A federated domain generalization method for cross-subject electroencephalogram emotion recognition

Abstract

The application discloses a federated domain generalization method for cross-subject electroencephalogram emotion recognition. Each client performs local session constraint data enhancement and trains a local model, and updated local model parameters are uploaded to a server. Meanwhile, a global mixed batch normalization layer is deployed in the local model. In the training stage, a local learnable mixing factor is used to fuse the global statistics and the local statistics corresponding to the current small batch of sample data for mixed normalization. The server aggregates the local model parameters, the local statistics and the local learnable mixing factor uploaded by each client into a global model, the global statistics of the current round and the global mixing factor of the current round, respectively. In the test stage, the global mixing factor is used to fuse the global statistics and the local statistics corresponding to the current test small batch of sample data for mixed normalization. The above process is repeated until the federated training is completed, and the server obtains a final global model.

Description

Technical Field

[0001] This invention belongs to the technical field of federated learning, specifically relating to a federated domain generalization method for cross-subject EEG emotion recognition. Background Technology

[0002] Brain-computer interface (BCI) technology enables direct interaction between humans and computers by analyzing brain neural signals and has been widely applied in fields such as entertainment and games, motor imagery, and emotion recognition. Electroencephalogram (EEG) signals, with their millisecond-level temporal resolution, portability, and non-invasiveness, have become the primary data modality for BCI systems. Among various EEG-based BCI paradigms, emotion recognition (ER) is widely recognized as one of the most challenging classification tasks to generalize across subjects due to significant differences in individual neurocognitive patterns and physiological characteristics, as well as the temporal non-stationarity of EEG signals.

[0003] In the deployment of actual brain-computer interface systems, there are two mutually restrictive core challenges.

[0004] First, cross-subject generalization is difficult. Due to physiological differences between individuals, even when different subjects perform the same brain-computer interface task, the resulting EEG signals deviate significantly in both frequency domain characteristics and spatial topology, exhibiting a severe cross-subject feature shift. Furthermore, when the same subject performs the same task at different times, their EEG signals also show significant temporal non-stationarity, causing cross-session challenges to often couple with cross-subject challenges. These factors collectively result in a non-independent and identically distributed (Non-IID) feature distribution between training and test data, greatly hindering the model's cross-subject generalization ability.

[0005] Second, there is the need for data privacy protection. EEG signals are highly sensitive personal biometric data, which can be used not only for identity verification but also contain rich information about cognitive states and emotions. Both user rights protection and relevant laws and regulations explicitly require that machine learning methods must protect user privacy when processing EEG data, and prohibit the uploading of raw EEG data to centralized servers for aggregation and training.

[0006] Federated learning (FL) allows multiple clients to collaboratively train a globally powerful model through federated communication, sharing only local model parameters rather than raw data. This approach protects data privacy while enabling cross-client knowledge sharing. Based on the visibility of target domain data during training, federated learning can be divided into two paradigms: federated domain adaptation (FL-DA) and federated domain generalization (FL-DG). In real-world brain-computer interface deployment scenarios, data from new participants is completely unavailable during the training phase; therefore, the federated domain generalization paradigm is more aligned with practical needs. In the federated domain generalization framework, each participant corresponds to a client. Each client trains independently on local data and then uploads only the model parameters. The server aggregates the parameters to form a global model, which is then directly applied to new participants who did not participate in the initial training after training.

[0007] However, existing federated domain generalization methods still face the following unresolved technical bottlenecks in EEG emotion recognition tasks: Bottleneck 1: Overfitting due to scarcity of local data. Collecting EEG data is costly, and each subject typically has only a limited number of labeled samples, resulting in a severe shortage of local datasets on each client side. Neural network-based learning paradigms follow the law of scale, where model performance improves with the amount of training data. However, the scarcity of local data makes local models highly susceptible to overfitting to session-specific noise, limiting the generalization ability of the global model. While existing federated data augmentation methods improve cross-subject generalization through sample mixing strategies, their method of replacing original samples with an equal amount of mixed samples does not increase the effective training set size, resulting in limited gains.

[0008] Bottleneck 2: Mismatch in normalized statistics during the testing phase. Batch normalization (BN) layers are a widely used standard normalization method in deep neural networks. In traditional centralized learning, batch normalization layers use global runtime statistics (i.e., runtime mean and runtime variance) accumulated during the training phase to normalize test samples during the inference phase. In the federated learning framework, the global statistics are obtained by weighted aggregation of the local statistics of each participating client, which can characterize the data distribution in the training domain to a certain extent. However, cross-subject EEG classification tasks differ fundamentally from traditional image classification tasks. In the non-independent, identically distributed federated learning setting of traditional image classification, the bias in statistics mainly stems from label distribution shifts, and the test set is typically sampled from the training domain distribution. In cross-subject EEG classification, however, there is a significant covariate shift between the neural signal distribution in the test domain (new subjects) and the training domain (known subjects). The global statistics obtained by aggregating local statistics from each client during the training phase are biased for the test subjects. Directly normalizing the small batch of tests with frozen global statistics will lead to severe normalization mismatch, amplifying the adverse effects of cross-subject feature bias and temporal nonstationarity. While hybrid batch normalization (HBN) explores the integration of global and local statistics in federated learning, it still directly uses frozen global statistics during the cross-subject EEG testing phase, failing to incorporate the current subject's distribution information during testing, thus limiting its improvement effect on cross-subject scenarios.

[0009] Bottleneck 3: Performance degradation due to convergence to sharp minima during local training. In the local training phase of federated learning, due to the heterogeneity of data distribution across clients, local models from different clients tend to converge to their respective local optima, i.e., sharp minima. Sharp minima are highly sensitive to parameter perturbations. When the parameters of local models from multiple clients are federated and averaged, the aggregated global model parameters may fall into high-loss regions, leading to a severe degradation in the generalization ability of the global model. Although the sharpness-aware minimization (SAM) optimizer can improve the sharp minima problem to some extent, it only focuses on the sharpness of the loss landscape without explicitly constraining the upper bound of the gradient norm in the neighborhood, still posing a risk of convergence to flat minima with high generalization error. The Gradient Norm Aware Minimization (GAM) optimization strategy introduces first-order flatness regularization. By constraining the maximum gradient norm in the neighborhood of the parameters, it seeks a flatter minimum with lower generalization error, thus maintaining better global generalization performance even after federated parameters.

[0010] In summary, existing methods have failed to address the core challenges of cross-subject EEG emotion recognition in federated domain generalization scenarios through a synergistic approach encompassing data augmentation, normalization adaptation, and landscape optimization. There is an urgent need to propose a unified framework that can achieve complementary enhancements across these three dimensions. Summary of the Invention

[0011] To address the aforementioned technical issues, this invention provides a federated domain generalization method for cross-subject EEG emotion recognition. This method synergistically enhances the cross-subject generalization capability of the global federated learning model from three complementary dimensions: data augmentation, feature normalization, and optimization strategies. While protecting the privacy of each subject's data, this method effectively alleviates the problems of data scarcity and distribution bias in cross-subject EEG emotion recognition.

[0012] To achieve the above objectives, the present invention provides the following technical solution: A federated domain generalization method for cross-subject EEG emotion recognition includes the following steps: Step 1: Initialization and Client Preparation The server establishes communication connections with multiple clients, initializes global model parameters, and at the same time, each client collects EEG signals locally and extracts EEG features to form a local dataset for each client. Step 2: Initiate Federated Communication Rounds and Client Selection The server selects a subset of clients from the entire client set to participate in the first round of training, and simultaneously sends the global model parameters for the first round of training to the selected clients. Step 3: Local training on the client side Each client first performs local session constraint data augmentation, and then trains the local model using the augmented local dataset based on the global model parameters issued by the server, to obtain updated local model parameters and upload them to the server. Meanwhile, a global hybrid batch normalization layer is deployed in the local model. During the training phase, the global statistics and the local statistics corresponding to the current small batch sample data are fused with the local learnable hybrid factor to perform hybrid normalization. After each round of federated communication, the local statistics and the local learnable hybrid factor are uploaded to the server. Step 4: Server aggregation yields the global model for the current training round. The server aggregates the local model parameters uploaded by each client into a global model using a federated average method. At the same time, it summarizes the local statistics uploaded by each client using a weighted aggregation method to obtain the global statistics for the current round. It also aggregates the local learnable mixture factors uploaded by each client using a federated average method to obtain the global mixture factors for the current round. During the testing phase, the global model uses a global mixing factor to fuse global statistics with local statistics corresponding to the current small batch of test sample data for mixed normalization. Step 5: Federated training is complete, and the server obtains the final global model. The server reselects a subset of clients from the entire client set to participate in the next round of training, and sends the global model parameters (excluding the global mixing factor) for that round of training to the selected clients. Steps three and four are executed repeatedly until the preset number of communication rounds is reached, resulting in a final global model with cross-subject generalization ability.

[0013] Furthermore, by performing convex combination mixing of samples of different categories within the same session, the mixed samples are added as an additional regularization term to the current mini-batch training samples, thereby expanding the effective training set size and suppressing overfitting to session-specific noise, thus achieving local session-constrained data augmentation.

[0014] Furthermore, the local session constraint data enhancement specifically includes: Each sample in the local dataset of each client is pre-assigned a discrete session identifier; In each mini-batch of training samples, for each sample within the batch, construct a feasible pairing set that satisfies the following three constraints: belonging to the same session, belonging to different categories, and not self-pairing; If the feasible pairing set is not empty, then uniformly and randomly select pairing partners from it, sample the mixing coefficients from the beta distribution, perform convex combination interpolation on the sample and its pairing partner to generate candidate mixed samples, and construct the corresponding soft-label mixing loss. According to a preset addition ratio, a number of mixed samples are uniformly drawn from the effective mixed samples of the current round, and then added to the original mini-batch training samples. They then participate in the parameter update of the local model in the current round, so that the number of effective training samples in the current round is greater than the number of original mini-batch training samples.

[0015] Furthermore, set up a client. The local dataset has a total of There are sessions, and the number of samples in each session is . If the samples are arranged in session order, then the first... Session identifier for each sample satisfy: In the In each mini-batch training sample update step, the sampling size is small batch ,in, The first The client number The feature vector and sentiment category label of each sample. For each sample index within the batch... The feasible pairing set is as follows: Sampling mixing coefficient Generate candidate mixed samples The corresponding mixing loss is: in, For the client Local model parameters, The basic loss function (such as cross-entropy loss). For the output function of the local model, paired partners .

[0016] Let the additional ratio be Effective number of mixed samples ,in, ; from Uniformly drawn subsets without replacement Make The effective experience loss for the current round is: Among them, the mixed experience loss term (when If the value is 0, then the value is 0; otherwise, it is 0.

[0017] Furthermore, let the first... The set of clients selected in round-fed communication is The client is calculated using the following formula. The uploaded local statistics are the mean. and variance , in, For the client In a single mini-batch of sample data of size B, the first... Feature vectors of each sample; Client Uploaded locally learnable mixture factors Gradient updates are continuously learned and updated locally on the client through backpropagation. The server aggregates the local statistics uploaded by each client using a sample size weighting method to obtain the global statistics: in, For the client The total number of local samples, and the total number of client samples participating in this round of aggregation. ; The server processes locally learned mixture factors uploaded by each client. Global mixing factor is obtained by aggregation using a federated average method. ; During the training phase, the client The local model uses locally learnable mixing factors for the current mini-batch sample data. Perform mixed normalization; during the testing phase, the server's global model applies a global mixing factor to the current test mini-batch sample data. Perform mixed normalization; Calculate the mean and variance of the mixture normalization using the following formula: Where e is the natural constant, and This is the statistic for the current small batch of sample data. During the training phase, locally learnable mixture factors are taken. During the testing phase, the global mixing factor is taken. .

[0018] Furthermore, in step three, an accelerated gradient norm-aware minimization optimization strategy is introduced into the local training of each client. The first-order flatness regularization term is approximated by finite difference, which guides the local model to converge to the minimum region where the loss landscape is flatter.

[0019] Furthermore, the accelerated gradient norm-aware minimization optimization strategy computes four gradients in each mini-batch sample data update step during local training on the client. , , , To update the current parameters of the local model. , Let the first Federal Communications Round The local training round number The current parameter in a small batch update step The effective experience loss for small batches is The following calculations will be performed: (1) Using finite difference step size Calculate the two gradients at the current point. , : (2) Constructing the difference direction To obtain the counter-disturbance point : (3) Calculate the two gradients at the point of resistance to perturbation with the same finite difference step size. , : (4) Construct gradient combination directions: (5) about Perform orthogonal decomposition: in, for Compared to Parallel components; (6) Execution parameter update: in, It is the numerical stability constant. The perturbation radius is the scale. For finite difference step size, For gradient combination weights, The orthogonal suppression strength, This is the learning rate.

[0020] Furthermore, the EEG features are differential entropy features, which are calculated on five frequency bands and then spliced ​​together to form the feature representation of each EEG sample.

[0021] Compared with the prior art, the beneficial effects of the present invention are: 1. The local session constraint data augmentation proposed in this invention, without sharing any original client data, expands the effective training set size while introducing stronger inductive bias by performing convex combination between cross-class samples within the same session and adding mixed samples as additional regularization terms to the training, thereby effectively suppressing overfitting to session-specific noise and improving the cross-subject generalization performance of the global model.

[0022] 2. The global hybrid batch normalization layer proposed in this invention addresses the fundamental problem of bias in global statistics for test subjects during the training phase in cross-subject EEG classification. By introducing a learnable hybrid factor, it dynamically fuses global statistics with statistics corresponding to the current small batch of sample data during the testing phase for normalization, effectively mitigating the impact of cross-subject feature bias and time non-stationarity on normalization accuracy. At the same time, by independently aggregating the global hybrid factor and global statistics, and not sending the global hybrid factor to the client in federated communication, it ensures that each client retains its own local adaptive hybrid factor, achieving an organic unity of local adaptability and global generalization.

[0023] 3. The accelerated gradient norm-aware minimization optimization strategy introduced in this invention guides each client's local model to converge to a minimum region with a flatter loss landscape by constraining the maximum gradient norm in the parameter neighborhood. This results in the global model after federated aggregation being in a wider low-generalization loss basin in the parameter space, improving the cross-domain generalization ability of the global model. Furthermore, the accelerated version uses finite difference approximation to replace the calculation of the Hessian vector product, significantly reducing computational complexity without introducing additional communication overhead.

[0024] In summary, this invention synergistically enhances the cross-subject generalization ability of the federated learning global model from three complementary dimensions: data augmentation, feature normalization, and optimization strategies. Its plug-and-play design allows for seamless integration with any backbone network architecture and other federated learning algorithms, achieving cumulative performance gains. Experiments on three mainstream publicly available EEG emotion recognition datasets, using convolutional neural networks and EEGNet-DE as the backbone, demonstrate that compared to the federated average baseline, this invention achieves improvements of +3.20% and +3.64% in macro-average accuracy (convolutional neural network backbone) and +5.99% and +6.07% in F1 score (EEGNet-DE backbone), respectively. Furthermore, when the test batch size is reduced to 2, the average accuracy decreases by only 1.58%, demonstrating strong value for real-time privacy-preserving brain-computer interface deployment. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the overall framework of FedPGG, the federated domain generalization method for cross-subject EEG emotion recognition, during the training phase of the present invention. Figure 2 This is a detailed internal mechanism diagram of the Global Hybrid Batch Normalization (GHBN) layer of the present invention. Figure 3 A detailed flowchart illustrating the local session constraint data enhancement PunitiveMix of the present invention; Figure 4 This table compares the cross-subject classification performance of the method described in this invention with eight federated learning baseline methods on three datasets under the leave-one-subject cross-validation protocol. It uses convolutional neural networks and EEGNet-DE as the backbone, and reports the mean and standard deviation of macro-average accuracy and F1 score. The best results are marked in bold and the second-best results are marked in underline. Figure 5 The graph shows the macro-average accuracy versus F1 score of different methods on the SEED and SEED-IV datasets as the number of participating clients selected in each round of federated communication, K_"select" (K_"select" ∈ {1,3,5,7,9}, where K_"select" is the number of clients randomly selected by the server from all clients to participate in this round of training in each round of federated communication). Figure 6The graph shows the comparison of macro-average accuracy and F1 score of different methods on the SEED and SEED-IV datasets as a function of the number of local training rounds E (E∈1,2,3,4,5, where E is the number of training rounds in which each selected client performs local gradient updates in one round of federated communication, i.e., the number of local training rounds). Figure 7 The three-dimensional heatmap of hyperparameter sensitivity analysis of the local session constraint data augmentation module PunitiveMix of the present invention shows the distribution of leave-one-subjects cross-validation accuracy under different combinations of mixing intensity ξ (ξ∈0.5,1,2,3,5) and additional proportion δ (δ∈0.2,0.4,0.6,0.8,1.0) on three datasets. Figure 8 This table presents the ablation experiment results of this invention on three datasets. Using a convolutional neural network (CNN) backbone and an EEGNet-DE backbone, the table reports the leave-one-subject cross-validation accuracy and F1 score after removing or replacing each module, verifying the independent contribution and synergistic gain of each module. Figure 9 The graphs show the average accuracy of leave-one-out cross-validation and the mean of the optimal accuracy at each fold on three datasets under different test batch sizes (from 2 to 128), demonstrating the performance stability of the global hybrid batch normalization layer in small-batch inference scenarios. Detailed Implementation

[0026] To make the technical means, creative features, objectives and effects of this invention easier to understand, the following embodiments, in conjunction with the accompanying drawings, specifically illustrate the federated domain generalization method for cross-subject EEG emotion recognition of this invention. It should be noted that the description of these embodiments is for the purpose of helping to understand this invention, but does not constitute a limitation of this invention.

[0027] This invention provides a federated domain generalization method for cross-subject EEG emotion recognition, such as... Figure 1The diagram illustrates the collaborative relationship and data flow among the server, clients, and three core modules—local session constraint data augmentation, global hybrid batch normalization, and accelerated gradient norm-aware minimization optimization strategy—within the federated communication framework. In the federated learning framework, each client extracts features such as differential entropy from local EEG data and performs local session constraint data augmentation (PunitiveMix): by performing convex combination mixing between sample pairs of different categories within the same session, the mixed samples are added as an additional regularization term to the training, expanding the effective training set size without sharing the original data and addressing the local data bias problem. Simultaneously, each client deploys a global hybrid batch normalization layer (GHBN), fusing global statistics with the current mini-batch statistics using locally learnable mixing factors during the training phase. The server aggregates local statistics to obtain global statistics using a weighted aggregation method and aggregates locally learnable mixing factors to obtain global mixing factors. During the testing phase, the global model dynamically fuses global statistics with the current test mini-batch statistics using global mixing factors for normalization, effectively mitigating cross-subject feature bias. In addition, an accelerated gradient norm-aware minimization optimization strategy is introduced to guide the local model to converge to the minimum region where the loss landscape is flatter, thereby improving the cross-domain generalization ability of the global model.

[0028] Specifically as follows: For ease of understanding, this invention uses three mainstream publicly available EEG emotion recognition datasets as examples: the SEED dataset, the SEED-IV dataset, and the SEED-V dataset. These three datasets correspond to three-class, four-class, and five-class emotion recognition tasks, respectively. The SEED dataset involved 15 participants, each completing three sessions. Each session used 15 Chinese film clips to induce positive, neutral, and negative emotions. The SEED-IV dataset also included three sessions from 15 participants, using 24 videos to induce four emotions: happiness, sadness, fear, and neutrality. The SEED-V dataset included three sessions from 16 participants, using 15 videos to induce five emotions: disgust, fear, sadness, neutrality, and happiness. All three datasets used a 62-channel EEG acquisition system, acquiring EEG signals at a sampling rate of 1000 Hz. Electrode placement followed the international 10-20 system standard. The raw EEG signals were preprocessed, downsampled to 200 Hz, and bandpass filtered to remove high-frequency EMG artifacts and low-frequency drift.

[0029] This invention employs a leave-one-out cross-validation strategy to evaluate cross-subject generalization performance. Specifically, each time one subject is used as the test subject, and all other subjects are used as training clients. After training, the server directly uses the global model to infer the test subject data without accessing any target subject data.

[0030] The neural network architecture used by the local and global models participating in federated learning in this invention is not limited, and can be convolutional neural networks, EEGNet, recurrent neural networks, or other deep learning architectures suitable for EEG signal classification. At the same time, the method proposed in this invention can be used in a plug-and-play manner in combination with any of the above-mentioned backbone architectures and other federated learning algorithms to achieve cumulative performance gains.

[0031] Step 1: Initialization and Client Preparation The server establishes communication connections with multiple clients, initializes global model parameters, and at the same time, each client collects EEG signals locally and extracts EEG features to form a local dataset for each client.

[0032] S1. EEG signal acquisition and preprocessing.

[0033] A multi-lead EEG acquisition system was used to collect emotion-evoked EEG signals from each subject, recording the potential values ​​of each channel over time. The raw EEG signals were preprocessed to remove artifacts and noise such as electrooculography (EOG) and electromyography (EMG) to obtain clean EEG data. To balance computational resource consumption and data integrity, this embodiment independently and randomly sampled each subject from each dataset for each trial, selecting one sample every four samples to reduce the total sample size.

[0034] S2, EEG feature extraction.

[0035] Differential entropy (DE) features were extracted from the preprocessed EEG signals and used as input features for the neural network model. The DE features were calculated in five frequency bands: delta, theta, alpha, beta, and gamma, with each band corresponding to a scalar feature for each lead. The SEED dataset used a non-overlapping one-second sliding window for feature extraction, while the SEED-IV and SEED-V datasets used a non-overlapping four-second sliding window. The SEED and SEED-IV datasets further employed a linear dynamic system (LDS) to perform temporal smoothing on the extracted DE features.

[0036] It should be noted that the present invention does not limit the type of EEG feature. Power spectral density features, original sliding time window segmented data, or other time-domain, frequency-domain, and time-frequency-domain features can also be used as model inputs. Differential entropy features are the mainstream feature type in the field of EEG emotion recognition, and this embodiment is represented by it.

[0037] Step 2: Initiate Federated Communication Rounds and Client Selection The server selects a subset of clients from the entire client set to participate in the first round of training, and simultaneously sends the global model parameters for the first round of training to the selected clients. Step 3: Local training on the client side Each client first performs local session constraint data augmentation, and then trains the local model using the augmented local dataset based on the global model parameters issued by the server, to obtain the locally updated model parameters and upload them to the server. Meanwhile, a global hybrid batch normalization layer is deployed in the local model. During the training phase, the global statistics and the local statistics corresponding to the current mini-batch sample data are fused together using local learnable hybrid factors for hybrid normalization. After each round of federated communication, the local statistics and local learnable hybrid factors are uploaded to the server.

[0038] 1. Enhanced local session constraint data Local session-constrained data augmentation is performed independently during local training on each client, without any data sharing between clients. For example... Figure 3 As shown, this demonstrates the complete process of cross-class sample pairing within the same session, mixed coefficient sampling, convex combination mixed sample generation, appending mixed samples to the original mini-batch, and jointly participating in model parameter updates.

[0039] (1) Session identifier construction. Each sample in the local dataset of each client is pre-assigned a discrete session identifier. If the dataset natively provides session information, it is used directly. If the dataset does not provide native session information, a pseudo-session identifier is constructed based on the following auxiliary information: collection date (data collected on different dates belong to different sessions), trial segment number (sessions are divided according to trial number), or a fixed sample size range (a fixed sample size is used as a pseudo-session unit).

[0040] Set up the client Total There are sessions, and the number of samples in each session is . If the samples are arranged in session order, then the first... The session identifiers of the samples satisfy: When dividing the training set and validation set, the same set of indices is used to synchronously divide the samples, labels and session identifiers to avoid inconsistencies among the three.

[0041] (2) Constraint pairing and mixed sample generation. In the first... The first round of Federal Communications The first local training round In each mini-batch update step, the sample size from the local dataset is... small batch Sampling mixing coefficients from the beta distribution The mixing coefficient is shared within a small batch. For each sample within the batch... Construct a feasible pairing set that satisfies the following three constraints: belonging to the same session, belonging to different categories, and not being self-pairs, i.e.: like Then, randomly select matching partners from them. Otherwise, This sample does not participate in the mixing. For samples that satisfy... samples Generate candidate mixed samples: The corresponding soft label mixing loss is: (3) Additional mixed training. Let the effective mixed sample set be... Preset additional ratio Effective additional mixed sample number .from Uniformly selected subsets without replacement Make ,Will After being appended to the original mini-batch, they participate in the local model parameter update for the current round. The effective empirical loss for the current round is: in , (when If the value is 0, then the value is 0; otherwise, it is 0.

[0042] The data augmentation of this invention introduces only two hyperparameters: mixing intensity. (Controlling the shape of the beta distribution, and thus the concentration of the mixing coefficient) and additional proportions (Control the proportion of additional mixed samples in each mini-batch).

[0043] 2. Global Mixed Batch Normalization Training Phase Traditional batch normalization layers use statistics from the current small batch for normalization during the training phase and frozen global runtime statistics accumulated during training for normalization during the inference phase. Hybrid batch normalization methods also directly use frozen global statistics during the inference phase, failing to incorporate the distribution information of the current test subjects during testing. This invention addresses the fundamental problem of bias in the global statistics during the training phase for test subjects in cross-subject EEG classification by introducing a learnable mixture factor into the global hybrid batch normalization layer. This allows the normalization process to dynamically integrate global statistics with current mini-batch statistics during both the training and testing phases.

[0044] This invention deploys a global hybrid batch normalization layer at the batch normalization layer of the neural network model, such as... Figure 2 As shown, the complete process is illustrated, including the training phase where each client performs mixture normalization using locally learnable mixture factors, the uploading of local statistics, and the server-side aggregation of global statistics; and the testing phase where the global model performs dynamic mixture normalization on the current test mini-batch using global mixture factors. This global mixture batch normalization layer shares the same normalization layer structure in both the training and testing phases, but the sources of the mixture factors used in the two phases differ, as detailed below: During the client training phase, the mixing factor used for normalization is a learnable mixing factor that is locally retained and continuously updated by the client. During the server testing phase, the mixing factor used for normalization is the global mixing factor obtained by federated average aggregation of the locally learnable mixing factors uploaded by each selected client. The global mixed batch normalization layer does not use frozen global running statistics for normalization during the training and testing phases. Instead, it dynamically combines the global statistics with the current mini-batch statistics by weighting the mixture factor and then uses the results for normalization to alleviate cross-subject feature bias and time non-stationarity. In each round of federated communication, the server does not distribute the global mixture factor to each client. Each client always retains and updates its own local learnable mixture factor. After the server aggregates all model parameters (including local learnable mixture factors) uploaded by each client, the resulting global mixture factor is only used by the server during the testing phase.

[0045] Let the first The set of clients selected in round-fed communication is The client is calculated using the following formula. The uploaded local statistics are the mean. and variance , Client Uploaded locally learnable mixture factors Gradient updates are continuously learned locally on the client through backpropagation, and are not overwritten by the global model parameters issued by the server.

[0046] During the client training phase, each client In each mini-batch update step, a locally learnable blending factor is used at the global blend batch normalization layer. Perform mixed normalization. Let the statistic for the current mini-batch sample data be... and The mean of the current small batch of sample data can be calculated using the standard batch normalization method. and variance Global statistics are and (Obtained from the server in the previous round of federated communication), then the hybrid statistics used by the global hybrid batch normalization layer are: After each round of local training, the client Upload local model parameters to the server (Including locally learnable mixture factors) ) and local statistics and .

[0047] 3. Accelerated gradient norm-aware minimization optimization strategy In each mini-batch update step of local training on each client, an accelerated gradient norm-aware minimization (accelerated GAM) optimization strategy is introduced to perform parameter updates in conjunction with standard stochastic gradient descent or other conventional optimizers.

[0048] The gradient norm-aware minimization optimization strategy introduces a first-order flatness regularization term to constrain the maximum gradient norm within the parameter neighborhood, seeking a flatter minimum with lower generalization error. An accelerated implementation replaces the Hessian vector product (HVP) calculation with a finite difference approximation, significantly reducing computational complexity and making it suitable for deployment on federated learning client-side. This accelerated strategy guides each client's local model to converge to a flatter minimum region in the loss landscape, resulting in a broader low-generalization loss basin in the parameter space for the federated global model, thus improving the global model's cross-domain generalization ability. An accelerated gradient norm-aware minimization optimization strategy is employed to compute four gradients in each mini-batch update step during local training on the client side. , , , To update the current parameters of the local model. Let the first Federal Communications Round The local training round number The current parameter in a small batch update step The effective experience loss for small batches is The following calculations will be performed: (1) Using finite difference step size Calculate the two gradients at the current point. , : (2) Constructing the difference direction The counter-disturbance point is obtained: (3) Calculate the two gradients at the point of resistance to perturbation with the same finite difference step size. , : (4) Construct gradient combination directions: (5) about Perform orthogonal decomposition: With orthogonal suppression strength inhibition orthogonal components With stable training, among which, for Compared to Parallel components.

[0049] (6) Execution parameter update: in, It is the numerical stability constant. The perturbation radius is the scale. For finite difference step size, For gradient combination weights, The orthogonal suppression strength, This represents the learning rate. If a dynamic learning rate decay strategy is used during training, then all hyperparameters should decay accordingly.

[0050] Step 4: Server aggregation yields the global model for the current training round. The server aggregates the local model parameters uploaded by each client into a global model using a federated average method. At the same time, it summarizes the local statistics uploaded by each client using a weighted aggregation method to obtain the global statistics for the current round. It also aggregates the local learnable mixture factors uploaded by each client using a federated average method to obtain the global mixture factors for the current round. During the testing phase, the global model uses a global mixing factor to fuse global statistics with local statistics corresponding to the current small batch of test sample data for mixed normalization.

[0051] 1. The server receives the first... The set of clients selected in the wheel After uploading the local model parameters, local statistics, and local learnable mixing factors, perform the following two independent aggregation operations: (1) Weighted aggregation of local statistics yields global statistics. Local statistics uploaded by each client are calculated based on the local sample size of each client. Weighted aggregation yields the global statistics for the current round: , in .

[0052] The local statistics uploaded by each client support the following two calculation methods.

[0053] Method 1 (default in this embodiment): Approximate estimation based on a single local training mini-batch sample data. In this case, the statistics are the same as those of the current mini-batch sample data, i.e.: in This is the output of the intermediate layer of the model before the normalization operation is performed in the normalization layer.

[0054] Method 2: Precise estimation based on all local data on the client side. In this case, the statistics are different from those of the current small batch of sample data, and they belong to a subordinate relationship. Therefore, a more accurate local distribution representation can be obtained, but it is necessary to traverse the entire local dataset, which results in greater computational overhead. There is only a slight difference in performance between the two methods.

[0055] (2) Federated average aggregation of local model parameters and local learnable mixture factors yields the global model and global mixture factors. The server processes the local model parameters (including local learnable mixture factors) uploaded by each client. The global model parameters and global mixture factors are aggregated using a sample size-weighted federated average method. : It is important to note that the aggregation methods for global statistics and global mixture factors are independent and different: the former uses sample size-weighted aggregation based on the variance pooling formula, while the latter uses standard federated average-weighted aggregation. The server will aggregate the global model parameters (excluding global mixture factors). The parameters are distributed to each client to initialize the next round of local training. The local learnable mixture factors of each client are retained locally and are not overwritten by the global model parameters distributed by the server.

[0056] 2. Global mixed batch normalization test phase.

[0057] In the testing phase, the global model directly infers from unseen test subjects; at this time, the global mixture batch normalization layer uses a global mixture factor. Perform dynamic hybrid normalization on the current test mini-batch: Where e is the natural constant, and This is a local statistic for the current small batch of test sample data. and These are global statistics stored on the server side. During the training phase, locally learnable mixture factors are taken. During the testing phase, the global mixing factor is taken. .

[0058] Unlike traditional batch normalization and hybrid batch normalization, which only use frozen global statistics during the inference phase, global hybrid batch normalization dynamically introduces the current test mini-batch statistics during the testing phase. It adaptively adjusts the fusion weight of the two with a learnable global mixing factor, so that the normalization process can adaptively perceive the local distribution characteristics of the test subjects and effectively mitigate the adverse effects of cross-subject feature bias and time non-stationarity.

[0059] In streaming real-time inference scenarios with extremely small batch sizes (including batch size of 1), the following two variant schemes can be adopted to enhance the stability of statistics: Scheme 1: Cache recent historical physiological samples within a preset sliding time window, and use the cached samples within the window to calculate local statistics instead of single-sample statistics for mixed normalization; Scheme 2: Use an exponential moving average strategy to weight and accumulate historical small-batch statistics, and use the exponential moving average statistics instead of the current small-batch statistics for mixed normalization, thereby enhancing the stability of inference statistics without increasing caching overhead. Figure 9 As shown, when the test batch size is reduced from 128 to 2, the average accuracy of the present invention on the three datasets decreases by only 1.58% overall, which demonstrates the robustness of the global hybrid batch normalization layer in small-batch inference scenarios and meets the practical requirements for real-time brain-computer interface deployment.

[0060] Step 5: Federated training is complete, and the server obtains the final global model. The server reselects a subset of clients from the entire client set to participate in the next round of training, and sends the global model parameters (excluding the global mixture factor) for that round of training to the selected clients. Steps three and four are repeated until the preset number of communication rounds is reached, for a total of [number missing]. Through rounds of federated communication, a final global model with cross-subject generalization ability is obtained.

[0061] In each round of federated learning, the server introduces two key hyperparameters: the number of clients randomly selected by the server to participate in this round of training during each round of federated communication, denoted as... The number of training epochs (i.e., the number of local training epochs) in which each selected client performs local gradient updates in a round of federated communication is denoted as . .in Controlling the size of the clients participating in aggregation in each round of communication directly affects the statistical representativeness of the global model and the communication bandwidth overhead. Control the computational intensity of local training on each client during each round of communication. When the size is too large, the client model is prone to deviating from the global optimum (i.e., client drift). When the time is too short, the local computation efficiency of each round of communication is low.

[0062] In each round of communication, the server randomly selects from all clients. Each selected client participates in this round of training. Each client initializes its local model using the global model parameters from the previous round and executes... After each local training epoch, the local model parameters, local statistics, and local learnable mixing factors are uploaded to the server. After aggregating the above information, the server distributes the updated global model parameters (excluding the global mixing factors) to each client to initialize the next round of local training.

[0063] In the default experimental settings of this embodiment, the number of federated communication rounds... Each round of selection Each client executes [a certain number of] commands. Number of local training rounds, training batch size The test batch size was 32, the learning rate of the convolutional neural network backbone was 0.01, the learning rate of the EEGNet-DE backbone was 0.0005, the base optimizer was stochastic gradient descent with weight decay (weight decay coefficient of 0.0001) and momentum (momentum coefficient of 0.9), and local session-constrained data augmentation hyperparameters were used. , Accelerated version of gradient norm-aware minimization of hyperparameters , , , , .

[0064] After the round of communication is completed, the final global model held by the server can be directly applied to any new subject who has not participated in the training, and reasoning is performed using the dynamic mixed normalization method of the global mixed batch normalization layer during the test phase, outputting the emotion category prediction result.

[0065] To verify the feasibility of this invention, we conducted the following experimental tests: like Figure 4 As shown, on three datasets, using convolutional neural networks and EEGNet-DE as the backbone and employing a leave-one-out cross-validation protocol, compared with eight federated learning baseline methods, this invention achieved the best results in 10 out of 12 evaluation metrics. Using convolutional neural networks as the backbone, this invention achieves macro-average accuracy and F1 score improvements of +3.94% and +4.17% (SEED dataset), +2.21% and +2.23% (SEED-IV dataset), and +3.47% and +4.51% (SEED-V dataset) on three datasets compared to the federated average baseline. Using EEGNet-DE as the backbone, the corresponding improvements are +5.33% and +5.64% (SEED dataset), +5.22% and +5.39% (SEED-IV dataset), and +7.43% and +7.17% (SEED-V dataset). The macro-average improvements under the two backbones are +3.20% and +3.64%, and +5.99% and +6.07%, respectively.

[0066] like Figure 5 As shown, in In the hyperparameter scanning experiments, this invention demonstrated a stable competitive advantage on both the SEED and SEED-IV datasets, without performance crashes or drastic fluctuations. For example... Figure 6 As shown, in In the hyperparameter scanning experiment, the present invention maintained a stable advantage on both datasets, verifying the robustness of the present invention to the two key hyperparameters of federated learning.

[0067] like Figure 7 As shown, the hyperparameter sensitivity analysis of the local session-constrained data augmentation indicates that the accuracy variation across the three datasets is only 1.05% (SEED dataset), 0.64% (SEED-IV dataset), and 2.26% (SEED-V dataset), respectively, demonstrating strong hyperparameter robustness. The optimal configuration, taking the average across the three datasets, is... The average accuracy rate reached 67.61%.

[0068] like Figure 8As shown, the ablation experiment results validated the contributions of the three modules: after removing local session constraint data augmentation, the average accuracy of EEGNet-DE as the backbone decreased by 1.84%; after replacing global mixed batch normalization with mixed batch normalization, the average accuracy of convolutional neural networks as the backbone decreased by 3.25%; after removing the accelerated gradient norm-aware minimization optimization strategy, the average accuracy of EEGNet-DE as the backbone decreased by 3.00%; the combined use of the three modules yielded the best performance, validating the design motivation of synergistic enhancement across three complementary dimensions.

[0069] It is important to note that the schemes and arrangements of this application shown in the exemplary embodiments are merely exemplary. Although only a few embodiments are described in detail in this disclosure, those who consult this disclosure will readily understand that many modifications are possible (e.g., variations in various parameter values ​​(temperature, power, humidity, etc.), installation arrangements, names, colors, logical orders, etc.) without substantially departing from the novel teachings and advantages of the subject matter described in this application. Therefore, all such modifications are also included within the scope of the invention, and the order or sequence of any process or method steps may be changed or rearranged according to alternative embodiments. In the claims, any "apparatus plus function" clause is intended to cover the structure described herein for performing the function, and not only structurally equivalent but also equivalent in structure. Other substitutions, modifications, alterations, and omissions may be made in the design, operation, and arrangement of the exemplary embodiments without departing from the scope of the invention. Therefore, the invention is not limited to the particular embodiments but extends to a variety of modifications that still fall within the scope of the appended claims.

[0070] Furthermore, in order to provide a concise description of exemplary embodiments, not all features of actual embodiments (i.e., those features that are not relevant to the best mode of carrying out the invention as currently considered, or those features that are not relevant to implementing the invention) may be omitted.

[0071] It should be understood that numerous specific implementation decisions can be made during the development of any practical implementation, such as in any engineering or design project. Such development efforts may be complex and time-consuming, but for those skilled in the art who benefit from this disclosure, the development effort will be a routine work of design, manufacturing, and production without requiring much experimentation.

[0072] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A federated domain generalization method for cross-subject EEG emotion recognition, characterized in that... Includes the following steps: Step 1: Initialization and Client Preparation The server establishes communication connections with multiple clients, initializes global model parameters, and at the same time, each client collects EEG signals locally and extracts EEG features to form a local dataset for each client. Step 2: Initiate Federated Communication Rounds and Client Selection The server selects a subset of clients from the entire client set to participate in the first round of training, and simultaneously sends the global model parameters for the first round of training to the selected clients. Step 3: Local training on the client side Each client first performs local session constraint data augmentation, and then trains the local model using the augmented local dataset based on the global model parameters issued by the server, to obtain updated local model parameters and upload them to the server. Meanwhile, a global hybrid batch normalization layer is deployed in the local model. During the training phase, the global statistics and the local statistics corresponding to the current small batch sample data are fused with the local learnable hybrid factor to perform hybrid normalization. After each round of federated communication, the local statistics and the local learnable hybrid factor are uploaded to the server. Step 4: Server aggregation to obtain the global model for the current round of training. The server aggregates the local model parameters uploaded by each client into a global model using a federated average method. At the same time, it summarizes the local statistics uploaded by each client using a weighted aggregation method to obtain the global statistics for the current round. It also aggregates the local learnable mixture factors uploaded by each client using a federated average method to obtain the global mixture factors for the current round. During the testing phase, the global model uses a global mixing factor to fuse global statistics with local statistics corresponding to the current small batch of test sample data for mixed normalization. Step 5: Federated training is complete, and the server obtains the final global model. The server reselects a subset of clients from the entire client set to participate in the next round of training, and sends the global model parameters (excluding the global mixing factor) for that round of training to the selected clients. Steps three and four are executed repeatedly until the preset number of communication rounds is reached, resulting in a final global model with cross-subject generalization ability.

2. The federated domain generalization method for cross-subject EEG emotion recognition according to claim 1, characterized in that: By performing convex combination mixing of samples of different categories within the same session, and adding the mixed samples as an additional regularization term to the current mini-batch training samples, the effective training set size is expanded and overfitting to session-specific noise is suppressed, thereby achieving local session-constrained data augmentation.

3. The federated domain generalization method for cross-subject EEG emotion recognition according to claim 1, characterized in that: The local session constraint data enhancement specifically includes: Each sample in the local dataset of each client is pre-assigned a discrete session identifier; In each mini-batch of training samples, for each sample within the batch, construct a feasible pairing set that satisfies the following three constraints: belonging to the same session, belonging to different categories, and not self-pairing; If the feasible pairing set is not empty, then uniformly and randomly select pairing partners from it, sample the mixing coefficients from the beta distribution, perform convex combination interpolation on the sample and its pairing partner to generate candidate mixed samples, and construct the corresponding soft-label mixing loss. According to a preset addition ratio, a number of mixed samples are uniformly drawn from the effective mixed samples of the current round, and then added to the original mini-batch training samples. They then participate in the parameter update of the local model in the current round, so that the number of effective training samples in the current round is greater than the number of original mini-batch training samples.

4. The federated domain generalization method for cross-subject EEG emotion recognition according to claim 3, characterized in that: Set up the client The local dataset has a total of There are sessions, and the number of samples in each session is . If the samples are arranged in session order, then the first... Session identifier for each sample satisfy: In the In each mini-batch training sample update step, the sampling size is small batch ,in, The first The first client The feature vector and sentiment category label of each sample. For each sample index within the batch... The feasible pairing set is as follows: Sampling mixing coefficient Generate candidate mixed samples The corresponding mixing loss is: in, For the client Local model parameters, The basic loss function (such as cross-entropy loss). For the output function of the local model, paired partners . Let the additional ratio be Effective number of mixed samples ,in, ; from Uniformly drawn subsets without replacement Make The effective experience loss for the current round is: Among them, the mixed experience loss term (when If the value is 0, then the value is 0; otherwise, it is 0.

5. The federated domain generalization method for cross-subject EEG emotion recognition according to claim 1, characterized in that: Let the first The set of clients selected in round-fed communication is The client is calculated using the following formula. The uploaded local statistics are the mean. and variance , in, For the client In a single mini-batch of sample data of size B, the first... Feature vectors of each sample; Client Uploaded locally learnable mixture factors Gradient updates are continuously learned and updated locally on the client through backpropagation. The server aggregates the local statistics uploaded by each client using a sample size weighting method to obtain the global statistics: in, For the client The total number of local samples, and the total number of client samples participating in this round of aggregation. ; The server processes locally learned mixture factors uploaded by each client. Global mixing factor is obtained by aggregation using a federated average method. ; During the training phase, the client The local model uses locally learnable mixing factors for the current mini-batch sample data. Perform mixed normalization; during the testing phase, the server's global model applies a global mixing factor to the current test mini-batch sample data. Perform mixed normalization; Calculate the mean and variance of the mixture normalization using the following formula: Where e is the natural constant, and This is the statistic for the current small batch of sample data. During the training phase, locally learnable mixture factors are taken. During the testing phase, the global mixing factor is taken. .

6. The federated domain generalization method for cross-subject EEG emotion recognition according to claim 1, characterized in that: In step three, an accelerated gradient norm-aware minimization optimization strategy is introduced into the local training of each client. The first-order flatness regularization term is approximated by finite difference, which guides the local model to converge to the minimum region where the loss landscape is flatter.

7. The federated domain generalization method for cross-subject EEG emotion recognition according to claim 6, characterized in that: The accelerated gradient norm-aware minimization optimization strategy computes four gradients in each mini-batch sample data update step during local training on the client. , , , To update the current parameters of the local model. , Let the first Federal Communications Round The local training round number The current parameter in a small batch update step The effective experience loss for small batches is The following calculations will be performed: (1) Using finite difference step size Calculate the two gradients at the current point. , : (2) Constructing the difference direction To obtain the counter-disturbance point : (3) Calculate the two gradients at the point of resistance to perturbation with the same finite difference step size. , : (4) Construct gradient combination directions: (5) about Perform orthogonal decomposition: in, for Compared to Parallel components; (6) Execution parameter update: in, It is the numerical stability constant. The perturbation radius is the scale. For finite difference step size, For gradient combination weights, The orthogonal suppression strength, This is the learning rate.

8. The federated domain generalization method for cross-subject EEG emotion recognition according to claim 1, characterized in that: The EEG features are differential entropy features. Differential entropy is calculated on each of the five frequency bands and then spliced ​​together to form the feature representation of each EEG sample.

Images