Federal learning method and system for cross-subject electroencephalogram classification and electronic equipment

Abstract

The embodiment of the invention provides a federal learning method and system for cross-subject electroencephalogram classification and electronic equipment. The method comprises the following steps: constructing a federated learning framework based on a plurality of dispersed clients, a global server, a global model and a shared data terminal; electroencephalogram data label pairs are extracted from a plurality of dispersed clients through a global server, mixed data enhancement is carried out according to differential entropy features, mixed data label pairs are obtained, and field generalization federated learning is carried out on a global model through the mixed data labels; obtaining label-free shared electroencephalogram data from the target domain data by using the shared data end based on the sharing rate and the aggregation parameter, and generating a mixed label for the label-free shared electroencephalogram data, so as to provide the mixed label for a plurality of dispersed clients to carry out domain adaptive federated learning; therefore, a plurality of dispersed clients cooperatively train a global model for cross-subject electroencephalogram classification. According to the embodiment of the invention, a novel federal learning framework is provided, and the generalization ability of a global model is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of brain-computer interfaces, and in particular to a federated learning method, system, and electronic device for cross-subject EEG classification. Background Art

[0002] BCIs (Brain Computer Interfaces) based on EEG (Electroencephalogram) signals are helping people in a wide range of fields. However, due to the different and diverse cognitive processes and physiological structures of each person, cross-user or cross-subject EEG classification is difficult. Existing EEG models are based on neural networks and require large amounts of data training to achieve high performance and versatility. However, EEG-related data is generally difficult to obtain. Most EEG data around the world is stored in the form of small datasets in various medical hospitals or research institutions. There is no large enough public EEG dataset to train EEG BCIs with good generalization capabilities.

[0003] To achieve EEG classification, existing technologies usually use: 1. Federated Learning: Federated learning is a machine learning technique in which algorithms are trained across multiple distributed edge devices or servers with local data samples. This approach differs significantly from traditional centralized machine learning techniques, which upload all local datasets to a single server, and more classic distributed approaches, which typically assume that local data samples are uniformly distributed.

[0004] 2. Federated Average Algorithm: FedAvg focuses on solving the Non-IID and Unbalanced problems, not considering a host of other issues that arise during implementation. Assume there are K clients, each with a fixed dataset. In each communication round, the server randomly selects C clients and sends the current model parameters to them. These clients then perform local computations and send updated status to the server. The server receives these updates, and this process repeats.

[0005] 3. Mixup data augmentation method (mixture of experts for EEG-based seizure subtype classification): Mixup achieves data augmentation by modeling the differences between different categories, while general data augmentation methods perform transformations within the same category. The idea is simple: two samples are randomly selected from the training set and a simple random weighted sum is performed. The labels of the samples are also weighted and summed accordingly. The prediction result is then compared with the weighted summed labels to calculate the loss, and the parameters are updated by reverse derivative.

[0006] 4. Manifold mixup: Manifold mixup extends mixup, extending the mixing of raw input data to also mix the outputs of intermediate hidden layers. The original paper provides a compelling explanation of the three advantages of this mixup: smoothing decision boundaries, widening the low-confidence space (increasing the distance between high-confidence spaces of different classes), and flattening the hidden layer outputs.

[0007] In the process of implementing the present invention, the inventors discovered that there are at least the following problems in the related art: Federation-related technologies use federated learning models, which make it difficult to train global models with better generalization. Strict data communication restrictions prevent the framework from properly processing EEG data from new subjects or from the target domain and protecting the privacy of this EEG data.

[0008] Mixup-related techniques linearly interpolate the original data or intermediate hidden layer vectors, ignoring the unique properties of EEG data. New mixup data can be generated by simply adding the two original images and their labels. However, for other data modalities, simple linear addition may not necessarily improve model generalization. Summary of the Invention

[0009] In order to at least solve the problem of poor generalization ability of EEG BCI in the existing technology when there is no sufficiently large public EEG dataset for training.

[0010] In a first aspect, an embodiment of the present invention provides a federated learning method for cross-subject EEG classification, comprising: A federated learning framework is constructed based on multiple distributed clients, a global server, a global model, and a shared data terminal. The multiple distributed clients each store EEG data label pairs across subjects, and data between the clients is not interoperable. The global server extracts the EEG data label pairs from the multiple distributed clients and performs mixed data enhancement according to differential entropy features to obtain mixed data label pairs, and uses the mixed data labels to perform domain generalization federated learning on the global model, wherein the mixed data enhancement includes: linear mixing, channel mixing, and frequency mixing; Obtaining unlabeled shared EEG data from target domain data based on the sharing rate and aggregation parameters using the shared data end, generating hybrid labels for the unlabeled shared EEG data, and providing them to the multiple distributed clients for domain adaptive federated learning; The domain generalization federated learning and the domain adaptation federated learning enable the multiple distributed clients to collaboratively train a global model for cross-subject EEG classification.

[0011] In a second aspect, an embodiment of the present invention provides a federated learning system for cross-subject EEG classification, including: A framework construction module is used to build a federated learning framework based on multiple distributed clients, a global server, a global model, and a shared data terminal, wherein the multiple distributed clients respectively store EEG data label pairs across subjects, and the data between the clients is not interoperable; A domain generalization module is configured to extract the EEG data label pairs from the multiple distributed clients through the global server and perform mixed data enhancement according to differential entropy features to obtain mixed data label pairs, and perform domain generalization federated learning on the global model using the mixed data labels, wherein the mixed data enhancement includes: linear mixing, channel mixing, and frequency mixing; a domain adaptation module, configured to obtain unlabeled shared EEG data from target domain data using the shared data end based on a sharing rate and an aggregation parameter, and generate hybrid labels for the unlabeled shared EEG data to provide to the multiple distributed clients for domain adaptive federated learning; A federated learning module is used to enable the multiple distributed clients to collaboratively train a global model for cross-subject EEG classification through the domain generalization federated learning and the domain adaptation federated learning.

[0012] According to a third aspect, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the federated learning method for cross-subject EEG classification according to any embodiment of the present invention.

[0013] In a fourth aspect, an embodiment of the present invention provides a storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the steps of the federated learning method for cross-subject EEG classification of any embodiment of the present invention are implemented.

[0014] In a fifth aspect, an embodiment of the present invention provides a computer program product, including a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, the steps of the federated learning method for cross-subject EEG classification of any embodiment of the present invention are implemented.

[0015] The beneficial effects of embodiments of the present invention include: This method proposes a novel federated learning framework for mixEEG, customizing mixup for EEG modalities to unlock its potential for developing better cross-subject EEG brain-computer interfaces (EEG BCIs). By mixing only local EEG data from each client, mixEEG further enhances the generalization of the global model in a DG FL setting. By leveraging unlabeled data from the target domain, even better performance is achieved in a DG FL setting. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 This is a flowchart of a federated learning method for cross-subject EEG classification provided by one embodiment of the present invention; Figure 2 Schematic diagram of learning a global model from multiple dispersed source domains by domain generalization joint learning and domain adaptation joint learning of a federated learning method for cross-subject EEG classification provided by one embodiment of the present invention; Figure 3 1 is a schematic diagram of an EEG hybrid method of a federated learning method for cross-subject EEG classification provided by an embodiment of the present invention; Figure 4 1 is a schematic diagram of scores under a DG FL setting for a federated learning method for cross-subject EEG classification provided by one embodiment of the present invention, where L, C, and F represent linear, channel, and frequency mixing strategies, respectively; Figure 5 1 is a schematic diagram of scores under a DA FL setting of a federated learning method for cross-subject EEG classification provided by one embodiment of the present invention; Figure 6It is the score of α on the dataset under the MLP architecture of a federated learning method for cross-subject EEG classification provided by one embodiment of the present invention; Figure 7 This is a schematic diagram showing the accuracy of a global model trained on a SEED dataset using different training set sizes in a federated learning method for cross-subject EEG classification provided by one embodiment of the present invention; Figure 8 3D heat map of LOSO on SEED datasets with different sharing rates r and aggregation parameters s for a federated learning method for cross-subject EEG classification provided by one embodiment of the present invention Figure 9 This is a UMAP visualization diagram of the learned features on the SEED dataset of each global model using different training strategies for a federated learning method for cross-subject EEG classification provided by an embodiment of the present invention, where purple, green, and pink represent negative, neutral, and positive emotions; Figure 10 This is a schematic diagram of the structure of a federated learning system for cross-subject EEG classification provided by one embodiment of the present invention; Figure 11 A schematic structural diagram of an embodiment of an electronic device for federated learning of EEG classification across subjects provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0019] like Figure 1 FIG. 1 is a flow chart of a federated learning method for cross-subject EEG classification provided by an embodiment of the present invention, comprising the following steps: S11: Constructing a federated learning framework based on multiple distributed clients, a global server, a global model, and a shared data terminal, wherein the multiple distributed clients respectively store EEG data label pairs across subjects, and data between the clients is not interoperable; S12: extracting the EEG data label pairs from the multiple distributed clients through the global server and performing mixed data enhancement according to differential entropy features to obtain mixed data label pairs, and performing domain generalization federated learning on the global model using the mixed data labels, wherein the mixed data enhancement includes: linear mixing, channel mixing, and frequency mixing; S13: using the shared data end to obtain unlabeled shared EEG data from the target domain data based on the sharing rate and the aggregation parameter, generating a hybrid label for the unlabeled shared EEG data, and providing it to the multiple distributed clients for domain adaptive federated learning; S14: The domain generalization federated learning and the domain adaptation federated learning enable the multiple distributed clients to collaboratively train a global model for cross-subject EEG classification.

[0020] In this embodiment, the cross-subject variability of EEG signals is considered to hinder the development of brain-computer interfaces. Existing DG (domain generalization) methods mainly focus on traditional supervised learning or unsupervised learning. Such methods cannot be directly used to train global models with multiple decentralized clients, thus limiting their access to large amounts of data. This method needs to enable BCI developers to train global models through the collaboration of multiple decentralized clients while protecting privacy. At the same time, considering the privacy protection of EEG, existing technologies introduce FL (Federated Learning) in EEG classification tasks, but the cross-subject problem has not yet been solved.

[0021] For step S11, this method aims to design a data augmentation strategy tailored to the EEG modality from a data augmentation perspective. It also proposes a method that leverages unlabeled data from the target domain to enhance the generalizability of the global model for cross-subject federated EEG learning. To address the cross-subject issue in federated EEG learning, this method proposes a generalized federated learning EEG model based on mixup, referred to as the federated learning framework (mixEEG). Specifically, it includes multiple distributed clients, a global server, a global model, and a shared data endpoint.

[0022] In EEG federated learning tasks, there are two cross-subject settings: DG (domain generalization) and DA (domain adaptation). DG does not require data from the unknown target domain, while DA allows the trainer to use unlabeled data from the unknown target domain.

[0023] This method sets DG and DA on FL federated learning, which are DG FL (domain generalization federated learning) and DA FL (domain adaptive federated learning), such as Figure 2 As shown in Figure 2, these two setups require training a global model on source domains provided by multiple clients. Multiple distributed clients store cross-subject EEG data-label pairs, i.e., (EEG data, label), while also protecting the data privacy of each source domain. Data communication between source domains is strictly prohibited.

[0024] make and Represents the source domain dataset and the target domain dataset, where X represents data and Y represents labels. The source domain dataset is the data stored in multiple scattered clients. For example, client 1 is an EEG emotion research institution that stores EEG data in the emotion field, and client 2 is an EEG neural research institution that stores EEG data in the neural field. The target domain dataset is prepared in advance to correspond to the training target. Different from the DG setting for training the global model of DL (Deep Learning), the DA setting allows the trainer to train unlabeled target data. Make additional visits.

[0025] In step S12, in order to extract EEG frequency domain features, DE (Differential Entropy) is extracted from the five frequency bands of each sample within a non-overlapping 1s time window: Where Ts, C, T and D represent the number of DE samples, EEG channels, the size of the overlapping window and the dimension of the extracted features, respectively.

[0026] In the federated learning setting, there are K clients whose dataset has multiple sub-datasets ,in, To protect privacy, direct data communication between clients is strictly prohibited, that is, each client only uses its own sub-dataset Train its own local model. The goal of distributed DG FL (Domain Generalized Federated Learning) is to train a model that can be applied to the target domain dataset. The global model.

[0027] In order to train a more generalized global EEG model, the mixup data augmentation strategy is used, but this is not enough. It is also necessary to generate features that are more suitable for EEG spatial representation and more discriminative. Therefore, this method verifies several different mixing strategies to fully study which type of mixing is suitable for EEG modality.

[0028] As an embodiment, when the mixed data enhancement is linear mixing, the method includes: The first data label pair and the second data label pair are mixed according to linear interpolation to obtain mixed data ,in: described , the λ is a preset hyperparameter.

[0029] The mixed label in the mixed data label pair , the λ is a preset hyperparameter.

[0030] When the mixed data is enhanced to channel mixing, the method includes: The first data label pair (x i ,y i ) and the second data label pair (x j ,y j ) Divide into non-overlapping subsets C1 and C2 according to the number of channels C; The first data label pair and the second data label pair are mixed according to the channel to obtain mixed data for learning local spatial representation ,in: described , the Mr is a 01 mask matrix.

[0031] When the mixed data enhancement is frequency mixing, the method includes: The first data label pair (x i ,y i ) and the second data label pair (x j ,y j ) Divide into non-overlapping subsets F1 and F2 according to the number of frequency bands F; The first data label pair and the second data label pair are mixed according to the frequency to obtain mixed data for capturing the discriminative characteristics of the spectral band ,in: described , the Mc is a 01 mask matrix.

[0032] In this embodiment, the mixup data enhancement strategy of various EEG modalities is the most important part of the model.

[0033] Mixup is a data augmentation technique that uses two data label pairs and Linear interpolation between to produce a mixed data label pair:

[0034] in, is a hyperparameter. By enhancing, the model obtains better robustness and generalization ability. In this method, three hybrid methods of EEG DE differential entropy features are explored and verified for the first time, such as Figure 3 shown.

[0035] (1) Linear mixing: Same as the original mixup above.

[0036] (2) Channel mixing: Channel mixing can help the model learn local spatial representations that are robust to inter-subject differences in channel activation. For C channels of EEG data, the entire C channel set is divided into two non-overlapping subsets C1 and C2. The mixed data is:

[0037] in, Represents a binary mask of 0, 1. Corresponding to the channel in C1 1 for the behavior, 0 for the other behaviors.

[0038] (3) Frequency mixing: Frequency mixing can capture the discriminative features of multiple spectral bands, which is very important for EEG analysis tasks. EEG data has F frequency bands. This method divides the entire F frequency band set into two non-overlapping subsets F1 and F2. The mixed data is:

[0039] in, A binary mask representing 0 and 1. The columns of Mc(F1) corresponding to the frequency bands in F1 are 1, and the other columns are 0.

[0040] Through the above method, a mixed data label pair is obtained.

[0041] Regarding the training of the global model, considering that the SGD algorithm used to update local models often causes these models to overfit to their own subsets, destroying the generalization ability of the global model and making it difficult for the global model to classify EEG across subjects, the mixEEG method of this method uses mixed data to enhance the generalization ability of the global EEG model learned in the FL setting.

[0042] Specifically, a global model is used to determine a predicted label of the mixed data in the mixed data label pair; The global model is trained based on the loss function determined by the mixed label in the predicted label and the mixed data label until the preset training target is achieved. Similarly, for the DG FL setting, the local update strategy of the SGD local client can also be replaced by mixup, that is, the local model is updated using the mixup technology .

[0043] For step S13, it is not enough to train the global EEG model in step S12. Since the training data comes from the client, domain-adaptive federated learning is also required for the distributed clients to achieve overall federated learning.

[0044] Obtaining unlabeled shared EEG data from target domain data based on the sharing rate and aggregation parameters using the shared data end includes: Determine the amount of unlabeled shared EEG data obtained by the sharing rate and the number of EEG data in the target domain EEG data label pair Dt; Setting an aggregation parameter s, wherein the aggregation parameter s is used to aggregate the data quantity of a single average data; According to the target domain data x t The aggregation parameter s determines the unlabeled shared EEG data ,in: .

[0045] For each unlabeled shared EEG data Generate a pseudo label , where c is the number of emotion categories; Align the pseudo labels with the source domain EEG data labels Based on onehot encoding, shared mixing is performed to obtain mixed labels ,in, , the source domain EEG data label pairs are determined by multiple decentralized clients.

[0046] In this implementation, compared to DG FL, the DA FL (Domain Adaptive Federated Learning) setting allows the client to access unlabeled data from the target domain. , making cross-subject analysis easier, but at the expense of the target domain Therefore, it is necessary to consider the trade-off between privacy protection and data sharing. This method defines a sharing ratio , represents the ratio of the amount of shared data to the target domain data. When r = 0, the DA FL setting is equal to the DG FL setting. A larger r indicates more data communication but less privacy protection.

[0047] For DA FL settings, local clients can see Unlabeled data. Two questions need to be answered: Q1: What is shared data? Q2: How can local clients utilize this unlabeled data? For question Q1, this method considers direct sharing The subset of will certainly damage the privacy security of the target domain. Therefore, this method averages the target domain data To generate shared data. Specifically, this method defines a hyperparameter , represents the number of data used to aggregate a single average data. This method randomly shuffles and from The multiple s data are linearly interpolated to generate a single shared data using the following formula. This process is repeated times to obtain sufficient shared data.

[0048]

[0049] For question Q2, by assuming that the probability of each emotion in the target dataset is equal, this method generates a pseudo label for each shared data: , where c is the number of emotion classes. After generating shared data and corresponding labels, mixEEG can utilize these data in the target dataset to improve the transfer capability. Different from the DG FL problem, in the DG FL problem, our method mixes the data in a single batch. Each data label pair in Mixed with shared data. Decode One() into onehot encoding, and the corresponding mixed label is:

[0050] In step S14, multiple decentralized clients are continuously updated through the above-mentioned domain generalization federated learning and domain adaptation federated learning, and a global model for cross-subject EEG classification is trained by enabling these multiple decentralized clients to collaborate.

[0051] As demonstrated in the preceding implementation, this method proposes a novel federated learning framework for mixEEG, customizing mixup for each EEG modality to unlock its potential for developing better cross-subject EEG brain-computer interfaces (EEG BCIs). By blending only the local EEG data of each client, mixEEG further enhances the generalization of the global model in a DG FL setting. By leveraging unlabeled data from the target domain, even better performance is achieved in a DG FL setting.

[0052] This method is experimentally demonstrated using two datasets: an epilepsy detection dataset and an emotion recognition dataset. Differential entropy (DE) features are extracted for classification. The CHB-MIT dataset is a dataset of 23 patients with epilepsy. This method crops and resamples the CHB-MIT dataset to construct an ED dataset containing four types of EEG: ictal, preictal, postictal, and interictal EEG.

[0053] Emotion Recognition (ER): SEED data records 62 channels of EEG data from 15 subjects, corresponding to three emotions: positive, neutral, and negative.

[0054] Each subject is treated as a decentralized source domain, and results are reported under the leave-one-subject-out (LOSO) setting. Specifically, our method selects n-1 subjects as source domains, trains a global model using FL methods on the source domains (direct data communication between source domains), and tests the performance on the last remaining subject, which is treated as the target domain.

[0055] The global server updates T = 50 times, and each client updates E = 5 times locally. This method uses SGD as the optimizer with a learning rate η = 0.01. The fraction φ is set to 0.2 for the ER dataset and 0.3 for the ED dataset. For the DA FL setting, the sharing ratio r is 0.1, and the aggregation hyperparameter s = 10.

[0056] There are many methods for classifying EEG data. This method selects the following two: (1) MLP: Multi-layer perception, here we use MLP with 2 hidden layers of 128 and 64 neurons.

[0057] (2) CNN: Convolutional neural network. Here, a 3-layer CNN with kernel sizes {(3,3), (3,3), and (7,1)} is used. The activation function is the GELU function, and the dropout probability is 0.5 for all models.

[0058] This method evaluates three hybrid approaches, but there are more implementation details to be evaluated: (1) Linear mixing: In this paper, two cases are tested: α = 0.2 and α = 5.

[0059] (2) Channel mixing: This method tests two situations: ① Divide the EEG channel into left scalp and right scalp; ② Randomly divide the EEG channels into two groups. For both cases, this method sets λ = 0.5.

[0060] (3) Frequency mixing: For DE features with five frequency bands (δ, θ, α, β, and γ), this method tests two cases: ① Divide the EEG frequency bands into {α, β, γ} and {δ, θ}; ② The EEG frequency bands are divided into {δ, α, γ} and {θ, β}. For both cases, this method sets λ to 0.6.

[0061] In the DG FL setting, all clients can only access their own data. This method compares the hybrid EEG with the baseline FedAvg method. LOSO results are shown in Figure 4As shown in Figure 3, simply updating the client model with local mixing already enhances the transferability of the global model, as most of the results of mixEEG are higher than those of the baseline FedAvg. mixEEG improves the cross-subject EEG classification from 77.0% to 86.3% for the ER task and from 43.6% to 47.6% for the ED task.

[0062] Under DA FL settings, there are Unlabeled data is shared from the target domain to the local client. The results are as follows Figure 5 As shown in Figure 3, it can be observed that mixEEG improves the performance by 46% to 49% for the ED task and by 90% to 93% for the ER task.

[0063] In this case, the channel mixing method using binary segmentation outperformed other mixing strategies for the MLP architecture on both datasets. For the CNN architecture, linear mixing showed the best improvement on both tasks. However, frequency mixing did not perform as well in the DAFL setting, and with the exception of the MLP architecture on the ED dataset, frequency mixing always compromised generalizability. Considering the performance in the DG FL setting, it can be concluded that frequency mixing may not be suitable for the EEG modality.

[0064] Through experiments on the DG / DA FL setting, it can be found that the performance of linear mixing is affected by the hyperparameter α. Therefore, an additional experiment is conducted to evaluate the impact of α. 14 values of α are selected and the performance of the CNN model is shown on the ED dataset, as shown in Figure 2. Figure 6 As shown in Figure 2, we can see that in this case, the performance is best when α is around 0.3. When α is greater than 1.0, the performance is similar.

[0065] This method further trains a global model with different numbers of source domain clients. Figure 7 As shown in Figure 3, for all hybrid strategies, performance fluctuates due to significant differences in EEG data across subjects. However, despite this fluctuation, performance improves when the training set becomes larger, suggesting that larger training data helps improve generalization.

[0066] We conducted additional experiments by adjusting the sharing ratio r and the aggregation hyperparameter s to verify the effectiveness of data sharing. r ranged from 0.01 to 0.1, and s ranged from 5 to 50 (in steps of 5). Since this experiment was quite large (consisting of 100 sets of hyperparameters), we only ran the global model for 10 iterations, i.e., T=10 in this case. Figure 8As shown in Figure 3, it can be observed that when the sharing rate r increases, the performance also improves, which indicates that more shared data helps cross-subject EEG classification. It can also be seen that although the performance fluctuates, larger aggregations can achieve better results.

[0067] To better present the differences between each hybrid strategy and explore why some hybrid strategies are better at learning transferable features, this method uses the UMAP technique to visualize the output of the last layer of the global model, i.e., the learned features of the target domain data.

[0068] like Figure 9 Figure 2 shows the learned features of the MLP architecture on the SEED dataset. As can be seen, the features of FedAvg have a somewhat chaotic distribution in the upper left corner. Linear mixing tends to learn more dispersed features, and therefore may improve generalization in some cases. In this case (MLP on the ER task), binary channel mixing is the best. As can be seen, because features of the same class are more tightly clustered, the learned features are more distinguishable. Frequency mixing also enables the global model to learn more tightly clustered features, but the boundaries between classes are relatively blurred, resulting in suboptimal generalization.

[0069] In summary, this method investigates two novel problem settings in EEG-based brain-computer interfaces: the DG FL setting and the DA FL setting. We propose a simple yet effective framework, mixEEG, and investigate various tailored blending strategies on two datasets. We demonstrate that, by sharing averaged data, mixEEG can leverage this unlabeled data to further improve the generalization of the global model. The experiments above validate the effectiveness of mixEEG and show that linear blending and channel blending can improve generalization.

[0070] like Figure 10 The figure shows a structural diagram of a federated learning system for cross-subject EEG classification provided by one embodiment of the present invention. The system can execute the federated learning method for cross-subject EEG classification described in any of the above embodiments and be configured in a terminal.

[0071] This embodiment provides a federated learning system 10 for cross-subject EEG classification, including: a framework construction module 11, a domain generalization module 12, a domain adaptation module 13 and a federated learning module 14.

[0072] Among them, the framework construction module 11 is used to construct a federated learning framework based on multiple distributed clients, a global server, a global model, and a shared data terminal, wherein the multiple distributed clients respectively store EEG data label pairs across subjects, and the data between the clients are not interoperable; the domain generalization module 12 is used to extract the EEG data label pairs from the multiple distributed clients through the global server, perform mixed data enhancement according to the differential entropy features, and obtain mixed data label pairs, and use the mixed data labels to perform domain generalization federated learning on the global model, wherein the mixed data enhancement includes: linear mixing, channel mixing, and frequency mixing; the domain adaptation module 13 is used to use the shared data terminal to obtain unlabeled shared EEG data from the target domain data based on the sharing rate and aggregation parameters, and generate mixed labels for the unlabeled shared EEG data to provide them to the multiple distributed clients for domain adaptive federated learning; the federated learning module 14 is used to enable the multiple distributed clients to collaborate to train a global model for cross-subject EEG classification through the domain generalization federated learning and the domain adaptive federated learning.

[0073] An embodiment of the present invention further provides a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions can execute the federated learning method for cross-subject EEG classification in any of the above method embodiments; As an embodiment, the non-volatile computer storage medium of the present invention stores computer-executable instructions, and the computer-executable instructions are configured as follows: A federated learning framework is constructed based on multiple distributed clients, a global server, a global model, and a shared data terminal. The multiple distributed clients each store EEG data label pairs across subjects, and data between the clients is not interoperable. The global server extracts the EEG data label pairs from the multiple distributed clients and performs mixed data enhancement according to differential entropy features to obtain mixed data label pairs, and uses the mixed data labels to perform domain generalization federated learning on the global model, wherein the mixed data enhancement includes: linear mixing, channel mixing, and frequency mixing; Obtaining unlabeled shared EEG data from target domain data based on the sharing rate and aggregation parameters using the shared data end, generating hybrid labels for the unlabeled shared EEG data, and providing them to the multiple distributed clients for domain adaptive federated learning; The domain generalization federated learning and the domain adaptation federated learning enable the multiple distributed clients to collaboratively train a global model for cross-subject EEG classification.

[0074] A non-volatile computer-readable storage medium can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the methods in the embodiments of the present invention. One or more program instructions stored in the non-volatile computer-readable storage medium, when executed by a processor, perform the federated learning method for cross-subject EEG classification in any of the above-described method embodiments.

[0075] Figure 11 This is a schematic diagram of the hardware structure of an electronic device for a federated learning method for cross-subject EEG classification provided by another embodiment of the present application, such as Figure 11 As shown, the device includes: One or more processors 1110 and memory 1120, Figure 11 A processor 1110 is used as an example. The apparatus for the federated learning method for cross-subject EEG classification may further include: an input device 1130 and an output device 1140.

[0076] The processor 1110, the memory 1120, the input device 1130 and the output device 1140 may be connected via a bus or other means. Figure 11 The bus connection is taken as an example.

[0077] Memory 1120, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs, and modules, such as the program instructions / modules corresponding to the federated learning method for cross-subject EEG classification in the embodiments of the present application. Processor 1110 executes the various functional applications and data processing of the server by running the non-volatile software programs, instructions, and modules stored in memory 1120, thereby implementing the federated learning method for cross-subject EEG classification in the above-mentioned method embodiment.

[0078] The memory 1120 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data, etc. In addition, the memory 1120 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 1120 may optionally include a memory remotely located relative to the processor 1110, and these remote memories may be connected to the mobile device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0079] The input device 1130 can receive input digital or character information. The output device 1140 can include a display device such as a display screen.

[0080] The one or more modules are stored in the memory 1120 , and when executed by the one or more processors 1110 , perform the federated learning method for cross-subject EEG classification in any of the above method embodiments.

[0081] The above-mentioned product can execute the method provided in the embodiment of this application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of this application.

[0082] The non-volatile computer-readable storage medium may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the device, etc. In addition, the non-volatile computer-readable storage medium may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state memory device. In some embodiments, the non-volatile computer-readable storage medium may optionally include a memory remotely located relative to the processor, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0083] An embodiment of the present invention also provides an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the federated learning method for cross-subject EEG classification of any embodiment of the present invention.

[0084] The electronic devices of the embodiments of the present application exist in various forms, including but not limited to: (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and their primary purpose is to provide voice and data communications. These terminals include smartphones, multimedia phones, feature phones, and low-end phones.

[0085] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers and have computing and processing capabilities, and generally also have mobile Internet access. These terminals include PDAs, MIDs, and UMPC devices, such as tablet computers.

[0086] (3) Portable entertainment devices: These devices can display and play multimedia content. They include audio and video players, handheld game consoles, e-books, smart toys, and portable car navigation devices.

[0087] (4) Other electronic devices with data processing functions.

[0088] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include" and "comprise" include not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article or device. In the absence of further limitations, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device that includes the elements.

[0089] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0090] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A federated learning method for cross-subject EEG classification, comprising: A federated learning framework is constructed based on multiple distributed clients, a global server, a global model, and a shared data terminal. The multiple distributed clients each store EEG data label pairs across subjects, and data between the clients is not interoperable. The global server extracts the EEG data label pairs from the multiple distributed clients and performs mixed data enhancement according to differential entropy features to obtain mixed data label pairs, and uses the mixed data labels to perform domain generalization federated learning on the global model, wherein the mixed data enhancement includes: linear mixing, channel mixing, and frequency mixing; Obtaining unlabeled shared EEG data from target domain data based on the sharing rate and aggregation parameters using the shared data end, generating hybrid labels for the unlabeled shared EEG data, and providing them to the multiple distributed clients for domain adaptive federated learning; The domain generalization federated learning and the domain adaptation federated learning enable the multiple distributed clients to collaboratively train a global model for cross-subject EEG classification.

2. The method according to claim 1, wherein The step of extracting the EEG data labels from the plurality of dispersed clients through the global server and performing hybrid data enhancement according to differential entropy features includes: Extract a first data tag pair (x) from the plurality of dispersed clients i ,y i ) and the second data label pair (x j ,y j ) Perform hybrid data enhancement according to differential entropy features, where the x i With the x j is the EEG data, the y i With the y j The number of channels of the EEG data in the first data label pair and the second data label pair is C, and the number of frequency bands is F.

3. The method according to claim 2, wherein: When the mixed data enhancement is frequency mixing, the method includes: The first data label pair (x i ,y i ) and the second data label pair (x j ,y j ) Divide into non-overlapping subsets F1 and F2 according to the number of frequency bands F; The first data label pair and the second data label pair are mixed according to the frequency to obtain mixed data for capturing the discriminative characteristics of the spectral band ,in: described , the Mc is a 01 mask matrix.

4. The method according to claim 2, wherein When the mixed data is enhanced to channel mixing, the method includes: The first data label pair (x i ,y i ) and the second data label pair (x j ,y j ) Divide into non-overlapping subsets C1 and C2 according to the number of channels C; The first data label pair and the second data label pair are mixed according to the channel to obtain mixed data for learning local spatial representation ,in: described , the Mr is a 01 mask matrix.

5. The method according to claim 2, wherein: When the mixed data enhancement is linear mixing, the method includes: The first data label pair and the second data label pair are mixed according to linear interpolation to obtain mixed data ,in: described , the λ is a preset hyperparameter.

6. The method according to claim 2, wherein: The mixed label in the mixed data label pair , the λ is a preset hyperparameter.

7. The method according to claim 1, wherein The target domain data is obtained from a pre-prepared target domain EEG data label pair Dt (xt, yt), and the target domain data is xt.

8. The method according to claim 7, wherein: Obtaining unlabeled shared EEG data from target domain data based on the sharing rate and aggregation parameters using the shared data end includes: Determine the amount of unlabeled shared EEG data obtained by the sharing rate and the number of EEG data in the target domain EEG data label pair Dt; Setting an aggregation parameter s, wherein the aggregation parameter s is used to aggregate the data quantity of a single average data; Determine the unlabeled shared EEG data according to the target domain data xt and the aggregation parameter s ,in: 。 9. The method according to claim 8, wherein Generating a mixed label for the unlabeled shared EEG data includes: For each unlabeled shared EEG data Generate a pseudo label , where c is the number of emotion categories; Align the pseudo labels with the source domain EEG data labels Based on onehot encoding, shared mixing is performed to obtain mixed labels ,in, , the source domain EEG data label pairs are determined by multiple decentralized clients.

10. The method according to claim 1, wherein The performing domain generalization federated learning on the global model using the mixed data labels includes: Determining predicted labels for the mixed data in the mixed data label pairs using a global model; The global model is trained based on a loss function determined by the predicted label and the mixed label in the mixed data label pair until a preset training target is reached.

11. A federated learning system for cross-subject EEG classification, comprising: A framework construction module is used to build a federated learning framework based on multiple distributed clients, a global server, a global model, and a shared data terminal, wherein the multiple distributed clients respectively store EEG data label pairs across subjects, and the data between the clients is not interoperable; A domain generalization module is configured to extract the EEG data label pairs from the multiple distributed clients through the global server and perform mixed data enhancement according to differential entropy features to obtain mixed data label pairs, and perform domain generalization federated learning on the global model using the mixed data labels, wherein the mixed data enhancement includes: linear mixing, channel mixing, and frequency mixing; a domain adaptation module, configured to obtain unlabeled shared EEG data from target domain data using the shared data end based on a sharing rate and an aggregation parameter, and generate hybrid labels for the unlabeled shared EEG data to provide to the multiple distributed clients for domain adaptive federated learning; A federated learning module is used to enable the multiple distributed clients to collaboratively train a global model for cross-subject EEG classification through the domain generalization federated learning and the domain adaptation federated learning.

12. A storage medium having a computer program product stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

13. A computer program product having instructions embedded on a storage medium, wherein the instructions implement the steps of the method according to any one of claims 1 to 10.

14. An electronic device comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the method according to any one of claims 1 to 10.

Images