Multitask motor imagery electroencephalogram decoding method based on federal learning

Abstract

The invention discloses a multitask motor imagery electroencephalogram decoding method based on federal learning, and belongs to the technical field of motor imagery brain-computer interfaces. In order to solve the problems that electroencephalogram data are high in isomerism and federated learning communication efficiency is low, a double-server federated learning framework and a knowledge distillation mixed communication method are designed and used for motor imagery electroencephalogram decoding. Once the server selects and shares the optimal features of the local model, the client updating strategy is improved, and the problem of federated learning client drift is solved. And the second server aggregates local model parameters and calculates global distillation loss based on knowledge distillation soft labels, so that fine adjustment and optimization of the global model parameters are realized, the model is enhanced to adapt to heterogeneous electroencephalogram data, and the universality of a federal learning algorithm is improved. A hybrid communication method is designed between the second server and the client, a client FL node adopts a lightweight model to transmit parameters, an FD node utilizes complex model knowledge to distill and transmit soft labels, and the communication overhead of federal learning is reduced.

Description

Technical Field

[0001] This invention belongs to the field of motor imagery brain-computer interface technology, specifically relating to a multi-task motor imagery EEG decoding method based on federated learning. Background Technology

[0002] Brain-computer interface (BCI) technology establishes a direct communication pathway between the brain and external devices, demonstrating great potential in fields such as neurorehabilitation, intelligent control, and human-computer interaction. Motor imagery brain-computer interface (MI-BCI), as an important branch of BCI, analyzes the electroencephalogram (EEG) signals generated when a user imagines limb movements, enabling control of external devices without actual action, thus providing entirely new interactive possibilities for patients with motor dysfunction.

[0003] Based on signal acquisition methods, brain-computer interfaces (BCIs) can be divided into invasive and non-invasive types. Non-invasive BCIs, due to their high safety and ease of use, have become the mainstream in current research and application. However, non-invasive BCIs exhibit low signal-to-noise ratios and are typically non-stationary with significant individual variability. This data heterogeneity, caused by the influence of physiological structure, psychological state, and other factors on different users' EEG data, makes it difficult for general decoding models trained on centralized data to adapt to new users, severely limiting the performance and generalization ability of motor imagery EEG decoding models.

[0004] Federated learning (FL) is a distributed machine learning paradigm that trains local models on various clients within a distributed network and shares these models with a global model on a server, ultimately constructing a universal global model. During federated learning, participating clients update their local model parameters using local data, and the server iteratively optimizes the model by aggregating these local parameters until the global model converges. Federated learning clients and the server only transmit model parameters, not local data, protecting local data privacy while ensuring that the server's global model achieves optimal performance across all clients. This distributed collaboration, with multiple clients jointly training the globally optimal model, provides a new approach to decoding heterogeneous EEG data from multiple subjects, offering superior decoding performance compared to traditional centralized training paradigms in this scenario.

[0005] However, applying federated learning directly to the MI-BCI field still faces two major bottlenecks: First, FL uses average aggregation of client model parameters, ignoring the fact that the local model parameter update direction tends to the local optimum, which leads to client drift problem, making it difficult for the server to converge to the global optimum during aggregation, ultimately reducing the model decoding accuracy; Second, federated learning requires multiple rounds of frequent model parameter transmission between the client and the server, which brings high communication overhead and limits its deployment in resource-constrained environments.

[0006] To address communication overhead issues, compression optimization has been widely adopted by researchers in federated learning as an effective method. Currently, there are three main compression optimization methods in federated learning: pruning, quantization, and knowledge distillation. Pruning removes weights and branches from the model that have little or no impact on the output, thus avoiding redundant parameter exchanges. This results in a smaller memory footprint and faster processing speed, but improper pruning can lead to decreased model accuracy, and requires active selection of pruning strategies and parameter adjustment. Quantization reduces storage, computation, and communication overhead by decreasing the precision of data representation, but often results in a loss of accuracy, complex parameter tuning after quantization, and incompatibility issues.

[0007] Federated knowledge distillation (FD), as a model compression technique, has emerged as a novel approach to address the communication efficiency problem in federated knowledge distillation (FL). In FD, clients no longer directly upload model parameters but instead upload their predictions of the local dataset (i.e., soft labels). This offers two advantages: First, the amount of soft label data is significantly smaller than that of model parameters, substantially reducing communication overhead between the client and server. Wu et al. applied FD to personalized news recommendation and text detection, proposing an adaptive mutual distillation method that uses soft labels as communication content, reducing communication overhead to 94.89% of the FL method while maintaining model accuracy. Second, the FL method requires all clients to use the same model architecture, while FD transmits knowledge through soft labels, allowing clients to choose differentiated models based on hardware limitations or data characteristics. Li et al.'s FedMD algorithm validated the feasibility of clients using models of varying complexity on the MNIST handwritten digit dataset and achieving co-training through knowledge distillation. However, the application of FD in brainwave imagery (BCI) has not been fully explored. Existing research struggles to achieve an effective balance between EEG decoding accuracy and communication efficiency, thus failing to improve the performance of both. Summary of the Invention

[0008] To address the issues of high heterogeneity in current motor imagery EEG data and low communication efficiency in federated learning, this invention provides a multi-task motor imagery EEG decoding method based on federated learning.

[0009] To achieve the above objectives, the present invention employs the following technical solutions:

[0010] A multi-task motor imagery EEG decoding method based on federated learning, the method comprising the following steps:

[0011] Step 1: Design a dual-server federated learning framework consisting of two servers and multiple clients. Each subject is treated as a client. At each client, local motor imagery EEG data is preprocessed and features are extracted, converting EEG data from different subjects into a unified format. The two servers are Server 1 and Server 2. Server 1 serves as a feature sharing server, where clients share its best features. Server 2 is responsible for optimizing the global model, employing two strategies: parameter aggregation and knowledge distillation.

[0012] The specific operation of step 1 is as follows:

[0013] Step 1.1: Bandpass filtering is performed on the motor imagery EEG signal to obtain the EEG rhythms related to motor imagery. The frequency of the bandpass filtering is 8Hz~30Hz, which includes the μ rhythm (8~13Hz) and β rhythm (14~30Hz) of the motor imagery EEG data.

[0014] Step 1.2: Perform Euclidean alignment on the filtered motor imagery EEG data to eliminate edge distribution differences;

[0015] Step 1.3: The aligned motor imagery EEG data is amplified using a time window sliding method. The time window is set to a preset duration and the data is slid at time intervals with a preset step size. The number of time sampling points for each data sample is changed from the original number to the number corresponding to the time window length.

[0016] Step 1.4: Optional The covariance matrix of EEG data from motor imagery tasks is calculated. Then, the covariance matrix is ​​decomposed into generalized eigenvalues ​​to obtain a projection matrix of a specific dimension. A number of feature vectors are selected from the projection matrix of each motor imagery task category. The feature vectors of each category are used to construct a spatial filter. The spatial filter is used to perform spatial filtering on the motor imagery EEG data to obtain a set of spatially filtered motor imagery EEG data. After co-spatial pattern preprocessing, the EEG data from different datasets and different acquisition devices are unified into a data matrix of the same size, i.e., the EEG co-spatial feature dimension × the number of EEG sampling points.

[0017] Step 2: Divide multiple clients into FL nodes and FD nodes; each client trains and updates its local model based on the preprocessed EEG data, combined with global model parameters and global soft labels; the global model is a complex model containing convolutional blocks, a Transformer multi-head attention module and a classification module, and the local model is divided into a lightweight model and a complex model. The lightweight model contains convolutional blocks and a classification module, while the complex model is the same as the global model.

[0018] In step 2, the convolutional blocks of the global model and the local model consist of two temporal convolutions and one spatial convolution to extract local spatiotemporal features of EEG, and batch normalization and average pooling operations are connected after each convolution.

[0019] In step 2, the Transformer multi-head attention module consists of four self-attention heads. The input sequence is processed in parallel by the four independent attention heads. Each head focuses on a different part of the time sequence to capture the long-distance dependencies of the motor imagery EEG sequence.

[0020] The specific operation of step 2 is as follows:

[0021] Step 2.1: Sort the data according to the classification accuracy of each client in the first round of training, and select the top... One client is an FL node, and the rest of the clients are FD nodes;

[0022] Step 2.2: The FL node client updates and optimizes the local model using local data and the distributed global model parameters; the FD node client updates and optimizes the local model using local data and the distributed global soft labels; the local data is EEG data, and the features extracted by the local model from the local data are local features; the specific update and optimization operations are as follows:

[0023] Each client adopts the best feature selected by the server. ,Will As a globally shared resource, and based on minimizing the maximum mean difference (MMD), it minimizes the optimal feature and local features. By minimizing the maximum mean difference distance, the distribution difference between two domains is reduced, guiding the training direction of multiple clients to converge towards the globally optimal feature, effectively solving the client drift problem caused by heterogeneous EEG data. Optimal Feature and the Local characteristics of each client Represented as: , During local model training, the MMD value of the best feature and the local features is calculated, and the loss value is... Represented as:

[0024]

[0025] In the formula, It is a function that maps the original data to the regenerated Hilbert space; the MMD value represents the difference in distribution between the best feature and the local feature, and the smaller the difference, the smaller the MMD value.

[0026] Objective function for local training on FL nodes for:

[0027]

[0028] In the formula, Weighting coefficients to balance the MMD loss; Represents cross-entropy loss;

[0029] FD nodes add distillation loss for global and local soft tags, and their objective function... for:

[0030]

[0031] In the formula, Indicates the dynamic weighting coefficient; Indicates a local soft tag; This indicates that the global model outputs soft labels; This represents the KL divergence distillation loss between the local soft label and the global model output soft label;

[0032] Step 3: Each client uploads the EEG features extracted from its local model to Server 1; FL nodes upload the local model parameters to Server 2, and FD nodes upload the local soft tags to Server 2;

[0033] Step 4: Server 1 selects the best features from the received EEG features; Server 2 uses the received local model parameters and local soft labels to fine-tune and update the parameters of the global model.

[0034] The specific operation of step 4 is as follows:

[0035] The server selects the best feature from the received EEG features based on the highest classification accuracy. The formula for selecting the best feature is as follows:

[0036]

[0037] In the formula, This represents the best feature selected by server 1; This indicates the local characteristics of each client uploading to server 1;

[0038] Server 2 uploads local model parameters to FL nodes Perform a weighted average and use it as a parameter for the global model. It also aggregates the local soft tags uploaded by the FD node, using the following formula:

[0039]

[0040] In the formula, Indicates the first Local data sample size for each client; This represents the local data sample size across all clients. Indicates the number of clients; Indicates the first Local model parameters uploaded by each client; This represents the aggregated soft tag; Indicates the first Local soft tags uploaded by each client;

[0041] Server 2 uses aggregated model parameters, soft labels, and data shared by all clients to fine-tune and optimize the global model parameters, forming the final global model, whose loss function is... As shown below:

[0042] By leveraging the synergy of two servers, the client drift problem can be mitigated to the greatest extent possible, thereby improving the model's generalization ability.

[0043]

[0044] In the formula, For cross-entropy loss, Output soft labels for the global model With aggregated soft tags KL divergence distillation loss, dynamic weighting coefficient Using a linear growth strategy, the formula is as follows:

[0045]

[0046] In the formula, The total number of communications represents the amount of knowledge in the soft tags that dynamically changes with each round of communication during the training process. This represents the minimum weight coefficient; This represents the largest weighting coefficient; Indicates the current number of communications.

[0047] In the early stages of training, soft labels contain less knowledge, so set smaller initial weights. To reduce the impact of soft tags; as the number of communication rounds increases, soft tags gradually stabilize and contain more effective knowledge information, therefore the distillation weight is linearly increased to... This allows for full utilization of soft-label knowledge in the later stages of training to further improve model performance.

[0048] Step 5: Server 1 sends the selected best features to all clients, and Server 2 sends the updated global model parameters to FL nodes and the updated global soft labels to FD nodes.

[0049] Step 6: Repeat steps 2 to 5 until the global model converges, obtaining the optimal federated learning model for EEG decoding.

[0050] The method for calculating the communication overhead of the method is as follows:

[0051] The communication overhead consists of the interaction between the client and the two servers; let the total number of communications be... The number of clients is ,Include FL node clients and One FD node client; the communication overhead calculation method is as follows:

[0052] 1) All client-server interactions involve the transmission of H-dimensional features, and the communication overhead is: ;

[0053] 2) Each FL node interacts with the server to transmit lightweight models. There are several parameters, and the communication overhead is as follows: ;

[0054] 3) Each FD node transmits motion images to the server. Categories The communication overhead for knowledge distillation, achieved by using soft labels for each sample, is as follows: ;

[0055] The total communication overhead is the sum of the three: .

[0056] Compared with the prior art, the present invention has the following advantages:

[0057] This invention provides a federated learning-based multi-task motor imagery EEG decoding method. The method's dual-server federated learning architecture, improved client update strategy, and global model parameter optimization effectively alleviate the client drift problem in federated learning, improving the decoding accuracy of heterogeneous motor imagery EEG data. Secondly, the proposed client differentiation model combined with a hybrid communication method leverages the advantages of parameter communication and knowledge distillation to achieve a high accuracy in multi-task motor imagery classification while maintaining a communication compression rate of 96.34%, thus improving decoding efficiency. This algorithm provides a new technical approach for the application of federated learning in the field of brain-computer interfaces, and its design concept has significant reference value for handling other scenarios with data heterogeneity and resource constraints. Attached Figure Description

[0058] Figure 1 A framework diagram for a federated learning method for decoding motor imagery EEG signals based on a dual-server architecture and hybrid communication.

[0059] Figure 2 Structure diagrams for lightweight and complex models;

[0060] Figure 3 The training accuracy and loss curves are shown on the BCI IV 2a and BCI III IIIa datasets.

[0061] Figure 4 Experimental diagrams for different numbers of client nodes;

[0062] Figure 5 A dimensionality reduction visualization of the characteristics of four types of motion imagination tasks. Detailed Implementation

[0063] To gain a deeper understanding of this invention, we will provide a comprehensive and detailed description. However, this invention has various implementations and is not limited to the specific examples listed herein. These examples are presented to enhance a full understanding of the disclosure of this invention.

[0064] A multi-task motor imagery EEG decoding method based on federated learning is proposed. This method balances model performance and communication overhead by designing a dual-server architecture based on knowledge distillation, a differentiated model, and a hybrid communication strategy. The method includes the following steps:

[0065] Step 1: Design a dual-server federated learning framework consisting of two servers and multiple clients. Each subject is treated as a client. At each client, local motor imagery EEG data is preprocessed and features are extracted, converting EEG data from different subjects into a unified format. The two servers are Server 1 and Server 2. Server 1 serves as a feature sharing server, where clients share its best features. Server 2 is responsible for optimizing the global model, employing two strategies: parameter aggregation and knowledge distillation.

[0066] like Figure 1 As shown, in the dual-server architecture, server one acts as a feature-sharing server, responsible for processing the EEG features extracted from the local models of each client. By dynamically selecting the best features from each client model and sharing them globally, it achieves knowledge transfer at the feature level, effectively solving the client drift problem in federated learning. Server two is responsible for optimizing the global model, integrating parameter aggregation and knowledge distillation communication strategies. It can both optimize the global model using parameter information and reduce communication overhead through soft-label communication.

[0067] The specific operation of step 1 is as follows:

[0068] Step 1.1: Bandpass filtering is performed on the motor imagery EEG signal to obtain the EEG rhythms related to motor imagery. The frequency of the bandpass filtering is 8Hz~30Hz, which includes the μ rhythm (8~13Hz) and β rhythm (14~30Hz) of the motor imagery EEG data.

[0069] Step 1.2: Perform Euclidean alignment on the filtered motor imagery EEG data to eliminate edge distribution differences;

[0070] Step 1.3: The aligned motor imagery EEG data is amplified using a time window sliding method. The time window is set to a preset duration and the data is slid at time intervals with a preset step size. The number of time sampling points for each data sample is changed from the original number to the number corresponding to the time window length.

[0071] Step 1.4: Calculate the covariance matrix of EEG data from four randomly selected motor imagery tasks; then perform generalized eigenvalue decomposition on the covariance matrix to obtain a projection matrix of a specific dimension; select a number of feature vectors from the projection matrix of each motor imagery task category, and construct a spatial filter using the feature vectors of each category; use the spatial filter to perform spatial filtering on the motor imagery EEG to obtain a set of spatially filtered motor imagery EEG data; after co-spatial pattern preprocessing, EEG data from different datasets and different acquisition devices are unified into a data matrix of the same size, i.e., EEG co-spatial feature dimension × number of EEG sampling points.

[0072] Step 2: Divide multiple clients into FL nodes and FD nodes; each client trains and updates its local model based on the preprocessed EEG data, combined with global model parameters and global soft labels; the global model is a complex model containing convolutional blocks, a Transformer multi-head attention module and a classification module, and the local model is divided into a lightweight model and a complex model. The lightweight model contains convolutional blocks and a classification module, while the complex model is the same as the global model.

[0073] In terms of client design, this invention meticulously categorizes clients based on the quality of their EEG data. Clients with high-quality EEG data are designated as FL nodes, employing a lightweight model and using parametric communication; clients with low-quality data are designated as FD nodes, using a complex model with the same architecture as the global model, and uploading soft tags via knowledge distillation. This differentiated design ensures both the computational efficiency of clients with high-quality data and the feature extraction capabilities of clients with low-quality data. Furthermore, each client possesses local EEG data of one user (subject) and shares some data with the server for fine-tuning and optimization of the global model, making the global model better suited to all users.

[0074] The specific operation of step 2 is as follows:

[0075] Step 2.1: Sort the clients according to their classification accuracy in the first round of training, select the top 5 clients as FL nodes, and the remaining clients as FD nodes;

[0076] Step 2.2: The FL node client updates and optimizes the local model using local data and the distributed global model parameters; the FD node client updates and optimizes the local model using local data and the distributed global soft labels; the local data is EEG data, and the features extracted by the local model from the local data are local features; the specific update and optimization operations are as follows:

[0077] Given the heterogeneity of EEG data from different clients, local model training tends to converge to its local optimum, leading to client drift. Therefore, each client incorporates the optimal features selected by the server. ,Will As a globally shared resource, and based on minimizing the maximum mean difference (MMD), it minimizes the optimal feature and local features. By minimizing the maximum mean difference distance, the distribution difference between two domains is reduced, guiding the training direction of multiple clients to converge towards the globally optimal feature, effectively solving the client drift problem caused by heterogeneous EEG data. Optimal Feature and the Local characteristics of each client Represented as: , During local model training, the MMD value of the best feature and the local features is calculated, and the loss value is... Represented as:

[0078] (1)

[0079] In the formula, It is a function that maps the original data to the regenerated Hilbert space; the MMD value represents the difference in distribution between the best feature and the local feature, and the smaller the difference, the smaller the MMD value.

[0080] Objective function for local training on FL nodes for:

[0081] (2)

[0082] In the formula, Weighting coefficients to balance the MMD loss; Represents cross-entropy loss;

[0083] FD nodes add distillation loss for global and local soft tags, and their objective function... for:

[0084] (3)

[0085] In the formula, Indicates the dynamic weighting coefficient; Indicates a local soft tag; This indicates that the global model outputs soft labels; This represents the KL divergence distillation loss between the local soft label and the global model output soft label;

[0086] In the initial phase, all clients uniformly used a lightweight model process for the first round of training, and then selected the top performers based on classification accuracy. One client is designated as the FL node, and the remaining clients as the FD nodes. Starting from the second round of training, each round consists of three training steps: First, after all clients complete their local updates, they upload the EEG features extracted by the model to Server 1. Simultaneously, the FL nodes upload their local model parameters to Server 2, and the FD nodes upload the soft labels output by their local models to Server 2. Second, Server 1 selects the optimal features and distributes them to all clients. Server 2 aggregates the model parameters uploaded by the FL nodes and the soft label knowledge uploaded by the FD nodes. It then uses the aggregated parameters and soft labels to fine-tune and optimize the global model, and distributes the updated model parameters and global soft labels to the FL and FD nodes. Finally, the clients receive the updated parameters and soft labels for the next round of local updates.

[0087] The entire framework achieves multi-level knowledge sharing and global model optimization through the collaborative work of two servers. Server 1's feature sharing effectively solves the client drift problem in federated learning, while Server 2's hybrid communication strategy retains both the model optimization capabilities of parameter updates and the efficiency advantages of soft-label communication. The final optimized global model is used for motion image multi-task recognition, efficiently completing MI-EEG decoding tasks.

[0088] Given the significant differences in the quality of EEG data among different participants, this invention designs a differentiated model strategy: for clients with high-quality data, a lightweight model can achieve good classification results, and a lightweight model is assigned to these clients; for clients with low-quality data, a complex model has stronger feature representation capabilities and can effectively handle low-quality data, therefore a complex model is used. To ensure the performance of the global model on the server side, a complex model structure is also adopted.

[0089] Complex and lightweight model structures, such as Figure 2As shown, the complex model includes convolutional blocks, a Transformer multi-head attention module, and a classification module, while the lightweight model only includes convolutional blocks and a classification module. The convolutional block consists of two temporal convolutions and one spatial convolution, extracting local spatiotemporal features of EEG. Batch normalization and average pooling operations are applied after each convolution. Although the Transformer has higher computational complexity due to its larger number of parameters, it can deeply capture the complex relationships and long-term dependencies in EEG time series, achieving good results in MI-EEG decoding and effectively improving decoding accuracy and feature representation capabilities. Therefore, this invention chooses the Transformer multi-head self-attention mechanism to improve the overall performance of the model. The Transformer multi-head attention module consists of multiple self-attention heads, processing the input sequence in parallel through multiple independent attention heads. Each head focuses on different parts of the sequence, capturing motor imagery EEG feature information and complex temporal relationships. Finally, the classification module outputs the classification result, and the server selects the best feature based on the classification accuracy. This differentiated model design and the application of the Transformer multi-head attention module enable each client model to effectively extract and classify features based on data quality characteristics, thereby improving the performance of motor imagery EEG decoding.

[0090] To ensure parameter communication compatibility between the lightweight model and the server model, the complex model and the lightweight model are designed with the same convolutional block structure. Table 1 shows the specific parameters.

[0091] Table 1. Network structure in the convolutional module

[0092]

[0093] Step 3: Each client uploads the EEG features extracted from its local model to Server 1; FL nodes upload the local model parameters to Server 2, and FD nodes upload the local soft tags to Server 2;

[0094] Step 4: Server 1 selects the best features from the received EEG features; Server 2 uses the received local model parameters and local soft labels to fine-tune and update the parameters of the global model.

[0095] In heterogeneous EEG data scenarios, the traditional single-server architecture of federated learning (FL) is prone to client drift. This is because a single global model struggles to adapt to the highly differentiated local data distributions of each client, and the lack of richer knowledge sharing through information transmission only via model parameters leads to discrepancies between the client's local model and the global model. This invention designs a federated learning dual-server architecture, adding a feature processing server. By providing additional feature knowledge sharing, it guides the client model towards the global optimum, effectively mitigating the client drift problem.

[0096] The specific operation of step 4 is as follows:

[0097] The server selects the best feature from the received EEG features based on the highest classification accuracy. The formula for selecting the best feature is as follows:

[0098] (4)

[0099] In the formula, This represents the best feature selected by server 1; This indicates the local characteristics of each client uploading to server 1;

[0100] Server 2 uploads local model parameters to FL nodes Perform a weighted average and use it as a parameter for the global model. It also aggregates the local soft tags uploaded by the FD node, using the following formula:

[0101] (5)

[0102] (6)

[0103] In the formula, Indicates the first Local data sample size for each client; This represents the local data sample size across all clients. Indicates the number of clients; Indicates the first Local model parameters uploaded by each client; This represents the aggregated soft tag; Indicates the first Local soft tags uploaded by each client;

[0104] Due to variations in local EEG data across clients, the parameters of the trained local models differ significantly, making it difficult for the global model obtained solely through aggregation to adapt to all clients. To improve the universality and generalization ability of the global model, server two fine-tunes and optimizes the global model parameters using the aggregated local model parameters, local soft labels, and data shared across clients, forming the final global model. Its loss function... As shown below:

[0105] (7)

[0106] In the formula, For cross-entropy loss, Output soft labels for the global model With aggregated soft tags KL divergence distillation loss, dynamic weighting coefficient Using a linear growth strategy, the formula is as follows:

[0107] (8)

[0108] In the formula, The total number of communications represents the amount of knowledge in the soft tags that dynamically changes with each round of communication during the training process. This represents the minimum weight coefficient; This represents the largest weighting coefficient; Indicates the current number of communications.

[0109] In the early stages of training, soft labels contain less knowledge, so set smaller initial weights. To reduce the impact of soft tags; as the number of communication rounds increases, soft tags gradually stabilize and contain more effective knowledge information, therefore the distillation weight is linearly increased to... This allows for full utilization of soft-label knowledge in the later stages of training to further improve model performance.

[0110] Step 5: Server 1 sends the selected best features to all clients, and Server 2 sends the updated global model parameters to FL nodes and the updated global soft labels to FD nodes.

[0111] Step 6: Repeat steps 2 to 5 until the global model converges, i.e. the loss function is minimized, and the optimal federated learning model is obtained for EEG decoding.

[0112] The method for calculating the communication overhead of the method is as follows:

[0113] The communication overhead consists of the interaction between the client and the two servers; let the total number of communications be... The number of clients is ,Include FL node clients and One FD node client; the communication overhead calculation method is as follows:

[0114] 1) All client-server interactions involve the transmission of H-dimensional features, and the communication overhead is: ;

[0115] 2) Each FL node interacts with the server to transmit lightweight models. There are several parameters, and the communication overhead is as follows: ;

[0116] 3) Each FD node transmits motion images to the server. Categories The communication overhead for knowledge distillation, achieved by using soft labels for each sample, is as follows: ;

[0117] The total communication overhead is the sum of the three: .

[0118] The overall training process of the algorithm of this invention is as follows:

[0119]

[0120] The hardware environment is an NVIDIA GeForce GTX 1650Ti GPU, and the model hyperparameter settings are shown in Table 2.

[0121] Local model training and global parameter fine-tuning use the same learning rate and batch size.

[0122] Table 2 Model Hyperparameter Settings

[0123]

[0124] The accuracy and loss function of the algorithm trained on the BCI IV 2a and BCI III IIIa datasets of this invention are as follows: Figure 3 As shown.

[0125] The algorithm of this invention analyzes different numbers of FL / FD nodes, such as... Figure 4 As shown in the figure, the lines in the box plot represent the average accuracy and also show the distribution of all experimental data. The horizontal axis represents the number of FL nodes (corresponding to the number of FD nodes, which is the total number of clients minus the number of FL nodes), and records the corresponding communication overhead and communication compression rate.

[0126] This invention's algorithm dynamically adjusts the number of FL nodes to observe the relationship between classification accuracy and communication efficiency. The results show that as the number of FL nodes increases, i.e., the number of clients communicating using model parameters increases, the average classification accuracy of the algorithm generally increases, while the communication compression rate shows a monotonically decreasing trend. This phenomenon proves that parameter communication can transmit richer model knowledge, which is beneficial to improving global model performance, but at the cost of higher communication overhead. When all clients are FD nodes (FL=0), all communication relies solely on soft labels for knowledge distillation, resulting in a communication overhead of only 17.73MB and a communication compression rate of 99.89% on the BCI IV 2a dataset. However, due to the heterogeneity of EEG data, relying solely on soft label communication leads to severe information loss, resulting in the worst system performance.

[0127] It is noteworthy that on the BCI IV 2a dataset, the average classification accuracy of the FL9 node (without hybrid communication, all clients communicated fully with server 2 using model parameters) only increased by 0.57% compared to the FL5 node (using federated distillation hybrid communication, with 5 clients communicating using model parameters and the other 4 clients communicating using soft labels). However, the communication overhead increased from 572.83MB to 1016.91MB, an increase of 77.5%. This indicates that in heterogeneous EEG data scenarios, the federated distillation hybrid communication method has significant advantages. While maintaining the integrity of knowledge in parameter communication, it fully leverages the communication efficiency advantages of knowledge distillation, ultimately achieving the optimal trade-off between communication efficiency and model performance in heterogeneous EEG tasks involving motor imagery.

[0128] To visually demonstrate the differences in EEG data features across the four types of motor imagery tasks, t-SNE was used to reduce the features to two dimensions and visualize them, such as... Figure 5 As shown, the distribution changes of EEG data features in the two datasets are illustrated. The original EEG data feature distribution (Figures a and c) shows that the EEG data for the four types of motor imagery tasks are completely mixed, making it difficult to distinguish the four task categories. The processing results of the method of this invention (Figures b and d) show a significant improvement; the four categories in the feature space can be clearly distinguished, verifying the effectiveness of the dual-server architecture and global model fine-tuning optimization method in handling the heterogeneity of EEG data. Compared with the original EEG, the method of this invention maintains better inter-class separability, indicating that the proposed hybrid communication method has practical value in balancing communication overhead and model performance.

[0129] Contents not described in detail in this specification are prior art known to those skilled in the art. Although illustrative specific embodiments of the invention have been described above to facilitate understanding by those skilled in the art, it should be understood that the invention is not limited to the scope of the specific embodiments. Various modifications are readily apparent to those skilled in the art as long as they fall within the spirit and scope of the invention as defined and determined by the appended claims, and all inventions utilizing the concept of this invention are protected.

Claims

1. A multi-task motor imagery electroencephalogram decoding method based on federated learning, characterized in that, The method comprises the following steps: Step 1: design a double-server federated learning framework comprising two servers and a plurality of clients, take each subject as a client, preprocess and extract features of local motor imagery electroencephalogram data at each client, and convert electroencephalogram data from different subjects into a unified format; the two servers are server one and server two, server one serves as a feature sharing server, and the clients share their best features; server two is responsible for optimizing a global model, and adopts two strategies of parameter aggregation and knowledge distillation; Step 2: divide the plurality of clients into FL nodes and FD nodes; each client trains and updates a local model based on preprocessed electroencephalogram data, in combination with global model parameters and global soft labels; the global model is a complex model comprising a convolution block, a Transformer multi-head attention module and a classification module, the local model comprises a lightweight model and a complex model, the lightweight model comprises a convolution block and a classification module, and the complex model is the same as the global model; Step 3: each client uploads electroencephalogram features extracted by the local model to server one; the FL nodes upload local model parameters to server two, and the FD nodes upload local soft labels to server two; Step 4: server one selects the best features from the received electroencephalogram features; server two updates the global model by fine-tuning parameters using the received local model parameters and local soft labels; Step 5: server one distributes the selected best features to all clients, server two distributes the updated global model parameters to the FL nodes, and distributes the updated global soft labels to the FD nodes; Step 6: repeat steps 2 to 5 until the global model converges, and obtain the best federated learning model for electroencephalogram decoding.

2. The multi-task motor imagery electroencephalogram decoding method based on federated learning according to claim 1, wherein, The specific operation of data preprocessing in step 1 is as follows: Step 1.1: band-pass filter processing is performed on the motor imagery electroencephalogram signal to obtain motor imagery related electroencephalogram rhythms; Step 1.2: the filtered motor imagery electroencephalogram data is subjected to Euclidean alignment to eliminate edge distribution differences; Step 1.3: the aligned motor imagery electroencephalogram data is expanded by a time window sliding method, with a preset time length as the time window length, sliding at a preset step time interval, the number of time sampling points of each data sample is changed from the original number to a number corresponding to the time window length; Step 1.4: optionally The covariance matrix of the EEG data of the motor imagery task is calculated; then the covariance matrix is subjected to generalized eigenvalue decomposition to obtain a projection matrix of a specific dimension; a certain number of eigenvectors are selected in the projection matrix of each motor imagery task category, and the eigenvectors of each category jointly construct a spatial filter; the spatial filter is used for spatial filtering of the motor imagery EEG to obtain a set of spatially filtered motor imagery EEG data; after common spatial pattern preprocessing, the EEG data from different data sets and different acquisition devices are unified into data matrices of the same size, i.e., EEG common spatial feature dimension x EEG sampling point number.

3. The multi-task motor imagery electroencephalogram decoding method based on federated learning according to claim 1, characterized in that, The specific operation of step 2 is as follows: Step 2.1: Sort the clients according to the classification accuracy of the first round of training, and select the top clients as FL nodes, and the remaining clients as FD nodes; Step 2.2: the FL node client updates and optimizes the local model using local data and the distributed global model parameters; the FD node client updates and optimizes the local model using local data and the distributed global soft labels; the local data is electroencephalogram data, and the features extracted from the local data by the local model are local features; the update optimization is specifically as follows: The best feature selected by the server is introduced to each client , As a global shared resource, and based on minimizing the maximum mean discrepancy, the distribution difference between the two domains is reduced by minimizing the minimum maximum mean discrepancy distance between the best feature and the local feature , The best feature and the local feature of the first client are represented as: , , During local model training, the MMD value of the best feature and the local feature is calculated, and the loss value is represented as: wherein is a function mapping the original data into the reproducing Hilbert space; Objective function for FL node local training is: wherein, is a weighting coefficient to balance the MMD loss; denotes the cross-entropy loss; The FD node adds the distillation loss of the global soft label and the local soft label, and the objective function of the FD node is: : wherein, denotes a dynamic weight coefficient; denotes a local soft label; denotes a global model output soft label; denotes a KL divergence distillation loss of the local soft label and the global model output soft label.

4. The multi-task motor imagery electroencephalogram decoding method based on federated learning according to claim 1, characterized in that, The specific operation of step 4 is as follows: Server one selects the best features from the received electroencephalogram features according to the highest classification accuracy, and the best feature selection formula is as follows: wherein, represents the selected best features of the server; represents the local features uploaded by each client to the server; Server two aggregates the local model parameters uploaded by FL nodes Performs weighted average, which is the parameter of the global model And aggregates the local soft labels uploaded by FD nodes, as follows: , In the formula, Indicates the first Local data sample size for each client; This represents the local data sample size across all clients. Indicates the number of clients; Indicates the first Local model parameters uploaded by each client; This represents the aggregated soft tag; Indicates the first Local soft tags uploaded by each client; The server two fine-tunes the global model parameters by using the aggregated local model parameters and local soft labels and the data shared by each client to form a final global model, and the loss function of the final global model is as follows: As follows: where, is the cross-entropy loss, is the global model output soft label is the KL divergence distillation loss between the aggregated soft label and the global model output soft label, dynamic weight coefficient adopting a linear growth strategy, as follows: , wherein, is the total number of communications, and the amount of knowledge of the soft label dynamically changes with the communication round in the training process; represents the minimum weight coefficient; represents the maximum weight coefficient; represents the current number of communications.

5. The multi-task motor imagery electroencephalogram decoding method based on federated learning according to claim 2, characterized in that, The frequency of the band-pass filter processing in step 1.1 is 8Hz-30Hz, which contains the mu rhythm and beta rhythm of the motor imagery electroencephalogram data.

6. The multi-task motor imagery electroencephalogram decoding method based on federated learning according to claim 1, characterized in that, The convolutional blocks of the global model and the local model in step 2 are composed of two time convolutions and one space convolution, which extract the local space-time features of the electroencephalogram, and are connected with batch normalization and average pooling operations after each convolution; The Transformer multi-head attention module in step 2 is composed of 4 self-attention heads, which process the input sequence in parallel through 4 independent attention heads, each head focusing on different parts of the time sequence to capture the long-distance dependence relationship of the motor imagery electroencephalogram sequence.

7. The multi-task motor imagery electroencephalogram decoding method based on federated learning according to claim 1, characterized in that, The communication overhead calculation method of the method is: The communication overhead is composed of the interaction of the clients with the dual servers; let the total communication times be , the number of clients be , the number of FL node clients be , and the number of FD node clients be ; The communication overhead calculation method is as follows: 1) All client-server interactions contain the transmission of H-dimensional features, which communication overhead is: ; 2) Each FL node interacts with the server to transfer the parameters of the light model of the light model; It Communication overhead is: ; 3) each FD node transmits motion imagination with the server one category soft labels of one sample, achieving knowledge distillation, and the communication overhead is: ; The total communication overhead is and the sum of the three: .

Images