Method and device for training electroencephalogram signal classification model based on federated learning

Abstract

This application is applicable to the field of bioinformatics technology, and provides a method and device for training an electroencephalogram signal classification model based on federated learning. The method includes: sending the electroencephalogram signal classification model on the server side to K client sides; receiving the local model gradients sent by each client side; obtaining the importance evaluation values of the client sides according to the local model gradients of the client sides; determining multiple target client sides from the K client sides according to the importance evaluation values of the K client sides; updating the network parameters of the electroencephalogram signal classification model on the server side according to the local model gradients and importance evaluation values of the target client sides; if the electroencephalogram signal classification model on the server side does not converge, return to the step of sending the electroencephalogram signal classification model on the server side to the K client sides until the electroencephalogram signal classification model on the server side converges. This application can improve the accuracy and convergence speed of the electroencephalogram signal classification model while making full use of the effective information of all users.

Description

Technical Field

[0001] This application belongs to the field of bioinformatics technology, and particularly relates to a method and device for training an electroencephalogram signal classification model based on federated learning. Background Art

[0002] A brain-computer interface (BCI, Brain Computer Interface) based on emotion recognition can collect a user's electroencephalogram signal in an emotion interaction experiment, and perform feature extraction and decoding on the electroencephalogram signal to identify the user's true emotional state and intention, thereby realizing friendly communication and interaction between the user and the device. Emotion analysis based on electroencephalogram signals has a wide range of application scenarios, such as the auxiliary diagnosis of emotional disorder diseases and psychological treatment interventions such as depression.

[0003] Emotion recognition models based on deep learning are often data-driven and require a large amount of training data. However, due to the cumbersome process of collecting electroencephalogram (EEG, Electroencephalographic) signals and the huge individual differences, EEG data often exists in a scattered form in multiple small data sets among various users. In order to build a high-precision emotion recognition model, existing methods are committed to effectively using the useful information of other users and improving the emotion recognition rate of the target user by sharing data between different users and using technologies such as knowledge transfer and domain adaptation. However, in the process of data sharing, if electroencephalogram signals containing private information such as a person's identity characteristics and thoughts and emotions are misused or illegally read and spread, it will cause the leakage of personal privacy.

[0004] Currently, the main EEG signal classification models are: the EEG signal classification model based on EEGNet (EEGNet is a general and compact convolutional neural network designed specifically for general EEG recognition tasks), and the EEG signal classification model based on federated transfer learning (FTL, Federate Transfer Learning). Among them, EEGNet takes the original EEG signal as input and trains an end-to-end competitive emotion recognition network for each user. However, due to the large individual differences in users' EEG signals, directly using the data of all users to train a unified network often results in a low accuracy of the shared model (i.e., the EEG signal classification model). Therefore, the network trained based on EEGNet can only use the local data of each user to train the emotion recognition network separately, ignoring the data of other users and the effective information they can provide, resulting in a problem of data waste. Although the method based on FTL uses federated learning to effectively utilize the data information of other users and also meets the requirement of not sharing local data of users. However, this method takes the spatial covariance matrix of EEG signals as input, losing some effective information of the original EEG signals. In addition, FTL relies on the federated averaging algorithm. In the process of joint training, this algorithm randomly selects the gradients of some local models and updates the gradients of the server by simple averaging without discrimination, ignoring the data quality and importance of different users. This will lead to unstable changes in the gradients of the server model updated each time, which is not conducive to the accuracy of the shared model (i.e., the EEG signal classification model), and often has a slow convergence speed, causing certain difficulties in model training. Summary of the Invention

[0005] The embodiments of the present application provide a method and device for training an EEG signal classification model based on federated learning, which can solve the problems of low accuracy and slow convergence speed of the EEG signal classification model.

[0006] In a first aspect, the embodiments of the present application provide a method for training an EEG signal classification model based on federated learning, which is applied to the server side. The method includes:

[0007] Sending the EEG signal classification model of the server side to K user terminals;

[0008] Receiving the local model gradients sent by each user terminal; the local model gradients are obtained by the user terminal training the EEG signal classification model using the local training set;

[0009] According to the local model gradients of each user terminal, obtaining the importance evaluation value of each user terminal;

[0010] Determining multiple target user terminals from the K user terminals according to the importance evaluation values of the K user terminals;

[0011] Update the network parameters of the EEG signal classification model on the server side according to the local model gradients and importance evaluation values of the multiple target client devices;

[0012] If the EEG signal classification model on the server side has not converged, return to execute the step of sending the EEG signal classification model on the server side to K client devices until the EEG signal classification model on the server side converges.

[0013] Among them, the step of determining multiple target client devices from K client devices according to the importance evaluation values of the K client devices includes:

[0014] Select a preset proportion of client devices from the K client devices as target client devices in the order of importance evaluation values from large to small.

[0015] Among them, the step of updating the network parameters of the EEG signal classification model on the server side according to the local model gradients and importance evaluation values of the multiple target client devices includes:

[0016] Perform normalization processing on the importance evaluation values of each of the target client devices;

[0017] Update the global gradient on the server side according to the normalized importance evaluation values and the local model gradients of all target client devices;

[0018] Update the network parameters of the EEG signal classification model on the server side according to the updated global gradient.

[0019] Among them, the step of updating the global gradient on the server side according to the normalized importance evaluation values and the local model gradients of all target client devices includes:

[0020] Through the formula Update the global gradient on the server side;

[0021] Among them, represents the global gradient obtained in the t-th round of update, C represents a preset proportion, K represents the number of client devices, represents the normalized importance evaluation value of the k-th client device, represents the local model gradient of the k-th client device in the t-th round of update, and t is an integer greater than 0.

[0022] Among them, the step of performing normalization processing on the importance evaluation values of each of the target client devices includes:

[0023] Through the formula Perform normalization processing on the importance evaluation values of each selected client device;

[0024] Among them, represents the importance evaluation value after normalization processing of the k-th client, μ k represents the importance evaluation value of the k-th client, C represents a preset ratio, and K represents the number of clients.

[0025] Among them, the step of obtaining the importance evaluation value of each client according to the local model gradient of each client includes:

[0026] By the formula μ k =α k ×β k , calculate the importance evaluation value of the k-th client;

[0027] Among them, μ k represents the importance evaluation value of the k-th client, α k =n k / n, n k represents the number of local samples included in the local training set of the k-th client, n represents the total number of local samples included in the local training sets of K clients, and K represents the number of clients, represents the global gradient of the server at the (t - 1)-th round of update, represents the local model gradient of the k-th client at the t-th round of update, and t is an integer greater than 0.

[0028] Among them, the method further includes:

[0029] When the electroencephalogram signal classification model of the server converges, send the electroencephalogram signal classification model of the server to the K clients.

[0030] In a second aspect, an embodiment of the present application provides an electroencephalogram signal classification model training device based on federated learning, which is applied to the server side. The device includes:

[0031] A sending module, configured to send the electroencephalogram signal classification model of the server to K clients;

[0032] A receiving module, configured to receive the local model gradients sent by each client; the local model gradients are obtained by the client training the electroencephalogram signal classification model using the local training set;

[0033] An obtaining module, configured to obtain the importance evaluation value of each client according to the local model gradient of each client;

[0034] The first determination module is configured to determine a plurality of target client terminals from the K client terminals according to the importance evaluation values of the K client terminals;

[0035] The update module is configured to update the network parameters of the electroencephalogram signal classification model on the server side according to the local model gradients and importance evaluation values of the plurality of target client terminals;

[0036] The second determination module is configured to, if the electroencephalogram signal classification model on the server side has not converged, return to execute the step of sending the electroencephalogram signal classification model on the server side to the K client terminals until the electroencephalogram signal classification model on the server side converges.

[0037] Wherein, the above-mentioned first determination module 304 is specifically configured to select client terminals with a preset ratio from the K client terminals as target client terminals in the order of decreasing importance evaluation values.

[0038] Wherein, the above-mentioned update module 305 includes:

[0039] The processing unit is configured to perform normalization processing on the importance evaluation values of each of the target client terminals;

[0040] The first update unit is configured to update the global gradient on the server side according to the normalized importance evaluation values and the local model gradients of all target client terminals;

[0041] The second update unit is configured to update the network parameters of the electroencephalogram signal classification model on the server side according to the updated global gradient.

[0042] Wherein, the above-mentioned first update unit is specifically configured to update the global gradient on the server side through the formula ;

[0043] Wherein, represents the global gradient obtained in the t-th round of update, C represents a preset ratio, K represents the number of client terminals, represents the normalized importance evaluation value of the k-th client terminal, represents the local model gradient of the k-th client terminal in the t-th round of update, and t is an integer greater than 0.

[0044] [[ID=·38]]Wherein, the above-mentioned processing unit is specifically configured to perform normalization processing on the importance evaluation values of each selected client terminal through the formula ;

[0045] Wherein, represents the normalized importance evaluation value of the k-th client terminal, μ k represents the importance evaluation value of the k-th client terminal, C represents a preset ratio, and K represents the number of client terminals.

[0046] Among them, the above-mentioned acquisition module 303 is specifically used to calculate the importance evaluation value of the k-th client through the formula μ k = α k × β k , and calculate the importance evaluation value of the k-th client;

[0047] Among them, μ k represents the importance evaluation value of the k-th client, α k = n k / n, n k represents the number of local samples included in the local training set of the k-th client, n represents the total number of local samples included in the local training sets of K clients, and K represents the number of clients, represents the global gradient of the server at the (t - 1)-th update, represents the local model gradient of the k-th client at the t-th update, and t is an integer greater than 0.

[0048] Among them, the above-mentioned electroencephalogram signal classification model training device further includes:

[0049] A distribution module, configured to distribute the electroencephalogram signal classification model of the server to the K clients when the electroencephalogram signal classification model of the server converges.

[0050] In a third aspect, an embodiment of the present application provides a server, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned method is implemented.

[0051] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned method is implemented.

[0052] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when running on a terminal device, enables the terminal device to execute the method described in any one of the above first aspects.

[0053] The beneficial effects of the embodiments of the present application compared with the prior art are:

[0054] In the embodiments of the present application, based on the federated learning framework, joint training and distributed training can be achieved on the premise of meeting data security requirements without sharing or exchanging the local data of each client, so as to fully utilize the effective information of all users to improve the accuracy of the electroencephalogram (EEG) signal classification model. At the same time, in the joint training, instead of randomly selecting target clients, the target clients that contribute greatly to the shared model are selected from all clients through the importance evaluation values of each client, and based on the local model gradients and importance evaluation values of the target clients, the network parameters of the EEG signal classification model on the server side are updated, thereby improving the accuracy and convergence speed of the EEG signal classification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0056] Figure 1 is a flowchart of a method for training an EEG signal classification model based on federated learning provided by an embodiment of the present application;

[0057] Figure 2 is a flowchart of step 15 provided by an embodiment of the present application;

[0058] Figure 3 is a schematic structural diagram of a device for training an EEG signal classification model based on federated learning provided by an embodiment of the present application;

[0059] Figure 4 is a schematic structural diagram of a server provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0060] In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0061] It should be understood that when used in the specification of the present application and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0062] It should also be understood that the term "and / or" used in the specification and appended claims of the present application refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0063] As used in the specification and appended claims of the present application, the term "if" can be interpreted as "when", "once", "in response to determining", or "in response to detecting" depending on the context. Similarly, the phrase "if determined" or "if [the described condition or event] is detected" can be interpreted as meaning "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]" depending on the context.

[0064] In addition, in the description of the specification and appended claims of the present application, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0065] Reference to "one embodiment" or "some embodiments" etc. described in the specification of the present application means that a specific feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of the present application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0066] Currently, the main EEG signal classification models are the EEG signal classification model based on EEGNet and the EEG signal classification model based on FTL. However, the accuracy of the EEG signal classification model based on EEGNet is low, and the convergence speed of the EEG signal classification model based on FTL is slow and the accuracy is not ideal.

[0067] In view of the above problems, embodiments of the present application are based on the federated learning framework. In distributed training, the EEG signal classification model on the server side is sent to K client sides, so that each client side uses the local training set to train the received EEG signal classification model, and sends the gradient of the locally trained model to the server side for joint training, thereby realizing joint training and its distributed training on the premise of meeting data security and without sharing or exchanging the local data of each client side, and achieving the effect of improving the accuracy of the EEG signal classification model while making full use of the effective information of all users.

[0068] Meanwhile, in the collaborative training, instead of randomly selecting target client devices, the target client devices that contribute significantly to the shared model are selected from all client devices based on the importance evaluation values of each client device. Then, based on the local model gradients and importance evaluation values of the target client devices, the network parameters of the EEG classification model on the server side are updated, thereby improving the accuracy and convergence speed of the EEG classification model.

[0069] The following provides an exemplary description of the EEG classification model training method based on federated learning provided by this application in combination with specific embodiments.

[0070] As Figure 1 shown, an embodiment of this application provides an EEG classification model training method based on federated learning, which is applied to the server side. The method includes the following steps:

[0071] Step 11: Send the EEG classification model of the server side to K client devices.

[0072] In some embodiments of this application, the above-mentioned K client devices are the client devices participating in federated learning with the above-mentioned server side. It should be noted that to ensure that the finally obtained EEG classification model is based on the valid information of users on the client side, before executing the steps of the above training method, the server side can initialize an EEG classification model (i.e., the EEG classification model in the above step 11). Specifically, the model weights can be initialized to 0, or other common initialization schemes can be adopted, such as Gaussian, Xavier initialization (Xavier initialization is a neural network initialization method).

[0073] Among them, the above-mentioned EEG classification model can be an EEGNet model, and of course, it can also be other deep learning networks, such as EEG classification neural networks such as convolutional neural networks (ConvNet).

[0074] Step 12: Receive the local model gradients sent by each of the client devices.

[0075] In some embodiments of this application, the above-mentioned local model gradients are obtained by the client devices training the EEG classification model using the local training sets.

[0076] That is, in some embodiments of this application, for each client device participating in federated learning, after receiving the EEG classification model sent by the server side, the received EEG classification model will be trained using the local training set of the client device, and the local model gradient will be obtained when the EEG classification model converges.

[0077] Step 13: Obtain the importance evaluation value of each client device according to the local model gradient of each client device.

[0078] In some embodiments of the present application, the above importance evaluation value is mainly used to characterize the importance degree of the user terminal, so as to subsequently select target user terminals that contribute greatly to the shared model (i.e., the electroencephalogram signal classification model on the server side) from the K user terminals in descending order of importance for joint training, thereby improving the accuracy and convergence speed of the electroencephalogram signal classification model.

[0079] Step 14: Determine multiple target user terminals from the K user terminals according to the importance evaluation values of the K user terminals.

[0080] In some embodiments of the present application, user terminals with a preset proportion can be selected from the K user terminals as target user terminals in descending order of the importance evaluation value, so as to screen out target user terminals with a high degree of importance from the K user terminals. Among them, the specific value of the above preset proportion can be set according to the actual situation.

[0081] It can be seen that in some embodiments of the present application, the importance degree of the above target user terminals is higher than that of other user terminals among the K user terminals, that is, the contribution of the target user terminals to the shared model (i.e., the electroencephalogram signal classification model on the server side) is greater than the contribution of other user terminals to the shared model. Subsequently, using these target user terminals for joint training can improve the accuracy and convergence speed of the electroencephalogram signal classification model.

[0082] Step 15: Update the network parameters of the electroencephalogram signal classification model on the server side according to the local model gradients and importance evaluation values of the multiple target user terminals.

[0083] In some embodiments of the present application, in joint training, by updating the network parameters of the electroencephalogram signal classification model on the server side according to the local model gradients and importance evaluation values of the target user terminals, the accuracy and convergence speed of the electroencephalogram signal classification model on the server side can be improved.

[0084] Step 16: If the electroencephalogram signal classification model on the server side has not converged, return to execute the step of sending the electroencephalogram signal classification model on the server side to the K user terminals until the electroencephalogram signal classification model on the server side converges.

[0085] In some embodiments of the present application, the converged electroencephalogram signal classification model in the above step 16 is a shared model, which can be used to classify the electroencephalogram signals of any user.

[0086] In some embodiments of the present application, after executing step 15, if the electroencephalogram signal classification model on the server side has not converged, return to step 11 to update the network parameters of the electroencephalogram signal classification model on the server side again until the electroencephalogram signal classification model on the server side converges.

[0087] It should be noted that after each update of the network parameters of the electroencephalogram (EEG) signal classification model on the server side, it is necessary to determine whether the updated EEG signal classification model converges. If it converges, the updated EEG signal classification model is the shared model. Otherwise, the EEG signal classification model with updated network parameters is sent to K client sides, so that the K client sides respectively use their own local training sets to train the received EEG signal classification model to obtain local model gradients, in order to update the network parameters of the EEG signal classification model on the server side again.

[0088] It is worth mentioning that in some embodiments of the present application, in the joint training, instead of directly using the local training set data of the client side, the local model gradients of the client side are used to jointly train the EEG signal classification model on the server side, thereby ensuring the privacy and security of the local data of the client side. Under the premise of meeting data security and without sharing or exchanging the local data of each client side, joint training and its distributed training can be realized, achieving the effect of fully utilizing the effective information of all users to improve the accuracy of the EEG signal classification model.

[0089] At the same time, in the joint training, instead of randomly selecting client sides, target client sides that contribute greatly to the shared model are selected from all client sides through the importance evaluation values of each client side, and based on the local model gradients and importance evaluation values of the target client sides, the network parameters of the EEG signal classification model on the server side are updated, thereby improving the accuracy and convergence speed of the EEG signal classification model.

[0090] In the embodiments of the present application, after performing the above step 16, the method further includes the following steps: when the EEG signal classification model on the server side converges, the EEG signal classification model on the server side is sent to the K client sides.

[0091] It should be noted that after receiving the EEG signal classification model, the client side can use its own local training set to train the EEG signal classification model to fine-tune the model parameters of the EEG signal classification model, so as to obtain an EEG signal classification model more suitable for the client side. Subsequently, the client side can use the fine-tuned EEG signal classification model to classify the user data of the client side to improve the classification accuracy.

[0092] Next, an exemplary description will be given of the process of the client side using the local training set to train the EEG signal classification model in combination with specific embodiments.

[0093] In some embodiments of the present application, the local training set of the client can be sourced from the Shanghai Jiao Tong University Emotion EEG Dataset (SEED). In the experiments of this dataset, 15 selected Chinese movie clips were chosen as the emotional stimulus sources in the experiments, and the labels included positive, neutral, and negative emotions. A total of 15 Chinese subjects (including 7 males and 8 females) were recruited for this dataset, and each subject conducted 3 experiments respectively. Each sample in this dataset contains 62 electrode channels, which were downsampled to 200 Hz, and a band-pass frequency filter of 0 - 75 Hz was applied. To expand the data volume, we cut each data into non-overlapping segments with a 1s data window, and finally obtained 3394 samples. Among the 62 channels collected, the embodiments of the present application selected 32 channels related to emotions, corresponding to Fp1, AF3, F3, F7, FC5, FC1, C3, T7, CP5, CP1, P3, P7, PO3, O1, Oz, Pz, Fp2, AF4, Fz, F4, F8, FC6, FC2, Cz, C4, T8, CP6, CP2, P4, P8, PO4, O2 respectively. For this reason, the size of each sample is 32×200. It should be noted that, in some embodiments of the present application, the data of 32 channels of any one of the 15 subjects can be used as the local training set of a client. To improve the accuracy of the EEG signal classification model, the client can use all the data in the local training set to train the EEG signal classification model each time. It should be further noted that the local training set corresponding to each client is different.

[0094] As a preferred example, according to the spatio-temporal attributes of the input original EEG signal, the above EEG signal classification model adopts the EEGNet model for extracting the feature representation and classification of EEG signals. The model parameters of the feature extractor and classifier in the present application are shown in Table 1. Of course, it can be understood that the number of convolutional layers, the size of the convolutional kernel, the pooling method, and the activation function can all be set according to the actual situation.

[0095]

[0096] Table 1

[0097] Among them, when the client uses the local training set to train the EEG signal classification model, the cross-entropy loss function can be used to evaluate the training results. The training loss function of the k-th client is as follows: Among them, n k represents the number of local samples contained in the local training set of the k-th client, and y i is the true label of the training sample (i.e., the local sample in the local training set), Is the prediction label. It should be noted that the above training loss function is a commonly used loss function, so the principle of this training loss function will not be elaborated here too much.

[0098] Next, an exemplary description of the process of obtaining the importance evaluation value and updating the network parameters will be given in combination with specific embodiments.

[0099] In some embodiments of the present application, the specific implementation of step 13 of obtaining the importance evaluation value for each client according to the local model gradient of each client may be: through the formula μ k =α k ×β k , calculate the importance evaluation value of the k-th client.

[0100] Among them, μ k represents the importance evaluation value of the k-th client, α k =n k / n, n k represents the number of local samples included in the local training set of the k-th client, n represents the total number of local samples included in the local training sets of K clients, and K represents the number of clients, represents the global gradient of the server side at the (t - 1)-th update, represents the local model gradient of the k-th client at the t-th update, and t is an integer greater than 0.

[0101] It should be noted that when t takes the value of 1, is the model gradient of the EEG signal classification model initialized by the server side, and the update in the formula refers to the update of the network parameters of the EEG signal classification model on the server side.

[0102] In some embodiments of the present application, in addition to calculating the importance evaluation value of the client through the above formula, other similarity metric learning methods or attention mechanism algorithms can also be used to measure the importance of the client.

[0103] In some embodiments of the present application, as Figure 2 shown, the specific implementation of step 15 of updating the network parameters of the EEG signal classification model on the server side according to the local model gradients and importance evaluation values of the multiple target clients includes the following steps:

[0104] Step 21, perform normalization processing on the importance evaluation value of each target client.

[0105] In some embodiments of the present application, the formula Normalize the importance evaluation values of each selected client.

[0106] Among them, represents the importance evaluation value of the k-th client after normalization, μ k represents the importance evaluation value of the k-th client, C represents a preset ratio, and K represents the number of clients.

[0107] Step 22: Update the global gradient of the server according to the importance evaluation value after normalization and the local model gradients of all target clients.

[0108] In some embodiments of the present application, the global gradient of the server can be updated through the formula Update the global gradient of the server.

[0109] Among them, represents the global gradient obtained in the t-th round of update, C represents a preset ratio, K represents the number of clients, represents the importance evaluation value of the k-th client after normalization, represents the local model gradient of the k-th client in the t-th round of update, and t is an integer greater than 0.

[0110] Step 23: Update the network parameters of the electroencephalogram signal classification model of the server according to the updated global gradient.

[0111] In some embodiments of the present application, the stochastic gradient descent method based on Stochastic Gradient Descent (SGD) can be used to solve the network parameters. It should be noted that when the server initializes the electroencephalogram signal classification model, the global gradient of the server will also be initialized to 0.

[0112] In summary, the electroencephalogram signal classification model training method based on federated learning provided by the embodiments of the present application has the following effects:

[0113] First, the electroencephalogram signal classification model uses the EEGNet model and applies it to the classification task of emotional electroencephalogram signals. Without manually extracting signal features, it can perform feature extraction and classification of emotional electroencephalogram signals end-to-end;

[0114] Second, applying the EEGNet model to the emotional electroencephalogram recognition network, using deep learning to automatically extract discriminative features of emotional electroencephalogram signals, and improving the accuracy of the electroencephalogram signal classification model of a single client;

[0115] Third, without performing complicated preprocessing on electroencephalogram signals, directly using electroencephalogram signals to train the electroencephalogram signal classification model can effectively perform feature extraction and classification of electroencephalogram signals;

[0116] Fourth, it can achieve joint training and distributed training under the premise of ensuring data security and without sharing or exchanging local data of each user end, so as to fully utilize the effective information of all users to improve the accuracy of the EEG signal classification model;

[0117] 5. Based on the importance of each user terminal, target user terminals that contribute most to the shared model are selected for joint training, thereby improving the accuracy and convergence speed of the EEG signal classification model.

[0118] The following is an illustrative description of the EEG signal classification model training device based on federated learning provided in this application in conjunction with specific embodiments.

[0119] like Figure 3 As shown, the embodiment of the present application provides an EEG signal classification model training device based on federated learning, which is applied to the server side. The EEG signal classification model training device 300 includes:

[0120] A sending module 301 is used to send the EEG signal classification model of the server to K user terminals;

[0121] The receiving module 302 is configured to receive a local model gradient sent by each user terminal; the local model gradient is obtained by the user terminal training the EEG signal classification model using a local training set;

[0122] An acquisition module 303 is configured to acquire an importance evaluation value of each user terminal according to a local model gradient of each user terminal;

[0123] A first determining module 304 is configured to determine a plurality of target user terminals from the K user terminals according to the importance evaluation values of the K user terminals;

[0124] An updating module 305 is configured to update the network parameters of the EEG signal classification model on the server side according to the local model gradients and importance evaluation values of the multiple target user sides;

[0125] The second determining module 306 is configured to return to the step of sending the server-side EEG signal classification model to K user terminals if the server-side EEG signal classification model has not converged, until the server-side EEG signal classification model converges.

[0126] The first determining module 304 is specifically configured to select a preset proportion of user terminals as target user terminals from the K user terminals in descending order of importance evaluation values.

[0127] The update module 305 includes:

[0128] A processing unit for normalizing the importance evaluation value of each of the target client terminals;

[0129] A first update unit for updating the global gradient of the server side according to the normalized importance evaluation value and the local model gradients of all target client terminals;

[0130] A second update unit for updating the network parameters of the electroencephalogram signal classification model of the server side according to the updated global gradient.

[0131] Among them, the above-mentioned first update unit is specifically used to update the global gradient of the server side through the formula ;

[0132] Among them, represents the global gradient obtained in the t-th round of update, C represents a preset ratio, K represents the number of client terminals, represents the importance evaluation value of the k-th client terminal after normalization, represents the local model gradient of the k-th client terminal at the t-th round of update, and t is an integer greater than 0.

[0133] Among them, the above-mentioned processing unit is specifically used to normalize the importance evaluation value of each selected client terminal through the formula ;

[0134] Among them, represents the importance evaluation value of the k-th client terminal after normalization, μ k represents the importance evaluation value of the k-th client terminal, C represents a preset ratio, and K represents the number of client terminals.

[0135] Among them, the above-mentioned acquisition module 303 is specifically used to calculate the importance evaluation value of the k-th client terminal through the formula μ k =α k ×β k ; [[ID=(42)]]

[0136] Among them, μ k represents the importance evaluation value of the k-th client terminal, α k =n k / n, n k represents the number of local samples included in the local training set of the k-th client terminal, n represents the total number of local samples included in the local training sets of K client terminals, K represents the number of client terminals, represents the global gradient of the server side at the (t - 1)-th round of update, represents the local model gradient of the k-th client terminal at the t-th round of update, and t is an integer greater than 0.

[0137] Among them, the above-mentioned electroencephalogram signal classification model training device further includes:

[0138] A distribution module, configured to, when the electroencephalogram signal classification model at the server side converges, distribute the electroencephalogram signal classification model at the server side to the K client sides.

[0139] It should be noted that, for the information interaction, execution process, etc. among the above-mentioned device / units, since they are based on the same concept as the method embodiment of the present application, for their specific functions and the technical effects brought, reference can be specifically made to the method embodiment part, and details will not be repeated here.

[0140] Those skilled in the art can clearly understand that, for the convenience and conciseness of description, only the above-mentioned division of each functional unit and module is used as an example for illustration. In practical applications, the above-mentioned functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of the present application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the foregoing method embodiment, and details will not be repeated here.

[0141] As Figure 4 shown, an embodiment of the present application provides a server. As Figure 4 shown, the server D10 in this embodiment includes: at least one processor D100 ( Figure 4 only one processor is shown in

[0142] The so-called processor D100 may be a central processing unit (CPU), and this processor D100 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or this processor may also be any conventional processor, etc.

[0143] In some embodiments, the memory D101 may be an internal storage unit of the server D10, such as the hard disk or memory of the server D10. In other embodiments, the memory D101 may also be an external storage device of the server D10, such as a plug-in hard disk equipped on the server D10, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory D101 may also include both the internal storage unit of the server D10 and the external storage device. The memory D101 is used to store an operating system, application programs, a boot loader, data, and other programs, such as the program code of the computer program, etc. The memory D101 may also be used to temporarily store data that has been output or will be output.

[0144] It should be noted that, for the content such as information interaction and execution process between the above-mentioned devices / units, since it is based on the same concept as the method embodiment of this application, for its specific functions and the technical effects brought, please refer to the method embodiment part specifically, and details will not be elaborated here.

[0145] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working processes of the units and modules in the above system can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated here.

[0146] An embodiment of this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the foregoing method embodiments can be implemented.

[0147] An embodiment of this application provides a computer program product. When the computer program product runs on a terminal device, the terminal device can implement the steps in the foregoing method embodiments when executed.

[0148] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above method embodiments of this application, a computer program can be used to instruct relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps in the foregoing method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can at least include: any entity or device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium that can carry the computer program code to the electroencephalogram signal classification model training device / terminal device. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0149] In the above embodiments, the descriptions of the various embodiments have their own emphases. For parts not described or recorded in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0150] Those of ordinary skill in the art will realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application.

[0151] In the embodiments provided in this application, it should be understood that the disclosed device / network device and method can be implemented in other ways. For example, the device / network device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling, direct coupling, or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be electrical, mechanical, or other forms.

[0152] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0153] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included in the protection scope of this application.

Claims

1. A training method for an electroencephalogram signal classification model based on federated learning, characterized in that Applied to the server side, the method includes: Sending the electroencephalogram signal classification model of the server side to K client sides; Receiving the local model gradients sent by each of the client sides; the local model gradients are obtained by the client sides training the electroencephalogram signal classification model using local training sets; Obtaining an importance evaluation value for each of the client sides according to the local model gradients of each of the client sides; Determining a plurality of target client sides from the K client sides according to the importance evaluation values of the K client sides; Updating the network parameters of the electroencephalogram signal classification model of the server side according to the local model gradients and importance evaluation values of the plurality of target client sides; If the electroencephalogram signal classification model of the server side does not converge, return to execute the step of sending the electroencephalogram signal classification model of the server side to K client sides until the electroencephalogram signal classification model of the server side converges; Wherein, the step of obtaining an importance evaluation value for each of the client sides according to the local model gradients of each of the client sides includes: Through the formula μ k = α k × β k , calculate the importance evaluation value of the k-th client; Among them, μ k represents the importance evaluation value of the k-th client, and α k = n k n, n k represents the number of local samples included in the local training set of the k-th client, n represents the total number of local samples included in the local training sets of K clients, and K represents the number of clients, represents the global gradient of the server side at the (t - 1)-th update, represents the local model gradient of the k-th client at the t-th update, where t is an integer greater than 0.

2. The method according to claim 1, wherein The step of determining a plurality of target client sides from the K client sides according to the importance evaluation values of the K client sides includes: Selecting a preset proportion of the client sides from the K client sides as target client sides in the order of the importance evaluation values from large to small.

3. The method according to claim 2, characterized in that, The step of updating the network parameters of the electroencephalogram signal classification model of the server side according to the local model gradients and importance evaluation values of the plurality of target client sides includes: Performing normalization processing on the importance evaluation value of each of the target client sides; Updating the global gradient of the server side according to the normalized importance evaluation values and the local model gradients of all the target client sides; Updating the network parameters of the electroencephalogram signal classification model of the server side according to the updated global gradient.

4. The method according to claim 3, wherein The step of updating the global gradient of the server side according to the normalized importance evaluation values and the local model gradients of all the target client sides includes: Update the global gradient of the server side through the formula Update the global gradient of the server side; Among them, represents the global gradient obtained in the t-th round of update, C represents a preset ratio, and K represents the number of clients, represents the importance evaluation value of the k-th client after normalization, represents the local model gradient of the k-th client in the t-th round of update, where t is an integer greater than 0.

5. The method according to claim 3, wherein The step of performing normalization processing on the importance evaluation value of each of the target client sides includes: Through the formula Normalize the importance evaluation value of each selected client; Among them, represents the importance evaluation value after normalization of the k-th client, μ k represents the importance evaluation value of the k-th client, C represents a preset ratio, and K represents the number of clients.

6. The method according to claim 1, wherein The method further includes: When the electroencephalogram signal classification model of the server side converges, sending the electroencephalogram signal classification model of the server side to the K client sides.

7. A training device for an electroencephalogram signal classification model based on federated learning, characterized in that, Applied to the server side, the device includes: A sending module, configured to send the electroencephalogram signal classification model of the server side to K client sides; A receiving module, configured to receive the local model gradients sent by each of the client sides; the local model gradients are obtained by the client sides training the electroencephalogram signal classification model using local training sets; An obtaining module, configured to obtain an importance evaluation value for each of the client sides according to the local model gradients of each of the client sides; A first determining module, configured to determine a plurality of target client sides from the K client sides according to the importance evaluation values of the K client sides; An updating module, configured to update the network parameters of the electroencephalogram signal classification model of the server side according to the local model gradients and importance evaluation values of the plurality of target client sides; A second determination module, configured to, if the EEG signal classification model at the server end has not converged, return to execute the step of sending the EEG signal classification model at the server end to K client ends until the EEG signal classification model at the server end converges; Among them, the obtaining module is specifically configured to calculate the importance evaluation value of the k-th client through the formula μ k = α k × β k , where α, β are coefficients, and k is a positive integer representing the number of the client. Among them, μ k represents the importance evaluation value of the k-th client, and α k = n k n, n k represents the number of local samples included in the local training set of the k-th client, n represents the total number of local samples included in the local training sets of K clients, and K represents the number of clients, represents the global gradient of the server side at the (t - 1)-th round of update, represents the local model gradient of the k-th client at the t-th round of update, where t is an integer greater than 0.

8. A server, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the method according to any one of claims 1 to 6 is implemented.

Images