A personalized federated learning implementation method with high generalization
By employing a dual-branch structure and batch normalization features in federated learning, a personalized federated learning method is achieved that improves model generalization on unknown data distributions while maintaining individual performance, thus solving the problem of reduced model generalization.
Patent Information
- Application Number
- CN202211206093.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-09-30
- Publication Date
- 2025-11-25
- Estimated Expiration
- 2042-09-30
AI Technical Summary
While existing personalized federated learning methods can improve performance on known data, they are prone to overfitting, resulting in reduced generalization and an inability to perform well on unknown data distributions.
A global model with a dual-branch structure is adopted, which combines task-independent personalized batch normalization and global batch normalization features. By learning simultaneously through global task sub-models and personalized task sub-models, the generalization and personalization capabilities of the model are improved.
Without increasing the number of additional federated communication and local training rounds, the generalization performance of the client-side local model on unknown data is improved, while maintaining personalized performance, thus solving the problem of poor model generalization.
Smart Images
Figure CN115511109B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of federated learning technology, and in particular to a highly generalizable personalized federated learning implementation method. Background Technology
[0002] Federated learning refers to the task of training a model on multiple isolated datasets. With increasingly stringent privacy policies, the traditional centralized approach of aggregating data from multiple data silos for data mining is becoming impractical. Furthermore, the limited effective data from a single data silo restricts data-driven modeling and data mining. This is where federated learning comes in. General federated learning involves all clients collaboratively training a consensus model without sharing their own data, aiming to learn as much as possible from data from multiple clients. The main steps of general federated learning include: client selection, model distribution, model training, and model aggregation, iterating until convergence to obtain an aggregated consensus model.
[0003] Due to the inherent data isolation of federated learning, the data distribution of clients is unknown, leading to strong heterogeneity in the learning of different client models. For example, clients from different geographical environments may solve different tasks, but the aggregated consensus model may be biased towards certain clients, resulting in poor overall performance. To address this heterogeneity among clients, personalized federated learning allows each client to retain and optimize its own independent, personalized model, rather than using a global consensus model. The aim is for clients to benefit from federated learning while achieving better performance on locally available data; that is, personalized models outperform models trained independently by isolated clients, and also outperform the federated consensus model.
[0004] While personalized federated learning offers a solution to the heterogeneity dilemma of federated clients, mainstream implementations of personalized federated learning focus on performance improvements on visible data. Due to further optimization of visible data, most mainstream methods generate personalized models prone to overfitting, ultimately leading to strong model bias and reduced generalization. However, model generalization is a crucial issue in real-world scenarios. For example, when hospital clients receive data from referred patients from unknown hospitals, they should not only focus on the federated model's performance on locally visible data but also on its performance on undistributed data.
[0005] Therefore, there is an urgent need for a personalized federated learning implementation method that can focus on the generalization of the model, so as to improve the generalization of the model while ensuring the effectiveness of personalized federated learning. Summary of the Invention
[0006] To address the problems in the background technology, this invention provides a highly generalizable personalized federated learning implementation method. It utilizes task-independent personalized batch normalization and global batch normalization features, and learns the model's personalization and generalization capabilities simultaneously through a dual-branch structure. This not only effectively improves the generalization ability of the client's local model in the face of unknown data, but also ensures the personalization capability of the client's local model under the client's local data distribution.
[0007] Firstly, the present invention provides a highly generalizable personalized federated learning implementation method, including,
[0008] Step 1: The server randomly initializes a global model with a dual-branch structure and sends the obtained initial model parameters to multiple selected clients; the global model includes global task sub-model branches and personalized task sub-model branches.
[0009] Step 2: Each client initializes its local model with a dual-branch structure using the initialization model parameters sent by the server, and performs the first round of local iterative training using local data to obtain the updated local model; the statistical parameters of the updated local model and the model parameters of the global task sub-model are then uploaded to the server.
[0010] Step 3: The server calculates the weighted average of the model parameters of the global task sub-models of all clients to obtain the model parameters of the new global task sub-model after aggregation, and sends the updated model parameters to multiple selected clients.
[0011] Step 4: The client uses the model parameters of the global task sub-model sent by the server to update the model parameters of the global task sub-model in the client's local model. Combined with the personalized task sub-model in the client's local model obtained from the iterative training in this round of federated training, the updated client local model is obtained, and one round of federated training is completed.
[0012] Step 5: The client uses the client-local model updated in Step 4 to perform another round of iterative training based on local data, updates the client-local model parameters, and uploads the updated client-local model statistical parameters and global task sub-model model parameters to the server. Then, it returns to Step 3 and repeats the process of updating the client-local model until it meets the preset standard.
[0013] Furthermore, the global model used on the server side and the local client model used on the client side have the same structure, that is, a batch normalization layer is added after the feature extraction layer of the model; among them, the feature extraction layer is a task-shared layer, and the batch normalization layer is a task-specific layer; the task-specific layer includes a global batch normalization layer and a personalized batch normalization layer.
[0014] Furthermore, the global task sub-model consists of a task sharing layer and a global batch normalization layer; the personalized task sub-model consists of a task sharing layer and a personalized batch normalization layer.
[0015] Furthermore, the statistical parameters include the amount of data the client participated in training.
[0016] Preferably, the process of obtaining the updated client-side local model through local iterative training in step 2 is as follows:
[0017] After inputting local data x into the client's local model, two branches are executed simultaneously to obtain the outputs of two tasks, namely the global task output y. g and personalized task output y l The global task loss is obtained by calculating the cross-entropy loss. g and personalized task loss l ;
[0018] The expression for cross-entropy loss is as follows:
[0019]
[0020] Where a takes the form g or l; y j To predict the target, This is the actual prediction result; m represents the number of clients participating in the training.
[0021] Using global task loss g and personalized task loss l Obtain the total loss overall The expression is:
[0022] loss overall =αloss g +(1-α)loss l
[0023] Where α is the loss ratio coefficient;
[0024] Combining the overall loss and the preset learning rate η, the client obtains the updated gradient of the overall computational model through stochastic gradient descent and backpropagation, thus obtaining the updated model parameters of the client's local model. The expression for updating the model parameters of the client's local model is as follows:
[0025]
[0026] Among them, g l g represents the overall gradient obtained from one iteration of the personalized task sub-model optimization; g These represent the overall gradient obtained from one iteration of the global task sub-model optimization; wg This represents the model parameters of the global task sub-model; w l t represents the model parameters of the personalized task sub-model; t represents the current round of federated training; i represents the i-th client.
[0027] Preferably, the model parameters of the new global task sub-model obtained by weighted averaging in step 3 are as follows:
[0028] Calculate the proportion of training data collected by the client to the total training data collected by all clients;
[0029] Model parameters w of the global task sub-model g The update formula is as follows:
[0030]
[0031] Where K represents the total number of clients participating in training; k represents the k-th client; n represents the total amount of data collected by all clients during training; n k This represents the amount of data trained by the k-th client; w represents the model parameters of the global task sub-model for the k-th client in the t-th round of federated training; g,t+1 This represents the model parameters of the global task sub-model for the k-th client in the (t+1)-th round of federated training.
[0032] Preferably, the preset standard in step 5 is as follows:
[0033] Judgment based on data and client distribution using the loss curve:
[0034] If the data and client distribution are stable and convergent, the client's local model will use the model parameters from the last round of federated training as the training result after a preset number of rounds.
[0035] If the data and client distribution cannot converge stably, the model parameters of the client-side local model that performs best in the validation set within the preset federated training rounds are selected as the training result by adding the validation set from the dataset to the federated training.
[0036] Furthermore, during the training process:
[0037] Model parameters can be updated by calculating the multi-task loss of the training set and backpropagation. If it is necessary to perform prediction and inference steps on the model, such as calculating the accuracy of the training set, validation set and test set, or inferring the category of the input data, the ensemble inference method can be used in the inference stage to obtain the output results of the client's local model on the input data.
[0038] The ensemble reasoning method is as follows:
[0039] Local data is input into the global task sub-model, and the global task output y is output in probabilistic form. g ;
[0040] Local data is input into the personalized task sub-model, and the output is a personalized task output y in probabilistic form. p .
[0041] Compare the probability values of all categories output by the two sub-models above, select the category with the highest probability value as the classification result of the client's local model, and calculate the model's accuracy.
[0042] Secondly, this invention provides a highly generalizable personalized federated learning implementation method, applied to the server side, including:
[0043] Step 1: The server randomly initializes a global model with a dual-branch structure, establishes a connection with the clients participating in the training, and sends the initialized model parameters to the clients participating in the training, waiting for the clients to train; the global model includes a global task sub-model branch and a personalized task sub-model branch.
[0044] Step 2: Receive the statistical parameters of the global model training, the model parameters of the global task sub-models, and the evaluation results of the local models of the clients uploaded by all participating clients;
[0045] Step 3: If the evaluation results of the federated training rounds or the aggregation meet the preset standards, then federated training is stopped; if the evaluation results of the federated training rounds or the aggregation do not meet the preset standards, then the uploaded global task sub-model parameters are weighted and averaged to obtain the aggregated global task sub-model parameters, and the aggregated model parameters are sent to the clients participating in the training, waiting for the clients to train, then returning to Step 2 for cyclical updates; the aggregated evaluation results refer to the final results after the server aggregates the evaluation results of the client-side local models uploaded by each participating client.
[0046] Thirdly, a highly generalizable personalized federated learning implementation method, applied to the client, includes:
[0047] S1: Establish a connection with the server and receive the initialization model parameters sent by the server to initialize the client's local model; the client's local model includes a global task sub-model branch and a personalized task sub-model branch.
[0048] S2: Use local data to perform one round of iterative training on the client-side local model to obtain the model parameters of the client-side local model. Upload the statistical parameters of the client-side local model, the evaluation results of whether the client-side local model meets the preset standards, and the model parameters of the global sub-model to the server.
[0049] S3: Wait for the server to send a command to end training. If the command is to end training, then end training and save the preset best local model on the client. If the command is to continue training, wait for the server to aggregate the model parameters of the global sub-model, receive the aggregated global sub-model parameters sent by the server, update the global sub-model parameters in the local model on the client, return to S2, and perform a loop update.
[0050] Beneficial effects
[0051] This invention provides a highly generalizable personalized federated learning method. The method utilizes a global model with a dual-branch structure, and simultaneously learns global generalization tasks and local personalized tasks through global task sub-models and personalized task sub-models. By leveraging the correlation between tasks to promote each other, the method effectively improves the performance of the client's local model on unknown distributed data and addresses the problem of poor generalization of the client's local model.
[0052] The participating clients train their local models using local data and upload global sub-model parameters to the server. The server then performs federated aggregation of all global task sub-model parameters uploaded by participating clients. This effectively reduces conflicts in personalized feature learning caused by federated aggregation, enhances the learning of personalized features by the client's local model, and preserves global features. Without adding extra federated communication rounds, local training rounds, or training models, it simultaneously completes the learning of personalized and global features, improving the generalization of the client's local model while ensuring its personalized performance. Attached Figure Description
[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0054] Figure 1 This is a global model structure diagram of the dual-branch structure provided by the method described in this invention;
[0055] Figure 2 This is a schematic diagram of the communication between the server and the client provided by the method described in this invention. Detailed Implementation
[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be described in detail below. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other implementation methods obtained by those skilled in the art without creative effort are within the scope of protection of this invention.
[0057] This invention provides a highly generalizable personalized federated learning implementation method to address the poor performance of personalized federated learning on image data with unknown distributions, while simultaneously focusing on the performance of the federated model on local image data. The technical solution provided by this invention is applicable to different neural network models; that is, a batch normalization layer is added after the feature extraction layer, allowing the selection of the neural network model type according to different needs. The technical solution of this invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0058] Example 1
[0059] like Figure 1-2 As shown, this embodiment provides a highly generalizable personalized federated learning method. Taking a convolutional neural network for image classification as an example, this embodiment includes the following steps:
[0060] Step 1: The server randomly initializes a global model with a dual-branch structure and sends the obtained initialization model parameters to multiple selected clients; the global model includes global task sub-model branches and personalized task sub-model branches.
[0061] All clients use the same model structure, which is identical to the model structure used by the server. When the server sends initialization model parameters, clients using the same model structure carry these parameters, ensuring that all clients have the same initialization state. The technical solution of this invention employs a global model with a dual-branch structure, as shown below. Figure 1 As shown.
[0062] Figure 1 The model participating in federated training includes convolutional layers, fully connected layers, and other neural network layers. Since data isolation is an inherent property of federated learning, the data distribution across different clients exhibits several common non-independent and identically distributed scenarios, such as class imbalance and feature distribution shift. In real-world applications, images generated by different medical institutions using different equipment may result in feature distribution shifts. To adapt to the feature distribution of different clients, this invention adds a batch normalization layer after the feature extraction layer. Specifically... Figure 1This is achieved by adding a batch normalization layer after each convolutional and fully connected layer (except for the last fully connected layer used for classification). The implementation involves calculating the mean and variance of a batch of training data; normalizing the output of the two trainable parameters to a value within a pre-defined reasonable range, ultimately producing a shift in the original input specific to the distribution of that batch of data. Since different clients hold different data distributions, the parameters trained by the batch normalization layer will also differ. This invention uses a separate batch normalization layer to implement a personalized federated model.
[0063] Due to the low generalization of personalized models, this invention designs a global task to enhance generalization. For different tasks, this invention, based on the original neural network model, designs two batch normalization layers at the same position after the feature extraction layer, one for the global task and the other for the personalized task, to form the personalized model parameters for the client. In this embodiment, model training has two task branches, each corresponding to a sub-model. Each sub-model consists of a task-shared layer and a task-specific layer. The task-shared layer is optimized jointly by both branches, while the task-specific layer is optimized independently. The global task sub-model consists of a task-shared layer and a global batch normalization layer; the personalized task sub-model consists of a task-shared layer and a personalized batch normalization layer, thus forming a dual-branch global model. Hyperparameters include federated training epochs, local iteration epochs, loss scaling factor, learning rate, and batch size.
[0064] Step 2: Each client initializes its local model with a dual-branch structure using the initialization model parameters sent by the server, and performs the first round of local iterative training using local data to obtain the updated local model; the statistical parameters of the updated local model and the model parameters of the global task sub-model are then uploaded to the server.
[0065] The client uses the initialized client-local model to perform one round of local iterative training in federated training. The specific process is as follows:
[0066] Client i uses locally private image data Where, n i This indicates that client i will n i The set of samples is used as training data; x j y represents the local image data of the j-th input in the training data; j This represents the true label of the j-th local image data in the training data. In practice, the number of local iterations can be set according to actual needs. In this example, one local iteration is selected to avoid overfitting of the local model to the local data.
[0067] Use w gRepresents the global task sub-model, w l This represents a personalized task sub-model. After inputting local image data x into the client's local model, both branches are executed simultaneously to obtain the outputs of two tasks, namely the global task output y. g and personalized task output y l The global task loss is obtained by calculating the cross-entropy loss. g and personalized task loss l ;
[0068] The expression for cross-entropy loss is as follows:
[0069]
[0070] Where a takes the form g or l; y j To predict the target, This represents the actual prediction result; m represents the number of clients participating in the training; the global task loss is used. g and personalized task loss l Obtain the total loss overakl The expression is:
[0071] loss overakl =αloss g +(1-α)loss l
[0072] Where α is the loss ratio coefficient.
[0073] Combining the overall loss and the preset learning rate η, the client obtains the updated gradient of the overall computational model through stochastic gradient descent and backpropagation, thus obtaining the updated model parameters of the client's local model. Specifically, the global task loss... g When calculating the gradient, the gradient of the batch normalization layer specific to the personalized task is 0; similarly, the gradient of the loss in the personalized task is 0. l When calculating the gradient, the gradient of the batch-normalized layer specific to the global task is 0. That is, the overall loss... pverall Optimizing the model is equivalent to simultaneously optimizing two sub-models. The parameters of the task-shared layer are optimized jointly by the two losses, while the parameters of the task-specific layer are optimized separately by the two losses. The model parameter update expression for the client-side local model is as follows:
[0074]
[0075] Among them, g l g represents the overall gradient obtained from one iteration of the personalized task sub-model optimization; g These represent the overall gradient obtained from one iteration of the global task sub-model optimization; wg This represents the model parameters of the global task sub-model; w l t represents the model parameters of the personalized task sub-model; t represents the current round of federated training; i represents the i-th client.
[0076] Step 3: The server calculates the weighted average of the model parameters of the global task sub-models of all clients to obtain the aggregated model parameters of the new global task sub-model, and sends the updated model parameters to multiple selected clients.
[0077] Specifically, the model parameters of the new global task sub-model obtained by weighted averaging are as follows:
[0078] Calculate the proportion of training data collected by the client to the total training data collected by all clients;
[0079] Model parameters w of the global task sub-model g The update formula is as follows:
[0080]
[0081] Where K represents the total number of clients participating in training; k represents the k-th client; n represents the total amount of data collected by all clients during training; n k This represents the amount of data trained by the k-th client; w represents the model parameters of the global task sub-model for the k-th client in the t-th round of federated training; g,t+1 This represents the model parameters of the global task sub-model for the k-th client in the (t+1)-th round of federated training.
[0082] It is worth noting that the federated aggregation of the global task sub-model and the local iterative training process of the local dual-branch structure model are decoupled. The global task sub-model is used to learn the consistency knowledge of the data, and other aggregation methods can also be used to improve the learning of consistency features according to actual needs.
[0083] Step 4: The client uses the model parameters of the global task sub-model sent by the server to update the model parameters of the global task sub-model in the client's local model. Combined with the personalized task sub-model in the client's local model obtained from the iterative training in this round of federated training, the updated client local model is obtained, and one round of federated training is completed.
[0084] In this process, the model parameters of the global sub-model of the client-side local model come from the updated data after the server aggregates the model parameters of the global sub-models of all clients in step 3, including the task-sharing layer and the global task-specific batch normalization layer; while the personalized task-specific batch normalization layer retains the model parameters of the personalized sub-model that was locally updated after training with local image data in step 2. Different client models form differences, so the client-side local global model generated in this step is a personalized model.
[0085] Step 5: The client uses the client-local model updated in Step 4 to perform another round of iterative training based on local data, updates the client-local model parameters, and uploads the updated client-local model statistical parameters and global task sub-model model parameters to the server. Then, it returns to Step 3 and repeats the process of updating the client-local model until it meets the preset standard.
[0086] The specific preset standards are as follows:
[0087] Judgment based on data and client distribution using the loss curve:
[0088] If the data and client distribution are stable and convergent, the client's local model will use the model parameters from the last round of federated training as the training result after a preset number of rounds.
[0089] If the data and client distribution cannot converge stably, the model parameters of the client-side local model that performs best in the validation set within the preset federated training rounds are selected as the training result by adding the validation set from the dataset to the federated training.
[0090] Furthermore, stable convergence refers to a model whose training loss changes little after a certain number of rounds, such as handwritten digit classification, which is a stable convergence data and client distribution. Unstable convergence refers to a model whose training loss still changes significantly after many federated rounds. The validation set is divided from the training set according to a certain proportion; it does not participate in training but is only used to select model parameters for training.
[0091] During the training process:
[0092] Model parameters can be updated by calculating the multi-task loss of the training set and backpropagation. If it is necessary to perform prediction and inference steps on the model, such as calculating the accuracy of the training set, validation set and test set, or inferring the category of the input data, the ensemble inference method can be used in the inference stage to obtain the output results of the client's local model on the input data.
[0093] The ensemble reasoning method is as follows:
[0094] The local image data is input into the global task sub-model, and the global task output y in probabilistic form is output.g ;
[0095] Local image data is input into the personalized task sub-model, and the personalized task output y in probabilistic form is output. p .
[0096] Compare the probability values of all categories output by the two sub-models above, select the category with the highest probability value as the classification result of the client's local model, and calculate the model's accuracy. For unlabeled prediction tasks, the same three steps are performed to output the prediction result.
[0097] Application examples:
[0098] This invention takes handwritten digit data with different feature distributions held by five clients as an example. First, the training and testing data for each client are divided in a 7:3 ratio. Different client data states are formed based on whether the client participates in federated training: federated visible data and federated invisible data. Specifically, a leave-one-out method is used to cyclically select four clients from the five to participate in federated training, providing training data for model training and testing data for model evaluation. The testing data is the visible data for the corresponding client. One client is left to not participate in training but only provide testing data; this data is federated invisible data. The four clients participating in federated training each generate a client-local model. The performance on the invisible data represents the model's generalization ability, while the performance on the client-local visible data represents the model's individual performance. Each client has 743 non-overlapping handwritten digit images. The selection of local models for local clients is as follows... Figure 1 The convolutional neural network shown is applied in practice.
[0099] When the method is applied to solve the data silo problem between different hospitals, each client can be considered an independent hospital, and different hospitals have different data distributions. In summary, the knowledge of a client in a federation can be understood by other clients without explicitly sharing their private data. Through federated aggregation and personalized optimization, the data value of each participant is further explored, improving the convergence, robustness, and generalization of model training.
[0100] Example 2
[0101] This embodiment provides a highly generalizable personalized federated learning implementation method, applied to the server side, including:
[0102] Step 1: The server randomly initializes a global model with a dual-branch structure, establishes a connection with the clients participating in the training, and sends the initialized model parameters to the clients participating in the training, waiting for the clients to train using local image data; the global model includes a global task sub-model branch and a personalized task sub-model branch.
[0103] Step 2: Receive the statistical parameters of the global model training, the model parameters of the global task sub-models, and the evaluation results of the local models of the clients uploaded by all participating clients;
[0104] Step 3: If the evaluation results of the federated training rounds or the aggregation meet the preset standards, then federated training stops; if the evaluation results of the federated training rounds or the aggregation do not meet the preset standards, then the uploaded global task sub-model parameters are weighted and averaged to obtain the aggregated global task sub-model parameters, and the aggregated model parameters are sent to the participating clients, waiting for the clients to train, then returning to Step 2 for cyclical updates; the aggregated evaluation result refers to the final result after the server aggregates the evaluation results of the client-side local models uploaded by each participating client. In this embodiment, the evaluation results of the client-side local models include the accuracy of the training set, validation set, and test set.
[0105] Example 3
[0106] This embodiment provides a highly generalizable personalized federated learning implementation method, applied to a client, including:
[0107] S1: Establish a connection with the server and receive the initialization model parameters sent by the server to initialize the client's local model; the client's local model includes a global task sub-model branch and a personalized task sub-model branch.
[0108] S2: Use local image data to perform one round of iterative training on the client-side local model to obtain the model parameters of the client-side local model. Upload the statistical parameters of the client-side local model, the evaluation results of whether the client-side local model meets the preset standards, and the model parameters of the global sub-model to the server.
[0109] S3: Wait for the server to send a command to end training. If the command is to end training, then end training and save the preset best local model on the client. If the command is to continue training, wait for the server to aggregate the model parameters of the global sub-model, receive the aggregated global sub-model parameters sent by the server, update the global sub-model parameters in the local model on the client, return to S2, and perform a loop update.
[0110] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.
[0111] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.
Claims
1. A highly generalizable personalized federated learning implementation method, characterized in that, include, Step 1: The server randomly initializes a global model with a dual-branch structure and sends the obtained initial model parameters to multiple selected clients; the global model includes global task sub-model branches and personalized task sub-model branches. Step 2: Each client initializes its local model with a dual-branch structure using the initialization model parameters sent by the server, and performs the first round of local iterative training using local image data to obtain the updated local model; the statistical parameters of the updated local model and the model parameters of the global task sub-model are then uploaded to the server. Step 3: The server calculates the weighted average of the model parameters of the global task sub-models of all clients to obtain the model parameters of the new global task sub-model after aggregation, and sends the updated model parameters to multiple selected clients. Step 4: The client uses the model parameters of the global task sub-model sent by the server to update the model parameters of the global task sub-model in the client's local model. Combined with the personalized task sub-model in the client's local model obtained from the iterative training in this round of federated training, the updated client local model is obtained, and one round of federated training is completed. Step 5: The client uses the client-local model updated in Step 4 to perform another round of iterative training based on local image data, updates the client-local model parameters, and uploads the updated client-local model statistical parameters and global task sub-model model parameters to the server. Then, it returns to Step 3 and repeats the update of the client-local model until the preset standard is met. The global model used on the server and the local model used on the client side have the same structure, including the model's feature extraction layer and the corresponding batch normalization layer; the feature extraction layer is a task-shared layer, and the batch normalization layer is a task-specific layer; the task-specific layer includes a global batch normalization layer and a personalized batch normalization layer. The client-side local model is used for image classification.
2. The highly generalizable personalized federated learning implementation method according to claim 1, characterized in that, The global task sub-model consists of a task sharing layer and a global batch normalization layer; the personalized task sub-model consists of a task sharing layer and a personalized batch normalization layer.
3. The highly generalizable personalized federated learning implementation method according to claim 1, characterized in that, The statistical parameters include the amount of data the client participated in training.
4. The highly generalizable personalized federated learning implementation method according to claim 1, characterized in that, The process of obtaining the updated client-side local model through local iterative training in S2 is as follows: Local image data After inputting the local model from the client, two branches are executed simultaneously to obtain the outputs of two tasks, namely the global task output. and personalized task output The global task loss is obtained by calculating the cross-entropy loss. and personalized task loss ; The expression for cross-entropy loss is as follows: ； Where a takes the form of g or l; To predict the target, This is the actual prediction result; m represents the number of clients participating in the training. Utilizing global task loss and personalized task loss Total loss The expression is: ； in, This is the loss ratio coefficient; Combining total loss and preset learning rate The client obtains the updated gradient of the overall computational model through stochastic gradient descent and backpropagation, and then obtains the updated model parameters of the client's local model. The update expression for the model parameters of the client's local model is as follows: ； in, This represents the overall gradient obtained from one iteration of the personalized task sub-model optimization; These represent the overall gradient obtained from one iteration of the global task sub-model optimization; = ; Represents the model parameters of the global task sub-model; represents the model parameters of the personalized task sub-model; t represents the current round of federated training; i represents the i-th client.
5. The highly generalizable personalized federated learning implementation method according to claim 1, characterized in that, The specific model parameters of the new global task sub-model obtained by weighted averaging in S3 are as follows: Calculate the proportion of training data collected by the client to the total training data collected by all clients; Model parameters of the global task sub-model The update formula is as follows: = ； Where K represents the total number of clients participating in training; k represents the k-th client; n represents the total amount of data collected by all clients during training; n k This represents the amount of data trained by the k-th client; This represents the model parameters of the global task sub-model for the k-th client in the t-th round of federated training; This represents the model parameters of the global task sub-model for the k-th client in the (t+1)-th round of federated training.
6. The highly generalizable personalized federated learning implementation method according to claim 1, characterized in that, The preset standards in S5 are as follows: Judgment based on data and client distribution using the loss curve: If the data and client distribution are stable and convergent, the client's local model will use the model parameters from the last round of federated training as the training result after a preset number of rounds. If the data and client distribution cannot converge stably, the validation set of the dataset is added to the federated training, and the model parameters of the client local model that performs best in the validation set within the preset federated training rounds are selected as the training result.
7. The highly generalizable personalized federated learning implementation method according to claim 6, characterized in that, During the training process: When reasoning about the output of the input data, the ensemble reasoning method is used in the reasoning phase to obtain the output of the client's local model on the input data. The ensemble reasoning method is as follows: Local image data is input into the global task sub-model, and the global task output is in probabilistic form. ; Local image data is input into the personalized task sub-model, and the output is a personalized task output in probabilistic form. ; Compare the probability values of all categories output by the two sub-models above, and select the category with the highest probability value as the output of the client's local model.
8. A highly generalizable personalized federated learning implementation method, applied to the server side, characterized in that, include: Step 1: The server randomly initializes a global model with a dual-branch structure, establishes a connection with the clients participating in the training, and sends the initialized model parameters to the clients participating in the training, waiting for the clients to train using local image data; the global model includes a global task sub-model branch and a personalized task sub-model branch. Step 2: Receive the statistical parameters of the local model training, the model parameters of the global task sub-model, and the evaluation results of the local model uploaded by all participating clients; Step 3: If the evaluation results of the federated training rounds or the aggregation meet the preset standards, then federated training stops; if the evaluation results of the federated training rounds or the aggregation do not meet the preset standards, then the uploaded global task sub-model parameters are weighted and averaged to obtain the aggregated global task sub-model parameters, and the aggregated model parameters are sent to the clients participating in the training, waiting for the clients to train, then returning to Step 2 for cyclical updates; the aggregated evaluation results refer to the final result after the server aggregates the evaluation results of the client-side local models uploaded by each participating client; The global model used on the server and the local model used on the client side have the same structure, including the model's feature extraction layer and the corresponding batch normalization layer; the feature extraction layer is a task-shared layer, and the batch normalization layer is a task-specific layer; the task-specific layer includes a global batch normalization layer and a personalized batch normalization layer. The client-side local model is used for image classification.
9. A highly generalizable personalized federated learning implementation method, applied to a client, characterized in that, include: S1: Establish a connection with the server and receive the initialization model parameters sent by the server to initialize the client's local model; the client's local model includes a global task sub-model branch and a personalized task sub-model branch. S2: Use local image data to perform one round of iterative training on the client-side local model to obtain the model parameters of the client-side local model. Upload the statistical parameters of the client-side local model, the evaluation results of whether the client-side local model meets the preset standards, and the model parameters of the global sub-model to the server. S3: Wait for the server to send an instruction. If the instruction is to end training, then end training and save the preset best local client model. If the instruction is to continue training, wait for the server to aggregate the model parameters of the global task sub-model, receive the aggregated global task sub-model parameters sent by the server, update the global task sub-model parameters in the local client model, return to S2, and perform loop updates. The global model used on the server and the local model used on the client side have the same structure, including the model's feature extraction layer and the corresponding batch normalization layer; the feature extraction layer is a task-shared layer, and the batch normalization layer is a task-specific layer; the task-specific layer includes a global batch normalization layer and a personalized batch normalization layer. The client-side local model is used for image classification.