Personalized federated learning method, device and system based on global feature sharing
By introducing global features and conditional mutual information regularization terms into federated learning, data features are shared, which solves the problems of poor generalization performance and overfitting of local models in federated learning, and achieves better model generalization and data privacy protection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI JIAOTONG UNIV
- Filing Date
- 2023-06-30
- Publication Date
- 2026-04-21
AI Technical Summary
Existing federated learning methods suffer from poor generalization performance when training global models due to the heterogeneity of node data, and personalized federated learning is prone to overfitting of local models.
By introducing global features and conditional mutual information regularization terms, data features are shared between the client and the server. Global features are used to train the node model to prevent overfitting of the local model, and the local model parameters are updated through backpropagation and gradient optimization.
This approach enables the training of local models with better generalization performance, prevents overfitting of local models, and improves the generalization ability of models while protecting data privacy.
Smart Images
Figure CN116777015B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, specifically to a personalized federated learning method, apparatus, and system based on global feature sharing. Background Technology
[0002] Federated learning is a distributed machine learning method that trains a model on multiple local nodes without transferring the original dataset to a central server. It is typically used for privacy-preserving tasks, such as healthcare and finance. The goal of standard federated learning is to train a global model with good generalization performance, in which nodes and the server share the same global model. However, because the data distribution on each node is different, the global model cannot adapt well to the heterogeneous data across all nodes, resulting in poor generalization performance.
[0003] Therefore, personalized federated learning is proposed. In personalized federated learning, each node is provided with a personalized model that performs best on its local data. Specifically, personalized federated learning requires training a private model for each node to fit the local dataset. However, due to the heterogeneity of the local data and the limitation of the number of samples, the local model of the node is prone to overfitting. Summary of the Invention
[0004] In view of the shortcomings of the prior art, the purpose of this disclosure is to provide a personalized federated learning method, apparatus and system based on global feature sharing.
[0005] To achieve the above objectives, according to a first aspect of this disclosure, a personalized federated learning method based on global feature sharing is provided, applied to a client, the client including a local model, comprising:
[0006] Receive the global feature extractor model and global features sent by the server;
[0007] Initialize the local model based on the global feature extractor model and the local classifier model;
[0008] Local image data is input into the initialized local model for model training, and the loss function of the local model is determined. The loss function includes the cross-entropy loss between the training labels and the real labels of the local image data, and the conditional mutual information regularization term.
[0009] Based on the loss function of the local model, the local model is updated using backpropagation.
[0010] When the local model converges, the target local model is determined.
[0011] Optionally, the local model includes a local feature extractor model and a local classifier model;
[0012] The first update process of the local model based on backpropagation according to the loss function of the local model includes:
[0013] Based on the loss function of the local model, the second update process is performed on the local feature extractor model and the third update process is performed on the local classifier model based on backpropagation.
[0014] Optionally, the method further includes:
[0015] The local image data is input into the local feature extractor model after the second update processing to determine the last updated features of each image in the local image data;
[0016] When the first update processing number of the local model reaches a preset threshold, the last update features of the local image data with the same image category are subjected to a first product processing according to the image category to determine the local features.
[0017] Optionally, the method further includes:
[0018] When the number of first update processes of the local model reaches a preset threshold, the local feature extractor model after the last second update process is determined.
[0019] The local feature extractor model and the local features after the last second update are sent to the server.
[0020] According to a second aspect of this disclosure, a personalized federated learning method based on global feature sharing is provided, applied to a server-side environment. The server-side environment includes a global feature extractor model, comprising:
[0021] Applied to the server side, the server side includes a global feature extractor model, comprising:
[0022] Initialize the global feature extractor model and global features;
[0023] The initialized global feature extractor model and the global features are sent to the client;
[0024] Receive the local feature extractor model and local features sent by the client after the last second update process;
[0025] The local feature extractor model is then subjected to global aggregation to determine the global feature extractor model.
[0026] The local features are then subjected to a second product based on the image category to determine the global features.
[0027] Optionally, the method further includes:
[0028] The global feature extractor model and the global features are sent to the client, and the client performs the step of receiving the global feature extractor model and the global features sent by the server.
[0029] According to a third aspect of this disclosure, a personalized federated learning device based on global feature sharing is provided, applied to a client, the client including a local model, comprising:
[0030] The client's first receiving module is used to receive the global feature extractor model and global features sent by the server.
[0031] The client initialization module is used to initialize the local model based on the global feature extractor model, the global features, and the local classifier.
[0032] The client-side first determination module is used to input local image data into the initialized local model for model training, and determine the loss function of the local model. The loss function includes the cross-entropy loss between the training labels and the real labels of the local image data, and the conditional mutual information regularization term.
[0033] The client-side first update module is used to perform a first update process on the local model based on backpropagation according to the loss function of the local model;
[0034] The second determination module on the client side is used to determine the target local model when the local model converges.
[0035] According to a fourth aspect of this disclosure, a personalized federated learning device based on global feature sharing is provided, applied on a server side, the server side including a global feature extractor model, comprising:
[0036] The server initialization module is used to initialize the global feature extractor model and global features.
[0037] The server sending module is used to send the initialized global feature extractor model and the global features to the client;
[0038] The server receiving module is used to receive the local feature extractor model and local features sent by the client after the last second update processing.
[0039] The server first determination module is used to perform global aggregation processing on the local feature extractor model to determine the global feature extractor model.
[0040] The server's second determination module is used to perform a second product process on the local features according to the image category to determine the global features.
[0041] According to the fifth aspect of this disclosure, a personalized federated learning system based on global feature sharing is provided, comprising:
[0042] The local model update module is used to perform a first update process on the local model based on backpropagation at the local training node. The first update process on the local model includes a second update process on the local feature extractor model and a third update process on the local classifier model.
[0043] The local feature extraction module is used to extract the update features of the local image data and determine the local features when the local model update module performs the first update process on the local model at the local training node.
[0044] The global feature extractor aggregation module is used to perform global aggregation processing on the local feature extractor model after the second update processing to determine the global feature extractor model.
[0045] The global feature update module is used to perform a second product processing on the local features determined by the local feature extraction module according to the image category to determine the global features;
[0046] The communication module is used to transmit the local feature extractor model and the local features of the client after the second update processing to the server, and to transmit the global feature extractor model and the global features of the server to the client.
[0047] Optionally, the client includes the local model update module and the local feature extraction module, the server includes the global feature extractor aggregation module and the global feature update module, and the communication module is also used to connect the client and the server.
[0048] Compared with the prior art, the embodiments disclosed herein have at least one of the following beneficial effects:
[0049] By introducing global features and conditional mutual information regularization terms, the data of the local training nodes on the client is shared. Global features with wide data distribution and comprehensive data characteristics are used to provide more global and generalized data information to the local training nodes. The local training nodes can use the node data of other nodes, thereby preventing overfitting of the local model. Attached Figure Description
[0050] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:
[0051] Figure 1 This is a flowchart illustrating a personalized federated learning method based on global feature sharing applied to a client, according to an exemplary embodiment.
[0052] Figure 2 This is a flowchart illustrating a personalized federated learning method based on global feature sharing applied to a client, according to another exemplary embodiment.
[0053] Figure 3 This is a flowchart illustrating a personalized federated learning method based on global feature sharing applied to a server, according to an exemplary embodiment.
[0054] Figure 4 This is a schematic diagram illustrating the signaling interaction between a client, a communication module, and a server according to an exemplary embodiment.
[0055] Figure 5 This is a block diagram illustrating a personalized federated learning device based on global feature sharing applied to a client, according to an exemplary embodiment.
[0056] Figure 6 This is a block diagram illustrating a personalized federated learning device based on global feature sharing applied to a server, according to an exemplary embodiment.
[0057] Figure 7 This is a block diagram illustrating a personalized federated learning system based on global feature sharing, according to an exemplary embodiment. Detailed Implementation
[0058] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.
[0059] Figure 1 This is a flowchart illustrating a personalized federated learning method based on global feature sharing applied to a client, according to an exemplary embodiment. Figure 1 As shown, a personalized federated learning method based on global feature sharing is applied to a client. The client includes a local module, comprising:
[0060] S11 receives the global feature extractor model and global features sent by the server.
[0061] The client's local infrastructure includes multiple local training nodes. These nodes train local models based on local privacy data, which is essentially local image data. Due to the poor stability of the communication connection, the local training nodes participating in the first round of model training are randomized.
[0062] In this round, the random local training nodes participating in local model training receive the global feature extractor model and global features sent by the server.
[0063] S12, initialize the local model based on the global feature extractor model, global features, and local classifier model.
[0064] The local model includes a local feature extractor model and a local classifier model. After receiving the global feature extractor model and global features from the server, the random local training node performs feature classification on the global features based on the global feature extractor, global features, and local classifier, thereby initializing the local feature extractor model.
[0065] S13. Input the local image data into the initialized local model for model training, and determine the loss function of the local model. The loss function includes the cross-entropy loss between the training labels and the real labels of the local image data, and the conditional mutual information regularization term.
[0066] In one possible embodiment, local image data is input into the local model of the random local training node in the current round for forward model training.
[0067] Specifically, local image data is input into the local feature extractor model of the local model, which outputs local image features; local image data is input into the local classifier model, which outputs predicted training labels.
[0068] Loss functions include:
[0069]
[0070] in, Let f represent the loss function. i (w i ) represents the local model w i The cross-entropy loss function, i.e., the cross-entropy loss between the training labels and the ground truth labels of the local image data, β||I i (z;x i |y i )-I(z;x|y i || represents the conditional mutual information regularization term, β represents the Lagrange multiplier, and x i This represents the input local image data, y iThe training label represents the local image data, z represents the generated image features, and I represents the training label corresponding to the local image data. i (z;x i |y i I(z; x|y) represents local conditional mutual information. i ) represents global conditional mutual information, and p represents the probability distribution.
[0071] In this disclosure, the conditional mutual information regularization term represents the difference between local conditional mutual information and global conditional mutual information. Local conditional mutual information represents the mutual information between local image data and local features at a given label; global conditional mutual information represents the mutual information between global image data and global features at a given label. The conditional mutual information regularization term is used to encourage the local and global conditional mutual information to be closer to each other.
[0072] S14, based on the loss function of the local model, perform the first update process on the local model based on backpropagation.
[0073] The first update process includes the second update process and the third update process.
[0074] In some possible embodiments, the local model is first updated based on backpropagation according to the loss function of the local model, including S141.
[0075] S141, based on the loss function of the local model, perform a second update process on the local feature extractor model and a third update process on the local classifier model based on backpropagation.
[0076] Based on the determined loss function of the local model, the gradient of the local model is determined through backpropagation and gradient calculation. The parameters of the local model can be optimized by using the stochastic gradient descent algorithm, that is, the parameters of the local feature extractor model and the local classifier model are updated.
[0077] Those skilled in the art should understand that other algorithms can also be used to optimize the parameters of the local model, all of which fall within the protection scope of this disclosure.
[0078] S15, when the local model converges, determine the target local model.
[0079] When the local model converges, training on the local model is stopped, and the local model trained in the last iteration is determined as the target local model. This determines the personalized model on the local training node and prevents overfitting of the local model.
[0080] By introducing global features and conditional mutual information regularization terms, the data of the local training nodes on the client is shared. Global features with wide data distribution and comprehensive data characteristics are used to provide more global and generalized data information to the local training nodes. The local training nodes can use the node data of other nodes, thereby preventing overfitting of the local model.
[0081] Figure 2 This is a flowchart illustrating a personalized federated learning method based on global feature sharing applied to a client, according to another exemplary embodiment.
[0082] In some possible embodiments, such as Figure 2 As shown, the personalized federated learning method based on global feature sharing, applied to the client, also includes S16 to S17.
[0083] S16, input the local image data into the local feature extractor model after the second update processing, and determine the last updated features of each image in the local image data.
[0084] During the first update process of the local model, the image features of the local image data output by the model after each second update process are saved.
[0085] When the number of repetitions of steps S11 to S14 reaches a preset threshold K (K≥1), the local image features output by the last second update processing of the local feature extractor model are determined, that is, the last update features of each image in the local image data.
[0086] S17, when the number of first update processing times of the local model reaches a preset threshold, the last update features of local image data with the same image category are subjected to the first product processing according to the image category to determine the local features.
[0087] The first product processing can be performed using an expert product aggregation method.
[0088] Following the example above, the local image features output from the last second update process of the local feature extractor model are multiplied according to the image category to generate features of different categories, and these are then identified as local features.
[0089] In some possible embodiments, such as Figure 2 As shown, the personalized federated learning method based on global feature sharing, applied to the client, also includes S18 to S19.
[0090] S18, when the number of first update processes of the local model reaches a preset threshold, determine the local feature extractor model after the last second update process.
[0091] S19, the local feature extractor model and local features after the second update are sent to the server.
[0092] Following the example above, when the number of repetitions of steps S11 to S14 reaches a preset threshold K (K≥1), the last local feature extractor model and local features of each local training node in this round of training, after undergoing the second update processing, are sent to the server through the communication module.
[0093] The communication module is used to connect the various nodes of the client with the server.
[0094] The above technical solution introduces a regularization term to the local training nodes to minimize the difference between local conditional mutual information and global conditional mutual information, thereby encouraging local training nodes to learn and share feature representations and alleviate the overfitting phenomenon of the local model.
[0095] Figure 3 This is a flowchart illustrating a personalized federated learning method based on global feature sharing applied to a server, according to an exemplary embodiment. Figure 3 As shown, a personalized federated learning method based on global feature sharing is applied to the server side, including S21 to S25.
[0096] S21, Initialize the global feature extractor model and global features.
[0097] S22 sends the initialized global feature extractor model and global features to the client.
[0098] The server first initializes the global feature extractor model and global features. Due to the unstable communication connection between the server and the client, the server sends a unified global feature extractor model and unified global features to the local training nodes of the client that are participating in this round of model training.
[0099] S23, receive the local feature extractor model and local features after the last second update processing sent by the client.
[0100] S24, perform global aggregation on the local feature extractor model to determine the global feature extractor model.
[0101] The global aggregation process can be performed using the arithmetic mean method, with the weight coefficient being the proportion of the number of local training nodes participating in the model training in this round to the total number of local training nodes.
[0102] The server receives the local feature extractor models sent by the local training nodes participating in the current round of model training, and performs global aggregation processing on the received local feature extractor models to determine the global feature extractor model.
[0103] S25, perform a second product of the local features according to the image category to determine the global features.
[0104] The second product processing can be performed using the expert product method.
[0105] The server multiplies the local features of each local training node participating in the model training in this round according to the image category of the local image data, and outputs global features of different categories.
[0106] In some possible embodiments, such as Figure 3 As shown, a personalized federated learning method based on global feature sharing is applied to the server side and also includes S26.
[0107] S26, the global feature extractor model and global features are sent to the client, and the client executes the steps of receiving the global feature extractor model and global features sent by the server.
[0108] Following the example above, the determined global feature extractor model and global features are sent to the client. The client executes S12 to S19, and the server executes S21 to S26 until the local model is updated based on backpropagation, at which point the local model converges and training stops.
[0109] The above technical solution can train a global feature extractor model with good generalization performance. In this process, the client can train a local model with superior generalization performance and store local image data in a local location to prevent data leakage.
[0110] Figure 4 This is a schematic diagram illustrating the signaling interaction between a client, a communication module, and a server according to an exemplary embodiment.
[0111] like Figure 4 As shown in S31, the server initializes the global feature extractor model and global features.
[0112] S32, the server sends the initialized global feature extractor model and global features to the client.
[0113] S33, the client receives the global feature extractor model and global features sent by the server.
[0114] S34, the client initializes the local model based on the global feature extractor model, global features, and local classifier model.
[0115] S35, the client inputs local image data into the initialized local model for model training and determines the loss function of the local model.
[0116] S36, the client performs a second update on the local feature extractor model and a third update on the local classifier model based on backpropagation, according to the loss function of the local model.
[0117] S37, the client inputs the local image data into the local feature extractor model after the second update processing to determine the last updated features of each image in the local image data.
[0118] S38, when the number of first update processes of the local model reaches a preset threshold, the client performs a first product process on the last update features of local image data with the same image category to determine the local features.
[0119] S39, when the number of times the local model is updated reaches a preset threshold, the client determines the local feature extraction model after the last second update.
[0120] S40, the client sends the local feature extractor model and local features, which have undergone the second update, to the server.
[0121] S41, the server receives the local feature extractor model and local features sent by the client after the second update.
[0122] S42, the server performs global aggregation on the local feature extractor model to determine the global feature extractor model.
[0123] S43, the server performs a second product of the local features based on the image category to determine the global features.
[0124] S44, the server sends the global feature extractor model and global features to the client.
[0125] S45, when the client local model converges, determine the target local model.
[0126] The communication module serves as the medium for signaling transmission between the client and the server.
[0127] After S31 to S44 are executed, S33 to S44 are executed in a loop. When it is determined that the local feature extractor model of the local model has converged, S45 is executed.
[0128] In some possible implementations, a local model with superior generalization performance can be trained for each client, and local data can be stored securely. Personalized federated learning methods based on global feature sharing offer promise for connecting decentralized medical data sources and protecting user data privacy.
[0129] As an example, in the diagnosis of intrathoracic lymph nodes, imaging is performed using an ultrasound-guided endobronchial probe, and a deep learning model is used for image classification to diagnose the patient's condition. To protect patient privacy, hospitals cannot publicly release the patient's image dataset, resulting in insufficient training data for the deep learning model. If individual hospitals train their models based on their own patient monitoring image datasets, the limited training data can easily lead to overfitting.
[0130] Based on a trusted third-party organization as the server and each hospital as the client, a communication network, namely a communication module, is built between the server and each hospital. In this module, the hospitals do not need to communicate directly with each other, and the third-party organization does not transmit the original disease monitoring image dataset between the hospitals.
[0131] Based on S31 to S45 of the personalized federated learning method based on global feature sharing described above, a deep learning model with good generalization performance is trained for each hospital.
[0132] (1) A third-party organization (server side) initializes the global feature extractor model and global features.
[0133] (2) The third-party organization (server) sends the unified initialized global feature extractor model and global features to the hospitals (clients) participating in the model training in this round. Due to the instability of the communication connection, the hospitals participating in the model training in this round can participate randomly.
[0134] (3) The hospital (client) receives the global feature extractor model and global features sent by the third-party organization (server).
[0135] (4) The hospital (client) initializes the local model based on the global feature extractor model, global features and local classifier model.
[0136] (5) The hospital (client) inputs the stored patient condition detection image dataset into the initialized local model for model training, and determines the loss function of the local model, which is the sum of the cross-entropy loss between the training labels and the true labels of the local image data and the conditional mutual information regularization term:
[0137]
[0138] (6) The hospital (client) performs a second update on the local feature extractor model and a third update on the local classifier model based on the loss function of the local model and backpropagation.
[0139] (7) The hospital (client) inputs the stored patient condition detection image dataset in the hospital into the local feature extractor model after the second update processing to determine the last updated features of each image in the stored patient condition detection image dataset in the hospital.
[0140] (8) When the number of first update processing times of the local model reaches the preset threshold, the hospital (client) performs the first product processing on the last update features of the images in the disease detection image dataset of patients with the same image category to determine the local features.
[0141] (9) When the number of times the local model is updated reaches the preset threshold, the hospital (client) determines the local feature extraction model after the last second update.
[0142] (10) The hospital (client) will send the local feature extractor model and local features after the last second update to the server.
[0143] (11) The third-party organization (server) receives the local feature extractor model and local features sent by the hospital (client) after the last second update.
[0144] (12) A third-party organization (server) performs global aggregation processing on the local feature extractor model to determine the global feature extractor model. The global aggregation processing can be performed using the arithmetic mean method.
[0145] (13) A third-party organization (server) performs a second product of the local features based on the image category to determine the global features.
[0146] (14) The third-party organization (server) will send the global feature extractor model and global features to the hospital (client) participating in the model training, and continue to train the local model of the hospital (client).
[0147] (15) Stop model training when the local model of the participating hospital (client) converges.
[0148] The data in the above process is transmitted through the communication network between the hospital and the third-party institution, and there is no need for data transmission between the hospitals.
[0149] The above technical solution can train a local model with superior generalization performance for each hospital, while storing its private data locally without leakage, providing significant application value for decentralized medical data sources and medical privacy protection.
[0150] In some possible embodiments, as shown in Table 1, based on the EMNIST-L, Fashion-MNIST, CIFRA-10, and CIFAR-100 image datasets, using non-independent data partitioning methods non-iid-1 and non-iid-2, the generalization performance of the personalized federated learning method based on global feature sharing disclosed herein, as well as the methods FedAvg, FedAvg-FT, FedPer, LG-FedAvg, FedRep, FedBABU, Ditto, FedSR-FT, and FedPAC, is verified.
[0151] In the non-iid-1 partitioning method, each local training node uniformly samples from the four different categories of image datasets: EMNIST-L, Fashion-MNIST, CIFRA-10, and CIFAR-100. The amount of sampled data is the same for all four categories. In the non-iid-2 partitioning method, each local training node samples according to the Dirichlet distribution. The training dataset of each local training node includes a variable number of image data from each category of the image dataset, and the amount of data in each category of the image dataset is different.
[0152]
[0153]
[0154] Table 1
[0155] Here, Ours represents the personalized federated learning method for global feature sharing disclosed herein.
[0156] As shown in Table 1, the personalized federated learning method with global feature sharing disclosed in this paper exhibits the best generalization performance on the EMNIST-L, Fashion-MNIST, CIFRA-10, and CIFAR-100 image datasets.
[0157] Based on the same concept, this disclosure also provides a personalized federated learning device based on global feature sharing, applied to a client. Figure 5 This is a block diagram illustrating a personalized federated learning device based on global feature sharing applied to a client, according to an exemplary embodiment. (Refer to...) Figure 5The personalized federated learning device 100 based on global feature sharing includes: a client first receiving module 110, a client initialization module 120, a client first determining module 130, a client first updating module 140, and a client second determining module 150.
[0158] The client's first receiving module 110 is used to receive the global feature extractor model and global features sent by the server.
[0159] The client initialization module 120 is used to initialize the local model based on the global feature extractor model, the global features, and the local classifier;
[0160] The client first determination module 130 is used to input local image data into the initialized local model for model training, and determine the loss function of the local model. The loss function includes the cross-entropy loss between the training labels and the real labels of the local image data and the conditional mutual information regularization term.
[0161] The client-side first update module 140 is used to perform a first update process on the local model based on backpropagation according to the loss function of the local model;
[0162] The second determination module 150 on the client side is used to determine the target local model when the local model converges.
[0163] By introducing global features and conditional mutual information regularization terms, the data of the local training nodes on the client is shared. Global features with wide data distribution and comprehensive data characteristics are used to provide more global and generalized data information to the local training nodes. The local training nodes can use the node data of other nodes, thereby preventing overfitting of the local model.
[0164] Optionally, the local model includes a local feature extractor model and a local classifier model.
[0165] Optionally, the client first update module 140 is further configured to perform the second update process on the local feature extractor model and the third update process on the local classifier model based on backpropagation according to the loss function of the local model.
[0166] Optionally, the device 100 further includes:
[0167] The third determination module of the client is used to input the local image data into the local feature extractor model after the second update processing, and determine the last updated features of each image in the local image data;
[0168] The fourth determination module of the client is used to determine the local features by performing a first product process on the last update features of the local image data with the same image category when the first update processing number of the local model reaches a preset threshold.
[0169] Optionally, the device 100 further includes:
[0170] The fifth determination module of the client is used to determine the local feature extractor model after the last second update process when the first update processing number of the local model reaches a preset threshold.
[0171] The client sending module is used to send the local feature extractor model and the local features after the last second update processing to the server.
[0172] This disclosure also provides a personalized federated learning device based on global feature sharing, applied to the server side. Figure 6 This is a block diagram illustrating a personalized federated learning device based on global feature sharing applied to a client, according to another exemplary embodiment. (Refer to...) Figure 6 The personalized federated learning device 200 based on global feature sharing includes: a server initialization module 210, a server sending module 220, a server receiving module 230, a server first determination module 240, and a server second determination module 250.
[0173] Server initialization module 210 is used to initialize the global feature extractor model and global features;
[0174] Server sending module 220 is used to send the initialized global feature extractor model and the global features to the client;
[0175] Server receiving module 230 is used to receive the local feature extractor model and local features sent by the client after the last second update processing.
[0176] The server first determination module 240 is used to perform global aggregation processing on the local feature extractor model to determine the global feature extractor model.
[0177] The server second determination module 250 is used to perform a second product processing on the local features according to the image category to determine the global features.
[0178] Optionally, the server sending module 220 is further configured to send the global feature extractor model and the global features to the client.
[0179] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0180] Figure 7 This is a block diagram illustrating a personalized federated learning system based on global feature sharing, according to an exemplary embodiment. Figure 7 As shown, a personalized federated learning system based on global feature sharing includes:
[0181] The local model update module is used to perform a first update process on the local model based on backpropagation at the local training node. The first update process on the local model includes a second update process on the local feature extractor model and a third update process on the local classifier model.
[0182] The local feature extraction module is used to extract the update features of the local image data and determine the local features when the local model update module performs the first update process on the local model at the local training node.
[0183] The global feature extractor aggregation module is used to perform global aggregation processing on the local feature extractor model after the second update processing to determine the global feature extractor model.
[0184] The global feature update module is used to perform a second product processing on the local features determined by the local feature extraction module according to the image category to determine the global features;
[0185] The communication module is used to transmit the local feature extractor model and the local features of the client after the second update processing to the server, and to transmit the global feature extractor model and the global features of the server to the client.
[0186] Optionally, the client includes the local model update module and the local feature extraction module, the server includes the global feature extractor aggregation module and the global feature update module, and the communication module is also used to connect the client and the server.
[0187] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.
Claims
1. A personalized federated learning method based on global feature sharing, characterized in that, Applied to a client, the client including a local model, including: Receive the global feature extractor model and global features sent by the server; Based on the global feature extractor model and the local classifier model, initialize the local model, which includes the local feature extractor model and the local classifier model; Local image data is input into an initialized local model for model training. The loss function of the local model is determined. The loss function is the sum of the cross-entropy loss between the training labels and the ground truth labels of the local image data and the conditional mutual information regularization term. The conditional mutual information regularization term represents the difference between the local conditional mutual information and the global conditional mutual information. The local conditional mutual information represents the mutual information between the local image data and local features at a given label. The global conditional mutual information represents the mutual information between the global image data and global features at a given label. Based on the loss function of the local model, the local model is updated using backpropagation. When the local model converges, the target local model is determined; The step of performing a first update process on the local model based on backpropagation according to the loss function of the local model includes: Based on the loss function of the local model, a second update process is performed on the local feature extractor model and a third update process is performed on the local classifier model based on backpropagation.
2. The method according to claim 1, characterized in that, The method further includes: The local image data is input into the local feature extractor model after the second update processing to determine the last updated features of each image in the local image data; When the first update processing number of the local model reaches a preset threshold, the last update features of the local image data with the same image category are subjected to a first product processing according to the image category to determine the local features.
3. The method according to claim 2, characterized in that, The method further includes: When the number of first update processes of the local model reaches a preset threshold, the local feature extractor model after the last second update process is determined. The local feature extractor model and the local features after the last second update are sent to the server.
4. The personalized federated learning method based on global feature sharing according to claim 1, characterized in that, Applied to the server side, the server side includes a global feature extractor model, and the server side interacts with the client via signaling, including: Initialize the global feature extractor model and global features; The initialized global feature extractor model and the global features are sent to the client; Receive the local feature extractor model and local features sent by the client after the last second update process; The local feature extractor model is then subjected to global aggregation to determine the global feature extractor model. The local features are then subjected to a second product based on the image category to determine the global features.
5. The method according to claim 4, characterized in that, The method further includes: The global feature extractor model and the global features are sent to the client, and the client performs the step of receiving the global feature extractor model and the global features sent by the server.
6. A personalized federated learning device based on global feature sharing, characterized in that, Applied to a client, the client including a local model, including: The first receiving module on the client side is used to receive the global feature extractor model and global features sent by the server; The client initialization module is used to initialize a local model based on the global feature extractor model, the global features, and the local classifier. The local model includes a local feature extractor model and a local classifier model. The client-side first determination module is used to input local image data into an initialized local model for model training, and determine the loss function of the local model. The loss function is the sum of the cross-entropy loss between the training labels and the ground truth labels of the local image data and the conditional mutual information regularization term. The conditional mutual information regularization term represents the difference between the local conditional mutual information and the global conditional mutual information. The local conditional mutual information represents the mutual information between the local image data and local features at a given label, and the global conditional mutual information represents the mutual information between the global image data and global features at a given label. The client-side first update module is used to perform a first update process on the local model based on backpropagation according to the loss function of the local model; The second determination module on the client side is used to determine the target local model when the local model converges; The client-side first update module is further configured to perform a second update on the local feature extractor model and a third update on the local classifier model based on backpropagation, according to the loss function of the local model.
7. The personalized federated learning device based on global feature sharing according to claim 6, characterized in that, Applied to the server side, the server side includes a global feature extractor model, and the signaling interaction between the server side and the client includes: The server initialization module is used to initialize the global feature extractor model and global features. The server sending module is used to send the initialized global feature extractor model and the global features to the client; The server receiving module is used to receive the local feature extractor model and local features sent by the client after the last second update processing. The server first determination module is used to perform global aggregation processing on the local feature extractor model to determine the global feature extractor model. The server's second determination module is used to perform a second product process on the local features according to the image category to determine the global features.
8. A personalized federated learning system based on global feature sharing, comprising: The local model update module is used to perform a first update process on the local model based on backpropagation according to the loss function of the local model at the local training node. The first update process of the local model includes a second update process of the local feature extractor model and a third update process of the local classifier model. The loss function of the local model is the sum of the cross-entropy loss between the training labels and the true labels of the local image data and the conditional mutual information regularization term. The conditional mutual information regularization term represents the difference between the local conditional mutual information and the global conditional mutual information. The local conditional mutual information represents the mutual information between the local image data and the local features when given a label. The global conditional mutual information represents the mutual information between the global image data and the global features when given a label. The local feature extraction module is used to extract the update features of the local image data and determine the local features when the local model update module performs the first update process on the local model at the local training node. The global feature extractor aggregation module is used to perform global aggregation processing on the local feature extractor model after the second update processing to determine the global feature extractor model. The global feature update module is used to perform a second product processing on the local features determined by the local feature extraction module according to the image category to determine the global features; The communication module is used to transmit the local feature extractor model and the local features of the client after the second update processing to the server, and to transmit the global feature extractor model and the global features of the server to the client.
9. The system according to claim 8, characterized in that, The client includes the local model update module and the local feature extraction module, the server includes the global feature extractor aggregation module and the global feature update module, and the communication module is also used to connect the client and the server.
Citation Information
Patent Citations
Two-stage federated learning method and system
CN113723617A
KR20220170583A