Knowledge distillation-based efficient personalized federal learning method, equipment and product

Abstract

The invention discloses an efficient personalized federal learning method, device and product based on knowledge distillation. The method comprises the following steps: firstly, clustering clients with similar data distribution by using a client clustering algorithm based on Euclidean distance; each client completes local model training on a local private data set, completes model prediction on a public data set based on dynamic temperature adjustment and uploads a soft prediction result to the server; the server completes global aggregation; and finally, the client downloads a global aggregation result, and completes local model updating and starts a new round of local model training by using a triple dynamic distillation strategy. According to the invention, more accurate and efficient personalized federal learning is realized.

Description

Technical Field

[0001] The present invention belongs to the field of information security technology and relates to a personalized federated learning method, device and product, and specifically to an accurate and efficient personalized federated learning method, device and product based on knowledge distillation. Background Art

[0002] Federated learning is one of the most promising distributed learning frameworks, which allows multiple clients to collaboratively train a global model under the coordination of a central server. The main goal is to provide privacy protection for the client's local data and solve the "data island" problem. Its applications involve industries such as finance, healthcare, and manufacturing. However, a single global model trained from all clients may not satisfy clients whose tasks or data distributions deviate significantly from the rest. Therefore, adopting personalized models seems to be an effective solution in federated learning, namely personalized federated learning. However, recent work on personalized federated learning inevitably has to be the same in model structure and size to aggregate parameters from all clients. In practical scenarios, clients usually prefer to use customized model architectures (i.e., different structures and sizes) to adapt to heterogeneity in computing, communication, and storage capabilities.

[0003] A promising approach is to use knowledge distillation to transfer knowledge by exchanging soft predictions instead of using model parameters. Therefore, personalized federated learning methods based on knowledge distillation are studied for collaborative training of heterogeneous models. Unfortunately, personalized federated learning based on knowledge distillation faces three major challenges: first, due to the distributed nature of clients, data from different clients are usually non-independent and identically distributed, which leads to low accuracy and huge training overhead (i.e., training time and communication overhead); second, the distillation temperature of traditional knowledge distillation is fixed, and the dynamics of the model training process are not considered; third, the knowledge source of simple knowledge distillation is single, which may ignore additional beneficial information, resulting in poor model performance. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides an accurate, efficient and personalized federated learning method, device and product based on knowledge distillation, which improves the accuracy of the model, accelerates the training of the model and reduces communication overhead.

[0005] The technical solution adopted by the method of the present invention is: an efficient personalized federated learning method based on knowledge distillation, involving entities including a server and N clients; comprising the following steps:

[0006] Step 1: The server clusters clients with similar data distribution using a client clustering algorithm based on Euclidean distance;

[0007] Step 2: Each client i uses its private dataset D i Train a local model and complete the local model performance evaluation on the public dataset D0 based on dynamic temperature adjustment. The evaluation result is used as the soft prediction result. Upload to the server, represents the soft prediction value of client i in round r;

[0008] Step 3: The server aggregates local soft prediction results And distribute the aggregation results to all clients;

[0009] Step 4: Each client i uses the triple dynamic distillation strategy to complete the local model update and start a new round of local model training based on the global aggregation results.

[0010] Preferably, the specific implementation of step 1 includes the following sub-steps:

[0011] Step 1.1: Randomly select a client z∈[N];

[0012] Step 1.2: Calculate the Euclidean distance [E(R1,R z ),E(R2,R z ),…,E(R N ,R z )], where R i represents the data distribution of client i∈[N], E(R i ,R j ) represents the Euclidean distance between the data distributions of client i and client j, and uploads the result to the server;

[0013] Step 1.3: The server clusters clients with similar data distribution based on the K-means clustering algorithm and broadcasts the results to all clients.

[0014] Preferably, the specific implementation of step 2 includes the following sub-steps:

[0015] Step 2.1: Client i∈[N] obtains a local private dataset D i Train the local model w i ;

[0016] calculate Where η1 represents the learning rate; in Represents the cross entropy loss function that measures the difference between the predicted results and the true labels on the private dataset; each client i∈[N] stores a size of n i Local private dataset x i represents the i-th data sample, yi ∈{1,2,…,C} represents x i The labels of the local datasets are obtained by combining all the local datasets D i Connections are formed, is the size of the global dataset;

[0017] Step 2.2: Client i∈[N] completes the model performance evaluation on the public dataset D0 and outputs the calculation results of the last layer of the model in Represents the calculation result of the last layer of the model in category C and calculates the soft prediction value Where T r represents the distillation temperature of the rth round, x represents the data sample collected from the public dataset D0; T r The calculation expression is as follows: in k1, k2, r0 are hyperparameters for dynamic temperature adjustment; T0 is the initial distillation temperature setting.

[0018] Preferably, in step 3, the server obtains the soft prediction results uploaded by each client, and then completes global aggregation according to the client clustering results to calculate Where |v| represents the total number of clients in cluster v, n v represents the size of the dataset in cluster v; the global dataset D is obtained by dividing all the datasets of size n i The local dataset D i Connections are formed, is the size of the global dataset.

[0019] Preferably, the specific implementation of step 4 includes the following sub-steps:

[0020] Step 4.1: Each client i uses the triple dynamic distillation strategy to extract three parts of knowledge in the rth round of training, namely local and global distillation, local and local distillation, and global and global distillation;

[0021] Each client i performs local and global distillation and calculates the local soft prediction for the rth round and the globally aggregated soft prediction result s of round r r The KL divergence of is used to obtain the distillation loss, i.e. T r represents the distillation temperature of the rth round;

[0022] Each client i performs local and local distillation, and calculates the best soft prediction of the previous round 1≤b≤r-1 and the local soft prediction for round r The KL divergence of is used to obtain the distillation loss, i.e. If the model performance in round r is better than the previous optimal model performance, it will be replaced;

[0023] Each client i performs global and global distillation and calculates the average of the global soft predictions of the previous rounds 1≤b≤r-1 and the global aggregated soft prediction s for round r r The KL divergence of is used to obtain the distillation loss, i.e. And update the global soft prediction average;

[0024] Step 4.2: Use the four kinds of knowledge to update the local model; the loss function is composed of the loss with the public dataset labels Local to Global Distillation Loss Part-to-part distillation losses and global-to-global distillation loss Composition, that is Among them, α1, α2, α3, and α4 represent the weight hyperparameters of the corresponding loss values; represents the cross entropy loss function that measures the difference between the predicted results and the true labels on the private dataset;

[0025] Step 4.3: Each client i completes a local update based on the acquired knowledge by applying a gradient descent step: Where η2 represents the learning rate; then a new round of local model training is started; when the number of training rounds reaches the preset value, the personalized federated learning task is completed.

[0026] The technical solution adopted by the device of the present invention is: an efficient personalized federated learning device based on knowledge distillation, including a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the efficient personalized federated learning method based on knowledge distillation when executing the program.

[0027] The technical solution adopted by the storage medium of the present invention is: a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the efficient personalized federated learning method based on knowledge distillation is implemented.

[0028] The technical solution adopted by the product of the present invention is: a computer program product, including a computer program, characterized in that when the computer program is executed by a processor, the efficient personalized federated learning method based on knowledge distillation is implemented.

[0029] Compared with the prior art, the beneficial effects of the present invention include:

[0030] (1) The present invention alleviates the adverse effects of non-independent and identically distributed data by designing a client clustering method based on Euclidean distance.

[0031] (2) The present invention also proposes dynamic distillation temperature adjustment to adapt to the dynamic personalized federated learning process, and adopts a triple distillation strategy to provide rich multi-source knowledge. The present invention improves the accuracy of the model, accelerates the training of the model, and reduces the communication overhead. This provides a possible solution for the deployment of personalized federated learning based on knowledge distillation in practical scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The technical solution of this invention is further described below using embodiments and specific implementation methods. In addition, some drawings are also used in the process of describing the technical solution. For those skilled in the art, other drawings and the intention of the invention can also be obtained based on these drawings without paying creative labor.

[0033] Figure 1 is a schematic diagram of a system method according to an embodiment of the present invention;

[0034] Figure 2 is a schematic diagram of a method flow of an embodiment of the present invention;

[0035] Figure 3 It is a local model structure diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to facilitate ordinary technicians in the field to understand and implement the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0037] Personalized federated learning can collaboratively personalize models for multiple clients without sharing their private data. However, many personalized federated learning methods rely on server-side model parameter aggregation, which requires all models to have the same structure and size. A promising approach is to train heterogeneous models by leveraging knowledge distillation to transfer knowledge between models by exchanging soft predictions instead of model parameters. However, existing personalized federated learning solutions based on knowledge distillation are difficult to be applied in practice due to insufficient knowledge extraction, resulting in excessive accuracy loss, and huge computational and communication overheads.

[0038] Knowledge distillation has been widely used in personalized federated learning as a knowledge transfer method that only exchanges soft predictions and avoids the need to share model parameters. Knowledge distillation allows multiple clients to absorb knowledge predicted by other clients by comparing their outputs on a common dataset.

[0039] This paper provides a possible solution for the deployment of personalized federated learning based on knowledge distillation in practical scenarios. A client-side clustering method based on Euclidean distance is designed to mitigate the adverse effects of non-IID data. Then, dynamic distillation temperature adjustment is proposed to adapt to the dynamic personalized federated learning process. Finally, a triple distillation strategy is adopted to provide rich multi-source knowledge for model updating.

[0040] Please see Figure 1 and Figure 2 , This embodiment provides an accurate, efficient and personalized federated learning method based on knowledge distillation, involving entities including a server and N clients;

[0041] The method comprises the following steps:

[0042] Step 1: The server clusters clients with similar data distribution using a client clustering algorithm based on Euclidean distance;

[0043] In one embodiment, the specific implementation of step 1 includes the following sub-steps:

[0044] Step 1.1: Randomly select a client z∈[N];

[0045] Step 1.2: Calculate the Euclidean distance [E(R1,R z ),E(R2,R z ),…,E(R N ,R z )], where R i represents the data distribution of client i∈[N], E(R i ,R j 0 represents the Euclidean distance between the data distributions of client i and client j, and the result is uploaded to the server;

[0046] Step 1.3: The server clusters clients with similar data distribution based on the K-means clustering algorithm and broadcasts the results to all clients.

[0047] Step 2: Each client i uses its private dataset D i Train a local model and complete the local model performance evaluation on the public dataset D0 based on dynamic temperature adjustment. The evaluation result is used as the soft prediction result. Upload to the server, represents the soft prediction value of client i in round r;

[0048] In one embodiment, see Figure 3, the local model is a feedforward neural network with multiple layers of interconnected nodes. It includes an input layer, one or more hidden layers, and an output layer. In each layer, a group of nodes performs linear or nonlinear transformations on the input signal and passes the processed information to the next layer. Linear transformations use weight matrices to linearly combine the inputs, while nonlinear transformations apply nonlinear activation functions (such as ReLU, Sigmoid, Tanh, etc.) to the results. In this system, the activation layer uses the ReLU activation function. This structure enables deep neural networks to learn complex features and show excellent performance in specific fields such as computer vision, natural language processing, and speech recognition.

[0049] The hidden layer includes one or more convolutional layers, which includes performing dot product calculations between the convolution kernel and the input matrix of the layer, and its activation function is usually ReLU; when the convolution kernel slides along the input matrix of the layer, the convolution operation generates a feature map, which is then contributed to the input of the next layer; followed by other layers, such as pooling layers and fully connected layers;

[0050] The convolution layer can generate a set of parallel feature maps, which are composed by sliding different convolution kernels on the input image and performing certain operations; at each sliding position, an element-by-element product and sum operation is performed between the convolution kernel and the input image to project the information in the receptive field to an element in the feature map; the size of the convolution kernel is much smaller than the input image, and it acts on the input image in an overlapping or parallel manner. All elements in a feature map are calculated by a convolution kernel, that is, a feature map shares the same weight and bias term;

[0051] The ReLU layer uses the function f(x)=max(0,x) as the activation function of this layer; it can enhance the nonlinear characteristics of the entire neural network without changing the convolutional layer calculation itself;

[0052] The pooling layer is actually a nonlinear form of downsampling. Among them, the maximum pooling is the most common. It divides the input image into several rectangular areas and outputs the maximum value for each sub-area. The pooling layer will continuously reduce the spatial size of the data, so the number of parameters and the amount of calculation will also decrease. Through the pooling layer, we can reduce the sensitivity of the convolution layer to the edge. The pooling layer calculates the output on a pooling window each time, and then moves the pooling window according to the stride. The pooling layer used has a stride of 2 and a pooling window of 2×2. The two-dimensional maximum pooling layer, and then takes the maximum value of the 4 numbers in each block, which will reduce the amount of data by 75%;

[0053] The fully connected layer connects each node in one layer to each node in another layer. After several convolutional and maximum pooling layers, the prediction part of the neural network model is completed through the fully connected layer. The nodes in the fully connected layer are associated with all activations in the previous layer; their activations can be calculated as an affine transformation, that is, multiplied by a matrix and then added with an offset.

[0054] In one embodiment, the deep neural network model is a locally initialized model;

[0055] In one embodiment, the specific implementation of step 2 includes the following sub-steps:

[0056] Step 2.1: Client i∈[N] obtains a local private dataset D i Train the local model w i ;

[0057] calculate Where η1 represents the learning rate; in Represents the cross entropy loss function that measures the difference between the predicted results and the true labels on the private dataset; each client i∈[N] stores a size of n i Local private dataset x i represents the i-th data sample, y i ∈{1,2,…,C} represents x i The labels of the local datasets are obtained by combining all the local datasets D i Connections are formed, is the size of the global dataset;

[0058] Step 2.2: Client i∈[N] completes the model performance evaluation on the public dataset D0 and outputs the calculation results of the last layer of the model in Represents the calculation result of the last layer of the model in category C and calculates the soft prediction value Where T r represents the distillation temperature of the rth round, x represents the data sample collected from the public dataset D0; T r The calculation expression is as follows: in k1, k2, r0 are hyperparameters for dynamic temperature adjustment; T0 is the initial distillation temperature setting.

[0059] Step 3: The server aggregates local soft prediction results And distribute the aggregation results to all clients;

[0060] In one embodiment, the server obtains the soft prediction results uploaded by each client, and then completes global aggregation based on the client clustering results to calculate Where |v| represents the total number of clients in cluster v, n v represents the size of the dataset in cluster v; the global dataset D is obtained by dividing all the datasets of size n i The local dataset D i Connections are formed, is the size of the global dataset.

[0061] Step 4: Each client i uses the triple dynamic distillation strategy to complete the local model update and start a new round of local model training based on the global aggregation results;

[0062] In one implementation, the specific implementation of step 4 includes the following sub-steps:

[0063] Step 4.1: Each client i uses a triple dynamic distillation strategy to extract three parts of knowledge in the rth round of training, namely, local and global distillation, local and local distillation, and global and global distillation; specifically, explicit knowledge is extracted from the aggregated global soft prediction, and two parts of implicit knowledge are extracted from the previous local optimal soft prediction and the previous global average soft prediction;

[0064] Each client i performs local and global distillation and calculates the local soft prediction for the rth round and the globally aggregated soft prediction result s of round r r The KL divergence of is used to obtain the distillation loss, i.e. T r represents the distillation temperature of the rth round;

[0065] Each client i performs local and local distillation, and calculates the best soft prediction of the previous round 1≤b≤r-1 and the local soft prediction for round r The KL divergence of is used to obtain the distillation loss, i.e. If the model performance in round r is better than the previous optimal model performance, it will be replaced;

[0066] Each client i performs global and global distillation and calculates the average of the global soft predictions of the previous rounds 1≤b≤r-1 and the global aggregated soft prediction s for round r r The KL divergence of is used to obtain the distillation loss, i.e. And update the global soft prediction average;

[0067] Step 4.2: Use the four kinds of knowledge to update the local model; the loss function is composed of the loss with the public dataset labels Local to Global Distillation Loss Part-to-part distillation losses and global-to-global distillation loss Composition, that is Among them, α1, α2, α3, and α4 represent the weight hyperparameters of the corresponding loss values; represents the cross entropy loss function that measures the difference between the predicted results and the true labels on the private dataset;

[0068] Step 4.3: Each client i completes a local update based on the acquired knowledge by applying a gradient descent step: Where η2 represents the learning rate; then a new round of local model training is started; when the number of training rounds reaches the preset value, the personalized federated learning task is completed.

[0069] This embodiment also provides an efficient personalized federated learning device based on knowledge distillation, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the efficient personalized federated learning method based on knowledge distillation when executing the program.

[0070] This embodiment also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the efficient personalized federated learning method based on knowledge distillation is implemented.

[0071] This embodiment also provides a computer program product, including a computer program, which implements the efficient personalized federated learning method based on knowledge distillation when executed by a processor.

[0072] The present invention is further described through specific experiments. See Table 1 below, which shows the test accuracy (%) experimental results of the scheme of running to buy that and other schemes under two non-independent and identically distributed data conditions (β=0.1, π=2). Table 2 shows the training time (min) and communication overhead (GB) experimental results of the scheme of the present invention and other schemes under two non-independent and identically distributed data conditions (β=0.1, π=2).

[0073] Table 1

[0074]

[0075] Table 2

[0076]

[0077] This experiment compares this implementation scheme with other schemes in terms of test accuracy, training time and communication overhead. The experiment selected three data sets (CIFAR-10, CIFAR-100, CheXpert), and considered two extreme non-independent and identically distributed data cases, namely β=0.1 and π=2, where β represents the specific parameters of the data conforming to the Dirichlet distribution, and π represents the total number of categories of data owned by each client. Table 1 shows the test accuracy of different schemes under two non-independent and identically distributed data conditions. It is observed that the test accuracy of this implementation scheme is 6.50% to 56.93% higher than all other schemes. Table 2 shows the training time and communication overhead of different schemes under two non-independent and identically distributed data conditions. It is found that the training time of this implementation scheme is up to 2.50 times less than other schemes, and the communication overhead is reduced by up to 184.33 times. The above experiments show that this implementation scheme improves the accuracy of the model, accelerates the training of the model, and reduces the communication overhead.

[0078] This embodiment designs an accurate, efficient and personalized federated learning solution based on knowledge distillation. A client clustering method based on Euclidean distance is designed to mitigate the adverse effects of non-independent and identically distributed data. Then, dynamic distillation temperature adjustment is proposed to adapt to the dynamic personalized federated learning process. Finally, we adopt a triple distillation strategy to provide rich multi-source knowledge. Compared with previous solutions, the present invention improves the accuracy of the model, accelerates the training of the model, and reduces the communication overhead. This provides a possible solution for the deployment of personalized federated learning based on knowledge distillation in actual scenarios.

[0079] It should be understood that the embodiments described above are part of the embodiments of the present invention, rather than all of the embodiments. In addition, the technical features in the various embodiments or single embodiments provided by the present invention can be combined with each other arbitrarily to form a feasible technical solution. Such combination is not restricted by the sequence of steps and / or the structural composition mode, but must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the protection scope required by the present invention.

[0080] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of patent protection of the present invention. Under the enlightenment of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested for the present invention shall be based on the attached claims.

Claims

1. An efficient personalized federated learning method based on knowledge distillation, involving entities including a server and N clients; characterized in that: The following steps are involved: Step 1: The server clusters clients with similar data distribution using a client clustering algorithm based on Euclidean distance; Step 2: Each client i uses its private dataset D i Train a local model and complete the local model performance evaluation on the public dataset D0 based on dynamic temperature adjustment. The evaluation result is used as the soft prediction result. Upload to the server, represents the soft prediction value of client i in round r; Step 3: The server aggregates local soft prediction results And distribute the aggregation results to all clients; Step 4: Each client i uses the triple dynamic distillation strategy to complete the local model update and start a new round of local model training based on the global aggregation results.

2. The efficient personalized federated learning method based on knowledge distillation according to claim 1, characterized in that: The specific implementation of step 1 includes the following sub-steps: Step 1.1: Randomly select a client z∈[N]; Step 1.2: Calculate the Euclidean distance [E(R1,R z ),E(R2,R z ),…,E(R N ,R z )], where R i represents the data distribution of client i∈[N], E(R i ,R j ) represents the Euclidean distance between the data distributions of client i and client j, and uploads the result to the server; Step 1.3: The server clusters clients with similar data distribution based on the K-means clustering algorithm and broadcasts the results to all clients.

3. The efficient personalized federated learning method based on knowledge distillation according to claim 1 is characterized in that: The specific implementation of step 2 includes the following sub-steps: Step 2.1: Client i∈[N] obtains a local private dataset D i Train the local model w i ; calculate Where η1 represents the learning rate; in Represents the cross entropy loss function that measures the difference between the predicted results and the true labels on the private dataset; each client i∈[N] stores a size of n i Local private dataset x i represents the i-th data sample, y i ∈{1,2,…,C} represents x i labels, C represents the total number of categories in the dataset; the global dataset D is obtained by combining all local datasets D i Connections are formed, is the size of the global dataset; Step 2.2: Client i∈[N] completes the model performance evaluation on the public dataset D0 and outputs the calculation results of the last layer of the model in Represents the calculation result of the last layer of the model in category C and calculates the soft prediction value Where T r represents the distillation temperature of the rth round, x represents the data sample collected from the public dataset D0; T r The calculation expression is as follows: in k1, k2, r0 are hyperparameters for dynamic temperature adjustment; T0 is the initial distillation temperature setting.

4. The efficient personalized federated learning method based on knowledge distillation according to claim 1, characterized in that: In step 3, the server obtains the soft prediction results uploaded by each client, and then completes global aggregation based on the client clustering results to calculate Where |v| represents the total number of clients in cluster v, n v represents the size of the dataset in cluster v; the global dataset D is obtained by dividing all the datasets of size n i The local dataset D i Connections are formed, is the size of the global dataset.

5. The efficient personalized federated learning method based on knowledge distillation according to claim 1, characterized in that: The specific implementation of step 4 includes the following sub-steps: Step 4.1: Each client i uses the triple dynamic distillation strategy to extract three parts of knowledge in the rth round of training, namely local and global distillation, local and local distillation, and global and global distillation; Each client i performs local and global distillation and calculates the local soft prediction for the rth round and the globally aggregated soft prediction result s of round r r The KL divergence of is used to obtain the distillation loss, i.e. T r represents the distillation temperature of the rth round; Each client i performs local and local distillation, and calculates the best soft prediction of the previous round 1≤b≤r-1 and the local soft prediction for round r The KL divergence of is used to obtain the distillation loss, i.e. If the model performance in round r is better than the previous optimal model performance, it will be replaced; Each client i performs global and global distillation and calculates the average of the global soft predictions of the previous rounds 1≤b≤r-1 and the global aggregated soft prediction s for round r r The KL divergence of is used to obtain the distillation loss, i.e. And update the global soft prediction average; Step 4.2: Use the four kinds of knowledge to update the local model; the loss function is composed of the loss with the public dataset labels Local to Global Distillation Loss Part-to-part distillation losses and global-to-global distillation loss Composition, that is Among them, α1, α2, α3, and α4 represent the weight hyperparameters of the corresponding loss values; represents the cross entropy loss function that measures the difference between the predicted results and the true labels on the private dataset; Step 4.3: Each client i completes a local update based on the acquired knowledge by applying a gradient descent step: Where η2 represents the learning rate; then a new round of local model training is started; when the number of training rounds reaches the preset value, the personalized federated learning task is completed.

6. An efficient personalized federated learning device based on knowledge distillation, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the efficient personalized federated learning method based on knowledge distillation as described in any one of claims 1 to 5 is implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the efficient personalized federated learning method based on knowledge distillation as described in any one of claims 1 to 5 is implemented.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the efficient personalized federated learning method based on knowledge distillation as described in any one of claims 1 to 5 is implemented.

Images