Self-adaptive bilateral distillation personalized federal learning method based on diffusion model

Abstract

The invention belongs to the technical field of federated learning, and particularly relates to a self-adaptive bilateral distillation personalized federated learning method based on a diffusion model. According to the method, a guidance mechanism is designed, so that mutual distillation is carried out between a global model and a local model, efficient transmission of knowledge is realized, and the adaptability of a personalized model to specific data distribution of a client is enhanced. Meanwhile, a conditional diffusion model is introduced to generate high-quality pseudo data, and the aggregated global model is finely adjusted by using the pseudo data. The process not only effectively makes up global information possibly lost in the local-global mutual distillation process, but also further optimizes the performance of the global model. By combining the mutual distillation and conditional diffusion fine tuning technology, the data privacy is protected, meanwhile, the balance between the personalized performance and the global generalization ability is achieved, and the method is suitable for a multi-client cooperation scene in a non-independent identically-distributed non-IID data environment.

Description

Technical Field

[0001] The present invention belongs to the technical field of federated learning, and more specifically, relates to an adaptive bilateral distillation personalized federated learning method based on a diffusion model. Background Art

[0002] Federated learning is a distributed machine learning method, which was originally designed to solve the "data island" problem. However, with the continuous iteration of technology and the sharp increase in the number of users, a single generalized global model can no longer meet users' needs for personalization. In order to solve this problem, personalized federated learning came into being. Personalized federated learning optimizes the performance of local models based on the specific data distribution and task requirements of each user while sharing global knowledge. Compared with traditional federated learning, it can not only alleviate the problem of data heterogeneity, but also significantly improve the performance of local models on specific clients at the expense of very little global generalization ability.

[0003] As an efficient means of knowledge transfer, knowledge distillation is widely used in personalized federated learning. However, due to privacy restrictions, the server cannot directly access the client's local data, which makes generating pseudo data an effective solution. The currently more popular pseudo data generation technology is the Generative Adversarial Network (GAN). However, GAN requires adversarial training between the generator and the discriminator, and "mode collapse" may occur in this process, that is, the generated samples lack diversity and only cover a small number of similar modes. To overcome this problem, the Conditional Diffusion Models was proposed. This method generates pseudo data by gradually adding noise to the original data and learning the denoising process, which fundamentally solves the mode collapse problem of GAN, and the quality and diversity of the generated samples are significantly improved.

[0004] Most existing personalized federated learning methods tend to focus on only one aspect of model personalization and generalization ability, resulting in an imbalance between the two, and may even forget previously learned knowledge. For example, FedDistill achieves information exchange by transferring logits between clients and servers. Although this improves the generalization performance of the global model, it ignores the support for client personalization needs, and the transfer of logits may bring privacy leakage risks. FedAMP achieves adaptive model collaboration aggregation by calculating model similarity between clients. However, its focus is still on the optimization of the global model and lacks effective support for client personalization. FML takes into account the learning of personalized knowledge and global knowledge through mutual distillation of local and global models, but the global model may forget some global knowledge when training with local data, resulting in a decrease in generalization performance. FedProto assumes that there are shared class prototypes between clients, but when the client class distribution is completely different or non-overlapping, the global prototype may not accurately express the global features, resulting in limited generalization ability.

[0005] In order to solve the above problems, some improvement schemes have been proposed in recent years. For example, FedKD achieves a balance between personalization and generalization performance through adaptive mutual distillation of local models and global models. However, it may still forget some global knowledge, thus affecting the overall effect. Therefore, there is an urgent need for a method to achieve more efficient personalization while ensuring the generalization performance of the model as much as possible and avoiding forgetting knowledge.

[0006] Chinese patent document CN117035057A discloses a personalized federated learning method based on model and data distillation, the steps of which are as follows: construct a local model on the client, including a shared encoder and a private decoder. The client trains the local model based on the private data set and uploads the model parameters of the shared encoder to the server. The client calculates the output logits of the public data set based on the local model and uploads the logits to the server. The server performs weighted aggregation on the logits and the shared encoder model parameters of multiple clients to obtain global logits and multiple global encoder model parameters. Each client downloads multiple global encoder models, updates multiple local shared encoder model parameters in the client, downloads global logits, and participates in the training of the decoder in the form of knowledge distillation. However, this method has the following shortcomings: 1. This method relies on the global encoder and global logits for knowledge distillation, but the global model has limitations in capturing the personalized features of the client, especially when the data distribution is not independent and identically distributed, it is difficult to meet the personalized needs. 2. This method does not fully consider the "catastrophic forgetting" problem that may occur in the local update process of the global model, that is, the loss of previous knowledge.

[0007] In view of this, the present invention designs an adaptive bilateral distillation personalized federated learning method based on a diffusion model. Summary of the invention

[0008] The present invention aims to overcome at least one defect of the above-mentioned prior art, and provides an adaptive bilateral distillation personalized federated learning method based on a diffusion model. By designing a guidance mechanism, the global model and the local model are mutually distilled, thereby realizing efficient knowledge transfer and enhancing the adaptability of the personalized model to the client-specific data distribution. At the same time, a conditional diffusion model is introduced to generate high-quality pseudo data, and these pseudo data are used to fine-tune the aggregated global model. This process not only effectively compensates for the global information that may be lost in the local-global mutual distillation process, but also further optimizes the performance of the global model. By combining mutual distillation and conditional diffusion fine-tuning technology, the present invention achieves a balance between personalized performance and global generalization ability while protecting data privacy, and is suitable for multi-client collaboration scenarios in non-independent and identically distributed non-IID data environments.

[0009] The detailed technical scheme of the present invention is as follows: An adaptive bilateral distillation personalized federated learning method based on a diffusion model, the method comprising: S1. Server initializes the global model , broadcast the initial global model to each participating client; S2. The client receives the global model from the server, and uses local data to train the received global model and the local partial model to obtain local loss and global loss. Then, the client uses local data to distill the global model and the local partial model, and uses the local loss and global loss to guide the distillation process to obtain a local global model. At the same time, the client uses local data and category vectors to train the local conditional diffusion model to obtain a local local generator. S3, the client sends the adjusted local global model and local local generator to the server; S4. The server compares the received local global model with the initial global model according to the local data volume of the client. Aggregate to obtain the aggregated global model; then use KL divergence to calculate the similarity between the local global model and the aggregated global model; S5. Use similarity to perform weighted aggregation on the received local local generators to obtain a global generator that can generate global pseudo data; S6. The server uses the global pseudo data generated by the global generator to perform knowledge distillation on the aggregated global model and the historical global model to further optimize the global model. S7. The server rebroadcasts the optimized global model to each participating client and repeats the above steps S1-S6 until the preset rounds are reached to obtain the final fine-tuned global model.

[0010] Preferably, according to the present invention, in step S2, the client trains the received global model and the local local model using local data, and obtains the local loss and the global loss as follows: Local data They are sent to the local model and the global model for training respectively, and the cross entropy loss function is used to calculate their respective losses in the process: (1) (2) In formulas (1) and (2), Indicates local loss, represents the global loss, is the result predicted by the local model, is the result predicted by the global model, Is a real label, Represents the total number of categories in the client dataset.

[0011] Preferably, in step S2, the global model and the local local model are mutually distilled using local data, and the distillation process is guided by local loss and global loss to obtain the local global model as follows: The strength of mutual distillation is controlled according to the accuracy of the local model and the global model in predicting the data set. If the accuracy is higher, the loss is smaller, and the distillation strength is smaller. If the accuracy is lower, the loss is greater, and the distillation strength is greater, and more knowledge needs to be learned. In the process of mutual distillation, KL divergence is used to control the difference between the local model prediction distribution and the global model prediction distribution, and the purpose of knowledge transfer is achieved by minimizing the difference between the two. The process is as follows: (3) (4) In formulas (3) and (4), represents the difference between the local model and the global model, represents the difference between the global model and the local model, is the result predicted by the local model, is the result predicted by the global model, Indicates local loss, Represents the global loss, and the denominator achieves the purpose of controlling the strength of knowledge transfer through the sum of two cross entropy losses.

[0012] Preferably, according to the present invention, in step S2, the client uses local data and category vectors to train a local conditional diffusion model to obtain a local local generator as follows: First, use the conditional diffusion model to perform forward diffusion: convert the local data As the target data, by gradually adding noise to the original data, the original data gradually approaches pure noise. The formula is as follows: (5) In formula (5), In the diffusion model, the original data arrive The conditional probability distribution of represents the original data, Represents the original data The data after the noise addition time step t, represents a Gaussian distribution, To control the noise level, represents the cumulative noise attenuation coefficient, and I represents the unit matrix; through this process, the original data gradually becomes pure noise; Then, reverse diffusion is performed. In the reverse diffusion process, the category embedding vector c is introduced to control the generation of data that conforms to the distribution of the local data set; then, the variance and mean of the added noise learned in the forward diffusion process are used to gradually denoise to restore the original data. The formula is as follows: (6) (7) In equations (6) and (7), Indicates from Reverse generation of state samples at time t-1 The conditional probability distribution of , c is the category embedding vector, is the mean of the noise, is the variance of the noise, t represents the time step, Represents the conditional probability distribution The sample at time state t-1 is obtained by sampling ; Through the reverse diffusion process, the data is gradually restored to the target data that conforms to the target distribution; The optimization goal of the reverse optimization process is: (8) In formula (8), represents the mean square error loss function, is standard Gaussian noise, is the noise predicted by the model, Represents the time step t, the original data and noise The conditional diffusion model minimizes the noise in the model predictions. and the real noise added The distance between them makes the reverse generation process more accurate; through this optimization process, the conditional diffusion model can gradually learn the reverse diffusion process; At this point, the local generator training process is completed, and the local generator used to generate data that conforms to the client data distribution is obtained. .

[0013] Preferably, according to the present invention, step S4 is specifically as follows: The server performs global aggregation on the global model based on its local data volume. The formula is as follows: (9) In formula (9), It is The global model after round aggregation, N represents the total number of clients participating in the aggregation, represents the local data volume of client j, |D| represents the total data volume of all clients, represents the global model of client j in round i; After the aggregation is completed, the server needs to calculate the similarity between each local global model and the aggregated global model. Since the model parameters here are all high-dimensional vectors, cosine similarity needs to be used. The calculation process is as follows: (10) In formula (10), It is to calculate the preliminary similarity between the local global model and the aggregated global model. represents the global model parameter vector of client k in round i, represents the global model parameter vector after the i-th round of aggregation, represents the L2 norm, and Indicates that and Normalized to a unit vector; then Perform standardization, as shown in formula (11): (11) In formula (11), It is Round Client The similarity between the local global model and the aggregated global model, satisfy .

[0014] Preferably, according to the present invention, step S5 is specifically as follows: Based on the calculated similarity , weighted aggregation is performed on each local generator, and the calculation formula is shown in (12): (12) In formula (12), represents the global generator after the i-th round of aggregation, Representative Round Client The similarity between the local global model and the aggregated global model, Represents the local generator of client k in round i.

[0015] According to the preferred embodiment of the present invention, since the aggregated global model may forget the previously learned knowledge after local training, in order to deal with this situation, it is considered to use the pseudo data generated by the global generator and the previous The saved global model performs knowledge distillation and fine-tuning on the aggregated global model, which fundamentally avoids the problem of "catastrophic forgetting" of the global model.

[0016] First, use the global generator Generate global pseudo data , then select Previous The global model of the round is weighted aggregated to obtain the aggregated front Global model parameters of the wheel , the specific formula is shown in (13): (13) In formula (13), represents the average value of the global model parameters in the first m rounds, Represents the global model parameter vector of the ijth round, and m represents the global model parameters of the first m rounds. In order to avoid unnecessary computational costs, ,in, Represents the number of communication rounds between the client and the server; Then, according to the obtained and right Perform knowledge distillation: First, calculate the probability distribution of the student and teacher models used in the distillation process: (14) (15) In formulas (14) and (15), is the probability distribution of the teacher model after smoothing, is the probability distribution of the student model after smoothing, represents the output logits of the teacher model on the global pseudo data, represents the output logits of the student model on the global pseudo data, represents the distillation temperature; Next, calculate the KL divergence between the probability distribution of the student model and the probability distribution of the teacher model : (16) In formula (16), Used to offset the gradient scaling caused by smoothing of probability distribution; By minimizing the KL divergence, we update , thus achieving the effect of fine-tuning, the formula is shown in (17): (17) In formula (17), Represents the learning rate, which is used to control the step size of each update. Represents the loss function The gradient of Finally, the fine-tuned global model parameters are broadcast to each client, which is used to update the local global model parameters. The above process is iterated continuously until the global model converges.

[0017] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention uses the adaptive mutual distillation of local models and global models to make full use of local loss and global loss, and guides the global model to achieve a dynamic balance between generalization performance and personalized needs.

[0018] (2) The present invention uses the pseudo data generated by the global generator to fine-tune the aggregated global model, effectively compensating for the global knowledge that may be lost during the distillation process and preventing the global model from forgetting historical knowledge due to the introduction of new knowledge.

[0019] (3) Since the amount of data and data distribution of each client may be different, the present invention avoids the impact of data heterogeneity by learning a personalized model for each client. For the common non-independent and identically distributed data in federated learning, the introduction of the mutual distillation mechanism and conditional diffusion model of the present invention can better adapt to and handle the heterogeneity of the data of each client.

[0020] (4) The present invention uses global pseudo data for global knowledge distillation, avoiding the risk of privacy leakage caused by directly using private data. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flow chart of the personalized federated learning method described in the present invention.

[0022] Figure 2 It is a schematic diagram of the framework of the personalized federated learning method of the present invention.

[0023] Figure 3 This is a comparison chart of the test accuracy of the personalized federated learning algorithm of the present invention and other federated learning algorithms under CIFAR10 data in Example 1 of the present invention.

[0024] Figure 4 This is a comparison chart of the test accuracy of the personalized federated learning algorithm of the present invention and other federated learning algorithms under CIFAR100 data in Example 1 of the present invention. DETAILED DESCRIPTION

[0025] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.

[0026] Embodiment 1, Ginseng Figure 1 and Figure 2 This embodiment provides an adaptive bilateral distillation personalized federated learning method based on a diffusion model, the method comprising: S1. Server initializes the global model , broadcast the initial global model to each participating client; S2. The client receives the global model from the server, and uses local data to train the received global model and the local partial model to obtain local loss and global loss. Then, the client uses local data to distill the global model and the local partial model, and uses the local loss and global loss to guide the distillation process to obtain a local global model. At the same time, the client uses local data and category vectors to train the local conditional diffusion model to obtain a local local generator. S3, the client sends the adjusted local global model and local local generator to the server; S4. The server compares the received local global model with the initial global model according to the local data volume of the client. Aggregate to obtain the aggregated global model; then use KL divergence to calculate the similarity between the local global model and the aggregated global model; S5. Use similarity to perform weighted aggregation on the received local local generators to obtain a global generator that can generate global pseudo data; S6. The server uses the global pseudo data generated by the global generator to perform knowledge distillation on the aggregated global model and the historical global model to further optimize the global model. S7. The server rebroadcasts the optimized global model to each participating client and repeats the above steps S1-S6 until the preset rounds are reached to obtain the final fine-tuned global model.

[0027] In step S2, the client trains the received global model and the local local model using local data, and obtains the local loss and the global loss as follows: Local data They are sent to the local model and the global model for training respectively, and the cross entropy loss function is used to calculate their respective losses in the process: (1) (2) In formulas (1) and (2), Indicates local loss, represents the global loss, is the result predicted by the local model, is the result predicted by the global model, It’s a real label.

[0028] The local data is used to distill the global model and the local local model, and the local loss and the global loss are used to guide the distillation process, and the local global model is obtained as follows: The strength of mutual distillation is controlled according to the accuracy of the local model and the global model in predicting the data set. If the accuracy is higher, the loss is smaller, and the distillation strength is smaller. If the accuracy is lower, the loss is greater, and the distillation strength is greater, and more knowledge needs to be learned. In the process of mutual distillation, KL divergence is used to control the difference between the local model prediction distribution and the global model prediction distribution, and the purpose of knowledge transfer is achieved by minimizing the difference between the two. The process is as follows: (3) (4) In formulas (3) and (4), represents the difference between the local model and the global model, represents the difference between the global model and the local model, is the result predicted by the local model, is the result predicted by the global model, Indicates local loss, Represents the global loss, and the denominator achieves the purpose of controlling the strength of knowledge transfer through the sum of two cross entropy losses.

[0029] The client uses local data and category vectors to train the local conditional diffusion model to obtain the local local generator as follows: First, use the conditional diffusion model to perform forward diffusion: convert the local data As the target data, by gradually adding noise to the original data, the original data gradually approaches pure noise. The formula is as follows: (5) In formula (5), In the diffusion model, the original data arrive The conditional probability distribution of represents the original data, Represents the original data After the noise time step The data after that, represents a Gaussian distribution, To control the noise level, represents the cumulative noise attenuation coefficient, and I represents the unit matrix; through this process, the original data gradually becomes pure noise; Then, reverse diffusion is performed. In the reverse diffusion process, the category embedding vector c is introduced to control the generation of data that conforms to the distribution of the local data set; then, the variance and mean of the added noise learned in the forward diffusion process are used to gradually denoise to restore the original data. The formula is as follows: (6) (7) In equations (6) and (7), Indicates from Reverse generation of state samples at time t-1 The conditional probability distribution of , c is the category embedding vector, is the mean of the noise, is the variance of the noise, t represents the time step, Represents the conditional probability distribution The sample at time state t-1 is obtained by sampling ; Through the reverse diffusion process, the data is gradually restored to the target data that conforms to the target distribution; The optimization goal of the reverse optimization process is: (8) In formula (8), represents the mean square error loss function, is standard Gaussian noise, is the noise predicted by the model, Represents the time step t, the original data and noise The conditional diffusion model minimizes the noise in the model predictions. and the real noise added The distance between them makes the reverse generation process more accurate; through this optimization process, the conditional diffusion model can gradually learn the reverse diffusion process; At this point, the local generator training process is completed, and the local generator used to generate data that conforms to the client data distribution is obtained. .

[0030] Step S4 is specifically as follows: The server performs global aggregation on the global model based on its local data volume. The formula is as follows: (9) In formula (9), It is The global model after round aggregation, N represents the total number of clients participating in the aggregation, represents the local data volume of client j, |D| represents the total data volume of all clients, represents the global model of client j in round i; After the aggregation is completed, the server needs to calculate the similarity between each local global model and the aggregated global model. Since the model parameters here are all high-dimensional vectors, cosine similarity needs to be used. The calculation process is as follows: (10) In formula (10), It is to calculate the preliminary similarity between the local global model and the aggregated global model. represents the global model parameter vector of client k in round i, represents the global model parameter vector after the i-th round of aggregation, represents the L2 norm, and Indicates that and Normalized to a unit vector; then Perform standardization, as shown in formula (11): (11) In formula (11), It is Round Client The similarity between the local global model and the aggregated global model, satisfy .

[0031] Step S5 is specifically as follows: Based on the calculated similarity , weighted aggregation is performed on each local generator, and the calculation formula is shown in (12): (12) In formula (12), represents the global generator after the i-th round of aggregation, Representative Round Client The similarity between the local global model and the aggregated global model, Represents the local generator of client k in round i.

[0032] According to the preferred embodiment of the present invention, since the aggregated global model may forget the previously learned knowledge after local training, in order to deal with this situation, it is considered to use the pseudo data generated by the global generator and the previous The saved global model performs knowledge distillation and fine-tuning on the aggregated global model, which fundamentally avoids the problem of "catastrophic forgetting" of the global model.

[0033] First, use the global generator Generate global pseudo data , then select Previous The global model of the round is weighted aggregated to obtain the aggregated front Global model parameters of the wheel , the specific formula is as follows: (13) In formula (13), represents the average value of the global model parameters in the first m rounds, Represents the global model parameter vector of the ijth round, and m represents the global model parameters of the first m rounds. In order to avoid unnecessary computational costs, ,in, Represents the number of communication rounds between the client and the server; Then, according to the obtained and right Perform knowledge distillation: First, calculate the probability distribution of the student and teacher models used in the distillation process: (14) (15) In formulas (14) and (15), is the probability distribution of the teacher model after smoothing, is the probability distribution of the student model after smoothing, represents the output logits of the teacher model on the global pseudo data, represents the output logits of the student model on the global pseudo data, represents the distillation temperature; Next, calculate the KL divergence between the probability distribution of the student model and the probability distribution of the teacher model : (16) In formula (16), Used to offset the gradient scaling caused by smoothing of probability distribution; By minimizing the KL divergence, we update , so as to achieve the effect of fine-tuning, the formula is as follows: (17) In formula (17), Represents the learning rate, which is used to control the step size of each update. Represents the loss function The gradient of Finally, the fine-tuned global model parameters are broadcast to each client, which is used to update the local global model parameters. The above process is iterated continuously until the global model converges.

[0034] The optimization objectives of the present invention are as follows: The present invention selects Clients participate in the aggregation, and each client The local datasets owned by , where | |Indicates the number of data sets it owns. Global data sets Represents the collection of all client data sets. The main goal of the invention is to obtain a local model at the same time and a global model , its optimization goal is to minimize the total loss value on the global dataset and , their calculation formulas are as follows: (18) in,

[0035] (19) in,

[0036] Representative The local model loss of each client, is the cross entropy loss, which is used to measure the difference between the predicted value and the true value. Representative The global model loss of the client. Here and It is the weighted KL divergence of the local model prediction distribution and the global model prediction distribution.

[0037] The adaptive bilateral distillation personalized federated learning method based on the diffusion model provided by the present invention can be applied to the scenario of personalized blood sugar prediction and management of diabetic patients. High-quality simulation data is generated through the diffusion model to solve the problem of insufficient individual data, and a high-precision personalized prediction model is generated for each patient by combining global knowledge and local characteristics through the bilateral distillation mechanism. In practice, the patient's smart device collects blood sugar, exercise, diet and other data in real time, and protects data privacy through the federated learning framework to avoid uploading sensitive information. At the same time, the model can dynamically adapt according to the patient's living habits and changes in the condition, providing patients with real-time blood sugar trend predictions and personalized management suggestions, such as diet, exercise and drug adjustment plans.

[0038] Experimental example Next, we will conduct experimental verification. First, we will give a basic introduction to the experiment: (1) Datasets: CIFAR10, CIFAR100; (2) Model: CNN; (3) Data partitioning method: practical non-IID Dirichlet partitioning method; (4) Benchmark algorithms: FedAvg, FedProx, FML, DaFKD, FedKD, FedTweet; To ensure fairness, all benchmark algorithms and this algorithm use the same network architecture, equipment, and hyperparameter settings. Figure 3 and Figure 4 The experimental results are analyzed in detail.

[0039] for Figure 3 , This section compares the performance of the proposed personalized federated learning algorithm with other benchmark algorithms on the CIFAR10 dataset, and the experimental condition is the heterogeneity coefficient α=0.1. As can be seen from the figure, the accuracy of personalized algorithms (such as FML, DaFKD, FedKD, FedFD, FedTweet) is significantly higher than that of traditional federated learning algorithms FedAvg and FedProx. This shows that the effectiveness of the personalized federated learning scheme has been verified in the data heterogeneity scenario. At the same time, among various personalized federated learning algorithms, the algorithm FedFDM of the present invention has better accuracy after convergence than other algorithms, showing its significant performance advantage.

[0040] for Figure 4,The dataset is changed from CIFAR10 to CIFAR100, and other experimental configurations remain unchanged.,From the experimental results, it can be seen that FedFDM still performs significantly better than other algorithms in the case of,heterogeneous data, which further verifies its generalization ability and,applicability.

[0041] The above analysis shows that the algorithm FedFDM of the present invention has significant advantages in personalized federated learning scenarios with heterogeneous data, and its key modules play a vital role in improving the overall performance.

Claims

1. Adaptive bilateral distillation personalized federated learning method based on diffusion model, characterized by: The method comprises: S1. Server initializes the global model , broadcast the initial global model to each participating client; S2. The client receives the global model from the server, and uses local data to train the received global model and the local partial model to obtain local loss and global loss. Then, the client uses local data to distill the global model and the local partial model, and uses the local loss and global loss to guide the distillation process to obtain a local global model. At the same time, the client uses local data and category vectors to train the local conditional diffusion model to obtain a local local generator. S3, the client sends the adjusted local global model and local local generator to the server; S4. The server compares the received local global model with the initial global model according to the local data volume of the client. Aggregate to obtain the aggregated global model; then use KL divergence to calculate the similarity between the local global model and the aggregated global model; S5. Use similarity to perform weighted aggregation on the received local local generators to obtain a global generator that can generate global pseudo data; S6. The server uses the global pseudo data generated by the global generator to perform knowledge distillation on the aggregated global model and the historical global model to further optimize the global model. S7. The server rebroadcasts the optimized global model to each participating client and repeats the above steps S1-S6 until the preset rounds are reached to obtain the final fine-tuned global model.

2. The adaptive bilateral distillation personalized federated learning method based on the diffusion model according to claim 1, characterized in that: In step S2, the client trains the received global model and the local local model using local data, and obtains the local loss and the global loss as follows: Local data They are sent to the local model and the global model for training respectively, and the cross entropy loss function is used to calculate their respective losses in the process: （1） （2） In formulas (1) and (2), Indicates local loss, represents the global loss, is the result predicted by the local model, is the result predicted by the global model, Is a real label, Represents the total number of categories in the client dataset.

3. The adaptive bilateral distillation personalized federated learning method based on the diffusion model according to claim 1, characterized in that: In step S2, the global model and the local local model are mutually distilled using local data, and the local loss and the global loss are used to guide the distillation process, and the local global model is obtained as follows: The strength of mutual distillation is controlled according to the accuracy of the local model and the global model in predicting the data set. If the accuracy is higher, the loss is smaller, and the distillation strength is smaller. If the accuracy is lower, the loss is greater, and the distillation strength is greater. In the process of mutual distillation, KL divergence is used to control the difference between the local model prediction distribution and the global model prediction distribution, and the purpose of knowledge transfer is achieved by minimizing the difference between the two. The process is as follows: （3） （4） In formulas (3) and (4), represents the difference between the local model and the global model, represents the difference between the global model and the local model, is the result predicted by the local model, is the result predicted by the global model, Indicates local loss, represents the global loss.

4. The adaptive bilateral distillation personalized federated learning method based on the diffusion model according to claim 1, characterized in that: In step S2, the client uses local data and category vectors to train a local conditional diffusion model to obtain a local local generator as follows: First, use the conditional diffusion model to perform forward diffusion: transform the local data As the target data, by gradually adding noise to the original data, the original data gradually approaches pure noise. The formula is as follows: （5） In formula (5), In the diffusion model, the original data arrive The conditional probability distribution of represents the original data, Represents the original data The data after the noise addition time step t, represents a Gaussian distribution, To control the noise level, represents the cumulative noise attenuation coefficient, I represents the unit matrix; Then, reverse diffusion is performed. In the reverse diffusion process, the category embedding vector c is introduced to control the generation of data that conforms to the distribution of the local data set; then, the variance and mean of the added noise learned in the forward diffusion process are used to gradually denoise to restore the original data. The formula is as follows: （6） （7） In equations (6) and (7), Indicates from Reverse generation of state samples at time t-1 The conditional probability distribution of , c is the category embedding vector, is the mean of the noise, is the variance of the noise, t represents the time step, Represents the conditional probability distribution The sample at time state t-1 is obtained by sampling ; Through the reverse diffusion process, the data is gradually restored to the target data that conforms to the target distribution; The optimization goal of the reverse optimization process is: （8） In formula (8), represents the mean square error loss function, is standard Gaussian noise, is the noise predicted by the model, Represents the time step t, the original data and noise The expected value of ; through this optimization process, the conditional diffusion model can gradually learn the reverse diffusion process; At this point, the local generator training process is completed, and the local generator used to generate data that conforms to the client data distribution is obtained. .

5. The adaptive bilateral distillation personalized federated learning method based on the diffusion model according to claim 1, characterized in that: The step S4 is specifically as follows: The server performs global aggregation on the global model based on its local data volume. The formula is as follows: （9） In formula (9), It is The global model after round aggregation, N represents the total number of clients participating in the aggregation, represents the amount of local data of client j, Represents the total amount of data from all clients. represents the global model of client j in round i; After the aggregation is completed, the server uses cosine similarity to calculate the similarity between each local global model and the aggregated global model. The calculation process is as follows: （10） In formula (10), It is to calculate the preliminary similarity between the local global model and the aggregated global model. represents the global model parameter vector of client k in round i, represents the global model parameter vector after the i-th round of aggregation, and Indicates that and Normalized to unit vector; Then will Perform standardization, as shown in formula (11): （11） In formula (11), It is Round Client The similarity between the local global model and the aggregated global model, satisfy .

6. The adaptive bilateral distillation personalized federated learning method based on the diffusion model according to claim 1, characterized in that: The step S5 is specifically as follows: Based on the calculated similarity , weighted aggregation is performed on each local generator, and the calculation formula is shown in (12): （12） In formula (12), represents the global generator after the i-th round of aggregation, Representative Round Client The similarity between the local global model and the aggregated global model, Represents the local generator of client k in round i.

7. The adaptive bilateral distillation personalized federated learning method based on the diffusion model according to claim 5, characterized in that: Using the pseudo data generated by the global generator and the previous The global model saved in the round performs knowledge distillation fine-tuning on the aggregated global model: First, use the global generator Generate global pseudo data , then select Previous The global model of the round is weighted aggregated to obtain the aggregated front Global model parameters of the wheel , the specific formula is shown in (13): （13） In formula (13), represents the average value of the global model parameters in the first m rounds, represents the global model parameter vector of the ijth round, m represents the global model parameters of the first m rounds, and ,in, Represents the number of communication rounds between the client and the server; Then, according to the obtained and right Perform knowledge distillation: First, calculate the probability distribution of the student and teacher models used in the distillation process: （14） （15） In formulas (14) and (15), is the probability distribution of the teacher model after smoothing, is the probability distribution of the student model after smoothing, represents the output logits of the teacher model on the global pseudo data, represents the output logits of the student model on the global pseudo data, represents the distillation temperature; Next, calculate the KL divergence between the probability distribution of the student model and the probability distribution of the teacher model : （16） In formula (16), Used to offset the gradient scaling caused by smoothing of probability distribution; By minimizing the KL divergence, we update , thus achieving the effect of fine-tuning, the formula is shown in (17): （17） In formula (17), Represents the learning rate, which is used to control the step size of each update. Represents the loss function The gradient of Finally, the fine-tuned global model parameters are broadcast to each client, which is used to update the local global model parameters. The above process is iterated continuously until the global model converges.

Images