Variational autoencoder-enhanced federated learning system

Abstract

The application provides a kind of federated learning system based on variational autoencoder enhancement, with such features, including server and multiple clients, wherein, client includes local data storage module, client data transmission module, according to original data and first sampling feature vector, private classification model generation module, private VAE model generation module and private training module are obtained by training classification model and variational autoencoder, server includes server data transmission module, global classifier aggregation module, global decoder aggregation module, second sampling module and knowledge distillation module for obtaining this round global classifier and this round global decoder by knowledge distillation to initial global classifier and initial global decoder according to all second sampling feature vectors, this round private classifier and this round private decoder. In short, the method can protect privacy while having good model accuracy.

Description

Technical Field

[0001] This invention relates to the field of federated learning, and more specifically to a federated learning system based on variational autoencoder enhancement. Background Technology

[0002] Federated learning (FL) is a novel distributed machine learning paradigm. Its core concept is to ensure the privacy and security of client data through collaborative model training across multiple clients. In this model, data is generally kept local, with only necessary model update information shared with the server, theoretically significantly enhancing data privacy protection. Although FL is designed as a privacy-preserving distributed machine learning paradigm, it faces a serious challenge in practical applications: deep gradient leakage (DLG) attacks. DLG attacks are a new threat within federated learning architectures that utilize complex algorithms to reconstruct the original sensitive training data by analyzing the gradient information shared by clients during training. This attack not only reveals the potential vulnerability of FL in maintaining data privacy but also seriously threatens its application in real-world scenarios. Figure 1 This is a diagram illustrating a conventional federated learning architecture and DLG attacks, such as... Figure 1 As shown, each client i Train the model using local data and calculate its gradient with respect to the local data. This gradient is then sent to the Global Server, which aggregates these gradients to update the global model ω. g . Figure 1 The text also highlights a potential gradient leakage attack scenario. If the global server is malicious, it could lead to the reconstruction of real data, i.e., "cats" and "dogs," from the gradient information uploaded by the client.

[0003] To effectively defend against DLG attacks, researchers and practitioners in the field of FL (Flash Logic) have explored various privacy protection techniques, including homomorphic encryption (HE) and differential privacy. Homomorphic encryption, as a technique that ensures secure computation of data in an encrypted state, can theoretically provide strong security for FL. However, the application of homomorphic encryption often comes with a huge computational and communication burden, especially when dealing with large-scale deep neural network models, severely limiting its practicality. Differential privacy offers a more lightweight privacy protection strategy, protecting individual privacy and reducing the risk of data leakage by adding noise when data is published. However, the implementation of differential privacy often comes at the cost of sacrificing some model accuracy. Furthermore, research shows that the effectiveness of differential privacy strategies may be limited when facing complex data reconstruction attacks, therefore its defensive capabilities still need improvement in some situations.

[0004] Furthermore, traditional Lightroom (FL) privacy protection techniques involve trade-offs between computational efficiency, privacy protection, and algorithm accuracy. Employing advanced encryption techniques and privacy-preserving algorithms can significantly impact computational efficiency and model performance. To address these issues, FL's new approach focuses on splitting the neural network into private and public models, sharing only the latter to improve the balance between privacy and efficiency. However, this strategy still presents a trade-off between data privacy and model performance.

[0005] In summary, current technologies cannot effectively defend against DLG attacks while maintaining the computational efficiency and model accuracy of FL. Summary of the Invention

[0006] This invention is made to solve the above-mentioned problems, and aims to provide a federated learning system based on variational autoencoder enhancement.

[0007] This invention provides a federated learning system based on variational autoencoder (VAE) enhancement, characterized by: a server; and multiple clients, each communicating with the server. Each client includes: a local data storage module for storing the client's original data, a private feature extractor obtained from the previous round of federated training, and a private encoder obtained from the previous round of federated training; a client data transmission module for receiving the global classifier and global decoder obtained from the previous round of federated training, and sending the private classifier and private decoder obtained in the current round of federated training to the server; a private classification model generation module for combining the previous round's global classifier and private feature extractor to obtain a classification model; a private VAE model generation module for combining the previous round's global decoder and private encoder to obtain a variational autoencoder; a first sampling module for generating multiple first sampling feature vectors; and a private training module for training the system based on the original data. The classification model and variational autoencoder are trained using the first sampled feature vector to obtain a private classification model and a private variational autoencoder. The private classification model includes a private feature extractor and a private classifier obtained from this round of federated training. The private variational autoencoder includes a private encoder and a private decoder obtained from this round of federated training. The server includes: a server data transmission module, used to receive the private classifier and private decoder from each client and send the global classifier and global decoder obtained from this round of federated training to each client as the global classifier and global decoder for this round; a global classifier aggregation module, used to aggregate all private classifiers from this round to obtain an initial global classifier; a global decoder aggregation module, used to aggregate all private decoders from this round to obtain an initial global decoder; a second sampling module, used to generate multiple second sampled feature vectors; and a knowledge distillation module, used to perform knowledge distillation on the initial global classifier and initial global decoder based on all second sampled feature vectors, the private classifiers from this round, and the private decoders from this round to obtain the global classifier and global decoder for this round.

[0008] The federated learning system based on variational autoencoder enhancement provided by this invention may also have the following features: the private training module includes: a classification loss calculation unit for calculating the classification loss of the classification model based on the original data; a reconstruction loss calculation unit for calculating the reconstruction loss of the variational autoencoder based on the original data, the first sampled feature vector, and the feature extractor of the classification model; a local loss calculation unit for calculating the local loss based on the classification loss and the reconstruction loss; and an update unit for updating the classification model and the variational autoencoder based on the classification loss, the reconstruction loss, and the local loss to obtain a private classification model and a private variational autoencoder.

[0009] The federated learning system based on variational autoencoder enhancement provided by this invention may also have the following feature: wherein the expression for calculating the local loss is: In the formula L VAE For localized losses, For reconstruction loss, -D KL (q(z|x)||p(z)) is the classification loss, p(x|z) is the distribution of the decoder of the variational autoencoder, q(z|x) is the distribution of the encoder of the variational autoencoder, and D KL Let p(z) be the Kullback-Leibler divergence, and p(z) be the feature probability distribution function of the original data extracted by the feature extractor of the classification model.

[0010] The federated learning system based on variational autoencoder enhancement provided by this invention may also have the following features: The calculation of reconstruction loss in the reconstruction loss calculation unit includes the following steps: Step S1, inputting the original data into the feature extractor of the classification model to obtain a first feature vector; Step S2, inputting the feature vector into the encoder of the variational autoencoder to obtain a second feature vector; Step S3, inputting the second feature vector into the decoder of the variational autoencoder to obtain a first output result; Step S4, inputting the first sampled feature vector into the encoder of the variational autoencoder to obtain a third feature vector; Step S5, inputting the third feature vector into the decoder of the variational autoencoder to obtain a second output result; Step S6, calculating the reconstruction loss based on the first and second output results.

[0011] The federated learning system based on variational autoencoder enhancement provided by this invention may also have the following feature: In the global classifier aggregation module, the expression for obtaining the initial global classifier through aggregation is: In the formula θ global Let K be the initial global classifier, K be the number of all private classifiers in this round, and f be the initial global classifier. k (z k ) is the k-th private classifier in this round.

[0012] The federated learning system based on variational autoencoder enhancement provided by this invention may also have the following features: In the knowledge distillation module, the specific process of knowledge distillation includes the following steps: Step T1, combining each current-round private classifier and its corresponding current-round private decoder into a corresponding private model; Step T2, selecting one second-sampled feature vector from all second-sampled feature vectors as a training feature vector; Step T3, inputting the training feature vector into each private model to obtain the corresponding logical value; Step T4, aggregating all logical values ​​to obtain the client model logical value; Step T5, inputting the training feature vector into the global model composed of the initial global classifier and the initial global decoder to obtain the global model logical value; Step T6, calculating the loss based on the client model logical value and the global model logical value to obtain the loss L. distill Step T7, based on the loss L distill Update the initial global classifier and the initial global decoder; in step T8, repeat steps T2 to T7 until the preset training termination condition is met, and then obtain the current round global classifier and the current round global decoder.

[0013] The federated learning system based on variational autoencoder enhancement provided in this invention may also have the following feature: wherein the loss L distill The expression is: In the formula, T is the temperature parameter, and z global z is the global model logical value. local This is the logical value for the client-side model.

[0014] In the federated learning system based on variational autoencoder enhancement provided by the present invention, it may also have the following feature: wherein the first sampled feature vector and the second sampled feature vector are Gaussian noise feature vectors of the same size.

[0015] The role and effect of invention

[0016] According to the federated learning system based on variational autoencoder enhancement of this invention, on the one hand, a private variational autoencoder capable of converting raw data into a privacy-hardened latent space representation is trained simultaneously with the private classification model through a private training module, and the private classifier and private encoder are used as uploaded data, thereby avoiding data leakage caused by DLG attacks; on the other hand, the global classifier and global encoder are optimized through a knowledge distillation module, reducing aggregation overhead while ensuring performance. Therefore, the federated learning system based on variational autoencoder enhancement of this invention can protect privacy while achieving good model accuracy. Attached Figure Description

[0017] Figure 1This is a schematic diagram of a conventional federated learning architecture and a DLG attack.

[0018] Figure 2 This is a block diagram of the federated learning system in an embodiment of the present invention;

[0019] Figure 3 This is a block diagram of the client in an embodiment of the present invention;

[0020] Figure 4 This is a schematic diagram of the process by which the reconstruction loss calculation unit calculates the reconstruction loss in an embodiment of the present invention;

[0021] Figure 5 This is a block diagram of the server in an embodiment of the present invention;

[0022] Figure 6 This is a schematic diagram of the knowledge distillation process in an embodiment of the present invention;

[0023] Figure 7 This is a schematic diagram of the process of using a federated learning system for federated training in an embodiment of the present invention. Detailed Implementation

[0024] To make the technical means, creative features, objectives and effects of this invention easy to understand, the following embodiments, in conjunction with the accompanying drawings, provide a detailed description of the federated learning system based on variational autoencoder enhancement.

[0025] This embodiment provides a federated learning system based on variational autoencoder enhancement, applied to a face recognition scenario. It is used by multiple institutions to train a global face recognition model based on their respective customer image data. This face recognition model is used for customer identity verification and authentication.

[0026] Figure 2 This is a block diagram of the federated learning system in an embodiment of the present invention.

[0027] like Figure 2 As shown, the federated learning system 100 includes multiple clients 10 and a server 20 that is communicatively connected to each client 10. In this embodiment, each institution holds a corresponding client 10.

[0028] Figure 3 This is a block diagram of the client in an embodiment of the present invention.

[0029] like Figure 3 As shown, the client 10 includes a local data storage module 101, a client data transmission module 102, a private classification model generation module 103, a private VAE model generation module 104, a first sampling module 105, a private training module 106, and a client control module 107 that controls the above modules.

[0030] The local data storage module 101 is used to store the original data of the client 10, the private feature extractor obtained from the previous round of federated training as the previous round of private feature extractor, and the private encoder obtained from the previous round of federated training as the previous round of private encoder.

[0031] The client data transmission module 102 is used to receive the global classifier obtained from the previous round of federated training as the previous round of global classifier and the global decoder obtained from the previous round of federated training as the previous round of global decoder, and send the private classifier obtained from the current round of federated training as the current round of private classifier and the private decoder obtained from the current round of federated training as the current round of private decoder to the server 20.

[0032] The private classification model generation module 103 is used to combine the previous round's global classifier and the previous round's private feature extractor to obtain a classification model. In this embodiment, each client 10 combines the global classifier obtained in the last round of federated training with the private feature extractor to obtain a classification model, which is the global face recognition model obtained by that client 10.

[0033] The private VAE model generation module 104 is used to combine the previous round global decoder and the previous round private encoder to obtain a variational autoencoder.

[0034] The first sampling module 105 is used to generate multiple first sampling feature vectors.

[0035] The private training module 106 is used to train the classification model and the variational autoencoder based on the original data and the first sampled feature vector to obtain a private classification model and a private variational autoencoder. The private classification model includes the private feature extractor and the private classifier obtained in this round of federated training, and the private variational autoencoder includes the private encoder and the private decoder obtained in this round of federated training.

[0036] The private training module 106 includes a classification loss calculation unit 1061, a reconstruction loss calculation unit 1062, a local loss calculation unit 1063, and an update unit 1064.

[0037] The classification loss calculation unit 1061 is used to calculate the classification loss of the classification model based on the original data.

[0038] The reconstruction loss calculation unit 1062 is used to calculate the reconstruction loss of the variational autoencoder based on the original data, the first sampled feature vector, and the feature extractor of the classification model.

[0039] Figure 4 This is a schematic diagram of the process by which the reconstruction loss calculation unit calculates the reconstruction loss in an embodiment of the present invention.

[0040] like Figure 4 As shown, calculating the reconstruction loss in the reconstruction loss calculation unit includes the following steps:

[0041] Step S1: Input the raw data into the feature extractor of the classification model to obtain the first feature vector.

[0042] Step S2: Input the feature vector into the encoder of the variational autoencoder to obtain the second feature vector.

[0043] Step S3: Input the second feature vector into the decoder of the variational autoencoder to obtain the first output result.

[0044] Step S4: Input the first sampled feature vector into the encoder of the variational autoencoder to obtain the third feature vector.

[0045] Step S5: Input the third feature vector into the decoder of the variational autoencoder to obtain the second output result.

[0046] Step S6: Calculate the reconstruction loss based on the first output result and the second output result.

[0047] The local loss calculation unit 1063 is used to calculate the local loss based on the classification loss and reconstruction loss.

[0048] The expression for calculating the local loss is as follows:

[0049]

[0050] In the formula L VAE For localized losses, For reconstruction loss, -D KL (q(z|x)||p(z)) is the classification loss, p(x|z) is the distribution of the decoder of the variational autoencoder, q(z|x) is the distribution of the encoder of the variational autoencoder, and D KL Let p(z) be the Kullback-Leibler divergence, and p(z) be the feature probability distribution function of the original data extracted by the feature extractor of the classification model.

[0051] The update unit 1064 is used to update the classification model and variational autoencoder based on the classification loss, reconstruction loss and local loss, to obtain a private classification model and a private variational autoencoder.

[0052] In the private training module 106, the specific training process of the private classification model and the private variational autoencoder is as follows: First, the classification model is trained using the original data. The update unit 1064 updates the classification model according to the classification loss calculated by the classification loss calculation unit 1061 to obtain a preliminary classification model. Next, the private variational autoencoder is trained according to the feature extractor of the preliminary classification model, combined with the original data and the first sampled feature vector. The update unit 1064 updates the private variational autoencoder according to the reconstruction loss calculated by the reconstruction loss calculation unit 1062 to obtain a preliminary private variational autoencoder. Finally, the preliminary private variational autoencoder and the preliminary classification model are trained together according to the original data and the first sampled feature vector. The update unit 1064 updates the preliminary private variational autoencoder and the preliminary classification model together by minimizing the local loss obtained by the classification loss calculation unit 1061, the reconstruction loss calculation unit 1062 and the local loss calculation unit 1063 to obtain the trained private classification model and the private variational autoencoder.

[0053] The client control module 107 stores the control program that controls the operation of each module of the client 10.

[0054] Figure 5 This is a block diagram of the server side in an embodiment of the present invention.

[0055] like Figure 5 As shown, the server 20 includes a server data transmission module 201, a global classifier aggregation module 202, a global decoder aggregation module 203, a second sampling module 204, and a knowledge distillation module 205.

[0056] The server-side data transmission module 201 is used to receive the private classifier and private decoder of the current round sent by each client 10, and send the global classifier obtained from the current round of federated training as the global classifier of the current round and the global decoder obtained from the current round of federated training as the global decoder of the current round to each client 10.

[0057] The global classifier aggregation module 202 is used to aggregate all private classifiers in this round to obtain an initial global classifier.

[0058] In the global classifier aggregation module 202, the expression for obtaining the initial global classifier through aggregation is as follows:

[0059]

[0060] In the formula θ global Let K be the initial global classifier, K be the number of all private classifiers in this round, and f be the initial global classifier. k (z k ) is the k-th private classifier in this round.

[0061] The global decoder aggregation module 203 is used to aggregate all the private decoders in this round to obtain the initial global decoder.

[0062] The second sampling module 204 is used to generate multiple second sampling feature vectors. Among them, the first sampling feature vector and the second sampling feature vector are Gaussian noise feature vectors of the same size.

[0063] The knowledge distillation module 205 is used to perform knowledge distillation on the initial global classifier and the initial global decoder based on all second sampled feature vectors, the current round private classifier and the current round private decoder, to obtain the current round global classifier and the current round global decoder.

[0064] Figure 6 This is a schematic diagram of the knowledge distillation process in an embodiment of the present invention.

[0065] like Figure 6 As shown, in the knowledge distillation module 205, the specific process of knowledge distillation includes the following steps:

[0066] Step T1: Combine each current round private classifier and its corresponding current round private decoder into a corresponding private model.

[0067] Step T2: Select one second sampling feature vector from all the second sampling feature vectors as the training feature vector.

[0068] Step T3: Input the training feature vectors into each private model to obtain the corresponding logical values.

[0069] Step T4: Aggregate all logical values ​​to obtain the client model logical values.

[0070] Step T5: Input the training feature vector into the global model composed of the initial global classifier and the initial global decoder to obtain the global model logical value.

[0071] Step T6: Calculate the loss based on the client-side model logic value and the global model logic value to obtain the loss L. distill .

[0072] Among them, the loss L distill The expression is:

[0073]

[0074] In the formula, T is the temperature parameter, and z global z is the global model logical value. local This is the logical value for the client-side model.

[0075] Step T7, based on the loss L distill Update the initial global classifier and the initial global decoder.

[0076] Step T8: Repeat steps T2 to T7 until the preset training termination condition is met, and then the global classifier and global decoder for this round are obtained.

[0077] The following, with reference to the accompanying diagram, explains the process of obtaining a global face recognition model using a federated learning system 100.

[0078] Figure 7 This is a schematic diagram of the process of using a federated learning system for federated training in an embodiment of the present invention.

[0079] like Figure 7 As shown, the process of obtaining a global face recognition model through federated training using a federated learning system 100 includes the following steps:

[0080] In step R1, the server 20 constructs an initial classification model and an initial variational autoencoder, and sends them to each client 10 through the server data transmission module 201. The initial classification model includes an initial classifier as the previous round global classifier and an initial feature extractor as the previous round private feature extractor. The initial variational autoencoder includes an initial encoder as the previous round private encoder and an initial decoder as the previous round global encoder.

[0081] In step R2, on each client 10, the corresponding private classification model generation module 103 combines the previous round global classifier and the previous round private feature extractor to obtain a classification model.

[0082] In step R3, on each client 10, the corresponding private VAE model generation module 104 combines the previous round global decoder and the previous round private encoder to obtain a variational autoencoder.

[0083] In step R4, the first sampling module 105 of each client 10 generates multiple first sampling feature vectors.

[0084] In step R5, on each client 10, the corresponding private training module 106 trains the classification model and variational autoencoder based on the original data stored in the local data storage module 101 and the first sampled feature vector, thereby obtaining the private classification model and the private variational autoencoder.

[0085] In step R6, each client 10 sends the private classifier obtained in this round of federated training as the private classifier of this round and the private decoder obtained in this round of federated training as the private decoder of this round to the server 20 through the corresponding client data transmission module 102.

[0086] In step R7, each client 10 stores the private feature extractor obtained in this round of federated training as the private feature extractor of the previous round and the private encoder obtained in this round of federated training as the private encoder of the previous round together in the local data storage module 101.

[0087] In step R8, the server 20 receives the current round private classifier and the current round private decoder sent by each client 10 through the server data transmission module 201.

[0088] Step R9: The global classifier aggregation module 202 aggregates all private classifiers from this round to obtain the initial global classifier.

[0089] Step R10: The global decoder aggregation module 203 aggregates all private decoders of this round to obtain the initial global decoder.

[0090] In step R11, the second sampling module 204 generates multiple second sampling feature vectors.

[0091] In step R12, the knowledge distillation module 205 performs knowledge distillation on the initial global classifier and the initial global decoder based on all the second sampled feature vectors, the current round private classifier, and the current round private decoder, to obtain the current round global classifier and the current round global decoder.

[0092] In step R13, the server-side data transmission module 201 sends the current round global classifier as the previous round global classifier and the current round global decoder as the previous round global decoder to each client 10.

[0093] In step R14, repeat steps R2 to R13 until the preset training termination condition is met. Then, each client 10 will combine the global classifier received in the last round with the stored private feature extractor to form the global face recognition model corresponding to that client 10.

[0094] Compared to traditional federated learning methods, this embodiment does not upload the gradient information trained by each client. Instead, each client trains a private variational autoencoder capable of converting the original data into a privacy-hardened latent space representation using a private training module 106 while simultaneously training its own classification model. The private classifier and encoder are then uploaded to the server. This approach avoids DLG attacks from inferring client privacy data (i.e., the original data) by aggregating gradient information, effectively preventing the leakage of sensitive data attributes. Furthermore, it allows the global classifier to benefit from the diverse knowledge of the client datasets, thus achieving a new balance between data privacy and model performance.

[0095] To verify that the federated learning system 100 of this embodiment, i.e., the proposed method, has better performance than existing methods, the model accuracy of this method is compared with existing local methods, fedavg methods, fedprox methods, feddf methods, fedsplit methods, and fedgen methods on existing Fmnist datasets, CIFAR10 datasets, digit5 datasets, Office datasets, and DomainNet datasets, respectively. The specific data is shown in the table below:

[0096]

[0097]

[0098] The first column of the table above lists the various methods. Columns two through seven show the accuracy of the models trained by each method on the FMNIST dataset with 4 clients, the CIFAR-10 dataset with 4 clients, the CIFAR-10 dataset with 8 clients, the Digit5 dataset with 4 clients, the Office dataset with 4 clients, and the DomainNet dataset with 4 clients, respectively. This shows that our proposed method achieves the best model training results on the FMNIST, Office, and DomainNet datasets, and also achieves good model accuracy on the CIFAR-10 and Digit5 datasets.

[0099] In this embodiment, the performance of our method is further compared with that of the existing fedavg method and its introduction of differential privacy (DP) strategy on three datasets: CIFAR10, digit5, and Office. The specific calculation results of two metrics, model accuracy and peak signal-to-noise ratio (PSNR), are shown in the table below:

[0100]

[0101] The first column of the table above represents the various datasets, the second column represents the various metrics, and the third to sixth columns represent the calculation results of the fedavg method, the fedavg method with differential privacy DP:0.001, the fedavg method with differential privacy DP:0.1, and the proposed method for each metric on each dataset, respectively. In this embodiment, the lower the peak signal-to-noise ratio (PSNR), the better the privacy protection performance of the method. On the CIFAR10 dataset, the proposed method has better privacy protection performance than the fedavg method. Compared to the fedavg method with DP, although the privacy protection performance is slightly inferior at higher DP levels (DP:0.1), it has a more significant advantage in model accuracy. On the digit5 and office datasets with heterogeneous data sources, the proposed method outperforms both the fedavg method with and without DP in terms of model accuracy and privacy protection. Therefore, the proposed method can adapt to data heterogeneity, maintaining good model performance while protecting privacy, thus achieving a good balance between privacy protection and performance preservation.

[0102] The role and effect of the embodiments

[0103] According to the federated learning system based on variational autoencoder enhancement involved in this embodiment, on the one hand, a private training module trains a private variational autoencoder that can transform the original data into a privacy-hardened latent space representation while training a private classification model. The private classifier and encoder are then used as uploaded data, thus avoiding data leakage caused by DLG attacks. On the other hand, a knowledge distillation module optimizes the global classifier and global encoder, reducing aggregation overhead while ensuring performance. In summary, this method can protect privacy while achieving good model accuracy.

[0104] Those skilled in the art should understand that this invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to this invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A federated learning system based on variational autoencoder enhancement, applied to face recognition scenarios, for multiple institutions to train a global face recognition model based on their respective customer image data. This face recognition model is used for customer identification and authentication. Its features include: include: Server-side; Multiple clients, each communicating with the server, The client includes: The local data storage module is used to store the client's original data, the private feature extractor obtained from the previous round of federated training as the previous round of private feature extractor, and the private encoder obtained from the previous round of federated training as the previous round of private encoder. The client data transmission module is used to receive the global classifier obtained from the previous round of federated training as the previous round global classifier and the global decoder obtained from the previous round of federated training as the previous round global decoder, and send the private classifier obtained from the current round of federated training as the current round private classifier and the private decoder obtained from the current round of federated training as the current round private decoder to the server. A private classification model generation module is used to combine the previous round global classifier and the previous round private feature extractor to obtain a classification model; A private VAE model generation module is used to combine the previous round global decoder and the previous round private encoder to obtain a variational autoencoder; The first sampling module is used to generate multiple first sampling feature vectors; A private training module is used to train the classification model and the variational autoencoder based on the original data and the first sampled feature vector, to obtain a private classification model and a private variational autoencoder. The private classification model includes a private feature extractor and a private classifier obtained in this round of federated training, and the private variational autoencoder includes a private encoder and a private decoder obtained in this round of federated training. The server includes: The server-side data transmission module is used to receive the current round private classifier and the current round private decoder sent by each of the clients, and send the current round federated training global classifier as the current round global classifier and the current round federated training global decoder as the current round global decoder to each of the clients. The global classifier aggregation module is used to aggregate all the private classifiers mentioned in this round to obtain an initial global classifier. The global decoder aggregation module is used to aggregate all the private decoders mentioned in this round to obtain the initial global decoder; The second sampling module is used to generate multiple second sampling feature vectors; The knowledge distillation module is used to perform knowledge distillation on the initial global classifier and the initial global decoder based on all the second sampled feature vectors, the current round private classifier, and the current round private decoder, to obtain the current round global classifier and the current round global decoder; Specifically, in the knowledge distillation module, the knowledge distillation process includes the following steps: Step T1: Combine each of the current round private classifiers and the corresponding current round private decoders into a corresponding private model; Step T2: Select one second sampled feature vector from all the second sampled feature vectors as the training feature vector; Step T3: Input the training feature vectors into each of the private models respectively to obtain the corresponding logical values; Step T4: Aggregate all the logical values ​​to obtain the client model logical values; Step T5: Input the training feature vector into the global model composed of the initial global classifier and the initial global decoder to obtain the global model logical value; Step T6: Calculate the loss based on the client model logic value and the global model logic value to obtain the loss L_distill; Step T7: Update the initial global classifier and the initial global decoder according to the loss L_distill; Step T8: Repeat steps T2 to T7 until the preset training termination condition is met, and then the current round global classifier and the current round global decoder are obtained.

2. The federated learning system based on variational autoencoder enhancement as described in claim 1, Its features are: The private training module includes: A classification loss calculation unit is used to calculate the classification loss of the classification model based on the original data; The reconstruction loss calculation unit is used to calculate the reconstruction loss of the variational autoencoder based on the original data, the first sampled feature vector, and the feature extractor of the classification model. A local loss calculation unit is used to calculate the local loss based on the classification loss and the reconstruction loss; The update unit is used to update the classification model and the variational autoencoder based on the classification loss, the reconstruction loss, and the local loss, to obtain the private classification model and the private variational autoencoder.

3. The federated learning system based on variational autoencoder enhancement according to claim 2, characterized in that: in, The expression for calculating the local loss is: ， In the formula For the local loss, For the reconstruction loss, For the classification loss, The distribution of the decoder for the variational autoencoder, The encoder distribution of the variational autoencoder, For Kullback-Leibler divergence, The feature probability distribution function of the original data extracted by the feature extractor of the classification model.

4. The federated learning system based on variational autoencoder enhancement according to claim 2, Its features are: The calculation of the reconstruction loss in the reconstruction loss calculation unit includes the following steps: Step S1: Input the raw data into the feature extractor of the classification model to obtain the first feature vector; Step S2: Input the first feature vector into the encoder of the variational autoencoder to obtain the second feature vector; Step S3: Input the second feature vector into the decoder of the variational autoencoder to obtain the first output result; Step S4: Input the first sampled feature vector into the encoder of the variational autoencoder to obtain the third feature vector; Step S5: Input the third feature vector into the decoder of the variational autoencoder to obtain the second output result; Step S6: Calculate the reconstruction loss based on the first output result and the second output result.

5. The federated learning system based on variational autoencoder enhancement according to claim 1, characterized in that: in, In the global classifier aggregation module, the expression for the initial global classifier obtained through aggregation is: ， In the formula For the initial global classifier, The number of all private classifiers mentioned in this round. For the first The private classifier described in this round.

6. The federated learning system based on variational autoencoder enhancement according to claim 1, characterized in that: in, The loss The expression is: ， In the formula For temperature parameters, For the global model logical value, For the client model logical value, D KL The divergence is Kullback-Leibler.

7. The federated learning system based on variational autoencoder enhancement according to claim 1, characterized in that: in, The first sampled feature vector and the second sampled feature vector are Gaussian noise feature vectors of the same size.

Images