Federal learning member reasoning defense method based on knowledge distillation and contribution degree aggregation

Abstract

The invention discloses a federal learning member reasoning defense method based on knowledge distillation and contribution degree aggregation, and the method comprises the steps: carrying out the systematic fusion of an improved entropy regularized teacher model training, a student model distillation module based on the data synthesis of a conditional variation auto-encoder, and a dynamic aggregation method based on contribution degree perception; and the combined suppression of the multi-dimensional safety risk in the training link is realized. According to the method, the identifiability of the model to member data is remarkably reduced, the robustness to malicious updating is improved, the efficient learning performance of the federated model is kept while privacy protection is achieved, and therefore the dual requirements for safety and usability in practical application are met.

Description

Technical Field

[0001] This invention relates to the field of privacy protection technology in federated learning in machine learning, and in particular to a federated learning member inference defense method based on knowledge distillation and contribution aggregation. Background Technology

[0002] Federated learning is a technique for training models in a distributed, multi-party environment. By offloading the training process locally and only uploading model gradients or parameter updates to the server, it avoids centralized storage of raw data, thus helping to protect user privacy. Therefore, this technology has been widely applied in scenarios such as healthcare, finance, and personalized recommendations, becoming an important direction in privacy-preserving machine learning. However, federated learning cannot completely prevent privacy leaks. Research shows that the gradients and parameter updates generated during model training often contain rich data distribution information. Attackers can use this information to infer whether training data was used in training, thereby launching member inference attacks. These attacks exploit the difference in the model's prediction confidence between training samples and unseen samples to identify whether something belongs to the training set, threatening the sensitivity and confidentiality of the data of the participating parties.

[0003] In member inference attacks, attackers' strategies can be broadly categorized into two types: passive attacks and active attacks. Passive attackers do not interfere with the training process, but infer member identities solely by analyzing model output or model updates. Active attackers, on the other hand, manipulate the local training process, inject anomalous samples, or perturb model updates to amplify the differences in responses between members and non-members, significantly increasing the success rate of inference. As federated learning applications become increasingly complex, both types of attacks pose a serious threat to system privacy and security.

[0004] To combat member inference attacks, existing research has proposed defensive techniques such as differential privacy, regularization, model masking, and knowledge distillation. From a theoretical perspective, differential privacy provides strict privacy guarantees, but often leads to a significant decrease in model accuracy in order to meet noise intensity requirements. Regularization methods can mitigate overfitting and reduce the success rate of passive attacks, but their effectiveness is limited against active attacks. Model masking techniques typically rely on post-processing based on output probability distributions and cannot defend against white-box attacks. Traditional knowledge distillation methods, while reducing privacy leaks, depend on unlabeled data, which often lacks semantic representativeness, resulting in a significant decrease in student model accuracy and often failing to balance model performance. Furthermore, the distillation process itself cannot cope with intervention by active attackers during training; therefore, its protective capabilities are mainly concentrated in passive attack scenarios. Thus, how to effectively defend against both passive and active member inference attacks without significantly reducing model performance remains a critical technical challenge that needs to be overcome in the field of federated learning. Summary of the Invention

[0005] This invention addresses the problem that existing defense techniques for member inference attacks in federated learning cannot simultaneously cover both passive and active member inference attacks, and that it is difficult to strike a balance between model utility and privacy protection. It proposes a federated learning member inference defense method based on knowledge distillation and contribution aggregation, which can resist both passive and active member inference attacks while taking into account the privacy protection of federated learning and the model utility.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] Firstly, a federated learning member reasoning defense method based on knowledge distillation and contribution aggregation includes the following steps:

[0008] On each client, a local teacher model is trained using an improved entropy regularization teacher model training module, where the training loss function includes the standard task loss and an improved prediction entropy regularization term to increase the uncertainty of the model’s output on the training samples.

[0009] On each client, a synthetic data with category labels is generated using a conditional variational autoencoder, and a local student model is trained using a student model distillation module based on the synthetic data and the soft labels output by the local teacher model.

[0010] Each client will update and upload the trained local student model to the server;

[0011] On the server side, a contribution-aware dynamic aggregation module evaluates the contribution of model updates uploaded by each client to the performance of the global model, and dynamically allocates aggregation weights based on the contribution to update the global model.

[0012] Furthermore, in the improved entropy regularization teacher model training module, the total loss function is:

[0013]

[0014] in, For standard task losses, For the improved prediction entropy regularization term, This is the balance coefficient;

[0015] The improved prediction entropy regularization term The calculation formula is:

[0016]

[0017]

[0018] in, For the local training dataset of client i, Let x be the probability distribution predicted by the teacher model for sample x. For the real category The predicted probability, The predicted probability for the non-true category. This is the improved prediction entropy.

[0019] Furthermore, in the student model distillation module, synthetic data with category labels is generated using a conditional variational autoencoder, including:

[0020] The conditional variational autoencoder is trained, and its loss function is the evidence lower bound loss:

[0021]

[0022] in, The posterior approximate distribution obtained by the encoder, For decoder output, Let KL divergence be a metric.

[0023] Based on a trained conditional variational autoencoder, latent vectors are sampled from a standard normal distribution. The target label y is then input into the decoder to generate synthetic samples of the corresponding category. .

[0024] Furthermore, in the student model distillation module, the loss function for training the local student model... for:

[0025]

[0026] in, Let i be the output probability distribution of the student model for participant i. For the synthetic sample, Here are the corresponding labels for the synthesized samples, and M is the number of synthesized training samples;

[0027] Distillation loss is:

[0028]

[0029] in, For the teacher model, synthetic samples In temperature parameters The smooth output probability distribution is as follows. For balance coefficient, This represents the KL divergence between the output distributions of the teacher model and the student model.

[0030] Furthermore, in the contribution-aware dynamic aggregation module, the contribution of client i in communication round t is calculated. include:

[0031]

[0032] in, This represents the accuracy obtained by updating the model of client i and evaluating it on the server validation set; Represents the previous round of the global model Baseline accuracy on the validation set.

[0033] Furthermore, the dynamic allocation of aggregate weights based on contribution to update the global model includes the following strategies:

[0034] If the contribution of all clients If the value is greater than 0, then weighted aggregation is used:

[0035]

[0036] in, This represents the model update for client i, where N is the total number of clients;

[0037] If the contribution of some clients If the contribution is less than 0, then clients with negative contributions are removed, and the remaining clients are aggregated.

[0038]

[0039] If the contribution of all clients If the value is less than 0, then the top n clients with the lowest contribution are removed, and the remaining clients are weighted and aggregated.

[0040]

[0041] in, This represents the set of indices of the n clients with the lowest contribution.

[0042] The present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method as described in any of the preceding claims.

[0043] The present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method as described in any of the preceding claims.

[0044] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0045] This invention proposes a federated learning member inference defense method based on knowledge distillation and contribution aggregation. This method systematically integrates an improved entropy-regularized teacher model training, a student model distillation module based on conditional variational autoencoder synthesized data, and a contribution-aware dynamic aggregation method to jointly suppress multi-dimensional security risks during the training process. This approach not only significantly reduces the model's identifiability to member data and improves robustness against malicious updates, but also maintains the efficient learning performance of the federated model while protecting privacy, thus meeting the dual requirements of security and usability in practical applications. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0047] Figure 1 This is an architecture diagram of a federated learning member reasoning defense method based on knowledge distillation and contribution aggregation, provided in an embodiment of the present invention. Detailed Implementation

[0048] To better understand this technical solution, the method of the present invention will be described in detail below with reference to the accompanying drawings.

[0049] Federated learning, as a distributed training framework, superficially improves data privacy by sharing only model parameters or gradients without transmitting the original data. However, recent research has shown that model updates exchanged during training can still leak sensitive information about participants, especially in scenarios involving member inference attacks. Attackers can analyze model outputs or gradient changes to infer whether a piece of data was used for training, thus undermining the privacy protection goal of "data not leaving the local machine." With the widespread deployment of federated learning in healthcare, finance, recommendation systems, and other scenarios, effectively defending against member inference attacks has become a critical issue that urgently needs to be addressed.

[0050] Therefore, this invention proposes a method based on knowledge distillation and contribution aggregation to implement member reasoning defense in federated learning, with the architecture as follows: Figure 1 As shown, it mainly includes three core functional modules: a teacher model training module based on improved entropy regularization, a student model distillation module based on synthetic data from a conditional variational autoencoder (CVAE), and a contribution-aware dynamic aggregation module. These three modules collaborate sequentially around the federated learning training cycle, jointly forming a unified defense mechanism against passive and active member inference attacks.

[0051] Specifically, the federated learning member reasoning defense method based on knowledge distillation and contribution aggregation proposed in this invention includes the following steps:

[0052] First, in the federated learning system, a local teacher model is trained on each client using an improved entropy regularization teacher model training module. This module's training loss function includes the standard task loss and an improved prediction entropy regularization term to increase the model's output uncertainty on training samples, reducing overfitting to member samples from the outset. The output of this module is a teacher model used only locally and not uploaded, which is then used in the knowledge distillation process in subsequent modules.

[0053] Secondly, this invention deploys a student model distillation module based on synthetic data generated by a conditional variational autoencoder on each client. This module uses the conditional variational autoencoder to generate synthetic data with category labels, and trains the local student model jointly based on the synthetic data and the soft labels output by the local teacher model, achieving privacy-enhanced supervised knowledge distillation. This module avoids the direct propagation of real privacy data during training, while maintaining the accuracy of distillation training through a label conditional generation mechanism, thus achieving a good balance between privacy protection and model performance in the student model.

[0054] Each client uploads its trained local student model to the server.

[0055] Finally, on the server side, this invention employs a contribution-aware dynamic aggregation module to evaluate the contribution of each client's uploaded model updates to the global model performance, and dynamically allocates aggregation weights based on these contributions to update the global model. Unlike traditional average aggregation strategies, this module assigns dynamic aggregation weights to each client's student model updates based on their contribution to improving global performance. Clients with high contributions receive higher weights, while updates from clients with low contributions or exhibiting abnormal characteristics are weakened or removed, effectively suppressing malicious client attacks and ensuring the stability and robustness of the global model.

[0056] Through the collaborative work of the three modules mentioned above, this invention provides systematic privacy protection in several key stages of the federated learning training process: it weakens the sensitivity to member samples during the local training stage, avoids the leakage of real data during the distillation stage, and suppresses malicious update interference during the aggregation stage, thereby achieving joint defense against passive and active member inference attacks while maintaining the efficient learning performance of the global model.

[0057] The following section details the implementation process of each module.

[0058] 1. Improved teacher model training module with entropy regularization

[0059] The first stage of this invention is implemented by a teacher model training module based on improved entropy regularization within the client. Its core objective is to increase the model's predictive uncertainty for training samples during local training, thereby effectively reducing the model's overfitting to member data. In traditional model training processes, teacher models typically improve classification accuracy by minimizing standard cross-entropy loss. However, such training strategies aimed at maximizing accuracy can lead to highly deterministic outputs on training data, meaning that member samples often receive extremely high-confidence predictions. This "high-confidence—low-entropy" pattern is precisely the main feature exploited by most passive member inference attacks, resulting in member samples being significantly more distinguishable than non-member samples. Therefore, if relying solely on traditional training methods, the teacher model itself becomes a source of privacy leakage during the student model distillation stage.

[0060] To address the aforementioned issues, this invention introduces an improved prediction entropy regularization term during the teacher model training process and designs a new training loss function to jointly optimize task accuracy and prediction uncertainty. Specifically, the total loss function constructed in this module consists of two parts: standard task loss... With entropy regularization loss This combined loss is used to guide the teacher model to output a more balanced probability distribution on the training samples, preventing it from becoming overconfident in any particular class of samples. In the improved entropy regularization teacher model training module, the total loss function is:

[0061]

[0062] in, For standard task losses, For the improved prediction entropy regularization term, This is a balancing coefficient used to control the balance between task accuracy and privacy regularization, enabling the model to improve privacy protection capabilities without significantly reducing performance.

[0063] Standard methods for calculating prediction entropy rely solely on the probability distribution of the model's output to measure information content, but they have significant limitations: regardless of whether the model makes a highly certain correct or incorrect prediction for a particular category, it may exhibit the same zero entropy value. Therefore, this method cannot effectively distinguish the model's predictive behavior on member and non-member samples, making it unreliable as a measure of uncertainty. To overcome this deficiency, this invention employs an improved method for calculating prediction entropy, which simultaneously penalizes both overconfident correct and incorrect classifications, thus more fully reflecting the differences in the model's output across categories.

[0064] Specifically, given a sample x and its predicted probability distribution The improved prediction entropy is defined as:

[0065]

[0066] in, Represents the true category The predicted probability, This represents the predicted probability of the non-true category. This formula significantly improves the sensitivity of prediction entropy to uncertainty by jointly penalizing "overconfident correct categories" and "indistinguishability of incorrect categories," resulting in a smoother output distribution for member samples and thus reducing the advantage of passive reasoning attacks.

[0067] To apply the improved entropy to the entire local training set, this invention averages the improved entropy of all training samples to construct an entropy regularization term. The improved prediction entropy regularization term... The calculation formula is:

[0068]

[0069] in, This is the local training dataset for client i. This regularization term continuously increases the predictive uncertainty of the teacher model on the training data during training, significantly reducing the identifiable features of member samples. The teacher model trained through this module is not uploaded to the server; it is only used in the student model distillation stage within the client. This preserves the teacher model's professional expressive ability for the task while effectively preventing the high confidence characteristics of the teacher model from leaking member identity information, making the subsequent distillation process safer and more reliable.

[0070] 2. Student model distillation module based on conditional variational autoencoder for synthesized data

[0071] The second core step of this invention is implemented by a student model distillation module based on synthetic data from a conditional variational autoencoder. Its main purpose is to enable the student model to learn the predictive patterns of the teacher model without directly relying on private data, thereby improving privacy protection. Existing research typically uses unlabeled external data for unsupervised distillation training, but this approach often results in student models performing significantly worse than teacher models, or even exhibiting severe degradation. To overcome this problem, this invention introduces a conditional variational autoencoder to generate labeled synthetic samples with a distribution similar to that of private data, thus supporting supervised student model training.

[0072] In this invention, the conditional variational autoencoder achieves conditional distribution by simultaneously introducing class labels into both the encoder and decoder. The modeling ensures that the generated synthetic samples are semantically consistent with the target category. The training objective of the conditional variational autoencoder is to minimize its lower bound loss of evidence, which consists of two parts: a reconstruction loss, used to measure the similarity between the synthetic sample and the original input; and a KL divergence loss, used to regularize the latent space to a standard normal distribution. The overall loss function is:

[0073]

[0074] in, The posterior approximate distribution obtained by the encoder, For decoder output, The KL divergence between the posterior distribution and the standard normal distribution;

[0075] Through the above training, given any label y, the conditional variational autoencoder can generate synthetic samples of the corresponding category. The generation process is as follows: based on the trained conditional variational autoencoder, it first samples latent vectors from a standard normal distribution. The sampled vector and the embedded label y are then input into the decoder to generate synthetic samples for the corresponding category. .

[0076] The aforementioned mechanism ensures the consistency of generated samples in terms of category semantics, while the randomness of the latent space ensures sufficient diversity within the same category. The resulting synthetic data effectively captures the statistical structure of real-world data while avoiding the leakage of any specific training samples, fundamentally eliminating the risk of privacy breaches.

[0077] During the student model training phase, labeled synthetic data generated by a conditional variational autoencoder is used to replace the original private data for supervised learning. For each participant, the conditional variational autoencoder generates synthetic samples for all categories in the dataset. and corresponding tags The student model uses standard cross-entropy loss to complete basic classification training, and its loss function is defined as follows: :

[0078]

[0079] in, Let i be the output probability distribution of the student model for participant i. For the synthetic sample, Here are the corresponding labels for the synthesized samples, and M is the number of synthesized training samples.

[0080] To further improve the learning performance of the student model and ensure it retains as much useful knowledge as possible from the teacher model, this invention introduces a knowledge distillation loss based on soft labels during training. The student model is jointly trained using both the ground truth labels (hard labels) provided by the conditional variational autoencoder synthetic data and the soft prediction probability distribution of the teacher model on the same synthetic sample. The teacher model predicts based on temperature parameters... Smoothing process is performed to obtain Distillation loss is defined as:

[0081]

[0082] in, For the teacher model, synthetic samples In temperature parameters The smooth output probability distribution is as follows. For balance coefficient, This represents the KL divergence between the output distributions of the teacher model and the student model.

[0083] Through the above design, the labeled synthetic data generated by the conditional variational autoencoder is used for supervised training, completely replacing real private data and fundamentally reducing the risk of member inference attacks. Simultaneously, the soft-label distillation mechanism further promotes the student model's learning of the teacher model's predictive behavior, maximizing the preservation of model utility. Because the synthetic data closely approximates the class distribution and statistical characteristics of real data, the student model achieves a good balance between privacy protection and performance, thus maintaining high task accuracy while improving security.

[0084] 3. Contribution-aware dynamic aggregation module

[0085] The third core step of this invention is implemented by a contribution-aware dynamic aggregation module. This module is mainly used to perform quality assessment and selective aggregation of student model updates uploaded by each participant on the server side of the federated learning system. Its goal is to suppress proactive attacks from malicious clients, including data poisoning attacks, gradient manipulation attacks, and malicious updates that may lead to leakage of member inference, thereby improving the robustness and privacy security of the global model.

[0086] During federated learning training, malicious clients may manipulate local training data or upload forged model updates, causing the global model to shift in a direction favorable to the attack. This further amplifies the distinguishability between member and non-member samples, significantly increasing the success rate of member inference attacks. To address this, this invention proposes a contribution-aware aggregation strategy. This strategy calculates the contribution of updates provided by each participant in each round of communication and dynamically allocates aggregation weights based on the magnitude of the contribution, fundamentally reducing the negative impact of malicious updates on the global model.

[0087] In communication round t, the server first retains the global model parameters from the previous round. The server independently evaluates the local student model updates for each client. Specifically, the server temporarily applies the parameter updates uploaded by client i to the previous round of the global model and evaluates its accuracy on a reserved validation set to estimate the extent to which the client update improves the performance of the global model.

[0088] Calculate the contribution of client i in communication round t. Defined as:

[0089]

[0090] in, This represents the accuracy obtained by updating the model of client i and evaluating it on the server validation set; Represents the previous round of the global model Baseline accuracy on the validation set. Contribution. This reflects the net gain of the model performance from the client update and is the core criterion for dynamic aggregation in this invention.

[0091] After the contribution calculation is completed, the server uses three aggregation strategies to allocate aggregation weights to update the global model based on different contribution distributions:

[0092] (1) The case where all client contributions are positive

[0093] If the contribution of all clients >0, this invention employs a contribution-weighted aggregation method, giving higher weights to clients with greater interests in the model. The global model update rule is:

[0094]

[0095] in, This represents the model update for client i, where N is the total number of clients.

[0096] This strategy ensures that each participant's impact on the global model is proportional to their actual contribution, thereby improving training efficiency and model performance.

[0097] (2) Cases where some clients have negative contribution levels

[0098] If the contribution of some clients A value <0 indicates that updates from these clients will negatively impact the performance of the global model, potentially due to low-quality training or malicious attacks. In this case, this invention removes all client updates with negative contributions and aggregates only those with positive contributions. The update method is as follows:

[0099]

[0100] This aggregation method can effectively filter out potential malicious updates while maintaining the training contributions of normal clients, thereby avoiding a decline in global model performance.

[0101] (3) The case where all client contributions are negative.

[0102] If the contribution of all clients A value less than 0 indicates poor overall training quality, and all updates will degrade model performance. In this case, the invention designs a recovery strategy of "removing the lowest contributing clients." Specifically, the server removes the top n clients with the lowest contributions and re-weights and aggregates the remaining clients as a candidate set:

[0103]

[0104] in, This represents the set of indices of the n clients with the lowest contribution.

[0105] By eliminating the most unfavorable updates, this invention can maintain the stability of the global model even in completely unfavorable training epochs, avoiding drastic performance fluctuations.

[0106] In summary, the contribution-aware dynamic aggregation module can dynamically adjust its aggregation weights based on the actual quality of updates from each client. This not only effectively resists proactive attacks from malicious clients but also maintains the robustness of global model performance under adverse training conditions. This module, together with the teacher model privacy enhancement and student model security distillation in the previous two stages, forms a complete defense chain, jointly achieving systematic protection against member inference attacks in federated learning.

[0107] Furthermore, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method as described in any of the preceding claims.

[0108] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method as described in any of the preceding claims.

[0109] The key points and inventive aspects of this invention are summarized as follows:

[0110] (1) Teacher model training mechanism based on improved entropy regularization

[0111] This invention introduces an improved prediction entropy regularization mechanism during the local training phase of the teacher model, which is the first key innovation. Unlike traditional cross-entropy training methods, this invention employs an improved prediction entropy calculation formula. By simultaneously penalizing overconfident correct and incorrect predictions, it enables the model to more accurately represent the uncertainty on the training samples. This regularization term is added to the teacher model's loss function, allowing the teacher model to automatically suppress overfitting to member samples during training.

[0112] This mechanism effectively reduces the model's high-confidence output on the training data, significantly reducing the distinguishability between member and non-member samples, thus weakening the attack surface of passive member inference attacks in the early stages of the model's lifecycle. Simultaneously, because the improved entropy term is introduced as an adjustable coefficient in addition to the task loss, the teacher model can improve privacy protection while ensuring task accuracy, achieving an effective balance between performance and uncertainty.

[0113] (2) Supervised student model distillation method based on synthetic data of conditional variational autoencoder

[0114] The second key innovation lies in the student model distillation stage. This invention combines labeled synthetic data generated by a conditional variational autoencoder to construct a supervised distillation method that does not require real data. By simultaneously introducing class labels into both the encoder and decoder, the conditional variational autoencoder can learn the class conditional distribution of real data and generate high-quality synthetic samples that conform to class semantics, thus replacing real local data when training the student model.

[0115] Based on synthetic data, this invention combines a double distillation loss mechanism using both hard and soft labels. During distillation, the student model not only learns the true labels from the synthetic data from the conditional variational autoencoder but also utilizes the soft prediction results from the teacher model on the same synthetic data to achieve knowledge transfer. This mechanism avoids the performance degradation problem of traditional unlabeled distillation and completely eliminates the exposure path of real private data during training, structurally reducing the risk of member inference attacks.

[0116] (3) Dynamic aggregation strategy based on contribution perception

[0117] The third key innovation is a contribution-aware dynamic aggregation strategy deployed on the server side. This strategy independently evaluates the student model updates uploaded by each client in each round of federated learning communication, calculating the client's contribution score based on the validation set performance gain. The contribution score reflects the actual effect of the client's update on the global model performance and serves as the sole basis for subsequent aggregation weights.

[0118] This invention designs three types of aggregation logic based on the distribution of contribution. When all client contributions are positive, the system adopts a weighted aggregation method based on contribution size; when some client contributions are negative, these negative contributing clients are automatically removed, and only positive contributing clients are aggregated; in the extreme case where all client contributions are negative, the system will remove several clients with the lowest contribution to avoid low-quality or malicious updates from damaging the global model.

[0119] Through the aforementioned mechanism, this invention can effectively resist proactive attacks from malicious clients, such as gradient poisoning and model manipulation, while ensuring the stability and performance improvement of the global model in adversarial environments. This aggregation strategy constitutes a key guarantee for the entire defense framework, enabling the federated learning system to maintain robustness even in unstable participation environments.

[0120] Compared with the prior art, the advantages of the present invention are as follows:

[0121] This invention constructs a multi-layered security protection system consisting of teacher model training, student model distillation, and global aggregation to address the privacy leakage risks during federated learning training. It can effectively solve the problems of existing technologies that can only defend against a single type of attack and that it is difficult to achieve both model utility and privacy protection. It has the following significant advantages.

[0122] First, this invention is the first to simultaneously cover both passive and active member inference attacks within a unified framework. By introducing improved entropy regularization during the local training phase, the overfitting of the teacher model to the training data is effectively mitigated, fundamentally weakening the feasibility of passive inference attacks. Subsequently, by using a conditional variational autoencoder to generate labeled synthetic data to replace real privacy data for student model training, the exposure path of sensitive data during the distillation process is completely cut off. Furthermore, the contribution-aware dynamic aggregation mechanism proposed in this invention can automatically identify and suppress malicious or low-quality updates on the server side, effectively resisting active attackers from amplifying the risk of member leakage through gradient manipulation, model poisoning, and other methods. These three aspects complement each other, making this invention a method capable of systematically and jointly defending against multiple types of attacks throughout the federated learning process.

[0123] Secondly, this invention achieves an effective balance between model utility and privacy protection. The improved entropy regularization strategy enhances prediction uncertainty while maintaining the classification performance of the teacher model; the conditional variational autoencoder synthesizes data that statistically closely approximates the real data distribution, enabling the student model to learn in a supervised manner and avoiding the performance degradation caused by traditional unlabeled distillation; the dynamic aggregation strategy assigns higher weights to high-contribution clients, allowing the global model to maintain stable training convergence performance while enhancing privacy. Thanks to these technical designs, this invention not only significantly reduces the success rate of member inferences but also ensures that the accuracy and generalization ability of the federated learning model are not significantly affected, thus balancing privacy and model practicality.

[0124] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. However, these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A federated learning member reasoning defense method based on knowledge distillation and contribution aggregation, characterized in that, Includes the following steps: On each client, a local teacher model is trained using an improved entropy regularization teacher model training module, where the training loss function includes the standard task loss and an improved prediction entropy regularization term to increase the uncertainty of the model’s output on the training samples. On each client, a synthetic data with category labels is generated using a conditional variational autoencoder, and a local student model is trained using a student model distillation module based on the synthetic data and the soft labels output by the local teacher model. Each client will update and upload the trained local student model to the server; On the server side, a contribution-aware dynamic aggregation module evaluates the contribution of model updates uploaded by each client to the performance of the global model, and dynamically allocates aggregation weights based on the contribution to update the global model.

2. The federated learning member reasoning defense method based on knowledge distillation and contribution aggregation according to claim 1, characterized in that, In the improved entropy regularization teacher model training module, the total loss function is: ， in, For standard task losses, For the improved prediction entropy regularization term, This is the balance coefficient; The improved prediction entropy regularization term The calculation formula is: ， ， in, For the local training dataset of client i, Let x be the probability distribution predicted by the teacher model for sample x. For the real category The predicted probability, The predicted probability for the non-true category. This is the improved prediction entropy.

3. The federated learning member reasoning defense method based on knowledge distillation and contribution aggregation according to claim 1, characterized in that, In the student model distillation module, synthetic data with category labels is generated using a conditional variational autoencoder, including: The conditional variational autoencoder is trained, and its loss function is the evidence lower bound loss: ， in, The posterior approximate distribution obtained by the encoder, For decoder output, Let KL divergence be a metric. Based on a trained conditional variational autoencoder, latent vectors are sampled from a standard normal distribution. The target label y is then input into the decoder to generate synthetic samples of the corresponding category. .

4. The federated learning member reasoning defense method based on knowledge distillation and contribution aggregation according to claim 3, characterized in that, In the student model distillation module, the loss function for training the local student model is... for: ， in, Let i be the output probability distribution of the student model for participant i. For the synthetic sample, Here are the corresponding labels for the synthesized samples, and M is the number of synthesized training samples; Distillation loss is: ， in, For the teacher model, synthetic samples In temperature parameters The smooth output probability distribution is as follows. For balance coefficient, This represents the KL divergence between the output distributions of the teacher model and the student model.

5. The federated learning member reasoning defense method based on knowledge distillation and contribution aggregation according to claim 1, characterized in that, In the contribution-aware dynamic aggregation module, the contribution of client i in communication round t is calculated. include: ， in, This represents the accuracy obtained by updating the model of client i and evaluating it on the server validation set; Represents the previous round of the global model Baseline accuracy on the validation set.

6. The federated learning member reasoning defense method based on knowledge distillation and contribution aggregation according to claim 1, characterized in that, The method of dynamically allocating aggregate weights based on contribution to update the global model includes the following strategies: If the contribution of all clients If the value is greater than 0, then weighted aggregation is used: ， in, This represents the model update for client i, where N is the total number of clients; If the contribution of some clients If the contribution is less than 0, then clients with negative contributions are removed, and the remaining clients are aggregated. ， If the contribution of all clients If the value is less than 0, then the top n clients with the lowest contribution are removed, and the remaining clients are weighted and aggregated. ， in, This represents the set of indices of the n clients with the lowest contribution.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method as described in any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images