Federal learning backdoor attack defense method based on dynamic Bayesian belief update and deep reinforcement learning belief
By employing a federated learning approach that combines dynamic Bayesian belief updates and deep reinforcement learning beliefs, the problem of identifying and suppressing backdoor attacks in federated learning is solved, thereby improving model robustness and training efficiency without sacrificing model accuracy and privacy protection.
Patent Information
- Application Number
- CN202511681330.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-17
- Publication Date
- 2026-02-10
AI Technical Summary
Existing backdoor attack defense methods in federated learning suffer from disadvantages in terms of information asymmetry and computational resource requirements, making it difficult to effectively identify and suppress malicious client attacks, while potentially sacrificing model accuracy and privacy protection.
We employ a dynamic Bayesian belief update and deep reinforcement learning belief approach. By fitting the client model update distribution with a Gaussian mixture model and combining Bayesian full probability inference and a reinforcement learning policy network, we dynamically estimate the malicious probability and adjust the aggregation weights to suppress malicious updates.
Without altering the privacy assumptions, the model dynamically identifies and suppresses malicious updates, improving its robustness and training efficiency while reducing the demand for computational and communication resources.
Smart Images
Figure CN121508971A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of federated learning technology, specifically to a method for defending against backdoor attacks in federated learning based on dynamic Bayesian belief updates and deep reinforcement learning beliefs. Background Technology
[0002] Federated learning, a training paradigm proposed by Google in 2016, involves clients collaboratively training a global model under the coordination of a central server, without sharing their local private data. While it excels in protecting user privacy and utilizing distributed computing resources, the diverse sources of clients also introduce security risks, including backdoor attacks. Backdoor attacks involve modifying local data labels or injecting specific triggering patterns during training to construct a backdoor model on the client. This causes the model to misclassify triggered samples during testing, while maintaining its original performance on normal samples. Due to the information asymmetry between the server and clients, the server cannot directly observe whether a client is malicious. Some malicious clients can easily infiltrate the federated learning training process, injecting backdoor data into their training data to construct malicious gradient updates, which are then uploaded to the server. On the server side, this malicious gradient is aggregated with gradients trained by other clients. After multiple iterations, the final model will also exhibit backdoor characteristics. Researchers have proposed many defense methods. Some are data-independent methods that rely solely on model parameters to determine malicious updates or suppress their spread. These methods improve efficiency and avoid data dependence through lightweight design, reducing computational overhead and privacy risks. However, due to limited information, they have significant disadvantages in attack tolerance and dynamic defense capabilities. Another type of method relies on auxiliary verification data to improve defense capabilities, which can effectively enhance robustness in extreme malicious environments. However, the assumptions of these methods weaken the privacy protection boundary, and the methods have high requirements for computational or communication resources, limiting their practical deployment. Summary of the Invention
[0003] To address the aforementioned shortcomings in existing technologies, this invention provides a federated learning backdoor attack defense method based on dynamic Bayesian belief updates and deep reinforcement learning beliefs.
[0004] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows:
[0005] A federated learning backdoor attack defense method based on dynamic Bayesian belief updates and deep reinforcement learning beliefs includes the following steps:
[0006] Retrieve model update data uploaded by all clients;
[0007] Based on the model update data uploaded by all clients, a Gaussian mixture model is used to fit the model update distribution of all clients;
[0008] Based on the model update distribution of all clients, a recursive update mechanism based on Bayesian full probability inference is used to update beliefs and obtain the posterior estimate of the malicious probability of all clients.
[0009] The malicious probability of all clients is used as the state input to the server reinforcement learning policy network to learn the optimal action combination for selecting aggregate weights, thus obtaining the optimal aggregate weights for all clients.
[0010] The global model is updated based on the optimal aggregate weights across all clients and then distributed to all clients.
[0011] Furthermore, fitting the model update distribution for all clients using a Gaussian mixture model includes:
[0012] The optimal number of clusters is automatically determined using the Bayesian information criterion.
[0013] Furthermore, a recursive update mechanism based on Bayesian full probability inference is employed for belief updates to obtain posterior estimates of the malicious probability for all clients, including:
[0014] Using the model update data uploaded by the client as observations, we modeled each client to obtain the distribution of benign model scores and the distribution of malicious model scores;
[0015] A recursive update mechanism based on Bayesian full probability inference is used, combining the current score and historical beliefs, to calculate the malicious probability of each client, specifically:
[0016]
[0017] in, Let π be the probability of malicious intent of client i in round t, and let π be a one-dimensional Gaussian density function. Let μ be the observation score for client i in round t. B σ is the mean of a benign model. B μ represents the standard deviation of a benign model. M Let σ be the mean of the malicious model. M Let d be the standard deviation of the malicious model, and d be the server.
[0018] Furthermore, the server reinforcement learning policy network, which inputs the malicious probability of all clients as state, includes:
[0019] Construct the server's state space using the estimated probability of malice for all clients in the current round;
[0020] The server's action space is constructed using the aggregate weights of all clients in the current round;
[0021] The server's reward function is constructed using a negative logarithmic penalty against the probability of malice, specifically:
[0022]
[0023] in, Let α be the reward for server d in round t, and α be the scaling factor. Let be the estimated probability of malicious activity for client i in round t. Let be the aggregate weight of client i in round t.
[0024] Furthermore, the server-side reinforcement learning policy network employs the following loss function:
[0025]
[0026] in, Let be the loss function for server d, and φ be the network parameters of the server's reinforcement learning policy. Let d be the target value for server d in round t. Let Q be the Q value of server d in round t. Let be the state of server d in round t. For the action of server d in round t;
[0027] And an ε-greedy strategy is used for action updates, specifically:
[0028]
[0029] in, Let Q be the Q value of server d under the target network parameter φ.
[0030] Furthermore, the client constructs a set of benign models and a set of malicious models based on the global model;
[0031] Based on the benign model set and the malicious model set, a Gaussian mixture model is used to fit the distribution of benign models and the distribution of malicious models.
[0032] Based on the distribution of benign and malicious models, a recursive update mechanism based on Bayesian full probability inference is used to estimate the posterior estimate of the probability that the client model update is suspected.
[0033] The client-side model updates the probability of being suspected, which is used as the state input to the client-side reinforcement learning policy network to learn to select the optimal attack strength.
[0034] A malicious model is trained based on the client's optimal attack strength, and the updated model data is uploaded to the server.
[0035] Furthermore, the client constructs a set of benign models and a set of malicious models based on the global model, including:
[0036] The client constructs a benign model set based on local data and gradually introduces the observation scores from the real training process after formal communication begins. The benign model set is dynamically updated in a sliding window manner, and the currently received global model is added to the benign model set.
[0037] Under the same settings, backdoor samples are injected into local data to construct a malicious model set, and then the malicious model set is appended with additional training malicious updates in each round after formal communication begins.
[0038] Furthermore, a recursive update mechanism based on Bayesian full probability inference is used to estimate the posterior estimate of the probability of the client model update being doubted, specifically as follows:
[0039]
[0040] in, Let f be the probability that client a's model update is doubted in round t. M f is the probability density function of the score of the malicious model. B Let s be the probability density function of the score of the benign model. t The structural consistency score of the local model in round t.
[0041] Furthermore, the client-side reinforcement learning policy network, which uses the client-side model's update of the suspected probability as state input, includes:
[0042] The client's state space is constructed based on the belief that the current round of malicious clients has a history of suspicion.
[0043] The client's action space is constructed based on the proportion of backdoors injected into local data by the malicious client in the current round;
[0044] The client's reward function is constructed using a dynamic trade-off mechanism between attack success rate and the probability of being suspected, specifically as follows:
[0045]
[0046] in, For client a's reward in round t, ASR t Let be the attack success rate in round t, κ be the weight of the attack success rate, and τ be the weight of the probability of being suspected.
[0047] Furthermore, the client-side reinforcement learning policy network employs the following loss function:
[0048]
[0049] in, Let θ be the loss function for client a, and θ be the network parameters of the client's reinforcement learning policy. Let be the target value for client a in round t. Let Q be the Q value of client a in round t. Let's consider the state of client a in round t. For client a, the action in round t;
[0050] And an ε-greedy strategy is used for action updates, specifically:
[0051]
[0052] in, Let Q be the Q value of client a under the target network parameters θ.
[0053] The present invention has the following beneficial effects:
[0054] This invention introduces game theory modeling between attackers and defenders to characterize the incomplete information interaction between the attacker and the server. It applies the idea of dynamic Bayesian game theory to backdoor attacks and defenses in federated learning. The attacker aims to minimize the risk of detection and maximize the effectiveness of the backdoor, while the server strives to identify and suppress potential malicious updates without sacrificing model accuracy. This allows for dynamic inference of malicious behavior and adjustment of aggregation strategies without changing the privacy assumptions of federated learning, thereby suppressing the attack behavior of malicious clients and making the training efficiency close to data-independent. Attached Figure Description
[0055] Figure 1 This is a schematic diagram of a federated learning backdoor attack defense architecture based on dynamic Bayesian belief updates and deep reinforcement learning beliefs.
[0056] Figure 2 This is a flowchart illustrating a federated learning backdoor attack defense method based on dynamic Bayesian belief updates and deep reinforcement learning beliefs.
[0057] Figure 3 This is a diagram illustrating the malicious client attack process. Detailed Implementation
[0058] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.
[0059] This invention proposes a dynamic defense method for federated learning that integrates Bayesian game theory, belief updating, and reinforcement learning mechanisms. Without altering the privacy assumptions of federated learning, it dynamically infers malicious behavior and adjusts the aggregation strategy to suppress malicious client attacks, achieving training efficiency close to that of a data-independent method. We introduce game modeling between the attacker and defender to characterize the incomplete information interaction between the attacker and the server. The idea of dynamic Bayesian game theory is applied to backdoor attacks and defenses in federated learning. The attacker aims to minimize the risk of detection and maximize the effectiveness of the backdoor, while the server strives to identify and suppress potential malicious updates without sacrificing model accuracy.
[0060] like Figure 1 As shown. For the server (defender), the workflow is as follows: 1) The server fits the client model update distribution and calculates the probability density of benign and malicious model updates; 2) The server continuously adjusts the posterior estimate of the client's malicious probability through a Bayesian update mechanism; 3) The client's malicious probability is used as the state input to the reinforcement learning policy network to learn the optimal action combination for selecting aggregation weights, so as to minimize the model perturbation caused by aggregation and suppress potential backdoor effects.
[0061] For attackers, the workflow is as follows: 1) Fit a benign model distribution using a global model and calculate the probability that the current model update is judged as benign; 2) Continuously adjust the posterior estimate of the suspicion of the model update through a Bayesian update mechanism; 3) Use the suspicion probability as the state input to a reinforcement learning policy network to learn and select the optimal attack strength.
[0062] like Figure 2 As shown, an embodiment of the present invention provides a federated learning backdoor attack defense method based on dynamic Bayesian belief update and deep reinforcement learning belief, comprising the following steps A1 to A5:
[0063] A1. Obtain model update data uploaded by all clients;
[0064] A2. Based on the model update data uploaded by all clients, use a Gaussian mixture model to fit the model update distribution of all clients;
[0065] In an optional embodiment of the present invention, after dimensionality reduction of the high-dimensional update through Principal Component Analysis (PCA) in step S2, the distribution of client updates can be well approximated by a Gaussian Mixture Model (GMM). Malicious client updates exhibit higher cosine similarity in the PCA feature space due to sharing the same backdoor target, and cluster together; while benign client updates are more dispersed, forming diversity. This stable clustering difference provides a reliable basis for malicious client detection. Considering that GMM can adapt to different covariance structures and has high flexibility, this embodiment chooses GMM as the clustering method. Meanwhile, since the update distribution of benign clients may have large dispersion, a fixed number of clusters may not fit the data well. Therefore, this embodiment uses the Bayesian Information Criterion (BIC) to automatically determine the optimal number of clusters. The BIC is calculated as: BIC = -2 ln(ζ) + k ln(v), where ζ is the maximum likelihood estimate of the model, k is the number of model parameters, and v is the number of samples. A smaller BIC value indicates a better model fit.
[0066] A3. Based on the model update distribution of all clients, a recursive update mechanism based on Bayesian full probability inference is used to update beliefs and obtain the posterior estimate of the malicious probability of all clients.
[0067] In an optional embodiment of the present invention, step S3 employs a recursive update mechanism based on Bayesian full probability inference to update beliefs and obtain a posterior estimate of the malicious probability for all clients, including:
[0068] Update data based on model uploaded by the client As an observation, modeling each client after the GMM yields parameters for two types of latent distributions, namely the benign model score distribution. and malicious model score distribution And has μ B >μ M .
[0069] set up Let i be the trust value of client i in round t. A recursive update mechanism based on Bayesian full probability reasoning is used to combine the current score and historical beliefs. Calculate the probability of malicious activity for each client, specifically as follows:
[0070]
[0071] in, Let π be the probability of malicious intent of client i in round t, and let π be a one-dimensional Gaussian density function. Let μ be the observation score for client i in round t. B σ is the mean of a benign model. Bμ represents the standard deviation of a benign model. M Let σ be the mean of the malicious model. M Let be the standard deviation of the malicious model, and d be the server. The larger the p-value, the higher the probability of the client being malicious.
[0072] A4. Input the malicious probability of all clients into the server reinforcement learning policy network as the state input, learn the optimal action combination to select the aggregate weight, and obtain the optimal aggregate weight for all clients.
[0073] In an optional embodiment of the present invention, the objective of step S4 is to dynamically adjust the model aggregation weights based on the malicious belief in each client, thereby enhancing robustness and suppressing backdoor injection. The server's state space, action space, and reward function designed in this embodiment are as follows:
[0074] State space: This describes the defender's estimate of the probability of malicious activity for all participating clients in the current round, denoted as a vector.
[0075] Action space: Describe the defender's actions in the current round, with each action as an aggregated weight vector. Ω={0.00, 0.05, 0.10,…,1.00}, This represents the aggregate weight of client i in round t.
[0076] Reward function: Describes the negative logarithmic penalty for the probability of malice. in This represents the estimated probability of the defender's malicious intent towards client i in round t. This represents the aggregate weight of client i in round t.
[0077] The update formula for the learning state-action value function Q(s, a) of the server-side reinforcement learning policy network is:
[0078]
[0079] In each round, pass through tuples To calculate the target value:
[0080]
[0081] And minimize the loss function:
[0082]
[0083] in, Let φ be the loss function of server d, and let φ be the network parameters of the server's reinforcement learning policy, which are updated synchronously from the current network every certain number of steps. Let d be the target value for server d in round t. Let Q be the Q value of server d in round t. Let be the state of server d in round t. For the action of server d in round t;
[0084] This embodiment employs an ε-greedy strategy to balance exploration and exploitation, namely:
[0085]
[0086] in, Let Q be the Q value of server d under the target network parameter θ.
[0087] A5. Update the global model based on the optimal aggregate weights of all clients and distribute it to all clients.
[0088] In an optional embodiment of the present invention, the malicious client uploads model update data including:
[0089] Construct a set of benign models and a set of malicious models based on the global model;
[0090] In this embodiment, during the initial training phase, the malicious client constructs a benign model set based on local data. The malicious client uses clean local data to train the model for several rounds, calculating the structure consistency score for each round of model updates to form an initial benign score set. After formal communication begins, the malicious client gradually introduces the observation scores from the actual training process, dynamically updating the benign set using a sliding window approach. The malicious client will receive the global model Θ at the moment. t Add to the benign set. A malicious client injects a backdoor sample under the same settings and calculates the corresponding updated structural consistency score, which serves as the initial malicious score set. After formal communication begins, the malicious updates from each round of additional training are added to the malicious set to obtain...
[0091] Based on the benign model set and the malicious model set, a Gaussian mixture model is used to fit the distribution of benign models and the distribution of malicious models.
[0092] In this embodiment, the malicious client targets... and PCA dimensionality reduction was performed separately, and a Gaussian mixture model (GMM) was used for fitting to obtain the conditional distribution of global model parameters in the "unsuspected" and "suspected" cases:
[0093]
[0094] Among them, f B with f M Let represent the probability density functions of the scores for benign and malicious models, respectively.
[0095] Based on the distribution of benign and malicious models, a recursive update mechanism based on Bayesian full probability inference is used to estimate the posterior estimate of the probability that the client model update is suspected.
[0096] In this embodiment, in each round of communication, the malicious client completes local model training and calculates the structural consistency score s of the model. t The malicious client uses the following Bayesian update formula to estimate the subjective probability that its attack behavior will be recognized by the server in the current round:
[0097]
[0098] in, Let f be the probability that client a's model update is doubted in round t. M f is the probability density function of the score of the malicious model. B Let s be the probability density function of the score of the benign model. t The score represents the structural consistency score of the local model in round t. If the score is s... t If the attack is closer to a malicious distribution, the attacker will increase the probability of identification; otherwise, they will consider the attack to still be covert.
[0099] The client-side model updates the probability of being suspected, which is used as the state input to the client-side reinforcement learning policy network to learn to select the optimal attack strength.
[0100] In this embodiment, each malicious client has an independent DQN policy network. The goal of each malicious client is to dynamically adjust its backdoor attack data injection ratio to maximize the attack success rate without being detected by the server. The state space, action space, and reward function of the malicious client designed in this embodiment are as follows:
[0101] State space: Used to describe the attacker's state, S i It represents the state of client i in round t. in This indicates that the attacker's history is being questioned, C a This represents the set of attackers.
[0102] Action space: A i It is the action taken by client i in round t, A i ={0,0.1,...,1.0},i∈|C a | Use the attack strength selection set to represent the proportion of injected backdoors in the data.
[0103] Reward function: Considering the trade-off between the effectiveness of the attack and the risk of detection, in
[0104] The update formula for the learning state-action value function Q(s,a) of each malicious client is:
[0105]
[0106] In each round, pass through tuples To calculate the target value:
[0107]
[0108] And minimize the loss function:
[0109]
[0110] in, Let θ be the loss function for client a, and let θ be the network parameters of the client's reinforcement learning policy, which are updated synchronously from the current network every certain number of steps. Let be the target value for client a in round t. Let Q be the Q value of client a in round t. Let's consider the state of client a in round t. For client a, the action in round t;
[0111] This embodiment employs an ε-greedy strategy to balance exploration and exploitation, namely:
[0112]
[0113] in, Let Q be the Q value of client a under the target network parameters θ.
[0114] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0115] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0116] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0117] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.
[0118] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.
Claims
1. A federated learning backdoor attack defense method based on dynamic Bayesian belief update and deep reinforcement learning beliefs, characterized in that, Includes the following steps: Retrieve model update data uploaded by all clients; Based on the model update data uploaded by all clients, a Gaussian mixture model is used to fit the model update distribution of all clients; Based on the model update distribution of all clients, a recursive update mechanism based on Bayesian full probability inference is used to update beliefs and obtain the posterior estimate of the malicious probability of all clients. The malicious probability of all clients is used as the state input to the server reinforcement learning policy network to learn the optimal action combination for selecting aggregate weights, thus obtaining the optimal aggregate weights for all clients. The global model is updated based on the optimal aggregate weights across all clients and then distributed to all clients.
2. The federated learning backdoor attack defense method based on dynamic Bayesian belief update and deep reinforcement learning beliefs as described in claim 1, characterized in that, The model update distribution fitted to all clients using a Gaussian mixture model includes: The optimal number of clusters is automatically determined using the Bayesian information criterion.
3. The federated learning backdoor attack defense method based on dynamic Bayesian belief update and deep reinforcement learning beliefs as described in claim 1, characterized in that, A recursive update mechanism based on Bayesian full probability inference is used for belief updates to obtain posterior estimates of the malicious probability for all clients, including: Using the model update data uploaded by the client as observations, we modeled each client to obtain the distribution of benign model scores and the distribution of malicious model scores; A recursive update mechanism based on Bayesian full probability inference is used, combining the current score and historical beliefs, to calculate the malicious probability of each client, specifically: in, Let π be the probability of malicious intent of client i in round t, and let π be a one-dimensional Gaussian density function. Let μ be the observation score for client i in round t. B σ is the mean of the benign model. B μ represents the standard deviation of a benign model. M Let σ be the mean of the malicious model. M Let d be the standard deviation of the malicious model, and d be the server.
4. The federated learning backdoor attack defense method based on dynamic Bayesian belief update and deep reinforcement learning beliefs as described in claim 3, characterized in that, The server reinforcement learning policy network, which uses the malicious probability of all clients as state input, includes: Construct the server's state space using the estimated probability of malice for all clients in the current round; The server's action space is constructed using the aggregate weights of all clients in the current round; The server's reward function is constructed using a negative logarithmic penalty against the probability of malice, specifically: in, Let α be the reward for server d in round t, and α be the scaling factor. Let be the estimated probability of malicious activity for client i in round t. Let be the aggregate weight of client i in round t.
5. The federated learning backdoor attack defense method based on dynamic Bayesian belief update and deep reinforcement learning beliefs according to claim 4, characterized in that, The server-side reinforcement learning strategy network uses the following loss function: in, Let be the loss function for server d, and φ be the network parameters of the server's reinforcement learning policy. Let d be the target value for server d in round t. Let Q be the Q value of server d in round t. Let be the state of server d in round t. For the action of server d in round t; And an ε-greedy strategy is used for action updates, specifically: in, Let Q be the Q value of server d under the target network parameter φ.
6. The federated learning backdoor attack defense method based on dynamic Bayesian belief update and deep reinforcement learning beliefs according to claim 1, characterized in that, The client constructs a set of benign models and a set of malicious models based on the global model; Based on the benign model set and the malicious model set, a Gaussian mixture model is used to fit the distribution of benign models and the distribution of malicious models. Based on the distribution of benign and malicious models, a recursive update mechanism based on Bayesian full probability inference is used to estimate the posterior estimate of the probability that the client model update is suspected. The client-side model updates the probability of being suspected, which is used as the state input to the client-side reinforcement learning policy network to learn to select the optimal attack strength. A malicious model is trained based on the client's optimal attack strength, and the updated model data is uploaded to the server.
7. The federated learning backdoor attack defense method based on dynamic Bayesian belief update and deep reinforcement learning beliefs according to claim 6, characterized in that, The client constructs a set of benign models and a set of malicious models based on the global model, including: The client constructs a benign model set based on local data and gradually introduces the observation scores from the real training process after formal communication begins. The benign model set is dynamically updated in a sliding window manner, and the currently received global model is added to the benign model set. Under the same settings, backdoor samples are injected into local data to construct a malicious model set, and then the malicious model set is appended with additional training malicious updates in each round after formal communication begins.
8. The federated learning backdoor attack defense method based on dynamic Bayesian belief update and deep reinforcement learning beliefs according to claim 7, characterized in that, A recursive update mechanism based on Bayesian total probability inference is used to estimate the posterior estimate of the probability of the client model update being doubted, specifically as follows: in, Let f be the probability that client a's model update is doubted in round t. M f is the probability density function of the score of the malicious model. B Let s be the probability density function of the benign model score. t The structural consistency score of the local model in round t.
9. The federated learning backdoor attack defense method based on dynamic Bayesian belief update and deep reinforcement learning beliefs according to claim 8, characterized in that, The client-side reinforcement learning policy network, which uses the client-side model update's suspected probability as state input, includes: The client's state space is constructed based on the belief that the current round of malicious clients has a history of suspicion. The client's action space is constructed based on the proportion of backdoors injected into local data by the malicious client in the current round; The client's reward function is constructed using a dynamic trade-off mechanism between attack success rate and the probability of being suspected, specifically as follows: in, For client a's reward in round t, ASR t Let be the attack success rate in round t, κ be the weight of the attack success rate, and τ be the weight of the probability of being suspected.
10. The federated learning backdoor attack defense method based on dynamic Bayesian belief update and deep reinforcement learning beliefs according to claim 9, characterized in that, The client-side reinforcement learning policy network uses the following loss function: in, Let θ be the loss function for client a, and θ be the network parameters of the client's reinforcement learning policy. Let be the target value for client a in round t. Let Q be the Q value of client a in round t. Let's consider the state of client a in round t. For client a, the action in round t; And an ε-greedy strategy is used for action updates, specifically: in, Let Q be the Q value of client a under the target network parameters θ.