Joint audit security defense method for federal learning multiple poisoning attacks

By using joint audit security defense methods in federated learning, detecting gradient updates uploaded by the client, combining joint estimates, cosine similarity and European distance indicators, adjusting the aggregate weights, the problem of multiple poisoning attacks in federated learning is solved, and the problem of multi-poisoning attacks in federated learning is realized. Good defense performance and model performance maintenance in multiple attack scenarios.

CN118916878AActive Publication Date: 2024-11-08SUN YAT SEN UNIVERSITY SHENZHEN +1

Patent Information

Application Number
CN202411201304.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-11-08
Estimated Expiration
2044-08-29

AI Technical Summary

Technical Problem

There are multiple poisoning attacks in federated learning, and existing defense methods are difficult to effectively resist, especially in targeted attacks, model performance is severely disturbed.

Method used

The joint audit security defense method is adopted to detect the gradient update uploaded by the client on the server side, use the three indicators of joint estimates, cosine similarity and European distance to perform abnormal detection, and adjust the aggregate weight according to the detection results to weaken The impact of the attacker.

Benefits of technology

In multi-various attack scenarios, it can effectively detect and resist multiple attack behaviors, maintain the performance and training efficiency of the model, reduce the impact of the attacker on the model accuracy, and improve the robustness of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916878A_ABST
    Figure CN118916878A_ABST
Patent Text Reader

Abstract

The invention discloses a joint audit security defense method for federal learning multi-poison attacks, and belongs to the technical field of artificial intelligence security and information security, and the method comprises the steps: S1, constructing a federal learning model which comprises a server and K clients; s2, training the federal learning model to obtain a global model; and S3, setting a defense algorithm at the server side, detecting the gradient update uploaded by the client side, scoring the client side through an evaluation algorithm, and modifying a corresponding aggregation weight so as to obtain a new round of global model. By adopting the joint audit security defense method for federal learning multi-poison attacks, various different malicious attack behaviors can be effectively detected, the training task can be ensured to be carried out, the attack of the attack to the training task is reduced, and meanwhile, the robustness and efficiency of the task are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence security and information security technology, and in particular to a joint audit security defense method against federated learning multi-poisoning attacks. Background Art

[0002] Artificial intelligence is a technology that has developed rapidly in recent years. It builds a suitable model, performs iterative training on the data set, and then obtains a model suitable for a variety of target tasks. The training of deep neural networks requires massive data input to extract the intrinsic feature information of the data. As the number of input samples increases, the trained model effect is often better. Therefore, how to obtain a large number of data samples is the key point of artificial intelligence technology.

[0003] Traditional centralized learning requires a lot of computing power. All data needs to be collected in a computing center, which will then train and process all the data. As the amount of data increases, the computing overhead will be very large. General computing centers are often unable to bear the huge computing burden, which increases computing costs and time costs. At the same time, collecting and uploading a large amount of data also increases additional communication overhead and storage pressure.

[0004] Distributed learning is an improvement on the centralized learning framework. It no longer requires data to be concentrated in a computing center, but allows data to be distributed on different nodes. A central server coordinates the training. The network model structure is determined according to the task and is usually published by the central server. Nodes are providers of computing power. They download shared models and part of the data set under the organization of the central server, and then perform iterative training, and then send the trained local models to the central server. The central server generates a global model by organizing and aggregating the local models of these nodes. This method can basically achieve the performance of centralized data training. The advantage of distributed learning is that it shares the computing pressure among the nodes. The central server only needs to manage the distribution and aggregation of these models, thereby reducing the computing overhead of the central server. It can be seen that the difference between distributed learning and centralized learning is that distributed learning no longer has high requirements for the computing power of the central server. The parallel structure reduces the computing time, but also increases the communication overhead with the nodes.

[0005] However, with the development of modern society, enterprises and individuals pay more attention to the anonymity of their own data, resulting in a tendency to upload their own private data less and less. It is more difficult to collect enough data samples to meet the performance requirements of the model. At the same time, the collected data often comes from different institutions and departments, and the organization and characteristics of these data may be different, forming a "data island". Therefore, it is also necessary to find a way to organize different types of data from different parties.

[0006] Driven by the two premises of privacy security and data silos, federated learning came into being. Federated learning also belongs to the category of distributed machine learning, but the scope of application is different. Similarly, common federated learning involves multiple nodes participating in training. These nodes may be different companies or departments, and then are managed uniformly by a central server. Unlike traditional distributed learning, they are not only providers of computing power, but also providers of data, which means that data does not need to be uploaded to the server, but can be saved locally. The central server broadcasts and publishes training tasks, and then the nodes decide whether to participate in the training tasks. Participants can join the training tasks at any time or quit at any time. The work of the participants is to download the shared model from the central server, then train it on the local private data set, and then upload their own model gradient updates. As a result, the participants avoid the risk of exposing personal privacy, and at the same time complete the task of training data. The central server aggregates these uploaded model updates to obtain a global model. At the same time, because the storage and training data sets are both performed on the client, the client and server do not need to transmit a large number of data samples back and forth, and the storage and computing pressure on the server will also be greatly reduced. In general, federated learning has two major benefits. First, it ensures the anonymity of the client's local data set, without the need to disclose its own private data to the outside world. Second, it coordinates the local data of all parties to provide strong support for training a good global model.

[0007] However, with a deeper exploration of federated learning, researchers have found that this training mode also has some security risks. The client's data set is not visible to the server, which makes the gradient update uploaded by the client unreliable, because the server will not be able to use the data set to verify the gradient of the participants. This raises a security issue in federated learning. There are some malicious attackers in the client who will upload wrong gradient updates to damage the training of the global model, which is called poisoning. The attacker's goals are divided into two types. One is untargeted attack, such as data poisoning and model poisoning, which means that there is no special attack orientation, just to reduce the performance of the final model, or even fail to converge. The other is targeted attack, such as backdoor attack. The attacker will formulate his own attack task and enable the final model to achieve this attack task. For example, in the recognition task, the attacker will specifically let the main model mistakenly identify a certain object as another object. Such an attack will greatly reduce the performance of the final model and cause serious harm in practical applications.

[0008] However, the current defense against poisoning attacks is not perfect. The anonymity of the client data set greatly increases the difficulty of anomaly detection. Some existing methods assume that there is an auxiliary data set on the server side to assist in checking the anomaly of the gradients uploaded by the participants, but this method is not reasonable because the client data distribution types are diverse and not public, so it is not realistic to construct a data set. Some classic algorithms can achieve good application results under the premise of independent and identical distribution, but the data in federated learning is not independent and identically distributed, so these defense methods cannot be well implemented in federated learning. In recent years, researchers have successively proposed some defense strategies suitable for federated learning scenarios, but with them, more complex and more variable attack methods have also emerged. Ordinary defense methods can no longer meet the defense needs in today's multiple attack scenarios. For example, some defense methods are effective against untargeted attacks, but have no obvious effect on targeted attacks. This means that such a defense system is still flawed.

[0009] Therefore, the present invention provides a security defense strategy for multiple attack scenarios based on the federated learning mode. It does not require the construction of auxiliary data sets on the server side, and only requires some special calculations on the gradient updates uploaded by the participants. At the same time, it can cope with various forms of attacks, maintain excellent training effects under different attacks, and has more stable performance than existing defense algorithms. Summary of the invention

[0010] The purpose of the present invention is to provide a joint audit security defense method for federated learning multi-poisoning attacks, so as to solve the security improvement measures for federated learning technology existing in the above-mentioned background technology. Federated learning technology is often used for big data model training in privacy protection scenarios. Under privacy protection, the local data information of the participants is usually not disclosed. The server cannot review the model provided by the user, which makes the training task very vulnerable to attacks by malicious users, thereby reducing the effectiveness of the model. Although there are some methods for detecting malicious gradients at this stage, they are still not enough to resist diversified malicious attacks, so that the model performance is still greatly disturbed in the joint attack mode.

[0011] To achieve the above object, the present invention provides a joint audit security defense method against federated learning multi-poisoning attacks, comprising the following steps:

[0012] S1. Build a federated learning model, including the server and K clients. Use k=0,1,2…,K to represent the kth client. The dataset owned by the kth client is The amount of data is Use z k =(x k ,yk ) represents the input data label pair. The data of different clients are not independent and identically distributed, which means that the input data features or labels of different clients are different, and the data volume is also different. The total amount of local data of all clients is recorded as The model is recorded as The gradient is recorded as

[0013] S2. Train the federated learning model to obtain a global model;

[0014] S3. Set up a defense algorithm on the server side to detect the gradient updates uploaded by the client, score the client through an evaluation algorithm, modify the corresponding aggregation weights, and obtain a new round of global model.

[0015] Preferably, the federated learning model training in step S2 is performed for T iterations, as follows:

[0016] S21, the server publishes the learning task, the client participating in the training applies to connect to the server, and the server broadcasts the task parameters to all clients, such as the model network, optimizer, and loss function F(ω,z);

[0017] S22, server-side initialization model parameters ω 0 , the client initializes the model parameters Set up the optimizer;

[0018] S23. In the iteration round t=1, 2, ..., T, a random number of clients join the aggregation of this round. These users are called participants, with S t Representing the collection of these clients, the server sends a message to the participant S t Send the current global model parameters ω t ;

[0019] S24, Participant i∈S t Replace the current global model parameters with your own model parameters According to the learning rate γ t , local iteration number j = 1, 2, ..., τ, train your local model according to the following expression:

[0020]

[0021] S25, Participant i∈S t After performing τ local iterations locally, the model is obtained Then calculate its own gradient update:

[0022]

[0023] S26, Participant i∈S t Upload your own gradient updates The server collects all gradient updates and aggregates them through the aggregation algorithm G(·) to obtain a new round of global model;

[0024] S27. Repeat steps S23 to S26 until the model convergence requirement is met or the maximum number of iterations T is reached, the model convergence is completed, the final global model is obtained, and the federated learning task is completed.

[0025] Step S23 simulates the client set that accesses the server at the current time in actual application. There may be malicious attackers mixed in, and the set of these attackers is recorded as For malicious attackers They may tamper with the dataset or upload fake gradient updates; if data poisoning is used, the attacker's local dataset will be tampered with Then formula (1) will be modified into the following form:

[0026]

[0027] If the model is poisoned, then (2) will be modified to in, It is an attack algorithm used by malicious actors to modify gradients. If it is a targeted attack, such as a backdoor attack, it is implemented by combining data poisoning and model poisoning.

[0028] Preferably, the aggregation algorithm in step S26 is:

[0029]

[0030] Preferably, since the malicious participants' attacks are all poisoning the global model through malicious gradients, and the gradient update is also the most important information presented by the participants to the server, the detection of gradients is a key step in defense. Through the detection of gradients, researchers have proposed many classic algorithms. For example, traditional defense methods, such as Krum, are to calculate the Euclidean distance from the participant set S. t Filter out users who are considered reliable Aggregate their gradient updates. This method assumes that malicious gradients will be far away from normal gradients. However, the federated learning architecture is heterogeneous, which means that the gradients of benign users may also show large differences, which does not mean that they are aggressive. There are also some methods that calculate the cosine similarity between the gradients of participants, remove users with high similarity, and filter out users that are considered reliable. However, their criteria for detecting malicious users are not reasonable enough, and they are often not competent for the defense task under multiple attacks, and they are single-minded. The present invention sets a defense algorithm on the server side to detect the gradient update uploaded by the participants, and scores the client through an evaluation algorithm to modify the corresponding aggregation weight, as follows:

[0031] S31, the server will detect the attacker from the three indicators of joint estimation value χ, cosine similarity α, and Euclidean distance β. Before the training starts, the server initializes the list l corresponding to these three indicators χ ,l α ,l β , record the number of bad behaviors of the participants;

[0032] S32. In the iteration round t=1,2,...,T, the server collects the data from the participant S t Gradient collection And calculate the current joint estimate χ for each user i , evaluate the credibility of the participant according to the joint estimation value, and then record it in the corresponding list;

[0033] S33. After calculating the joint estimated value of the participants, the server has made a preliminary judgment on the maliciousness of the participants and eliminated the participants with obvious malicious behavior. Calculate a mean gradient for all users in

[0034]

[0035] S34, this mean gradient It is called the standard for subsequent detection, and the server calculates the remaining gradient The cosine similarity with the mean gradient is then used to evaluate the user's suspiciousness and recorded in the list;

[0036] S35. The server detects the Euclidean distance β of the participants and calculates the residual gradient The Euclidean distance from the mean gradient is recorded in a list according to the degree of suspicion;

[0037] S36. In the iteration round t=1, 2, ..., T, the server records the results of the first t-1 tests and records them in the list l χ ,l α ,l βAt the same time, the server also records the number of times each client is selected as an aggregation participant. For each client k = 0, 1, 2, .., K, the ratio of the number of times it is recorded in the list to the number of times it is selected as an aggregation participant is calculated, and three scores s related to the joint estimate, cosine similarity, and Euclidean distance are obtained. k ,χ,s k,α ,s k,β , and then use the relevant calculation method to obtain the malicious evaluation value of client k from these three scores

[0038] S37. For the aggregated set of participants Initial aggregation weight is the ratio of the client dataset size;

[0039]

[0040] S38. For the tth iteration, the server divides the suspicious users into the list according to the three indicators. In , if participant i exists in Then reduce the aggregate weight of the participant; the malicious evaluation value of each client is obtained from step S36 The server regards the top 30% of clients with the largest malicious evaluation values ​​as attackers. If participant i belongs to the attacker set, its aggregate weight is further reduced. The final modified weight is

[0041] S39. The server obtains a new set of participants. We also get a new set of weights Then, according to the weighted aggregation method, a new round of global model is obtained as follows

[0042]

[0043] Preferably, step S32 specifically includes:

[0044] S321. For each participant Computing and other parties The cosine similarity cs of the gradient update of j≠i ij ;

[0045]

[0046] S322, the server pre-sets a hyperparameter n x , Participants Select The largest nx and calculate the mean to obtain the joint estimate χ of participant i i ;

[0047] S323, the server obtains the joint estimated value of the participants and checks them one by one. If χ i >0.7, the participant i is considered to be a malicious user and is removed from the set In addition, we get a new set of participants If no participant meets the above formula, then select the n with the largest joint estimate x Participants are regarded as suspected attackers and are considered bad behaviors and recorded in the list. middle.

[0048] The joint estimate is an important indicator for judging the degree of abnormality. Generally speaking, if χ i If the threshold is exceeded, then the participant is likely to be a malicious attacker, because attackers will implement joint attacks and they usually show similar characteristics to achieve a higher level of attack, so the joint estimate is an effective detection indicator. i If the threshold is not exceeded, it means that the degree of union among the participants is not significant and further testing is needed.

[0049] Preferably, step S34 specifically includes:

[0050] S341. For each participant According to formula (7), the cosine similarity α between it and the mean gradient is calculated i , and then get an array of cosine similarities for all participants and mean gradients Calculate the median of this array Mean and variance

[0051] S342, if It is believed that α i Smaller parties are suspicious targets of attack; It is believed that α i Larger parties are suspicious targets of attack;

[0052] S343. If the new set of participants is equal to the original set That is, the joint estimation value does not show the obvious joint attack tendency of the participants. The server pre-sets a hyperparameter n α , select α from the range in step S342 i Maximum or minimum n α Gradients, record their corresponding participant numbers in the list In the case of a misconduct, it is considered as a misconduct;

[0053] S344: If some malicious users have been identified in step S32, Then it is considered that the detection has been completed based on the similarity feature, and no additional identification of suspicious users is required based on cosine similarity.

[0054] Both the cosine similarity α and the joint estimate χ are calculated based on the cosine value. Although malicious users may show abnormal characteristics in both the cosine similarity and the joint estimate, their detection principles are different. Combining the two is to prevent more diversified attacks.

[0055] Preferably, step S35 specifically includes:

[0056] S351. For each participant The Euclidean distance β between it and the mean gradient is calculated according to the following formula i , and then get an array of Euclidean distances between all participants and the mean gradient

[0057] (8)

[0058] S352, calculate the Euclidean distance array β t The mean of

[0059] S353: The server pre-sets the maximum threshold β of the Euclidean distance max and the minimum threshold β min , check the gradients in the set of participants one by one With β t Compared with the mean, if it is too large, the gradient is clipped. If it is too small, the gradient is amplified. Then record the gradients that need to be clipped or enlarged in a list middle.

[0060] The difference from the previous method is that participants whose joint estimation value exceeds a certain threshold are regarded as malicious users and are excluded from the set. If the gradient detected by Euclidean distance exceeds the threshold, the participant will not be excluded, but the gradient will be corrected. Because the joint estimation value judges the joint behavior, once it meets the malicious behavior judgment standard, it means that there are a large number of attackers among the participants, and their attack effect will be positively correlated with the number of attackers. At this time, their attack effect is very large, and methods need to be taken to prevent it. However, if the Euclidean distance judgment indicator meets the malicious behavior standard, it does not mean that a joint attack has been carried out, but perhaps it is just a single behavior attack. In this case, as long as their attack level is limited, their attack can be weakened, and even the difference of the aggregated gradient can be improved, which promotes model convergence.

[0061] Therefore, the present invention adopts the above-mentioned joint audit security defense method against federated learning multi-poisoning attacks, which has the following beneficial effects:

[0062] (1) A security defense measure against poisoning attacks in federated learning. It mainly detects gradients uploaded by participants and performs gradient anomaly detection based on three indicators: joint estimate, cosine similarity, and Euclidean distance. Combined with the detection algorithm, it can take into account most attack behaviors, because most attack behaviors will show special forms on the gradient.

[0063] (2) The present invention does not simply determine whether a malicious attacker is based on a single detection. It can create a list on the server side to record the client's historical behavior and comprehensively calculate the client's malicious evaluation value. At the same time, the present invention is not limited to eliminating malicious attackers, but also adds them to the aggregation, but reduces their aggregation weight, which not only ensures the convergence efficiency of the model, but also weakens the attacker's attack effect;

[0064] (3) It can maintain the performance of the model to the greatest extent under the premise of attacks, reduce the impact of attackers on the accuracy of the model, and improve the robustness of model training.

[0065] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 This is a training flow chart of the federated learning model of the present invention;

[0067] Figure 2 This is a flow chart of the defense method adopted by the present invention;

[0068] Figure 3 It is a schematic diagram of the defense performance of an embodiment of the present invention in the data_poison scenario;

[0069] Figure 4 It is a schematic diagram of the defense performance of an embodiment of the present invention in a reverse scenario;

[0070] Figure 5 It is a schematic diagram of the defense performance of an embodiment of the present invention in a LIE scenario;

[0071] Figure 6 It is a schematic diagram of the defense performance of an embodiment of the present invention in a SF scenario;

[0072] Figure 7 It is a schematic diagram of the defense performance of an embodiment of the present invention in the label_flip scenario;

[0073] Figure 8 This is a schematic diagram of the attacker success rate in the label_flip scenario according to an embodiment of the present invention;

[0074] Fig. 9 It is a schematic diagram of the defense performance of an embodiment of the present invention in a backdoor scenario;

[0075] Fig.10 The figure is a schematic diagram of the success rate of an attacker in a backdoor scenario according to an embodiment of the present invention. DETAILED DESCRIPTION

[0076] Example

[0077] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0078] like Figure 1-2 As shown, a joint audit security defense method for federated learning multi-poisoning attack includes the following steps:

[0079] S1. Build a federated learning model, including the server and K clients. Use k=0,1,2…,K to represent the kth client. The dataset owned by the kth client is The amount of data is Use z k =(x k ,t k ) represents the input data label pair. The data of different clients are not independent and identically distributed, which means that the input data features or labels of different clients are different, and the data volume is also different. The total amount of local data of all clients is recorded as The model is recorded as The gradient is recorded as

[0080] S2. Federated learning model training requires T iterations to obtain a global model, as follows:

[0081] S21, the server publishes the learning task, the client participating in the training applies to connect to the server, and the server broadcasts the task parameters to all clients, such as the model network, optimizer, and loss function F(ω,z);

[0082] S22, server-side initialization model parameters ω 0 , the client initializes the model parameters Set up the optimizer;

[0083] S23. In the iteration round t=1, 2, ..., T, a random number of clients join the aggregation of this round. These users are called participants, with S t Representing the collection of these clients, the server sends a message to the participant S t Send the current global model parameters ω t ;

[0084] S24, Participant i∈S t Replace the current global model parameters with your own model parameters According to the learning rate γ t , local iteration number j = 1, 2, ..., τ, train your local model according to the following expression:

[0085]

[0086] S25, Participant i∈S t After performing τ local iterations locally, the model is obtained Then calculate its own gradient update:

[0087]

[0088] S26, Participant i∈S t Upload your own gradient updates The server collects all gradient updates and aggregates them through the aggregation algorithm G(·) to obtain a new round of global model; the general aggregation algorithm is

[0089]

[0090] S27. Repeat steps S23 to S26 until the model convergence requirement is met or the maximum number of iterations T is reached, the model convergence is completed, the final global model is obtained, and the federated learning task is completed.

[0091] Step S23 simulates the client set that accesses the server at the current time in actual application. There may be malicious attackers mixed in, and the set of these attackers is recorded as For malicious attackers They may tamper with the dataset or upload fake gradient updates; if data poisoning is used, the attacker's local dataset will be tampered with Then formula (1) will be modified into the following form:

[0092]

[0093] If the model is poisoned, then (2) will be modified to in, It is an attack algorithm used by malicious actors to modify gradients. If it is a targeted attack, such as a backdoor attack, it is implemented by combining data poisoning and model poisoning.

[0094] S3. Since the malicious participants' attacks are all poisoning the global model through malicious gradients, and the gradient update is also the most important information presented by the participants to the server, the detection of gradients is a key step in defense. Through the detection of gradients, researchers have proposed many classic algorithms. For example, traditional defense methods, such as Krum, calculate the Euclidean distance to obtain the gradient from the participant set S. t Filter out users who are considered reliable Aggregate their gradient updates. This method assumes that malicious gradients will be far away from normal gradients. However, the federated learning architecture is heterogeneous, which means that the gradients of benign users may also show large differences, which does not mean that they are aggressive. There are also some methods that calculate the cosine similarity between the gradients of participants, remove users with high similarity, and filter out users that are considered reliable. However, their criteria for detecting malicious users are not reasonable enough, and they are often not competent for the defense task under multiple attacks, and they are single-minded. The present invention sets a defense algorithm on the server side to detect the gradient update uploaded by the participants, and scores the client through an evaluation algorithm to modify the corresponding aggregation weight, as follows:

[0095] S31, the server will detect the attacker from the three indicators of joint estimation value χ, cosine similarity α, and Euclidean distance β. Before the training starts, the server initializes the list l corresponding to these three indicators χ ,l a ,l β , record the number of bad behaviors of the participants;

[0096] S32. In the iteration round t=1,2,...,T, the server collects the data from the participant S t Gradient collection And calculate the current joint estimate χ for each user i , evaluate the credibility of the participant according to the joint estimation value, and then record it in the corresponding list;

[0097] S321. For each participant Computing and other parties The cosine similarity cs of the gradient update of j≠i ij ;

[0098]

[0099] S322, the server pre-sets a hyperparameter n x , Participants Select The largest n x and calculate the mean to obtain the joint estimate χ of participant i i ;

[0100] S323, the server obtains the joint estimated value of the participants and checks them one by one. If χ i >0.7, the participant i is considered to be a malicious user and is removed from the set In addition, we get a new set of participants If no participant meets the above formula, then select the n with the largest joint estimate x Participants are regarded as suspected attackers and are considered bad behaviors and recorded in the list. middle.

[0101] The joint estimate is an important indicator for judging the degree of abnormality. Generally speaking, if χ i If the threshold is exceeded, then the participant is likely to be a malicious attacker, because attackers will implement joint attacks and they usually show similar characteristics to achieve a higher level of attack, so the joint estimate is an effective detection indicator. i If the threshold is not exceeded, it means that the degree of union among the participants is not significant and further testing is needed.

[0102] S33. After calculating the joint estimated value of the participants, the server has made a preliminary judgment on the maliciousness of the participants and eliminated the participants with obvious malicious behavior. Calculate a mean gradient for all users in

[0103]

[0104] S34, this mean gradient It is called the standard for subsequent detection, and the server calculates the remaining gradient The cosine similarity with the mean gradient is then used to evaluate the user's suspiciousness and recorded in the list;

[0105] S341. For each participant According to formula (7), the cosine similarity α between it and the mean gradient is calculated i , and then get an array of cosine similarities for all participants and mean gradients Calculate the median of this array Mean and variance

[0106] S342, if It is believed that α i Smaller parties are suspicious targets of attack; It is believed that α i Larger parties are suspicious targets of attack;

[0107] S343. If the new set of participants is equal to the original set That is, the joint estimation value does not show the obvious joint attack tendency of the participants. The server pre-sets a hyperparameter n α , select α from the range in step S342 i Maximum or minimum n α Gradients, record their corresponding participant numbers in the list In the case of a misconduct, it is considered as a misconduct;

[0108] S344: If some malicious users have been identified in step S32, Then it is considered that the detection has been completed based on the similarity feature, and no additional identification of suspicious users is required based on cosine similarity.

[0109] Both the cosine similarity α and the joint estimate χ are calculated based on the cosine value. Although malicious users may show abnormal characteristics in both the cosine similarity and the joint estimate, their detection principles are different. Combining the two is to prevent more diversified attacks.

[0110] S35. The server detects the Euclidean distance β of the participants and calculates the residual gradient The Euclidean distance from the mean gradient is recorded in a list according to the degree of suspicion;

[0111] S351. For each participant The Euclidean distance β between it and the mean gradient is calculated according to the following formula i , and then get an array of Euclidean distances between all participants and the mean gradient

[0112]

[0113] S352, calculate the Euclidean distance array β t The mean of

[0114] S353: The server pre-sets the maximum threshold β of the Euclidean distance max and the minimum threshold β min , check the gradients in the set of participants one by one With β t Compared with the mean, if it is too large, the gradient is clipped. If it is too small, the gradient is amplified. Then record the gradients that need to be clipped or enlarged in a list middle.

[0115] The difference from the previous method is that participants whose joint estimation value exceeds a certain threshold are regarded as malicious users and are excluded from the set. If the gradient detected by Euclidean distance exceeds the threshold, the participant will not be excluded, but the gradient will be corrected. Because the joint estimation value judges the joint behavior, once it meets the malicious behavior judgment standard, it means that there are a large number of attackers among the participants, and their attack effect will be positively correlated with the number of attackers. At this time, their attack effect is very large, and methods need to be taken to prevent it. However, if the Euclidean distance judgment indicator meets the malicious behavior standard, it does not mean that a joint attack has been carried out, but perhaps it is just a single behavior attack. In this case, as long as their attack level is limited, their attack can be weakened, and even the difference of the aggregated gradient can be improved, which promotes model convergence.

[0116] S36. In the iteration round t=1, 2, ..., T, the server records the results of the first t-1 tests and records them in the list l χ ,l α ,l β At the same time, the server also records the number of times each client is selected as an aggregation participant. For each client k = 0, 1, 2, .., K, the ratio of the number of times it is recorded in the list to the number of times it is selected as an aggregation participant is calculated, and three scores s related to the joint estimate, cosine similarity, and Euclidean distance are obtained. k ,χ,s k,α ,sk,β , and then use the relevant calculation method to obtain the malicious evaluation value of client k from these three scores

[0117] S37. For the aggregated set of participants Initial aggregation weight is the ratio of the client dataset size;

[0118]

[0119] S38. For the tth iteration, the server divides the suspicious users into the list according to the three indicators. In , if participant i exists in Then reduce the aggregate weight of the participant; the malicious evaluation value of each client is obtained from step S36 The server regards the top 30% of clients with the largest malicious evaluation values ​​as attackers. If participant i belongs to the attacker set, its aggregate weight is further reduced. The final modified weight is

[0120] S39. The server obtains a new set of participants. We also get a new set of weights Then, according to the weighted aggregation method, a new round of global model is obtained as follows

[0121]

[0122] In order to compare with existing methods, some commonly used defense algorithms are briefly described below:

[0123] (1) Krum: It is a classic distributed machine learning Robin algorithm. It calculates the Euclidean distance between each model, sums the distances of the nearest neighbors for each user, and takes the model with the smallest sum of distances as the aggregation result. The experimental result graph uses the multi-krum method, that is, the average of several models with the smallest sum of distances is taken as the aggregation result.

[0124] (2) Trimmed-mean: This is also a classic distributed machine learning security algorithm. It needs to find the geometric median of the gradients of the participants, and then select several gradients with the smallest distance to the geometric median as the aggregation object, also known as truncated mean. This method is similar to Krum in terms of calculation and principle.

[0125] (3) RFA: An advanced method for finding the geometric median gradient. The definition of the geometric median is to find a geometric mean that minimizes the sum of the Euclidean distances from all user gradients to the median. This method uses an iterative calculation method to find a more accurate geometric median gradient as the aggregation result.

[0126] (4) FABA: This method assumes that the gradient that deviates greatly from the mean is a malicious gradient. It removes suspicious gradients iteratively. Each time, it calculates a mean gradient and then removes the gradient with the largest deviation from the mean gradient.

[0127] (5) Density: This method uses density as a detection metric. A gradient with a higher density means that the average sum of the Euclidean distances between it and its neighboring gradients is smaller. It assumes that outliers with small density are high and reduces the weight of participants with high outliers.

[0128] (6) FoolsGold: This is a detection method based on cosine similarity. It calculates the cosine similarity between each user and other users, and takes the cosine similarity of the nearest gradient as the basis for judgment. If it exceeds a certain threshold, the learning rate of the participant is corrected.

[0129] In this embodiment, attacks will be injected into the CIFAR-10 image recognition task, and multiple defense measures will be added to compare the superiority of the present invention. Several advanced attacks will be listed below, and the above-mentioned defense measures will be added to compare with this embodiment.

[0130] (1) Data_poison: Data poisoning attack. The attacker has a normal data set. They will modify the features or labels of most of their data sets to make them different from the normal data sets. They will train poisonous gradients on the poisoned data sets, thereby reducing the accuracy of the final model. It does not change the trained gradients, but only changes the data sets. This type of attack does not intend to complete a certain task and is an untargeted attack.

[0131] like Figure 3 As shown, the final accuracy of the model using this embodiment is higher than that of the existing defense method.

[0132] (2) Reverse: Model poisoning attack. The attacker’s data set is good, but they will modify the gradient. The main method is to reverse the direction of the gradient, thereby affecting the orderly progress of the global model training gradient descent, ultimately reducing the accuracy of the model and even destroying its convergence.

[0133] like Figure 4 As shown, the final accuracy of the model using this embodiment is close to the baseline, and is not much different from the RFA method.

[0134] (3) LIE: This is a common attack and belongs to model poisoning. Attackers will work together to share information and jointly tamper with the uploaded gradient updates. They do not need to modify the dataset or even the training data, they only need to send a forged gradient. The effectiveness of this type of attack is no longer based on the strong offensiveness of a single attacker, but rather on multiple attackers working together to achieve greater results.

[0135] like Figure 5 As shown, the final accuracy of the model using this embodiment is close to the baseline, far exceeding other defense methods, most of which cannot converge.

[0136] (4) SF: This is a special form of attack. This type of attack not only represents attackers who attempt to destroy the model, but also represents some lazy clients who are unwilling to pay computing power but want to gain benefits. Malicious clients only train a small part of the data set, but exaggerate the size of their own data set, thereby trying to gain large benefits at a small cost.

[0137] like Figure 6 As shown, the final accuracy of the model using this embodiment is close to the baseline and is better than the existing defense method.

[0138] (5) Label_flip: refers to label flipping, which is a targeted attack that combines data poisoning and model poisoning. The attacker needs to first tamper with his own data set according to the preset task, and then obtain the malicious gradient that fits the attacker's task. In order to achieve higher effectiveness, it is often necessary to scale the gradient so that it has a greater influence in the global model.

[0139] like Figure 7 As shown in the figure, the training performance of various defense methods in the Label_flipping attack scenario is demonstrated. The higher the accuracy, the better the task effect. The final accuracy of the model in this embodiment is close to the baseline. Figure 8 As shown, the attacker carries out targeted attacks. The higher the attacker's accuracy rate is, the closer the final training result is to the malicious task pre-set by the attacker. Under the model effect of this embodiment, the attacker's success rate is reduced by more than 20% compared with the defenseless state. Compared with other defenses, it can perform at a level close to the best.

[0140] (6) Backdoor attack: This is a targeted attack that combines data poisoning and model poisoning. It is a relatively complex attack form in actual scenarios. The attacker needs to ensure the correct execution of the main task while implementing the malicious task preset by the attacker. For example, a trigger is added to the data set so that the data with the trigger can be trained to produce the result of the malicious task.

[0141] Fig. 9The training effects of various defense technologies in the Backdoor attack scenario are shown. The higher the accuracy, the better the result. The final accuracy of the model using this embodiment is close to the baseline. Fig.10 The figure shows the attack success rate of Backdoor attackers in different defense schemes. The lower the curve, the better the effect of the defense model. It can be seen that the attack success rate of the attacker has also dropped significantly under the model of this embodiment. The RFA method has obvious effect in accuracy, but it cannot effectively reduce the attack success rate.

[0142] Therefore, the present invention adopts the above-mentioned joint audit security defense method for federated learning multi-poisoning attacks, which can demonstrate good defense performance in multiple attack scenarios. Compared with various existing methods, it can effectively improve the accuracy of the task and reduce the success rate of attackers.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A joint audit security defense method for federated learning multi-poisoning attacks, characterized in that: The following steps are involved: S1. Build a federated learning model, including the server and K clients. Use k=0,1,2…,K to represent the kth client. The dataset owned by the kth client is The amount of data is Use z k =(x k ,y k ) represents the input data label pair, the data of different clients are not independent and identically distributed, and the total amount of local data of all clients is recorded as The model is recorded as The gradient is recorded as S2. Train the federated learning model to obtain a global model; S3. Set up a defense algorithm on the server side to detect the gradient updates uploaded by the client, score the client through an evaluation algorithm, modify the corresponding aggregation weights, and obtain a new round of global model.

2. According to claim 1, a joint audit security defense method for federated learning multi-poisoning attacks is characterized in that: The federated learning model training in step S2 needs T iterations, as follows: S21, the server publishes the learning task, the client participating in the training applies to connect to the server, and the server broadcasts the task parameters to all clients, including the model network, optimizer and loss function F(ω,z); S22, server-side initialization model parameters ω 0 , the client initializes the model parameters Set up the optimizer; S23. In the iteration round t=1, 2, ..., T, a random number of clients join the aggregation of this round. These users are called participants, with S t Representing the collection of these clients, the server sends a message to the participant S t Send the current global model parameters ω t ; S24, Participant i∈S t Replace the current global model parameters with your own model parameters According to the learning rate γ t , local iteration number j = 1, 2, ..., τ, train your local model according to the following expression: S25, Participant i∈S t After performing τ local iterations locally, the model is obtained Then calculate its own gradient update: S26, Participant i∈S t Upload your own gradient updates The server collects all gradient updates and aggregates them through the aggregation algorithm G(·) to obtain a new round of global model; S27. Repeat steps S23 to S26 until the model convergence requirement is met or the maximum number of iterations T is reached, the model convergence is completed, the final global model is obtained, and the federated learning task is completed.

3. According to claim 2, a joint audit security defense method for federated learning multi-poisoning attacks is characterized in that: The aggregation algorithm in step S26 is:

4. According to claim 1, a joint audit security defense method for federated learning multi-poisoning attacks is characterized in that: Step S3 is as follows: S31, the server will detect the attacker from the three indicators of joint estimation value χ, cosine similarity α, and Euclidean distance β. Before the training starts, the server initializes the list l corresponding to these three indicators χ ,l α ,l β , record the number of bad behaviors of the participants; S32. In the iteration round t=1,2,...,T, the server collects the data from the participant S t Gradient collection And calculate the current joint estimate χ for each user i , evaluate the credibility of the participant according to the joint estimation value, and then record it in the corresponding list; S33. The server side calculates the joint estimated value of the participants and removes the participants with obvious malicious behavior. Calculate a mean gradient for all users in S34, this mean gradient It is called the standard for subsequent detection, and the server calculates the remaining gradient The cosine similarity with the mean gradient is then used to evaluate the user's suspiciousness and recorded in the list; S35. The server detects the Euclidean distance β of the participants and calculates the residual gradient The Euclidean distance from the mean gradient is recorded in a list according to the degree of suspicion; S36. In the iteration round t=1, 2, ..., T, the server records the results of the first t-1 tests and records them in the list l χ ,l α ,l β At the same time, the server also records the number of times each client is selected as an aggregation participant. For each client k = 0, 1, 2, .., K, the ratio of the number of times it is recorded in the list to the number of times it is selected as an aggregation participant is calculated, and three scores s related to the joint estimate, cosine similarity, and Euclidean distance are obtained. k ,χ,s k,α ,s k,β , and then use the relevant calculation method to obtain the malicious evaluation value of client k from these three scores S37. For the aggregated set of participants Initial aggregation weight is the ratio of the client dataset size; S38. For the tth iteration, the server divides the suspicious users into the list according to the three indicators. In , if participant i exists in Then reduce the aggregate weight of the participant; the malicious evaluation value of each client is obtained from step S36 The server regards the top 30% of clients with the largest malicious evaluation values ​​as attackers. If participant i belongs to the attacker set, its aggregate weight is further reduced. The final modified weight is S39. The server obtains a new set of participants. We also get a new set of weights Then, according to the weighted aggregation method, a new round of global model is obtained as follows 5. According to claim 4, a joint audit security defense method for federated learning multi-poisoning attacks is characterized in that: Step S32 specifically includes: S321. For each participant Computing and other parties The cosine similarity cs of the gradient update of j≠i ij ; S322, the server pre-sets a hyperparameter n x , Participants Select The largest n x and calculate the mean to obtain the joint estimate χ of participant i i ; S323, the server obtains the joint estimated value of the participants and checks them one by one. If χ i >0.7, the participant i is considered to be a malicious user and is removed from the set In addition, we get a new set of participants If no participant meets the above formula, then select the n with the largest joint estimate x Participants are regarded as suspected attackers and are considered bad behaviors and recorded in the list. middle.

6. According to claim 4, a joint audit security defense method for federated learning multi-poisoning attacks is characterized in that: Step S34 specifically includes: S341. For each participant According to formula (7), the cosine similarity α between it and the mean gradient is calculated i , and then get an array of cosine similarities for all participants and mean gradients Calculate the median of this array Mean and variance S342, if It is believed that α i Small parties are suspicious targets; if It is believed that α i Large parties are suspicious targets; S343. If the new set of participants is equal to the original set The server pre-sets a hyperparameter n α , select α from the range in step S342 i Maximum or minimum n α Gradients, record their corresponding participant numbers in the list In the case of a misconduct, it is considered as a misconduct; S344: If some malicious users have been identified in step S32, Then it is considered that the detection has been completed based on the similarity feature, and no additional identification of suspicious users is required based on cosine similarity.

7. A joint audit security defense method for federated learning multi-poisoning attacks according to claim 4, characterized in that: Step S35 specifically includes: S351. For each participant The Euclidean distance β between it and the mean gradient is calculated according to the following formula i , and then get an array of Euclidean distances between all participants and the mean gradient S352, calculate the Euclidean distance array β t The mean of S353: The server pre-sets the maximum threshold β of the Euclidean distance max and the minimum threshold β min , check the gradients in the set of participants one by one With β t If the mean is larger, the gradient is clipped. If it is small, the gradient is amplified. These clipped or enlarged gradients are then recorded in a list middle.

Citation Information

Patent Citations

  • Defense method for federated learning neuron gradient attack

    CN115333825A

  • Robustness federated learning method based on client defense

    CN117035056A

  • Target-free model poisoning attack dynamic joint learning defense framework based on historical information

    CN118094566A

  • Federal learning poisoning attack defense method

    CN118246009A

  • Robustness federated learning method capable of resisting backdoor attack based on joint defense

    CN118297178A

Cited By

  • Poison defense method, device and equipment for federal learning, medium and product

    CN119520096A

  • Federal learning Krum algorithm hardware acceleration and optimization system and method

    CN119623583A

  • Federal learning hybrid attack data security defense system and method

    CN119961922A

  • A federated learning hybrid attack data security defense system and method

    CN119961922B

  • Defense method for non-independent identically distributed grouping aggregation attack based on federal learning

    CN120257291A