A backdoor attack defense method and system based on federated learning

Abstract

The application belongs to the technical field of machine learning, and discloses a backdoor attack defense method and system based on federated learning, which comprises the following steps: determining backdoor image data for training, initializing a training task on a server side and distributing a global model; a client receives the global model distributed by the server, trains a local model, and uploads the local model update to the server side; the global learning rate of the server side is adjusted according to the symbolic information of the local model update of the client; and the global model is updated according to a dynamic weight aggregation rule based on historical global updates. Through the application, the problem that the traditional federated learning backdoor defense method is limited by data distribution and the number of attack types is solved, and at the same time, good defense effect can be achieved without affecting the model performance, thereby improving the security and robustness of federated learning.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of machine learning, and particularly relates to a backdoor attack defense method and system based on federated learning. BACKGROUND

[0002] Current machine learning development faces data silos and data privacy protection problems. The data silo problem greatly limits the availability of data, and there is a lack of trust between institutions, making it difficult to achieve full sharing of data. At the same time, in the era of emerging privacy regulations such as the General Data Protection Regulation (GDPR), data privacy leakage problems are highlighted. With the rapid growth of the complexity of machine learning models and the demand for large-scale training data sets, more and more research has begun to focus on distributed machine learning. In this context, federated learning has emerged as a new distributed learning paradigm. Unlike traditional machine learning methods, federated learning allows multiple clients to collaborate in training a specific neural network model without sharing raw data, mainly including several client training to generate local updates, and a central server constantly iterating and optimizing the federated global model and distributing it to each client, which has been widely applied in autonomous driving, edge computing, medical care and other fields.

[0003] However, because the client can completely control the local data and the local training process, federated learning is vulnerable to adversarial machine learning attacks initiated by malicious clients. For example, by poisoning local data or replacing the update parameters sent by a client to the server, an attacker can gradually replace the global model with a backdoor model, a process known as backdoor attack. The goal of backdoor attack is to make the global model classify the input malicious data as the class specified by the attacker, but not affect the classification results of normal data, which ensures the overall performance of the training main task. Backdoor attack poses a potential threat to learning systems, and is triggered only when the model receives a specific input, which has strong concealment and poses great challenges to attack defense.

[0004] Due to the high emphasis on client privacy in federated learning, the center server has no access to client data and cannot determine whether the local updates uploaded by the clients are correctly generated according to the privacy agreement, which makes it more difficult to detect backdoor attacks. Existing defense methods against backdoor attacks mainly include Byzantine robust aggregation rules, clustering-based anomaly detection methods, similarity measurement-based defense, and differential privacy defense methods. However, due to the stealthy nature of backdoor attacks, the effectiveness of these defense measures is lacking. Robust aggregation rules based on statistical characteristics such as mean and median are limited to specific attack models, and their defense effectiveness depends on data distribution and environmental settings. Cluster-based anomaly detection methods with fewer clusters cannot meet the diverse needs of attacks, and attackers can inject multiple backdoor triggers to group at least one backdoor model update with benign model updates, making it difficult to detect all backdoors. Moreover, when there is no malicious client, this method may also misjudge benign local updates with biased data distribution as malicious local updates, making it difficult to be compatible with the case of no malicious client. When the target update directions of honest clients are similar, it is difficult to identify malicious model updates by calculating similarity alone, and it cannot defend against the case of only a single malicious client. There are also research methods based on differential privacy technology to defend against backdoor attacks, which are suitable for general attack models, but ensuring that the amount of noise added to eliminate backdoors will also affect the performance of the aggregated model to some extent, and it is difficult to balance privacy and accuracy.

[0005] In summary, there is currently no defense method for federated learning backdoor attacks that can effectively reduce the success rate of backdoor attacks without assuming specific settings such as data distribution, while ensuring the performance of the global model. Therefore, how to design an effective defense scheme to resist backdoor attacks is an important problem that federated learning needs to solve. SUMMARY

[0006] In view of the deficiencies of the prior art, the present application provides a backdoor attack defense method and system based on federated learning, which calculates the sign information of the local update of the client, quantifies the influence of the local model update on the global model, and adjusts the global learning rate of the server side accordingly to promote the model to update in a direction away from the malicious target; a dynamic weight aggregation rule based on historical global updates is designed, which considers the influence of gradient size and direction on the final model, on the one hand, by controlling the size of the local model update, the contribution of the malicious client to the global model is limited, and on the other hand, the cosine similarity is calculated using the sum of the client model update and the historical global update, which is more accurate and stable than the similarity calculation between clients, effectively reducing the success rate of backdoor attacks, weakening the effect of backdoor attacks, and ensuring the comprehensiveness of the defense. The present application is a lightweight backdoor attack defense method suitable for federated learning scenarios, which solves the problem of limited data distribution and attack type number in traditional federated learning backdoor defense methods, and can achieve good defense effect without affecting the model performance, improving the security and robustness of federated learning.

[0007] To solve the above technical problems, the technical scheme adopted by the present application is:

[0008] Firstly, the present application provides a backdoor attack defense method based on federated learning, comprising the following steps: step S1, determining the backdoor image data for training, initializing the training task at the server side and distributing the global model;

[0009] Step S2, the client receives the global model issued by the server, trains the local model using local data, and uploads the generated local model update to the server side;

[0010] Step S3, adjusting the global learning rate of the server side according to the sign information of the local model update of the client;

[0011] Step S4, processing the size of the local model update of each client to scale it to the same order of magnitude as the global model update;

[0012] Step S5, updating the global model according to the dynamic weight aggregation rule based on historical global updates:

[0013] The server-side aggregation generates a global model update using the scaled local model updates, and for training rounds with a constant global learning rate, the server-side calculates a client aggregation score, filters and removes client local updates with a cosine similarity less than zero to the sum of historical global updates, and for training rounds with a negative global learning rate, calculates a client aggregation score according to the negative value of the cosine similarity of the client local model update to the sum of historical global updates; finally, the scaled client local model updates are weighted and averaged according to the aggregation score to generate a global model update, and iterative training is performed until the final global model is obtained.

[0014] Further, in step S1, the server first determines the backdoor image dataset, the model training target, the initialized global model w0 and the hyperparameters, and broadcasts the initial global model w0 to the clients; wherein the backdoor image dataset is generated by adding different shapes of triggers in the data samples, and they are assigned to designated clients according to different data distributions, and the set of clients C = {c1, c2, …, cN} is set, and the local dataset of each client is k}, The global model training target can be regarded as a process of solving the optimal global model according to empirical risk minimization, that is, finding a set of optimal model parameters satisfying wherein, represents a d-dimensional weight vector, N i represents the size of the dataset D i , the total number of training data F(w; D i ) represents the empirical loss function of client i.

[0015] Further, in step S2, each client receives the global model w t-1 of the last round from the server as its local model w i,t = w t-1 , wherein w i,t represents the local model of client i in the tth round, and then client i optimizes the current local model using the local dataset D i obtains the parameter difference between the local models generated by client i when training in adjacent rounds, and this parameter difference is called client local model update, Δw i,t = w i,t -w i,t-1 , wherein w i,t and w i,t-1 represent the local models of client i in the tth and (t-1)th rounds respectively, and l represents the client local learning rate, represents the client gradient loss function; after local training, each client will generate a local model update Δw i,t ​upload to the server.

[0016] Further, in step S3, whenever a round of global training is completed, the server side obtains all the local model updates uploaded by the clients and calculates the sign sum S t , which is defined as follows:

[0017]

[0018] where n is the number of clients, Δw i,t is the local model update generated by client i in the tth round, and sgn(Δw i,t ) is a sign function, sgn(Δw i,t ) = 1 if Δw i,t > 0, and sgn(Δw i,t ) = -1 otherwise.

[0019] A learning rate threshold θ is introduced. If the sign sum St of the local model updates in the current round is less than the threshold θ, the server-side global learning rate is adjusted to -η, which maximizes the model training loss. If the sign sum is greater than or equal to the threshold θ, the global learning rate remains unchanged and is updated normally, i.e.,

[0020]

[0021] where η represents the initial global learning rate of the server side, S t represents the sign sum of the local model updates, and η θ,t represents the global learning rate in the tth round given the threshold θ.

[0022] The learning rate threshold θ is set according to the number of clients and the proportion of malicious clients. The value range of θ needs to satisfy CP+1<θ<C(1-P), where C is the number of clients and P is the proportion of malicious clients.

[0023] Further, in step S4, before performing the aggregation operation, the size of each client's local model update is processed to scale it to the same order of magnitude as the global model update; the local model update after scaling processing is calculated by the following formula:

[0024]

[0025] where t represents the model training round, Δw t-1 represents the global model update generated in the previous round, Δw i,t represents the local model update generated by client i in the tth round, and ||·|| represents the vector two-norm.

[0026] Further, step S5, using the scaled local model update, the server side aggregates to generate a global update, the specific method is:

[0027] S51, the server side obtains all global updates generated by the previous round of the current training task, referred to as the historical global update sum Δw pre,t-1 t-1 , respectively, calculate the cosine similarity CS i,t of each client local model update and the historical global update sum:

[0028]

[0029] Where Δw t-1 represents the global model update of the t-1 round, Δw pre,t-1 represents the historical global update sum, represents the scaled local model update of client i in the t round; If the cosine similarity is negative, it means that the local model update generated by the current client i has a negative impact on the global model;

[0030] S52, for the training round with unchanged global learning rate, it is necessary to filter out the malicious local model update opposite to the global update direction in the model aggregation process, that is, the local model update with negative cosine similarity, which is calculated by using the Relu operation to process the calculated cosine similarity, and is defined as the aggregation score T i,t of the current client i in the t round, and is used as the aggregation basis of the current round global model update; For the training round with negative global learning rate, the calculated cosine similarity is also processed by using the Relu operation, the difference is that, since the global learning rate is negative at this time, in order to make the model update in the direction away from the malicious target, it is necessary to maximize the model training loss, at this time, the client aggregation score T i,t is calculated according to the negative value of the cosine similarity, and the client aggregation score T i,t is calculated by the following formula:

[0031]

[0032] Where Δw pre,t-1 represents the historical global update sum, represents the scaled local model update of client i in the t round, if CS i,t > 0, then Relu(CS i,t ) = CS i,t , otherwise Relu(CS i,t ) = 0, CS i,t represents the cosine similarity of the local model update of client i and the historical global update sum; ​

[0033] S53, the server side calculates the weighted average value as the aggregated global model update according to the client scaled local update and the aggregated score:

[0034]

[0035] wherein n represents the number of clients, T i,t is the aggregated score of the current client i in the tth round, is the scaled local model update of the client i.

[0036] Further, since the historical global update of the previous round cannot be calculated in the first round of training, the median of the local model update is taken as the first round global model update

[0037] Further, in step S5, finally, the server side obtains the final global model according to the calculated global learning rate and global model update:

[0038]

[0039] wherein n represents the number of clients, η represents the initial global learning rate of the server side, S t is the sign sum of the local model update, Δw t-1 is the global model update generated in the previous round of training, CS i,t is the cosine similarity of the local model update of the client i and the historical global update sum, is the scaled local model update of the client i.

[0040] Secondly, the present application also provides a backdoor attack defense system based on federated learning, which is used to realize the backdoor attack defense method based on federated learning as described above, and the backdoor attack defense system based on federated learning comprises a local training module, a parameter uploading module, a learning rate modification module, a parameter processing module and an aggregated update module,

[0041] The local training module is used to train the local model of the client and generate the corresponding local model update.

[0042] The parameter uploading module is connected with the local training module and is used to upload the local model update of each client to the server side, and the server side receives all the local model updates of the current training round and marks them as w t ={Δw i,t ,i∈n}, wherein t is the model training round, n is the number of clients, Δw i,t is the local model update generated by the client i in the tth round;

[0043] The learning rate modification module is connected with the parameter uploading module and is used for judging whether the global learning rate of the current training needs to be modified according to the local model update of the global client.

[0044] The parameter processing module is connected with the parameter uploading module and is used for processing the size of the local model update of the client, so that the size is scaled to the same order of magnitude as the global model update, the contribution of the malicious client to the global model is reduced, and the benign model update is promoted to play a role.

[0045] The aggregation update module is connected with the learning rate modification module and the parameter processing module if the system starts the backdoor attack defense method proposed in the application, and is used for dynamic weight aggregation based on the historical global update: for the training round in which the global learning rate remains unchanged, the server calculates the client aggregation score, filters and removes the local update of the client whose cosine similarity with the sum of the historical global update is less than zero, and for the training round in which the global learning rate is negative, the client aggregation score is calculated according to the negative value of the cosine similarity; the scaled client local model update is weighted and averaged according to the aggregation score to generate the global model update, and iterative training is performed until the final global model is obtained; if the system does not start the defense method, the aggregation update module is connected with the parameter uploading module and is used for the federated average aggregation algorithm, which directly uses the average value of each client model update as the global model update to update the global model.

[0046] Compared with the prior art, the application has the following advantages:

[0047] (1) The application proposes a backdoor attack defense method based on federated learning, generates a plurality of local updates by using a plurality of clients for training, adjusts the global learning rate of the server side according to the sign information of the local update of each client, improves the traditional FedAvg federated average aggregation algorithm, and in each round of training, the cosine similarity between the local model update of each client and the sum of the historical global update of the previous round is calculated to obtain the aggregation score of each round of training of the client, which is used as the aggregation basis for the global model update of the current round, reduces the influence of malicious local update on the backdoor defense effect, ensures the comprehensiveness of the defense, and improves the robustness of federated learning.

[0048] The method is suitable for most federated learning frameworks and data distribution scenarios, has sufficient universality, is an unsupervised method deployed on the server side of federated learning, does not occupy the computer resources of the terminal device, has simple implementation steps and small computing overhead, and the existing researches are often limited to a certain data distribution scenario and have large computing overhead.

[0049] (2) The application adjusts the server-side global learning rate according to the overall direction of the client local model update, promotes the model to update in the direction opposite to the malicious target, and weakens the backdoor attack effect. The method is relatively independent and can be combined with different model aggregation rules to defend against different types of backdoor attacks. Few existing methods change the learning rate to defend against backdoors. Some studies reduce the learning rate of malicious clients to prevent backdoor attacks, but due to the limitations of detection accuracy, the learning rate of some honest clients is disturbed during the training process. (3) The dynamic weight aggregation rule based on historical global update proposed in the application considers the influence of gradient size and direction on the final model, controls the size of local model update, limits the contribution of malicious clients to the global model, and calculates the similarity by using the cosine distance between the client model update and the sum of the historical global update, which is more accurate and stable than the similarity calculation between clients, effectively reducing the success rate of backdoor attacks. Existing model aggregation rules often estimate the general range of global update based on statistical methods such as mean and median, and cannot fundamentally remove the influence of backdoor local update on the global model.

[0050] (4) The application improves the training process of the existing federated learning and can be applied to the federated learning scene with untrusted clients. The server-side global learning rate is adjusted by calculating the sign sum of the client local model update, and then the client aggregation score is calculated based on the historical global update to obtain the final model by weighted aggregation, reducing the contribution of suspicious local update to the global model and achieving comprehensive and effective defense against backdoor attacks. The defense method can also be combined with other defense methods (such as clipping update and adding noise) to enhance the defense effect. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0052] Figure 1 The flowchart of the backdoor attack defense method based on federated learning of the application;

[0053] Figure 2 The specific flowchart of adjusting the server-side global learning rate of the application;

[0054] Figure 3 The specific flowchart of the dynamic weight aggregation rule based on historical global update of the application;

[0055] Figure 4A federal learning-based backdoor attack defense system structure schematic diagram of the application. DETAILED DESCRIPTION

[0056] The application will be further described below in combination with the drawings and specific embodiments.

[0057] In view of the security problem caused by local data anomaly in the federal learning training process, the application provides a federal learning-based backdoor attack defense method and system, which is improved based on the general federal learning training process. The improved training process can effectively resist the backdoor attack initiated by the client and improve the security and robustness of the federal learning system. The following will be introduced in combination with specific embodiments.

[0058] Embodiment 1

[0059] In combination with Figure 1 As shown in the figure, the embodiment provides a federal learning-based backdoor attack defense method, which includes the following steps:

[0060] Step S1, determine the backdoor image data for training, initialize the training task at the server end and distribute the global model;

[0061] Step S2, the client receives the global model distributed by the server, trains the local model using the local data, and uploads the local model update to the server end;

[0062] Step S3, adjust the server end global learning rate according to the symbol information of the client local model update;

[0063] Step S4, process the size of each client local model update to scale it to the same order of magnitude as the global model update;

[0064] Step S5, update the global model according to the dynamic weight aggregation rule based on the historical global update:

[0065] With the scaled local model update, the server-side aggregation generates a global model update, for the training round with a constant global learning rate, the server-side calculates the client aggregation score, filters and removes the client local update with a cosine similarity less than zero with the historical global update sum, for the training round with a negative global learning rate, the client aggregation score is calculated according to the negative value of the cosine similarity of the client local model update with the historical global update sum; finally, the scaled client local model update is weighted and averaged according to the aggregation score to generate the global model update, and iterative training is performed until the final global model is obtained. As a preferred embodiment, in step S1, the server first determines the backdoor image dataset, the model training target, the initialized global model w0 and the hyperparameters (such as the number of training rounds, the learning rate, the number of clients participating in training each time, the proportion of malicious clients, etc.), and broadcasts the initial global model w0 to the clients.

[0066] The application is applicable to data independent and non-independent distributed application scenarios. The independent distributed scenario means that the training data is randomly and uniformly distributed in each client, and the non-independent distributed scenario means that the data is independently generated by the data owner, and the data label types or data sample quantities are unevenly distributed. In this embodiment, the selected data set is divided into a non-independent distributed form by using a Dirichlet distribution with a hyperparameter α=0.05.

[0067] The backdoor image dataset is generated by adding different shapes of triggers in the data samples, and they are assigned to specified clients according to different data distributions. The set of clients C={c1, c2, …, cN} is set, and the local data set of each client is k} is set, and the local data set of each client is The global model training target can be regarded as a process of solving the optimal global model according to the empirical risk minimization, that is, finding a set of optimal model parameters satisfying wherein, represents a d-dimensional weight vector, N i represents the size of the data set D i , the total number of training data F(w; D i ) represents the empirical loss function of the client i.

[0068] As a preferred embodiment, in step S2, each client (including malicious clients) receives the global model w t-1 of the last round issued by the server as its local model w i,t =w t-1 , wherein w i,t represents the local model of the client i in the tth round, and then the client optimizes the current local model using the local data set D i ​The parameter differences between the local models generated by client i in adjacent training rounds are obtained, and this parameter difference is called the client local model update, Δw. i,t =w i,t -w i,t-1 , where w i,t and w i,t-1 Let represent the local models of client i in rounds t and t-1, respectively, and let k represent the client's local learning rate. This represents the client-side gradient loss function.

[0069] After local training is complete, each client (including malicious clients) will update Δw with the generated local model. i,t Uploaded to the server.

[0070] Example 2

[0071] Combination Figure 2 As shown, this embodiment improves the defense against backdoor attacks by adjusting the global learning rate on the server side. In step S3, after each round of global training, the server obtains all local model updates uploaded by clients and calculates the symbolic sum S of the local model updates. t The definition is as follows:

[0072]

[0073] Where n is the number of clients, Δw i,t For the local model update generated by client i in round t, sgn(Δw) i,t ) is a sign function, if Δw i,t If >0, then sgn(Δw) i,t ) = 1, otherwise sgn(Δw i,t ) = -1.

[0074] In this embodiment, a learning rate threshold θ is introduced. If the sum of the signs of the local model updates in the current round S... t If the sum of signs is less than the threshold θ, it indicates that the uploaded local model updates are inconsistent in direction, and the probability of being attacked by a backdoor during local training is high. In this case, the global learning rate on the server side needs to be adjusted to -η to maximize the model training loss and prevent the global model from converging towards a malicious target. If the sum of signs is greater than or equal to the threshold θ, the global learning rate remains unchanged and updates normally.

[0075]

[0076] Where η represents the initial global learning rate on the server side, and S t η represents the sum of signs in the local model updates. θ,t Let θ represent the global learning rate in round t, given a threshold θ.

[0077] It should be noted that the learning rate threshold θ is set according to the number of clients and the proportion of malicious clients, and the value range of θ needs to meet CP+1<θ<C(1-P), where C is the number of clients, and P is the proportion of malicious clients.

[0078] Embodiment 3

[0079] In order to accurately identify the backdoor local update with high attack influence and eliminate its influence on the global model, this embodiment proposes a model aggregation method based on historical global update as the aggregation basis of the current round global model update. Since malicious clients can make their malicious model updates dominate the aggregated global model update by expanding the scale of the malicious model updates. Therefore, before performing the aggregation operation, the size of each client's local model update needs to be processed to scale it to the same order of magnitude as the global model update.

[0080] In combination with Figure 3 As shown in FIG. 4, step S4, before performing the aggregation operation, the size of each client's local model update is processed to scale it to the same order of magnitude as the global model update. On the one hand, the scaling vector ensures that a single local model update will not have too much influence on the aggregated global model update, which to some extent reduces the contribution of malicious clients to the global model. On the other hand, in order to maximize the attack influence, the attacker needs to generate a malicious model update with a large order of magnitude, so the small-scale local model update is more likely to come from an honest client in theory, and appropriately expanding the vector size of the model update helps the benign update to play a role and maximally weaken the effect of the backdoor attack.

[0081] The local model update after scaling processing is calculated by the following formula:

[0082]

[0083] where t represents the model training round, Δw t-1 represents the global model update generated in the last round of training, Δw i,t represents the local model update generated by client i in the tth round, and ||·|| represents the vector two norm;

[0084] Step S5, using the scaled local model update, the server side aggregates to generate a global update, the specific method is:

[0085] S51, the server side obtains all global updates generated in the previous round of the current training task, referred to as the historical global update sum Δw pre,t-1 = Δw1+ Δw2+…+ Δw t-1 , respectively calculate the cosine similarity CS of each client's local model update and the historical global update sumi,t :

[0086]

[0087] where Δw t-1 represents the global model update of the t-1th round, Δw pre,t-1 represents the historical global update sum, represents the local model update of the client i after the scaling processing of the tth round; if the cosine similarity calculation is negative, it indicates that the local model update generated by the current client i has a negative impact on the global model.

[0088] S52, for the training round with the unchanged global learning rate, the malicious local model update opposite to the global update direction, that is, the local model update with the negative cosine similarity calculation, needs to be filtered out in the model aggregation process, specifically, the calculated cosine distance is processed using the Relu operation, which is defined as the aggregation score T i,t of the current client i in the tth round, and is used as the aggregation basis of the global model update of the current round; for the training round with the negative global learning rate, the calculated cosine distance is also processed using the Relu operation, except that, since the global learning rate is negative at this time, in order to make the model update in the direction away from the malicious target, the model training loss needs to be maximized, at this time, the client aggregation score T i,t is calculated according to the negative value of the cosine similarity. i,t It is calculated by the following formula:

[0089]

[0090] where Δw pre,t-1 represents the historical global update sum, represents the local model update of the client i after the scaling processing of the tth round, if CS i,t > 0, then Relu(CS i,t ) = CS i,t , otherwise Relu(CS i,t ) = 0, CS i,t represents the cosine similarity of the local model update of the client i and the historical global update sum.

[0091] S53, according to the local update and the aggregation score after the scaling processing of the client, the server end calculates the weighted average value as the aggregated global model update:

[0092]

[0093] where n represents the number of clients, T i,t is the aggregation score of the current client i in the tth round, scaled local model update of the client i.

[0094] It should be noted that, since the historical global update of the last round cannot be calculated in the first round of training, the median of the local model update is taken as the first round global model update

[0095] Finally, the server obtains the final global model according to the calculated global learning rate and global model update:

[0096]

[0097] Where n represents the number of clients, η represents the initial global learning rate of the server, S t is the symbol sum of the local model update, and Δw t-1 is the global model update generated in the last round of training, CS i,t is the cosine similarity between the local model update of the client i and the historical global update sum, scaled local model update of the client i.

[0098] Embodiment 4

[0099] In combination Figure 4 As shown in the figure, the embodiment provides a backdoor attack defense system based on federated learning, which includes a local training module, a parameter uploading module, a learning rate modification module, a parameter processing module and an aggregation update module. Using the system can realize the backdoor attack defense method based on federated learning described in the previous embodiments, and the specific implementation is not repeated here.

[0100] The local training module is configured to train the local model of the client and generate the corresponding local model update;

[0101] The parameter uploading module is connected with the local training module and is configured to upload the local model update of each client to the server side. The server side receives all the local model updates of the current training round and records them as w t ={Δw i,t , i∈n}, where t is the model training round, n is the number of clients, Δw i,t is the local model update generated by the client i in the tth round.

[0102] The learning rate modification module is connected with the parameter uploading module and is configured to determine whether the global learning rate of the current training needs to be modified according to the local model update of the global client.

[0103] The parameter processing module is connected with the parameter uploading module and is used for processing and unifying the size of the local model update of the client, so that the size is scaled to the same order of magnitude as the global model update, the contribution of the malicious client to the global model is reduced, and the benign model update is promoted to play a role.

[0104] The aggregation update module is connected with the learning rate modification module and the parameter processing module if the system starts the backdoor attack defense method proposed in the application, and is used for dynamic weight aggregation based on historical global updates: for the training round in which the global learning rate remains unchanged, the server end calculates the client aggregation score, filters and removes the client local update whose cosine similarity with the sum of historical global updates is less than zero, and for the training round in which the global learning rate is negative, the client aggregation score is calculated according to the negative value of the cosine similarity; the scaled client local model update is weighted and averaged according to the aggregation score to generate the global model update, and iterative training is performed until the final global model is obtained. If the system does not start the defense method, the aggregation update module is connected with the parameter uploading module, and is used for the default federated average aggregation algorithm (FedAvg) (the algorithm is prior art and is not the design point of the application, and is not described here), which directly uses the average value of each client model update as the global model update to update the global model.

[0105] Embodiment 5

[0106] The embodiment provides an application of a backdoor attack defense system based on federated learning. The backdoor attack defense is performed by the backdoor attack defense system based on federated learning, the global model is updated, the updated global model is used, and a recognition prediction result is output.

[0107] In summary, for the problem of backdoor attack defense in federated learning, the existing methods cannot simultaneously meet the following requirements: 1. The method has sufficient universality and is suitable for different data distribution scenarios. 2. The method is simple and easy to understand, and the calculation overhead should not be too large. 3. The method needs to identify and remove the backdoor local update, rather than only reducing the influence of the backdoor attack. 4. The method needs to minimize the detection error and does not affect the behavior of the honest client, so as to ensure the accuracy of the global model.

[0108] In the backdoor attack, the attacker needs to add a specific trigger in the target dataset and use the poisoned dataset for training, so that the global model identifies the target sample as the target class. For the finally trained global model, the backdoor attack will be effective only when the client model update has a negative impact on the global model, and the impact of the attacker's backdoor update is greater than that of the benign update of the honest client. In order to meet the above four requirements, the present application proposes a backdoor attack defense method and system based on federated learning from two aspects. First, in order to quantify the impact of this round of model update on the global model, the present application adjusts the server-side learning rate according to the sign and information of the local model update, preventing the model from converging to the malicious target. Then, in order to remove suspicious local model updates before aggregating the global model updates and prevent the model from converging to the malicious target, the present application calculates the client aggregation score based on the historical global update, reducing the impact of abnormal local updates on the global model. In addition, model backdoor attack is also a kind of backdoor attack in federated learning. In order to achieve the purpose of attack, the attacker needs to roughly understand the system parameters and the current model training state, so as to replace the global model update with the backdoor model update when the model approaches convergence. However, this attack is generally accompanied by parameter scaling and other operations, which can be easily detected by the defense method based on gradient clipping and adding noise. The operations performed by the attacker need to strike a balance between concealment and the success rate of backdoor attack, and have higher requirements for the attacker's ability. In addition, the defense method of the present application can also be combined with the defense method based on gradient clipping and adding noise to enhance the defense effect. Finally, the present application provides a backdoor attack defense system based on federated learning, including a local training module, a parameter uploading module, a learning rate modification module, a parameter processing module and an aggregation update module. The defense system corresponds to the defense method proposed one by one, reduces the attack accuracy, and improves the anti-backdoor attack ability of the system.

[0109] Of course, the above description is not a limitation of the present application, and the present application is not limited to the above examples. Within the scope of the present application, changes, modifications, additions or substitutions made by those skilled in the art should be within the scope of the present application.

Claims

1. A backdoor attack defense method based on federated learning, characterized in that, The method comprises the following steps: Step S1, determining backdoor image data for training, initializing a training task at a server end and distributing a global model; Step S2, receiving the global model distributed by the server at the client end, training a local model using local data and uploading the generated local model update to the server end; Step S3, adjusting the global learning rate at the server end according to the symbol information of the local model update of the client end; In step S3, whenever a round of global training is completed, the server end obtains all the local model updates uploaded by the clients and calculates the sign sum of the local model updates , which is defined as follows: ； where n is the number of clients, is the local model update generated by client i in round t, ( ) is the sign function, i.e., > 0, then , otherwise ; Introducing a learning rate threshold If the sign sum of the local model updates of the current round is less than the threshold , then the server-side global learning rate is adjusted to , maximizing the model training loss, if the sign sum is greater than or equal to the threshold , then the global learning rate is unchanged, normal update, that is: ； wherein, denotes the server-side initial global learning rate, denotes the summation of the local model updates, denotes a given threshold the global learning rate at the t-th round; Learning rate threshold According to the number of clients and the proportion of malicious clients, The value range of C and P should satisfy Where C is the number of clients and P is the proportion of malicious clients. Step S4, processing the size of the local model update of each client end to scale it to the same order of magnitude as the global model update; Step S5, updating the global model according to the dynamic weight aggregation rule based on the historical global update: Using the scaled local model update, the server end aggregates to generate a global model update. For training rounds with an unchanged global learning rate, the server end calculates a client aggregation score, filters and removes the client local update with a cosine similarity less than zero with the sum of the historical global update. For training rounds with a negative global learning rate, the client aggregation score is calculated according to the negative value of the cosine similarity between the client local model update and the sum of the historical global update. Finally, the scaled client local model update is weighted and averaged according to the aggregation score to generate a global model update, and iterative training is performed until the final global model is obtained.

2. The federated learning based backdoor attack defense method according to claim 1, characterized in that, In step S1, the server first determines a backdoor image dataset, a model training objective, an initial global model and hyperparameters, and broadcasts the initial global model to the clients; wherein the backdoor image dataset is generated by adding triggers of different shapes in the data samples, and they are assigned to designated clients according to different data distributions, and a set of clients is set , each client's local dataset , the global model training objective can be regarded as a process of solving the optimal global model according to empirical risk minimization, that is, finding a set of optimal model parameters satisfying , wherein represents a d-dimensional weight vector, represents the size of the dataset , and the total number of training data , represents the empirical loss function of client i.

3. The federated learning based backdoor attack defense method according to claim 1, characterized in that, In step S2, each client receives the global model of the last round issued by the server As its local model Wherein represents the local model of client i in the tth round, then client i uses the local data set Optimize the current local model , obtain the parameter difference between the local models generated by client i in adjacent rounds of training, which is called client local model update Wherein And respectively represent the local model of client i in the tth round and the (t-1)th round, l represents the client local learning rate, represents the client gradient loss function; after local training, each client uploads the generated local model update To the server.

4. The federated learning based backdoor attack defense method according to claim 1, characterized in that, Step S4, before performing the aggregation operation, the size of each client local model update is first processed to be scaled to the same order of magnitude as the global model update; the local model update after scaling processing is calculated by the following formula: ； wherein, denotes a model training round, denotes a global model update generated by the previous round of training, denotes a client a local model update generated in the t-th round, denotes the vector two-norm.

5. The federated learning based backdoor attack defense method according to claim 4, characterized in that, Step S5, using the scaled local model update, the server end aggregates to generate a global update, the specific method is: S51, the server end acquires all global updates generated in the previous round of the current training task, referred to as a historical global update sum , respectively calculate the cosine similarity of each client local model update and the historical global update sum : ； wherein, denotes the global model update of the t-1th round, denotes the sum of historical global updates, denotes the local model update of the client i after the scaling processing of the tth round; if the cosine similarity calculation is negative, it indicates that the current client the generated local model update has a negative impact on the global model; S52, for the training round with a constant global learning rate, it is necessary to filter out the malicious local model updates opposite to the global update direction in the model aggregation process, that is, the local model updates with negative cosine similarity calculation, specifically using Operation processing the calculated cosine similarity, which is defined as the current client In the first The aggregation score of the round , and as the basis for the aggregation of the current round global model update; for the training round with a negative global learning rate, the calculated cosine similarity is also processed using Operation, the difference is that, since the global learning rate is negative at this time, in order to make the model update in the direction away from the malicious target, it is necessary to maximize the model training loss, at this time, the client aggregation score , the client aggregation score Is calculated by the following formula: ； wherein, represents the sum of the historical global updates, represents the local model update of the client i after the scaling process in the tth round, if then , otherwise , represents the cosine similarity between the local model update of the client i and the sum of the historical global updates; S53, according to the client scaled local update and the aggregation score, the server end calculates the weighted average value as the aggregated global model update: ； where n represents the number of clients, for the current client In the first the aggregated score of the round, is the scaled local model update for client i.

6. The federated learning based backdoor attack defense method according to claim 5, characterized in that, Since the history global update of the last round cannot be calculated at the first round of training, the median of the local model updates is taken as the first round global model update .

7. The federated learning based backdoor attack defense method according to claim 5, characterized in that, In step S4, finally, the server end obtains the final global model according to the calculated global learning rate and global model update: ； where, represents the number of clients, denotes the initial global learning rate on the server side, is the symbol sum of local model updates, is the global model update generated by the last round of training, is the cosine similarity between the local model update of client i and the sum of historical global updates, is the scaled local model update of client i.

8. A backdoor attack defense system based on federated learning, characterized in that, The method for preventing backdoor attacks based on federated learning according to any one of claims 1-7, the system for preventing backdoor attacks based on federated learning comprises a local training module, a parameter uploading module, a learning rate modification module, a parameter processing module and an aggregation update module, The local training module is configured to train a local model at the client end and generate a corresponding local model update. The parameter uploading module is connected with the local training module and is configured to upload the local model updates of the respective clients to a server end, and the server end receives all the local model updates of the current training round as where t is a model training round, n is the number of clients, is the local model update generated by the client i in the tth round. The learning rate modification module is connected with the parameter uploading module and is configured to determine whether the global learning rate of the current training needs to be modified according to the local model update of the global client end. The parameter processing module is connected with the parameter uploading module and is configured to process the size of the local model update of the client end to scale it to the same order of magnitude as the global model update, reduce the contribution of malicious clients to the global model and promote the role of benign model updates. The aggregation update module is connected with the learning rate modification module and the parameter processing module if the system starts the backdoor attack defense method based on federated learning, and is used for dynamic weight aggregation based on historical global update: for a training round with an unchanged global learning rate, the server calculates a client aggregation score, filters and removes a client local update with a cosine similarity less than zero to a historical global update sum, and calculates a client aggregation score according to a negative value of the cosine similarity for a training round with a negative global learning rate; the scaled client local model update is weighted and averaged according to the aggregation score to generate a global model update, and iterative training is performed until a final global model is obtained; if the system does not start the defense method, the aggregation update module is connected with the parameter upload module, and is used for a default federated averaging aggregation algorithm, which directly uses an average value of each client model update as a global model update to update the global model.

Images