Poison attack defense method and device based on federal learning

Abstract

The embodiment of the invention discloses a poisoning attack defense method and device based on federated learning, and the method comprises the steps: a central server randomly selects a client participating in training from all clients participating in federated learning, and synchronizes a global model with the client participating in training, and serves as an initial global model; the clients participating in training perform local training on the initial global model to obtain an uploading gradient of a normal client; the central server performs feature extraction on a long-term historical gradient to obtain a score index sequence so as to remove malicious clients in the clients participating in training, and candidate clients are obtained; and the central server performs model aggregation based on the candidate clients, the clients participating in training and the uploading gradient, and updates model parameters of the global model, the updated global model is used as a new round of initial global model, iteration is performed until a preset number of iterations is reached, and the initial global model is obtained. And obtaining a trained global model.

Description

Technical Field

[0001] The present invention relates to the fields of computer technology and image classification technology, and in particular to a method and device for defending against poisoning attacks based on federated learning. Background Art

[0002] Poisoning attack defense technology is a defense method in which the server identifies abnormal updates from malicious attackers and deletes or weights them according to aggregation rules.

[0003] Traditional defense technology can identify some abnormal updates affected by poisoning attacks, but it still faces problems such as insufficient use of historical data and unmet privacy requirements. With the development of technology, historical gradient technology and distance-based defense technology have emerged to solve the shortcomings of traditional defense technology. Among them, historical gradient technology, as an important technology of deep learning, can effectively capture the long-term dependencies and information flow in historical data. Applying this technology to traditional defense technology can improve the latter's ability level in terms of detection effect and defense capability. However, historical gradient technology cannot defend against attacks in the long-term historical dimension. In each round, the server only identifies the client's update based on the current round of data. The long and short historical gradient technology can solve the above problems, but the collection of long-term data may lead to unmet privacy requirements, and new technologies are needed to fully analyze the characteristics of historical gradients. Distance-based defense technology is a simple and fast defense method, which calculates the distance between uploaded model parameters or gradients or the similarity between model parameters. Through the calculation of the above distance and similarity, the updates with the farthest distance and the smallest similarity are deleted or weighted. At present, this technology is mainly represented by Krum. However, distance-based defense technology cannot defend against long-term attacks and makes insufficient use of historical data, which results in some long-standing malicious behaviors not being identified in a timely manner and poses a risk of privacy leakage.

[0004] Based on this, this specification provides a poisoning attack defense method and device based on federated learning. Summary of the invention

[0005] The embodiments of this specification provide a method and device for defending against poisoning attacks based on federated learning, which are used to solve the following technical problems: the historical gradient technology cannot defend against attacks in the long-term historical dimension. In each round, the server only identifies the client's updates based on the current round of data. The long and short historical gradient technology can solve the above problems, but the collection of long-term data may result in privacy requirements not being met, and new technologies are needed to fully analyze the characteristics of historical gradients. Distance-based defense technology cannot defend against long-term attacks, and insufficient use of historical data leads to the inability to identify certain long-standing malicious behaviors in a timely manner, and there is a risk of privacy leakage.

[0006] To solve the above technical problems, the embodiments of this specification are implemented as follows:

[0007] The embodiment of this specification provides a poisoning attack defense method based on federated learning, which is implemented by a client and a central server. The poisoning attack defense method includes:

[0008] The central server randomly selects a client participating in training from all clients participating in federated learning, and synchronizes a global model with the client participating in training as an initial global model;

[0009] The clients participating in the training perform local training on the initial global model to obtain uploaded gradients of normal clients among the clients participating in the training;

[0010] The central server performs feature extraction on the long-term historical gradient to obtain a score index sorting, wherein the score index sorting is used to remove malicious clients from the clients participating in the training to obtain candidate clients, and the long-term historical gradient is obtained based on the uploaded gradient;

[0011] The central server performs model aggregation based on the candidate clients, the clients participating in training, and the uploaded gradients, and updates the model parameters of the global model. The updated global model is used as the initial global model of a new round and iterated until a preset number of iterations is reached to obtain a trained global model. The trained global model is used for poisoning attack defense.

[0012] The embodiment of this specification also provides a poisoning attack defense device based on federated learning, the poisoning attack defense device is implemented by a client and a central server, and the poisoning attack defense device based on federated learning includes:

[0013] Initialization module, the central server randomly selects clients participating in training from all clients participating in federated learning, and synchronizes the global model with the clients participating in training as the initial global model;

[0014] A local training module, wherein the client participating in the training performs local training on the initial global model to obtain uploaded gradients of normal clients among the clients participating in the training;

[0015] A screening module, wherein the central server extracts features from the long-term historical gradients to obtain score index sorting, wherein the score index sorting is used to remove malicious clients from the clients participating in the training to obtain candidate clients, and the long-term historical gradients are obtained based on the uploaded gradients;

[0016] Iteration module, the central server performs model aggregation based on the candidate clients, the clients participating in training and the uploaded gradients, and updates the model parameters of the global model. The updated global model is used as the initial global model of a new round and iterates until a preset number of iterations is reached to obtain a trained global model. The trained global model is used for poisoning attack defense.

[0017] The poisoning attack defense method based on federated learning provided in the embodiments of the present specification is implemented by a client and a central server. The central server randomly selects a client participating in training from all clients participating in federated learning, and synchronizes a global model with the client participating in training as an initial global model; the client participating in training performs local training on the initial global model to obtain the uploaded gradients of normal clients among the clients participating in training; the central server performs feature extraction on the long-term historical gradient to obtain a score index sorting, and the score index sorting is used to remove malicious clients from the clients participating in training to obtain candidate clients, and the long-term historical gradient is obtained based on the uploaded gradient; the central server performs model aggregation based on the candidate clients, the clients participating in training, and the uploaded gradient, and updates the model parameters of the global model. The updated global model is used as the initial global model of a new round, and it is iterated until a preset number of iterations is reached to obtain a trained global model. The trained global model is used for poisoning attack defense, which can achieve better training effect, smaller impact caused by attacks, more accurate malicious client detection results, and can achieve malicious update identification and privacy protection requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0019] Figure 1 A schematic diagram of the system architecture of a poisoning attack defense method based on federated learning provided in an embodiment of this specification;

[0020] Figure 2 A flowchart of a method for defending against poisoning attacks based on federated learning provided in an embodiment of this specification;

[0021] Figure 3 A flowchart showing the perturbation principle of the short-term historical gradient provided in the embodiments of this specification;

[0022] Figure 4 A flow chart of the perturbation principle of the long-term historical gradient provided in the embodiments of this specification;

[0023] Figure 5 A framework diagram of a poisoning attack defense method based on federated learning provided in an embodiment of this specification;

[0024] Figure 6 A schematic diagram of a poisoning attack defense device based on federated learning provided in an embodiment of this specification. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0026] Figure 1 A schematic diagram of the system architecture of a poisoning attack defense method based on federated learning provided in the embodiments of this specification. Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0027] The terminal devices 101, 102, 103 interact with the server 105 through the network 104 to receive or send messages, etc. Various client applications may be installed on the terminal devices 101, 102, 103. For example, a dedicated program such as a poisoning attack defense method based on federated learning may be implemented.

[0028] The terminal devices 101, 102, 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various dedicated or general electronic devices, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, etc. When the terminal devices 101, 102, 103 are software, they may be installed in the electronic devices listed above. They may be implemented as multiple software or software modules (e.g., multiple software or software modules for providing distributed services), or they may be implemented as a single software or software module.

[0029] The server 105 may be a server that provides various services, such as a backend server that provides services for client applications installed on the terminal devices 101, 102, and 103. For example, the server may perform poisoning attack defense based on federated learning, so as to display the poisoning attack defense result based on federated learning on the terminal device servers 101, 102, and 103.

[0030] The server 105 may be hardware or software. When the server 105 is hardware, it may be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server 105 is software, it may be implemented as multiple software or software modules (e.g., multiple software or software modules for providing distributed services), or as a single software or software module.

[0031] Figure 2 The following is a flowchart of a method for defending against poisoning attacks based on federated learning provided in the embodiments of this specification. From the program perspective, the execution subject of the process can be a program installed on an application server or an application terminal. It can be understood that the method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. Figure 2 As shown, the poisoning attack defense method based on federated learning is applied to the server, that is, the central server. The poisoning attack defense method based on federated learning includes:

[0032] Step S201: The central server randomly selects clients participating in training from all clients participating in federated learning, and synchronizes a global model with the clients participating in training as an initial global model.

[0033] Clients participating in training, that is, clients participating in federated learning, refer to devices or nodes that perform local computing tasks in the network and collaborate with the central server to jointly train machine learning models.

[0034] In the embodiments of this specification, the clients participating in the training are the server randomly selecting a part of all the clients as the subset N(i) of the clients in each round of federated learning to participate in the round of federated learning. At the same time, the server synchronizes the global model with the clients participating in the training. In a specific embodiment, the number of all clients is not less than 50, and the proportion of the randomly selected subset of clients from all the clients is 10%-30%. In one embodiment of this specification, 20% of all the clients are selected as the subset of clients. The subset of clients may be benign clients or malicious clients. It is precisely because of the existence of malicious clients that poisoning attack defense is needed.

[0035] The global model refers to the central model that is jointly trained by clients in the federated learning framework. The central model is managed by the server. Its main function is to share learning results among various clients, thereby achieving joint optimization of the model without directly exchanging local client data.

[0036] In the embodiments of this specification, the global model is a model for processing picture data, and / or image data, and / or text data, and / or for data classification. In practical applications, the types of data processed by the global model are all types in a broad sense. For example, picture data also includes data that can be converted into pictures, image data also includes data that can be converted into images, and text data also includes data that can be converted into text. For example, video image data can be converted into multiple image data.

[0037] After the central server synchronizes the global model with the clients participating in the training, the global model is used as the initialization model for subsequent iterations.

[0038] Step S203: the clients participating in the training perform local training on the initial global model to obtain uploaded gradients of normal clients among the clients participating in the training.

[0039] The client participating in the training performs local training on the initial global model to obtain the uploaded gradient of a normal client among the clients participating in the training, specifically including:

[0040] The client participating in the training performs local training on the initial global model using local data, uses cross entropy as a loss function, and aims to minimize the initial global model parameter w to complete the local training;

[0041] Based on the initial global model that has completed the local training, obtaining the uploaded gradients of normal clients among the clients participating in the training;

[0042] The expression of the uploaded gradient is:

[0043]

[0044] in,

[0045] is the uploaded gradient of the tth round of the i-th client;

[0046] w t is the model parameter of the client participating in the training in the tth round;

[0047] x represents the actual data of the client participating in the training;

[0048] y represents the label data of the client participating in the training;

[0049] h w () represents the prediction function of the model involved in training;

[0050] h i,w (x) represents the predicted output made by the model participating in the training on client i based on the input data x and the current model parameters w;

[0051] l represents the loss function.

[0052] In the embodiments of this specification, the long-term historical gradient can be generated in the central server, or uploaded to the central server after being formed by the client participating in the training. Specifically, the long-term historical gradient is obtained based on the uploaded gradient, specifically including:

[0053] The client participating in the training performs gradient aggregation on the uploaded gradient and the sliding window size of the short-term historical gradient of the client participating in the training to obtain the short-term historical gradient of the client participating in the training. The calculation formula of the short-term historical gradient is:

[0054]

[0055] in,

[0056] Represents the short-term historical gradient of the i-th client participating in the training;

[0057] S represents the sliding window size of the short-term historical gradient;

[0058] E represents the local training round;

[0059] t represents the current training round;

[0060] k represents the kth training round of the sliding window of short-term historical gradients;

[0061] The client participating in the training perturbs the short-term historical gradient, and filters and scales the perturbation parameters to meet the differential privacy requirements, thereby updating the short-term historical gradient and obtaining an updated short-term historical gradient;

[0062] The central server obtains the long-term historical gradient based on the received updated short-term historical gradient; or

[0063] The client participating in the training performs gradient aggregation based on the uploaded gradient and the sliding window size of the long-term historical gradient of the client participating in the training to obtain the long-term historical gradient of the client participating in the training. The long-term historical gradient is uploaded to the central server. The calculation formula of the long-term historical gradient is:

[0064]

[0065] in,

[0066] Represents the long-term historical gradient of the i-th client participating in the training;

[0067] Represents the uploaded gradient of the i-th client participating in the training;

[0068] represents the uploaded gradient of the i-th client participating in the training in the j-th training round of the long-term sliding window;

[0069] L is the sliding window size of the long-term historical gradient;

[0070] j represents the jth training epoch of the sliding window of long-term historical gradients.

[0071] The label y is used to identify or mark the data. The specific content of the label y varies with the training data. In one embodiment of the present specification, when the training data is the MNIST data set, the label y is data between 0-9, and the label indicates the true value of the handwritten number represented by the image. When the training data is the CIFAR-10 data set, the label represents ten categories, specifically, including: airplane, car, bird, cat, deer, dog, frog, horse, ship and truck.

[0072] Malicious clients have different attack methods depending on the type of attack. Model attacks directly modify the uploaded gradients. Data poisoning attacks modify the label y so that it does not match the actual data x, thereby affecting the gradient obtained during training.

[0073] In this specification, short-term historical gradient refers to the gradient information in the most recent iterations. Long-term historical gradient refers to the gradient information that takes into account a longer time span. In a specific embodiment, the determination of long-term historical gradient and short-term historical gradient can be determined according to the business scenario and is not limited here.

[0074] In order to further understand the short-term historical gradient and the long-term historical gradient, the principles of the short-term historical gradient and the long-term historical gradient will be explained separately below.

[0075] Figure 3 The perturbation principle flow chart of the short-term historical gradient provided in the embodiment of this specification. The principle of the short-term historical gradient is as follows: (1) Collect short-term historical gradients Collect short-term historical gradients from a series of weight matrices. These weight matrices W1, W2, ..., Wk represents the model parameters at different time points. (2) Calculate the difference value: Calculate the gradient difference between two adjacent weight matrices. These difference values ​​g1, g2, ..., g k-1 Reflects the changes of model parameters over time (3) Re-organize gradients into patches: According to the gradient difference values ​​obtained in the previous step, reorganize them into a block structure. Each block in the block structure represents the changes of model parameters in a certain aspect. (4) Perturbation: Finally, by applying perturbations to these block structures, differential privacy protection of gradients is achieved. The perturbed gradients can be used for subsequent model training while protecting the privacy of the original data.

[0076] Figure 4 The perturbation principle flow chart of the long-term historical gradient provided in the embodiment of this specification. Figure 4 As shown in the figure, Start: The process starts. Dataset: Use the dataset for training. Gradients of t-th round: Calculate the gradients g1, g2, ..., g of the current round t n ; Dynamic smoothing: Dynamically smooth these gradients to generate long-term historical gradients Rectified conv feature maps: Rectified convolution feature maps. Flattened gradients: Flatten the rectified feature maps to get the flattened gradients. Random sampling: Randomly sample these flattened gradients. SVD: Perform singular value decomposition (SVD) on the sampled gradients to get the left singular vector matrix U, the right singular vector matrix V and the singular values. Calculate scores: Calculate scores based on singular values ​​and centralized data Filter malicious clients by scores: Use these scores to filter malicious clients.

[0077] In the embodiments of this specification, on the basis of accelerating model convergence through short-term historical gradient aggregation, the characteristics of multi-round smoothing of short-term historical gradients are utilized to eliminate the randomness of single-round local gradient values, which helps the model converge to the expected stable state.

[0078] In the embodiment of this specification, the client participating in the training perturbs the short-term historical gradient, and filters and scales the perturbation parameters to meet the differential privacy requirements, thereby updating the short-term historical gradient and obtaining an updated short-term historical gradient, specifically including:

[0079] A Laplace noise matrix B is randomly generated with a mean value of 0 and a standard deviation of σ as a disturbance;

[0080] If the weight difference matrix of the short-term historical gradient is less than the preset threshold of each dimension, the disturbance is added to the short-term historical gradient as a candidate short-term historical gradient, and the candidate short-term historical gradient is clipped using the differential privacy method with gradient depersonalization to ensure that the gradient norm of the candidate short-term historical gradient is less than the gradient clipping upper limit C, thereby updating the short-term historical gradient to obtain the updated short-term historical gradient, and the preset threshold of each dimension is 1 / 2 of the standard deviation;

[0081] If the weight difference matrix of the short-term historical gradient is greater than or equal to the preset threshold of each dimension, the short-term historical gradient is clipped using the gradient depersonalization differential privacy method to ensure that the gradient norm of the short-term historical gradient is less than the gradient clipping upper limit C, thereby updating the short-term historical gradient as the updated short-term historical gradient.

[0082] In the embodiments of this specification, the standard deviation σ is inversely proportional to the privacy budget ε. The privacy budget ε is determined before training. As a hyperparameter, the privacy budget ε is related to the sensitivity of the database and the accuracy requirements of the model. The determination of the privacy budget ε depends on the user's attention to privacy. If you want to ensure higher privacy protection, you will choose a small privacy budget ε, which will lead to more noise and reduce the accuracy of the model. The privacy budget ε is determined by the data set and the model itself, and is adjusted according to the specific application.

[0083] In the embodiments of this specification, gradient depersonalization differential privacy is a method of combining gradient depersonalization with differential privacy to protect data privacy. In order to prevent the gradient norm (size) from exceeding the predetermined gradient clipping upper limit C, it is necessary to reduce it in proportion so that the gradient norm does not exceed the gradient clipping upper limit C.

[0084] The calculation formula for the gradient clipping upper limit C is:

[0085]

[0086] in,

[0087] Represents the gradient of the i-th client participating in the training;

[0088] express The L2 norm of ;

[0089] Represents the gradient of the i-th client participating in the training after clipping.

[0090] In a specific embodiment, Then the gradient will not change and no clipping is required; when When , the gradient is scaled by the scaling factor In the embodiment of this specification, the calculation formula of the weight difference matrix of the short-term historical gradient is:

[0091]

[0092] in,

[0093] W * The weight difference matrix representing the short-term historical gradient;

[0094] represents the short-term historical gradient of the i-th client in the t-th round;

[0095] represents the short-term historical gradient of the i-th client in the t-1th round.

[0096] In the embodiment of the present specification, the threshold value of each dimension preset based on the weight difference matrix of the short-term historical gradient is used to determine whether to add the disturbance to the short-term historical gradient, specifically including:

[0097] If the weight difference matrix of the short-term historical gradient is less than the preset threshold of each dimension, the disturbance is added to the short-term historical gradient, and the preset threshold of each dimension is 1 / 2 of the standard deviation.

[0098] In the embodiment of the present specification, each dimension, that is, each layer of the model, has a threshold value |γ| of each dimension that is preset, but generally speaking, the threshold value |γ| of each dimension is determined by the model itself.

[0099] When W * <|γ|, the weight difference of the current dimension changes less with training. If there is a malicious attack, it may be hidden in these model parameter kernels that are not affected by training changes. Perturbations are added to them to reduce their possible long-term attack effects. At this time, they are added to the short-term historical gradient.

[0100] The embodiments of this specification perturb the short-term historical gradient of the local model training of the benign client, and minimize the loss of the local benign task through the judgment matrix to maintain the performance of the benign task. And by perturbing the core of the Hessian matrix, the attack impact is prevented from being hidden in the historical gradient of the benign device, thereby reducing the impact of malicious attacks.

[0101] Step S205: The central server performs feature extraction on the long-term historical gradient to obtain a score index sorting, wherein the score index sorting is used to remove malicious clients from the clients participating in the training and obtain candidate clients, and the long-term historical gradient is obtained based on the uploaded gradient.

[0102] In the embodiment of the present specification, the central server extracts features from the long-term historical gradient to obtain a score index sorting, and the score index sorting is used to remove malicious clients from the clients participating in the training to obtain candidate clients, specifically including:

[0103] The central server performs singular value decomposition on the long-term historical gradient to extract features to obtain a score sorting index;

[0104] Based on the score sorting index, malicious clients are removed from the clients participating in the training to obtain the candidate clients.

[0105] In the embodiment of this specification, the central server performs singular value decomposition on the long-term historical gradient to extract features to obtain a score sorting index, specifically including:

[0106] The central server obtains the average value of the long-term historical gradient after it is expanded by dimension;

[0107] The difference between the average value and the average value of all clients participating in the training is used as the centralized gradient of each client participating in the training;

[0108] Performing singular value decomposition on the centralized gradient to obtain a right singular matrix;

[0109] Obtaining an outlier score based on the centralized gradient, the average value, and the right singular matrix;

[0110] The outlier value scores are sorted from large to small to obtain the score sorting index.

[0111] In the embodiments of this specification, the calculation formula of the abnormal value score is:

[0112] in,

[0113] Score(i) represents the outlier value of the i-th client;

[0114] represents the long-term historical gradient of the i-th client;

[0115] μ represents the average value of the long-term historical gradient after expansion according to the dimension;

[0116] v represents a right singular matrix.

[0117] In the embodiment of the present specification, removing malicious clients from the clients participating in the training based on the score sorting index to obtain the candidate clients specifically includes:

[0118] Based on the score sorting index, determining a preset ratio as malicious clients among the clients participating in the training;

[0119] The malicious client is removed from the clients participating in the training to obtain the candidate client.

[0120] In a specific embodiment, first, based on the uploaded long-term historical gradient of client N(i), the average value after flattening by dimension is calculated.

[0121] Next, calculate the centralization gradient for each client That is, the difference between it and the average value of all clients:

[0122]

[0123] Further, the difference between each client and the average gradient in multiple dimensions is the centralized gradient. Extract the specificity of its features. The specific method is to centralize the gradient Perform singular value decomposition to obtain left and right singular matrices, where the right singular matrix v represents the specificity of the data, which is flipped and aligned with μ.

[0124] Finally, the anomaly score Score(i) is calculated according to the specificity of its features as follows, and the anomaly scores are sorted to construct the score sorting index S(i), and then malicious clients are removed based on the score sorting index S(i).

[0125]

[0126] The sorting index is obtained based on the score, which is the correspondence between the score and the model parameter. After the score sorting index is arranged in order from high to low, the part with higher score in the score sorting index is selected and removed to remove malicious clients. Generally, 10-30% of the higher scores in the score sorting index are selected for removal, preferably 20%.

[0127] Step S207: The central server performs model aggregation based on the candidate clients, the clients participating in training, and the uploaded gradients, and updates the model parameters of the global model. The updated global model is used as the initial global model of a new round and iterated until a preset number of iterations is reached to obtain a trained global model. The trained global model is used for poisoning attack defense.

[0128] In the embodiment of this specification, the calculation formula of the model parameters of the updated global model is:

[0129]

[0130] in,

[0131] ω t Represents the model parameters of the tth round of the global model of federated learning;

[0132] N(b) represents the set of candidate clients;

[0133] D represents the amount of data from clients participating in model aggregation;

[0134] represents the uploaded gradient of the tth round of the i-th candidate client.

[0135] In this embodiment, the parameters of the updated global model obtained are used as parameters for a new round of model iterative training, that is, initialization parameters.

[0136] In the embodiments of this specification, the preset number of iterations is determined depending on the data set used for global model training and the complexity of the model. In one embodiment of this specification, the MNIST data set is used as training data, a simple CNN architecture with two convolutional layers is used as a model, and the preset number of iterations is 100; in another embodiment of this specification, the CIFAR-10 data set is used as training data, ResNet18 is used as a model, and the preset number of iterations is 300.

[0137] In order to further understand the poisoning attack defense method based on federated learning in the embodiment of this specification, it will be explained in conjunction with a specific framework below.

[0138] Figure 5 The framework diagram of a poisoning attack defense method based on federated learning provided in the embodiments of this specification. Figure 5As shown, the central server synchronizes the global model to the clients participating in the training, and then the clients participating in the training perform multiple rounds of local training based on local data. Based on multiple rounds of local training, the uploaded gradients of normal clients are obtained, and short-term historical gradients are further generated. After adding disturbances to the short-term historical gradients, they are uploaded to the central server. The central server generates long-term historical gradients. Subsequently, the central server removes malicious clients based on the generated long-term historical gradients to obtain candidate clients; then based on the candidate clients, the global model is updated, and the trained global model is obtained through paper training. Of course, the long-term historical gradients can also be generated by the clients participating in the training and then uploaded to the central server.

[0139] The poisoning attack defense method based on federated learning provided in the embodiments of the present specification is implemented by a client and a central server. The central server randomly selects a client participating in training from all clients participating in federated learning, and synchronizes a global model with the client participating in training as an initial global model; the client participating in training performs local training on the initial global model to obtain the uploaded gradients of normal clients among the clients participating in training; the central server performs feature extraction on the long-term historical gradient to obtain a score index sorting, and the score index sorting is used to remove malicious clients from the clients participating in training to obtain candidate clients, and the long-term historical gradient is obtained based on the uploaded gradient; the central server performs model aggregation based on the candidate clients, the clients participating in training, and the uploaded gradient, and updates the model parameters of the global model. The updated global model is used as the initial global model of a new round, and it is iterated until a preset number of iterations is reached to obtain a trained global model. The trained global model is used for poisoning attack defense, which can achieve better training effect, smaller impact caused by attacks, more accurate malicious client detection results, and can achieve malicious update identification and privacy protection requirements.

[0140] The above content describes in detail a method for defending against poisoning attacks based on federated learning. Correspondingly, this specification also provides a device for defending against poisoning attacks based on federated learning, such as Figure 6 shown. Figure 6 A schematic diagram of a poisoning attack defense device based on federated learning provided in an embodiment of this specification, the poisoning attack defense device is implemented by a client and a central server, and the poisoning attack defense device includes:

[0141] Initialization module 601, the central server randomly selects a client participating in training from all clients participating in federated learning, and synchronizes a global model with the client participating in training as an initial global model;

[0142] A local training module 603, wherein the client participating in the training performs local training on the initial global model to obtain the uploaded gradients of normal clients among the clients participating in the training;

[0143] Screening module 605, the central server extracts features from the long-term historical gradient to obtain a score index ranking, the score index ranking is used to remove malicious clients from the clients participating in the training, and obtain candidate clients, the long-term historical gradient is obtained based on the uploaded gradient;

[0144] Iteration module 607, the central server performs model aggregation based on the candidate clients, the clients participating in training and the uploaded gradients, and updates the model parameters of the global model. The updated global model is used as the initial global model of a new round and iterates until a preset number of iterations is reached to obtain a trained global model. The trained global model is used for poisoning attack defense.

[0145] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0146] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0147] The apparatus, electronic device, non-volatile computer storage medium and method provided in the embodiments of this specification correspond to each other, and therefore, the apparatus, electronic device and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding apparatus, electronic device and non-volatile computer storage medium will not be repeated here.

[0148] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0149] The specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media including storage devices.

[0150] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0151] The above is only an embodiment of this specification and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A poisoning attack defense method based on federated learning, characterized in that: The poisoning attack defense method is implemented by the client and the central server together, and the poisoning attack defense method includes: The central server randomly selects a client participating in training from all clients participating in federated learning, and synchronizes a global model with the client participating in training as an initial global model; The clients participating in the training perform local training on the initial global model to obtain uploaded gradients of normal clients among the clients participating in the training; The central server performs feature extraction on the long-term historical gradient to obtain a score index sorting, wherein the score index sorting is used to remove malicious clients from the clients participating in the training to obtain candidate clients, and the long-term historical gradient is obtained based on the uploaded gradient; The central server performs model aggregation based on the candidate clients, the clients participating in training, and the uploaded gradients, and updates the model parameters of the global model. The updated global model is used as the initial global model of a new round and iterated until a preset number of iterations is reached to obtain a trained global model. The trained global model is used for poisoning attack defense.

2. The poisoning attack defense method according to claim 1, characterized in that: The client participating in the training performs local training on the initial global model to obtain the uploaded gradient of a normal client among the clients participating in the training, specifically including: The client participating in the training performs local training on the initial global model using local data, uses cross entropy as a loss function, and aims to minimize the initial global model parameter w to complete the local training; Based on the initial global model that has completed the local training, obtaining the uploaded gradients of normal clients among the clients participating in the training; The expression of the uploaded gradient is: in, is the uploaded gradient of the tth round of the i-th client; w t is the model parameter of the client participating in the training in the tth round; x represents the actual data of the client participating in the training; y represents the label data of the client participating in the training; h w () represents the prediction function of the model involved in training; h i,w (x) represents the predicted output made by the model participating in the training on client i based on the input data x and the current model parameters w; l represents the loss function.

3. The poisoning attack defense method according to claim 1, characterized in that: The long-term historical gradient is obtained based on the uploaded gradient, specifically including: The client participating in the training performs gradient aggregation on the uploaded gradient and the sliding window size of the short-term historical gradient of the client participating in the training to obtain the short-term historical gradient of the client participating in the training. The calculation formula of the short-term historical gradient is: in, Represents the short-term historical gradient of the i-th client participating in the training; S represents the sliding window size of the short-term historical gradient; E represents the local training round; t represents the current training round; k represents the kth training round of the sliding window of short-term historical gradients; The client participating in the training perturbs the short-term historical gradient, and filters and scales the perturbation parameters to meet the differential privacy requirements, thereby updating the short-term historical gradient and obtaining an updated short-term historical gradient; The central server obtains the long-term historical gradient based on the received updated short-term historical gradient; or The client participating in the training performs gradient aggregation based on the uploaded gradient and the sliding window size of the long-term historical gradient of the client participating in the training to obtain the long-term historical gradient of the client participating in the training. The long-term historical gradient is uploaded to the central server. The calculation formula of the long-term historical gradient is: in, Represents the long-term historical gradient of the i-th client participating in the training; Represents the uploaded gradient of the i-th client participating in the training; represents the uploaded gradient of the i-th client participating in the training in the j-th training round of the long-term sliding window; L is the sliding window size of the long-term historical gradient; j represents the jth training epoch of the sliding window of long-term historical gradients.

4. The poisoning attack defense method according to claim 3, characterized in that: The client participating in the training perturbs the short-term historical gradient, and filters and scales the perturbation parameters to meet the differential privacy requirements, thereby updating the short-term historical gradient and obtaining an updated short-term historical gradient, specifically including: randomly generating a Laplace noise matrix B as a perturbation according to an average value of 0 and a standard deviation of σ; If the weight difference matrix of the short-term historical gradient is less than the preset threshold of each dimension, the disturbance is added to the short-term historical gradient as a candidate short-term historical gradient, and the candidate short-term historical gradient is clipped using the differential privacy method with gradient depersonalization to ensure that the gradient norm of the candidate short-term historical gradient is less than the gradient clipping upper limit C, thereby updating the short-term historical gradient to obtain the updated short-term historical gradient, and the preset threshold of each dimension is 1 / 2 of the standard deviation; If the weight difference matrix of the short-term historical gradient is greater than or equal to the preset threshold of each dimension, the short-term historical gradient is clipped using the gradient depersonalization differential privacy method to ensure that the gradient norm of the short-term historical gradient is less than the gradient clipping upper limit C, thereby updating the short-term historical gradient as the updated short-term historical gradient.

5. The poisoning attack defense method according to claim 1, characterized in that: The central server extracts features from the long-term historical gradients to obtain score index sorting, and the score index sorting is used to remove malicious clients from the clients participating in the training to obtain candidate clients, specifically including: The central server performs singular value decomposition on the long-term historical gradient to extract features to obtain a score sorting index; Based on the score sorting index, malicious clients are removed from the clients participating in the training to obtain the candidate clients.

6. The poisoning attack defense method according to claim 5, characterized in that: The central server performs singular value decomposition on the long-term historical gradient to extract features to obtain a score sorting index, specifically including: The central server obtains the average value of the long-term historical gradient after it is expanded by dimension; The difference between the average value and the average value of all clients participating in the training is used as the centralized gradient of each client participating in the training; Performing singular value decomposition on the centralized gradient to obtain a right singular matrix; Obtaining an outlier score based on the centralized gradient, the average value, and the right singular matrix; The outlier value scores are sorted from large to small to obtain the score sorting index.

7. The poisoning attack defense method according to claim 6, characterized in that: The calculation formula of the outlier score is: in, Score(i) represents the outlier value of the i-th client; represents the long-term historical gradient of the i-th client; μ represents the average value of the long-term historical gradient after expansion according to the dimension; v represents a right singular matrix.

8. The poisoning attack defense method according to claim 5, characterized in that: The step of removing malicious clients from the clients participating in the training based on the score sorting index to obtain the candidate clients specifically includes: Based on the score sorting index, determining a preset ratio as malicious clients among the clients participating in the training; The malicious client is removed from the clients participating in the training to obtain the candidate client.

9. The poisoning attack defense method according to claim 1, characterized in that: The calculation formula of the model parameters of the updated global model is: in, ω t Represents the model parameters of the tth round of the global model of federated learning; N(b) represents the set of candidate clients; D represents the amount of data from clients participating in model aggregation; represents the uploaded gradient of the tth round of the i-th candidate client.

10. A poisoning attack defense device based on federated learning, characterized in that: The poisoning attack defense device is implemented by the client and the central server. The poisoning attack defense device based on federated learning includes: Initialization module, the central server randomly selects clients participating in training from all clients participating in federated learning, and synchronizes the global model with the clients participating in training as the initial global model; A local training module, wherein the client participating in the training performs local training on the initial global model to obtain uploaded gradients of normal clients among the clients participating in the training; A screening module, wherein the central server extracts features from the long-term historical gradients to obtain score index sorting, wherein the score index sorting is used to remove malicious clients from the clients participating in the training to obtain candidate clients, and the long-term historical gradients are obtained based on the uploaded gradients; Iteration module, the central server performs model aggregation based on the candidate clients, the clients participating in training and the uploaded gradients, and updates the model parameters of the global model. The updated global model is used as the initial global model of a new round and iterates until a preset number of iterations is reached to obtain a trained global model. The trained global model is used for poisoning attack defense.

Images