A federated learning image classification method capable of defending against byzantine attacks

Abstract

The application discloses a federated learning image classification method capable of defending against a Byzantine attack, utilizes feature outputs of each local image classification model on a virtual data set, processes a Gram matrix based on Median and MAD to eliminate malicious models subjected to the Byzantine attack, and then selects node training gradients submitted by normal worker nodes to participate in aggregation, so that the influence of the Byzantine attack on a global model is completely avoided, and the global model has high classification accuracy in normal image data. In addition, the application belongs to an unsupervised method, is suitable for a federated learning scene, and can be widely applied in federated learning defense, and the robustness of the federated learning is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of machine learning security, and particularly relates to a federated learning image classification method capable of defending against Byzantine attacks. BACKGROUND

[0002] Data islands and data privacy are one of the main factors restricting the development of artificial intelligence technology. The proposal of federated learning breaks through this limitation. Federated learning is a machine learning framework for distributed data, allowing model training participants to collaboratively train a global model without sharing data. By protecting data privacy, federated learning breaks down data silos, accelerates the training of machine learning models, and is suitable for non-balanced and non-independent and identically distributed (Non-IID) data. Many machine learning tasks can be trained using federated learning, including image classification tasks.

[0003] In federated learning, the typical training structure is the parameter server structure (PS), which includes server nodes (Server) and worker nodes (Worker). When using the PS structure for model training, the following four steps are usually included: first, each Worker node collects image data for local training and receives a global model from the Server node, uses the received global model for local training to obtain the gradient of the local model; second, the Worker node sends the gradient of the local model to the Server node for updating the global model; third, the Server node updates the global model for the next round of iteration using the received gradients of each Worker node; fourth, the Server node broadcasts the updated global model to all Worker nodes and starts the next round of iteration update process.

[0004] In federated learning based on the PS structure, Worker nodes are usually deployed on edge nodes, while Server nodes are deployed in the cloud. Federated average (FedAvg) is a classic aggregation algorithm, and its expression is:

[0005]

[0006] where w t is the parameter update of the global model in the tth round, Δw k is the gradient of Worker node k, K is the number of Worker nodes participating in aggregation, n k is the data volume of the local data set of Worker node k, n is the total data volume, and has

[0007] Worker nodes use the latest global model they receive, combined with local data and training methods (such as Stochastic Gradient Descent, SGD), to train the model, and then upload the gradients to the Server nodes for aggregation.

[0008] However, the distributed and privacy-preserving nature of federated learning makes it vulnerable to numerous security issues, such as Byzantine attacks. During model training, worker nodes controlled by attackers can send task model gradients to server nodes. When the server node uses these worker node gradients to update the global model, the performance of the global model is compromised, resulting in poor performance when using this global model for image classification prediction. Summary of the Invention

[0009] In view of this, the present invention provides a federated learning image classification method that can defend against Byzantine attacks, effectively defending against Byzantine attacks that may occur in image classification based on federated learning methods, and ensuring high classification accuracy.

[0010] This invention provides a federated learning image classification method that can defend against Byzantine attacks, comprising the following steps:

[0011] Step 1: Determine the local image data for model training on each Worker node; the Server node generates a virtual dataset containing training and test sets and completes the initialization of the global image classification model; the Server node distributes the global image classification model and training set to each Worker node.

[0012] Step 2: Each Worker node uses local image data and training set to train the global image classification model, obtains the node training gradient, and uploads the node training gradient to the Server node.

[0013] Step 3: The Server node uses the node training gradients of each Worker node and the global image classification model from the previous round to generate a local image classification model for each Worker node; the feature outputs h of the local image classification models of each Worker node on the test set are merged to form a feature matrix H, and the Gram feature matrix R is calculated; the upper limit threshold T of the deviation value T of the Gram feature matrix R is calculated. u and lower limit threshold T l ;

[0014] Step 4: Calculate the deviation value T for each Worker node i. i If Ti In the range of [T l ,T u ] the Worker node is a benign node, otherwise it is excluded;

[0015] Step 5, aggregate the node training gradients of all benign nodes to obtain a global image classification model for the next round of iteration, and distribute it to each Worker node, if the classification accuracy of the global image classification model no longer changes or the change is less than a threshold on the test set, then the global image classification model is taken as a federated learning image classification model; otherwise, step 2 is performed;

[0016] Step 6, after pre-processing the image to be classified, input it into the federated learning image classification model trained to obtain the type of the image to be classified.

[0017] Further, when the Byzantine attack is a Bit-flip attack, the manner of uploading the node training gradient to the Server node in step 2 is that the benign Worker node uploads the real node training gradient to the Server node, and the Byzantine attacker uploads the node training gradient after Bit-flip processing to the Server node, and the Bit-flip attack is Δw' = k x Δw, where Δw' = k x Δw is the node training gradient uploaded by the Byzantine attacker, Δw' = k x Δw is the node training gradient before the Byzantine attack, and k is a coefficient with a value of -1.

[0018] Further, the manner of obtaining the feature output h of the local image classification model of each Worker node on the test set is to obtain the feature output of each layer of the local image classification model on the virtual test set, and then obtain the feature output h by concatenating the one-dimensional vector after unfolding.

[0019] Further, the manner of calculating the upper threshold Tu and the lower threshold Tl of the deviation value T of the Gram feature matrix R is to calculate the deviation value Ti of each feature vector Ri using Median and MAD, and then calculate the upper threshold Tu and the lower threshold Tl of the deviation value T using Median and MAD.

[0020] Further, the manner of calculating the deviation value T of each feature vector Ri using Median and MAD is: i i

[0021]

[0022]

[0023] min = R med -R mad ​​

[0024] max = R med + R mad

[0025]

[0026] T i =∑δ i

[0027] where N represents the total number of Worker nodes, [N] denotes [1, 2, 3, …, N], R med represents the median of each column of the Gram feature matrix R, R mad represents the absolute median difference of the Gram feature matrix R, min represents the minimum threshold value set for the Gram feature matrix, max represents the maximum threshold value set for the Gram feature matrix, δ i represents the deviation value of the Worker node i, T i represents the total deviation value of the Worker node i.

[0028] Further, the manner of calculating the upper threshold value T u and the lower threshold value T l of the deviation value T using the Median and MAD is as follows:

[0029]

[0030]

[0031] T l = T med - 0.8 × T mad

[0032] T u = T med + 0.8 × T mad

[0033] where T med represents the median of the deviation value set T, T med represents the absolute median difference of the deviation value set T, T u and T l respectively represent the upper threshold value and the lower threshold value.

[0034] Advantages:

[0035] The application utilizes the feature output of each local image classification model on the virtual data set, and processes the Gram matrix based on Median and MAD to eliminate malicious models attacked by the Byzantine attack, and then selects the node training gradient submitted by the normal Worker node to participate in aggregation, completely avoiding the influence of the Byzantine attack on the global model, and making the global model have high classification accuracy in normal image data. In addition, the application belongs to an unsupervised method, is suitable for a federated learning scene, and can be widely applied in federated learning defense, and improves the robustness of federated learning. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 A training flowchart of the federated learning image classification method capable of defending against the Byzantine attack is provided in the application. DETAILED DESCRIPTION

[0037] The application will be described in detail below with reference to the drawings and examples.

[0038] The basic idea of the federated learning image classification method capable of defending against the Byzantine attack provided in the application is as follows: training local image classification models on each Worker node by using a virtual data set, observing the feature performance of the local image classification models on the virtual test set on the Server side, calculating the deviation value and threshold value of each Worker node according to the feature performance by using Median and MAD, and selecting the local image classification models of the Worker nodes within the threshold value range for aggregation, so that the influence of the Byzantine attack on the global model can be completely avoided, and the global model can maintain high classification accuracy on the test image.

[0039] The federated learning image classification method capable of defending against the Byzantine attack provided in the application has the process as shown in Figure 1 The specific steps include the following steps:

[0040] Step 1, on each Worker node, the image sample data used for training the local image classification model is determined as local image data; on the Server node, a virtual data set containing a training set and a test set is generated, and the initialization of the global image classification model is completed, and the specific mode of the Byzantine attack is determined.

[0041] In the application, an image preprocessing method of pixel point value normalization on image data and virtual data is adopted to form image sample data, in addition, the Byzantine attack can adopt Bit-flip attack. The global image classification model can adopt an image classification model based on a neural network.

[0042] Step 2, the Server node downloads the initialized global image classification model and the training set in the virtual data set to each Worker node.

[0043] Step 3, each Worker node trains the received global image classification model using local image data and the training set in the virtual data set, and adopts gradient descent method in the training process, and the optimizer is Adam. After multiple rounds of training, the node training gradient of the global image classification model is obtained.

[0044] Step 4, after local training, the benign Worker node uploads the real node training gradient to the Server node, and the Byzantine attacker uploads the node training gradient processed by Bit-flip to the Server node. The Bit-flip attack is as follows:

[0045] Δw' = k x Δw

[0046] In the formula, Δw' is the node training gradient uploaded by the Byzantine attacker, Δw is the node training gradient before the Byzantine attack, and k is the coefficient, k = -1.

[0047] Step 5, the Server node generates the local image classification model corresponding to each Worker node using the node training gradient uploaded by each Worker node and the global image classification model of the last round.

[0048] Step 6, obtain the feature output h of the local image classification model of the Worker node i on the virtual test set, merge the feature outputs of all local image classification models to form a feature matrix H, and calculate to obtain a Gram feature matrix R.

[0049] When obtaining the feature output of the local image classification model on the virtual test set, the feature output of each layer of the model on the virtual test set is obtained, then a one-dimensional vector is unfolded, and splicing is performed to obtain a feature vector h. For example, for a certain local image classification model, assuming that it is a three-layer neural network model, the image is input into the neural network to obtain the output of each layer, denoted as o1, o2 and o3. Since the network structure of each layer is different, the dimension of the obtained feature output is also different. The output needs to be unfolded to form a one-dimensional vector, and then the three one-dimensional vectors are spliced in front and back to form the feature vector h of the local image classification model. The feature vectors h of all obtained local image classification models are spliced in the row direction to obtain a feature matrix H, H = [h1, h2, …, hn], and the Gram feature matrix R is calculated according to the following formula, N

[0050] R = HH T

[0051] ​The dimension of the feature matrix H is high, and each row of the matrix is the feature output of the model, which is a high-dimensional vector. If the feature matrix H is directly used for calculation, the calculation complexity is high, and the extraction of potential features is affected. The use of Gram feature matrix can reduce the complexity, extract and amplify the potential relationship between different model features, and be more conducive to identifying Byzantine attackers.

[0052] Step 7, calculate the deviation value T of each feature vector R using Median and MAD i , the calculation formula is as follows: i

[0053]

[0054]

[0055] min = R med -R mad

[0056] max = R med +R mad

[0057]

[0058] T i =∑δ i

[0059] In the formula, N represents the total number of Worker nodes, [N] represents [1, 2, 3, …, N], R med represents the median of each column of the Gram feature matrix R, R mad represents the absolute median difference of the Gram feature matrix R, min represents the minimum threshold value of the Gram feature matrix, max represents the maximum threshold value of the Gram feature matrix, δ i represents the deviation value of the Worker node i, and T i represents the total deviation value of the Worker node i.

[0060] Because R med is the median of each column of the matrix, R med is a vector, and R mad , min, max and δ i are also vectors.

[0061] Step 8, calculate the upper threshold T u and the lower threshold T l of the deviation value T using Median and MAD, the calculation formula is as follows:

[0062] ​

[0063]

[0064] T l =T med -0.8×T mad

[0065] T u =T med +0.8×T mad

[0066] In the formula, T med T represents the median of the set of deviations T. mad T represents the absolute median difference of the set of deviations T. u and T l These represent the upper and lower thresholds, respectively.

[0067] Step 9: Calculate the deviation value T for each Worker node i. i Is it in [T] l ,T u Within the range, if it is within the range, the local image classification model of Worker node i is retained; otherwise, it is determined to be a Byzantine attacker and is removed.

[0068] Step 10: Aggregate the node training gradients uploaded by the Worker nodes obtained in Step 9 using FedAvg to generate the global image classification model for the next iteration.

[0069] Step 11: Distribute the global image classification model to each Worker node.

[0070] Step 12: Repeat steps 2 to 11. When the classification accuracy of the global image classification model no longer changes or changes little on the test set, the global image classification model is considered to have converged, and the final global image classification model can be obtained and used as the federated learning image classification model.

[0071] Step 13: After preprocessing the image to be classified, input it into the trained federated learning image classification model to obtain the type of the image to be classified.

[0072] The performance of the federated learning image classification method provided in this invention, which is resistant to Byzantine attacks, was evaluated through experiments. Based on the MNIST, GTSRB, and CIFAR-10 datasets, image classification defense tests were conducted using a Bit-flip attack model. In terms of accuracy, the federated learning image classification method provided in this invention can completely resist Byzantine attacks, achieving classification accuracies of 99%, 97%, and 91% on normal image data, respectively, comparable to the accuracy of image classification models obtained without Byzantine attackers.

[0073] To sum up, the above is only the preferred embodiment of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A federated learning image classification method that can defend against Byzantine attacks, characterized in that, Includes the following steps: Step 1: Determine the local image data for model training on each Worker node; the Server node generates a virtual dataset containing training and test sets and completes the initialization of the global image classification model; the Server node distributes the global image classification model and training set to each Worker node. Step 2: Each Worker node uses local image data and training set to train the global image classification model, obtains the node training gradient, and uploads the node training gradient to the Server node. Step 3: The Server node uses the node training gradients of each Worker node and the global image classification model from the previous round to generate a local image classification model for each Worker node; the feature outputs h of the local image classification models of each Worker node on the test set are merged to form a feature matrix H, and the Gram feature matrix R is calculated. Calculate the upper threshold T of the deviation T of the Gram feature matrix R. u and lower limit threshold T l ; Step 4: Calculate the deviation value T for each Worker node i. i If T i In [T] l ,T u If the node is within the specified range, it is considered a benign node; otherwise, it is removed. Step 5: Aggregate the node training gradients of all benign nodes to obtain the global image classification model for the next iteration, and distribute it to each Worker node. If the classification accuracy of the global image classification model no longer changes on the test set or the change is less than the threshold, then the global image classification model is used as the federated learning image classification model; otherwise, proceed to step 2. Step 6: After preprocessing the image to be classified, input it into the trained federated learning image classification model to obtain the type of the image to be classified. When the Byzantine attack is a Bit-flip attack, the method of uploading the node training gradient to the Server node in step 2 is as follows: the benign Worker node uploads the real node training gradient to the Server node, and the Byzantine attacker uploads the node training gradient after Bit-flip processing to the Server node. The Bit-flip attack is: Δw′=k×Δw, where Δw′ is the node training gradient uploaded by the Byzantine attacker, Δw is the node training gradient before the Byzantine attack, and k is a coefficient with a value of -1.

2. The federated learning image classification method according to claim 1, characterized in that, The feature output h of the local image classification model of each Worker node on the test set is obtained as follows: the feature output of each layer of the local image classification model on the virtual test set is obtained, unfolded into a one-dimensional vector, and then concatenated to obtain the feature output h.

3. The federated learning image classification method according to claim 1, characterized in that, The upper threshold T for calculating the deviation value T of the Gram feature matrix R. u and lower limit threshold T l The method is as follows: Calculate each feature vector R using Median and MAD. i Deviation value T i Then, Median and MAD are used to calculate the upper limit threshold T of the deviation value T. u and lower limit threshold T l .

4. The federated learning image classification method according to claim 3, characterized in that, The calculation of each feature vector R using Median and MAD is described. i Deviation value T i The method is as follows: min=R med -R mad max=R med +R mad T i =∑δ i In the formula, N represents the total number of Worker nodes, [N] represents [1,2,3,…,N], and R med R represents the median of each column of the Gram eigenma matrix R. mad δ represents the absolute median difference of the Gram feature matrix R, min represents the minimum threshold set for the Gram feature matrix, max represents the maximum threshold set for the Gram feature matrix, and δ i T represents the deviation value of Worker node i. i This represents the total deviation of Worker node i.

5. The federated learning image classification method according to claim 3, characterized in that, The upper threshold T for calculating the deviation value T using Median and MAD is described. u and lower limit threshold T l The method is as follows: T l =T med -0.8×T mad T u =T med +0.8×T mad In the formula, T med T represents the median of the set of deviations T. mad T represents the absolute median difference of the set of deviations T. u and T l These represent the upper and lower thresholds, respectively.

Images