Back door robust image classification method based on federated learning

Abstract

The invention discloses a federated learning-based backdoor robust image classification method, which comprises the following steps that: 1, a server initializes a global model and issues the global model to a client; 2, the client initializes a local model based on the received global model, and trains a local neural network based on a local data set; 3, the client side calculates local model update and sends a local update vector to the server; 4, the server records the update vector submitted by the client; 5, the server performs model aggregation according to an improved federal aggregation algorithm; 6, the server issues the trained global neural network to each client; and 7, the client performs image classification by using the trained global neural network. According to the method, federal learning is utilized, and collaborative image classification with higher robustness for backdoor attacks is realized.

Description

Technical Field

[0001] The present invention relates to the field of anomaly detection, and in particular to a backdoor robust image classification method based on federated learning. Background Art

[0002] Image classification is a key task in computer vision, which aims to assign a class label from a predefined set to each image. Although humans can easily identify the content in an image, machines must rely on complex algorithms to achieve similar results. In recent years, although deep learning has made some progress in the field of image classification, with the growth of data volume and the strengthening of personal privacy protection awareness, it has become increasingly challenging to obtain and centralize the high-quality data required for training.

[0003] Federated Learning (FL) is an emerging distributed machine learning framework that allows devices to train models on their local data and then upload only model updates to a central server, thereby collaboratively training global models without sharing original data. However, when applied to downstream tasks such as image classification, federated learning also faces challenges in terms of model security. Due to its distributed nature, federated learning is vulnerable to perturbations from backdoor attacks. Attackers hiding as normal clients can inject adversarial triggers into the model to make the model perform malicious tasks without affecting the performance of the main task, thereby posing a threat to system security. Therefore, how to enhance the robustness of federated learning models against such backdoor attacks in image classification problems still needs to be explored, especially while ensuring the classification performance of the model. Summary of the invention

[0004] In order to address the deficiencies of the above-mentioned prior art, the present invention proposes a backdoor robust image classification method based on federated learning, so that when applied to image classification task scenarios, the robustness of the federated learning image recognition model can be enhanced, thereby achieving high-precision image classification.

[0005] In order to achieve the above-mentioned purpose, the present invention adopts the following technical scheme:

[0006] The invention discloses a backdoor robust image classification method based on federated learning, which is applied to a central server and In a federated system consisting of 10 clients, the following steps are performed:

[0007] Step 1. Define the total number of global training rounds of the federated system as The total number of local training rounds on the client is ,definition Among the clients Clients are malicious clients, initialize the current global training round number , the central server initializes the global image recognition model under the t-1th global training round And send it to each client;

[0008] Step 2. Clients receive the global image recognition model After that, initialize the t-1th global training round Client-side image recognition models ;

[0009] Step 3. Define The local dataset of a client in the tth global training round is ,in, express The jth image sample in express The real label, Indicates Client local dataset The total number of image samples in ;

[0010] Step 4. enter Perform local training in the t+1th global training round to obtain Client-side image recognition models ;

[0011] Step 5. After receiving the historical global gradient set of the client model trained in the tth global training round sent by N clients, the central server calculates the similarity and maliciousness score between the global gradients of each client model to obtain Weight ;

[0012] Step 6. The central server calculates the weighted geometric center of the global gradient of each client image recognition model in the tth global training round based on the weight of each client image recognition model, and uses it to aggregate each client image recognition model in the tth global training round to obtain the global image recognition model in the tth global training round. ;

[0013] Step 7. After assigning t+1 to t, determine Is it true? If true, it means that R malicious clients are in the first The attack is carried out in round 1, and step 8 is executed. The remaining NR honest clients return to step 4 and execute sequentially. Otherwise, it means that all clients are in the first round. Rounds of normal training are performed, and step 4 is returned to execute sequentially until t>T, thus obtaining the final trained global image recognition model. , used to classify and identify the input image samples; where, represents a global training round in which R malicious clients launch a backdoor attack;

[0014] Step 8. A malicious client from The random selection ratio is and implant black pixel blocks at the specified coordinates of the image sample as backdoor triggers; Indicates The number of local image samples selected by the malicious client;

[0015] Step 9. Initialize The malicious client is in Number of local training rounds for global training rounds =1;

[0016] Initialize Global training round Local training round Malicious Client Image Recognition Model = ;

[0017] Step 10. The malicious client is in Local dataset for global training rounds enter Training is performed in The local gradient , thus obtaining Global training round Local training round Malicious Client Image Recognition Model ;

[0018] Step 11. Assign to Then return to step 10 and execute sequentially until > So far, we get After training in the global training round Malicious Client Image Recognition Model , and calculate The global gradient Then, save it in The first round of global training The historical global gradient set of the malicious client image recognition model and sent to the central server to execute step 5; wherein, Represents the total number of local training rounds of the malicious client.

[0019] The backdoor robust image classification method based on federated learning described in the present invention is also characterized in that step 4 includes:

[0020] Step 4.1. Initialize The number of local training rounds of a client under the tth global training round =1; Initialize the tth global training round The local training round Client-side image recognition models = ;

[0021] Step 4.2. enter Train and calculate The local gradient , thus obtaining the tth global training round The local training round Client-side image recognition models ,in, Indicates The learning rate of the client model;

[0022] Step 4.3. Assign to Then return to step 4.2 and execute sequentially until > So far, we can get the tth global training round after training. Client-side image recognition models , and calculate The global gradient And store it in the tth global training round The historical global gradient set of the client image recognition model After being received, it is sent to the central server.

[0023] Further, the step 5 comprises:

[0024] Step 5.1. Use formula (1) to calculate the tth global training round after training Client-side image recognition models After training Client-side image recognition models The similarity between , thus obtaining Similarity with all other client-side image recognition models:

[0025] (1)

[0026] In formula (1), It is a non-zero value to avoid the denominator being zero; represents the tth global training round The historical global gradient set of the client image recognition model;

[0027] Step 5.2. From The maximum value among the similarities between the image recognition models of all other clients is selected as the first training round after training in the tth global training round. Client-side image recognition models Malice score ;

[0028] Step 5.3. When When , the updated similarity is obtained using formula (2) , and keep and The similarity between constant:

[0029] (2)

[0030] Step 5.4. Follow the process in step 5.3 to Malice score The malicious scores of all other client image recognition models are judged and updated respectively to obtain Updated similarity with all other client image recognition models;

[0031] Step 5.5. From The maximum value among the updated similarities between the image recognition models of all other clients is selected as the first training step after training in the tth global training round. Client-side image recognition models Updated malicious score ;

[0032] Step 5.6. Use formula (3) to calculate the first Client-side image recognition models Weight :

[0033] (3)

[0034] In formula (3), represents the tth global training round after training Client-side image recognition models Updated malicious score;

[0035] Step 5.7. Use formula (4) to calculate the tth global training round Client-side image recognition models Updated weights :

[0036] (4)

[0037] Step 5.8. After normalization, we get the tth global training round Client-side image recognition models The final weight .

[0038] Further, the step 6 comprises:

[0039] Step 6.1. Define the current aggregation iteration number as , and initialize =1;

[0040] Step 6.2. The central server uses equations (6) and (7) to calculate the first global training round The weight of the image recognition model of the i-th client in the aggregation iteration :

[0041] (6)

[0042] (7)

[0043] In formula (6) and formula (7), is the smoothing parameter; is the tth global training round The geometric center in the aggregation iteration, is the tth global training round -1 geometric center in aggregation iteration, when = 1, using formula (8) we get :

[0044]

[0045] Step 6.3. Determine whether equation (9) is true. If true, then we get And as the optimal geometric center in the tth global training round Otherwise, +1 assigned to Then, return to step 6.2 and execute sequentially;

[0046] (9)

[0047] In formula (9), is the preset iteration threshold, represents the objective function of iterative optimization of the geometric center, and has:

[0048] (10)

[0049] Step 6.4. Use formula (11) to obtain the preliminary global image recognition model under the tth global training round :

[0050] (11)

[0051] Step 6.5. Use equation (12) to Update to obtain the global image recognition model under the tth global training round :

[0052] (12)

[0053] In formula (12), is the clipping threshold in the tth global training round, is the noise that follows the Gaussian distribution in the tth global training round.

[0054] An electronic device of the present invention includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the backdoor robust image classification method, and the processor is configured to execute the program stored in the memory.

[0055] The present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program executes the steps of the backdoor robust image classification method when the computer program is executed by a processor.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] 1. The present invention proposes a novel weight parameter clustering anomaly detection method in federated learning. This method calculates the malicious score of the client before aggregation, calculates the updated geometric center of the client's image recognition model instead of simple weighted aggregation, and performs perturbation pruning on the global image recognition model after aggregation. This enhances the robustness of federated learning against backdoor attacks in image recognition tasks and ensures high-precision image classification.

[0058] 2. The present invention introduces a weighted geometric median estimation algorithm, which takes into account the importance differences of the image recognition models of each client when evaluating the geometric median of the model update parameters. Compared with traditional methods, the aggregation results obtained by this algorithm are more robust. While ensuring the classification accuracy of the model, it effectively improves the ability of the image recognition model to resist malicious interference. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0060] In this embodiment, a backdoor robust image classification method based on federated learning is provided, such as Figure 1 As shown, it is applied by the central server and In a federated system consisting of 10 clients, the following steps are performed:

[0061] Step 1. Define the total number of global training rounds of the federated system as The total number of local training rounds on the client is ,definition Among the clients Clients are malicious clients, initialize the current global training round number , the central server initializes the global image recognition model under the t-1th global training round And sent to each client; This embodiment uses the MNIST data set to train and evaluate the model. The MNIST data set consists of 70,000 grayscale images in 10 categories, each category contains 6,000 training examples and 1,000 test examples; This embodiment randomly selects 10,000 from the 60,000 training examples as public data, and the remaining 50,000 are used as local data of the client; A multi-classification logistic regression model is used, which consists of a linear layer with a softmax function, and the cross entropy function is used to calculate the loss; In this embodiment, T is 100, Take 30, the total number of clients N take 20, and the number of malicious clients R take 4.

[0062] Step 2. Clients receive the global image recognition model After that, initialize the t-1th global training round Client-side image recognition models ;

[0063] Step 3. Define The local dataset of a client in the tth global training round is ,in, express The jth image sample in express The real label, Indicates Client local dataset The total number of image samples in ;

[0064] Step 4. enter Perform local training in the t+1th global training round to obtain Client-side image recognition models ;

[0065] Step 4.1. Initialize The number of local training rounds of a client under the tth global training round =1; Initialize the tth global training round The local training round Client-side image recognition models = ;

[0066] Step 4.2. enter Train and calculate The local gradient , thus obtaining the tth global training round The local training round Client-side image recognition models ,in, Indicates The learning rate of the client model; in this embodiment, the learning rate Take 0.001.

[0067] Step 4.3. Assign to Then return to step 4.2 and execute sequentially until > So far, we can get the tth global training round after training. Client-side image recognition models , and calculate The global gradient And store it in the tth global training round The historical global gradient set of the client image recognition model After the completion, it is sent to the central server;

[0068] Step 5. After the central server receives the historical global gradient set of the client model trained in the tth global training round sent by N clients, it calculates the similarity and maliciousness score between the global gradients of each client model to obtain Weight ;

[0069] Step 5.1. Use formula (1) to calculate the tth global training round after training Client-side image recognition models After training Client-side image recognition models The similarity between , thus obtaining Similarity with all other client-side image recognition models:

[0070] (1)

[0071] In formula (1), It is a non-zero value to avoid the denominator being zero; represents the tth global training round A historical global gradient set of a client image recognition model; in this embodiment, Pick .

[0072] Step 5.2. From The maximum value among the similarities between the image recognition models of all other clients is selected as the first training round after training in the tth global training round. Client-side image recognition models Malice score ;

[0073] Step 5.3. When When , the updated similarity is obtained using formula (2) , and keep and The similarity between constant:

[0074] (2)

[0075] Step 5.8. After normalization, we get the tth global training round Client-side image recognition models The final weight ;

[0076] Step 6. The central server calculates the weighted geometric center of the global gradient of each client image recognition model in the tth global training round based on the weight of each client image recognition model, and uses it to aggregate each client image recognition model in the tth global training round to obtain the global image recognition model in the tth global training round. ;

[0077] Step 6.1. Define the current aggregation iteration number as , and initialize =1;

[0078] Step 6.2. The central server uses equations (6) and (7) to calculate the first The weight of the image recognition model of the i-th client in the aggregation iteration :

[0079] (6)

[0080] (7)

[0081] In formula (6) and formula (7), is the smoothing parameter; is the tth global training round The geometric center in the aggregation iteration, is the tth global training round -1 geometric center in aggregation iteration, when = 1, using formula (8) we get : In this embodiment, Pick .

[0082]

[0083] Step 6.3. Determine whether equation (9) is true. If true, then we get And as the optimal geometric center in the tth global training round Otherwise, +1 assigned to After that, return to step 6.2 and execute sequentially; in this embodiment, Pick .

[0084] (9)

[0085] In formula (9), is the preset iteration threshold, represents the objective function of iterative optimization of the geometric center, and has:

[0086] (10)

[0087] Step 6.4. Use formula (11) to obtain the preliminary global image recognition model under the tth global training round :

[0088] (11)

[0089] Step 6.5. Use equation (12) to Update to obtain the global image recognition model under the tth global training round :

[0090] (12)

[0091] In formula (12), is the clipping threshold in the tth global training round, is the noise that obeys Gaussian distribution in the tth global training round; in this embodiment, Take 0.1t+2 (t is the current global iteration), Pick .

[0092] Step 7. After assigning t+1 to t, determine Is it true? If true, it means that R malicious clients attack in this round, the malicious clients execute step 8, and the remaining NR honest clients return to step 4 to execute sequentially; otherwise, it means that all clients perform training normally in this round, return to step 4 to execute sequentially until t>T, so as to obtain the final trained global image recognition model , used to classify and identify the input image samples; where, represents a global training round in which R malicious clients launch a backdoor attack; in this embodiment, Take 10.

[0093] Step 8. A malicious client from The random selection ratio is and implant black pixel blocks at the specified coordinates of the image sample as backdoor triggers; Indicates The number of local image samples selected by the malicious client. In this embodiment, The value is 0.05, and the coordinates of the pixel blocks implanted by the malicious client are [23,25], [24,24], [25,23], [25,25].

[0094] Step 9. Initialize The malicious client is in Number of local training rounds for global training rounds =1;

[0095] Initialize Global training round Local training round Malicious Client Image Recognition Model = ;

[0096] Step 10. The malicious client is in Local dataset for global training rounds enter Training is performed in The local gradient , thus obtaining Global training round Local training round Malicious Client Image Recognition Model ;

[0097] Step 11. Assign to Then return to step 10 and execute sequentially until > So far, we get After training in the global training round Malicious Client Image Recognition Model , and calculate The global gradient Then, save it in The first round of global training The historical global gradient set of the malicious client image recognition model and sent to the central server to execute step 5; wherein, Represents the total number of local training rounds of the malicious client; in this embodiment, Take 30.

[0098] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0099] In this embodiment, a computer-readable storage medium stores a computer program on the computer-readable storage medium, and the computer program executes the steps of the above method when executed by a processor.

Claims

1. A backdoor robust image classification method based on federated learning, characterized in that: Applied to the central server and In a federated system consisting of 10 clients, the following steps are performed: Step 1. Define the total number of global training rounds of the federated system as The total number of local training rounds on the client is ,definition Among the clients Clients are malicious clients, initialize the current global training round number , the central server initializes the global image recognition model under the t-1th global training round And send it to each client; Step 2. Clients receive the global image recognition model After that, initialize the t-1th global training round Client-side image recognition models ; Step 3. Define The local dataset of a client in the tth global training round is ,in, express The jth image sample in express The real label, Indicates Client local dataset The total number of image samples in ; Step 4. enter Perform local training in the t+1th global training round to obtain Client-side image recognition models ; Step 5. After receiving the historical global gradient set of the client model trained in the tth global training round sent by N clients, the central server calculates the similarity and maliciousness score between the global gradients of each client model to obtain Weight ; Step 6. The central server calculates the weighted geometric center of the global gradient of each client image recognition model in the tth global training round based on the weight of each client image recognition model, and uses it to aggregate each client image recognition model in the tth global training round to obtain the global image recognition model in the tth global training round. ; Step 7. After assigning t+1 to t, determine Is it true? If true, it means that R malicious clients are in the first The attack is carried out in round 1, and step 8 is executed. The remaining NR honest clients return to step 4 and execute sequentially. Otherwise, it means that all clients are in the first round. Rounds of normal training are performed, and step 4 is returned to execute sequentially until t>T, thus obtaining the final trained global image recognition model. , used to classify and identify the input image samples; where, represents a global training round in which R malicious clients launch a backdoor attack; Step 8. A malicious client from The random selection ratio is and implant black pixel blocks at the specified coordinates of the image sample as backdoor triggers; Indicates The number of local image samples selected by the malicious client; Step 9. Initialize The malicious client is in Number of local training rounds for global training rounds =1; Initialize Global training round Local training round Malicious Client Image Recognition Model = ; Step 10. The malicious client is in Local dataset for global training rounds enter Training is performed in The local gradient , thus obtaining Global training round Local training round Malicious Client Image Recognition Model ; Step 11. Assign to Then return to step 10 and execute sequentially until > So far, we get After training in the global training round Malicious Client Image Recognition Model , and calculate The global gradient Then, save it in The first round of global training The historical global gradient set of the malicious client image recognition model and sent to the central server to execute step 5; wherein, Represents the total number of local training rounds of the malicious client.

2. According to claim 1, a backdoor robust image classification method based on federated learning is characterized in that: The step 4 comprises: Step 4.

1. Initialize The number of local training rounds of a client under the tth global training round =1; Initialize the tth global training round The local training round Client-side image recognition models = ; Step 4.

2. enter Train and calculate The local gradient , thus obtaining the tth global training round The local training round Client-side image recognition models ,in, Indicates The learning rate of the client model; Step 4.

3. Assign to Then return to step 4.2 and execute sequentially until > So far, we can get the tth global training round after training. Client-side image recognition models , and calculate The global gradient And store it in the tth global training round The historical global gradient set of the client image recognition model After being received, it is sent to the central server.

3. The backdoor robust image classification method based on federated learning according to claim 2, characterized in that: The step 5 comprises: Step 5.

1. Use formula (1) to calculate the tth global training round after training Client-side image recognition models After training Client-side image recognition models Similarity between , thus obtaining Similarity with all other client-side image recognition models: (1) In formula (1), It is a non-zero value taken to avoid the denominator being zero; represents the tth global training round The historical global gradient set of the client image recognition model; Step 5.

2. From The maximum value among the similarities between the image recognition models of all other clients is selected as the first training round after training in the tth global training round. Client-side image recognition models Malice score ; Step 5.

3. When When , the updated similarity is obtained using formula (2) , and keep and Similarity between constant: (2) Step 5.

4. Follow the process in step 5.3 to Malice score The malicious scores of all other client image recognition models are judged and updated respectively to obtain Updated similarity with all other client image recognition models; Step 5.

5. From The maximum value among the updated similarities between the image recognition models of all other clients is selected as the first training step after training in the tth global training round. Client-side image recognition models Updated malicious score ; Step 5.

6. Use formula (3) to calculate the first Client-side image recognition models Weight : (3) In formula (3), represents the tth global training round after training Client-side image recognition models Updated malicious score; Step 5.

7. Use formula (4) to calculate the tth global training round Client-side image recognition models Updated weights : (4) Step 5.

8. After normalization, we get the tth global training round Client-side image recognition models The final weight .

4. The backdoor robust image classification method based on federated learning according to claim 3 is characterized in that: The step 6 comprises: Step 6.

1. Define the current aggregation iteration number as , and initialize =1; Step 6.

2. The central server uses equations (6) and (7) to calculate the first global training round The weight of the image recognition model of the i-th client in the aggregation iteration : (6) (7) In formula (6) and formula (7), is the smoothing parameter; is the tth global training round The geometric center in the aggregation iteration, is the tth global training round -1 geometric center in aggregation iteration, when = 1, using formula (8) we get : (8) Step 6.

3. Determine whether equation (9) is true. If true, then we get And as the optimal geometric center in the tth global training round Otherwise, +1 assigned to Then, return to step 6.2 and execute sequentially; (9) In formula (9), is the preset iteration threshold, represents the objective function of iterative optimization of the geometric center, and has: (10) Step 6.

4. Use formula (11) to obtain the preliminary global image recognition model under the tth global training round : (11) Step 6.

5. Use equation (12) to Update to obtain the global image recognition model under the tth global training round : (12) In formula (12), is the clipping threshold in the tth global training round, is the noise that follows the Gaussian distribution in the tth global training round.

5. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the backdoor robust image classification method described in any one of claims 1 to 5, and the processor is configured to execute the program stored in the memory.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the backdoor robust image classification method according to any one of claims 1 to 5 are executed.

Images