Federal learning client forgetting method and image recognition method

Abstract

The invention discloses a federated learning-based client forgetting method and an image recognition method, which are applied to the field of federated learning, and are characterized in that a forgotten client is determined based on a hostile attack detection result, and a global model is calibrated and retrained in the remaining clients to obtain a knowledge carrier model; generating an adversarial sample in the forgotten client to perform parameter fine tuning on the old global model to obtain forgotten client model gradient information; in the remaining clients, knowledge distillation is carried out based on the knowledge carrier model and the old global model to obtain remaining client model gradient information; and performing gradient aggregation based on the forgotten client model gradient information and the remaining client model gradient information until convergence to obtain a global model after the client is forgotten. The hostile attack client in federal learning is forgotten through a combined confrontation sample technology and knowledge distillation, and the performance and privacy of the model after forgetting are improved while the forgetting efficiency is guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of federated learning, and in particular to a federated learning client forgetting method, an image recognition method, a federated learning client forgetting device, an electronic device and a computer readable storage medium. BACKGROUND

[0002] In the federated learning framework, multiple clients jointly participate in training a global model while keeping data localization to protect privacy. During the training process of federated learning, there may be malicious attack clients attacking the global model, resulting in a decrease in the performance of the global model, such as a decrease in the recognition accuracy of a target in an image. At this time, the malicious attack clients need to be forgotten.

[0003] The current client forgetting method cannot effectively trade off the three indicators of high availability, privacy and efficiency of the forgotten model. If high availability of the forgotten model is achieved, more statistical features of non-forgotten clients need to be retained to improve their generalization ability, which requires processing more data, resulting in a significant decrease in forgetting efficiency. If stronger forgetting effect and privacy protection are achieved, it may introduce a significant decrease in model performance, affecting its availability in subsequent tasks. SUMMARY

[0004] The present application relates to the field of federated learning, and in particular to a federated learning client forgetting method, an image recognition method, a federated learning client forgetting device, an electronic device and a computer readable storage medium.

[0005] To solve the above technical problems, the present application provides a federated learning client forgetting method, comprising:

[0006] Based on the malicious attack detection result, determine the malicious attack clients from the clients, determine the malicious attack clients as forgotten clients, and determine the non-malicious attack clients as remaining clients;

[0007] Calibrate and retrain the global model in the remaining clients to obtain a trained knowledge carrier model;

[0008] Generate an adversarial sample in the forgotten client, and perform parameter fine-tuning on the old global model based on the adversarial sample to obtain forgotten client model gradient information;

[0009] In the remaining clients, the knowledge carrier model is determined as a teacher model, and the old global model is determined as a student model. Knowledge distillation is performed based on the teacher model and the student model to obtain remaining client model gradient information.

[0010] Gradient aggregation is performed based on the forgotten client model gradient information and the remaining client model gradient information until convergence, to obtain the global model after forgetting of the clients.

[0011] Optionally, the global model is calibrated and retrained in the remaining clients to obtain a trained knowledge carrier model, comprising:

[0012] Old gradient information of the old global model of each of the remaining clients in the federated learning process is obtained.

[0013] A new global model is generated, and training of the new global model is performed in the remaining clients to obtain new gradient information.

[0014] The new gradient information is calibrated based on the old gradient information to obtain new calibrated gradient information.

[0015] After the new calibrated gradient information converges, the knowledge carrier model after calibration and retraining is obtained.

[0016] Optionally, the calibration and retraining loss function is:

[0017]

[0018] In the formula, L retrain is the calibration and retraining loss function, C r is the total number of the remaining clients, is the number of samples of the c r th remaining client, n is the total number of samples of all clients, CE is a cross-entropy loss function, is the prediction result of the new global model with a weight of in the c r th remaining client, is the local data set of the c r th remaining client, is the local sample of the c r th remaining client, is the local sample label of the c r th remaining client.

[0019] Optionally, an adversarial sample is generated in the forgotten client, and parameter fine-tuning of the old global model is performed based on the adversarial sample to obtain forgotten client model gradient information, comprising: ​

[0020] The adversarial sample of the forgotten client is generated based on the PGD attack, and the adversarial sample is input into the old global model to obtain the output adversarial sample label;

[0021] The local samples of the forgotten client and the adversarial sample labels are combined into a new local dataset;

[0022] Based on the new local dataset, the parameters of the old global model in the forgotten client are fine-tuned to obtain the gradient information of the forgotten client model.

[0023] Optionally, the parameter fine-tuning loss function is:

[0024] ;

[0025] In the formula, L adjust Fine-tune the loss function for the parameters, C f The total number of the forgotten clients, For the cth f The number of samples from the forgotten clients, where n is the total number of samples, and CE is the cross-entropy loss function. For the cth f The weight of the forgotten clients is The prediction results of the old global model, For the cth f The new local dataset of the forgotten clients. For the cth f The local sample of the forgotten client, For the cth f The adversarial sample label of the forgotten client.

[0026] Optionally, the knowledge distillation loss function is:

[0027] ;

[0028] In the formula, L transfer Let α be the knowledge distillation loss function, and let α be the weighting parameter. For the cth f The local datasets of the remaining clients, For the cth f Local samples of the remaining clients, For the cth f The sample tags of the remaining clients, D KL Let KL divergence function be denoted as KL divergence function, and CE be the cross-entropy loss function. For the c-th knowledge transfer round of the T-th round f The weight of the remaining clients is The prediction results of the old global model, c f The weight of the remaining clients is The prediction results of the knowledge carrier model.

[0029] To address the aforementioned technical problems, this invention provides an image recognition method, comprising:

[0030] The image to be detected is acquired. When it is determined that the image recognition model has a malicious client, the image to be detected is input into the image recognition model after the client has forgotten it, and the image recognition result output by the model is obtained.

[0031] Specifically, the image recognition model is subjected to client-side forgetting based on the client-side forgetting method of federated learning described above.

[0032] To address the aforementioned technical problems, this invention provides a client-side forgetting device for federated learning, comprising:

[0033] The first module is used to identify malicious attack clients from the clients based on the malicious attack detection results, identify the malicious attack clients as forgotten clients, and identify non-malicious attack clients as remaining clients.

[0034] The second module is used to perform global model calibration and retraining in the remaining clients to obtain the trained knowledge carrier model;

[0035] The third module is used to generate adversarial examples in the forgotten client, and to fine-tune the parameters of the old global model based on the adversarial examples to obtain the gradient information of the forgotten client model.

[0036] The fourth module is used to determine the knowledge carrier model as the teacher model and the old global model as the student model in the remaining clients, and to perform knowledge distillation based on the teacher model and the student model to obtain the gradient information of the remaining client model.

[0037] The fifth module is used to perform gradient aggregation based on the gradient information of the forgotten client model and the gradient information of the remaining client model until convergence, so as to obtain the global model after the client is forgotten.

[0038] To solve the above-mentioned technical problems, the present invention provides an electronic device, characterized in that it comprises:

[0039] Memory, used to store computer programs;

[0040] A processor for implementing the client-side forgetting method of federated learning or the image recognition method described above when executing the computer program.

[0041] To address the aforementioned technical problems, the present invention provides a computer-readable storage medium storing computer-executable instructions. When these computer-executable instructions are executed by a processor, they implement the aforementioned client-side forgetting method of federated learning or the aforementioned image recognition method.

[0042] As can be seen, this invention identifies malicious attack clients from among the clients based on malicious attack detection results, designates these malicious attack clients as forgotten clients, and identifies non-malicious attack clients as remaining clients; it then performs global model calibration and retraining in the remaining clients to obtain a trained knowledge carrier model; adversarial examples are generated in the forgotten clients, and the parameters of the old global model are fine-tuned based on the adversarial examples to obtain the gradient information of the forgotten client model; in the remaining clients, the knowledge carrier model is identified as the teacher model, and the old global model is identified as the student model, and knowledge distillation is performed based on the teacher model and student model to obtain the gradient information of the remaining client model; gradient aggregation is performed based on the gradient information of the forgotten client model and the gradient information of the remaining client model until convergence, to obtain the global model after client forgetting.

[0043] This invention uses a combination of adversarial sample techniques and knowledge distillation to forget malicious attack clients in federated learning, thereby improving the performance and privacy of the model after forgetting while ensuring the efficiency of forgetting. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0045] Figure 1 A flowchart illustrating a client-side forgetting method for federated learning provided in an embodiment of the present invention;

[0046] Figure 2 This is a structural block diagram of a client-side forgetting device for federated learning provided in an embodiment of the present invention. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] The following combination Figure 1 , Figure 1 A flowchart illustrating a client-side forgetting method for federated learning provided in an embodiment of the present invention, the method comprising:

[0049] S101: Based on the malicious attack detection results, identify the malicious attacking client from the client pool, identify the malicious attacking client as the forgotten client, and identify the non-malicious attacking client as the remaining client.

[0050] In a federated learning framework, multiple clients (such as mobile devices and edge computing nodes) collaborate to train a global model while maintaining data locality to protect privacy. In practical applications, some clients may wish to withdraw from the federated learning process and request that their influence on the final model be completely removed, i.e., client forgetfulness.

[0051] Client unlearning in federated learning refers to the process in which a specific client wants its data to be completely removed during federated training so that it no longer affects the global model. In other words, it is the process of eliminating the influence of the forgotten client from the global model and obtaining a new global model.

[0052] There are many scenarios where client-side forgetting is needed, such as various situations involving user data, including social networks, healthcare, and the financial industry, to ensure that the global model can efficiently forget the model's memory of specific data features when needed. This embodiment does not limit the application scenarios of client-side forgetting; it only needs to determine the forgotten clients and the remaining clients from the federated learning clients based on corresponding rules to achieve client-side forgetting in the corresponding application scenario.

[0053] For example, during the training process of federated learning, a client might insert a backdoor or poison the dataset, leading to an attack on the global model. This can manifest in several ways: the attacker can manipulate the global model to produce incorrect outputs; or the attacker can degrade the model's usability, significantly reducing its accuracy. In such scenarios, malicious attacking clients can be identified as forgotten clients, and their impact can be removed from the global model using a federated forgetting method, thereby improving the accuracy of the global model.

[0054] For example, during the training process of federated learning, there may be some clients with low data quality, which may lead to poor performance of the final global model. In this scenario, once the clients with low data quality are identified as forgotten clients, the federated forgetting method can be used to remove their influence from the global model, thereby improving the accuracy of the global model.

[0055] In this embodiment, a client-side malicious attack scenario can be used as a specific example. Based on the malicious attack detection results, the malicious attacking client is identified from the clients, the malicious attacking client is identified as the forgotten client, and the non-malicious attacking client is identified as the remaining client.

[0056] This embodiment does not limit the specific type of malicious attack detection results, and can generally include poisoning attack detection results and backdoor attack detection results.

[0057] S102: Perform global model calibration and retraining in the remaining clients to obtain the trained knowledge carrier model.

[0058] In this embodiment, after determining the remaining clients, the global model can be calibrated and retrained on the remaining clients to obtain the trained knowledge carrier model. This embodiment does not limit the specific method of calibrating and retraining the global model, and can be set according to the actual application.

[0059] For example, this embodiment can obtain the old gradient information of the old global model of each remaining client during the federated learning process; generate a new global model, train the new global model in the remaining clients to obtain new gradient information; calibrate the new gradient information based on the old gradient information to obtain new calibration gradient information; and obtain the knowledge carrier model after calibration and retraining when the new calibration gradient information converges.

[0060] Specifically, the old gradient information of the old global model during the federated learning process can be obtained first, and the old gradient information of each remaining client can be determined from all the old gradient information.

[0061] Furthermore, a new global model is initialized and sent to the remaining clients for training. This embodiment does not limit the training method of the new global model; it can be set based on the actual application. Generally, the loss function during the training process of the new global model can be:

[0062] ;

[0063] In the formula, L retrain To calibrate the retraining loss function, C r This represents the total number of remaining clients. For the cth r The number of samples from the remaining clients, where n is the total number of samples from all clients, and CE is the cross-entropy loss function. For the cth r Among the remaining clients, the weight is The prediction results of the new global model, For the cth r Local datasets of the remaining clients, For the cth r Local samples from the remaining clients, For the cth r Local sample labels for the remaining clients.

[0064] After the new global model is trained, the new gradient information of the remaining clients in each training round can be obtained. In this embodiment, the new gradient information can be calibrated based on the old gradient information to obtain new calibrated gradient information. This embodiment does not limit the specific method of calibrating gradient information, but it can generally be as follows:

[0065] The new gradient information is input into the gradient correction function to obtain the new calibrated gradient information at output. The expression for the gradient correction function is:

[0066] ;

[0067] In the formula, For the tth r In the cth round of federated learning r Old gradient information for the remaining clients, For the tth r In the cth round of federated learning r New gradient information for the remaining clients, For the tth r In the cth round of federated learning r New corrected gradient information for the remaining clients.

[0068] Finally, the training and correction process of the new global model can be repeated until the new calibration gradient information converges, resulting in the trained new global model, which is also the knowledge carrier model.

[0069] S103: Generate adversarial examples in the forgotten client, and fine-tune the parameters of the old global model based on the adversarial examples to obtain the gradient information of the forgotten client model.

[0070] In this embodiment, adversarial examples can be generated in the forgotten client. Based on these adversarial examples, the parameters of the old global model are fine-tuned to obtain the gradient information of the forgotten client model. This embodiment does not limit the specific method of generating adversarial examples; it can be set according to the actual application. For example, adversarial examples can be generated using PGD (Projected Gradient Descent) attacks.

[0071] In this embodiment, adversarial samples of forgotten clients are generated based on PGD attacks. The adversarial sample labels are obtained by inputting the sample into the old global model. The local samples of forgotten clients and the adversarial sample labels are combined into a new local dataset. Based on the new local dataset, the parameters of the old global model in the forgotten clients are fine-tuned to obtain the gradient information of the forgotten client model.

[0072] Specifically, adversarial examples can be generated iteratively based on local samples from forgotten clients, as shown in the following formula:

[0073] ;

[0074] In the formula, For the cth f The forgotten client at the tth adv Adversarial examples are generated in each round of adversarial example generation. c f The forgotten client at the tth adv +1 adversarial sample generation round generates adversarial samples. The gradient function of the model loss function is generated to represent the adversarial examples. Adversarial example generation model parameters, For the cth f The local sample labels of a forgotten client, β and ε are hyperparameters, and clip is a clipping function that projects the perturbed adversarial sample back into the legal constraint space.

[0075] After generating adversarial examples, the generated adversarial examples can be input into the old global model to obtain the adversarial example labels output by the model. By combining the local samples and adversarial example labels of the remaining clients, a new local dataset for the remaining clients can be obtained.

[0076] This embodiment can obtain the gradient information of the forgotten client model by fine-tuning the parameters of the old global model in the forgotten client based on a new local dataset. This embodiment does not limit the specific method of parameter fine-tuning and can be set according to the actual application. Generally, the parameter fine-tuning loss function is:

[0077] ;

[0078] In the formula, L adjust For fine-tuning the loss function for the parameters, C f The total number of forgotten clients, For the cth f The number of samples from forgotten clients, where n is the total number of samples, and CE is the cross-entropy loss function. For the cth f The weight of the forgotten clients is The prediction results of the old global model, For the cth f A new local dataset for a forgotten client. For the cth f A local sample of a forgotten client. For the cth f A forgotten client's adversarial sample label.

[0079] This embodiment can obtain gradient information during the parameter fine-tuning process, that is, the gradient information of the forgotten client model.

[0080] S104: In the remaining clients, the knowledge carrier model is determined as the teacher model, and the old global model is determined as the student model. Knowledge distillation is performed based on the teacher model and the student model to obtain the gradient information of the remaining client model.

[0081] In the remaining clients, this embodiment can determine the knowledge carrier model as the teacher model and the old global model as the student model, and perform knowledge distillation based on the teacher model and student model to obtain the gradient information of the remaining client model.

[0082] This embodiment does not limit the specific method of knowledge distillation; it can be set according to the actual application. Generally, the knowledge distillation loss function is:

[0083] ;

[0084] In the formula, L transfer Let α be the knowledge distillation loss function, and α be the weight parameter. For the cth f Local datasets of the remaining clients, For the cth f Local samples from the remaining clients, For the cth f The sample labels of the remaining clients, D KL Let KL divergence function be denoted as KL divergence function, and CE be the cross-entropy loss function. For the c-th knowledge transfer round of the T-th round f Among the remaining clients, the weight is The prediction results of the old global model, c f Among the remaining clients, the weight is The prediction results of the knowledge carrier model.

[0085] In this embodiment, gradient information during the knowledge distillation process can be obtained, that is, the gradient information of the remaining client model.

[0086] S105: Perform gradient aggregation based on the gradient information of the forgotten client model and the gradient information of the remaining client model until convergence, to obtain the global model after client forgetting.

[0087] This embodiment can perform gradient aggregation based on the gradient information of the forgotten client model and the gradient information of the remaining client model to obtain aggregated gradient information, and repeatedly execute the above steps of obtaining the gradient information of the forgotten client model and the gradient information of the remaining client model until the aggregated gradient information converges. The global model after client forgetting can be obtained by updating the converged aggregated gradient information to the global model.

[0088] Current client-side forgetting methods cannot effectively balance the three metrics of high availability, privacy, and efficiency of the model after forgetting. To achieve high availability after forgetting, it is necessary to retain more statistical features of non-forgotten clients, and even fine-tune the model through additional retraining processes to improve its generalization ability. This requires processing more data, leading to a significant decrease in client-side forgetting efficiency and wasting computational resources. On the other hand, to achieve stronger forgetting effects and privacy protection, it may lead to a significant decline in model performance, affecting its availability in subsequent tasks.

[0089] This embodiment integrates knowledge distillation and adversarial example techniques, combined with a custom loss function and regularization term, to improve the performance and privacy of the global model while ensuring client-side forgetting efficiency.

[0090] Based on the above embodiments, the present invention uses a combination of adversarial sample techniques and knowledge distillation to forget malicious attack clients in federated learning, thereby improving the performance and privacy of the model after forgetting while ensuring the efficiency of forgetting.

[0091] The following is an image recognition method provided by an embodiment of the present invention, which may include:

[0092] The image to be detected is acquired. When it is determined that the image recognition model has a malicious client, the image to be detected is input into the image recognition model after the client has forgotten it, and the image recognition result output by the model is obtained.

[0093] Specifically, the image recognition model is subjected to client-side forgetting based on the aforementioned client-side forgetting method of federated learning.

[0094] In this embodiment, the type of global model can be set based on the actual application scenario. For example, it can be an image recognition model, a text recognition model, or a large language model. This embodiment does not limit it.

[0095] When the global model is an image recognition model, if the image recognition model is attacked by the client, such as an image dataset poisoning attack or an image dataset backdoor attack, the performance of the image recognition model will be degraded.

[0096] Therefore, in this embodiment, when the image recognition model is attacked by a client, the malicious attacking client can be forgotten according to the client forgetting method of federated learning, so as to obtain the forgotten image recognition model.

[0097] In this embodiment, the image to be detected is input into the image recognition model after the client has forgotten it to obtain the image recognition result. Compared with the image before the client forgot it, the accuracy of the image recognition model is improved, making the obtained image recognition result more accurate.

[0098] The following combination Figure 2 , Figure 2 This is a structural block diagram of a client-side forgetting device for federated learning provided in an embodiment of the present invention. The device may include:

[0099] The first module 100 is used to identify malicious attack clients from the clients based on the malicious attack detection results, identify the malicious attack clients as forgotten clients, and identify non-malicious attack clients as remaining clients.

[0100] The second module 200 is used to calibrate and retrain the global model in the remaining clients to obtain the trained knowledge carrier model;

[0101] The third module 300 is used to generate adversarial examples in the forgotten client, and to fine-tune the parameters of the old global model based on the adversarial examples to obtain the gradient information of the forgotten client model.

[0102] The fourth module 400 is used to determine the knowledge carrier model as the teacher model and the old global model as the student model in the remaining clients, and to obtain the gradient information of the remaining client model by knowledge distillation based on the teacher model and the student model.

[0103] The fifth module, 500, is used to perform gradient aggregation based on the gradient information of the forgotten client model and the gradient information of the remaining client model until convergence, so as to obtain the global model after client forgetting.

[0104] Based on the above embodiments, the present invention uses a combination of adversarial sample techniques and knowledge distillation to forget malicious attack clients in federated learning, thereby improving the performance and privacy of the model after forgetting while ensuring the efficiency of forgetting.

[0105] Based on the above embodiments, the second module 200 may include:

[0106] The first unit is used to obtain the old gradient information of the old global model of each remaining client during the federated learning process;

[0107] The second unit is used to generate a new global model, and the new global model is trained in the remaining clients to obtain new gradient information;

[0108] The third unit is used to calibrate the new gradient information based on the old gradient information to obtain the new calibrated gradient information;

[0109] The fourth unit is used to obtain the knowledge carrier model after the new calibration gradient information converges and the calibration and retraining are completed.

[0110] Based on the above embodiments, the calibration retraining loss function is:

[0111] ;

[0112] In the formula, L retrain To calibrate the retraining loss function, C r This represents the total number of remaining clients. For the cth r The number of samples from the remaining clients, where n is the total number of samples from all clients, and CE is the cross-entropy loss function. For the cth r Among the remaining clients, the weight is The prediction results of the new global model, For the cth r Local datasets of the remaining clients, For the cth r Local samples from the remaining clients, For the cth r Local sample labels for the remaining clients.

[0113] Based on the above embodiments, the third module 300 may include:

[0114] The fifth unit is used to generate adversarial examples of forgotten clients based on PGD attacks. The adversarial examples are input into the old global model to obtain the adversarial example labels of the output.

[0115] Unit 6 is used to combine local samples and adversarial sample labels from forgotten clients into a new local dataset;

[0116] Unit 7 is used to fine-tune the parameters of the old global model in the forgotten client based on the new local dataset to obtain the gradient information of the forgotten client model.

[0117] Based on the above embodiments, the parameter fine-tuning loss function is:

[0118] ;

[0119] In the formula, L adjust For fine-tuning the loss function for the parameters, C f The total number of forgotten clients, For the cth f The number of samples from forgotten clients, where n is the total number of samples, and CE is the cross-entropy loss function. For the cth f The weight of the forgotten clients is The prediction results of the old global model, For the cth f A new local dataset for a forgotten client. For the cth f A local sample of a forgotten client. For the cth f A forgotten client's adversarial sample label.

[0120] Based on the above embodiments, the knowledge distillation loss function is:

[0121] ;

[0122] In the formula, L transfer Let α be the knowledge distillation loss function, and α be the weight parameter. For the cth f Local datasets of the remaining clients, For the cth f Local samples from the remaining clients, For the cth f The sample labels of the remaining clients, D KL Let KL divergence function be denoted as KL divergence function, and CE be the cross-entropy loss function. For the c-th knowledge transfer round of the T-th round f Among the remaining clients, the weight is The prediction results of the old global model, c f Among the remaining clients, the weight is The prediction results of the knowledge carrier model.

[0123] The following is an image recognition device provided by an embodiment of the present invention, which may include:

[0124] The sixth module is used to acquire the image to be detected. When it is determined that the image recognition model has a malicious attack on the client, the image to be detected is input into the image recognition model after the client has forgotten it, and the image recognition result output by the model is obtained.

[0125] Specifically, the image recognition model is subject to client-side forgetting based on the aforementioned client-side forgetting mechanism of federated learning.

[0126] Based on the above embodiments, the present invention also provides an electronic device, which may include a memory and a processor. The memory stores a computer program, and when the processor calls the computer program in the memory, it can implement the steps provided in the above embodiments. Of course, the device may also include various necessary network interfaces, a power supply, and other components.

[0127] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by an execution terminal or processor, can implement the method provided in the embodiments of the present invention; the storage medium may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0128] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

Claims

1. A client-side forgetting method for federated learning, characterized in that, include: Based on the malicious attack detection results, the malicious attack client is identified from the client, and the malicious attack client is identified as the forgotten client, while the non-malicious attack client is identified as the remaining client; The remaining clients are used to calibrate and retrain the global model to obtain the trained knowledge carrier model. Adversarial examples are generated in the forgotten client, and the parameters of the old global model are fine-tuned based on the adversarial examples to obtain the gradient information of the forgotten client model. In the remaining clients, the knowledge carrier model is determined as the teacher model, and the old global model is determined as the student model. Knowledge distillation is performed based on the teacher model and the student model to obtain the gradient information of the remaining client model. Gradient aggregation is performed based on the gradient information of the forgotten client model and the gradient information of the remaining client model until convergence, to obtain the global model after client forgetting.

2. The client-side forgetting method for federated learning according to claim 1, characterized in that, The remaining clients are used to perform global model calibration and retraining to obtain the trained knowledge carrier model, including: Obtain the old gradient information of the old global model for each of the remaining clients during the federated learning process; A new global model is generated, and the new global model is trained in the remaining clients to obtain new gradient information; The new calibrated gradient information is obtained by calibrating the new gradient information based on the old gradient information. Once the new calibration gradient information converges, the knowledge carrier model that has been calibrated and retrained is obtained.

3. The client-side forgetting method for federated learning according to claim 2, characterized in that, The calibration retraining loss function is: ； In the formula, L retrain Let C be the calibration retraining loss function. r The total number of the remaining clients, For the cth r The number of samples from the remaining clients, where n is the total number of samples from all clients, and CE is the cross-entropy loss function. For the cth r The weight of the remaining clients is The prediction results of the new global model, For the cth r The local datasets of the remaining clients, For the cth r Local samples of the remaining clients, For the cth r The local sample labels of the remaining clients.

4. The client-side forgetting method for federated learning according to claim 1, characterized in that, Adversarial examples are generated in the forgotten client, and the gradient information of the forgotten client model is obtained by fine-tuning the parameters of the old global model based on the adversarial examples, including: The adversarial sample of the forgotten client is generated based on the PGD attack, and the adversarial sample is input into the old global model to obtain the output adversarial sample label; The local samples of the forgotten client and the adversarial sample labels are combined into a new local dataset; Based on the new local dataset, the parameters of the old global model in the forgotten client are fine-tuned to obtain the gradient information of the forgotten client model.

5. The client-side forgetting method for federated learning according to claim 4, characterized in that, The parameter fine-tuning loss function is: ； In the formula, L adjust Fine-tune the loss function for the parameters, C f The total number of the forgotten clients, For the cth f The number of samples from the forgotten clients, where n is the total number of samples, and CE is the cross-entropy loss function. For the cth f The weight of the forgotten clients is The prediction results of the old global model, For the cth f The new local dataset of the forgotten clients. For the cth f The local sample of the forgotten client, For the cth f The adversarial sample label of the forgotten client.

6. The client-side forgetting method for federated learning according to claim 1, characterized in that, The knowledge distillation loss function is: ； In the formula, L transfer Let α be the knowledge distillation loss function, and let α be the weighting parameter. For the cth f The local datasets of the remaining clients, For the cth f Local samples of the remaining clients, For the cth f The sample tags of the remaining clients, D KL Let KL divergence function be denoted as KL divergence function, and CE be the cross-entropy loss function. For the c-th knowledge transfer round of the T-th round f The weight of the remaining clients is The prediction results of the old global model, c f The weight of the remaining clients is The prediction results of the knowledge carrier model.

7. An image recognition method, characterized in that, include: The image to be detected is acquired. When it is determined that the image recognition model has a malicious client, the image to be detected is input into the image recognition model after the client has forgotten it, and the image recognition result output by the model is obtained. The image recognition model is subjected to client-side forgetting using the client-side forgetting method of federated learning according to any one of claims 1 to 6.

8. A client-side forgetting device for federated learning, characterized in that, include: The first module is used to identify malicious attack clients from the clients based on the malicious attack detection results, identify the malicious attack clients as forgotten clients, and identify non-malicious attack clients as remaining clients. The second module is used to perform global model calibration and retraining in the remaining clients to obtain the trained knowledge carrier model; The third module is used to generate adversarial examples in the forgotten client, and to fine-tune the parameters of the old global model based on the adversarial examples to obtain the gradient information of the forgotten client model. The fourth module is used to determine the knowledge carrier model as the teacher model and the old global model as the student model in the remaining clients, and to perform knowledge distillation based on the teacher model and the student model to obtain the gradient information of the remaining client model. The fifth module is used to perform gradient aggregation based on the gradient information of the forgotten client model and the gradient information of the remaining client model until convergence, so as to obtain the global model after the client is forgotten.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the client-side forgetting method of federated learning as described in any one of claims 1 to 6, or the image recognition method as described in claim 7, when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the client-side forgetting method of federated learning as described in any one of claims 1 to 6, or the image recognition method as described in claim 7.

Images