Federal learning-based privacy-protecting and robust personalized image recognition method
By combining differential privacy and K-means clustering algorithm in personalized federated learning, the problem that existing technology cannot protect privacy and defense against attacks at the same time is solved, effective protection and attack defense of training data is achieved, and the system is improved Strong and operational efficiency.
Patent Information
- Application Number
- CN202410984825.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-07-22
AI Technical Summary
The existing personalized federated learning method based on multitasking learning cannot simultaneously protect privacy and attack defense, and is vulnerable to attacks such as data poisoning and model poisoning during training.
Using differential privacy mechanism and K-means clustering algorithm, the training data privacy protection is ensured by executing a noisy stochastic gradient descent algorithm when training the model locally and performing a robust aggregation method on the server side. Defend against multiple attacks.
In personalized federated learning, users' privacy is protected and defense against attacks such as data poisoning and model poisoning, effectively filter out potential attackers with a proportion of less than 50%, and improve the operating efficiency and robustness of the system.
Smart Images

Figure CN118941853A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of federated learning in privacy computing, and in particular to a privacy-preserving and robust personalized image recognition method based on federated learning. Background Art
[0002] In recent years, with the improvement of hardware performance and the rapid growth of data volume, the performance of machine learning has been continuously improved, and the application scenarios have gradually become richer. In traditional centralized machine learning, massive data needs to be concentrated on one machine for training. However, such unregulated transfer is very dangerous for sensitive data. With the increasing attention paid to data privacy and the introduction of relevant protection regulations, federated learning, as one of the paradigms of distributed machine learning, has gradually attracted widespread attention. In federated learning, data does not need to be transferred. Participants train models locally and send training results to the server. The server aggregates the results submitted by all participants as a global model. While protecting the data privacy of participants, federated learning can achieve training effects similar to centralized learning and solve the problem of data islands between different institutions. Since the concept of federated learning was first proposed in 2016, related research and applications have developed in full swing. At present, mature federated learning systems have been applied to government affairs, medical care, finance and other fields.
[0003] Although the fact that the training data does not leave the local machine protects the privacy of the participants to a certain extent, a large number of studies have shown that the server can launch an inference attack, that is, to recover the valid information of the data based on the model or gradient submitted by the participant. Therefore, it is not sufficient to rely solely on the federated learning mechanism itself to protect sensitive data. At present, participants can choose cryptographic-related secure aggregation methods or differential privacy mechanisms to prevent the server from directly observing the initial values of the submitted models or gradients. In addition, the system environment of federated learning is relatively complex, with a large number of participants and no identity authentication. There may be malicious attackers who attempt to disrupt the convergence process of the global model, reduce the performance of the global model, or insert backdoors into the global model to conduct data poisoning attacks, model poisoning attacks, or backdoor attacks. At present, the server can use anomaly detection methods to remove updates submitted by potential attackers and only aggregate updates that are judged to be non-malicious.
[0004] The general federated learning scenario aims to train a single overall model. When the data is not independent and identically distributed, the participants apply the model in different directions, and a single model cannot meet the personalized needs of the participants. Therefore, a personalized federated learning method that aims to train different models for different participants has been proposed, and multi-task learning is one of the representative methods. In personalized federated learning based on multi-task learning, each type of participant generates a model suitable for the local data distribution during the training process, and the differences between the models of different types of participants are large or small. However, the existing multi-task learning framework cannot effectively defend against model poisoning attacks, backdoor attacks and other attacks while protecting the privacy of training data. Therefore, it is very important to introduce privacy protection measures into multi-task learning and combine them with defense measures suitable for personalized scenarios.
[0005] IV. Disadvantages of the prior art and technical problems to be solved by the present invention
[0006] Existing personalized federated learning methods based on multi-task learning cannot simultaneously protect privacy and defend against attacks. Several studies have added differential privacy mechanisms to the training process of participants to solve the problem of gradient leakage, but simply adding noise still cannot effectively defend against multiple attacks, and the training process or the final model obtained by the participants is still vulnerable to attackers. In addition, some studies focus on the defense effect brought by the personalized framework itself, or simply combine it with the defense measures in general federated learning scenarios, without considering the protection of training data privacy and the applicability of these defense measures in multi-task learning. Summary of the invention
[0007] To solve the above problems, the present invention provides a privacy-preserving and robust personalized image recognition method based on federated learning, which can defend against various attacks such as data poisoning and model poisoning while protecting the privacy of training data, thereby improving the operating efficiency of the system and reducing the maintenance cost of the system.
[0008] A privacy-preserving and robust personalized image recognition method based on federated learning, wherein each participant uses a local image recognition model to perform personalized recognition on an image; wherein the training method of the local image recognition model of each participant includes the following steps:
[0009] S1: The central server sends the initialized global image recognition model to each participant; the number of image samples contained in the local training set of each participant is the same, and each image sample corresponds to a label. The number of label categories corresponding to the image samples contained by all participants is N, and the label categories corresponding to the image samples contained by each participant are not exactly the same, and the number of label categories corresponding to the image samples contained by each participant does not exceed N / 2;
[0010] S2: Determine whether it is the first iteration. If so, the average gradient of each participant based on the local training set Execute the stochastic gradient descent algorithm with differential privacy to update its own local image recognition model; if not, each participant updates the local model based on the average gradient of the local training set. and the model parameters θ of the global model for image recognition g Obtain the fused gradient, and then execute the stochastic gradient descent algorithm with differential privacy based on the fused gradient to update its own local image recognition model; wherein the local image recognition model has the same structure as the global image recognition model;
[0011] S3: Each participant obtains the model parameters θ of its own updated local image recognition model l and the model parameters θ of the global model for image recognition g The difference between k =θ l -θ g , where k = 1, 2, …, K, K is the number of participants;
[0012] S4: The central server calculates the difference M of each participant k Calculate a score k , and use K-means clustering algorithm to cluster all scores k Perform binary classification;
[0013] S5: Get all the differences M in the category with more scores k The average And the average With model parameters θ g The sum of is used as the model parameter of the global image recognition model in the next round;
[0014] S6: Determine whether the current number of iterations has reached the upper limit. If yes, obtain the final image recognition local model of each participant. If no, send the image recognition global model obtained in step S5 to each participant, and then re-execute steps S2 to S6.
[0015] Furthermore, the average gradient of the local training set of any participant in step S2 The specific calculation method is:
[0016] S21: For each image sample x i The resulting gradient g(x i ) to clip and get the clipped gradient
[0017]
[0018] Where C is the maximum bi-norm of the gradient of a single image sample, and ||·||2 indicates the calculation of the bi-norm;
[0019] S22: From Gaussian distribution The noise is sampled in the image, and the noise is added to the sum of the clipped gradients to obtain the average gradient of the image samples contained in the local training set.
[0020]
[0021] Where L is the number of image samples contained in the local training set, and σ is the number that satisfies the inequality is an auxiliary variable of , δ is the probability of differential privacy, ε is the privacy budget of differential privacy, and I is the unit matrix.
[0022] Furthermore, the difference M of any participant k in step S4 k Corresponding score k The calculation method is:
[0023] S41: Assume that the difference M corresponding to participant k k =[m k,1 ,m k,2 ,…,m k,n ], where m k,1 ~m k,n is the difference of each parameter included in the model parameters corresponding to participant k, and n is the model parameter θ l The number of parameters contained in ;
[0024] S42: Obtain the Euclidean distance dis between participant k and other participants k,j :
[0025]
[0026] Among them, the difference M corresponding to participant j j =[m j,1 ,m j,2 ,…,m j,n ],m j,1 ~m j,n The model parameters corresponding to participant k include the difference of each parameter;
[0027] S43: Distance between participant k and all participants k,j The sum of the values is the difference M of participant k k Corresponding score k .
[0028] Furthermore, the calculation method of the fusion gradient of any participant in step S2 is:
[0029]
[0030] in, is the fusion gradient corresponding to participant k, is the average gradient corresponding to participant k, λ is the set weight, θ l [k] is the model parameter of the local model for image recognition corresponding to participant k.
[0031] Furthermore, in step S4, the K-means clustering algorithm is used to cluster all the scores. k The specific method for binary classification is:
[0032] S4a: Randomly select two scores as the starting centroids c1 and c2 of the two clusters;
[0033] S4b: Calculate each score separately k The distance d from the starting centroid c1 k,1 , the distance d from the starting centroid c2 k,2 :
[0034] d k,1 =score k -c1|
[0035] d k,2 =|score k -c2|
[0036] S4c: For each score k , compared with its corresponding d k,1 and d k,2 , and the scores k Divide into clusters with smaller distances to complete the current binary classification;
[0037] S4d: Determine whether the binary classification result currently obtained is the same as the binary classification result obtained in the previous iteration. If yes, complete the binary classification. If no, proceed to step S4e.
[0038] S4e: Get the average values of the scores of the two clusters of the current binary classification, and use the two average values as the centroids of the two clusters;
[0039] S4f: Repeat steps S4b to S4d based on the new centroids of the two clusters until the binary classification is completed.
[0040] Beneficial effects:
[0041] The present invention provides a privacy-preserving and robust personalized image recognition method based on federated learning. First, a differential privacy mechanism is adopted to protect user privacy. Compared with cryptographic means such as homomorphic encryption and secure multi-party computing, there is no need to distribute keys or consider the user's access and exit mechanism, thereby improving the operation efficiency of the system and reducing the maintenance cost of the system; secondly, compared with the existing framework that only focuses on privacy or poisoning attacks and does not implement a training framework that simultaneously considers the above two, the present invention can simultaneously protect user privacy and defend against potential attacks in personalized federated learning; in summary, the present invention is a personalized image recognition method based on average regularized multi-task learning, which can defend against various attacks such as data poisoning and model poisoning while protecting the privacy of training data. The central server applies this rule to effectively filter out potential attackers accounting for less than 50%. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 A framework diagram of a privacy-preserving and robust personalized image recognition method based on federated learning provided by the present invention. DETAILED DESCRIPTION
[0043] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.
[0044] In order to protect the privacy of participants' training data and prevent the training process from being affected by attackers in personalized federated learning for training image recognition network models, the present invention proposes a privacy-preserving and robust personalized image recognition method based on average regularized multi-task learning. The roles in the framework are divided into two categories: learning participants and servers. To avoid the leakage of image privacy information, participants execute a stochastic gradient descent algorithm with differential privacy during training. After local training is completed, model parameters are submitted to a semi-honest server. The server executes a robust aggregation method, filters out potential malicious updates according to specific criteria, calculates a new round of global models, and sends the results to all participants.
[0045] like Figure 1 As shown, a privacy-preserving and robust personalized image recognition method based on federated learning is characterized in that each participant uses a local image recognition model to perform personalized recognition on an image; wherein the training method of the local image recognition model of each participant includes the following steps:
[0046] S1: The central server sends the initialized global image recognition model to each participant; the number of image samples contained in the local training set of each participant is the same, and each image sample corresponds to a label. The number of label categories corresponding to the image samples contained by all participants is N, and the label categories corresponding to the image samples contained by each participant are not exactly the same, and the number of label categories corresponding to the image samples contained by each participant does not exceed N / 2;
[0047] It should be noted that the federated learning system of the present invention includes multiple participants and a central server; during initialization, it is necessary to negotiate hyperparameters such as the specific image recognition task of this training, the image recognition model used, the standard for stopping training, the differential privacy parameters ε and δ of the participants, the maximum bi-norm C of the gradient of a single sample, the personalization parameter λ, and the learning rate η.
[0048] S2: Determine whether it is the first iteration. If so, the average gradient of each participant based on the local training set Execute the stochastic gradient descent algorithm with differential privacy to update its own local image recognition model; if not, each participant updates the local model based on the average gradient of the local training set. and the model parameters θ of the global model for image recognition g Obtain the fused gradient, and then execute the stochastic gradient descent algorithm with differential privacy based on the fused gradient to update its own local image recognition model; wherein the local image recognition model has the same structure as the global image recognition model;
[0049] Specifically, the average gradient of the local training set of any participant The specific calculation method is:
[0050] S21: For each image sample x i The resulting gradient g(x i ) to clip and get the clipped gradient
[0051]
[0052] Where C is the maximum bi-norm of the gradient of a single image sample, and ||·||2 indicates the calculation of the bi-norm;
[0053] S22: From Gaussian distribution The noise is sampled in the image, and the noise is added to the sum of the clipped gradients to obtain the average gradient of the image samples contained in the local training set.
[0054]
[0055] Where L is the number of image samples contained in the local training set, and σ is the number that satisfies the inequality is an auxiliary variable of , δ is the probability of differential privacy, ε is the privacy budget of differential privacy, and I is the unit matrix.
[0056] Furthermore, the calculation method of the fusion gradient of any participant is:
[0057]
[0058] in, is the fusion gradient corresponding to participant k, is the average gradient corresponding to participant k, λ is the set weight, θ l [k] is the model parameter of the local model for image recognition corresponding to participant k.
[0059] That is to say, according to the definition of differential privacy, the present invention assumes that there are L image samples in a training batch, and for each image sample x in a batch i The resulting gradient g(x i ) is clipped and denoised. Since the information of whether each sample participates in the training is protected, the gradient of a single sample is clipped and denoised to blur the difference between the gradient and the gradient generated by the samples similar to it; the participants clip the gradients generated by a batch of samples according to the specified maximum norm C of the sample gradient, and the clipped gradient is From Gaussian distribution The noise is sampled in , and the noise is added to the sum of the clipped gradients, and then the average gradient obtained by the batch of samples is obtained, that is, According to the definition, a differential privacy mechanism based on Gaussian distribution, if applicable to a function with sensitivity C, then When , the mechanism satisfies (ε,δ)-differential privacy.
[0060] If this round is the first round of federated learning, a complete local training is performed and the participants average the gradients. Execute the stochastic gradient descent algorithm to update the model parameters θ of the local model l ; If it is not the first round, then the global model at this time is meaningful, and the model parameters θ of the local model need to be adjusted l The model parameters θ of the global model g Close; Specifically, in the obtained image sample x i Gradient In , add the product of λ and the difference between this parameter and the corresponding parameter of the global model, that is, Participants get new gradients After that, perform stochastic gradient descent, that is
[0061] S3: Each participant obtains the model parameters θ of its own updated local image recognition model l and the model parameters θ of the global model for image recognition g The difference between k =θ l -θ g , where k = 1, 2, …, K, K is the number of participants;
[0062] S4: The central server calculates the difference M of each participant k Calculate a score k , and use K-means clustering algorithm to cluster all scores k Perform binary classification;
[0063] Furthermore, the difference M of any participant k k Corresponding score k is the sum of the Euclidean distances between participant k and all other participants. The specific calculation method is as follows:
[0064] S41: Assume that the difference M corresponding to participant k k =[m k,1 ,m k,2 ,...,m k,n ], where m k,1 ~m k,n is the difference of each parameter included in the model parameters corresponding to participant k, and n is the model parameter θ l The number of parameters contained in ;
[0065] S42: Obtain the Euclidean distance dis between participant k and other participants k,j :
[0066]
[0067] Among them, the difference M corresponding to participant j j =[m j,1 ,m j,2 ,…,m j,n ],m j,1 ~m j,n The model parameters corresponding to participant k include the difference of each parameter;
[0068] S43: Distance between participant k and all participants k,j The sum of the values is the difference M of participant k k Corresponding score k =∑ j∈ Ndis k,j .
[0069] Furthermore, the K-means clustering algorithm is used to cluster all the scores. k The specific method for binary classification is:
[0070] S4a: Randomly select two scores as the starting centroids c1 and c2 of the two clusters;
[0071] S4b: Calculate each score separately k The distance d from the starting centroid c1 k,1 , the distance d from the starting centroid c2 k,2 :
[0072] d k,1 =|score k -c1|
[0073] d k,2 =|score k -c2|
[0074] S4c: For each score k , compared with its corresponding d k,1 and d k,2 , and the scores k Divide into clusters with smaller distances to complete the current binary classification;
[0075] That is to say, for each score k , calculate the distance between it and the two centroids respectively, and assign the score to the cluster with the smaller distance to the centroid.
[0076] S4d: Determine whether the binary classification result currently obtained is the same as the binary classification result obtained in the previous iteration. If yes, complete the binary classification. If no, proceed to step S4e.
[0077] S4e: Get the average of the scores of the two clusters in the current binary classification, and use the two averages as the centroids of the two clusters; assuming that the number of scores of the two clusters is n1 and n2 respectively, the new centroid
[0078] S4f: Repeat steps S4b to S4d based on the new centroids of the two clusters until the binary classification is completed.
[0079] S5: Get all the differences M in the category with more scores k The average And the average With model parameters θ g The sum of As the model parameters of the global image recognition model in the next round;
[0080] It should be noted that, considering that the proportion of attackers generally does not exceed half of the number of all participants, a class with a larger number is selected for aggregation, and all updates of this class are averaged.
[0081] S6: Determine whether the current number of iterations has reached the upper limit. If yes, obtain the final image recognition local model of each participant. If no, send the image recognition global model obtained in step S5 to each participant, and then re-execute steps S2 to S6.
[0082] Taking a federated learning system with 50 federated participants and one server responsible for the aggregation model as an example, a privacy-preserving and robust personalized image recognition method based on federated learning provided by the present invention is described in detail below.
[0083] Among them, the federation participants can use mobile phones, tablets, laptops and other devices. Ideally, both types of members are honest and curious (that is, each party abides by the agreement honestly, but also tries to obtain more other information from the information received). Each federation participant has a local training data set for image recognition, which is part of the cifar10 data set. The cifar10 data set contains 50,000 training samples and 10,000 test samples. The samples are divided into color pictures and labels. The pictures are 32×32 pixel matrices, and the labels are 10 categories of common objects. In order to simulate the situation where the data between participants are not independent and identically distributed, each participant randomly has 4 of the 10 categories of labels.
[0084] In addition, there are two types of attackers in the system, passive attackers and active attackers. Passive attackers eavesdrop on the image recognition model sent by the federation participants to the server, trying to obtain valid information contained in the images of other participants; active attackers poison the image collection or tamper with the updates to be submitted, trying to disrupt the training process or insert backdoors in the global model to achieve the purpose of destroying the local image recognition model of other participants. The number of active attackers generally does not exceed half of the federation participants.
[0085] If the privacy-preserving and robust personalized image recognition method based on average regularized multi-task learning proposed in the present invention is implemented, the data privacy of the participants and the defense against potential attacks can be protected at the same time. Specifically, it includes the following sub-steps:
[0086] Step 1: Initialize the federated learning training system.
[0087] Step 1.1: Determine that the task of this training is cifar10 image classification, the model used is a simple neural network ConvNet, the criterion for stopping training is when the number of rounds of federated learning reaches 200, the differential privacy parameters of the participants ε=15, δ=0.001, the maximum binary norm of the gradient of a single sample C=2, the personalization parameter λ=0.25, the learning rate η=0.1 and other hyperparameters.
[0088] Step 1.2: Use the Pytorch machine learning library to import and distribute the dataset and establish a federated learning system with 50 participants and 1 server. All participants and servers initialize local or global neural network models.
[0089] Step 1.3: The server sends the initialized global model to all participants.
[0090] Step 2: Participants conduct local training.
[0091] Step 2.1: According to the definition of differential privacy, there are L = 16 samples in a training batch. For each sample x in a batch i The resulting gradient g(x i ) is clipped and denoised. The participant clips the gradients generated by a batch of samples according to the maximum norm of the sample gradient C=2 specified in step 1. The clipped gradients are From Gaussian distribution The noise is sampled in , and the noise is added to the sum of the clipped gradients, and then the average gradient obtained by the batch of samples is obtained, that is, This mechanism satisfies (15, 0.001)-differential privacy. To speed up the above process of gradient clipping and denoising for each sample, the Opacus tool library is used to process a single batch of samples. The Opacus library uses the tensor hook provided in the Pytorch library to vectorize the clipping and denoising process, thereby increasing the speed of DP-SGD.
[0092] Step 2.2: If this is the first round of federated learning, perform full local training, and participants use the average gradient obtained in step 2.1 to perform the stochastic gradient descent algorithm; if this is not the first round, adjust the local parameter θ l To the global model θ g Specifically, the gradient of a parameter i obtained in step 2.1 In , add the product of λ and the difference between this parameter and the corresponding parameter of the global model, that is, After the participants get the new gradient, they perform stochastic gradient descent, that is,
[0093] Step 2.3: Repeat steps 2.1-2.2 until all samples are trained once, then the local training is completed.
[0094] Step 2.4: The participant calculates the difference M = θ between the parameters of the local model and the global model l -θ g , and send the value to the server.
[0095] Step 3: The server performs robust aggregation to filter out model updates submitted by potential attackers.
[0096] Potential attackers may conduct data poisoning or model poisoning attacks. In the random update attack, the attacker submits a random number with the same dimension as the model, trying to reduce the accuracy of the local model of other participants or destroy the model convergence process of other participants. In the update boosting attack, the attacker amplifies the difference between the models obtained by calculation by several times, such as 6 times, to change the overall model in the direction of the local model. In the grid backdoor attack, during training, the attacker changes the 3×3 nine-square grid pixels in the lower right corner of the sample into a fixed black and white pattern, and sets the label of such samples to a fixed value, with the purpose of inserting a backdoor in the local model of other participants, hoping that when the picture containing this pattern is input into the local model of other participants, the model will also classify it as the target label. In the edge backdoor attack, the attacker inserts a small number or zero samples in the training data of other participants into the local training set and assigns them a fixed label, with the purpose of inserting a backdoor in the local model of other participants, hoping that when the rare picture of this type is input into the local model of other participants, the model will also classify it as the target label.
[0097] Step 3.1: The server gets all updates M uploaded by N = 50 participants i After that, a score is calculated for each participant. i Specifically, the local model parameters M trained by participant i i =[m i,1 ,m i,2 ,...,m i,n ], the local model parameters M trained by participant j j =[m j,1 ,m j,2 ,…,m j,n ], both have n dimensions. i and M j The Euclidean distance calculation formula is: The score of participant i is i =∑j∈{1…50} dis i,j .
[0098] Step 3.2: The server screens out non-malicious updates and aggregates them. Specifically, the server performs unsupervised K-means binary classification on all scores, using the Kmeans class of the cluster package in the scikit-learn library to perform clustering. Considering that the proportion of attackers generally does not exceed half of the number of all participants, a class with a larger number is selected for aggregation, and all updates of this class are averaged.
[0099] Step 3.3: The server updates the average obtained in step 3.2 Adding it to the global model of the previous round gives the global model of the new round, that is, The results are sent to all participants.
[0100] Step 4: Repeat steps 2-3 until the number of training rounds reaches 200.
[0101] Since the applicable scenario is personalized federated learning, the data distribution of each participant's test set is similar to that of the training set, but the test sets of the participants are different. Therefore, during the test, the average test accuracy of all participants is taken as the standard for measuring the performance of the framework. In addition, in the two backdoor attacks, a backdoor attack test set is assigned to non-attacker participants. The samples in the test set have backdoors that trigger attacks. The average test accuracy of the backdoor attack test sets of all participants is taken as the standard for measuring the success rate of the backdoor attack. The specific experimental results are shown in Tables 1 and 2. When testing the effect of the backdoor attack, we follow the existing work and adopt a more complex model structure resnet18. Note that in the grid backdoor attack, only introducing the differential privacy mechanism in training is sufficient to reduce the success rate of the attack to an ideal range.
[0102] Table 1 The defensive effect of the method proposed in the present invention on model poisoning attack
[0103]
[0104] Table 2 The defensive effect of the method proposed in this invention on backdoor attacks
[0105]
[0106] It can be seen that the present invention provides a privacy-preserving and robust personalized image recognition method based on federated learning, which is a personalized image recognition method based on average regularized multi-task learning. While introducing differential privacy to protect data in a type of multi-task learning, it adopts defense measures suitable for the scenario to effectively resist various attacks, protect the privacy of federated learning participants, and improve the robustness of personalized image recognition.
[0107] Of course, the present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may certainly make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.
Claims
1. A privacy-preserving and robust personalized image recognition method based on federated learning, characterized in that: Each participant uses a local image recognition model to perform personalized image recognition; wherein the training method of the local image recognition model of each participant includes the following steps: S1: The central server sends the initialized global image recognition model to each participant; the number of image samples contained in the local training set of each participant is the same, and each image sample corresponds to a label. The number of label categories corresponding to the image samples contained by all participants is N, and the label categories corresponding to the image samples contained by each participant are not exactly the same, and the number of label categories corresponding to the image samples contained by each participant does not exceed N / 2; S2: Determine whether it is the first iteration. If so, the average gradient of each participant based on the local training set Execute the stochastic gradient descent algorithm with differential privacy to update its own local image recognition model; if not, each participant updates the local model based on the average gradient of the local training set. and the model parameters θ of the global model for image recognition g Obtain the fused gradient, and then execute the stochastic gradient descent algorithm with differential privacy based on the fused gradient to update its own local image recognition model; wherein the local image recognition model has the same structure as the global image recognition model; S3: Each participant obtains the model parameters θ of its own updated local image recognition model l and the model parameters θ of the global model for image recognition g The difference between k =θ l -θ g , where k = 1, 2, …, K, K is the number of participants; S4: The central server calculates the difference M of each participant k Calculate a score k , and use K-means clustering algorithm to cluster all scores k Perform binary classification; S5: Get all the differences M in the category with more scores k The average And the average With model parameters θ g The sum of is used as the model parameter of the global image recognition model in the next round; S6: Determine whether the current number of iterations has reached the upper limit. If yes, obtain the final image recognition local model of each participant. If no, send the image recognition global model obtained in step S5 to each participant, and then re-execute steps S2 to S6.
2. A privacy-preserving and robust personalized image recognition method based on federated learning as claimed in claim 1, characterized in that: The average gradient of the local training set of any participant in step S2 The specific calculation method is: S21: For each image sample x i The resulting gradient g(x i ) to clip and get the clipped gradient Where C is the maximum bi-norm of the gradient of a single image sample, and ||·||2 indicates the calculation of the bi-norm; S22: From Gaussian distribution The noise is sampled in the image, and the noise is added to the sum of the clipped gradients to obtain the average gradient of the image samples contained in the local training set. Where L is the number of image samples contained in the local training set, and σ is the number that satisfies the inequality is an auxiliary variable of , δ is the probability of differential privacy, ε is the privacy budget of differential privacy, and I is the unit matrix.
3. The privacy-preserving and robust personalized image recognition method based on federated learning as claimed in claim 1, characterized in that: The difference M of any participant k in step S4 k Corresponding score k The calculation method is: S41: Assume that the difference M corresponding to participant k k =[m k,1 ,m k,2 ,...,m k,n ], where m k,1 ~m k,n is the difference of each parameter included in the model parameters corresponding to participant k, and n is the model parameter θ l The number of parameters contained in ; S42: Obtain the Euclidean distance dis between participant k and other participants k,j : Among them, the difference M corresponding to participant j j =[m j,1 ,m j,2 ,…,m j,n ],m j,1 ~m j,n The model parameters corresponding to participant k include the difference of each parameter; S43: Distance between participant k and all participants k,j The sum of the values is taken as the difference M of participant k k Corresponding score k .
4. The privacy-preserving and robust personalized image recognition method based on federated learning as claimed in claim 1, characterized in that: The calculation method of the fusion gradient of any participant in step S2 is: in, is the fusion gradient corresponding to participant k, is the average gradient corresponding to participant k, λ is the set weight, θ l [k] is the model parameter of the local model for image recognition corresponding to participant k.
5. The privacy-preserving and robust personalized image recognition method based on federated learning as claimed in claim 1, characterized in that: In step S4, the K-means clustering algorithm is used to cluster all scores. k The specific method for binary classification is: S4a: Randomly select two scores as the starting centroids c1 and c2 of the two clusters; S4b: Calculate each score separately k The distance d from the starting centroid c1 k,1 , the distance d from the starting centroid c2 k,2 : d k,1 =|score k -c1| d k,2 =|score k -c2| S4c: For each score k , compared with its corresponding d k,1 and d k,2 , and the scores k Divide into clusters with smaller distances to complete the current binary classification; S4d: Determine whether the binary classification result currently obtained is the same as the binary classification result obtained in the previous iteration. If yes, complete the binary classification. If no, proceed to step S4e. S4e: Get the average values of the scores of the two clusters of the current binary classification, and use the two average values as the centroids of the two clusters; S4f: Repeat steps S4b to S4d based on the new centroids of the two clusters until the binary classification is completed.
Citation Information
Patent Citations
Federated learning image classification method based on local differential privacy
CN113850272A
Self-adaptive clustering federal learning method with differential privacy
CN117634594A
TMI system satisfying localized differential privacy based on federated learning
CN117807622A
Federal learning gradient attack defense method based on joint differential privacy
CN118296651A
Private computation of an agent data attribution score in collaborated task
US20220188692A1
