Anti-collusion federated learning method and system based on correlation analysis, and storage medium

Abstract

The invention discloses an anti-collusion federal learning method and system based on correlation analysis and a storage medium. The method comprises the steps that each client side calculates an accuracy difference value before and after model updating according to a local privacy data set and calculates an audit score according to the accuracy difference value; the central server calculates correlation scores of the groups according to the audit scores and screens the groups with the correlation scores lower than a preset threshold out of a training system; re-dividing client groups according to the model gradient norm of the groups; calculating a local model gradient average number of each client in the group, and issuing the local model gradient average number to the corresponding group as a new global model gradient of the group; iterations are repeated until a preset number of iterations is completed. According to the invention, the problems that the existing anti-collusion privacy protection technology is only suitable for a scene in which the proportion of an attacker who puts up a car to a total client is relatively low and the performance is poor when the data distribution is Non-IId are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of federated learning and privacy protection, and particularly relates to an anti-collusion federated learning method, system, and storage medium based on correlation analysis. Background Art

[0002] With the increasing concern about data privacy, federated learning, as a powerful and secure distributed machine learning paradigm, is widely used in the Internet of Things (IoT). To undermine the privacy protection of federated learning, many privacy attack means have emerged in the past few years, such as the free-riding attack proposed in 2021. This type of attack allows uploading forged updates to obtain the normal updates sent by the server and benefit from the well-trained global model without contributing its own private dataset and computing resources.

[0003] To counter the free-riding attack, the current privacy protection methods are mainly divided into three categories: (1) Anti-collusion privacy protection method based on update statistics: Calculate the similarity between the global update and the local update to calculate the contribution of the client. If an update deviates more from other updates or the global update, it is more likely to be identified as a free-riding attacker and thus excluded from the entire system. However, this method can only work well in scenarios where the proportion of free-riding attackers in the total number of clients is relatively low. Once the proportion is too high (for example, the proportion exceeds 40%), fair clients will be misclassified as free-riding opponents, resulting in a higher false alarm rate. (2) Anti-collusion privacy protection method based on aggregation neutralization: Aggregate operators that are insensitive to outliers, such as relying on the median, group the clients uploading the same category of updates, and update them separately by group when updating the model. In this way, updates that are too different from other updates are divided into independent groups, reducing the impact on the learning process. The disadvantage of this method is that it is only applicable to scenarios where the proportion of free-riding attackers in the total number of clients is relatively low. Once the proportion of free-riding attackers is too high (for example, the proportion exceeds 50%), the updates of honest clients will be identified as outliers and aggregated independently; and this method does not delete free-riding attackers, but only tolerates the negative impact brought by the attackers. (3) Anti-collusion privacy protection method based on model metrics: Rely on the local model and the local dataset for verification. If merging the update into the model will reduce the model accuracy, the client transmitting the update will be marked as an attacker. Although this method can work in scenarios where the proportion of free-riding attackers is relatively low or high, it fails when facing collusion attacks improved based on free-riding, and its performance is not good when the number of clients is large and the data distribution is Non-IId.

[0004] To address the problem of resisting collusion attacks while ensuring good performance in scenarios where the number of free-riding attackers is large, an anti-collusion federated learning method, system, and storage medium based on correlation analysis are proposed. Summary of the Invention

[0005] An embodiment of the present invention proposes an anti-collusion federated learning method, system, and storage medium based on correlation analysis to at least solve the problem that existing anti-collusion privacy protection technologies are only applicable to scenarios where the proportion of free-riding attackers in the total number of client devices is relatively low and the performance is poor when the data distribution is Non-IID.

[0006] According to an embodiment of the present invention, an anti-collusion federated learning method based on correlation analysis is proposed, including:

[0007] Step 1: Establish a federated learning training system based on a centralized architecture using the EMNITS dataset;

[0008] Step 2: After all local client devices join the training system, they are initially divided into a group, and a part of the client devices are randomly selected to become the first type of client devices, and the other client devices are the second type of client devices;

[0009] Step 3: All client devices that have not been screened out of the system receive the global model gradient sent by the central server and perform iterative training;

[0010] Step 4: If this is not the first round of iteration, each client device calculates the difference value of the accuracy before and after the model update based on the local privacy dataset and calculates the audit score based on this;

[0011] Step 5: The central server calculates the correlation score of the group based on the audit score and screens out the groups with a correlation score lower than the preset threshold from the training system;

[0012] Step 6: Re-divide the client device groups according to the model gradient norm of the group;

[0013] Step 7: Calculate the average value of the local model gradients of each client device within the group and use the average value of the local model gradients as the new global model gradient of the group and send it to the corresponding group;

[0014] Step 8: Repeat steps 3 to 7 until the predetermined number of iteration rounds is completed.

[0015] In an exemplary embodiment, the federated learning training system based on a centralized architecture is composed of a central server node and local client nodes; both the central server node and the local client nodes adopt a multi-process working mode; the processes are divided into communication processes, gradient aggregation processes, grouping processes, accuracy management processes, and correlation analysis processes according to their functions.

[0016] In an exemplary embodiment, the first type of client is a collusive attacker, which uses the free-riding attack as the basic attack method, does not train the model, and transmits random model gradients to the central server. When the model metrics mutually audit and score, the first type of clients will give high scores to each other and randomly score other second type of clients; the second type of clients are honest clients, which use the global model gradients for iterative training and upload the local model gradients to the central server.

[0017] In an exemplary embodiment, each client calculates the accuracy difference value before and after the model update based on the local privacy dataset and calculates the audit score therefrom, including the steps of:

[0018] After iterative training, the local client updates to the global model according to the global model gradients.

[0019] The local client uses the global model to calculate the accuracy before the model update on its own local privacy dataset.

[0020] After obtaining the model gradients uploaded by the local client, the server adds Gaussian noise to them for privacy protection and transmits each model gradient to each local client within the system.

[0021] The client updates the model by bringing in the model gradients on its own local model and calculates the accuracy after the model update of each client using the local privacy dataset.

[0022] Calculate the accuracy difference value before and after the model update based on the accuracy before the model update and the accuracy after the model update of each client, and use it as the audit score for the accuracy of each client.

[0023] In an exemplary embodiment, the central server calculates the correlation score of the group based on the audit score and filters out the group with the correlation score lower than the preset threshold from the training system, including the steps of:

[0024] The central server constructs a one-dimensional array of audit scores for each local client based on the audit scores.

[0025] Calculate the group audit score array for each group based on the average value of the audit score arrays of the clients within the group.

[0026] Calculate the Spearman correlation coefficient between different groups, which is the correlation score of the group.

[0027] Filter out the group with the correlation score lower than the preset threshold from the training system.

[0028] In an exemplary embodiment, the re-grouping of clients according to the model gradient norm of the group includes the steps of:

[0029] The central server traverses all groups and calculates the average model gradient norm and the maximum model gradient norm of the groups;

[0030] If the average model gradient norm of a group is lower than the threshold while the maximum model gradient norm exceeds the threshold, the clients within the group are re-divided into two groups.

[0031] In an exemplary embodiment, the re-dividing the clients within the group into two groups includes the steps of:

[0032] Calculating the cosine similarity of the model gradients of each client within the group;

[0033] Calculating the Euclidean distance of the model gradients of each client within the group;

[0034] Calculating the Manhattan distance of the model gradients of each client within the group;

[0035] Calculating the mean norm of the model gradients of each client within the group;

[0036] Calculating the similarity based on the cosine similarity and / or Euclidean distance and / or Manhattan distance and / or mean norm of the model gradients of each client within the group;

[0037] Dividing the clients within the group with similarity greater than the preset threshold into one group, and the remaining clients into another group.

[0038] In an exemplary embodiment, the calculating the average of the local model gradients of each client within the group and using the average of the local model gradients as the new global model gradient of the group and sending it to the corresponding group includes the steps of:

[0039] After re-grouping, the central server calculates the average of the local model gradients of each client within the group by group;

[0040] Using the average of the local model gradients as the new global model gradient of the group;

[0041] Sending the new global model gradients of each group to the corresponding groups.

[0042] A computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program causes a computer to execute the above method.

[0043] According to another embodiment of the present invention, there is provided an anti-collusion federated learning system based on correlation analysis, including:

[0044] A processor;

[0045] A memory;

[0046] And

[0047] One or more programs, where the one or more programs are stored in a memory and configured to be executed by the processor, and the programs cause a computer to execute the above method.

[0048] The advantages of the anti-collusion federated learning method, system, and storage medium based on correlation analysis of the present invention are as follows:

[0049] (1) Configure the first type of client as a collusive attacker and the second type of client as an honest client in the training system, and perform iterative training using their respective training methods and audit scoring methods. Compared with the traditional training method that only targets free-riding attacks, it can resist both free-riding attacks and collusive attacks simultaneously.

[0050] (2) Each client calculates the accuracy difference value before and after model update based on the accuracy before model update and the accuracy after model update of each client and uses it as the audit score for the accuracy of each client. Compared with the traditional anti-collusion federated learning training method, it can effectively distinguish the audit scores of clients using collusive attacks and honest clients, facilitating the accurate identification of collusive attackers.

[0051] (3) The central server calculates the Spearman correlation coefficient between different groups based on the audit scores and filters out the groups with a correlation score lower than the preset threshold from the training system. Compared with the traditional anti-collusion federated learning training method, it can effectively distinguish clients using collusive attacks and honest clients and screen out collusive attack clients from the training system, and still ensure a high recognition accuracy when the number of clients is large and the data distribution is Non-IId.

[0052] (4) Calculate the similarity between clients based on the cosine similarity and / or Euclidean distance and / or Manhattan distance and / or norm mean of client signatures and re-divide the client groups accordingly. Compared with the traditional technical solution that only evaluates model similarity based on cosine similarity or Manhattan distance, it can effectively improve the accuracy of multiple iterative groupings, thereby effectively improving the efficiency of anti-collusion federated learning. Description of the Drawings

[0053] Figure 1 is a flowchart of an anti-collusion federated learning method based on correlation analysis according to an embodiment of the present invention;

[0054] Figure 2 is a flowchart of sub-step S04 of an embodiment of the present invention;

[0055] Figure 3 is a flowchart of sub-step S05 of an embodiment of the present invention;

[0056] Figure 4 is a flowchart of sub-step S06 of an embodiment of the present invention;

[0057] Figure 5 It is a flowchart of sub-step S062 of an embodiment of the present invention;

[0058] Figure 6 It is a flowchart of sub-step S07 of an embodiment of the present invention;

[0059] Figure 7 It is a schematic structural diagram of an anti-collusion federated learning system based on correlation analysis according to an embodiment of the present invention. Detailed implementation manners

[0060] The present invention will be described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the invention, but do not limit the invention in any form. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several changes and improvements can be made. These all belong to the protection scope of the present invention.

[0061] An anti-collusion federated learning method based on correlation analysis according to an embodiment of the present invention, the flowchart is as Figure 1 shown, including the steps:

[0062] Step 01: Establish a federated learning training system based on a centralized architecture using the EMNITS dataset;

[0063] Step 02: After all local clients join the training system, they are initially divided into a group and a part of the clients are randomly selected to become the first type of clients, and the other clients are the second type of clients;

[0064] Step 03: All clients not screened out of the system receive the global model gradient issued by the central server and perform iterative training;

[0065] Step 04: If this is not the first round of iteration, each client calculates the accuracy difference value before and after the model update based on the local privacy dataset and calculates the audit score accordingly;

[0066] Step 05: The central server calculates the correlation score of the group according to the audit score and screens out the group with the correlation score lower than the preset threshold from the training system;

[0067] Step 06: Re-divide the client groups according to the model gradient norm of the group;

[0068] Step 07: Calculate the average local model gradient of each client in the group and use the average local model gradient as the new global model gradient of the group and issue it to the corresponding group;

[0069] Step 08: Repeat steps 03 to 07 until the predetermined number of iteration rounds is completed.

[0070] In an exemplary embodiment, the federated learning training system based on a centralized architecture consists of a central server node and local client nodes; both the central server node and the local client nodes adopt a multi-process working mode; the processes are classified into communication processes, gradient aggregation processes, grouping processes, accuracy management processes, and correlation analysis processes according to their functions. In this embodiment, an EMNITS dataset is used to establish a federated learning training system based on a centralized architecture, and the data of different clients has Non-IId characteristics. Different clients choose to modify different dataset labels. For example, for client i, the label of data 2 is modified to 4 and the label of data 4 is modified to 2; for client j, the label of data 9 is modified to and the label of data 1 is modified to 9. The federated learning training system based on a centralized architecture consists of two types of nodes, namely, a central server node and local client nodes; the local client nodes are further divided into two types: honest local clients and colluding attackers; both the central server node and the local client nodes adopt a multi-process working mode, and the processes are classified into the following types according to their functions:

[0071] Communication processes: The number is equal to the number of local client nodes, and is used to maintain the communication of local model gradients and audit results with each client node;

[0072] Gradient aggregation processes: Used to aggregate the local model gradients sent by each client node to the central server node and handle the update of the global model gradient;

[0073] Grouping processes: Group according to the similarity of the local model gradients sent by local clients to the central server;

[0074] Correlation analysis processes: Used to perform correlation analysis based on each audit score and thereby control the screening of local clients;

[0075] Accuracy management processes: Used to manage the audit scores of each client.

[0076] In an exemplary embodiment, the first type of client is a collusive attacker, which uses the free-riding attack as the basic attack method. It does not train the model and transmits random model gradients to the central server. When the model metrics mutually audit and score, the first type of clients will give high scores to each other and randomly score other second type of clients; the second type of clients are honest clients, which use the global model gradients for iterative training and upload the local model gradients to the central server. In this embodiment, a part of the clients are randomly selected to be collusive attackers, that is, the first type of clients, and the other clients are honest clients, that is, the second type of clients. The collusive attackers (the first type of clients) use the free-riding attack as the basic attack method. They do not train the model and transmit forged random model gradients to the central server. When encountering a resistance attack method such as mutual audit and scoring of model metrics, the colluders will give high scores to each other to avoid screening and randomly score other non-collusive clients (the second type of clients) to disrupt the scoring system.

[0077] All N clients in total join the training system and are initially divided into a group to receive the initial global model issued by the central server. All clients not screened out of the system receive the global model gradients issued by the central server; the honest local clients use the global model gradients for iterative training and upload the local model gradients to the central server; the collusive attackers do not train and upload fabricated random gradient data to the central server.

[0078] In an exemplary embodiment, in step S04, each client calculates the accuracy difference value before and after the model update based on the local privacy dataset and calculates the audit score accordingly. The flowchart is as Figure 2 shown, including the steps:

[0079] Step S041: After iterative training, the local client updates to the global model according to the global model gradients.

[0080] Step S042: The local client uses the global model to calculate the accuracy before the model update in its own local privacy dataset.

[0081] Step S043: After the server obtains the model gradients uploaded by the local client, it adds Gaussian noise for privacy protection and transmits each model gradient to every local client within the system.

[0082] Step S044: The client brings the model gradients into its local model to update the model and uses the local privacy dataset to calculate the accuracy of each client after the model update.

[0083] Step S045: Calculate the accuracy difference value before and after the model update based on the accuracy before the model update and the accuracy of each client after the model update and use it as the audit score for the accuracy of each client.

[0084] In this embodiment, if this is not the first round of iteration, each client performs accuracy auditing (i.e., calculates the audit score).

[0085] After iterative training, the local client model is updated to the global model relying on the global model gradient. Each client uses the global model to calculate the accuracy before model update on its own local private dataset

[0086] After the central server obtains the model gradients uploaded by local clients 1 to N, it adds Gaussian noise to them for privacy protection and passes these N model gradients to N local clients.

[0087] After client i (i ∈ N) receives the model gradients of client j (j = 1, 2, 3, … N, j ≠ i), it respectively brings the model gradients into its own local model to update the model, and uses the local private dataset to calculate the accuracy of each client's model after update

[0088] Subtract from to obtain the accuracy difference value before and after model update Take the accuracy difference value before and after model update as the audit score of client i for the accuracy of other clients, and upload the audit score to the central server.

[0089] In an exemplary embodiment, in step S05, the central server calculates the correlation score of the group according to the audit score and screens out the training systems for the groups whose correlation scores are lower than the preset threshold. The flowchart is as Figure 3 shown, including the steps:[[]]

[0090] Step S051: The central server constructs a one-dimensional array of audit scores for each local client according to the audit scores.

[0091] Step S052: Calculate the group audit score array for each group according to the average value of the audit score arrays of the clients within the group.

[0092] Step S053: Calculate the Spearman correlation coefficient between different groups, which is the correlation score of the group.

[0093] Step S054: Screen out the training systems for the groups whose correlation scores are lower than the preset threshold.

[0094] In this embodiment, after the central server receives the audit scores sent by each client, it counts the audit scores of each client to form a one-dimensional array (the audit score of the client for itself is uniformly set to 1), denoted as Calculate the group audit score array for each group based on the average value of the audit score arrays of the clients within the group.

[0095] The server traverses each group (initially 1 group) to calculate the correlation between the audit score of audit group k and the audit score of group v, and calculates the Spearman correlation coefficient of group k as That is the correlation score of group k;

[0096] After obtaining the correlation scores of all groups, the central server filters out the groups with correlation scores lower than the preset threshold for the training system.

[0097] In an exemplary embodiment, in step S06, re-divide the client groups according to the model gradient norm of the groups, and the flowchart is as Figure 4 shown, including the steps:

[0098] Step S061: The central server traverses all groups and calculates the average model gradient norm and the maximum model gradient norm of the groups;

[0099] Step S062: If the average model gradient norm of the group is lower than the threshold while the maximum model gradient norm exceeds the threshold, re-divide the clients within the group into two groups.

[0100] In this embodiment, the central server traverses all groups, calculates the average model gradient norm and the maximum model gradient norm of the groups. If the average model gradient norm of the group is lower than the threshold while the maximum model gradient norm exceeds the threshold, re-group according to each model gradient into two groups.

[0101] In an exemplary embodiment, in step S062, re-divide the clients within the group into two groups, and the flowchart is as Figure 5 shown, including the steps:

[0102] Step S0621: Calculate the cosine similarity of the model gradients of the clients within the group;

[0103] Step S0622: Calculate the Euclidean distance of the model gradients of the clients within the group;

[0104] Step S0623: Calculate the Manhattan distance of the model gradients of the clients within the group;

[0105] Step S0624: Calculate the norm mean of the model gradients of the clients within the group;

[0106] Step S0625: Calculate the similarity according to the cosine similarity and / or Euclidean distance and / or Manhattan distance and / or norm mean of the model gradients of the clients within the group;

[0107] Step S0626: Divide the clients with an intra-group similarity greater than a preset threshold into one group, and the remaining clients into another group.

[0108] In this embodiment, the cosine similarity of the model gradients of each client within the group is calculated by taking the ratio of the vector product to the product of the norm lengths between the model gradients of two clients to calculate the cosine value of the included angle, which is the cosine similarity, and the cosine similarity is represented by c.

[0109] Calculate the Euclidean distance of the model gradients of each client within the group according to the distance between the model gradient vectors of each client within the group. The Euclidean distance is represented by the variable d.

[0110] Calculate the Manhattan distance of the model gradients of each client within the group according to the sum of the absolute axis distances of the model gradients of each client within the group on the standard coordinate system. The Manhattan distance is represented by the variable n.

[0111] Calculate the norm mean of the model gradients of each client within the group according to the mean value of the gradient norms of each client within the group. The norm mean is represented by p.

[0112] The calculation of the similarity according to the cosine similarity and / or Euclidean distance and / or Manhattan distance and / or norm mean of the model gradients of each client within the group is based on the positive correlation between the cosine similarity between models and the similarity and / or the negative correlation between the Euclidean distance and the similarity and / or the negative correlation between the Manhattan distance and the similarity and / or the negative correlation between the norm mean and the similarity to calculate the similarity between clients. The similarity between clients is represented by e.

[0113] Examples A1 to A15 represent different implementation manners for calculating the similarity between clients. For the sake of convenience of expression, the cosine similarity c, Euclidean distance d, Manhattan distance n, and norm mean p are calculated by using the method described in any of the above implementation manners.

[0114] Example A1: Calculate the similarity between clients according to the cosine similarity.

[0115] Specifically, calculate the cosine value of the included angle by taking the ratio of the vector product to the product of the norm lengths between the model gradients of two clients, which is the cosine similarity, and the cosine similarity is represented by c; calculate the similarity e between clients according to the positive correlation between the cosine similarity c and the similarity between clients. In a preferred implementation manner, calculate the similarity e between clients = o1·c o2 + o3, where o1, o2 (o1·o2 > 0), and o3 are calculation coefficients obtained by prior training.

[0116] Example A2: Calculate the similarity between clients according to the Euclidean distance.

[0117] Specifically, the Euclidean distance of the gradients of each client model within the group is calculated based on the distances between the gradient vectors of each client model within the group, and the Euclidean distance is represented by the variable d; the similarity e between clients is calculated based on the negative correlation between the Euclidean distance d and the similarity between clients. In a preferred embodiment, the similarity e between clients is calculated as e = o4·d o5 + o6, where o4, o5 (o4·o5 < 0), and o6 are calculation coefficients obtained through prior training.

[0118] Example A3: Calculate the similarity between clients based on the Manhattan distance.

[0119] Specifically, the Manhattan distance of the gradients of each client model within the group is calculated based on the total sum of the absolute axis distances of the gradients of each client model within the group on the standard coordinate system, and the Manhattan distance is represented by the variable n; the similarity e between clients is calculated based on the negative correlation between the Manhattan distance n and the similarity between clients. In a preferred embodiment, the similarity e between clients is calculated as e = o7·n o8 + o9, where o7, o8 (o7·o8 < 0), and o9 are calculation coefficients obtained through prior training.

[0120] Example A4: Calculate the similarity between clients based on the mean norm.

[0121] Specifically, the mean norm of the gradients of each client model within the group is calculated based on the mean of the gradient norms of the gradients of each client model within the group, and the mean norm is represented by the variable p. The similarity e between clients is calculated based on the negative correlation between the mean norm p and the similarity between clients. In a preferred embodiment, the similarity e between clients is calculated as e = o10·p o11 + o12, where o10, o11 (o10·o11 < 0), and o12 are calculation coefficients obtained through prior training.

[0122] Example A5: Calculate the similarity between clients based on the cosine similarity and the Euclidean distance.

[0123] Specifically, the cosine value of the included angle is calculated based on the ratio of the vector product to the product of the moduli between the gradient vectors of two client models, which is the cosine similarity, and the cosine similarity is represented by c; the Euclidean distance of the gradients of each client model within the group is calculated based on the distances between the gradient vectors of each client model within the group, and the Euclidean distance is represented by the variable d; the similarity e between clients is calculated based on the positive correlation between the cosine similarity c and the similarity between clients and the negative correlation between the Euclidean distance d and the similarity between clients. In a preferred embodiment, the similarity e between clients is calculated as e = o13·c o14 + o15·d o16+o17, where o13, o14 (o13·o14 > 0), o15, o16 (o15·o16 < 0), and o17 are calculation coefficients obtained through prior training. In another preferred embodiment, the similarity e between clients is calculated as e = o18·c o19 ·d o20 +o21, where o18, o19, o20, and o21 are calculation coefficients obtained through prior training.

[0124] Example A6: Calculate the similarity between clients based on cosine similarity and Manhattan distance.

[0125] Specifically, calculate the cosine value of the included angle based on the ratio of the vector product to the modulus product between the gradients of two client models, which is the cosine similarity, denoted by c; calculate the Manhattan distance of the gradients of each client model within the group based on the sum of the absolute axis distances of the gradients of each client model within the group on the standard coordinate system, denoted by the variable n; calculate the similarity e between clients based on the positive correlation between the cosine similarity c and the similarity between clients and the negative correlation between the Manhattan distance n and the similarity between clients. In a preferred embodiment, the similarity e between clients is calculated as e = o22·c o23 +o24·n o25 +o26, where o22, o23 (o22·o23 > 0), o24, o25 (o24·o25 < 0), and o26 are calculation coefficients obtained through prior training. In another preferred embodiment, the similarity e between clients is calculated as e = o27·c o28 ·n o29 +o30, where o22, o23, o24, and o30 are calculation coefficients obtained through prior training.

[0126] Example A7: Calculate the similarity between clients based on cosine similarity and norm mean.

[0127] Specifically, calculate the cosine value of the included angle based on the ratio of the vector product to the modulus product between the gradients of two client models, which is the cosine similarity, denoted by c; calculate the norm mean of the gradients of each client model within the group based on the mean of the gradient norms of each client model within the group, denoted by p; calculate the similarity e between clients based on the positive correlation between the cosine similarity c and the similarity between clients and the negative correlation between the norm mean p and the similarity between clients. In a preferred embodiment, the similarity e between clients is calculated as e = o31·c o32 +o33·p o34+ o35, where o31, o32 (o31·o32 > 0), o33, o34 (o32·o33 < 0), and o35 are calculation coefficients obtained through prior training. In another preferred embodiment, the similarity e between clients is calculated as e = o36·c o37 ·p o38 + o39, where o36, o37, o38, and o39 are calculation coefficients obtained through prior training.

[0128] Example A8: Calculate the similarity between clients based on the Euclidean distance and the Manhattan distance.

[0129] Specifically, calculate the Euclidean distance of the gradients of each client model within the group based on the distance between the gradient vectors of each client model within the group. The Euclidean distance is represented by the variable d; calculate the Manhattan distance of the gradients of each client model within the group based on the sum of the absolute axis distances of the gradients of each client model within the group on the standard coordinate system. The Manhattan distance is represented by the variable n; calculate the similarity e between clients based on the negative correlation between the Euclidean distance d and the Manhattan distance n and the similarity between clients. In a preferred embodiment, the similarity e between clients is calculated as e = o40·d o41 + o42·n o43 + o44, where o40, o41 (o40·o41 < 0), o42, o43 (o42·o43 < 0), and o44 are calculation coefficients obtained through prior training. In another preferred embodiment, the similarity e between clients is calculated as e = o45·d o46 ·n o47 + o48, where o45, o46, o47, and o48 are calculation coefficients obtained through prior training.

[0130] Example A9: Calculate the similarity between clients based on the Euclidean distance and the mean norm.

[0131] Specifically, calculate the Euclidean distance of the gradients of each client model within the group based on the distance between the gradient vectors of each client model within the group. The Euclidean distance is represented by the variable d; calculate the mean norm of the gradients of each client model within the group based on the mean of the gradient norms of each client model within the group. The mean norm is represented by p; calculate the similarity e between clients based on the negative correlation between the Euclidean distance d and the mean norm p and the similarity between clients. In a preferred embodiment, the similarity e between clients is calculated as e = o49·d o50 + o51·p o52 + o53, where o49, o50 (o49·o50 < 0), o51, o52 (o51·o52 < 0), and o53 are calculation coefficients obtained through prior training. In another preferred embodiment, the similarity e between clients is calculated as e = o54·d o55 ·p o56+o57, where o54, o55, o56, and o57 are calculation coefficients obtained through prior training.

[0132] Example A10: Calculate the similarity between clients based on the Manhattan distance and the mean of the norms.

[0133] Specifically, calculate the Manhattan distance of the gradients of each client model within the group based on the sum of the absolute axis distances of the gradients of each client model within the group on the standard coordinate system. The Manhattan distance is represented by the variable n; calculate the mean of the norms of the gradients of each client model within the group based on the mean of the gradient norms of each client model within the group. The mean of the norms is represented by the variable p; calculate the similarity e between clients based on the negative correlation between the Manhattan distance n and the mean of the norms p and the similarity between clients. In a preferred implementation, calculate the similarity e between clients = o58 · n o59 +o60 · p o61 +o62, where o58, o59 (o58 · o59 < 0), o60, o61 (o60 · o61 < 0), and o62 are calculation coefficients obtained through prior training. In another preferred implementation, calculate the similarity e between clients = o63 · n o64 · p o65 +o66, where o63, o64, o65, and o66 are calculation coefficients obtained through prior training.

[0134] Example A11: Calculate the similarity between clients based on the cosine similarity, Euclidean distance, and Manhattan distance.

[0135] Specifically, calculate the cosine value of the included angle, which is the cosine similarity, based on the ratio of the vector product to the product of the moduli between the gradients of two client models. The cosine similarity is represented by c; calculate the Euclidean distance of the gradients of each client model within the group based on the distance between the gradient vectors of each client model within the group. The Euclidean distance is represented by the variable d; calculate the Manhattan distance of the gradients of each client model within the group based on the sum of the absolute axis distances of the gradients of each client model within the group on the standard coordinate system. The Manhattan distance is represented by the variable n; calculate the similarity e between clients based on the positive correlation between the cosine similarity c and the similarity between clients, the negative correlation between the Euclidean distance d and the similarity between clients, and the negative correlation between the Manhattan distance n and the similarity between clients. In a preferred implementation, calculate the similarity e between clients = o67 · c o68 +o69 · d o70 +o71 · n o72 +o73, where o67, o68 (o67 · o68 > 0), o69, o70 (o69 · o70 < 0), o71, o72 (o71 · o72 < 0), and o73 are calculation coefficients obtained through prior training. In another preferred implementation, calculate the similarity e between clients = o74 · co75 ·d o76 ·n o77 + o78, where o74, o75, o76, o77, o78 are calculation coefficients obtained through prior training.

[0136] Example A12: Calculate the similarity between clients based on cosine similarity, Euclidean distance, and norm mean.

[0137] Specifically, calculate the cosine value of the angle according to the ratio of the product of the vectors between the gradients of two client models to the product of the norms, which is the cosine similarity, denoted by c; calculate the Euclidean distance between the gradient vectors of each client model within the group according to the distance between the gradient vectors of each client model within the group, denoted by the variable d; calculate the norm mean of the gradients of each client model within the group according to the norm mean of the gradients of each client model within the group, denoted by p; calculate the similarity e between clients according to the positive correlation between the cosine similarity c and the similarity between clients, the negative correlation between the Euclidean distance d and the similarity between clients, and the negative correlation between the norm mean p and the similarity between clients. In a preferred embodiment, calculate the similarity e between clients = o79·c o80 + o81·d o82 + o83·p o84 + o85, where o79, o80 (o79·o80 > 0), o81, o82 (o81·o82 < 0), o83, o84 (o83·o84 < 0), o85 are calculation coefficients obtained through prior training. In another preferred embodiment, calculate the similarity e between clients = o86·c o87 ·d o88 ·p o89 + o90, where o86, o87, o88, o89, o90 are calculation coefficients obtained through prior training.

[0138] Example A13: Calculate the similarity between clients based on cosine similarity, Manhattan distance, and norm mean.

[0139] Specifically, the cosine value of the included angle is calculated according to the ratio of the vector product to the product of the norms between the gradients of two client models, which is the cosine similarity, denoted by c; the Manhattan distance of the gradients of each client model within the group is calculated according to the total sum of the absolute axis distances of the gradients of each client model within the group on the standard coordinate system, and the Manhattan distance is denoted by the variable n; the mean of the gradient norms of the gradients of each client model within the group is calculated according to the mean of the gradient norms of the gradients of each client model within the group, and the mean of the norms is denoted by p; the similarity e between clients is calculated according to the positive correlation between the cosine similarity c and the similarity between clients, the negative correlation between the Manhattan distance n and the similarity between clients, and the negative correlation between the mean of the norms p and the similarity between clients. In a preferred embodiment, the similarity e between clients is calculated as e = o91·c o92 + o93·n o94 + o95·p o96 + o97, where o91, o92 (o91·o92 > 0), o93, o94 (o93·o94 < 0), o95, o96 (o95·o96 < 0), o97 are calculation coefficients obtained through prior training. In another preferred embodiment, the similarity e between clients is calculated as e = o98·c o99 ·n o100 ·p o101 + o102, where o98, o99, o100, o101, o102 are calculation coefficients obtained through prior training.

[0140] Example A14: Calculate the similarity between clients according to the Euclidean distance, Manhattan distance, and mean of the norms.

[0141] Specifically, the Euclidean distance of the gradients of each client model within the group is calculated according to the distance between the gradient vectors of each client model within the group, and the Euclidean distance is denoted by the variable d; the Manhattan distance of the gradients of each client model within the group is calculated according to the total sum of the absolute axis distances of the gradients of each client model within the group on the standard coordinate system, and the Manhattan distance is denoted by the variable n; the mean of the gradient norms of the gradients of each client model within the group is calculated according to the mean of the gradient norms of the gradients of each client model within the group, and the mean of the norms is denoted by p; the similarity e between clients is calculated according to the negative correlation between the Euclidean distance d, Manhattan distance n, and mean of the norms p and the similarity between clients. In a preferred embodiment, the similarity e between clients is calculated as e = o103·d o104 + o105·n o106 + o107·p o108+o109, where o103, o104 (o103·o104 < 0), o105, o106 (o105·o106 < 0), o107, o108 (o107·o108 < 0), o109 are calculation coefficients obtained through prior training. In another preferred embodiment, the similarity e between clients is calculated as e = o110·d o111 ·n o112 ·p o113 +o114, where o110, o111, o112, o113, o114 are calculation coefficients obtained through prior training.

[0142] Example A15: Calculate the similarity between clients based on cosine similarity, Euclidean distance, Manhattan distance, and norm mean.

[0143] Specifically, calculate the cosine value of the angle according to the ratio of the vector product to the modulus product between the gradients of two client models, which is the cosine similarity, denoted by c; calculate the Euclidean distance of the gradients of each client model within the group according to the distance between the gradient vectors of each client model within the group, denoted by the variable d; calculate the Manhattan distance of the gradients of each client model within the group according to the total sum of the absolute axis distances of the gradients of each client model within the group on the standard coordinate system, denoted by the variable n; calculate the norm mean of the gradients of each client model within the group according to the mean of the gradient norms of each client model within the group, denoted by the variable p; calculate the similarity e between clients according to the positive correlation between the cosine similarity c and the similarity between clients, the negative correlation between the Euclidean distance d and the similarity between clients, the negative correlation between the Manhattan distance n and the similarity between clients, and the negative correlation between the norm mean p and the similarity between clients. In a preferred embodiment, the similarity e between clients is calculated as e = o115·c o116 +o117·d o118 +o119·n o120 +o121·p o122 +o123, where o115, o116, o117, o118, o119, o120, o121, o122, o123 are calculation coefficients obtained through prior training (o115·o116 > 0), (o117·o118 < 0), (o119·o120 < 0), (o121·o122 < 0). In another preferred embodiment, the similarity e between clients is calculated as e = o124·c o125 ·d o126 ·n o127 ·p o128 +o129, where o124, o125, o126, o127, o128, o129 are calculation coefficients obtained through prior training.

[0144] According to the similarity e between clients calculated in the manner described in any one of Embodiments A1 to A15, clients with a similarity greater than a preset threshold are grouped into one group, and the remaining clients are grouped into another group.

[0145] As the number of iterations increases for honest clients, the model gradually stabilizes, and their average and maximum model gradient norms also gradually tend to lower values. For colluding clients, their maximum or average model gradient norms remain at a high level. These characteristics make it highly likely that honest clients and colluding attack clients will be grouped into different groups.

[0146] In an exemplary embodiment, in step S07, calculate the average of the local model gradients of each client within the group and use the average of the local model gradients as the new global model gradient for the group, and distribute it to the corresponding group. The flowchart is as Figure 6 shown, including the steps:

[0147] Step S071: After regrouping, the central server calculates the average of the local model gradients of each client within the group according to the group;

[0148] Step S072: Use the average of the local model gradients as the new global model gradient for the group;

[0149] Step S073: Distribute the new global model gradients of each group to the corresponding group.

[0150] In this embodiment, after grouping, the central server calculates the average of the local model gradients of each client within the group and uses the average of the local model gradients as the new global model gradient for the group. This reduces the harmful impact of colluding attacks on the entire system when the system training has just started and colluding attack clients have not been screened out. Subsequently, the central server distributes the new global model gradient to the corresponding group.

[0151] A computer-readable storage medium according to an embodiment of the present invention stores a computer program for electronic data exchange, wherein the computer program causes a computer to execute the above method.

[0152] A collusion-resistant federated learning system based on correlation analysis according to an embodiment of the present invention, the structural schematic diagram is as Figure 7 shown, including:

[0153] A processor;

[0154] A memory;

[0155] And

[0156] One or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the processor, and the programs cause a computer to execute the above method.

[0157] Of course, those of ordinary skill in the art should recognize that the above embodiments are only used to illustrate the present invention and are not intended to limit the present invention. As long as it is within the scope of the present invention, changes and modifications to the above embodiments will fall within the protection scope of the present invention.

Claims

1. An anti-collusion federated learning method based on correlation analysis, characterized in that include: Step 1: Use the EMNITS dataset to establish a centralized federated learning training system. Step 2: After all local clients join the training system, they are initially divided into a group and a portion of them are randomly selected as the first type of clients, and the rest are the second type of clients; Step 3: All clients that have not been screened out of the system receive the global model gradient sent by the central server and perform iterative training; Step 4: If this is not the first iteration, each client calculates the accuracy difference before and after the model update based on the local privacy dataset and uses this to calculate the audit score; Step 5: The central server calculates the relevance scores of the groups based on the audit scores and filters out the groups with relevance scores below a preset threshold from the training system; Step 6: Re-divide the client groups according to the model gradient norm of the group; Step 7: Calculate the average local model gradient of each client in the group and send the average local model gradient as the new global model gradient of the group to the corresponding group; Step 8: Repeat steps 3 to 7 until the predetermined number of iterations is completed.

2. The anti-collusion federated learning method based on correlation analysis according to claim 1, wherein The centralized architecture-based federated learning training system consists of a central server node and local client nodes; both the central server node and the local client node adopt a multi-process working mode; the processes are divided into communication processes, gradient aggregation processes, grouping processes, accuracy management processes and correlation analysis processes according to their functions.

3. The anti-collusion federated learning method based on correlation analysis according to claim 1, characterized in that The first type of client is a collusion attacker that uses a free-rider attack as its basic attack method. It does not train the model and transmits random model gradients to the central server. When the model metrics audit and score each other, the first type of clients will give each other high scores and randomly score other second type clients. The second type of client is an honest client that uses the global model gradient for iterative training and uploads the local model gradient to the central server.

4. The anti-collusion federated learning method based on correlation analysis according to claim 3, characterized in that, Each client calculates the accuracy difference before and after the model update based on the local privacy data set and calculates the audit score based on this, including the following steps: After iterative training, the local client is updated to the global model based on the global model gradient; The local client uses the global model to calculate the accuracy of the model before the update on its own local private dataset; After the server obtains the model gradients uploaded by the local client, it adds Gaussian noise to them for privacy protection and transmits each model gradient to the local client in each system. The client updates the model by introducing the model gradient on its local model and uses the local private dataset to calculate the accuracy of each client model after the update; The accuracy difference before and after the model update is calculated based on the accuracy of the model before the update and the accuracy of each client model after the update, and is used as the audit score for the accuracy of each client.

5. The anti-collusion federated learning method based on correlation analysis according to claim 4, characterized in that The central server calculates the relevance scores of the groups based on the audit scores and filters out the groups with relevance scores below a preset threshold from the training system, including the steps of: The central server constructs a one-dimensional array of audit scores for each local client based on the audit scores; Calculate the group audit score array for each group based on the average of the audit score arrays of each client in the group; The Spearman correlation coefficient between different groups was calculated, which was the correlation score of the groups; Select the groups with relevance scores lower than the preset threshold for training the system.

6. The anti-collusion federated learning method based on correlation analysis according to claim 5, characterized in that, The re - grouping of client groups according to the model gradient norm of the group includes the steps: The central server traverses all groups and calculates the average model gradient norm and the maximum model gradient norm of the groups; If the average model gradient norm of a group is lower than the threshold while the maximum model gradient norm exceeds the threshold, the clients within this group are re - divided into two groups.

7. The anti-collusion federated learning method based on correlation analysis according to claim 6, wherein The step of re - dividing the clients within the group into two groups includes the steps: Calculate the cosine similarity of the model gradients of each client within the group; Calculate the Euclidean distance of the model gradients of each client within the group; Calculate the Manhattan distance of the model gradients of each client within the group; Calculate the mean norm of the model gradients of each client within the group; Calculate the similarity based on the cosine similarity and / or Euclidean distance and / or Manhattan distance and / or mean norm of the model gradients of each client within the group; Divide the clients with similarity within the group greater than the preset threshold into one group, and the remaining clients into another group.

8. The anti-collusion federated learning method based on correlation analysis according to claim 1, characterized in that The step of calculating the average local model gradient of each client within the group and using the average local model gradient as the new global model gradient of the group and sending it to the corresponding group includes the steps: After re - grouping, the central server calculates the average local model gradient of each client within the group by group; Use the average local model gradient as the new global model gradient of the group; Send the new global model gradients of each group to the corresponding groups.

9. A computer-readable storage medium storing a computer program for electronic data exchange, wherein, The computer program causes the computer to execute the method according to any one of claims 1 - 8.

10. An anti-collusion federated learning system based on correlation analysis, characterized in that It includes: A processor; A memory; And One or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the processor, and the program causes the computer to execute the method according to any one of claims 1 - 8.

Images