An Unsupervised Federated Learning Poisoning Defense Method Based on High-Dimensional Space Clustering

Abstract

This invention discloses a method for defending against poisoning attacks in unsupervised federated learning based on high-dimensional space clustering, belonging to the field of federated learning technology. The overall framework of this method mainly comprises four parts: a data preprocessor, a clusterer, a selector, and an aggregator. The method first preprocesses the model parameters from the client, extracting one layer of the multi-layer network model for cluster analysis, and then uses PCA to reduce the dimensionality of the extracted model parameters, extracting better model parameter features. Next, the dimensionality-reduced model parameters are fed into m K-means clusterers for clustering. Then, a selector selects the cluster to which benign participants belong. Finally, the clusterer aggregates the model parameters of the benign participants. Experimental results show that the method of this invention can effectively defend against various typical federated learning poisoning attacks.

Description

Technical Field

[0001] This invention belongs to the field of federated learning technology, specifically relating to an unsupervised federated learning poisoning defense method based on high-dimensional space clustering. Background Technology

[0002] Federated learning systems are vulnerable to attacks from malicious clients, which has become a major obstacle to their practical deployment. In federated learning systems, the central server can neither control client behavior nor access client private data. Therefore, malicious clients can launch attacks on the global model by sending modified and harmful model updates, thereby degrading the global model's learning performance or forcing targeted poisoning attacks. This invention considers two types of poisoning attacks: untargeted attacks and targeted attacks. Untargeted attacks aim to reduce the overall performance of the model, typically such as Byzantine attacks and the adversarial example poisoning attacks described in Chapter 3 of this invention. Targeted attacks aim to modify the model's behavior on certain specific data instances chosen by the attacker (e.g., identifying an image of a cat as a dog) while keeping the model's performance unaffected on other data instances, such as backdoor attacks. Both untargeted and targeted attacks can lead to catastrophic consequences. Therefore, timely detection of attackers and their harmful model updates, and their removal from the federated learning system to prevent malicious models from poisoning the global federated learning model, becomes crucial.

[0003] Research has been conducted on poisoning attack defenses against federated learning, with typical examples including pruning defenses such as Krum, Trimmed Mean, and Geometric Median (GeoMed). These defenses are effective when the number of malicious participants is small, but their performance is limited when the number of attackers exceeds half of all participants. Furthermore, these defenses are more suitable for data with IID distributions; their performance is relatively poor in federated learning scenarios with non-IID data distributions. Che et al. used model performance to determine users, and Li et al. used autoencoders to detect anomalous model updates. However, these methods all assume the existence of a validation set on the server side, which could leak data from benign clients, contradicting the privacy protection goals of federated learning. In summary, there are three main challenges in defending against poisoning attacks: (1) the data in each client consists of different amounts and distribution characteristics, i.e., the data is non-IID; (2) the performance of defense methods is limited when the number of malicious participants exceeds half of the total training participants; and (3) defense methods based on server-side validation sets can leak data from benign participants, contradicting the privacy protection goals of federated learning. Summary of the Invention

[0004] To address the aforementioned problems, this invention proposes an unsupervised federated learning poisoning defense method based on high-dimensional space clustering. This method comprises four parts: a data preprocessor, a clusterer, a selector, and an aggregator. First, the model parameters from the client are preprocessed, extracting one layer of the multi-layer network model for cluster analysis. PCA is then used to reduce the dimensionality of the extracted model parameters, extracting better model parameter features. Next, the dimensionality-reduced model parameters are fed into m K-means clusterers for clustering. Then, a selector selects the cluster to which benign participants belong. Finally, the clusterer aggregates the model parameters of the benign participants. The working steps of each module are as follows:

[0005] The data preprocessor first processes the model parameters uploaded by each participant, selects a certain layer of model parameters as clustering data, extracts features using PCA, and distributes the dimensionality-reduced model parameters to the clusterer.

[0006] The clusterer contains m clustering models, and each clustering model uses the K-means clustering algorithm to cluster the parameters after dimensionality reduction;

[0007] The selector performs two main operations. First, it filters the clustering results of each clustering model and selects clusters whose sample size meets the set threshold. Then, it evaluates all clusters that meet the conditions using intra-cluster distance and selects the cluster to which the model parameters finally belong for clustering.

[0008] The aggregator aggregates the selected model parameters based on the FedAvg aggregation algorithm.

[0009] Furthermore, the data preprocessor obtains a low-dimensional representation of the model parameters from each training participant. Considering a scenario with n participants, each participant uses the same local network model. Each network model contains multiple layers of model parameters. Only one layer of model parameters is extracted for cluster analysis, and the extracted model parameters are reduced in dimensionality using principal component analysis (PCA) to extract better parameter features.

[0010] Assume that the model parameters of each extracted participant contain m-dimensional features W = {W1, W2, ..., W...} m The PCA dimensionality reduction process involves the following steps:

[0011] S1. Calculate the covariance matrix of the model parameters: R = E((WX)(WX)) T (4.1)

[0012] Where E(.) is the expectation operation, and X is the mean of W;

[0013] S2. Perform eigenvalue decomposition on the covariance matrix obtained in step S1 to obtain eigenvalues ​​and vectors:

[0014] Where λ and x are the eigenvalues ​​and eigenvectors of the covariance matrix R, respectively, and σ 2 Let be the noise variance, and λ1≥λ2≥…≥σ. 2 The first l eigenvalues, corresponding to the principal components of the eigenvectors, contain the main information of the original model parameters; the last ml eigenvalues ​​represent the secondary components of the eigenvectors, containing random noise in the original model parameters.

[0015] S3. Select the number of eigenvalues ​​that account for 90% of the total energy of the eigenvalue spectrum as the number of principal components l, that is:

[0016] The original model parameters w are projected onto l principal component spaces to achieve dimensionality reduction.

[0017] Furthermore, the clusterer contains multiple clustering models, each using the K-means clustering algorithm to cluster the dimensionality-reduced model parameters. In the K-means algorithm, the similarity between data is measured by Euclidean distance. Let the model parameter set W have n samples participating in clustering, each sample having l-dimensional features. Any two samples W... i and W j The Euclidean distance between them is denoted as d. i,j =d(W i W j ),

[0018]

[0019]

[0020] The overall process of using the K-means algorithm to cluster model parameters is as follows:

[0021] S11. Randomly select K sets of model parameters as initial cluster centers, and define the set of initial cluster centers as: S (0) ={W1 (0) W2 (0) W3 (0) ,...,W k (0)} (4.5)

[0022] S12. Divide each set of model parameters and determine its cluster; the division of any parameter W satisfies the rule of equation (4.5), that is, all data are divided into K non-overlapping clusters, and the initial category division C is obtained. (0) ;

[0023]

[0024]

[0025] S13. Recalculate the cluster center of each cluster, and take the mean of all parameters in each cluster as the new cluster point, to obtain a new set of cluster centers S. (1) And according to the data partitioning rules in equation (4.6), a new classification set C is obtained. (1) ;

[0026]

[0027] S (1) ={W1 (1) W2 (1) W3 (1) ,...,W k (1)} (4.7)

[0028] Where, m i It is the sum of all data in category i;

[0029] S14. Repeat steps S12 and S13 until the cluster centers no longer change or the preset number of iterations is reached, at which point the classification ends.

[0030] Furthermore, the selector first filters the clustering results of each clustering model, selecting clusters whose sample size meets the set threshold; the selector evaluates all clusters that meet the conditions using intra-cluster distance, selecting the cluster to which the model parameters finally belong for clustering; the selector calculates the intra-cluster distance of all clusters that meet the conditions selected in the first step, and selects the cluster with the smallest intra-cluster distance as a benign participant; the intra-cluster distance is represented by the average Euclidean distance from each set of model parameters within the cluster to the cluster center, as shown in Equation (4.8), where S is the distance between the cluster center and the cluster center. i The set of all model parameters for the cluster whose center is:

[0031]

[0032] Among them, W p Let |C| represent the model parameters of the p-th group in the i-th cluster. i | represents the total number of data in the i-th cluster, q, and t represents the total number of clusters in the m clustering models that meet the sample size requirement;

[0033] The purpose of the aggregator is to aggregate the model parameters of the selected benign participants to obtain the final global model;

[0034]

[0035] As shown in Equation (4.9), the aggregator uses the classic federated learning aggregation algorithm for aggregation, W globThis is the global model obtained through aggregation.

[0036] The beneficial effects of this invention are as follows: The overall framework of this method mainly includes four parts: a data preprocessor, a clusterer, a selector, and an aggregator. This method first preprocesses the model parameters from the client, extracts one layer of the multi-layer network model for cluster analysis, and uses PCA to reduce the dimensionality of the extracted model parameters, extracting better model parameter features. Then, the dimensionality-reduced model parameters are fed into K-means clusterers for clustering. Next, a selector selects the cluster to which benign participants belong. Finally, the clusterer aggregates the model parameters of the benign participants. Experimental results show that the method of this invention can effectively defend against various typical federated learning poisoning attacks. Attached Figure Description

[0037] Figure 1 This is a diagram of the unsupervised federated learning poisoning defense framework based on multidimensional spatial clustering of the present invention.

[0038] Figure 2 This is a bar chart showing the global model test accuracy under the gradient ascent attack of this invention.

[0039] Figure 3 This is a bar chart showing the global model's accuracy under adversarial sample poisoning attacks according to the present invention.

[0040] Figure 4 The graph shows the accuracy of the model and backdoor under the tag inversion attack of the present invention (30% attacker).

[0041] Figure 5 The figure shows the accuracy of the model and backdoor test (50% attacker) under the tag reversal attack of the present invention;

[0042] Figure 6 The figure shows the accuracy of the model and backdoor test (60% attacker) under the tag reversal attack of the present invention;

[0043] Figure 7 This is a test accuracy graph of a global model using different defense methods under the gradient ascent attack of this invention;

[0044] Figure 8 The graph shows the backdoor test accuracy (column 1 / column 2 / column 3: 30% / 50% / 60% attacker) under the tag reversal attack of the present invention;

[0045] Figure 9 Accuracy graphs of different defense models tested under the tag inversion attack of this invention;

[0046] Figure 10This is a graph showing the model test accuracy (Non-IID) under different defense methods against the tag inversion attack of this invention;

[0047] Figure 11 The graph shows the backdoor accuracy test results (column 1 / column 2 / column 3: 30% / 50% / 60% attack) under the tag reversal attack of the present invention using different defenses.

[0048] Figure 12 This is a graph showing the backdoor test accuracy (Non-IID) under different defense methods during the gradient ascent attack of this invention. Detailed Implementation

[0049] The purpose of this invention is to overcome the shortcomings of the prior art and provide an unsupervised federated learning poisoning defense method based on high-dimensional space clustering.

[0050] The overall framework of the method is as follows Figure 1 As shown, this method first preprocesses the model parameters from the client, extracts one layer of the multi-layer network model for cluster analysis, and uses PCA to reduce the dimensionality of the extracted model parameters to extract better model parameter features. Then, the dimensionality-reduced model parameters are fed into m K-means clusterers for clustering. Next, a selector selects the cluster to which benign participants belong. Finally, the clusterers aggregate the model parameters of the benign participants. Experimental results show that the method of this invention can effectively defend against various typical federated learning poisoning attacks.

[0051] Module 1: Data Preprocessor

[0052] The data preprocessor mainly obtains low-dimensional representations of model parameters from various training participants. Considering a scenario with n participants, each participant uses the same local network model, and each network model contains multiple layers of model parameters. This invention extracts only one layer of model parameters for cluster analysis, and uses Principal Component Analysis (PCA) [3] to reduce the dimensionality of the extracted model parameters and extract better parameter features.

[0053] PCA is a representative and widely used data dimensionality reduction method. Its basic idea is to linearly combine the correlated indicators in the data to obtain a smaller number of independent composite indicators. These composite indicators are called principal components, which can represent the useful information in the original data to the greatest extent.

[0054] Assume that the model parameters of each extracted participant contain m-dimensional features W = {W1, W2, ..., W...} m The PCA dimensionality reduction process involves the following steps:

[0055] Calculate the covariance matrix of the model parameters: R = E((WX)(WX)) T (4.1)

[0056] Where E(.) is the expectation operation, and X is the mean of W.

[0057] (1) Perform eigenvalue decomposition on the covariance matrix obtained in (1) to obtain eigenvalues ​​and vectors:

[0058] Where λ and x are the eigenvalues ​​and eigenvectors of the covariance matrix R, respectively, and σ 2 Let be the noise variance, and λ1≥λ2≥…≥σ. 2 The first l eigenvalues, corresponding to the principal components of the eigenvectors, contain the main information of the original model parameters. The last ml eigenvalues, representing the secondary components of the eigenvectors, mainly contain the random noise in the original model parameters.

[0059] (3) Select the number of eigenvalues ​​that account for 90% of the total energy of the eigenvalue spectrum as the number of principal components l, that is:

[0060] (2) Project the original model parameters w onto l principal component spaces to achieve dimensionality reduction.

[0061] Module 2: Clusterer Module

[0062] The clusterer contains multiple clustering models, each using the K-means clustering algorithm. [4] Cluster the parameters of the dimensionality-reduced model.

[0063] K-means is a partitioned clustering algorithm whose main goal is to divide the data into K clusters, where each data point belongs to only one cluster. Data within each cluster has high similarity, while data between clusters has relatively low similarity. In K-means, the similarity between data points is measured using Euclidean distance. Let the model parameter set W have n samples participating in clustering, each sample having l-dimensional features. For any two samples W... i and W j The Euclidean distance between them is denoted as d. i,j =d(W i W j ).

[0064]

[0065]

[0066] The overall idea of ​​using the K-means algorithm to cluster model parameters can be summarized as follows:

[0067] (1) Randomly select K sets of model parameters as initial cluster centers, and define the set of initial cluster centers as: S (0) ={W1 (0) W2 (0) W3 (0) ,...,W k (0)} (4.5)

[0068] (2) Divide each set of model parameters and determine its cluster. The partitioning of any parameter W satisfies the rule of equation (4.5), that is, all data are divided into K non-overlapping clusters, resulting in the initial category partitioning C. (0) .

[0069]

[0070]

[0071] (3) Recalculate the cluster center of each cluster, and take the mean of all parameters in each cluster as the new cluster point to obtain a new set of cluster centers S. (1) And according to the data partitioning rules in equation (4.6), a new classification set C is obtained. (1) . S (1) ={W1 (1) W2 (1) W3 (1) ,...,W k (1)} (4.7)

[0072] Where, m i It is the sum of all data in category i.

[0073] (4) Repeat steps (2) and (3) until the cluster center no longer changes or the preset number of iterations is reached, and the classification ends.

[0074] Module 3: Selectors and Aggregators

[0075] The selector first filters the clustering results of each clustering model, selecting clusters whose sample size meets a set threshold. Considering that when K is large during clustering, some discrete points may appear, meaning the number of samples within a cluster is very small, and the intra-cluster distance is also relatively small. However, in federated learning, when the vast majority of participants are attackers and only a very small number are benign participants, this is of little significance to the federated learning system. Therefore, the algorithm of this invention sets the number of samples within each cluster in each cluster model to be greater than one-third of the total number of training participants.

[0076] Next, the selector evaluates all clusters that meet the conditions using intra-cluster distance to select the cluster to which the model parameters belong for final clustering. In federated learning, benign participants have high similarity in their model parameter distributions. Therefore, intuitively, when the clusterer performs clustering, the parameters of benign clients will be clustered into the same cluster. In small-scale attack scenarios, where the number of malicious participants is much smaller than that of benign participants, benign participants can be distinguished by the number of samples in each cluster. However, in large-scale attacks, where the number of malicious participants exceeds half of the total training participants, it is impossible to classify benign and malicious participants by the number of samples in each cluster. This invention considers classification from the perspective of intra-cluster distance. Compared to malicious participants, the intra-cluster distance of the clusters to which benign participants belong is smaller. This is because for poisoning attacks, especially untargeted attacks, the attack objective of malicious participants is only to reduce the overall performance of the federated learning model. Therefore, even if each malicious participant uses the same attack method, their optimization objectives differ, and the intra-cluster distance of the model parameters of these participants is larger when they are clustered into the same cluster. For targeted attacks, most targeted poisoning attacks currently employ gradient amplification strategies to achieve better attack results, which can lead to greater intra-cluster distances among malicious participants. Furthermore, considering real-world scenarios, even if all participants use the same attack method, differences in parameter design can cause variations in model parameter distribution. For example, in label inversion attacks, different participants selecting different source and target categories will have different optimization objectives and different model parameter distributions.

[0077] Based on the above analysis, the selector calculates the intra-cluster distances of all clusters that meet the criteria selected in the first step, and selects the cluster with the smallest intra-cluster distance as a benign participant. This invention uses the average Euclidean distance from each set of model parameters within the cluster to the cluster center to represent the intra-cluster distance. As shown in equation (4.8), S is the distance between the C... i The set of all model parameters for the cluster whose center is the cluster.

[0078]

[0079] Among them, W p Let |C| represent the model parameters of the p-th group in the i-th cluster. i | represents the total number of data in the i-th cluster, q, and t represents the total number of clusters in the m clustering models that meet the sample size requirement. The purpose of the aggregator is to aggregate the model parameters of the selected benign participants to obtain the final global model. As shown in Equation (4.9), the aggregator uses the classic federated learning aggregation algorithm for aggregation, W glob This is the global model obtained through aggregation.

[0080] Experimental setup

[0081] This invention considers a typical federated learning scenario with 100 training participants, examining both IID and non-IID scenarios for each participant. In each round of communication, all participants engage in training. A certain number of these participants are malicious attackers; experiments are conducted to discuss the defense performance when the number of malicious participants is 30%, 50%, and 60%.

[0082] Two types of attacks are considered to evaluate defense methods: untargeted attacks and targeted attacks. For untargeted attacks, a gradient ascent attack is used. [5] Adversarial examples of poisoning attacks. [6] Using a backdoor attack. [7] This invention evaluates defense performance against targeted attacks. All experiments are based on image classification tasks, using the MNIST dataset for performance evaluation, which is described in detail in section 3.4.1. For Non-IID scenarios, this invention uses the MNIST dataset...

[10] The data is divided into 200 groups based on labels, with 300 training samples in each group. Two groups are randomly assigned to each training participant, ensuring each participant has data from both classes of labels. This invention uses a neural network with two convolutional layers for training. Each participant has 5 local training rounds, and the global training rounds are set to 50. The poisoning attack method and contrastive defense method used in this invention will be described in detail below.

[0083] Gradient ascent attack refers to a malicious actor performing gradient ascent instead of gradient descent during local training. In the experiment, the malicious actor reversed the sign of the local gradient updates when uploading model parameters to achieve the attack. Backdoor attack is a targeted attack aimed at changing the behavior of a machine learning model on a small amount of data while maintaining the main model performance on the entire test dataset. For backdoor attacks, this invention considers a semantic backdoor label reversal attack, where the malicious actor forcibly changes the model labels "7", "2", and "1" to labels "8", "1", and "5", respectively. Bhagoji et al.

[19] Gradient amplification is used to modify their updates so that the attack is not negated by the averaging mechanism of the FedAvg aggregation algorithm. This strategy is also employed in the attack in this invention. The adversarial poisoning attack uses adversarial examples for local training and employs a "learning rate amplification" attack strategy to achieve better attack results.

[0084] Krum [8]Among m local models, a local model similar to the others is selected as the global model. Specifically, in a federated learning scenario with m training participants and c attackers, the sum of the L2 distances between each local model parameter and its mc-2 nearest neighbors is calculated as the score for that model parameter, and the local model with the smallest score is selected as the global model for the next round. GeoMed [9] The idea behind the defense method is to sort the j-th parameter of all local models and use the median as the j-th parameter of the global model in the next round. When the number of training participants is even, the median is the average of the two middle parameters.

[0085] I. Experimental Study on the Performance of Poisoning Defense Methods

[0086] This section first examines the performance of the defense method of this invention under different attack methods when the data IID is distributed. Figure 2 and Figure 3 The figures show the experimental results for gradient ascent attack and adversarial poisoning attack, respectively. Without any attack, the global model test accuracy using the defense method of this invention is very close to the accuracy using the FedAvg aggregation algorithm. Under gradient ascent attack, when the number of malicious participants is 30%, 50%, and 60%, the global model test accuracy using the defense method of this invention increases from 13.67%, 11.30%, and 9.88% to 97.96%, 97.06%, and 97.33%, respectively. Under adversarial poisoning attack, the global model test accuracy using the defense method of this invention increases from 53.21%, 37.93%, and 26.55% to 96.44%, 94.91%, and 93.21%, respectively. Under both of the above-mentioned untargeted poisoning attack methods, the defense method of this invention achieves performance comparable to that without an attack.

[0087] Figure 4 , Figure 5 and Figure 6 The model test accuracy and backdoor test accuracy are calculated when the number of malicious participants is 30%, 50%, and 60%, respectively. Figure 4 As shown in (a) and 6(a), the model testing accuracy stabilizes at around 80% and 70% respectively under label inversion attacks. Under the defense method of this invention, the model testing accuracy is improved to 97%. Meanwhile, Figure 4 (b) to Figure 6 (b) shows the accuracy of classifying backdoor samples into the original and target categories when there is no defense. The test accuracy for classifying into the original and target categories fluctuates significantly when the proportion of malicious actors is 30% and 50%. When the proportion of malicious actors is 60%, the target category is predicted with high accuracy, while the test accuracy for the original category is almost 0%. (Comparison) Figure 4 (c) to 6(c) When the defense method of the present invention is used on the server side for aggregation, when the number of malicious participants is 30%, 50% and 60%, the original category is correctly predicted with high accuracy, while the accuracy rate on the target attack category is 0%, which reflects the effectiveness of the defense method of the present invention.

[0088] II. Comparative Experiment

[0089] Next, this invention compares the performance of its defense method with that of Krum and GeoMed under gradient ascent and label inversion attacks. The performance comparison results under gradient ascent attack are as follows: Figure 7 As shown, when no attacks are detected, the global model test accuracy using the defense method of this invention is 97.69%, compared to 94.65% and 94.69% for GeoMed and Krum defense methods, respectively, indicating a 3% improvement in global model test accuracy. Analysis reveals that GeoMed and Krum defenses select only one participant's model parameters as the global model parameters for each round of aggregation, while the defense method of this invention selects model parameters from more training participants for aggregation. When the number of malicious participants is 30%, all three defense methods exhibit good performance, but the defense method of this invention performs optimally, reaching 97.66%, an 82.99% improvement compared to the test accuracy of the undefended global model. When the proportion of malicious participants is 50% and 60%, the GeoMed defense method fails, with global model test accuracies of 10.57% and 9.86%, respectively. In contrast, the defense method of this invention still maintains the best defense performance, with global model accuracies of 97.07% and 97.33%, respectively. Under label reversal attacks, when the number of malicious participants is 30%, 50%, and 60%, the accuracy of classifying backdoor samples into the original and target categories under different defense methods is as follows: Figure 8 As shown, when the proportion of malicious participants is 50%, all three defense methods have good defense performance. However, when the proportion of malicious participants is greater than or equal to 50%, the accuracy of classifying backdoor samples into the original category and the target category under the GeoMed defense method has a relatively large fluctuation range. Both Krum and the defense method of this invention have achieved good defense performance, but judging from the experimental curves, the average accuracy of the defense method of this invention is higher.

[0090] like Figure 9 The figures show the model test accuracy using three different defense methods. In the absence of attack, the test accuracy using the defense method of this invention is almost the same as that using the FedAvg aggregation algorithm, while the test accuracy on the Krum and GeoMed aggregation algorithms both decrease. Under attack, the defense model using this invention also achieves the highest test accuracy, reaching 97.6%.

[0091] III. Performance Experiments in Non-IID Scenarios

[0092] Furthermore, this invention also investigates the performance comparison of different defense methods under Non-IID data distribution, using label inversion attacks and gradient ascent attacks. Under label inversion attacks, the global model test accuracy is as follows: Figure 10 As shown, the average test accuracies of models with malicious participant proportions of 30%, 50%, and 60% were 82.6%, 71.2%, and 66.8%, respectively. The experimental results indicate that the Krum defense method fails in Non-IID scenarios, with an average accuracy of only 13.8%. When the number of malicious participants is 30%, both the GeoMed defense method and the method of this invention exhibit excellent defense performance. When the number of malicious participants is 50% and 60%, the defense method of this invention shows greater oscillations in the global model's test accuracy, while the GeoMed defense is more stable. After 20 epochs, the average test accuracy of the global model of this invention is higher than that of the GeoMed method.

[0093] Figure 11 As shown, the classification accuracy of backdoor samples in both the original and target categories exhibits results similar to the global model's test accuracy. The Krum defense method fails under different proportions of malicious actors. When the proportion of malicious actors is 30%, the GeoMed defense method performs best, with an accuracy of 96.5% in classifying samples into the original category and 0% in classifying them into the target attack category. Under the defense method of this invention, the accuracy of classifying samples into the original category is 94.2%, and the accuracy of classifying them into the target attack category is 0.8%. When the proportion of malicious actors is 50% and 60%, the GeoMed defense method cannot correctly classify backdoor samples into the original category, while the defense method of this invention can achieve test accuracies of 89.6% and 93.5%, respectively.

[0094] Figure 12 The experimental results are shown under gradient ascent attack. Similar to the test performance on backdoor attacks, the Krum defense method fails, and the global model test accuracy is only 16.34% without attack. When the proportion of malicious participants is 30%, the test accuracy of the defense method of this invention is 90.73%, and the test accuracy of the GeoMed defense method is 78.93%. However, when the number of malicious participants is 50% and 60%, respectively, none of the three defense methods can provide effective defense.

[0095] IV. The Impact of the Number of Cluster Models on Defense Performance

[0096] This section first discusses the impact of the number of clustering models, *m*, on defense performance. In the experiment, *m* was set to 6. Under label inversion attacks, with malicious participants representing 30% and 60% of the total, five epochs were randomly selected. The number of intra-cluster data and intra-cluster distances for each clustering model are shown in Tables 4.5 and 4.6. The experimental results show that 2-means can generally distinguish between benign and malicious participants, but it performs incorrect clustering in several epochs. For example, in the 33rd epoch in Table 4.5 and the 9th epoch in Table 4.6, the clustering results of 2-means are 100 and 0 respectively, meaning all participants are grouped into the same class. Therefore, using multiple clustering models simultaneously can ensure defense performance. However, the experimental results show that when using five classification models simultaneously (i.e., 2-means to 6-means), the selector's selection result is not optimal each time. As shown in Table 4.5, the clustering models and clusters selected for the 33rd and 42nd epochs are 4-0 and 5-4, respectively, but the optimal results should be 3-0 and 2-0. Analysis reveals that clustering models with K greater than 2 may disperse benign or malicious participants during clustering. When benign participants are divided into multiple clusters, the selected cluster may have a small number of members, and the intra-cluster distance may be less than that of all benign participants' clusters, indicating that cluster selection is not optimal. Experimental results show that using both 2-means and 3-means clustering models simultaneously achieves optimal defense performance.

[0097] Furthermore, we analyzed the feasibility of the "minimum intra-cluster distance" clustering method used in this invention. When K is less than 5, malicious participants can be completely separated from benign participants, and benign participants are clustered together. As shown in Table 4.5, when epochs are 3, 8, 15, and 42, 2-means, 3-means, and 4-means all group benign participants into the same cluster, with 70 data points within each cluster. In Table 4.6, when epochs are 5, 12, 37, and 48, K values ​​of 2, 3, 4, and 5 all group benign participants into the same cluster, with 40 data points within each cluster. The experimental data shows that the intra-cluster distance of benign participants is smaller than that of malicious participants. In Table 4.5, for the third epoch, the clustering results of 2-means show that the intra-cluster distance of the cluster belonging to benign participants is 2.97, while the intra-cluster distance of the cluster belonging to malicious participants is 5.51. In Table 4.5, at the 5th epoch, the clustering results for 2-means show that the intra-cluster distance of the cluster belonging to benign participants is 1.40, while the intra-cluster distance of the cluster belonging to malicious participants is 3.41. In Table 4.6, at epoch 12, the clustering results for 4-means show three clusters with a total of 30 members; however, the intra-cluster distance of the cluster belonging to benign participants is the smallest.

[0098] Table 4.5 Clustering results and final cluster selection results of the clustering model (30% of attackers)

[0099] Tab.4.5:The clustering results for each clustermodel and final selectresult(30%attackers)

[0100]

[0101]

[0102] Table 4.6 Clustering results and final cluster selection results of the clustering model (60% of attackers)

[0103] Tab.4.6:The clustering results for each cluster model and finalselectresult(60%attackers)

[0104]

[0105]

[0106] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. An unsupervised federated learning poisoning defense method based on high-dimensional space clustering, characterized in that, It consists of four parts: data preprocessor, clusterer, selector and aggregator. First, the model parameters from the client are preprocessed, one layer of the multi-layer network model is extracted for cluster analysis, and PCA is used to reduce the dimensionality of the extracted model parameters and extract better model parameter features. The dimensionality-reduced model parameters are then fed into m K-means clusterers for clustering; a selector is then used to select the cluster to which benign participants belong; finally, the clusterers are used to aggregate the model parameters of the benign participants; the working steps of each module are as follows: The data preprocessor first processes the model parameters uploaded by each participant, selects a certain layer of model parameters as clustering data, extracts features using PCA, and distributes the dimensionality-reduced model parameters to the clusterer. The clusterer contains m clustering models, and each clustering model uses the K-means clustering algorithm to cluster the parameters after dimensionality reduction; The selector performs two main operations. First, it filters the clustering results of each clustering model and selects clusters whose sample size meets the set threshold. Then, it evaluates all clusters that meet the conditions using intra-cluster distance and selects the cluster to which the model parameters to be used for the final clustering. The aggregator aggregates the selected model parameters based on the FedAvg aggregation algorithm.

2. The unsupervised federated learning poisoning defense method based on high-dimensional space clustering as described in claim 1, characterized in that, The data preprocessor obtains low-dimensional representations of model parameters from each training participant, considering the total number of participants. In a scenario with multiple participants, each participant uses the same local network model. Each network model contains multiple layers of model parameters. Only one layer of model parameters is extracted for cluster analysis, and the extracted model parameters are dimensionality reduced using principal component analysis (PCA) to extract better parameter features. Assume that the model parameters extracted for each participant include 3D features The PCA dimensionality reduction process involves the following steps: S1. Calculate the covariance matrix of the model parameters: (4.1) in, For the expected operation, yes The mean; S2. Perform eigenvalue decomposition on the covariance matrix obtained in step S1 to obtain eigenvalues ​​and vectors: (4.2) in, and These are the covariance matrices. eigenvalues ​​and eigenvectors, Let V be the noise variance, and The above equation is obtained by decomposing it. and Two parts of eigenvalues, the first The principal components of the eigenvectors corresponding to each eigenvalue contain the main information about the original model parameters; (The rest of the text appears to be incomplete and requires further context.) Each eigenvalue represents a sub-component of the eigenvector, containing random noise from the original model parameters; S3. Select the number of eigenvalues ​​that account for 90% of the total energy of the eigenvalue spectrum as the number of principal components. ,Right now: (4.3) Original model parameters Projected to Dimensionality reduction is achieved in each principal component space.

3. The unsupervised federated learning poisoning defense method based on high-dimensional space clustering as described in claim 1, characterized in that, The clusterer contains multiple clustering models, each using the K-means clustering algorithm to cluster the dimensionality-reduced model parameters. In the K-means algorithm, the similarity between data is measured by Euclidean distance. Let the model parameter set... Total Each sample participates in clustering, and each sample has... 3D features, any two samples and The Euclidean distance between them is expressed as = , （4.4) The overall process of using the K-means algorithm to cluster model parameters is as follows: S11, Random selection The model parameters are used as the initial cluster centers, and the set of initial cluster centers is defined as follows: (4.5) S12. Divide each group of model parameters into its corresponding cluster; for any parameter The partitioning satisfies rule (4.5), that is, all data are divided into non-overlapping groups. Clusters, resulting in initial category classification ; (4.6) S13. Recalculate the cluster center of each cluster, and take the mean of all parameters in each cluster as the new cluster point to obtain a new set of cluster centers. And according to the data partitioning rules in equation (4.6), a new classification set is obtained. ; (4.7) in, It is a category The sum of all the data in the middle; S14. Repeat steps S12 and S13 until the cluster centers no longer change or the preset number of iterations is reached, at which point the classification ends.

4. The unsupervised federated learning poisoning defense method based on high-dimensional space clustering as described in claim 3, characterized in that, The selector first filters the clustering results of each clustering model, selecting clusters whose sample size meets the set threshold; then, the selector evaluates all clusters that meet the conditions using intra-cluster distance, and selects the cluster to which the model parameters finally belong for clustering. Selector Calculate the intra-cluster distances of all eligible clusters selected in the first step, and select the cluster with the smallest intra-cluster distance as the benign participant; the intra-cluster distance is represented by the average Euclidean distance from each set of model parameters within the cluster to the cluster center, as shown in Equation (4.8). For The set of all model parameters for the cluster whose center is: (4.8) in, Indicates the first The first in the cluster Group model parameters, Indicates the first Total number of data in the cluster , express The total number of clusters that meet the sample size requirement in a clustering model; The purpose of the aggregator is to aggregate the model parameters of the selected benign participants to obtain the final global model; (4.9) As shown in Equation (4.9), the aggregator uses the classic federated learning aggregation algorithm for aggregation. This is the global model obtained through aggregation.

Images