A non-iid federated learning backdoor attack defense method, system and medium

Abstract

This invention discloses a method, system, and medium for defending against backdoor attacks in Non-IID federated learning, relating to the fields of federated learning and cybersecurity. Addressing the problems of existing methods relying on model parameters that are easily circumvented by malicious clients and poor adaptability to Non-IID scenarios, this invention involves the client performing truncated singular value decomposition to extract the first p left singular vectors and uploading them to the server. The server constructs a matrix based on the cosine similarity of the left singular vectors and performs hierarchical clustering. Combining this with left singular vectors from a clean benchmark dataset, a scaled dot product attention mechanism is used to calculate the client similarity score and aggregation weights. The server distributes a global model, and the client trains locally and uploads the parameters. The server then weights and aggregates the model parameters within the clusters and iteratively optimizes them. This invention identifies malicious clients based on the essential characteristics of the data, without requiring a pre-defined malicious proportion. It suppresses backdoor attacks while protecting the contributions of benign clients and data privacy, is suitable for Non-IID scenarios, and improves model robustness and main task performance.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of federated learning and network security, in particular to a method and system for preventing backdoor attacks in a non-independent and identically distributed (Non-IID) scenario by extracting data features, clustering and grouping, and dynamically assigning weights, and a medium. BACKGROUND

[0002] Federated learning is a distributed machine learning paradigm that can build a global model by local training of clients and aggregation of server models without converging the original data of each client, effectively protecting data privacy. However, in real-world applications, client data often exhibits non-independent and identically distributed characteristics, i.e., the data distribution of different clients is significantly different, resulting in slower convergence and lower accuracy of the global model. At the same time, the distributed architecture of federated learning is vulnerable to backdoor attacks: malicious clients can inject harmful trigger conditions into the global model by tampering with local data or model parameters, causing the model to output incorrect results under certain inputs. Such attacks are more difficult to detect in Non-IID (Not identically and independently distributed) scenarios due to the strong heterogeneity of model updates.

[0003] Existing defense methods rely on model update parameters (or gradient directions) for anomaly detection, distinguishing between benign and malicious clients by clustering model parameters. However, malicious clients can evade detection by deliberately mimicking the model update behavior of benign clients, leading to ineffective defense. In addition, some methods preset the proportion of malicious clients or use fixed thresholds for screening, which is difficult to adapt to dynamic changes in data distribution in Non-IID scenarios, and is prone to misjudgment of benign clients, resulting in performance loss of the global model. Therefore, there is an urgent need for a federated learning defense scheme that starts from the essential features of data, adapts to Non-IID environments, and accurately identifies hidden backdoor attacks. SUMMARY

[0004] To solve the above problems, the present application aims to provide a Non-IID federated learning backdoor attack defense method, system and medium, which extracts key features from client data using truncated singular value decomposition (SVD), groups clients with similar data distribution using hierarchical clustering, and assigns aggregated weights to clients using a scaled dot-product attention mechanism, reducing the impact of malicious client updates during model aggregation, improving model robustness while ensuring the training effectiveness of benign clients and data privacy.

[0005] To achieve the above purpose, the technical solution of the present application is as follows:

[0006] A Non-IID federated learning backdoor attack defense method, comprising the following steps:

[0007] S1, client feature extraction and uploading: the client performs truncated singular value decomposition on the local dataset, extracts the first p left singular vectors, and uploads them to the server;

[0008] S2, server clustering grouping: the server calculates the cosine similarity of the left singular vectors between clients, constructs a similarity matrix, and uses hierarchical clustering to group;

[0009] S3, server weight allocation: the server performs truncated singular value decomposition on the clean benchmark dataset to obtain benchmark vectors, calculates the similarity score between the client and the benchmark using the scaled dot product attention mechanism, and allocates the aggregation weight;

[0010] S4, model distribution and local training: the server initializes the global model and randomly selects a client for distribution; the selected client trains with local data and uploads local model parameters;

[0011] S5, model aggregation and iterative optimization: the server aggregates model parameters within the cluster according to the weight, generates a new global model, and iterates the process until convergence.

[0012] Further, the client feature extraction and uploading specifically refers to: each client based on the local dataset performs truncated singular value decomposition, through SVD: , the dataset is decomposed into left singular vector matrix , singular value matrix and right singular vector matrix , the left singular vector matrix corresponding to the first p largest singular values is retained, and is uploaded to the central server.

[0013] Further, the server clustering grouping specifically refers to: after the central server receives the left singular vectors of all clients , ,… , it calculates the cosine similarity of the left singular vector matrices and of any two clients and , constructs a similarity matrix S; then the server uses hierarchical clustering method to cluster the clients according to the similarity matrix , and groups the clients that meet the preset similarity threshold into the same group.

[0014] Further, the server clustering grouping includes the following steps:

[0015] Step S21: The server calculates for each client pair and The left singular vector matrix between cosine similarity ,in and Represent the left singular vector matrix respectively A left singular vector is used to obtain the similarity matrix. ,in Indicates client and Cosine similarity measure between them;

[0016] Step S22: Using all clients as initial individual clusters, perform bottom-up hierarchical clustering based on the similarity matrix calculated in step S21: continuously merge the two clusters with the highest similarity metrics until the preset minimum inter-cluster similarity threshold β is met or the preset number of clusters M is reached.

[0017] Furthermore, the server weight allocation specifically refers to: the server pre-acquiring a clean benchmark dataset. ,right Performing truncated singular value decomposition yields the reference left singular vector matrix. In each cluster, the server uses a scaled dot product attention mechanism to compute the attention value obtained from step S1 for each client. and The similarity between them is calculated, and the results are used to assign weights to each client.

[0018] Furthermore, the server weight allocation includes the following steps:

[0019] Step S31: The server uses truncated singular value decomposition. Obtain its left singular vector matrix ;

[0020] Step S32: The client set in the k-th cluster is The server calculates the i-th client of Compared with benchmark datasets of Attention score between scaling dot products ,in express The dimension is used to scale the dot product result to ensure the stability of the value;

[0021] Step S33: Within each cluster, calculate the client's attention score based on the scaled dot product attention score obtained in step S32. Weights during aggregation wherein is the total number of clients in the current cluster, is an exponential function, i.e. .

[0022] Further, the model distribution and local training refer to: the server initializes a global model , randomly selects part of the clients at the tth round of communication, and distributes the global model to the selected clients; the selected clients train the model based on the local data set using the stochastic gradient descent algorithm to obtain a local model , and upload to the server.

[0023] Further, the model aggregation and iterative optimization refer to: the server, in each cluster, performs weighted aggregation on the local models of the clients in the cluster based on the aggregation weight obtained in step S33 to obtain a new round of global model in the cluster, wherein represents the set of clients in the kth cluster; and distributes to the clients in the corresponding cluster, and repeats steps S1-S5 until the model converges or reaches a preset training round.

[0024] In order to achieve the above purpose, the present application also provides a Non-IID federated learning backdoor attack defense system, comprising:

[0025] A client feature extraction and uploading module: the client performs truncated singular value decomposition on the local data set, extracts the first p left singular vectors, and uploads them to the server;

[0026] A server clustering grouping module: the server calculates the cosine similarity of the left singular vectors between the clients, constructs a similarity matrix, and groups using hierarchical clustering;

[0027] A server weight allocation module: the server performs truncated singular value decomposition on the clean benchmark data set to obtain benchmark vectors, calculates the similarity scores of the clients and the benchmark using the scaled dot-product attention mechanism, and allocates aggregation weights;

[0028] A model distribution and local training module: the server initializes a global model, randomly selects clients for distribution; the selected clients train using local data, and upload local model parameters;

[0029] A model aggregation and iterative optimization module: the server performs weighted aggregation of model parameters in the cluster according to the weights, generates a new global model, and repeats the process until convergence.

[0030] To achieve the above object, the application further provides a computer readable storage medium storing a computer program, which, when executed by a processor, causes the processor to execute the Non-IID federated learning backdoor attack defense method.

[0031] Beneficial effects: The application significantly improves the ability of federated learning to resist backdoor attacks in a non-independent and identically distributed environment by introducing a left singular vector-based data distribution feature extraction and attention weight dynamic allocation mechanism. On the one hand, by extracting the essential features of client data through singular value decomposition, the data distribution anomaly caused by backdoor data can be accurately identified, effectively distinguishing between malicious clients and benign clients. On the other hand, combined with hierarchical clustering and scaled dot-product attention mechanism, appropriate aggregation weights can be adaptively allocated to different clients while preserving the similarity information of data distribution, thereby suppressing malicious updates and maximizing the contribution of benign clients. BRIEF DESCRIPTION OF DRAWINGS

[0032] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application, illustrate the preferred embodiments of the application and assist in the explanation of the application. The drawings included are:

[0033] Figure 1 The flowchart of the Non-IID federated learning backdoor attack defense method described in the embodiments of the application;

[0034] Figure 2 The interaction flow framework diagram between the client and the server of the Non-IID federated learning backdoor attack defense method described in the embodiments of the application; which includes client feature extraction, server clustering, weight calculation, model distribution and aggregation, etc.

[0035] Figure 3 The 3DPCA visualization comparison diagram of the left singular vectors extracted from the original data and the backdoor data in the embodiments of the application; wherein, Figure 3 (a) shows that there is a significant reverse alignment difference between the left singular vectors of part of the client's original data and backdoor data, Figure 3 (b) shows that when the difference between a few client vectors is small, the left singular vector set can still be used as a basis for differentiation;

[0036] Figure 4 The structural diagram of the Non-IID federated learning backdoor attack defense system described in the embodiments of the application. DETAILED DESCRIPTION

[0037] It should be noted that the embodiments in the application and the features in the embodiments can be combined with each other without conflict.

[0038] The application will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0039] Embodiment 1

[0040] Referring to Figures 1-3 A Non-IID federated learning backdoor attack defense method, comprising the following steps:

[0041] S1, client feature extraction and uploading: the client performs truncated singular value decomposition (truncated SVD) on the local data set, extracts the first p left singular vectors, and uploads them to the server;

[0042] It should be noted that each client does not send the original data, but only sends the "key label" representing the core characteristics of its own data to the server; in this way, the original data privacy is protected, and specific information is not disclosed; the essence of the data is grasped in advance, laying a foundation for subsequent identification of malicious data.

[0043] S2, server clustering grouping: the server calculates the cosine similarity of the left singular vectors between the clients, constructs a similarity matrix, and groups using hierarchical clustering;

[0044] It should be noted that the server judges the data characteristics of which clients according to the "key label", and groups similar ones together; it can be understood that the clients with similar data distribution are grouped together to avoid interference from different types of data, facilitating subsequent targeted screening.

[0045] S3, server weight distribution: the server performs truncated singular value decomposition on the clean benchmark data set to obtain a benchmark vector, and uses a scaling dot product attention mechanism to calculate the similarity score of the client and the benchmark to allocate an aggregation weight;

[0046] In this embodiment, the server uses a "non-abnormal data template" as a standard to score the clients in each group: the more similar to the template, the higher the score and the greater the weight; this achieves accurate differentiation between benign (similar to the template) and malicious (not similar to the template) clients; in addition, it avoids blindly guessing malicious clients based on a fixed proportion, and is more flexible and accurate.

[0047] S4, model distribution and local training: the server initializes a global model and randomly selects a client to distribute; the selected client trains using local data (SGD algorithm) and uploads local model parameters;

[0048] In this embodiment, the server first initializes a global model framework, randomly selects some clients and distributes the global model; the selected clients train the model based on the local data set, and upload the local model parameters to the server after training is completed; the above process completes the basic cycle of model training, enabling each client to achieve local optimization of the model based on its own data;

[0049] S5. Model Aggregation and Iterative Optimization: The server aggregates model parameters in clusters according to weights, generates a new global model, and distributes it. The process is repeated until convergence.

[0050] In this embodiment, the server assigns greater influence to high-weight (benign) client models and less influence to low-weight (malicious) models within each group. The server then aggregates the best models and sends them back to the clients, repeatedly optimizing them until the models are stable. This process minimizes the damage caused by malicious clients and preserves the effective contributions of benign clients, ensuring the overall performance of the model.

[0051] It should be noted that, considering that traditional methods rely on model parameter updates (or gradient directions) for clustering, and addressing the issue of detection failure due to malicious clients deliberately mimicking benign behavior, this embodiment innovatively moves the defense and identification stage forward to the level of essential data features. The core of the method includes: extracting the set of left singular vectors from the client's local data based on truncated singular value decomposition; performing hierarchical clustering based on the principal angle similarity between left singular vectors; and introducing a scaled dot product attention mechanism to achieve dynamic weight allocation and secure aggregation.

[0052] The key finding and advantage of this embodiment is that even if a malicious client is difficult to distinguish from some benign clients in the gradient space, its features in the data subspace can still be effectively identified through the left singular vector. For example... Figure 3 As shown, by extracting left singular vectors from the original client data and the backdoor data and visually comparing them, it was found that at least one pair of left singular vectors in most clients showed significant differences. Figure 3 (a) thus amplifying the essential differences between covert malicious clients and benign clients from a multidimensional perspective; even if the differences between a few client vector pairs are small ( Figure 3 (b) The set of left singular vectors can still serve as a stable and effective distinguishing criterion. This embodiment overcomes the detection challenges of strong heterogeneity in model updates and concealed malicious behavior in Non-IID scenarios by directly analyzing the essential characteristics of data distribution, achieving efficient and accurate defense against backdoor attacks, and maintaining the data privacy of benign clients while ensuring the performance of the main task.

[0053] In a specific example, the client feature extraction and uploading specifically refers to: each client Based on local dataset Perform truncated singular value decomposition (SVD) using SVD: , the dataset Decomposed into a left singular vector matrix Singular value matrix and right singular vector matrix The left singular vector matrix is ​​obtained by retaining the left singular vectors corresponding to the first p largest singular values. and will Uploaded to the central server.

[0054] In the specific implementation, each client (benign / malicious) performs truncated singular value decomposition (SVD) on the local CIFAR-10 dataset: via the formula , local dataset matrix (The dimension can be "number of samples × 3072") Decomposed into a left singular vector matrix Singular value matrix Right singular vector matrix ;

[0055] reserve The left singular vectors corresponding to the first 50 largest singular values ​​are obtained. (Dimensions are 3072×50), only The data is uploaded to the central server in encrypted form, without transmitting the original image data, thus protecting data privacy.

[0056] Key effect: Malicious clients contain backdoor data locally, and their... With benign clients There are essential differences ( Figure 3 Visual verification lays the foundation for subsequent identification.

[0057] In a specific instance, the server clustering grouping specifically refers to: the central server receiving all client { , ,… After calculating the left singular vector of}, compute any two clients and Left singular vector matrix and The cosine similarity is used to construct a similarity matrix S; then the server uses the similarity matrix... Hierarchical clustering is used to cluster clients, grouping clients with high similarity into the same group; the server clustering grouping includes the following steps:

[0058] Step S21: The server calculates for each client pair and The left singular vector matrix between cosine similarity ,in and Represent the left singular vector matrix respectively A left singular vector is used to obtain the similarity matrix. ,in Indicates client and Cosine similarity measure between them.

[0059] Step S22: Using all clients as initial individual clusters, perform bottom-up hierarchical clustering based on the similarity matrix calculated in step S21: continuously merge the two clusters with the highest similarity metrics until the preset minimum inter-cluster similarity threshold β is met or the preset number of clusters M is reached.

[0060] In the specific implementation, the similarity matrix is ​​constructed as follows: the server receives data from 100 clients. Then, through the formula Calculate the cosine similarity of the left singular vectors of any two clients and construct a 100×100 similarity matrix S;

[0061] Calculation logic: The similarity of the "least similar vector pair" among the two sets of left singular vectors is taken as the overall similarity of the client to avoid misjudgment caused by a single vector anomaly;

[0062] Hierarchical clustering is performed as follows: each client is the initial individual cluster, and the two clusters with the highest similarity are merged from bottom to top until the similarity between clusters is <0.6, resulting in 8 clusters (each cluster contains 12-13 clients).

[0063] Clustering results: 20 malicious clients were identified due to... With similar characteristics, they mainly cluster in two clusters. Benign clients are clustered in the remaining six clusters according to data distribution type (such as "animals" and "transportation vehicles"), thus achieving "grouping of similar data clients".

[0064] In a specific instance, the server weight allocation specifically refers to: the server pre-acquiring a clean benchmark dataset. ,right Performing truncated singular value decomposition yields the reference left singular vector matrix. Within each cluster, the server uses a scaled dot product attention mechanism to compute the attention value obtained from step 1 for each client. and The similarity between the servers is used to assign weights to each client based on the calculated results; the server weight assignment includes the following steps:

[0065] Step S31: The server uses truncated singular value decomposition. Obtain its left singular vector matrix ;

[0066] Step S32: The client set in the k-th cluster is The server calculates the i-th client of scaled dot product attention score between the reference dataset scaled dot product attention score between the reference dataset wherein denotes the dimension of , used to scale the dot product result to ensure numerical stability;

[0067] Step S33: Calculate the client weight at aggregation wherein is the total number of clients in the current cluster, is an exponential function, i.e. .

[0068] In a specific implementation, the reference vector extraction: the server performs truncated SVD on the clean reference dataset to obtain the reference left singular vector matrix with a dimension of 3072x50;

[0069] Attention score calculation: for the kth cluster (e.g., containing 13 clients), the scaled dot product attention score of each client with is calculated by the formula wherein is the dimension of .

[0070] Score difference: benign client scores are concentrated in 0.7-0.9 (high similarity with clean reference), and malicious client scores are concentrated in 0.2-0.4 (large difference from clean reference);

[0071] Aggregation weight calculation: the weight is calculated by the formula , the weight of benign clients is 0.08-0.12, and the weight of malicious clients is reduced to 0.01-0.03, achieving dynamic allocation of "high weight for benign clients and low weight for malicious clients".

[0072] In a specific example, the model distribution and local training refer to: the server initializes a global model , randomly selects part of the clients at the tth round of communication, and distributes the global model to the selected clients; the selected clients train the model based on the local dataset using the stochastic gradient descent (SGD) algorithm to obtain a local model , and upload to the server.

[0073] ​​In a specific implementation, the server initializes ResNet-18 as a global model In the tth communication round, 40 clients are randomly selected (5 from each cluster), and are sent to the selected clients.

[0074] The selected clients train local models based on local data sets (clean data for benign clients and backdoor data for malicious clients) using the SGD algorithm. After 8 epochs of training, the are obtained, and the are encrypted and uploaded to the server.

[0075] Training guarantee: Clients only train with local data and do not interact with other clients to avoid data leakage.

[0076] In a specific example, the model aggregation and iterative optimization refer to: the server, in each cluster, based on the aggregation weights obtained in step S33 weights the local models of the clients in the cluster to obtain a new round of global model in the cluster where represents the set of clients in the kth cluster; and is sent to the clients in the corresponding cluster, and steps S1-S5 are repeated until the model converges or the preset training round is reached.

[0077] In a specific implementation, the weighted aggregation in the cluster: the server, in each cluster, aggregates the local models according to the weights obtained in step S3 through the formula is the set of clients in the kth cluster), to generate a new global model in the cluster.

[0078] Aggregation effect: the malicious client's backdoor update has a significant impact on the global model due to the extremely low weight (0.01-0.03).

[0079] Model distribution and iteration: the server sends to the clients in the corresponding cluster, updates the training round t = t + 1, and repeats steps S1-S5 for 100 training rounds.

[0080] Implementation effect verification:

[0081]

[0082] ​In summary, the defense process of the present application starts from local data processing and feature extraction of the client. Each client (including benign clients and malicious clients) first performs truncated singular value decomposition on its data set locally, extracts the left singular vectors corresponding to the first p largest singular values, and uploads them to the server as data distribution features. After collecting the left singular vectors of all clients, the server performs hierarchical clustering based on cosine similarity, and divides the clients into different cluster groups according to the similarity of their data distribution. Then, within each cluster group, the server uses the benchmark left singular vectors extracted from a clean benchmark data set to calculate the similarity score of each client to the benchmark through the scaled dot-product attention mechanism, and dynamically assigns the corresponding aggregation weight. Finally, the server aggregates the client models in each cluster by weighting according to the obtained weight, generates an updated global model, thereby effectively suppressing the influence of malicious clients while maintaining high performance of the global model on the main task.

[0083] Compared with the prior art, the present embodiment has the following advantages:

[0084] (1) By projecting the client data into a multi-dimensional feature space spanned by the left singular vectors, the data distribution is viewed from multiple principal component perspectives, thereby significantly enhancing the contrast between hidden malicious clients and benign clients, and achieving accurate identification.

[0085] (2) The introduction of hierarchical clustering and scaled dot-product attention mechanism provides double protection, which not only realizes the reasonable grouping of similar data distribution clients, but also dynamically adjusts the aggregation weight without pre-setting a fixed proportion of malicious clients.

[0086] (3) While effectively suppressing backdoor attacks, the attention-weighted aggregation mechanism maximizes the contribution of benign client models, significantly reducing the performance degradation caused by simply removing suspected malicious clients in traditional methods.

[0087] (4) Even in complex non-independent and identically distributed data environments and scenarios where backdoor attacks coexist, the present application can still maintain excellent model performance.

[0088] Embodiment 2

[0089] In order to achieve the above-mentioned purpose, see Figure 4 The present embodiment also provides a Non-IID federated learning backdoor attack defense system, comprising:

[0090] Client feature extraction and upload module: perform truncated singular value decomposition on the local data set by the client, extract the first p left singular vectors and upload to the server;

[0091] Server clustering grouping module: construct similarity matrix by cosine similarity of left singular vectors between clients, and group by hierarchical clustering;

[0092] Server weight allocation module: the server uses the clean benchmark dataset to do truncated singular value decomposition to get the benchmark vector, and uses the scaled dot product attention mechanism to calculate the similarity score of the client and the benchmark to allocate the aggregation weight;

[0093] Model distribution and local training module: initialize the global model through the server, and randomly select the client for distribution; the selected client trains with local data and uploads local model parameters;

[0094] Model aggregation and iterative optimization module: the server aggregates the model parameters within the cluster according to the weight to generate a new global model, and the process is repeated until convergence.

[0095] The Non-IID federated learning backdoor attack defense system of the embodiment has the same advantages as the Non-IID federated learning backdoor attack defense method described above, and will not be described here.

[0096] Embodiment 3

[0097] In order to achieve the above purpose, the embodiment also provides a computer readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor executes the Non-IID federated learning backdoor attack defense method as described above.

[0098] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for defending Non-IID federated learning backdoor attack, characterized in that, The method comprises the following steps: S1, client feature extraction and uploading: the client performs truncated singular value decomposition on the local data set, extracts the first p left singular vectors, and uploads them to the server; S2, server clustering grouping: the server calculates the cosine similarity of the left singular vectors between clients, constructs a similarity matrix, and groups using hierarchical clustering; S3, server weight allocation: the server performs truncated singular value decomposition on the clean benchmark data set to obtain benchmark vectors, calculates the similarity score of the client and the benchmark using the scaled dot product attention mechanism, and allocates aggregation weights; S4, model distribution and local training: the server initializes the global model, randomly selects a client for distribution; the selected client trains with local data and uploads local model parameters; S5, model aggregation and iterative optimization: the server aggregates model parameters within the cluster according to the weight to generate a new global model, and repeats the process until convergence.

2. The Non-IID federated learning backdoor attack defense method of claim 1, wherein, The client feature extraction and uploading specifically refers to: each client based on the local dataset Perform truncated singular value decomposition, through SVD: , the data set Decomposed into left singular vector matrix , singular value matrix And right singular vector matrix , the left singular vector corresponding to the first p largest singular value is obtained Left singular vector matrix , and Upload to the central server.

3. The Non-IID federated learning backdoor attack defense method of claim 2, wherein, The server clustering grouping specifically refers to: the central server receiving all client { , ,… After calculating the left singular vector of}, compute any two clients and Left singular vector matrix and The cosine similarity is used to construct a similarity matrix S; Then the server clusters the clients according to the similarity matrix The clients are clustered using a hierarchical clustering method, and the clients satisfying a preset similarity threshold are classified into the same group.

4. The Non-IID federated learning backdoor attack defense method of claim 3, wherein, The server clustering grouping comprises the following steps: Step S21: The server calculates the cosine similarity between each pair of clients and between the left singular vector matrices and where and represent a left singular vector in the left singular vector matrices , respectively, to obtain a similarity matrix where represents the cosine similarity measure between clients and ;​ Step S22: taking all clients as initial individual clusters, performing bottom-up hierarchical clustering based on the similarity matrix calculated in step S21: constantly merging the two clusters with the highest similarity measure until the preset minimum similarity threshold β between clusters is met or the preset number of clusters M is reached.

5. The Non-IID federated learning backdoor attack defense method according to claim 2, characterized in that, The server weight distribution specifically refers to: the server pre-acquires a clean benchmark data set , and performs truncated singular value decomposition to obtain a benchmark left singular vector matrix ; in each cluster, the server calculates the similarity between the feature vector of each client obtained in step S1 and the feature vector of the cluster using the scaled dot product attention mechanism, and assigns a weight to each client using the calculated result. , and ​ 6. The Non-IID federated learning backdoor attack defense method according to claim 5, characterized in that, The server weight allocation comprises the following steps: Step S31: The server uses truncated singular value decomposition obtains its left singular vector matrix ; Step S32: The client set in the kth cluster is , the server calculates the scaled dot product attention score between the i-th client and the reference dataset , where represents the dimension of , which is used to scale the dot product result to ensure the stability of the numerical value;​​​ Step S33: Compute the client Weight at aggregation where is the total number of clients in the current cluster, is the exponential function, i.e. .

7. The Non-IID federated learning backdoor attack defense method according to claim 6, characterized in that, The model distribution and local training refer to that the server initializes a global model , randomly selects part of the clients in the tth round of communication, and sends the global model to the selected clients; the selected clients train the model based on the local data set by using a stochastic gradient descent algorithm to obtain a local model , and upload the local model to the server. ​​ 8. The Non-IID federated learning backdoor attack defense method according to claim 7, characterized in that, The model aggregation and iterative optimization refers to: the server, in each cluster, performs weighted aggregation on the local models of the clients in the cluster based on the aggregation weights obtained in step S33 to obtain a new round of global model in the cluster The local model of the client in the cluster is updated as follows: wherein represents the set of clients in the kth cluster; and is sent to the clients in the corresponding cluster, and steps S1-S5 are repeated until the model converges or a preset training round is reached.

9. A Non-IID federated learning backdoor attack defense system, characterized in that, It comprises: A client feature extraction and uploading module: perform truncated singular value decomposition on the local data set through the client, extract the first p left singular vectors, and upload them to the server; A server clustering grouping module: calculate the cosine similarity of the left singular vectors between clients through the server, construct a similarity matrix, and group using hierarchical clustering; A server weight allocation module: the server performs truncated singular value decomposition on the clean benchmark data set to obtain benchmark vectors, calculates the similarity score of the client and the benchmark using the scaled dot product attention mechanism, and allocates aggregation weights; A model distribution and local training module: the server initializes the global model, randomly selects a client for distribution; the selected client trains with local data and uploads local model parameters; A model aggregation and iterative optimization module: the server aggregates model parameters within the cluster according to the weight to generate a new global model, and repeats the process until convergence.

10. A computer readable storage medium storing a computer program, characterized in that, The computer program, when executed by a processor, causes the processor to perform the Non-IID federated learning backdoor attack defense method according to any one of claims 1 to 8.

Images