Federal learning based byzantine attack defense method, device, equipment and medium

Abstract

The application relates to the technical field of security protection, can be applied to a business system platform such as financial technology and medical health, and discloses a Byzantine attack defense method, device and equipment based on federated learning and a medium. The method comprises the following steps: acquiring local models and sample data sets of a plurality of target clients, performing forward propagation on the sample data sets by using the local models, constructing a plurality of probability matrices, determining the Euclidean distance between any two probability matrices, constructing a distance matrix, performing hierarchical clustering on the target clients, obtaining a plurality of client clusters, acquiring a probability standard matrix of a standard model, determining the similarity between the probability matrices and the probability standard matrix one by one, marking the types of the to-be-marked matrices according to the similarity, updating the information of the client clusters, obtaining updated clusters, aggregating all the updated clusters, and obtaining a final defense model. The application can effectively improve the defense efficiency and defense scene coverage rate for the Byzantine attack.

Description

Technical Field

[0001] This invention relates to the field of security protection technology, and in particular to a method, apparatus, device and medium for defending against Byzantine attacks based on federated learning. Background Technology

[0002] In the field of federated learning, existing technologies offer several defenses against Byzantine attacks. Some Byzantine Robust Aggregation Rules (BRARs) defend against attacks by directly analyzing local model updates. Others rely on gradient statistics, gradient distance, additional validation data, optimization algorithm compensation, and differential privacy. In privacy-preserving federated learning, defenses are based on secure multi-party computation, trusted execution environments, and measurable additive masks. However, these existing technologies have numerous shortcomings and problems. Many traditional defense methods, due to their direct analysis of local model updates, incur enormous computational overhead when facing modern models with large parameter scales, such as Large Language Models (LLMs), significantly reducing their effectiveness. Moreover, existing defense mechanisms often only defend against certain types of Byzantine attacks and struggle to cope with diverse attack methods. Their robustness is clearly insufficient in complex scenarios with a high proportion of Byzantine clients or heterogeneous data.

[0003] In the healthcare field, federated learning requires collaborative modeling across multiple medical institutions while protecting patient privacy. However, medical data is typically characterized by high dimensionality, sparsity, small sample sizes, and strong heterogeneity, placing higher demands on Byzantine robustness. Existing defense mechanisms struggle to maintain model convergence stability when faced with highly heterogeneous distributions, and combining them with privacy protection measures such as differential privacy or secure multi-party computation significantly increases system communication and computational overhead, limiting practical deployment efficiency. Furthermore, the data sensitivity in healthcare scenarios prevents the use of external validation sets, making defense methods based on additional validation data difficult to apply.

[0004] In the fintech sector, federated learning is frequently used for high-value tasks such as cross-institutional credit assessment and fraud detection, where data privacy and model accuracy are equally critical. However, financial data is extremely unevenly distributed and significantly affected by the heterogeneity of business systems, making models susceptible to interference from non-independent, identically distributed data. Meanwhile, Byzantine attackers can launch more covert and targeted attacks through forged transactions or model drift strategies, which traditional defense mechanisms struggle to accurately identify and isolate. Furthermore, to meet compliance and security requirements, financial scenarios often require deployment in constrained environments, posing greater challenges to the compatibility of computational resources and model complexity, further weakening the applicability and practicality of existing defense strategies.

[0005] In summary, direct analysis of local model updates in high-dimensional models such as large language models incurs huge computational overhead, which seriously affects defense efficiency; most mechanisms can only deal with specific types of attacks and are difficult to cover diverse and complex attack scenarios; in environments with a high proportion of Byzantine clients or highly heterogeneous data distribution, the model is prone to instability and lacks robustness, making it difficult to meet the dual requirements of security and generalization ability in actual deployment.

[0006] Therefore, current technologies suffer from low defense efficiency against Byzantine attacks and low coverage of defense scenarios. Summary of the Invention

[0007] This invention provides a method, apparatus, device, and medium for defending against Byzantine attacks based on federated learning. Its main purpose is to solve the problems of low defense efficiency and low coverage of defense scenarios against Byzantine attacks.

[0008] Firstly, to achieve the above objectives, this invention provides a Byzantine attack defense method based on federated learning, comprising:

[0009] Obtain local models and sample datasets for several target clients, perform forward propagation on the sample datasets using the local models to obtain the output probability vectors for each target client, and construct several probability matrices based on the output probability vectors and the sample datasets;

[0010] Determine the Euclidean distance between any two of the probability matrices, and construct a distance matrix based on the Euclidean distance;

[0011] Based on the distance matrix, hierarchical clustering is performed on the target clients to obtain several client clusters;

[0012] Obtain the standard model and the probability standard matrix of each target client, determine the similarity between the probability matrix and the probability standard matrix one by one, and generate a matrix to be labeled based on the similarity.

[0013] The matrix to be labeled is then labeled with a type, and the labeled matrix is ​​used to update the information of the client cluster to obtain the updated cluster;

[0014] Aggregate all the updated clusters to obtain the final defense model.

[0015] Secondly, the present invention also provides a Byzantine attack defense device based on federated learning, comprising:

[0016] The probability matrix construction module is used to obtain local models and sample datasets of several target clients, perform forward propagation on the sample datasets using the local models to obtain the output probability vectors of each target client, and construct several probability matrices based on the output probability vectors and the sample datasets.

[0017] A distance matrix construction module is used to determine the Euclidean distance between any two probability matrices and to construct a distance matrix based on the Euclidean distance.

[0018] The client clustering module is used to perform hierarchical clustering of the target clients based on the distance matrix to obtain several client clusters;

[0019] The similarity determination module is used to obtain the standard model of each target client and the probability standard matrix of the standard model, determine the similarity between the probability matrix and the probability standard matrix one by one, and generate a matrix to be labeled based on the similarity.

[0020] The information update module is used to mark the type of the matrix to be marked, and to update the information of the client cluster using the marked matrix to obtain the updated cluster;

[0021] The update cluster aggregation module is used to aggregate all the updated clusters to obtain the final defense model.

[0022] Thirdly, the present invention also provides an electronic device, the electronic device comprising:

[0023] At least one processor; and,

[0024] A memory communicatively connected to the at least one processor; wherein,

[0025] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the aforementioned federated learning-based Byzantine attack defense method.

[0026] Fourthly, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the above-described federated learning-based Byzantine attack defense method.

[0027] This invention acquires local models and sample datasets of several target clients, performs forward propagation on the sample datasets using the local models to obtain the output probability vectors of each target client, and constructs several probability matrices based on the output probability vectors and the sample datasets. This allows for a comprehensive characterization of the response behavior of each client model to the same input without revealing model parameters and original data. It determines the Euclidean distance between any two probability matrices and constructs a distance matrix based on the Euclidean distance, which can intuitively quantify the output differences of different client models on a unified sample dataset, revealing the similarity or difference in their prediction behavior. Based on the distance matrix, hierarchical clustering of the target clients is performed to obtain several client clusters. The squared error within each cluster is used as a metric for the merging cost, ensuring that each merging minimizes the increase in intra-cluster differences, thereby maintaining the compactness and consistency of the clusters. By iteratively comparing merging costs and selecting the optimal merging pair, the degradation of clustering quality caused by arbitrary merging is avoided, achieving high-quality client grouping. The standard model and probability standard matrix of each target client are obtained. The similarity between each probability matrix and the probability standard matrix is ​​determined, and a labeling matrix is ​​generated based on the similarity. The labeling matrix is ​​then type-labeled, and the client clusters are updated using the labeled matrix to obtain updated clusters. This approach balances security and flexibility, enhances the system's ability to identify abnormal clients, and effectively improves the accuracy of client clustering and the robustness of the aggregation strategy during federated learning. All updated clusters are aggregated to obtain the final defense model. By dynamically detecting and controlling the difference between client updates and the global model, abnormal updates are intelligently scaled to prevent abnormal or malicious clients from negatively impacting model training, effectively improving the robustness and security of the aggregation process. Combined with a weighted aggregation mechanism, this ensures that updates from trusted clients play a full role while suppressing potential attacks and abnormal behavior, effectively improving the defense efficiency against Byzantine attacks and the coverage of defense scenarios. Attached Figure Description

[0028] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a schematic diagram of an application environment for a Byzantine attack defense method based on federated learning, according to an embodiment of the present invention.

[0030] Figure 2A flowchart illustrating a Byzantine attack defense method based on federated learning, provided as an embodiment of the present invention;

[0031] Figure 3 A flowchart illustrating the probability matrix construction process in a federated learning-based Byzantine attack defense method according to an embodiment of the present invention;

[0032] Figure 4 A schematic diagram of a module for a Byzantine attack defense device based on federated learning, provided as an embodiment of the present invention;

[0033] Figure 5 A schematic diagram of the structure of an electronic device that implements a Byzantine attack defense method based on federated learning, according to an embodiment of the present invention;

[0034] Figure 6 This is another schematic diagram of an electronic device that implements a Byzantine attack defense method based on federated learning, as provided in an embodiment of the present invention.

[0035] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0036] To enable those skilled in the art to better understand the technical solutions of this disclosure, and to fully understand and implement the process of how this disclosure applies technical means to solve technical problems and achieve corresponding technical effects, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, not all embodiments. The embodiments of this disclosure and the various features within them can be combined with each other without conflict, and the resulting technical solutions are all within the protection scope of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort should fall within the protection scope of this disclosure.

[0037] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0038] This application provides a Byzantine attack defense method based on federated learning. The execution entity of this method includes, but is not limited to, at least one electronic device that can be configured to execute the device provided in this application, such as a server or a terminal. In other words, the Byzantine attack defense method based on federated learning can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0039] This invention provides a Byzantine attack defense method based on federated learning, which can be applied to applications such as... Figure 1In this application environment, the client communicates with the server via a network. The server can obtain local models and sample datasets of several target clients from the client, perform forward propagation on the sample dataset using the local models to obtain the output probability vector of each target client, and construct several probability matrices based on the output probability vectors and the sample dataset. This allows for a comprehensive characterization of the response behavior of each client model to the same input without revealing model parameters and original data. The server can determine the Euclidean distance between any two probability matrices and construct a distance matrix based on the Euclidean distance. This allows for a direct quantification of the output differences of different client models on a unified sample dataset, revealing the similarity or difference in their predictive behavior. Hierarchical clustering of the target clients is performed based on the distance matrix to obtain several client clusters. The intra-cluster squared error is used as a metric for merging costs to ensure that each merge minimizes the increase in intra-cluster differences, thereby maintaining the compactness and consistency of the clusters. By iteratively comparing merging costs and selecting the optimal merging pair, the degradation of clustering quality caused by arbitrary merging is avoided, achieving high-quality client grouping. The standard model and probability standard matrix of each target client are obtained. The similarity between each probability matrix and the probability standard matrix is ​​determined, and a labeling matrix is ​​generated based on the similarity. The labeling matrix is ​​then type-labeled, and the client clusters are updated using the labeled matrix to obtain updated clusters. This approach balances security and flexibility, enhances the system's ability to identify abnormal clients, and effectively improves the accuracy of client clustering and the robustness of the aggregation strategy during federated learning. All updated clusters are aggregated to obtain the final defense model. By dynamically detecting and controlling the difference between client updates and the global model, abnormal updates are intelligently scaled to prevent abnormal or malicious clients from negatively impacting model training, effectively improving the robustness and security of the aggregation process. Combined with a weighted aggregation mechanism, this ensures that updates from trusted clients play a full role while suppressing potential attacks and abnormal behavior, effectively improving the defense efficiency against Byzantine attacks and the coverage of defense scenarios. Finally, the final defense model output is fed back to the user client. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will now be described in detail through specific embodiments.

[0040] The following explanation of this invention relates to the present invention. This invention applies the final updated value obtained from weighted aggregation to the current global model, performing an iterative optimization to update and adjust the model parameters. The final updated value, as the effective gradient or parameter difference contributed collectively by trusted clients in this round, is superimposed on the current global model to complete a new round of model updates. This not only integrates valuable information from highly trusted clients but also effectively suppresses interference from abnormal or malicious updates, thereby improving the stability and robustness of the global model. This can effectively improve the defense efficiency against Byzantine attacks and the coverage of defense scenarios.

[0041] Reference Figure 2 The diagram shown is a flowchart illustrating a Byzantine attack defense method based on federated learning according to an embodiment of the present invention. In this embodiment, the Byzantine attack defense method based on federated learning includes:

[0042] S1. Obtain local models and sample datasets for several target clients, perform forward propagation on the sample datasets using the local models to obtain the output probability vectors for each target client, and construct several probability matrices based on the output probability vectors and the sample datasets.

[0043] In this embodiment of the invention, by applying the local model of each target client to a unified sample dataset and performing forward propagation sequentially, the original output vector corresponding to each sample is obtained. These original output vectors are normalized and transformed into output probability vectors representing the category distribution, thereby constructing a probability matrix corresponding to each client, thus characterizing its prediction behavior features on the same dataset.

[0044] In specific healthcare scenarios, this can be applied to the construction of personalized disease diagnosis models within a federated learning framework. Each hospital, while protecting patient privacy, uses its local model to perform forward inference on standardized clinical samples (such as images or medical records) to generate predicted probability vectors for patient samples. A probability matrix is ​​then constructed to reflect the response patterns of different institutions' models to the same case. The server can use these probability matrices to evaluate the consistency of model behavior and identify nodes of abnormal behavior (such as training data bias or abnormal model configuration), thereby improving the robustness of model aggregation and the accuracy of clinical diagnosis.

[0045] In specific fintech scenarios, this can be used to build cross-institutional risk assessment models. For example, each financial institution's local model predicts a standardized sample of credit applications (including user profiles, transaction behavior, etc.), outputting a probability vector of default or fraud, and constructing a probability matrix to reflect the behavioral patterns of different institutional models in risk judgment. The platform can use these matrices to identify participants with potentially anomalous strategies (such as overly aggressive or conservative risk control models), thereby dynamically adjusting them when aggregating models to improve the robustness and risk control capabilities of the overall credit model.

[0046] Figure 3 This is a flowchart illustrating the probability matrix construction process in a federated learning-based Byzantine attack defense method according to an embodiment of the present invention.

[0047] In this embodiment of the invention, the step of using the local model to perform forward propagation on the sample dataset to obtain the output probability vector of each target client, and constructing several probability matrices based on the output probability vectors and the sample dataset, includes:

[0048] The local model is used to perform forward propagation on each sample data in the sample dataset to obtain the original output vector of each sample data.

[0049] The original output vector is normalized to obtain the output probability vector;

[0050] The output probability vectors are stacked according to the order of the sample data in the sample dataset to obtain the probability matrix of each sample data in the sample dataset.

[0051] In detail, each target client uses its locally trained model to perform forward propagation operation on each sample in the pre-distributed unified sample dataset. That is, each input sample first passes through the input layer of the model, and then passes through multiple hidden layers (such as convolutional layers, fully connected layers, activation functions, normalization layers, etc.) in sequence, and finally reaches the output layer to generate an original output vector (usually logits), which represents the model's unnormalized score or activation value for each category, reflecting the local model's initial judgment on each sample.

[0052] The unnormalized scores (logits) output by the model are converted into an interpretable probability distribution. The softmax function is applied to the original output vector to exponentialize each score and normalize it by summing the results, so that all values ​​become positive and the sum is 1. This yields an output probability vector representing the predicted probability of each category, reflecting the model's relative confidence in different categories, which is convenient for subsequent probability analysis or comparison of model behavior.

[0053] After completing the forward propagation and probability normalization for each sample, the output probability vectors corresponding to each sample are combined row by row according to the sample arrangement order in the original dataset, and finally a two-dimensional matrix is ​​formed. Each row of the matrix corresponds to a sample, and each column corresponds to a class. The matrix as a whole reflects the predicted probability distribution of the client model for the entire sample dataset, which is the output probability matrix of the client.

[0054] By performing forward propagation on the local models of each client using a unified sample dataset and normalizing the outputs into probability vectors to construct a probability matrix, it is possible to comprehensively characterize the response behavior of each client model to the same input without revealing model parameters and original data. The probability matrix can be used to evaluate the prediction consistency among clients, identify potentially abnormal or malicious models, and thus achieve a more robust model aggregation and defense mechanism, improving the security and overall performance of the federated learning system.

[0055] S2. Determine the Euclidean distance between any two of the probability matrices, and construct a distance matrix based on the Euclidean distance.

[0056] In this embodiment of the invention, two probability matrices are randomly selected from all clients as target matrices. The difference between corresponding elements of the target matrices is calculated to obtain a difference matrix. The sum of the squares of all elements in the difference matrix is ​​then taken as the square root to obtain the Euclidean distance between the two probability matrices. By iterating through all client matrix pairs and repeating the above operations, a symmetric distance matrix is ​​finally constructed, where each element represents the degree of difference in the outputs of the two client models on a unified sample dataset.

[0057] In specific healthcare scenarios, hospitals, without sharing patient privacy data, utilize their respective local diagnostic models to predict standardized pathological images or structured case data, constructing output probability matrices to reflect the model's judgment tendency across different disease categories. By calculating the Euclidean distance between the output probability matrices of each hospital's model, the central server can identify institutions with abnormal predictive behavior (such as biased model training data or potential attacks), excluding abnormal nodes during model aggregation, thereby improving the security and reliability of the multi-institution joint diagnostic system.

[0058] In specific fintech scenarios, multiple financial institutions utilize local credit risk control models to score standardized application samples, generating their own output probability matrices representing the distribution of risk levels such as default and fraud. The platform measures the similarity of model prediction behavior by calculating the Euclidean distance between the probability matrices of each institution, identifying nodes of abnormal strategy or model manipulation risk (such as over-lending or malicious false alarms). This helps optimize model fusion strategies in joint modeling, prevent single points of risk, and enhance the stability and reliability of cross-institutional risk control systems.

[0059] In this embodiment of the invention, determining the Euclidean distance between any two probability matrices and constructing a distance matrix based on the Euclidean distance includes:

[0060] Two probability matrices are randomly selected as the two target matrices;

[0061] The difference between the two target matrices is obtained, and the sum of the squares of the difference is obtained.

[0062] The summation result is squared to obtain the Euclidean distance between the two target matrices;

[0063] All the Euclidean distances are summarized into a distance matrix.

[0064] In detail, two target matrices are randomly selected from the output probability matrices of all clients. The difference between them is obtained by subtracting each element, representing the prediction difference between the two clients on the corresponding samples. The sum of the squares of all elements in the difference is then taken as the square root to calculate the Euclidean distance between the two target matrices, reflecting the similarity of their overall output behavior. The formula for calculating the Euclidean distance is as follows:

[0065]

[0066] in, Indicates the first The output probability vector of the first target matrix Indicates the first The output probability vector of the second objective matrix Represents the vector dimension. This represents the Euclidean distance between the first target matrix and the second target matrix.

[0067] By repeating this calculation process and traversing all client matrix pairs, a symmetric distance matrix is ​​finally generated, where each element represents the predicted distance between any two clients, providing a quantitative basis for subsequent clustering analysis and anomaly identification.

[0068] By calculating the Euclidean distance between any two probability matrices and constructing a distance matrix, the output differences of different client models on a unified sample dataset can be intuitively quantified, revealing the similarities or differences in their predictive behaviors. This method not only provides an objective numerical basis for subsequent clustering analysis and anomaly detection, but also helps to effectively identify abnormal or malicious clients, improve the robustness and security of models in the federated learning process, and ensure the reliability and accuracy of global model updates.

[0069] S3. Perform hierarchical clustering on the target client based on the distance matrix to obtain several client clusters.

[0070] In this embodiment of the invention, the distance matrix corresponding to each target client is initialized as an independent cluster. Then, two clusters are randomly selected repeatedly as merging objects, and their intra-cluster squared errors before and after merging are calculated. A cost function is constructed based on the error change to evaluate the merging cost increment. By comparing the merging costs of all possible cluster pairs, the cluster pair with the lowest cost is selected for merging, gradually forming new merged clusters. The merging process iterates continuously, repeating the random selection and merging operations until a preset number of clusters is reached. Finally, these merged clusters are used as client clusters, realizing hierarchical clustering based on the distance matrix.

[0071] In specific healthcare scenarios, hierarchical clustering based on Euclidean distance can be used to analyze the similarity of patient diagnostic model outputs from multiple medical institutions. By constructing probability matrix distances between the models of each institution on a unified case sample, clustering similarity models helps to identify nodes exhibiting abnormal behavior or potential attacks, ensuring the secure aggregation of models and the accuracy of clinical diagnosis in federated learning.

[0072] In specific fintech scenarios, this can be applied to the behavioral pattern analysis of cross-institutional risk assessment models. Financial institutions can use distance matrices to perform hierarchical clustering on the model outputs of a unified credit sample to identify model groups with abnormal risk judgments. This helps the platform effectively filter out abnormal nodes and improve the robustness and fraud prevention capabilities of the joint risk control model.

[0073] In this embodiment of the invention, the step of performing hierarchical clustering of the target client based on the distance matrix to obtain several client clusters includes:

[0074] Use the distance matrix of each target client as the initial cluster;

[0075] Two initial clusters are randomly selected as the first target cluster and the second target cluster, and the first target cluster and the second target cluster are taken as a target cluster pair;

[0076] The target cluster pairs are merged to obtain a merged cluster;

[0077] Determine the intra-cluster squared errors of the first target cluster, the second target cluster, and the merged cluster, respectively;

[0078] Construct a cost function based on the intra-cluster squared error;

[0079] The cost function is used to determine the merging cost increment of the target cluster pair;

[0080] The merging cost increments of each target cluster pair are compared, and the target cluster pair corresponding to the smallest merging cost increment is selected based on the comparison results.

[0081] The selected target cluster pairs are merged to obtain the merged updated cluster;

[0082] The process of randomly selecting two initial clusters as the first target cluster and the second target cluster, and using the first target cluster and the second target cluster as a target cluster pair, continues until a preset number of merged update clusters are found, at which point the process stops.

[0083] Obtain the target clients corresponding to the distance matrix in the final merged update cluster, and summarize the final target clients to obtain the client cluster.

[0084] In detail, the distance matrix corresponding to each target client is initialized as an independent initial cluster. Two initial clusters are randomly selected as the first target cluster and the second target cluster to form a cluster pair to be merged. This pair of target clusters is merged to form a new merged cluster, thereby gradually realizing the aggregation of client clusters and the construction of hierarchical structure.

[0085] First, the center points of the first and second target clusters are determined. Then, the sum of the squared Euclidean distances between all samples in each cluster and the center point is calculated to obtain the intra-cluster squared error. The two clusters are merged to form a new cluster. The center point of the merged cluster is calculated, and the sum of the squared distances between all merged samples and the new center point is obtained to obtain the intra-cluster squared error of the merged cluster. By comparing the difference between the squared error of the merged cluster and the sum of the squared errors of the two original clusters, a cost function is constructed. The cost function reflects the impact of the merging operation on the intra-cluster compactness. The smaller the cost, the less the intra-cluster compactness decreases after merging, thus guiding the selection of the optimal cluster merging scheme. The formula for calculating the cost function is shown below:

[0086]

[0087]

[0088]

[0089] in, Indicates the first target cluster, Indicates the second target cluster, Indicates a merged cluster. Represents the cost function, This represents the within-cluster squared error of the merged cluster. This represents the within-cluster squared error of the first target cluster. This represents the within-cluster squared error of the second target cluster. Indicates the target cluster The intra-cluster squared error, Indicates the target cluster The distance matrix, Indicates the target cluster The mean of the mid-distance matrix.

[0090] Calculate the change in intra-cluster squared error before and after merging each cluster pair. This is calculated by subtracting the sum of the squared errors of the two clusters before merging from the intra-cluster squared error after merging. The difference represents the merging cost increment for that target cluster pair. Compare the merging cost increments of all target cluster pairs one by one. By sorting or traversing, find the pair with the smallest merging cost increment. This indicates the smallest decrease in intra-cluster compactness after merging, and is therefore selected as the optimal merging pair in this round of clustering. This ensures that the clustering process merges clusters step-by-step with minimal cost, improving the overall clustering quality.

[0091] The selected target cluster pairs are merged to form a new merged update cluster. The system randomly selects two merged update clusters as the new first target cluster and the second target cluster to form a new target cluster pair. The merging cost is calculated and the optimal merging pair is selected. This process is repeated until the number of merged update clusters reaches the preset target number. At this point, the merging stops, and the clients corresponding to these merged update clusters that have reached the preset number are taken as the final cluster partitioning result, thus completing the hierarchical clustering process.

[0092] Hierarchical clustering based on distance matrices effectively aggregates target clients with similar behaviors into several clusters. Using the squared error within each cluster as a metric for merging cost ensures that each merge minimizes the increase in intra-cluster differences, thus maintaining cluster compactness and consistency. By iteratively comparing merging costs and selecting the optimal merge pair, the degradation in clustering quality caused by arbitrary merging is avoided, achieving high-quality client grouping. This data-driven adaptive clustering approach not only accurately identifies behavioral differences between clients but also provides a solid foundation for subsequent aggregation strategies such as anomaly detection, Byzantine attack defense, and federated learning, enhancing the overall system's robustness and security.

[0093] S4. Obtain the standard model of each target client and the probability standard matrix of the standard model, determine the similarity between the probability matrix and the probability standard matrix one by one, and generate a matrix to be labeled based on the similarity.

[0094] In this embodiment of the invention, the similarity between the probability matrix of each target client and the standard matrix is ​​calculated, such as using cosine similarity, the inverse of Euclidean distance, or other similarity measures. The calculation results are compared with a set similarity threshold to perform a preliminary classification of the client model behavior, laying the foundation for subsequent anomaly detection, labeling, or removal operations.

[0095] In specific healthcare scenarios, this can be used to assess whether personalized diagnostic models uploaded by different hospitals or devices are consistent with authoritative medical models. For example, under a federated learning framework, multiple hospitals train models based on local patient data and return output probability matrices. The system compares these matrices with the probability standard matrix generated by a national clinical diagnostic standard model. When a hospital's model differs significantly from the standard, it is identified as a matrix to be labeled that meets a preset second condition, indicating that the model may have data bias, training anomalies, or system configuration problems. This guides further manual review or adjustment, ensuring the clinical consistency and safety of the diagnostic model nationwide.

[0096] In specific fintech scenarios, this can be used for compliance assessment and fraud detection of risk control models. Multiple financial institutions or branches train credit scoring models based on local credit user behavior data. By calculating the similarity between the output probability matrix of these models and the output of the standard credit scoring model set by regulatory agencies or headquarters, models that deviate excessively from the standard are selected as matrices to be labeled based on a preset second condition for focused review. This mechanism helps to promptly identify risk scoring deviations caused by model overfitting, malicious tampering, or data quality issues, thereby improving the transparency and stability of risk control throughout the financial system.

[0097] In this embodiment of the invention, determining the similarity between each probability matrix and the standard probability matrix, and generating a matrix to be labeled based on the similarity, includes:

[0098] The similarity between the probability matrix and the standard probability matrix of each target client is determined one by one;

[0099] Determine whether the similarity is greater than a preset similarity threshold;

[0100] If the similarity is greater than the similarity threshold, then the probability matrix corresponding to the similarity greater than the similarity threshold is taken as the matrix to be labeled that meets the preset first condition;

[0101] If the similarity is less than or equal to the similarity threshold, then the probability matrix corresponding to the similarity that is less than or equal to the similarity threshold is taken as the matrix to be labeled that meets the preset second condition.

[0102] In detail, the output probability matrix of each client on a unified auxiliary sample set is extracted, and the output probability matrix is ​​matched with the standard probability matrix generated by the standard model on the same sample set. By calculating the similarity index between the two matrices (such as cosine similarity, Euclidean distance, or KL divergence), the closeness between the output results of the client model and the standard model is quantified, thereby providing numerical basis for subsequent model selection or anomaly identification.

[0103] After calculating the similarity between the probability matrix of each target client and the standard probability matrix, the system compares the similarity results with a preset similarity threshold to determine whether the client model's behavior conforms to the standard. If the similarity of a client is higher than the threshold, it indicates that the output result is relatively consistent with the standard model and has high credibility; the system then classifies the corresponding probability matrix as a matrix to be labeled that meets the first preset condition. If the similarity is lower than or equal to the threshold, it indicates that the behavior may be abnormal or deviate; therefore, the corresponding probability matrix is ​​classified as a matrix to be labeled that meets the second preset condition. This threshold-based discrimination mechanism allows for the initial screening of the reliability of the client model's output.

[0104] By calculating the similarity between the probability matrix of each target client and the standard probability matrix, and classifying them according to a set similarity threshold, the system effectively achieves automatic identification and classification of client model behavior. By assigning matrices with high similarity to the matrix to be labeled that meets the first preset condition, the system can quickly locate trustworthy clients that are highly consistent with the standard model; while assigning matrices with low similarity to the matrix to be labeled that meets the second preset condition facilitates focusing on models that may exhibit abnormal, distorted, or malicious behavior, improving the efficiency and accuracy of model screening. This helps ensure the stability, security, and controllability of the overall system decision-making, and is particularly suitable for the dynamic monitoring and management of client model quality in multi-party collaborative environments such as federated learning.

[0105] S5. Type-label the matrix to be labeled, and use the labeled matrix to update the information of the client cluster to obtain the updated cluster.

[0106] In this embodiment of the invention, clients are categorized using a similarity threshold. Clients with similarity values ​​above the threshold are marked as benign, while those below or equal to the threshold are marked as malicious. The marking results are then fed back to each client cluster. Based on the proportion of malicious clients within each cluster, it is determined whether a preset threshold has been reached, and thus, a decision is made on whether to remove, update, or retain that cluster. This achieves dynamic filtering and optimization of client clusters, which helps improve the aggregation reliability and overall system security during the federated learning process.

[0107] In specific healthcare scenarios, this technology can be used for quality control of disease prediction models uploaded by hospitals or medical institutions under a federated learning framework. By comparing the prediction probabilities of each institution's model on a unified auxiliary sample with a nationally recognized authoritative standard model, the system determines the reliability of the model output based on the similarity results. If some models deviate significantly from the standard, they can be marked as anomalies, and at the clustering level, groups of hospitals with potential data quality or modeling problems can be identified, thereby eliminating unreliable models and ensuring the stability and healthcare safety of the federated model nationwide.

[0108] In specific fintech scenarios, this technology can be used for risk identification in risk control model review and multi-institutional collaborative modeling. Credit scoring models uploaded by various branches will be centrally evaluated. By comparing the scoring probabilities of the credit scoring models on a standard customer set with the output of the headquarters' standard model, the system automatically identifies models that may exhibit overfitting, strategy bias, or fraudulent operations. It also dynamically assesses the overall risk level of the institutional group to which these models belong, helping to eliminate high-risk model sources, optimize the quality of cross-institutional collaborative modeling, and improve the robustness and compliance of the financial decision-making system.

[0109] In this embodiment of the invention, the step of labeling the matrix to be labeled with a type and updating the client cluster using the labeled matrix to obtain the updated cluster includes:

[0110] Mark the target clients corresponding to the matrix to be labeled that meet the preset first condition as benign clients;

[0111] The target client corresponding to the matrix to be labeled that meets the preset second condition is marked as a malicious client;

[0112] The benign clients and the malicious clients are respectively matched with the target clients in each client cluster, and the proportion of malicious clients in each client cluster is calculated.

[0113] Determine whether the proportion of malicious clients is less than a preset proportion threshold;

[0114] If the proportion of malicious clients is greater than or equal to the proportion threshold, then the client clusters corresponding to the proportion of malicious clients that is greater than or equal to the proportion threshold are subjected to credibility detection to obtain credibility detection results.

[0115] When the credibility detection result indicates that the client cluster is untrustworthy, delete the client cluster corresponding to the proportion of malicious clients that is greater than or equal to the proportion threshold.

[0116] When the credibility detection result indicates that the client cluster is trustworthy, the malicious clients of the client cluster are reclassified and detected to obtain the final classification result;

[0117] The client cluster is updated using the final classification result to obtain the updated cluster;

[0118] If the proportion of malicious clients is less than the proportion threshold, then the information of the client clusters corresponding to the proportion of malicious clients that is less than the proportion threshold is updated to obtain updated clusters.

[0119] In detail, the target clients corresponding to the matrix to be labeled that meets the first preset condition and the matrix to be labeled that meets the second preset condition are labeled respectively. The former is identified as a benign client and the latter as a malicious client. The labeled client status is matched one by one with the members in each client cluster to confirm the number of benign and malicious clients in each cluster, and the proportion of malicious clients in each client cluster is calculated, providing a basis for subsequent cluster credibility assessment and processing strategies.

[0120] In this embodiment of the invention, the step of performing credibility detection on the client clusters corresponding to the proportion of malicious clients greater than or equal to the proportion threshold, and obtaining credibility detection results, includes:

[0121] The client clusters corresponding to the proportion of malicious clients that is greater than or equal to the aforementioned proportion threshold are designated as clusters to be detected.

[0122] Obtain the initial trust weight and target weight of each target client in the cluster to be detected, as well as the classification label of the target client;

[0123] The target weight is updated based on the classification label and the initial confidence weight to obtain the updated weight;

[0124] Determine whether the updated weight is less than a null value;

[0125] If the updated weight is greater than or equal to a null value, then the target client corresponding to the updated weight is determined to be a benign client.

[0126] If the updated weight is less than a null value, the target client corresponding to the updated weight is identified as a malicious client, and the malicious ratio is determined based on the malicious client.

[0127] Determine whether the malicious ratio is less than the ratio threshold;

[0128] If the malicious ratio is less than the ratio threshold, then the trustworthiness of the cluster to be detected is taken as the trustworthiness detection result;

[0129] If the malicious proportion is greater than or equal to the proportion threshold, then the untrusted cluster to be detected is taken as the credibility detection result.

[0130] In detail, the credibility of a client is dynamically adjusted by combining the initial credibility weight of the target client in each client cluster, the current classification label (e.g., benign or malicious), and the target weight. The initial credibility weight of each client and the target weight reference value set in the current task are obtained, and the credibility of the behavior is determined based on the classification label: if it is a benign client, the weight is increased to enhance its influence in model aggregation; if it is a malicious client, the weight is decreased accordingly to suppress its interference with the global model. This achieves adaptive weight update based on behavior classification, which helps to build a more robust and secure model aggregation process.

[0131] By applying a threshold judgment to the updated client weights, the trustworthiness of the clients is further identified. It is determined whether the updated weight of each target client is less than a set null value (i.e., the lower trustworthiness threshold): if the weight is greater than or equal to the null value, the client is considered benign; if the weight is lower than the null value, it is identified as a malicious client. The proportion of malicious clients in the entire client cluster is then calculated and compared with a threshold: if the proportion is lower than the threshold, the cluster is considered trustworthy overall and marked as a trustworthy client cluster; if the proportion exceeds or equals the threshold, the cluster is considered to have a significant abnormal risk and marked as an untrustworthy client cluster, thus achieving dynamic judgment of the trustworthiness of client clusters.

[0132] By assessing the proportion of malicious clients within a client cluster, a tiered trustworthiness processing strategy is further implemented. First, it checks if the proportion of malicious clients in a given cluster is less than a preset threshold. If it is greater than or equal to this threshold, the cluster may pose a high risk and requires trustworthiness testing. If the test results indicate the cluster is untrustworthy, it is directly deleted to prevent interference with the global model. If the test results indicate trustworthiness, the abnormal behavior is controllable, and the system reclassifies and tests the malicious clients within the cluster, updating the cluster based on the final classification results to obtain a structurally optimized updated cluster. If the proportion of malicious clients is below the threshold, the cluster is considered to have a low overall risk, and only its information status is updated, generating a corresponding updated cluster. This achieves tiered processing and dynamic optimization of client clusters, enhancing the system's responsiveness and defense efficiency against abnormal behavior.

[0133] Based on the final classification results, the identity attributes of each client (such as benign or malicious) are confirmed, and their trust level, participation weight, or status label in the cluster are adjusted accordingly. Clients re-identified as benign can have their trust restored, while confirmed malicious clients can have their influence reduced or be removed.

[0134] By classifying the target matrix into benign and malicious categories and analyzing the corresponding labeling results with client cluster members, dynamic updates and risk control of client clusters are achieved. By statistically analyzing the proportion of malicious clients in each cluster and comparing it with a preset threshold, credibility detection can be performed on high-risk clusters, and untrustworthy clusters can be directly removed to ensure that abnormal behavior is not included in model aggregation. For clusters that remain trustworthy, further refined classification and member status updates are performed. For low-risk clusters, information updates are completed quickly, improving processing efficiency. Overall, this approach balances security and flexibility, enhances the system's ability to identify abnormal clients, and effectively improves the accuracy of client clustering and the robustness of the aggregation strategy during federated learning.

[0135] S6. Aggregate all the updated clusters to obtain the final defense model.

[0136] In this embodiment of the invention, by detecting the differences between client updates and the global model, the impact of abnormal updates is identified and limited, ensuring the stability of model aggregation. For clients with differences exceeding a threshold, a scaling factor is used to compress client updates exceeding the threshold; normal updates are directly retained. All processed client updates are weighted and aggregated to optimize the global model, thereby achieving defense against abnormal behavior and improving the overall security and robustness of the federated learning system.

[0137] In specific healthcare scenarios, it can be used in federated learning involving multiple medical institutions to detect and regulate differences in model updates uploaded by each institution. By limiting the impact of abnormal updates, it prevents model bias caused by data anomalies or equipment malfunctions in certain institutions, ensuring the accuracy and stability of the global diagnostic model, thereby improving the reliability of remote collaborative diagnosis and treatment systems and patient safety.

[0138] In specific fintech scenarios, this technology is applied to the training of joint risk control models across multiple branches. By monitoring and scaling model updates uploaded by abnormal branches, it prevents risk control model deviations caused by fraud or data anomalies, effectively maintaining the robustness of the overall credit scoring model, reducing financial risks, and ensuring the compliance and decision-making accuracy of financial institutions in a multi-institutional collaborative environment.

[0139] In this embodiment of the invention, the aggregation of all the updated clusters to obtain the final defense model includes:

[0140] Obtain the current global model and determine the difference norm between each target client in the update cluster and the current global model;

[0141] Determine whether the difference norm is greater than a preset update norm threshold;

[0142] If the difference norm is greater than the update norm threshold, a scaling factor is obtained, and the target client corresponding to the difference norm greater than the update norm threshold is scaled using the scaling factor to obtain the updated client, and the updated client is replaced in the update cluster;

[0143] If the difference norm is less than or equal to the update norm threshold, then the updated cluster remains unchanged;

[0144] The final update value is obtained by weighted aggregation of all the update clusters.

[0145] The final updated value is used to iteratively optimize the current global model to obtain the final defense model.

[0146] In detail, the impact of each client on the global model is assessed by measuring the difference between the current global model and the updates of each client model. This involves obtaining the global model parameters for the current round, extracting the local model updates of the target clients in each update cluster, comparing each client's update with the global model, and calculating the difference norm. The Euclidean norm (L2 norm) is typically used to measure the distance between the two in the parameter space. A larger difference norm indicates a more significant difference between the client update and the global model, potentially indicating anomalies or deviations, providing a quantitative basis for subsequent update control and anomaly suppression.

[0147] Determine if the difference norm is greater than the update norm threshold: If the difference norm is greater than the update norm threshold, suppress the interference of client updates that deviate too much from the global model on the overall model. Calculate a scaling factor, usually determined based on the ratio between the difference norm and the threshold, to compress client update amplitudes that deviate too much from the global model to a safe range. Use the scaling factor to scale the client's model update, generating an adjusted updated client. Replace the original client update content with the updated client and update it in the client cluster, thus ensuring that the aggregation phase only merges client updates within a reasonable range. If the difference norm is less than or equal to the update norm threshold, keep the update cluster unchanged.

[0148] By weighted aggregation of model update results from clients across all update clusters, and considering the credibility weights or contributions of each client, a global final update value is calculated. Each client is assigned a corresponding aggregation weight, typically based on data quality, historical performance, or credibility metrics. The model updates from each client are then integrated using a weighted average method, ensuring that high-credibility clients have a greater impact on the results, while the impact of abnormal or low-credibility clients is limited. The final update value serves as an important basis for global model optimization.

[0149] By applying the final updated value obtained from weighted aggregation to the current global model, an iterative optimization is performed to update and adjust the model parameters. The final updated value, as the effective gradient or parameter difference contributed collectively by trusted clients in this round, is then superimposed onto the current global model to complete a new round of model updates. This not only integrates valuable information from highly trusted clients but also effectively suppresses interference from abnormal or malicious updates, thereby improving the stability and robustness of the global model and ultimately resulting in a defensive optimization outcome.

[0150] By dynamically detecting and controlling the differences between client updates and the global model, abnormal updates are intelligently scaled to prevent negative impacts on model training from abnormal or malicious clients, effectively improving the robustness and security of the aggregation process. Combined with a weighted aggregation mechanism, this ensures that updates from trusted clients are fully utilized while suppressing potential attacks and abnormal behavior, thereby enhancing the overall defense capabilities and model performance stability of the federated learning system. This ensures that the final defense model possesses higher robustness and reliability, thus improving the defense efficiency against Byzantine attacks and the coverage of defense scenarios.

[0151] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0152] like Figure 4 The diagram shown is a functional block diagram of a Byzantine attack defense device based on federated learning provided in an embodiment of the present invention.

[0153] This disclosure provides a Byzantine attack defense device based on federated learning, which corresponds one-to-one with the Byzantine attack defense method based on federated learning described in the previous embodiments. Figure 4 As shown, this Byzantine attack defense device 100 based on federated learning can be installed in an electronic device. According to its functions, the Byzantine attack defense device 100 based on federated learning includes a probability matrix construction module 101, a distance matrix construction module 102, a client clustering module 103, a similarity determination module 104, an information update module 105, and an update cluster aggregation module 106. Detailed descriptions of each functional module are as follows:

[0154] The probability matrix construction module 101 is used to obtain local models and sample datasets of several target clients, perform forward propagation on the sample datasets using the local models to obtain the output probability vector of each target client, and construct several probability matrices based on the output probability vectors and the sample datasets.

[0155] The distance matrix construction module 102 is used to determine the Euclidean distance between any two probability matrices and construct a distance matrix based on the Euclidean distance.

[0156] The client clustering module 103 is used to perform hierarchical clustering of the target client based on the distance matrix to obtain several client clusters;

[0157] The similarity determination module 104 is used to obtain the standard model of each target client and the probability standard matrix of the standard model, determine the similarity between the probability matrix and the probability standard matrix one by one, and generate a matrix to be labeled based on the similarity.

[0158] The information update module 105 is used to mark the type of the matrix to be marked, and to update the information of the client cluster using the marked matrix to obtain the updated cluster;

[0159] The update cluster aggregation module 106 is used to aggregate all the updated clusters to obtain the final defense model.

[0160] In one embodiment, when the probability matrix construction module 101 performs forward propagation on the sample dataset using the local model to obtain the output probability vector of each target client, and constructs a plurality of probability matrices based on the output probability vector and the sample dataset, it is used to:

[0161] The local model is used to perform forward propagation on each sample data in the sample dataset to obtain the original output vector of each sample data.

[0162] The original output vector is normalized to obtain the output probability vector;

[0163] The output probability vectors are stacked according to the order of the sample data in the sample dataset to obtain the probability matrix of each sample data in the sample dataset.

[0164] In one embodiment, when the distance matrix construction module 102 performs the task of determining the Euclidean distance between any two of the probability matrices and constructing a distance matrix based on the Euclidean distance, it is configured to:

[0165] Two probability matrices are randomly selected as the two target matrices;

[0166] The difference between the two target matrices is obtained, and the sum of the squares of the difference is obtained.

[0167] The summation result is squared to obtain the Euclidean distance between the two target matrices;

[0168] All the Euclidean distances are summarized into a distance matrix.

[0169] In one embodiment, when the client clustering module 103 performs hierarchical clustering of the target client based on the distance matrix to obtain several client clusters, it is used to:

[0170] Use the distance matrix of each target client as the initial cluster;

[0171] Two initial clusters are randomly selected as the first target cluster and the second target cluster, and the first target cluster and the second target cluster are taken as a target cluster pair;

[0172] The target cluster pairs are merged to obtain a merged cluster;

[0173] Determine the intra-cluster squared errors of the first target cluster, the second target cluster, and the merged cluster, respectively;

[0174] Construct a cost function based on the intra-cluster squared error;

[0175] The cost function is used to determine the merging cost increment of the target cluster pair;

[0176] The merging cost increments of each target cluster pair are compared, and the target cluster pair corresponding to the smallest merging cost increment is selected based on the comparison results.

[0177] The selected target cluster pairs are merged to obtain the merged updated cluster;

[0178] The process of randomly selecting two initial clusters as the first target cluster and the second target cluster, and using the first target cluster and the second target cluster as a target cluster pair, continues until a preset number of merged update clusters are found, at which point the process stops.

[0179] Obtain the target clients corresponding to the distance matrix in the final merged update cluster, and summarize the final target clients to obtain the client cluster.

[0180] In one embodiment, when the information update module 105 performs type labeling on the matrix to be labeled and updates the information of the client cluster using the labeled matrix to obtain the updated cluster, it is used to:

[0181] Mark the target clients corresponding to the matrix to be labeled that meet the preset first condition as benign clients;

[0182] The target client corresponding to the matrix to be labeled that meets the preset second condition is marked as a malicious client;

[0183] The benign clients and the malicious clients are respectively matched with the target clients in each client cluster, and the proportion of malicious clients in each client cluster is calculated.

[0184] Determine whether the proportion of malicious clients is less than a preset proportion threshold;

[0185] If the proportion of malicious clients is greater than or equal to the proportion threshold, then the client clusters corresponding to the proportion of malicious clients that is greater than or equal to the proportion threshold are subjected to credibility detection to obtain credibility detection results.

[0186] When the credibility detection result indicates that the client cluster is untrustworthy, delete the client cluster corresponding to the proportion of malicious clients that is greater than or equal to the proportion threshold.

[0187] When the credibility detection result indicates that the client cluster is trustworthy, the malicious clients of the client cluster are reclassified and detected to obtain the final classification result;

[0188] The client cluster is updated using the final classification result to obtain the updated cluster;

[0189] If the proportion of malicious clients is less than the proportion threshold, then the information of the client clusters corresponding to the proportion of malicious clients that is less than the proportion threshold is updated to obtain updated clusters.

[0190] In one embodiment, when the information update module 105 performs credibility detection on the client clusters corresponding to the proportion of malicious clients greater than or equal to the proportional threshold and obtains the credibility detection result, it is used to:

[0191] The client clusters corresponding to the proportion of malicious clients that is greater than or equal to the aforementioned proportion threshold are designated as clusters to be detected.

[0192] Obtain the initial trust weight and target weight of each target client in the cluster to be detected, as well as the classification label of the target client;

[0193] The target weight is updated based on the classification label and the initial confidence weight to obtain the updated weight;

[0194] Determine whether the updated weight is less than a null value;

[0195] If the updated weight is greater than or equal to a null value, then the target client corresponding to the updated weight is determined to be a benign client.

[0196] If the updated weight is less than a null value, the target client corresponding to the updated weight is identified as a malicious client, and the malicious ratio is determined based on the malicious client.

[0197] Determine whether the malicious ratio is less than the ratio threshold;

[0198] If the malicious ratio is less than the ratio threshold, then the trustworthiness of the cluster to be detected is taken as the trustworthiness detection result;

[0199] If the malicious proportion is greater than or equal to the proportion threshold, then the untrusted cluster to be detected is taken as the credibility detection result.

[0200] In one embodiment, when the update cluster aggregation module 106 performs aggregation on all said update clusters to obtain the final defense model, it is used to:

[0201] Obtain the current global model and determine the difference norm between each target client in the update cluster and the current global model;

[0202] Determine whether the difference norm is greater than a preset update norm threshold;

[0203] If the difference norm is greater than the update norm threshold, a scaling factor is obtained, and the target client corresponding to the difference norm greater than the update norm threshold is scaled using the scaling factor to obtain the updated client, and the updated client is replaced in the update cluster;

[0204] If the difference norm is less than or equal to the update norm threshold, then the updated cluster remains unchanged;

[0205] The final update value is obtained by weighted aggregation of all the update clusters.

[0206] The final updated value is used to iteratively optimize the current global model to obtain the final defense model.

[0207] In this invention, a Byzantine attack defense device based on federated learning is proposed. First, the invention acquires local models and sample datasets of several target clients. The local models are then used to perform forward propagation on the sample datasets to obtain the output probability vectors of each target client. Several probability matrices are constructed based on these output probability vectors and the sample datasets. This allows for a comprehensive characterization of the response behavior of each client model to the same input without revealing model parameters or original data. The Euclidean distance between any two probability matrices is determined, and a distance matrix is ​​constructed based on this Euclidean distance. This allows for a direct quantification of the output differences of different client models on a unified sample dataset, revealing the similarity or difference in their prediction behaviors. Then, hierarchical clustering of the target clients is performed based on the distance matrices to obtain several client clusters. The intra-cluster squared error is used as a metric for merging costs to ensure that each merge minimizes the increase in intra-cluster differences, thereby maintaining the compactness and consistency of the clusters. By iteratively comparing merging costs and selecting the optimal merging pair, the degradation of clustering quality caused by arbitrary merging is avoided, achieving high-quality client grouping. The standard model and probability standard matrix of each target client are obtained. The similarity between each probability matrix and the probability standard matrix is ​​determined, and a labeling matrix is ​​generated based on the similarity. The labeling matrix is ​​then type-labeled, and the client clusters are updated using the labeled matrix to obtain updated clusters. This approach balances security and flexibility, enhances the system's ability to identify abnormal clients, and effectively improves the accuracy of client clustering and the robustness of the aggregation strategy during federated learning. Finally, all updated clusters are aggregated to obtain the final defense model. By dynamically detecting and controlling the difference between client updates and the global model, abnormal updates are intelligently scaled to prevent abnormal or malicious clients from negatively impacting model training, effectively improving the robustness and security of the aggregation process. Combined with a weighted aggregation mechanism, this approach ensures that updates from trusted clients play a full role while suppressing potential attacks and abnormal behavior, effectively improving the defense efficiency against Byzantine attacks and the coverage of defense scenarios. For specific limitations regarding a federated learning-based Byzantine attack defense device, please refer to the limitations of a federated learning-based Byzantine attack defense method described above, which will not be repeated here. The modules in the aforementioned federated learning-based Byzantine attack defense device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0208] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements server-side functions or steps of a federated learning-based Byzantine attack defense method.

[0209] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements client-side functions or steps of a federated learning-based Byzantine attack defense method.

[0210] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0211] Obtain local models and sample datasets for several target clients, perform forward propagation on the sample datasets using the local models to obtain the output probability vectors for each target client, and construct several probability matrices based on the output probability vectors and the sample datasets;

[0212] Determine the Euclidean distance between any two of the probability matrices, and construct a distance matrix based on the Euclidean distance;

[0213] Based on the distance matrix, hierarchical clustering is performed on the target clients to obtain several client clusters;

[0214] Obtain the standard model and the probability standard matrix of each target client, determine the similarity between the probability matrix and the probability standard matrix one by one, and generate a matrix to be labeled based on the similarity.

[0215] The matrix to be labeled is then labeled with a type, and the labeled matrix is ​​used to update the information of the client cluster to obtain the updated cluster;

[0216] Aggregate all the updated clusters to obtain the final defense model.

[0217] In the several embodiments provided by this invention, it should be understood that the disclosed devices and apparatuses can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0218] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0219] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0220] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0221] In some embodiments of this example, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that the computer program, when executed by a processor, implements the steps of the method described in the above embodiments.

[0222] The readable storage medium of the present invention stores a computer program, which, when executed by a processor of an electronic device, can perform the following:

[0223] Obtain local models and sample datasets for several target clients, perform forward propagation on the sample datasets using the local models to obtain the output probability vectors for each target client, and construct several probability matrices based on the output probability vectors and the sample datasets;

[0224] Determine the Euclidean distance between any two of the probability matrices, and construct a distance matrix based on the Euclidean distance;

[0225] Based on the distance matrix, hierarchical clustering is performed on the target clients to obtain several client clusters;

[0226] Obtain the standard model and the probability standard matrix of each target client, determine the similarity between the probability matrix and the probability standard matrix one by one, and generate a matrix to be labeled based on the similarity.

[0227] The matrix to be labeled is then labeled with a type, and the labeled matrix is ​​used to update the information of the client cluster to obtain the updated cluster;

[0228] Aggregate all the updated clusters to obtain the final defense model.

[0229] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0230] Computer-readable storage media may also store at least one computer-executable program / instruction, such as computer-readable instructions. Computer-readable storage media include, but are not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Computer-readable storage media may include, for example, read-only memory (ROM), hard disk, flash memory, etc. For example, a non-transitory computer-readable storage medium may be connected to a computing device such as a computer, and then, when the computing device executes the computer-readable instructions stored on the computer-readable storage medium, the various methods described above can be performed.

[0231] In addition, the computer device may include (but is not limited to) a data bus, an input / output (I / O) bus, a display, and input / output devices (e.g., keyboard, mouse, speakers, etc.).

[0232] The processor can communicate with external devices via the I / O bus through wired or wireless networks.

[0233] In one embodiment, the at least one computer-executable instruction may also be compiled into or comprise a software product / computer program product, wherein one or more computer-executable instructions are executed by a processor to perform the steps of the various functions and / or methods in the embodiments described herein.

[0234] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0235] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0236] In the embodiments provided in this disclosure, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0237] It should be noted that, in this disclosure, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element limited by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0238] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

[0239] It should be noted that if any software tools or components not belonging to our company appear in the embodiments of this application, they are merely for illustrative purposes and do not represent actual use.

Claims

1. A Byzantine attack defense method based on federated learning, characterized in that, The method includes: Obtain local models and sample datasets for several target clients, perform forward propagation on the sample datasets using the local models to obtain the output probability vectors for each target client, and construct several probability matrices based on the output probability vectors and the sample datasets; Determine the Euclidean distance between any two of the probability matrices, and construct a distance matrix based on the Euclidean distance; Based on the distance matrix, hierarchical clustering is performed on the target clients to obtain several client clusters; Obtain the standard model and the probability standard matrix of each target client, determine the similarity between the probability matrix and the probability standard matrix one by one, and generate a matrix to be labeled based on the similarity. The matrix to be labeled is then labeled with a type, and the labeled matrix is ​​used to update the information of the client cluster to obtain the updated cluster; Aggregate all the updated clusters to obtain the final defense model.

2. The Byzantine attack defense method based on federated learning as described in claim 1, characterized in that, The process involves forward propagating the sample dataset using the local model to obtain the output probability vector for each target client, and constructing several probability matrices based on the output probability vectors and the sample dataset, including: The local model is used to perform forward propagation on each sample data in the sample dataset to obtain the original output vector of each sample data. The original output vector is normalized to obtain the output probability vector; The output probability vectors are stacked according to the order of the sample data in the sample dataset to obtain the probability matrix of each sample data in the sample dataset.

3. The Byzantine attack defense method based on federated learning as described in claim 1, characterized in that, Determining the Euclidean distance between any two probability matrices and constructing a distance matrix based on the Euclidean distance includes: Two probability matrices are randomly selected as the two target matrices; The difference between the two target matrices is obtained, and the sum of the squares of the difference is obtained. The summation result is squared to obtain the Euclidean distance between the two target matrices; All the Euclidean distances are summarized into a distance matrix.

4. The Byzantine attack defense method based on federated learning as described in claim 1, characterized in that, The hierarchical clustering of the target clients based on the distance matrix yields several client clusters, including: Use the distance matrix of each target client as the initial cluster; Two initial clusters are randomly selected as the first target cluster and the second target cluster, and the first target cluster and the second target cluster are taken as a target cluster pair; The target cluster pairs are merged to obtain a merged cluster; Determine the intra-cluster squared errors of the first target cluster, the second target cluster, and the merged cluster, respectively; Construct a cost function based on the intra-cluster squared error; The cost function is used to determine the merging cost increment of the target cluster pair; The merging cost increments of each target cluster pair are compared, and the target cluster pair corresponding to the smallest merging cost increment is selected based on the comparison results. The selected target cluster pairs are merged to obtain the merged updated cluster; The process of randomly selecting two initial clusters as the first target cluster and the second target cluster, and using the first target cluster and the second target cluster as a target cluster pair, continues until a preset number of merged update clusters are found, at which point the process stops. Obtain the target clients corresponding to the distance matrix in the final merged update cluster, and summarize the final target clients to obtain the client cluster.

5. The Byzantine attack defense method based on federated learning as described in claim 1, characterized in that, The step of labeling the matrix to be labeled with a type and then updating the client cluster using the labeled matrix to obtain the updated cluster includes: Mark the target clients corresponding to the matrix to be labeled that meet the preset first condition as benign clients; The target client corresponding to the matrix to be labeled that meets the preset second condition is marked as a malicious client; The benign clients and the malicious clients are respectively matched with the target clients in each client cluster, and the proportion of malicious clients in each client cluster is calculated. Determine whether the proportion of malicious clients is less than a preset proportion threshold; If the proportion of malicious clients is greater than or equal to the proportion threshold, then the client clusters corresponding to the proportion of malicious clients that is greater than or equal to the proportion threshold are subjected to credibility detection to obtain credibility detection results. When the credibility detection result indicates that the client cluster is untrustworthy, delete the client cluster corresponding to the proportion of malicious clients that is greater than or equal to the proportion threshold. When the credibility detection result indicates that the client cluster is trustworthy, the malicious clients of the client cluster are reclassified and detected to obtain the final classification result; The client cluster is updated using the final classification result to obtain the updated cluster; If the proportion of malicious clients is less than the proportion threshold, then the information of the client clusters corresponding to the proportion of malicious clients that is less than the proportion threshold is updated to obtain updated clusters.

6. The Byzantine attack defense method based on federated learning as described in claim 5, characterized in that, The step of performing credibility detection on client clusters corresponding to the proportion of malicious clients greater than or equal to the proportion threshold, and obtaining credibility detection results, includes: The client clusters corresponding to the proportion of malicious clients that is greater than or equal to the aforementioned proportion threshold are designated as clusters to be detected. Obtain the initial trust weight and target weight of each target client in the cluster to be detected, as well as the classification label of the target client; The target weight is updated based on the classification label and the initial confidence weight to obtain the updated weight; Determine whether the updated weight is less than a null value; If the updated weight is greater than or equal to a null value, then the target client corresponding to the updated weight is determined to be a benign client. If the updated weight is less than a null value, the target client corresponding to the updated weight is identified as a malicious client, and the malicious ratio is determined based on the malicious client. Determine whether the malicious ratio is less than the ratio threshold; If the malicious ratio is less than the ratio threshold, then the trustworthiness of the cluster to be detected is taken as the trustworthiness detection result; If the malicious proportion is greater than or equal to the proportion threshold, then the untrusted cluster to be detected is taken as the credibility detection result.

7. The Byzantine attack defense method based on federated learning as described in claim 1, characterized in that, The aggregation of all the updated clusters to obtain the final defense model includes: Obtain the current global model and determine the difference norm between each target client in the update cluster and the current global model; Determine whether the difference norm is greater than a preset update norm threshold; If the difference norm is greater than the update norm threshold, a scaling factor is obtained, and the target client corresponding to the difference norm greater than the update norm threshold is scaled using the scaling factor to obtain the updated client, and the updated client is replaced in the update cluster; If the difference norm is less than or equal to the update norm threshold, then the updated cluster remains unchanged; The final update value is obtained by weighted aggregation of all the update clusters. The final updated value is used to iteratively optimize the current global model to obtain the final defense model.

8. A Byzantine attack defense device based on federated learning, characterized in that, The device includes: The probability matrix construction module is used to obtain local models and sample datasets of several target clients, perform forward propagation on the sample datasets using the local models to obtain the output probability vectors of each target client, and construct several probability matrices based on the output probability vectors and the sample datasets. A distance matrix construction module is used to determine the Euclidean distance between any two probability matrices and to construct a distance matrix based on the Euclidean distance. The client clustering module is used to perform hierarchical clustering of the target clients based on the distance matrix to obtain several client clusters; The similarity determination module is used to obtain the standard model of each target client and the probability standard matrix of the standard model, determine the similarity between the probability matrix and the probability standard matrix one by one, and generate a matrix to be labeled based on the similarity. The information update module is used to mark the type of the matrix to be marked, and to update the information of the client cluster using the marked matrix to obtain the updated cluster; The update cluster aggregation module is used to aggregate all the updated clusters to obtain the final defense model.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform a federated learning-based Byzantine attack defense method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements a Byzantine attack defense method based on federated learning as described in any one of claims 1 to 7.

Images