Federal learning-based Byzantine attack defense method, apparatus and device, and medium

Abstract

The invention relates to the technical field of safety protection, can be applied to business system platforms of financial science and technology, medical health and the like, and discloses a Byzantine attack defense method, device, equipment and medium based on federated learning. Carrying out forward propagation on the sample data set by utilizing a local model, constructing a plurality of probability matrixes, determining the Euclidean distance between any two probability matrixes, constructing a distance matrix, carrying out hierarchical clustering on target clients to obtain a plurality of client clusters, obtaining a probability standard matrix of a standard model, and obtaining the probability standard matrix of the standard model; and determining the similarity between the probability matrix and the probability standard matrix one by one, performing type marking on the to-be-marked matrix according to the similarity, performing information updating on the client cluster to obtain an updated cluster, and aggregating all the updated clusters to obtain a final defense model. According to the method, the defense efficiency and the defense scene coverage rate of the Byzantine attack can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of security protection technology, and in particular to a Byzantine attack defense method, apparatus, equipment, and medium based on federated learning. Background Art

[0002] In the field of federated learning, existing technologies offer several defenses against Byzantine attacks. Some Byzantine Robust Aggregation Rules (BRARs) directly analyze local model updates to mitigate attacks. Other defenses leverage gradient statistics, inter-gradient distances, additional verification data, optimization algorithm compensation, and differential privacy. In privacy-preserving federated learning, defenses leverage secure multi-party computation, trusted execution environments, and measurable additive masking. However, these existing technologies suffer from numerous drawbacks and challenges. Many traditional defenses, due to their direct analysis of local model updates, incur significant computational overhead when dealing with modern models with large parameter sizes, such as large language models (LLMs), significantly reducing their effectiveness. Furthermore, existing defense mechanisms often only protect against certain types of Byzantine attacks and struggle to address diverse attack vectors. Their robustness is significantly limited in complex scenarios with a high proportion of Byzantine clients or heterogeneous data.

[0003] In the healthcare sector, federated learning requires collaborative modeling across multiple medical institutions while protecting patient privacy. However, medical data is often characterized by high dimensionality, sparsity, small sample sizes, and strong heterogeneity, which places higher demands on Byzantine robustness. Existing defense mechanisms struggle to maintain model convergence stability when faced with highly heterogeneous distributions. Furthermore, incorporating privacy protection measures such as differential privacy or secure multi-party computation significantly increases system communication and computational overhead, limiting practical deployment efficiency. Furthermore, due to the sensitive nature of the data in medical scenarios, external validation sets cannot be used, making defense methods based on additional validation data difficult to apply.

[0004] In the fintech sector, federated learning is often used for high-value tasks such as cross-institutional credit assessment and fraud detection, where data privacy and model accuracy are equally critical. However, financial data is extremely unevenly distributed and significantly impacted by the heterogeneity of business systems, making models susceptible to interference from non-IID data. Furthermore, Byzantine attackers can implement more subtle and targeted attacks through forged transactions or model drift strategies, making them difficult for traditional defense mechanisms to accurately identify and isolate. Furthermore, to meet compliance and security requirements, financial scenarios often require deployment in restricted environments, which poses greater challenges to the compatibility of computing resources and model complexity, further weakening the applicability and practicality of existing defense strategies.

[0005] In summary, direct analysis of local model updates incurs enormous computational overhead in high-dimensional models such as large language models, severely impacting defense efficiency. Most mechanisms can only handle specific types of attacks and struggle to cover diverse and complex attack scenarios. In environments with a high proportion of Byzantine clients or highly heterogeneous data distribution, models are prone to instability and lack robustness, making it difficult to meet the dual requirements of security and generalization capabilities in actual deployments.

[0006] Therefore, current technologies have the problems of low defense efficiency against Byzantine attacks and low coverage of defense scenarios. Summary of the Invention

[0007] The present invention provides a Byzantine attack defense method, apparatus, device and medium based on federated learning, the main purpose of which is to solve the problems of low defense efficiency and low defense scenario coverage against Byzantine attacks.

[0008] In a first aspect, to achieve the above-mentioned objectives, the present invention provides a Byzantine attack defense method based on federated learning, comprising: Obtaining local models and sample data sets of several target clients, performing forward propagation on the sample data sets using the local models to obtain an output probability vector for each target client, and constructing several probability matrices based on the output probability vector and the sample data sets; Determining the Euclidean distance between any two of the probability matrices, and constructing a distance matrix based on the Euclidean distance; Performing hierarchical clustering on the target clients according to the distance matrix to obtain a plurality of client clusters; Obtaining a standard model of each target client and a probability standard matrix of the standard model, determining the similarity between the probability matrix and the probability standard matrix one by one, and generating a matrix to be labeled according to the similarity; Marking the type of the matrix to be marked, and using the marked matrix to update information of the client cluster to obtain an updated cluster; All the update clusters are aggregated to obtain the final defense model.

[0009] In a second aspect, the present invention further provides a Byzantine attack defense device based on federated learning, comprising: A probability matrix construction module is used to obtain local models and sample data sets of several target clients, perform forward propagation on the sample data sets using the local models to obtain an output probability vector for each target client, and construct several probability matrices based on the output probability vectors and the sample data sets; a distance matrix construction module, configured to determine the Euclidean distance between any two of the probability matrices and construct a distance matrix based on the Euclidean distance; A client clustering module, configured to perform hierarchical clustering on the target clients according to the distance matrix to obtain a plurality of client clusters; a similarity determination module, configured to obtain a standard model of each target client and a probability standard matrix of the standard model, determine the similarity between the probability matrix and the probability standard matrix one by one, and generate a matrix to be labeled based on the similarity; An information updating module, configured to mark the type of the matrix to be marked, and update the information of the client cluster using the marked matrix to obtain an updated cluster; The update cluster aggregation module is used to aggregate all the update clusters to obtain a final defense model.

[0010] In a third aspect, the present invention further provides an electronic device, comprising: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the above-mentioned Byzantine attack defense method based on federated learning.

[0011] In a fourth aspect, the present invention also provides a computer-readable storage medium, which stores at least one computer program, and the at least one computer program is executed by a processor in an electronic device to implement the above-mentioned Byzantine attack defense method based on federated learning.

[0012] The present invention obtains local models and sample data sets of several target clients, uses the local models to perform forward propagation on the sample data sets to obtain the output probability vector of each target client, and constructs several probability matrices based on the output probability vectors and the sample data sets. It can comprehensively characterize the response behavior of each client model to the same input without leaking model parameters and original data, determine the Euclidean distance between any two probability matrices, and construct a distance matrix based on the Euclidean distance. It can intuitively quantify the output differences of different client models on the same sample data set, revealing the similarities or differences in their predicted behaviors, hierarchically cluster the target clients according to the distance matrix, and obtain several client clusters. The intra-cluster squared error is used as a metric for the merging cost to ensure that each merging minimizes the increase in intra-cluster differences, thereby maintaining the compactness and consistency of the clusters. By iteratively comparing merging costs and selecting the optimal merging pair, the system avoids the degradation of clustering quality caused by random merging and achieves high-quality client grouping. It obtains the standard model of each target client and the probability standard matrix of the standard model. It then determines the similarity between the probability matrix and the probability standard matrix one by one, generates a labeled matrix based on the similarity, labels the labeled matrix by type, and uses the labeled matrix to update the client cluster information to obtain an updated cluster. This balances security and flexibility, enhances the system's ability to identify abnormal clients, and effectively improves the accuracy of client clustering and the robustness of the aggregation strategy during federated learning. All updated clusters are aggregated to obtain the final defense model. By dynamically detecting and controlling the difference between client updates and the global model, it intelligently scales abnormal updates to prevent abnormal or malicious clients from negatively impacting model training, effectively improving the robustness and security of the aggregation process. Combined with a weighted aggregation mechanism, this ensures that trusted client updates are fully effective while suppressing potential attacks and abnormal behavior, effectively improving the efficiency of defense against Byzantine attacks and the coverage of defense scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0014] Figure 1 Schematic diagram of an application environment of a Byzantine attack defense method based on federated learning in one embodiment of the present invention; Figure 2 A flowchart of a Byzantine attack defense method based on federated learning provided by one embodiment of the present invention; Figure 3 A flowchart illustrating a probability matrix construction process in a Byzantine attack defense method based on federated learning provided in one embodiment of the present invention; Figure 4 A schematic diagram of a module of a Byzantine attack defense device based on federated learning provided by one embodiment of the present invention; Figure 5 A schematic diagram of the structure of an electronic device for implementing a Byzantine attack defense method based on federated learning provided in one embodiment of the present invention; Figure 6 Another structural diagram of an electronic device implementing a Byzantine attack defense method based on federated learning provided by one embodiment of the present invention.

[0015] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0016] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, and to fully understand and implement how the present disclosure applies technical means to solve technical problems and achieve the corresponding technical effects, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. The embodiments of the present disclosure and the various features in the embodiments can be combined with each other without conflict, and the technical solutions formed are all within the scope of protection of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present disclosure.

[0017] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, apparatus, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0018] The embodiment of the present application provides a Byzantine attack defense method based on federated learning, and the execution subject of the Byzantine attack defense method based on federated learning includes but is not limited to at least one of the electronic devices such as a server, a terminal, etc. that can be configured to execute the device provided by the embodiment of the present application. In other words, the Byzantine attack defense method based on federated learning can be executed by software or hardware installed on a terminal device or a server device. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0019] The embodiment of the present invention provides a Byzantine attack defense method based on federated learning, which can be applied in Figure 1In the application environment. The client communicates with the server through the network. The server can obtain the local models and sample data sets of several target clients through the client, use the local model to perform forward propagation on the sample data set to obtain the output probability vector of each target client, and construct several probability matrices based on the output probability vector and the sample data set. It can comprehensively characterize the response behavior of each client model to the same input without leaking the model parameters and the original data, determine the Euclidean distance between any two probability matrices, and construct a distance matrix based on the Euclidean distance. It can intuitively quantify the output differences of different client models on the same sample data set, reveal the similarity or difference of their predicted behaviors, perform hierarchical clustering on the target clients according to the distance matrix, and obtain several client clusters. The intra-cluster squared error is used as a metric for the merging cost to ensure that each merger minimizes the increase in intra-cluster differences, thereby maintaining the compactness and consistency of the cluster. By iteratively comparing merging costs and selecting the optimal merging pair, the system avoids the degradation of clustering quality caused by random merging and achieves high-quality client grouping. It then obtains the standard model and the probability standard matrix of each target client, determines the similarity between the probability matrix and the probability standard matrix, and generates a labeled matrix based on the similarity. The labeled matrix is ​​then type-labeled and used to update the client cluster information to obtain an updated cluster. This balances security and flexibility, enhances the system's ability to identify abnormal clients, and effectively improves the accuracy of client clustering and the robustness of the aggregation strategy during federated learning. All updated clusters are aggregated to obtain the final defense model. By dynamically detecting and controlling the difference between client updates and the global model, it intelligently scales abnormal updates to prevent abnormal or malicious clients from negatively impacting model training, effectively improving the robustness and security of the aggregation process. Combined with a weighted aggregation mechanism, this ensures that trusted client updates are fully effective while suppressing potential attacks and abnormal behavior, effectively improving the efficiency and coverage of defense against Byzantine attacks. Finally, the final defense model output is fed back to the user client. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablet computers, and portable wearable devices. The server can be implemented as an independent server or a server cluster consisting of multiple servers. The present invention is described in detail below through specific embodiments.

[0020] The following explains the present invention's specification. By applying the final update value obtained through weighted aggregation to the current global model, the invention performs an iterative optimization to update and adjust model parameters. This final update value, as the effective gradient or parameter difference of the collective contribution of trusted clients in this round, is superimposed on the current global model to complete a new round of model updates. This not only incorporates valuable information from highly trusted clients but also effectively suppresses interference from abnormal or malicious updates, thereby improving the stability and robustness of the global model and effectively enhancing the defense efficiency and coverage of defense scenarios against Byzantine attacks.

[0021] Reference Figure 2 FIG2 is a flow chart of a Byzantine attack defense method based on federated learning provided by one embodiment of the present invention. In this embodiment, the Byzantine attack defense method based on federated learning includes: S1. Obtain local models and sample data sets of several target clients, use the local model to forward propagate the sample data sets to obtain an output probability vector for each target client, and construct several probability matrices based on the output probability vector and the sample data sets.

[0022] In an embodiment of the present invention, the local model of each target client is applied to a unified sample data set, and forward propagation is performed in sequence to obtain the original output vector corresponding to each sample. These original output vectors are normalized and converted into output probability vectors representing the category distribution. A probability matrix corresponding to each client is constructed, thereby characterizing its prediction behavior characteristics for the same data set.

[0023] In specific healthcare scenarios, this approach can be applied to building personalized disease diagnosis models within a federated learning framework. While protecting patient privacy, each hospital uses a local model to perform forward inference on uniformly constructed standardized clinical samples (such as images or medical records), generating predicted probability vectors for the patient samples. This vector then constructs a probability matrix to reflect the response patterns of models from different institutions to the same case. Based on these probability matrices, the server can evaluate the consistency of model behavior and identify nodes with anomalous behavior (such as training data bias or model configuration anomalies), thereby improving the robustness of model aggregation and the accuracy of clinical diagnoses.

[0024] In specific FinTech scenarios, this can be used to build cross-institutional risk assessment models. For example, each financial institution's local model predicts a standardized set of credit application samples (including user profiles and transaction behaviors), outputting a probability vector for default or fraud. A probability matrix is ​​then constructed to reflect the risk assessment behavior of different institutional models. The platform can use this matrix to identify participants with potentially anomalous strategies (such as overly aggressive or conservative risk control models), enabling dynamic adjustments when aggregating models, thereby improving the robustness and risk management capabilities of the overall credit model.

[0025] Figure 3 A flowchart illustrating the process of constructing a probability matrix in a Byzantine attack defense method based on federated learning provided in one embodiment of the present invention.

[0026] In an embodiment of the present invention, forward propagation of the sample data set using the local model to obtain an output probability vector for each target client, and constructing several probability matrices based on the output probability vector and the sample data set, including: Performing forward propagation on each sample data in the sample data set using the local model to obtain an original output vector of each sample data; Normalizing the original output vector to obtain an output probability vector; The output probability vectors are stacked in the order of the sample data in the sample data set to obtain a probability matrix for each sample data in the sample data set.

[0027] In detail, each target client uses its locally trained model to perform forward propagation operations on each sample in the pre-distributed unified sample dataset one by one. That is, each input sample first passes through the input layer of the model, and then passes through multiple hidden layers (such as convolutional layers, fully connected layers, activation functions, normalization layers, etc.) in sequence, and finally reaches the output layer to generate an original output vector (usually logits), which represents the model's unnormalized score or activation value for each category, reflecting the local model's initial judgment on each sample.

[0028] The unnormalized scores (logits) output by the model are converted into an interpretable probability distribution. The softmax function is applied to the original output vector. Each score is exponentially normalized by the sum so that all values ​​become positive and the sum is 1. This gives an output probability vector representing the predicted probability of each category, reflecting the model's relative confidence in different categories, which facilitates subsequent probability analysis or model behavior comparison.

[0029] After completing the forward propagation and probability normalization of each sample, the output probability vectors corresponding to each sample are combined row by row according to the order of sample arrangement in the original data set, and finally a two-dimensional matrix is ​​formed. Each row of the matrix corresponds to a sample, and each column corresponds to a category. The overall matrix reflects the predicted probability distribution of the client model for the entire sample data set, which is the output probability matrix of the client.

[0030] By forward-propagating each client's local model using a unified sample dataset and normalizing the output into a probability vector, a probability matrix is ​​constructed. This allows for a comprehensive characterization of each client's model's response to the same input without leaking model parameters or raw data. This probability matrix can be used to assess prediction consistency across clients and identify potentially abnormal or malicious models, enabling more robust model aggregation and defense mechanisms, improving the security and overall performance of the federated learning system.

[0031] S2. Determine the Euclidean distance between any two probability matrices, and construct a distance matrix based on the Euclidean distance.

[0032] In this embodiment of the present invention, two probability matrices are randomly selected from all clients as target matrices. The corresponding elements of the target matrices are subtracted one by one to obtain a difference matrix. The square root of all elements in the difference matrix is ​​then summed and taken to obtain the Euclidean distance between the two probability matrices. This process is repeated for all pairs of client matrices, ultimately constructing a symmetric distance matrix in which each element represents the degree of difference in the output of the two client models on the same sample dataset.

[0033] In specific healthcare scenarios, each hospital uses its own local diagnostic model to make predictions based on standardized pathology images or structured case data, without sharing private patient data. This model then constructs an output probability matrix reflecting the model's judgment bias across disease categories. By calculating the Euclidean distance between the probability matrices output by each hospital's model, the central server can identify institutions with abnormal prediction behavior (such as bias in model training data or potential attacks) and eliminate abnormal nodes during model aggregation, thereby improving the security and reliability of the multi-institutional joint diagnostic system.

[0034] In specific FinTech scenarios, multiple financial institutions utilize local credit risk control models to assess risk on standardized application samples, generating their own output probability matrices representing the distribution of perceived risk levels for default, fraud, and other factors. The platform calculates the Euclidean distance between each institution's probability matrices to measure the similarity of model predictions and identify strategic anomalies or points of model manipulation risk (e.g., overlending or malicious false positives). This helps optimize model fusion strategies in joint modeling, mitigate single-point risks, and enhance the stability and credibility of cross-institutional risk control systems.

[0035] In an embodiment of the present invention, determining the Euclidean distance between any two probability matrices and constructing a distance matrix according to the Euclidean distance includes: Randomly selecting two of the probability matrices as two target matrices; Subtracting the two target matrices to obtain a subtraction result, and square-summing the subtraction result to obtain a summation result; Performing square root processing on the summation result to obtain the Euclidean distance between the two target matrices; All said Euclidean distances are summarized into a distance matrix.

[0036] Specifically, two target matrices are randomly selected from the output probability matrices of all clients as the target matrices. The difference is obtained by element-by-element subtraction, which represents the difference in predictions between the two clients on the corresponding samples. The square root of all elements in the difference is summed and taken to calculate the Euclidean distance between the two target matrices, reflecting the similarity of their overall output behaviors. The formula for calculating the Euclidean distance is as follows:

[0037] in, Indicates the The output probability vector of the first target matrix, Indicates the The output probability vector of the second target matrix, represents the vector dimension, Represents the Euclidean distance between the first target matrix and the second target matrix.

[0038] By repeating this calculation process and traversing all client matrix pairs, a symmetric distance matrix is ​​finally generated, in which each element represents the predicted distance between any two clients, providing a quantitative basis for subsequent clustering analysis and anomaly identification.

[0039] By calculating the Euclidean distance between any two probability matrices and constructing a distance matrix, we can intuitively quantify the differences in the outputs of different client models on a unified sample dataset, revealing the similarities or differences in their predictive behaviors. This approach not only provides an objective numerical basis for subsequent cluster analysis and anomaly detection, but also helps effectively identify anomalous or malicious clients, improving the robustness and security of models in federated learning, and ensuring the reliability and accuracy of global model updates.

[0040] S3. Perform hierarchical clustering on the target clients according to the distance matrix to obtain several client clusters.

[0041] In this embodiment of the present invention, the distance matrix corresponding to each target client is initialized as an independent cluster. Two clusters are then randomly selected for merging. The intra-cluster squared error before and after merging is calculated, and a cost function is constructed based on the change in error to estimate the incremental merging cost. By comparing the merging costs of all possible cluster pairs, the cluster pair with the lowest cost is selected for merging, gradually forming a new merged cluster. The merging process is iterative, repeating the random selection and merging operations until the preset number of clusters is reached. These merged clusters are then used as client clusters, achieving hierarchical clustering based on the distance matrix.

[0042] In healthcare scenarios, hierarchical clustering methods based on Euclidean distance can be used to analyze the similarity of patient diagnostic model outputs across multiple medical institutions. By constructing a probability matrix distance between each institution's models on a unified case sample, similar models can be clustered together, helping to identify nodes with abnormal performance or potential attacks, ensuring secure model aggregation and clinical diagnostic accuracy in federated learning.

[0043] In specific FinTech scenarios, this can be applied to behavioral pattern analysis in cross-institutional risk assessment models. Financial institutions perform hierarchical clustering of model outputs for a unified credit sample using a distance matrix to identify groups of models with abnormal risk assessments. This helps the platform effectively screen out abnormal nodes and improve the robustness and anti-fraud capabilities of the joint risk control model.

[0044] In an embodiment of the present invention, hierarchical clustering is performed on the target client according to the distance matrix to obtain a plurality of client clusters, including: Using the distance matrix of each target client as an initial cluster; Randomly selecting two of the initial clusters as a first target cluster and a second target cluster, and using the first target cluster and the second target cluster as a target cluster pair; Merging the target cluster pairs to obtain a merged cluster; Determining intra-cluster squared errors of the first target cluster, the second target cluster, and the merged cluster respectively; constructing a cost function based on the intra-cluster squared error; Determining a merging cost increment of the target cluster pair using the cost function; Comparing the merge cost increments of each group of target cluster pairs, and selecting the target cluster pair corresponding to the minimum merge cost increment according to the comparison result; Merge the selected target cluster pairs to obtain a merged update cluster; Returning the merged update cluster to the step of randomly selecting two initial clusters as a first target cluster and a second target cluster, and using the first target cluster and the second target cluster as a target cluster pair, until a preset number of merged update clusters exist, then stopping the return; The target clients corresponding to the distance matrix in the final merged updated cluster are obtained, and the final target clients are summarized to obtain a client cluster.

[0045] Specifically, the distance matrix corresponding to each target client is initialized as an independent initial cluster. Two initial clusters are randomly selected as the first target cluster and the second target cluster to form a cluster pair to be merged. The pair of target clusters are merged to form a new merged cluster, thereby gradually realizing the aggregation of client clusters and the construction of a hierarchical structure.

[0046] First, find the center points of the first and second target clusters. Then, calculate the sum of the squared Euclidean distances between all samples in each cluster and the center point to obtain the respective intra-cluster squared errors. Merge the two clusters to form a new cluster. Calculate the center point of the merged cluster and calculate the sum of the squared distances between all merged samples and the new center point to obtain the intra-cluster squared error of the merged cluster. By comparing the difference between the squared error of the merged cluster and the sum of the squared errors of the two original clusters, a cost function is constructed. The cost function reflects the impact of the merge operation on the density of the cluster. The smaller the cost, the less the density of the data in the merged cluster decreases, thus guiding the selection of the optimal cluster merging solution. The cost function calculation formula is as follows:

[0047]

[0048]

[0049] in, represents the first target cluster, represents the second target cluster, represents the merged clusters, represents the cost function, represents the intra-cluster squared error of the merged cluster, represents the intra-cluster squared error of the first target cluster, represents the intra-cluster squared error of the second target cluster, Represents the target cluster The intra-cluster squared error, Represents the target cluster The distance matrix, Represents the target cluster The mean of the distance matrix.

[0050] The change in intra-cluster squared error before and after merging each pair of clusters is calculated. This is the difference between the intra-cluster squared error after merging and the sum of the squared errors of the two clusters before merging. This difference is the incremental merging cost for the target cluster pair. The incremental merging costs of all target cluster pairs are compared one by one. The smallest incremental merging cost is found through sorting or traversal. This indicates that this pair has the smallest decrease in intra-cluster compactness after merging, and is therefore selected as the optimal merging cluster pair in this clustering round. This ensures that the clustering process gradually merges clusters at the minimum cost, improving overall clustering quality.

[0051] The filtered target cluster pairs are merged to form a new merged update cluster. The system randomly selects two merged update clusters as the new first target cluster and the second target cluster to form a new target cluster pair. The system continues to calculate the merge cost and select the optimal merge pair. This process is repeated and the cluster pairs are merged continuously until the number of merged update clusters reaches the preset target number. The merging is stopped and the clients corresponding to these merged update clusters that have reached the preset number are used as the final cluster division results to complete the hierarchical clustering process.

[0052] Through hierarchical clustering based on a distance matrix, target clients with similar behaviors are effectively aggregated into several clusters. Using the intra-cluster squared error as a metric for merging costs, each merge ensures that the increase in intra-cluster variance is minimized, thereby maintaining cluster compactness and consistency. By iteratively comparing merge costs and selecting the optimal merge pair, the degradation of clustering quality caused by random merges is avoided, achieving high-quality client grouping. This data-driven adaptive clustering approach not only accurately identifies behavioral differences between clients but also provides a solid foundation for subsequent anomaly detection, Byzantine attack defense, and federated learning aggregation strategies, improving the robustness and security of the overall system.

[0053] S4. Obtain a standard model of each target client and a probability standard matrix of the standard model, determine the similarity between the probability matrix and the probability standard matrix one by one, and generate a matrix to be labeled according to the similarity.

[0054] In this embodiment of the present invention, the similarity between the probability matrix of each target client and the standard matrix is ​​calculated, such as using cosine similarity, the inverse of the Euclidean distance, or other similarity metrics. The calculated result is compared with a set similarity threshold to perform a preliminary classification of the client model behavior, laying the foundation for subsequent anomaly detection, labeling, or removal operations.

[0055] In specific healthcare scenarios, it can be used to assess whether individualized diagnostic models uploaded by different hospitals or devices are consistent with authoritative medical models. For example, under the federated learning framework, multiple hospitals train models based on local patient data and return output probability matrices. The system then compares these matrices with the probability standard matrices generated by the national clinical diagnostic standard model for similarity. When a hospital model differs significantly from the standard, it is identified as a matrix to be labeled that meets the preset second condition, indicating that the model may have data bias, training anomalies, or system configuration problems, thereby guiding further manual review or adjustment to ensure the clinical consistency and safety of the diagnostic model nationwide.

[0056] In specific FinTech scenarios, this can be used for compliance assessment and fraud detection of risk control models. Multiple financial institutions or branches train credit scoring models based on local credit user behavior data. By calculating the similarity between the output probability matrix and the output of the credit scoring standard model set by the regulator or headquarters, models that deviate significantly from the standard are screened as candidate matrices that meet the pre-set second condition and are then flagged for review. This mechanism helps promptly identify risk scoring deviations caused by model overfitting, malicious tampering, or data quality issues, thereby improving the transparency and stability of risk control across the entire financial system.

[0057] In an embodiment of the present invention, determining the similarities between the probability matrix and the probability standard matrix one by one, and generating a matrix to be labeled according to the similarities, includes: Determining the similarity between the probability matrix of each target client and the probability standard matrix one by one; Determining whether the similarity is greater than a preset similarity threshold; If the similarity is greater than the similarity threshold, the probability matrix corresponding to the similarity greater than the similarity threshold is used as a matrix to be marked that meets the preset first condition; If the similarity is less than or equal to the similarity threshold, the probability matrix corresponding to the similarity less than or equal to the similarity threshold is used as the matrix to be marked that meets the preset second condition.

[0058] Specifically, the output probability matrix of each client on the unified auxiliary sample set is extracted, and the output probability matrix is ​​matched with the probability standard matrix generated by the standard model on the same sample set. By calculating the similarity index between the two matrices (such as cosine similarity, Euclidean distance or KL divergence, etc.), the closeness of the output results of the client model and the standard model is quantified, thereby providing a numerical basis for subsequent model screening or anomaly identification.

[0059] After calculating the similarity between each target client's probability matrix and the standard probability matrix, the system compares the similarity results with a preset similarity threshold to determine whether the client model's behavior meets the standard. If a client's similarity is higher than the threshold, it indicates that the output is relatively consistent with the standard model and has a high degree of credibility. The system then classifies the corresponding probability matrix as a matrix to be labeled, meeting the first preset condition. If the similarity is lower than or equal to the threshold, indicating that the behavior may be abnormal or deviated, the corresponding probability matrix is ​​then classified as a matrix to be labeled, meeting the second preset condition. This threshold discrimination mechanism allows for a preliminary screening of the reliability of the client model output.

[0060] By calculating the similarity between each target client's probability matrix and the standard probability matrix one by one, and combining them with the set similarity threshold for classification, the automatic identification and classification of client model behavior is effectively achieved. By classifying matrices with high similarity into the matrix to be marked that meets the preset first condition, the system can quickly locate trusted clients that are highly consistent with the standard model; while classifying matrices with low similarity into the matrix to be marked that meets the preset second condition makes it easier to focus on models that may have anomalies, distortions, or malicious behavior, improving the efficiency and accuracy of model screening, helping to ensure the stability, security, and controllability of the system's overall decision-making. It is particularly suitable for the dynamic monitoring and management of client model quality in multi-party collaborative environments such as federated learning.

[0061] S5. Mark the type of the matrix to be marked, and use the marked matrix to update the information of the client cluster to obtain an updated cluster.

[0062] In this embodiment of the present invention, clients are tagged by type using a similarity threshold. Clients above the threshold are marked as benign, and those below or equal to the threshold are marked as malicious. This tagging result is then fed back to each client cluster. Based on the proportion of malicious clients in each cluster, a determination is made as to whether a preset threshold has been reached, and then whether the cluster should be removed, updated, or retained. This enables dynamic screening and optimization of client clusters, helping to improve aggregation reliability and overall system security during federated learning.

[0063] In specific healthcare scenarios, this system can be used to control the quality of disease prediction models uploaded by hospitals or medical institutions within the federated learning framework. By comparing the predicted probabilities of each institution's model on a unified auxiliary sample with the nationally authoritative standard model, the system determines the credibility of the model output based on the similarity results. Models that deviate significantly from the standard are flagged as anomalies, and clusters of hospitals with potential data quality or modeling issues are identified at the cluster level. Untrustworthy models can then be eliminated, ensuring the nationwide stability and medical safety of the federated model.

[0064] In specific FinTech scenarios, this system can be used to review risk control models and identify risks in multi-institutional collaborative modeling. Credit scoring models uploaded by each branch are centrally evaluated. By comparing the similarity between the scoring probabilities of the credit scoring models for a standard customer set and the output of the headquarters' standard model, the system automatically flags models that may exhibit overfitting, strategic bias, or fraudulent operations. It also dynamically assesses the overall risk level of the institutional groups hosting these models, helping to eliminate high-risk model sources, optimize the quality of cross-institutional joint modeling, and enhance the robustness and compliance of financial decision-making systems.

[0065] In an embodiment of the present invention, the step of marking the matrix to be marked by type and updating the client cluster using the marked matrix to obtain an updated cluster includes: Mark the target client corresponding to the to-be-marked matrix that meets the preset first condition as a benign client; Mark the target client corresponding to the to-be-marked matrix that meets the preset second condition as a malicious client; Matching the benign clients and the malicious clients to target clients in each client cluster, respectively, and counting the proportion of malicious clients in each client cluster; Determining whether the proportion of malicious clients is less than a preset proportion threshold; If the malicious client ratio is greater than or equal to the ratio threshold, performing credibility detection on the client cluster corresponding to the malicious client ratio greater than or equal to the ratio threshold to obtain a credibility detection result; When the credibility detection result indicates that the client cluster is untrustworthy, deleting the client cluster corresponding to the malicious client ratio greater than or equal to the ratio threshold; When the credibility detection result indicates that the client cluster is credible, reclassifying and detecting malicious clients in the client cluster to obtain a final classification result; Using the final classification result, the client cluster is updated to obtain an updated cluster; If the proportion of malicious clients is less than the proportion threshold, information of the client cluster corresponding to the proportion of malicious clients less than the proportion threshold is updated to obtain an updated cluster.

[0066] In detail, by marking the target clients corresponding to the matrix to be marked that meets the preset first condition and the matrix to be marked that meets the preset second condition, the former are identified as benign clients and the latter are identified as malicious clients. The marked client states are matched one by one with the members of each client cluster, the number of benign and malicious clients contained in each cluster is confirmed, and the proportion of malicious clients in each client cluster is counted, providing a basis for subsequent cluster credibility evaluation and processing strategies.

[0067] In the embodiment of the present invention, performing credibility detection on the client cluster corresponding to the malicious client ratio greater than or equal to the ratio threshold to obtain the credibility detection result includes: The client cluster corresponding to the malicious client ratio greater than or equal to the ratio threshold is used as the cluster to be detected; Obtaining an initial trust weight and a target weight of each target client in the cluster to be detected and a classification label of the target client; Updating the target weight according to the classification label and the initial credible weight to obtain an updated weight; Determine whether the update weight is less than a null value; If the updated weight is greater than or equal to a null value, the target client corresponding to the updated weight is determined as a benign client; If the updated weight is less than a null value, the target client corresponding to the updated weight is determined as a malicious client, and a malicious proportion is determined based on the malicious client; Determining whether the malicious ratio is less than the ratio threshold; If the malicious ratio is less than the ratio threshold, the cluster to be detected is considered to be credible as a credibility detection result; If the malicious ratio is greater than or equal to the ratio threshold, the to-be-detected cluster is regarded as untrustworthy as a credibility detection result.

[0068] Specifically, the client's trustworthiness is dynamically adjusted by combining the target client's initial trust weight, current classification label (e.g., benign or malicious), and target weight within each client cluster. Each client's initial trust weight and the target weight reference value set for the current task are obtained. The trustworthiness of the behavior is then determined based on the classification label: if the client is benign, the weight is increased to enhance its influence in model aggregation; if the client is malicious, the weight is reduced accordingly to suppress its interference with the global model. This enables adaptive weight updates based on behavior classification, contributing to a more robust and secure model aggregation process.

[0069] The client's trustworthiness is further identified by applying a threshold judgment to the updated client weight. The updated weight of each target client is determined to be less than a set null value (i.e., the lower limit of the credibility threshold). If the weight is greater than or equal to the null value, the client is considered benign. If the weight is less than the null value, the client is considered malicious. The proportion of malicious clients in the entire client cluster is calculated and compared with the proportion threshold. If the malicious proportion is less than the threshold, the cluster as a whole is trustworthy and is marked as a trusted client cluster. If the malicious proportion exceeds or equals the threshold, the cluster is considered to have a significant abnormal risk and is marked as an untrustworthy client cluster, thus achieving dynamic judgment of the client cluster's trustworthiness.

[0070] By determining the proportion of malicious clients in a client cluster, a hierarchical credibility processing strategy is further implemented. First, it is determined whether the proportion of malicious clients in a client cluster is less than a preset ratio threshold. If it is greater than or equal to the threshold, it indicates that the cluster may be at high risk and requires credibility testing. If the detection result determines that the cluster is untrustworthy, the client cluster is directly deleted to prevent it from interfering with the global model. If the detection result is trustworthy, it indicates that the abnormal behavior is controllable. The system will reclassify and detect the malicious clients in the cluster, and update the client cluster based on the final classification result to obtain an updated cluster with optimized structure. If the proportion of malicious clients is lower than the ratio threshold, the overall risk of the cluster is considered to be low, and only the information status therein is updated to generate the corresponding updated cluster, realizing hierarchical processing and dynamic optimization of client clusters, enhancing the system's responsiveness to abnormal behavior and defense efficiency.

[0071] Based on the final classification results, each client's identity attributes (e.g., benign or malicious) are confirmed, and their trustworthiness, participation weight, or status label within the cluster are adjusted accordingly. Clients re-identified as benign can have their trust restored, while confirmed malicious clients can have their influence reduced or even removed.

[0072] By classifying and labeling the labeled matrix as benign or malicious and analyzing the corresponding labeling results with client cluster members, dynamic updates and risk control of client clusters are achieved. By counting the proportion of malicious clients in each cluster and comparing it with a preset threshold, credibility checks can be performed on high-risk clusters, and untrustworthy clusters can be directly eliminated to ensure that abnormal behavior is not included in model aggregation. For clusters that remain trustworthy, further refine the classification and update the member status. For low-risk clusters, information updates are completed quickly to improve processing efficiency. Overall, it balances security and flexibility, enhances the system's ability to identify abnormal clients, and effectively improves the accuracy of client clustering and the robustness of aggregation strategies during federated learning.

[0073] S6. Aggregate all the update clusters to obtain a final defense model.

[0074] In this embodiment, by detecting differences between client updates and the global model, the impact of anomalous updates is identified and limited, ensuring the stability of model aggregation. For clients whose updates differ by more than a threshold, a scaling factor is applied to compress them, while normal updates are retained. All processed client updates are then weighted and aggregated to optimize the global model, thereby preventing anomalous behavior and improving the overall security and robustness of the federated learning system.

[0075] In healthcare scenarios, this can be used in federated learning involving multiple medical institutions to detect and regulate discrepancies in model updates uploaded by each institution. By limiting the impact of abnormal updates, model deviations caused by data anomalies or equipment failures at certain institutions can be prevented, ensuring the accuracy and stability of the global diagnostic model, thereby improving the reliability of remote collaborative diagnosis and treatment systems and patient safety.

[0076] In specific FinTech scenarios, joint risk control model training applied to multiple branches prevents risk control model deviations caused by fraud or data anomalies by monitoring and scaling model updates uploaded by abnormal branches, effectively maintaining the robustness of the overall credit scoring model, reducing financial risks, and ensuring the compliance and decision-making accuracy of financial institutions in a multi-institutional collaborative environment.

[0077] In the embodiment of the present invention, aggregating all the update clusters to obtain a final defense model includes: Obtaining a current global model, and determining a difference norm between each target client in the update cluster and the current global model; Determining whether the difference norm is greater than a preset update norm threshold; If the difference norm is greater than the update norm threshold, obtaining a scaling factor, scaling the target client corresponding to the difference norm greater than the update norm threshold using the scaling factor to obtain an updated client, and replacing the updated client into the update cluster; If the difference norm is less than or equal to the update norm threshold, keeping the update cluster unchanged; Performing weighted aggregation on all the update clusters to obtain a final update value; The current global model is iteratively optimized using the final updated value to obtain a final defense model.

[0078] Specifically, the impact of each client on the global model is assessed by measuring the difference between the current global model and each client's model update. The global model parameters for the current round are obtained, and the local model updates of the target client in each update cluster are sequentially extracted. Each client's update is compared with the global model, and the difference norm is calculated. The Euclidean norm (L2 norm) is typically used to measure the distance between the two in parameter space. A larger difference norm indicates a more significant difference between the client's update and the global model, indicating the presence of anomalies or deviations from the trend. This provides a quantitative basis for subsequent update control and anomaly suppression.

[0079] Determine whether the difference norm is greater than the update norm threshold: If the difference norm is greater than the update norm threshold, the interference of client updates that deviate too much from the global model on the overall model is suppressed. A scaling factor will be calculated, which is usually determined based on the ratio between the difference norm and the threshold. It is used to compress the client update amplitude that deviates too much from the global model to a safe range. The scaling factor is used to scale the client's model update to generate an adjusted updated client. The updated client replaces the original client update content and updates it to the client cluster to which it belongs, thereby ensuring that only client updates within a reasonable range are integrated in the aggregation stage. If the difference norm is less than or equal to the update norm threshold, the update cluster remains unchanged.

[0080] By performing weighted aggregation on the model update results of all clients in the update cluster, and integrating the trust weight or contribution of each client, a global final update value is calculated, and a corresponding aggregation weight is assigned to each client, usually based on data quality, historical performance or credibility indicators. The model updates of each client are then integrated together in a weighted average manner to ensure that high-trust clients have a greater impact on the results and that the impact of abnormal or low-trust clients is limited. The final update value serves as an important basis for global model optimization.

[0081] By applying the final update value obtained through weighted aggregation to the current global model, an iterative optimization is performed to update and adjust the model parameters. The final update value is used as the effective gradient or parameter difference contributed by the collective trusted clients in this round, and is superimposed on the current global model to complete a new round of model updates. This not only incorporates valuable information from highly trusted clients, but also effectively suppresses interference from abnormal or malicious updates, thereby improving the stability and robustness of the global model, ultimately forming a defensive optimization result.

[0082] By dynamically detecting and controlling discrepancies between client updates and the global model, intelligently scaling abnormal updates prevents abnormal or malicious clients from negatively impacting model training, effectively improving the robustness and security of the aggregation process. Combined with a weighted aggregation mechanism, this not only ensures that trusted client updates are fully effective, but also suppresses potential attacks and abnormal behavior, thereby enhancing the overall defense capabilities of the federated learning system and the stability of model performance. This ensures that the final defense model is more robust and reliable, thereby improving the defense efficiency and coverage of defense scenarios against Byzantine attacks.

[0083] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0084] like Figure 4, which is a functional module diagram of a Byzantine attack defense device based on federated learning provided by one embodiment of the present invention.

[0085] In the embodiment of the present disclosure, a Byzantine attack defense device based on federated learning is provided. The Byzantine attack defense device based on federated learning corresponds one-to-one with the Byzantine attack defense method based on federated learning in the above embodiment. Figure 4 As shown, the Byzantine attack defense device 100 based on federated learning can be installed in an electronic device. According to the functions to be implemented, the Byzantine attack defense device 100 based on federated learning includes a probability matrix construction module 101, a distance matrix construction module 102, a client clustering module 103, a similarity determination module 104, an information update module 105, and an update cluster aggregation module 106. The functional modules are described in detail as follows: A probability matrix construction module 101 is configured to obtain local models and sample data sets of several target clients, perform forward propagation on the sample data sets using the local models to obtain an output probability vector for each target client, and construct several probability matrices based on the output probability vectors and the sample data sets; a distance matrix construction module 102, configured to determine the Euclidean distance between any two probability matrices and construct a distance matrix based on the Euclidean distance; The client clustering module 103 is configured to perform hierarchical clustering on the target clients according to the distance matrix to obtain a plurality of client clusters; A similarity determination module 104 is configured to obtain a standard model of each target client and a probability standard matrix of the standard model, determine the similarity between the probability matrix and the probability standard matrix one by one, and generate a matrix to be labeled based on the similarity; The information updating module 105 is configured to mark the type of the matrix to be marked, and update the information of the client cluster using the marked matrix to obtain an updated cluster; The update cluster aggregation module 106 is configured to aggregate all the update clusters to obtain a final defense model.

[0086] In one embodiment, when the probability matrix construction module 101 performs forward propagation on the sample dataset using the local model to obtain an output probability vector for each target client and constructs a plurality of probability matrices based on the output probability vector and the sample dataset, it is configured to: Performing forward propagation on each sample data in the sample data set using the local model to obtain an original output vector of each sample data; Normalizing the original output vector to obtain an output probability vector; The output probability vectors are stacked in the order of the sample data in the sample data set to obtain a probability matrix for each sample data in the sample data set.

[0087] In one embodiment, when determining the Euclidean distance between any two probability matrices and constructing a distance matrix based on the Euclidean distance, the distance matrix construction module 102 is configured to: Randomly selecting two of the probability matrices as two target matrices; Subtracting the two target matrices to obtain a subtraction result, and square-summing the subtraction result to obtain a summation result; Performing square root processing on the summation result to obtain the Euclidean distance between the two target matrices; All said Euclidean distances are summarized into a distance matrix.

[0088] In one embodiment, when performing hierarchical clustering on the target clients according to the distance matrix to obtain a plurality of client clusters, the client clustering module 103 is configured to: Using the distance matrix of each target client as an initial cluster; Randomly selecting two of the initial clusters as a first target cluster and a second target cluster, and using the first target cluster and the second target cluster as a target cluster pair; Merging the target cluster pairs to obtain a merged cluster; Determining intra-cluster squared errors of the first target cluster, the second target cluster, and the merged cluster respectively; constructing a cost function based on the intra-cluster squared error; Determining a merging cost increment of the target cluster pair using the cost function; Comparing the merge cost increments of each group of target cluster pairs, and selecting the target cluster pair corresponding to the minimum merge cost increment according to the comparison result; Merge the selected target cluster pairs to obtain a merged update cluster; Returning the merged update cluster to the step of randomly selecting two initial clusters as a first target cluster and a second target cluster, and using the first target cluster and the second target cluster as a target cluster pair, until a preset number of merged update clusters exist, then stopping the return; The target clients corresponding to the distance matrix in the final merged updated cluster are obtained, and the final target clients are summarized to obtain a client cluster.

[0089] In one embodiment, when the information updating module 105 performs type marking on the matrix to be marked and updates the information of the client cluster using the marked matrix to obtain an updated cluster, it is configured to: Mark the target client corresponding to the to-be-marked matrix that meets the preset first condition as a benign client; Mark the target client corresponding to the to-be-marked matrix that meets the preset second condition as a malicious client; Matching the benign clients and the malicious clients to target clients in each client cluster, respectively, and counting the proportion of malicious clients in each client cluster; Determining whether the proportion of malicious clients is less than a preset proportion threshold; If the malicious client ratio is greater than or equal to the ratio threshold, performing credibility detection on the client cluster corresponding to the malicious client ratio greater than or equal to the ratio threshold to obtain a credibility detection result; When the credibility detection result indicates that the client cluster is untrustworthy, deleting the client cluster corresponding to the malicious client ratio greater than or equal to the ratio threshold; When the credibility detection result indicates that the client cluster is credible, reclassifying and detecting malicious clients in the client cluster to obtain a final classification result; Using the final classification result, the client cluster is updated to obtain an updated cluster; If the proportion of malicious clients is less than the proportion threshold, information of the client cluster corresponding to the proportion of malicious clients less than the proportion threshold is updated to obtain an updated cluster.

[0090] In one embodiment, when performing credibility detection on a client cluster corresponding to a malicious client ratio greater than or equal to the ratio threshold and obtaining a credibility detection result, the information updating module 105 is configured to: The client cluster corresponding to the malicious client ratio greater than or equal to the ratio threshold is used as the cluster to be detected; Obtaining an initial trust weight and a target weight of each target client in the cluster to be detected and a classification label of the target client; Updating the target weight according to the classification label and the initial credible weight to obtain an updated weight; Determine whether the update weight is less than a null value; If the updated weight is greater than or equal to a null value, the target client corresponding to the updated weight is determined as a benign client; If the updated weight is less than a null value, the target client corresponding to the updated weight is determined as a malicious client, and a malicious proportion is determined based on the malicious client; Determining whether the malicious ratio is less than the ratio threshold; If the malicious ratio is less than the ratio threshold, the cluster to be detected is considered to be credible as a credibility detection result; If the malicious ratio is greater than or equal to the ratio threshold, the to-be-detected cluster is regarded as untrustworthy as a credibility detection result.

[0091] In one embodiment, when the update cluster aggregation module 106 aggregates all the update clusters to obtain the final defense model, it is configured to: Obtaining a current global model, and determining a difference norm between each target client in the update cluster and the current global model; Determining whether the difference norm is greater than a preset update norm threshold; If the difference norm is greater than the update norm threshold, obtaining a scaling factor, scaling the target client corresponding to the difference norm greater than the update norm threshold using the scaling factor to obtain an updated client, and replacing the updated client into the update cluster; If the difference norm is less than or equal to the update norm threshold, keeping the update cluster unchanged; Performing weighted aggregation on all the update clusters to obtain a final update value; The current global model is iteratively optimized using the final updated value to obtain a final defense model.

[0092] In the present invention, a Byzantine attack defense device based on federated learning is provided. First, the present invention obtains local models and sample data sets of several target clients, uses the local models to forward propagate the sample data sets, obtains the output probability vector of each target client, and constructs several probability matrices based on the output probability vector and the sample data sets. It can comprehensively characterize the response behavior of each client model to the same input without leaking model parameters and original data, determine the Euclidean distance between any two probability matrices, and construct a distance matrix based on the Euclidean distance. It can intuitively quantify the output differences of different client models on the same sample data set and reveal the similarity or difference in their predicted behaviors. Then, the target clients are hierarchically clustered according to the distance matrix to obtain several client clusters. The intra-cluster squared error is used as a metric for the merging cost to ensure that each merging minimizes the increase in intra-cluster differences, thereby maintaining the compactness and consistency of the clusters. By iteratively comparing merging costs and selecting the optimal merging pair, the degradation of clustering quality caused by random merging is avoided, achieving high-quality client grouping. The standard model and the probability standard matrix of each target client are obtained. The similarity between the probability matrix and the probability standard matrix is ​​determined one by one, and a labeled matrix is ​​generated based on the similarity. The labeled matrix is ​​then type-labeled and used to update the client cluster information to obtain an updated cluster. This balances security and flexibility, enhances the system's ability to identify abnormal clients, and effectively improves the accuracy of client clustering and the robustness of the aggregation strategy during federated learning. Finally, all updated clusters are aggregated to obtain the final defense model. By dynamically detecting and controlling the difference between client updates and the global model, abnormal updates are intelligently scaled to prevent abnormal or malicious clients from negatively impacting model training, effectively improving the robustness and security of the aggregation process. Combined with a weighted aggregation mechanism, this ensures that trusted client updates are fully effective while suppressing potential attacks and abnormal behavior, effectively improving the efficiency of defense against Byzantine attacks and the coverage of defense scenarios. The specific definition of a federated learning-based Byzantine attack defense device can be found in the definition of a federated learning-based Byzantine attack defense method described above and will not be repeated here. Each module in the aforementioned federated learning-based Byzantine attack defense device can be implemented in whole or in part through software, hardware, or a combination thereof. Each of these modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0093] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5As shown. The computer device includes a processor, memory, a network interface, and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When executed by the processor, the computer program implements the functions or steps on the server side of a Byzantine attack defense method based on federated learning.

[0094] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, memory, a network interface, a display screen, and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When executed by the processor, the computer program implements the client-side functions or steps of a Byzantine attack defense method based on federated learning.

[0095] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are performed: Obtaining local models and sample data sets of several target clients, performing forward propagation on the sample data sets using the local models to obtain an output probability vector for each target client, and constructing several probability matrices based on the output probability vector and the sample data sets; Determining the Euclidean distance between any two of the probability matrices, and constructing a distance matrix based on the Euclidean distance; Performing hierarchical clustering on the target clients according to the distance matrix to obtain a plurality of client clusters; Obtaining a standard model of each target client and a probability standard matrix of the standard model, determining the similarity between the probability matrix and the probability standard matrix one by one, and generating a matrix to be labeled according to the similarity; Marking the type of the matrix to be marked, and using the marked matrix to update information of the client cluster to obtain an updated cluster; All the update clusters are aggregated to obtain the final defense model.

[0096] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and apparatuses can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the module division is merely a logical function division, and actual implementation may employ other division methods.

[0097] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0098] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0099] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0100] In some implementations of this embodiment, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that when the computer program is executed by a processor, the steps of the method described in the above embodiment are implemented.

[0101] The readable storage medium of the present invention stores a computer program, which, when executed by a processor of an electronic device, can implement: Obtaining local models and sample data sets of several target clients, performing forward propagation on the sample data sets using the local models to obtain an output probability vector for each target client, and constructing several probability matrices based on the output probability vector and the sample data sets; Determining the Euclidean distance between any two of the probability matrices, and constructing a distance matrix based on the Euclidean distance; Performing hierarchical clustering on the target clients according to the distance matrix to obtain a plurality of client clusters; Obtaining a standard model of each target client and a probability standard matrix of the standard model, determining the similarity between the probability matrix and the probability standard matrix one by one, and generating a matrix to be labeled according to the similarity; Marking the type of the matrix to be marked, and using the marked matrix to update information of the client cluster to obtain an updated cluster; All the update clusters are aggregated to obtain the final defense model.

[0102] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can be found in the relevant descriptions of the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0103] The computer-readable storage medium may also store at least one computer-executable program / instruction, such as a computer-readable instruction. Computer-readable storage media include, but are not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Computer-readable storage media may include, for example, read-only memory (ROM), a hard disk, a flash memory, etc. For example, a non-transitory computer-readable storage medium may be connected to a computing device such as a computer, and then, when the computing device executes the computer-readable instructions stored on the computer-readable storage medium, the various methods described above may be performed.

[0104] In addition, the computer device may also include (but is not limited to) a data bus, an input / output (I / O) bus, a display, and input / output devices (eg, keyboard, mouse, speaker, etc.).

[0105] The processor can communicate with external devices via an I / O bus via a wired or wireless network.

[0106] In one embodiment, the at least one computer executable instruction may also be compiled into or constitute a software product / computer program product, wherein one or more computer executable instructions are executed by a processor to perform the various functions and / or method steps in the embodiments described in the present technology.

[0107] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0108] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0109] In the embodiments provided in the present disclosure, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a portion of code, and the above-mentioned module, program segment or a portion of code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0110] It should be noted that, in this disclosure, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element limited by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0111] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

[0112] It should be noted that if software tools or components other than those of our company appear in the embodiments of this application, they are only used for illustration and do not represent actual use.

Claims

1. A Byzantine attack defense method based on federated learning, characterized in that: The method comprises: Obtaining local models and sample data sets of several target clients, performing forward propagation on the sample data sets using the local models to obtain an output probability vector for each target client, and constructing several probability matrices based on the output probability vector and the sample data sets; Determining the Euclidean distance between any two of the probability matrices, and constructing a distance matrix based on the Euclidean distance; Performing hierarchical clustering on the target clients according to the distance matrix to obtain a plurality of client clusters; Obtaining a standard model of each target client and a probability standard matrix of the standard model, determining the similarity between the probability matrix and the probability standard matrix one by one, and generating a matrix to be labeled according to the similarity; Marking the type of the matrix to be marked, and using the marked matrix to update information of the client cluster to obtain an updated cluster; All the update clusters are aggregated to obtain the final defense model.

2. The Byzantine attack defense method based on federated learning according to claim 1, characterized in that: The method of using the local model to perform forward propagation on the sample data set to obtain an output probability vector for each target client, and constructing a plurality of probability matrices based on the output probability vector and the sample data set, including: Performing forward propagation on each sample data in the sample data set using the local model to obtain an original output vector of each sample data; Normalizing the original output vector to obtain an output probability vector; The output probability vectors are stacked in the order of the sample data in the sample data set to obtain a probability matrix for each sample data in the sample data set.

3. The Byzantine attack defense method based on federated learning according to claim 1, characterized in that: Determining the Euclidean distance between any two probability matrices and constructing a distance matrix according to the Euclidean distance includes: Randomly selecting two of the probability matrices as two target matrices; Subtracting the two target matrices to obtain a subtraction result, and square-summing the subtraction result to obtain a summation result; Performing square root processing on the summation result to obtain the Euclidean distance between the two target matrices; All said Euclidean distances are summarized into a distance matrix.

4. The Byzantine attack defense method based on federated learning according to claim 1, characterized in that: The target clients are hierarchically clustered according to the distance matrix to obtain a plurality of client clusters, including: Using the distance matrix of each target client as an initial cluster; Randomly selecting two of the initial clusters as a first target cluster and a second target cluster, and using the first target cluster and the second target cluster as a target cluster pair; Merging the target cluster pairs to obtain a merged cluster; Determining intra-cluster squared errors of the first target cluster, the second target cluster, and the merged cluster respectively; constructing a cost function based on the intra-cluster squared error; Determining a merging cost increment of the target cluster pair using the cost function; Comparing the merge cost increments of each group of target cluster pairs, and selecting the target cluster pair corresponding to the minimum merge cost increment according to the comparison result; Merge the selected target cluster pairs to obtain a merged update cluster; Returning the merged update cluster to the step of randomly selecting two initial clusters as a first target cluster and a second target cluster, and using the first target cluster and the second target cluster as a target cluster pair, until a preset number of merged update clusters exist, then stopping the return; The target clients corresponding to the distance matrix in the final merged updated cluster are obtained, and the final target clients are summarized to obtain a client cluster.

5. The Byzantine attack defense method based on federated learning according to claim 1, characterized in that: The step of marking the matrix to be marked by type and updating the client cluster using the marked matrix to obtain an updated cluster includes: Mark the target client corresponding to the to-be-marked matrix that meets the preset first condition as a benign client; Mark the target client corresponding to the to-be-marked matrix that meets the preset second condition as a malicious client; Matching the benign clients and the malicious clients to target clients in each client cluster, respectively, and counting the proportion of malicious clients in each client cluster; Determining whether the proportion of malicious clients is less than a preset proportion threshold; If the malicious client ratio is greater than or equal to the ratio threshold, performing credibility detection on the client cluster corresponding to the malicious client ratio greater than or equal to the ratio threshold to obtain a credibility detection result; When the credibility detection result indicates that the client cluster is untrustworthy, deleting the client cluster corresponding to the malicious client ratio greater than or equal to the ratio threshold; When the credibility detection result indicates that the client cluster is credible, reclassifying and detecting malicious clients in the client cluster to obtain a final classification result; Using the final classification result, the client cluster is updated to obtain an updated cluster; If the proportion of malicious clients is less than the proportion threshold, information of the client cluster corresponding to the proportion of malicious clients less than the proportion threshold is updated to obtain an updated cluster.

6. The Byzantine attack defense method based on federated learning according to claim 5, characterized in that: The performing credibility detection on the client cluster corresponding to the malicious client ratio greater than or equal to the ratio threshold to obtain a credibility detection result includes: The client cluster corresponding to the malicious client ratio greater than or equal to the ratio threshold is used as the cluster to be detected; Obtaining an initial trust weight and a target weight of each target client in the cluster to be detected and a classification label of the target client; Updating the target weight according to the classification label and the initial credible weight to obtain an updated weight; Determine whether the update weight is less than a null value; If the updated weight is greater than or equal to a null value, the target client corresponding to the updated weight is determined as a benign client; If the updated weight is less than a null value, the target client corresponding to the updated weight is determined as a malicious client, and a malicious proportion is determined based on the malicious client; Determining whether the malicious ratio is less than the ratio threshold; If the malicious ratio is less than the ratio threshold, the cluster to be detected is considered to be credible as a credibility detection result; If the malicious ratio is greater than or equal to the ratio threshold, the to-be-detected cluster is regarded as untrustworthy as a credibility detection result.

7. The Byzantine attack defense method based on federated learning according to claim 1, characterized in that: Aggregating all the update clusters to obtain a final defense model includes: Obtaining a current global model, and determining a difference norm between each target client in the update cluster and the current global model; Determining whether the difference norm is greater than a preset update norm threshold; If the difference norm is greater than the update norm threshold, obtaining a scaling factor, scaling the target client corresponding to the difference norm greater than the update norm threshold using the scaling factor to obtain an updated client, and replacing the updated client into the update cluster; If the difference norm is less than or equal to the update norm threshold, keeping the updated cluster unchanged; Performing weighted aggregation on all the update clusters to obtain a final update value; The current global model is iteratively optimized using the final updated value to obtain a final defense model.

8. A Byzantine attack defense device based on federated learning, characterized in that: The device comprises: A probability matrix construction module is used to obtain local models and sample data sets of several target clients, perform forward propagation on the sample data sets using the local models to obtain an output probability vector for each target client, and construct several probability matrices based on the output probability vectors and the sample data sets; a distance matrix construction module, configured to determine the Euclidean distance between any two of the probability matrices and construct a distance matrix based on the Euclidean distance; A client clustering module, configured to perform hierarchical clustering on the target clients according to the distance matrix to obtain a plurality of client clusters; a similarity determination module, configured to obtain a standard model of each target client and a probability standard matrix of the standard model, determine the similarity between the probability matrix and the probability standard matrix one by one, and generate a matrix to be labeled based on the similarity; An information updating module, configured to mark the type of the matrix to be marked, and update the information of the client cluster using the marked matrix to obtain an updated cluster; The update cluster aggregation module is used to aggregate all the update clusters to obtain a final defense model.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute a Byzantine attack defense method based on federated learning as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the computer program implements a Byzantine attack defense method based on federated learning as described in any one of claims 1 to 7.

Images