Distributed Byzantine robust weighted aggregation method based on event triggering and gradient similarity

Abstract

The invention relates to the technical field of distributed optimization, solves the technical problems that traditional federated learning cannot cope with Byzantine attacks and resources are wasted due to a fixed communication period, and particularly relates to a distributed Byzantine robust weighted aggregation method based on event triggering and gradient similarity. An ET-BGWA algorithm which is used for updating global parameters of a server and reversely updating local parameters by aggregating parameters of nodes is introduced in a federated learning scene until a global objective function reaches a minimum value so as to obtain optimal parameters. The method shows remarkable advantages in the aspects of Byzantine defense capability and communication efficiency optimization, has the advantages of being high in anti-interference capability, low in communication cost and high in privacy security, can be more suitable for multiple fields, and particularly has important application value in scenes with high data privacy requirements and limited communication resources.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of distributed optimization, and particularly relates to a distributed Byzantine robust weighted aggregation method based on event triggering and gradient similarity. BACKGROUND

[0002] The rapid evolution of artificial intelligence computing and its breakthrough progress in various fields is increasingly becoming the main force driving human society into the intelligent era. As one of the key technologies of artificial intelligence, machine learning is committed to enabling computer systems to discover laws and patterns from data, thereby endowing them with the ability to automatically perform specific tasks. The development of machine learning has significantly improved the intelligence, adaptability and automation level of artificial intelligence applications.

[0003] With the continuous iteration of technology and the widening of application scope, federated learning, as an important direction of distributed optimization, has grown into a vibrant research hotspot. Its core lies in allowing multiple participants to share model parameters without exchanging raw data, thereby realizing collaborative modeling of multiple parties while ensuring data privacy. Compared with traditional centralized methods, federated learning has outstanding advantages in system fault tolerance and privacy protection. This training feature without the need for data to leave the local makes it have broad application potential in the fields of Internet of Things, intelligent manufacturing, financial technology and other highly sensitive fields of data security and privacy.

[0004] With the increasing demand for user data privacy protection, the drawbacks of the traditional centralized data processing mode are becoming increasingly apparent, and federated learning has become a key technology for solving data security problems due to its distributed nature. Its core mechanism is to rely on local devices to complete model training and only upload parameters to the central server for aggregation and update, thereby theoretically realizing data privacy protection. However, existing research is generally based on the premise of trusted computing nodes, which contradicts the security risks such as malicious hijacking or device failure that may exist in actual applications. Misleading parameters transmitted by such abnormal nodes (i.e. Byzantine nodes) will seriously interfere with model training. It is worth noting that existing defense schemes generally have the problem of mismatch between communication efficiency and computing resources, which is highlighted by excessive reliance on high-frequency parameter exchange or complex computational verification.

[0005] With the expansion of the scale of multi-agent systems and the increase in data complexity, although federated learning has made significant progress in user data privacy protection, further research is still needed to consider how to develop more efficient and robust distributed algorithms under the condition of multi-agent information communication to deal with Byzantine attacks and alleviate the problem of resource waste caused by fixed communication cycles. SUMMARY

[0006] In view of the deficiencies of the prior art, the application provides a distributed Byzantine robust weighted aggregation method based on event triggering and gradient similarity, which solves the technical problems that the traditional federated learning cannot cope with Byzantine attacks and resources are wasted due to fixed communication periods.

[0007] To solve the above technical problems, the application provides the following technical solutions: a distributed Byzantine robust weighted aggregation method based on event triggering and gradient similarity, which comprises the following steps:

[0008] The parameters of the ET-BGWA algorithm are initialized, and the global target function and the local target function corresponding to the server side and the node side respectively are established;

[0009] The server side receives the local model w i (t) of the node i, and calculates the difference value v a (t) between the local model w i (t) and the global model w i (0);

[0010] The gradient abnormality degree κ i (t) used to quantify the abnormality degree of the Byzantine node is calculated based on the difference value v i (t), and the aggregation weight γ i (t) used to aggregate the parameters of each node is calculated according to the gradient abnormality degree κ i (t);

[0011] The global model w a (t) used to suppress the abnormal update of the Byzantine node is obtained by weighted aggregation according to the aggregation weight γ a (t);

[0012] A dynamic event triggering strategy including a global triggering condition and a local triggering condition is established;

[0013] When the change amount ||Δw a || of the global model w a (t) exceeds the threshold value μ a (t), the updated global model w a (t) is broadcast to all nodes;

[0014] Each node receives the global model w i (t-1) broadcast by the server side, and each node performs gradient update based on local data to update the local model w i (t);

[0015] When the change amount ||Δw i || of the local model w i (t) exceeds the threshold value μ i (t) or the initial step t=1, the local model w i(t) to the server side.

[0016] Further, the parameters of the initialized ET-BGWA algorithm include: initializing the global model w a (0) and the local model w i (0), setting the learning rate η(t), the threshold μ a (t) and μ i (t).

[0017] Further, the global objective function of the server side is:

[0018]

[0019] The local objective function of the node side is:

[0020]

[0021] where w a is the global model parameter; w i is the local model parameter; is the global model parameter at the last triggered time; γ i is the dynamic weight; l(w a ; ξ) is the global loss function of node i; l(w i ; ξ k ) is the local loss function of node i; D i is the local data distribution of node i; λ is the regularization coefficient; m is the local batch size, and α is the model consistency coefficient.

[0022] Further, the calculation of the gradient anomaly degree κ i (t) adopts Gaussian bell curve normalization mapping, that is:

[0023]

[0024] where v i (t) is the Euclidean distance between the local model and the global model; ι is an adjustable scaling parameter, which maps the deviation to the interval [0, 1] to quantify the anomaly degree of the Byzantine node.

[0025] Further, the calculation formula of the aggregation weight γ i (t) is:

[0026]

[0027] whereòis a zero-prevention protection constant;

[0028] The distribution of the aggregation weight γ i (t) satisfies:

[0029]

[0030] By normalizing the denominator Ensure the sum of weights is 1, give higher weights to normal nodes, suppress the weights of Byzantine nodes, example: when κ = [0.9, 0.2, 0.1], γ = [0.75, 0.17, 0.08].

[0031] Further, the global trigger condition of the server side in the dynamic event trigger strategy is: only when the aggregation model change amount is broadcast, is the global model parameter at the last trigger time;

[0032] The local trigger condition of the node side in the dynamic event trigger strategy is: only when the local model change amount is uploaded, is the corresponding local model after the update timestamp , the threshold μ i (t), μ a (t) can be dynamically configured, such as μ i (t) = 0.01 x 0.95 t .

[0033] Further, the establishment of the dynamic event trigger strategy includes:

[0034] In the federated learning scenario, the global model parameter error e a (t) of the server side is defined, that is:

[0035]

[0036] Wherein, represents the global model at the last time of triggering the event by the server;

[0037] The local model parameter error e i (t) of node i is defined, that is:

[0038]

[0039] Wherein, represents the local model at the last time of triggering the event by the current local model w i (t);

[0040] The event trigger mechanism is designed by limiting the norm of the error, that is:

[0041] ‖e a (t)‖≤μ a (t), ‖e i (t)‖≤μ i (t), i = 1, 2,..., n

[0042] When the error ||e a (t) || of the server side exceeds the threshold μ a (t), that is:

[0043]

[0044] wherein, is the trigger time known at the last time, and is consistent at the non-trigger time.

[0045] When the error ||e i (t) || of the node i exceeds the threshold μ i (t), that is:

[0046]

[0047] wherein, is the trigger time known at the last time, and is consistent at the non-trigger time.

[0048] Further, after the trigger condition of the dynamic event trigger strategy is met, the global model w a (t) of the server side is updated by aggregating the node parameters, since the parameters of the previous nodes are all allocated by the aggregation weight γ i (t), thus:

[0049]

[0050] wherein, is the weighted average of all node parameters, and the global model w a (t-1) of the previous time is expressed as:

[0051]

[0052] The model parameters of the server side at the last trigger time can be decomposed as:

[0053]

[0054] wherein, w a (t-1) is the model parameter of the server side at time t-1; e a (t-1) is the error of the server side at time t-1; e0(t-1) is the global comprehensive error, defined as:

[0055]

[0056] Since the norm of the global comprehensive error e0(t) satisfies:

[0057]

[0058] wherein, μ i (t) is the error upper bound of node i; μ a (t) is the error upper bound of the server side; μ0(t) is the upper bound of the global comprehensive error.

[0059] Further, the dynamic event triggering strategy execution process further comprises a triggering timestamp updating rule, namely:

[0060] Node i uploads the local model w i (t) to update the timestamp and the counter k i (t) = k i (t-1) + 1; the server side broadcasts the global model w a (t) to update the timestamp and the counter k a (t) = k a (t-1) + 1; when not triggered, the counter k i (t) = k i (t-1) or k a (t) = k a (t-1).

[0061] The timestamp recording mechanism is: record the time of the k i th triggering of node i; record the time of the k a th broadcasting of the server side; only the recent triggering state needs to be stored, and the storage complexity is O(n+d).

[0062] By means of the above technical solution, the application provides a distributed Byzantine robust weighted aggregation method based on event triggering and gradient similarity, which has at least the following beneficial effects:

[0063] 1. The application introduces the ET-BGWA algorithm for updating the global parameters of the server by aggregating the parameters of each node and then updating the local parameters in reverse until the global objective function reaches the minimum value to obtain the optimal parameters, thereby suppressing the abnormal update of the Byzantine node in the parameter aggregation stage, and finally updating the global model by weighted average, thereby enhancing the robustness of the method in dealing with Byzantine attacks.

[0064] 2、The application triggers model parameter update only when the deviation of the client gradient from the global model exceeds the preset threshold by designing a dynamic event triggering strategy, and further balances communication and timeliness and efficiency by introducing a maximum silent period control mechanism, and the frequency limit of forced triggering is flexibly controlled by adjusting the period value.

[0065] 3、The method proposed in the application has significant advantages in Byzantine defense capability and communication efficiency optimization, has the advantages of strong anti-interference (especially facing Byzantine attack) capability, low communication cost, high privacy security, and can be more applicable to multiple fields, especially has important application value in scenes with high data privacy demand and limited communication resources. BRIEF DESCRIPTION OF DRAWINGS

[0066] The drawings described herein are used to provide further understanding of the present application, and constitute a part of the present application, the illustrative embodiments of the present application and the description thereof are used to explain the present application, and do not constitute improper limitation on the present application. In the drawings:

[0067] Figure 1 The flowchart of the ET-BGWA algorithm in the application is shown in the figure;

[0068] Figure 2 The framework diagram of the ET-BGWA algorithm in the application is shown in the figure;

[0069] Figure 3 The classification accuracy rate comparison chart of the ET-BGWA algorithm in the application and the other four known algorithms respectively resisting Gaussian noise attack in the IID and Non-IID data scenes is shown in the figure;

[0070] Figure 4 The classification accuracy rate comparison chart of the ET-BGWA algorithm in the application and the other four known algorithms respectively resisting constant vector attack in the IID and Non-IID data scenes is shown in the figure;

[0071] Figure 5 The classification accuracy rate comparison chart of the ET-BGWA algorithm in the application and the other four known algorithms respectively resisting gradient scaling attack in the IID and Non-IID data scenes is shown in the figure;

[0072] Figure 6 The classification accuracy rate comparison chart of the ET-BGWA algorithm in the application and the other four known algorithms respectively resisting symbol flip attack in the IID and Non-IID data scenes is shown in the figure;

[0073] Figure 7A comparison chart of classification accuracy of ET-BGWA algorithm and other four known algorithms in the IID and Non-IID scenarios of resisting zero gradient attack;

[0074] Figure 8 A comparison chart of overall classification accuracy of ET-BGWA algorithm and other four known algorithms in the IID and Non-IID scenarios;

[0075] Figure 9 A schematic diagram of the normalization abnormality degree function in the ET-BGWA algorithm. DETAILED DESCRIPTION

[0076] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below in combination with the drawings and specific embodiments. The implementation process of how to apply technical means to solve technical problems and achieve technical effects can be fully understood and implemented by the present application.

[0077] The present embodiment aims at the potential vulnerabilities after facing the Byzantine attack on a large-scale distributed cluster, and the resource waste problem caused by the fixed communication cycle in traditional federated learning. A distributed Byzantine robust weighted aggregation method based on event triggering and gradient similarity is proposed, which realizes system robustness and stability, efficient communication and client privacy security.

[0078] By introducing the ET-BGWA algorithm for updating the global parameters of the server by aggregating the parameters of each node, and then updating the local parameters in reverse until the global objective function reaches the minimum value to obtain the optimal parameters in the federated learning scenario, the method comprises the following steps:

[0079] S1, initialize the parameters of the ET-BGWA algorithm, and import the data set to be distributed for preprocessing, and establish global objective functions and local objective functions corresponding to the server side and node i respectively. As a preferred embodiment of step S1, the global objective function and the local objective function are set as follows:

[0080] The global objective function is:

[0081]

[0082] The local objective function is:

[0083]

[0084] wherein, w a is the global model parameter; w i is the local model parameter; is the global model parameter at the last triggering time; γ iis the dynamic weight (by κ i (t) is the local batch size; l(w a ; ξ) is the global loss function of node i; l(w i ; ξ k ) is the local loss function of node i; D i is the local data distribution of node i; λ is the regularization coefficient; m is the local batch size, and α is the model consistency coefficient (controls the deviation degree of the local and global models).

[0085] In the present embodiment, the parameter initialization is performed by setting the initial parameters of the global model as w a (0), the initial parameters of the local model of each node as w i (0); configuring the learning rate η(t), the node-side triggering threshold μ i (t) ≥ 0, the server-side triggering threshold μ a (t) ≥ 0; initializing the gradient normalization scaling factor ι > 0, the zero constant ò = 10 -9 ; setting the timestamp counter initial value k a (0) = k i (0) = 0 and the triggering time sequence The server saves the last triggered model of each node

[0086] Then, the imported data set to be distributed is preprocessed. The MNIST data set contains 60,000 training samples and 10,000 test samples, which are distributed in 10 categories. Each data sample is represented as a one-dimensional matrix with a length of 784, corresponding to a 28x28 pixel handwritten image. In the Non-IID scenario, a stratified random sampling strategy is used to achieve balanced data distribution. To achieve IID data distribution, the experiment first performs global randomization preprocessing on the 60,000 samples of the MNIST training set, completely disrupting the category order of the original data, eliminating the inherent bias of label arrangement, and ensuring the randomness and independence of the samples.

[0087] Subsequently, the shuffled data is evenly divided into 20 consecutive blocks (corresponding to 20 clients), each client is allocated 3,000 samples, and global randomness is used to make the class distribution of each client data approximately consistent with the global distribution (i.e. about 10% for each class). For example, client m1 may contain 592 samples of label "0", 608 samples of label "1", and 600 samples of label "9", thereby satisfying the core assumption of IID - the data of each client can be regarded as a subset sampled independently and identically from the same global distribution. In the Non-IID scenario, a heterogeneous data environment is constructed by a label-oriented allocation strategy - 10 categories of digital labels are sequentially allocated to 20 clients, ensuring that each two clients share the same digital class of training samples. For example, clients m1 and m2 share all samples of label "0", m3 and m4 share samples of label "1", and so on. This allocation can effectively simulate the typical data heterogeneity characteristics in federated learning and provide a challenging test environment for algorithm robustness evaluation. In model training, the Softmax regression algorithm is used, and the batch size is set to 32. Top-1 accuracy and loss are used as performance indicators.

[0088] S2, server-side iterative processing;

[0089] a. Model receiving and updating:

[0090] If the local model w i (t) updated by node i is received, update

[0091] Update timestamp: k i (t) = k i (t-1) + 1,

[0092] Otherwise, maintain k i (t) = k i (t-1);

[0093] b. Gradient anomaly detection:

[0094] Calculate the model deviation of each node:

[0095] Normalize the degree of abnormality:

[0096] c. Dynamic weight allocation:

[0097] Calculate the aggregation weight:

[0098] d. Robust aggregation and broadcast:

[0099] Aggregate the global model:

[0100] If broadcast global model w a (t) to all nodes, update update timestamp: k a (t) = k a (t-1) + 1,

[0101] else maintain k a (t) = k a (t-1).

[0102] S3, device end parallel processing:

[0103] a. Global model receives:

[0104] If received server broadcast global model w a (t-1), update timestamp: k a (t-1) = k a (t-2) + 1,

[0105] else maintain k a (t-1) = k a (t-2).

[0106] b. Local model update:

[0107] Calculate local gradient update: where is the local loss function gradient of node i;

[0108] c. Event triggered upload:

[0109] If or t = 1, upload local model w i (t) to the server,

[0110] and update timestamp: k i (t) = k i (t-1) + 1,

[0111] else maintain k i (t) = k i (t-1).

[0112] S4, iteration termination:

[0113] Repeat steps S2-S3 until the model converges or reaches the maximum number of iterations, output global model sequence {w a (t)} t≥1 or local model sequence {w i (t) of each node.(t)} t≥1 .

[0114] In this embodiment, in order to solve the problem of resource waste caused by fixed communication period, on the basis of experimental setting, a dynamic event triggering strategy is designed to realize, the specific implementation strategy is as follows:

[0115] In the scene of federated learning, the global model parameter error e a (t) is defined as:

[0116]

[0117] represents the difference between the current global model w a (t) and the global model w at the last time when the event is triggered, and the timestamp t is dynamically updated, if the event is not triggered at time t, then At this time, the error is: If the event is triggered at time t, then The error is forced to be zero: e a (t) = w a (t) - w a (t) = 0.

[0118] Similarly, the local model parameter error of each computing node i is defined as:

[0119]

[0120] represents the difference between the current local model w i (t) and the local model w at the last time when the event is triggered, and the timestamp t is dynamically updated, if the event is not triggered at time t, then At this time, the error is: If the event is triggered at time t, then The error is forced to be zero: e i (t) = w i (t) - w i (t) = 0.

[0121] The event triggering mechanism is designed by limiting the norm of the error, that is:

[0122] ‖e a (t)‖≤μ a (t),‖e i (t)‖≤μ i (t), i = 1, 2,..., n

[0123] where the error ||e a (t) of the global model (server side) parameters is constrained within a threshold μ a (t); the error ||e i (t) of the local model (node / device side) parameters is constrained within a threshold μ i (t). When the error ||e a (t) of the server side exceeds the threshold μ a (t), i.e.:

[0124]

[0125] where, is the known trigger time at the previous time, and the e in the error definition is consistent at the non-trigger time. At this time, the trigger condition is met, and the server updates the trigger time to The server broadcasts the global model w a (t) and resets the error e a (t) = 0. When the error ||e i (t) of node i exceeds the threshold μ i (t), i.e.:

[0126]

[0127] where, is the known trigger time at the previous time, and the e in the error definition is consistent at the non-trigger time. At this time, the trigger condition is met, and each node updates the trigger time to Send the local model w i (t) to the server and reset the error e i (t) = 0.

[0128] After the trigger condition is met, the global model w a (t) on the server side is updated by aggregating the node parameters, and since the parameters of each node in the previous time are distributed by the aggregation weight γ i (t), i.e.:

[0129]

[0130] Therefore,

[0131]

[0132] where, is the weighted average of all node parameters, then the global model w a (t-1) at the previous time is represented as:

[0133]

[0134] The model parameters of the server side at the last triggering time can be decomposed as:

[0135]

[0136] where w a (t-1) is the model parameters of the server at time t-1; e a (t-1) is the error of the server side at time t-1; e0(t-1) is the global comprehensive error, defined as:

[0137]

[0138] Since The norm of the global comprehensive error e0(t) satisfies:

[0139]

[0140] where μ i (t) is the error upper bound of node i; μ a (t) is the error upper bound of the server side; μ0(t) is the upper bound of the global comprehensive error.

[0141] In addition, in order to comprehensively evaluate the robustness of the ET-BGWA algorithm, on the basis of the experimental initialization setting in step S1, a test environment containing a non-attack control group and five typical Byzantine attacks is constructed, and the parameter configuration and code implementation of each attack mode are strictly corresponding. The specific implementation strategy is as follows:

[0142] No attack (benchmark control group): all computing nodes strictly follow the federated learning protocol, and transmit the real gradient vector calculated by the local computing to the server side in each iteration, and establish the algorithm performance benchmark.

[0143] Gaussian attack: the malicious node applies zero-mean Gaussian disturbance to the real gradient, that is: where ∈~N(0,σ 2 ,I d ) and the standard deviation is set to σ=100.0;

[0144] Constant vector attack: the gradient vector is replaced by a high amplitude constant vector, that is: where c is set to 100.0;

[0145] Sign flip attack: the sign of the gradient vector is reversed and the amplitude is enhanced, that is: where β is set to 10.0;

[0146] Zero gradient attack: the malicious node sends all-zero gradient, that is

[0147] Gradient scaling attack: hyperparameter amplification on real gradient: where θ is set to 100.

[0148] In addition, the embodiment is provided with strong convexity condition, Lipschitz continuity condition and unbiasedness and bounded variance of random gradient The global objective function and the local objective function are evaluated, specifically:

[0149] Let the global objective function f(w) satisfy the strong convexity, that is, f(w) is s strongly convex, that is, there is a positive scalar s>0 such that for any satisfies:

[0150]

[0151] where s is the strong convexity constant;

[0152] Let the local objective function f i satisfy the Lipschitz continuity condition, that is:

[0153] For each device i=1,2,...,n, the objective function is differentiable, and its gradient is Lipschitz continuous on the entire parameter space . That is, there is a constant L i >0 such that for any satisfies:

[0154]

[0155] where L i is the Lipschitz constant.

[0156] Let the random gradient satisfy the unbiasedness condition, that is: for each device i=1,2,...,n, any parameter and time step the random gradient is unbiased under the condition of given historical information F(t-1), that is:

[0157]

[0158] Further, the variance of the random gradient is upper bounded by the bounded variance , which is specifically represented as:

[0159] For each device i=1,2,...,n, the random gradient has a bounded variance under the condition of given historical information F(t-1). That is, there is a constant So that:

[0160]

[0161] Under the above four assumptions, the overall model is iterated multiple times. In this process, by combining the strong convexity, Lipschitz continuity condition and bounded variance assumption, the parameter update effect is gradually evaluated and optimized until the global objective function reaches the minimum value. Finally, the node parameters are as close as possible to the global parameters on the server, the algorithm converges and the optimal parameters are obtained.

[0162] Finally, the embodiment also uses the Softmax regression problem in machine learning to test the effectiveness of the proposed algorithm. First, the problem is described as follows:

[0163] Given the input data set {(x1, y1), (x2, y2),..., (x m , y m )}, where y i ∈{1, 2,..., k} represents the class label of sample i, and k is the number of classes. The goal of Softmax regression is to estimate the probability of each class to which the input data x i belongs, that is, the prediction probability vector of the model for the sample x i is:

[0164]

[0165] where θ = {θ1, θ2,..., θ k} is the parameter of the model, represents the weight vector of the jth class, and d is the feature dimension. θ j x i represents the inner product of the vector θ j and the sample feature vector x i . The denominator is normalized to ensure that the probability sum is 1, and each probability value is in the interval [0, 1]. The conditional probability of Softmax regression that the input sample x i belongs to class j is:

[0166]

[0167] Let be the model parameter matrix, then the loss function of Softmax regression (without regularization term) is as follows:

[0168]

[0169] where m is the number of samples, and 1(·) is the indicator function (1 if the condition is true, otherwise 0). The partial derivative of the corresponding loss function is:

[0170]

[0171] The goal of federated learning algorithm is to minimize the loss function L(θ) through distributed training, and finally obtain the global optimal parameter θ * .

[0172] In the federated learning scenario, accuracy is an ideal evaluation index due to its computational efficiency and global interpretability. It is defined as:

[0173]

[0174] Where argmax j p(y=j∣x i ;θ) represents the probability of the model predicting the sample x i as class j (corresponding to the maximum probability class in the Softmax output probability vector, i.e. the model prediction result). This index directly reflects the global generalization ability of the model in the multi-classification task, and only needs to count the number of correct predictions without transmitting the original probability distribution, which is suitable for efficient communication between clients and servers in federated learning.

[0175] In the algorithm model training process, the algorithm will try to minimize this loss function L(θ), that is, by continuously adjusting the parameter θ to make the model's prediction more accurate. In actual testing, differentiated evaluation criteria are adopted: for independent and identically distributed (IID) data, whether the model can stably converge to the limit accuracy of 90% is used as the criterion for attack resistance; while in the non-independent and identically distributed (Non-IID) scenario, whether it can reach the benchmark accuracy of 50% is used as the evaluation index of algorithm robustness. It can be seen that the accuracy of all algorithms converges faster in the IID scenario and can all maintain above 0.85. In the Non-IID scenario, the accuracy of the algorithm decreases significantly, indicating that data heterogeneity significantly affects the stability of the algorithm. The defense performance of each comparative algorithm against Byzantine attacks after 500 rounds of iteration training on the MNIST dataset in different scenarios is summarized as shown in Figure 8 .

[0176] Figure 3The classification accuracy of different algorithms against Gaussian noise attack is shown in the IID data scenario and the Non-IID data scenario. In the IID data scenario, the classification accuracy of each algorithm gradually converges with the increase of training iteration, and the overall performance is relatively stable. ET-BGWA algorithm and Geomed algorithm perform best in the whole training process, and after multiple rounds of training, their accuracy eventually stabilizes at about 0.92, showing strong anti-noise ability. RSA algorithm has a large accuracy fluctuation in the early training stage, but eventually converges to 0.90 with the progress of iteration. The accuracy of Median algorithm is slightly lower than the previous methods, and stabilizes at about 0.88. Although Krum algorithm improves rapidly in the early training stage, the final accuracy is only 0.85, which is significantly lower than other algorithms, indicating that its robustness is slightly insufficient. In the Non-IID data, the accuracy of each algorithm generally decreases, and the robustness of some algorithms is greatly affected. ET-BGWA algorithm still leads, with an ultimate accuracy of 0.82, far exceeding other methods, indicating that ET-BGWA algorithm has stronger robustness against Gaussian noise attack in the Non-IID scenario. Geomed algorithm and Median algorithm perform similarly, with final accuracies of about 0.65 and 0.60, respectively. RSA algorithm converges to 0.50 after fluctuation, which is significantly lower than in the IID scenario. Krum algorithm is most affected by Non-IID data, with an accuracy of only 0.20, showing its limitations in this scenario.

[0177] Figure 4The classification accuracy of different algorithms against the constant vector attack is shown in the IID data scenario and the Non-IID data scenario. In the IID data environment, the accuracy of each algorithm gradually converges with the increase of training rounds, and the overall performance is ideal. ET-BGWA and Geomed perform best throughout the training process, with an ultimate accuracy close to 0.92, showing strong stability. The RSA algorithm fluctuates significantly in the early training period, but still converges to 0.90. The accuracy of the Median algorithm stabilizes at 0.87-0.88, while the Krum algorithm improves rapidly in the early stage, but the final accuracy is lower than other algorithms, only 0.85, showing certain limitations. In the Non-IID data environment, the performance of each algorithm is significantly differentiated. Geomed still performs best, with an ultimate accuracy of about 0.65. ET-BGWA remains relatively stable, with an ultimate accuracy of about 0.55, slightly lower than Geomed, also showing certain robustness. RSA fluctuates greatly during training, with an accuracy of 0.45 to 0.55, and is significantly affected by the attack. The accuracy of the Median algorithm stabilizes at 0.50, while the Krum algorithm performs the worst, with an accuracy of only 0.20, indicating that it has extremely weak robustness against constant vector attacks and is difficult to adapt to non-independent and identically distributed data environments.

[0178] Figure 5 The classification accuracy of different algorithms against the constant vector attack is shown in the IID data scenario and the Non-IID data scenario. In the IID data environment, the accuracy of each algorithm gradually converges with the increase of training rounds, and the overall performance is ideal. ET-BGWA and Geomed perform best throughout the training process, with an ultimate accuracy close to 0.92, showing strong stability. The RSA algorithm fluctuates significantly in the early training period, but still converges to 0.90. The accuracy of the Median algorithm stabilizes at 0.87-0.88, while the Krum algorithm improves rapidly in the early stage, but the final accuracy is lower than other algorithms, only 0.85, showing certain limitations. In the Non-IID data environment, the performance of each algorithm is significantly differentiated. Geomed still performs best, with an ultimate accuracy of about 0.65. ET-BGWA remains relatively stable, with an ultimate accuracy of about 0.55, slightly lower than Geomed, also showing certain robustness. RSA fluctuates greatly during training, with an accuracy of 0.45 to 0.55, and is significantly affected by the attack. The accuracy of the Median algorithm stabilizes at 0.50, while the Krum algorithm performs the worst, with an accuracy of only 0.20, indicating that it has extremely weak robustness against constant vector attacks and is difficult to adapt to non-independent and identically distributed data environments.

[0179] Figure 6 The classification accuracy of different algorithms against sign-flipping attack is shown in IID and Non-IID data scenarios. In IID data environment, the accuracy of each algorithm gradually stabilizes with the increase of training iterations, and the overall performance is good. ET-BGWA algorithm experiences a short fluctuation at the beginning of training and then quickly converges, with an ultimate accuracy of 0.92, becoming the best-performing algorithm. RSA algorithm performs well at the beginning, with a rapid rise in accuracy and a stable accuracy of about 0.90, showing good adaptability. The accuracy of Geomed and Median algorithms is 0.88 and 0.87 respectively, with stable performance but slightly inferior. Krum algorithm has a low accuracy throughout the training process, with an ultimate accuracy of only 0.85, indicating its weak robustness against sign-flipping attack. After switching to Non-IID data scenario, the performance difference of each algorithm is significantly intensified. In Non-IID data environment, the performance of each algorithm varies greatly, and the robustness of some algorithms decreases significantly. Geomed algorithm ranks first with an ultimate accuracy of 0.65, showing strong anti-attack ability, followed by ET-BGWA with an accuracy of 0.55. RSA and Median algorithms have an accuracy of about 0.50, with moderate performance, while Krum algorithm performs the worst with an accuracy of only 0.20, indicating that it is almost unable to effectively deal with sign-flipping attack in non-independent and identically distributed data.

[0180] Figure 7 The classification accuracy of different algorithms against zero-gradient attack is shown in IID and Non-IID data scenarios. In IID data environment, the accuracy of each algorithm gradually converges with the increase of training iterations, and the overall performance is good. ET-BGWA algorithm performs best, with an ultimate accuracy close to 0.92 and fast convergence speed. RSA algorithm fluctuates greatly at the beginning of training, and finally stabilizes at 0.89. Median and Geomed perform similarly, with an accuracy of 0.86 to 0.87, while Krum algorithm performs relatively poorly, with an ultimate accuracy of only 0.84. In Non-IID data environment, the performance difference of each algorithm is further highlighted. Geomed performs best in this scenario, with an ultimate accuracy of about 0.60. ET-BGWA still performs well, with an ultimate accuracy of about 0.55, slightly lower than Geomed but stable. The accuracy of Median algorithm stabilizes at 0.45, RSA algorithm fluctuates between 0.30 and 0.40, and Krum algorithm performs the worst, with an accuracy of only 0.10, almost unable to resist zero-gradient attack, showing its vulnerability in non-independent and identically distributed scenarios.

[0181] In the experiment, the ET-BGWA algorithm is based on the MNIST dataset to construct multiple types of Byzantine attack scenarios (including Gaussian noise, symbol flipping, constant vector, etc.), and the experimental results show that:

[0182] Compared with the traditional method (Geomed, Median, etc.), the present application can maintain stable model accuracy under IID and non-IID data distribution, and the communication round is reduced by about 40%, which verifies the comprehensive advantages of resisting malicious attacks and improving system efficiency.

[0183] The present application calculates the Euclidean distance between the client gradient and the server historical gradient, and according to the characteristics of the Gaussian function image, it is normalized to a (inverse) proportional weight (the greater the gradient distance, the smaller the weight), so that in the parameter aggregation stage, a smaller weight is given to weighted aggregation to suppress the abnormal update of the Byzantine node, and finally the global model is updated by weighted average, which enhances the robustness of the method in dealing with Byzantine attacks. In addition, a dynamic event triggering strategy is designed, which triggers the model parameter update only when the deviation of the client gradient from the global model exceeds the preset threshold. To further balance the communication and timeliness and efficiency, a maximum silent period control mechanism is introduced, and the upper limit of the frequency of forced triggering is flexibly controlled by adjusting the period value. The method shows significant advantages in Byzantine defense capability and communication efficiency optimization, and has important application value in resource limited scenarios.

[0184] Compared with the prior art GeoMed, Krum, Median, RSA and other federated learning algorithms, the present application has better performance. Under the consideration of strong convexity condition, the present application can converge to the global optimal solution faster, and can ensure that the error bound is smaller; under the consideration of Lipschitz continuity condition and bounded variance, the present application has better stability and convergence accuracy.

[0185] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment methods can be completed by programs instructing related hardware, therefore, the present application can adopt a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer usable program codes.

[0186] Various embodiments are described herein with reference to the following items, by way of progressive implementation, each of which focuses on the differences from other embodiments, and the same or similar parts between various embodiments can be mutually referred to. For the above embodiments, since they are basically similar to the method embodiments, they are described relatively simply, and the relevant parts can be referred to the part of the method embodiments.

[0187] The above embodiments have introduced the present application in detail, and the principles and embodiments of the present application are described by applying specific examples. The above embodiment is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, according to the idea of the present application, the specific embodiments and application scope will be changed, and the above description should not be understood as a limitation of the present application.

Claims

1. A distributed Byzantine robust weighted aggregation method based on event triggering and gradient similarity, characterized in that, The method includes the following steps: Initialize the parameters of the ET-BGWA algorithm, and establish the global objective function and local objective function corresponding to the server side and the node side respectively; The server receives the local model w of node i. i After (t), calculate its relationship with the global model w. a (0) difference value v i (t); Based on the difference value v i (t) Calculate the gradient anomaly κ used to quantify the anomaly degree of Byzantine nodes. i (t), and based on the gradient anomaly degree κ i (t) Calculate the aggregation weight γ used to aggregate the parameters of each node. i (t); Based on aggregation weight γ i (t) Perform weighted aggregation to obtain a global model w used to suppress anomalous updates of Byzantine nodes. a (t); Establish a dynamic event triggering strategy that includes global triggering conditions and local triggering conditions; When the global model w a The change in (t)||Δw a ||Exceeds threshold μ a At time (t), the broadcast updated global model w a (t) to all nodes; Each node receives the global model w broadcast by the server. a (t-1), each node performs gradient updates based on local data to update the local model w. i (t); When the local model w i The change in (t)||Δw i ||Exceeds threshold μ i (t) or at the initial step t=1, upload the local model w i (t) to the server.

2. The distributed Byzantine robust weighted aggregation method according to claim 1, characterized in that, The parameters for initializing the ET-BGWA algorithm include: initializing the global model w a (0) and local model w i (0), set the learning rate η(t) and the threshold μ a (t) and μ i (t).

3. The distributed Byzantine robust weighted aggregation method according to claim 1, characterized in that, The global objective function on the server side is: The local objective function at the node end is: Among them, w a These are global model parameters; w i These are local model parameters; It is the global model parameter at the previous trigger moment; γ i It is a dynamic weight; It is the global loss function of node i; It is the local loss function of node i; D i λ is the local data distribution of node i; λ is the regularization coefficient; m is the local batch size; and α is the model consistency coefficient.

4. The distributed Byzantine robust weighted aggregation method according to claim 1, characterized in that, The gradient anomaly degree κ i The calculation of (t) uses a Gaussian bell curve normalization mapping, that is: In the formula, v i (t) represents the Euclidean distance between the local and global models; ι is an adjustable scaling parameter that maps the bias to the [0,1] interval to quantify the anomaly of the Byzantine nodes.

5. The distributed Byzantine robust weighted aggregation method according to claim 1, characterized in that, The aggregation weight γ i The formula for calculating (t) is: In the formula, ò is the zero protection constant; The aggregation weight γ i The allocation of (t) satisfies: By normalizing the denominator Ensure the total weight is 1, so that normal nodes receive higher weights, while the weights of Byzantine nodes are suppressed.

6. The distributed Byzantine robust weighted aggregation method according to claim 1, characterized in that, The global triggering condition on the server side in the dynamic event triggering strategy is as follows: Only when the aggregate model changes Time broadcast, These are the global model parameters from the previous trigger moment; The local triggering conditions at the node end in the dynamic event triggering strategy are as follows: Only the local model change Upload in time, To update timestamps The corresponding local model.

7. The distributed Byzantine robust weighted aggregation method according to claim 1 or 6, characterized in that, The establishment of the dynamic event triggering strategy includes: In a federated learning scenario, the global model parameter error e on the server side is defined. a (t), that is: in, This represents the global model at the time of the last triggered event on the server side; Define the local model parameter error e of node i i (t), that is: in, Represents the current local model w i (t) Local model at the time of the last triggering event; The event triggering mechanism is designed by limiting the norm of the error, that is: ‖e a (t)‖≤μ a (t),‖e i (t)‖≤μ i (t),i=1,2,...,n When the server-side error is ||e a (t)‖exceeds the threshold μ a When (t), that is: in, It is the known trigger time from the previous moment, and the time defined in the error definition. Consistent at non-triggered times; When the error at node i is ||e i (t)‖exceeds the threshold μ i When (t), that is: in, It is the known trigger time from the previous moment, and the time defined in the error definition. Consistent at non-triggered times.

8. The distributed Byzantine robust weighted aggregation method according to claim 7, characterized in that, After the triggering conditions of the dynamic event triggering strategy are met, the global model w on the server side... a (t) Updates parameters by aggregating node parameters, since the parameters of each node have previously been aggregated by weight γ. i (t) allocation, therefore: in, The weighted average of all node parameters is the global model w at the previous time step. a (t-1) is represented as: The server-side model parameters at the last trigger moment can be decomposed into: Among them, w a (t-1) represents the model parameters on the server side at time t-1; e a (t-1) is the error on the server side at time t-1; e0(t-1) is the global comprehensive error, defined as: because The norm of the global synthesis error e0(t) satisfies: Where, μ i (t) is the upper bound of the error at node i; μ a μ0(t) is the upper bound of the error on the server side; μ0(t) is the upper bound of the global integrated error.

9. The distributed Byzantine robust weighted aggregation method according to claim 8, characterized in that, The dynamic event triggering strategy also includes triggering timestamp update rules during execution, namely: Node i uploads local model w i (t) Update timestamp And counter k i (t)=k i (t-1)+1; Server-side broadcast global model w a (t) Update timestamp And counter k a (t)=k a (t-1)+1; Maintain counter k when not triggered i (t)=k i (t-1) or k a (t)=k a (t-1).

Images