Data aggregation method based on Byzantine fault-tolerant algorithm

Abstract

The invention relates to the field of distributed learning, in particular to a data aggregation method based on a Byzantine fault-tolerant algorithm. The method comprises the following steps: acquiring model parameters obtained by aggregating distributed learning model training scenes; if the current distributed learning model training scene is a distributed learning scene of a non-hierarchical structure, selecting mechanism aggregation model parameters through a Byzantine fault-tolerant algorithm to obtain model parameters used for updating a global model; and if the current distributed learning model training scene is a distributed learning scene of a hierarchical structure, aggregating model parameters through a Byzantine fault-tolerant algorithm selection mechanism according to preset parameters to obtain model parameters for updating a global model. According to the method, the problem that the existing Byzantine fault-tolerant algorithm cannot adapt to a distributed learning structure is solved, and the failure risk of distributed learning is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of distributed learning, and more particularly, to a data aggregation method based on a Byzantine fault-tolerant algorithm. BACKGROUND

[0002] With the continuous growth of data, the computing resources of a single machine have been unable to handle such a large amount of data.

[0003] Therefore, the concept of distributed computing is proposed. The most important concept in distributed computing is MapReduce, a programming model and related implementation that can handle various large data sets, and its main steps are the Map phase responsible for data mapping and the Reduce phase responsible for data aggregation. Usually, Reduce is run under a distributed system, which can greatly improve the computing efficiency without being limited by the computing resources of a single computer. However, in distributed computing, some nodes may be malicious, or due to node damage, or because of network errors during transmission, resulting in errors when reducing data. Such nodes are called Byzantine nodes. The concept of Byzantine nodes comes from the Byzantine General problem, which can be simply described as follows: in a system (which can be distributed), some nodes will exhibit a series of problems, such as sending incorrect information to other parts of the system.

[0004] The Byzantine General problem not only occurs in distributed computing, but also exists in many other fields today. For example, in the scenario of artificial intelligence model training, due to the need for massive machine nodes to participate in training, if there are Byzantine nodes, the risk of failure of distributed learning is also increasing. In some distributed learning scenarios based on gradient descent, the presence of Byzantine nodes greatly reduces the training results, and the accuracy cannot be improved, and the correctness of the model cannot be guaranteed. SUMMARY

[0005] The present application provides a data aggregation method based on a Byzantine fault-tolerant algorithm to overcome at least one of the deficiencies of the prior art.

[0006] The present application aims to at least partially solve the above technical problems.

[0007] To solve the above technical problems, the technical solutions of the present application are as follows: A data aggregation method based on a Byzantine fault-tolerant algorithm, comprising the following steps: Obtaining model parameters aggregated by a distributed learning model training scenario; wherein the distributed learning model training scenario includes a non-hierarchical structure distributed learning scenario and a hierarchical structure distributed learning scenario; If the current distributed learning model training scenario is a non-hierarchical structure distributed learning scenario, the model parameters are aggregated by the Byzantine fault-tolerant algorithm selection mechanism to obtain the model parameters for updating the global model. If the current distributed learning model training scenario is a hierarchical structure distributed learning scenario, the model parameters are aggregated by the Byzantine fault-tolerant algorithm selection mechanism according to the preset parameters to obtain the model parameters for updating the global model.

[0008] Further, the Byzantine fault-tolerant algorithm selection mechanism includes any one or a combination of the mean aggregation algorithm, the first aggregation algorithm, or the second aggregation algorithm.

[0009] Further, the hierarchical structure distributed learning scenario includes a center server, at least one server, and a plurality of distributed devices corresponding to each server, and at least one Byzantine node is unevenly distributed in the plurality of distributed devices.

[0010] Further, if the current distributed learning model training scenario is a hierarchical structure distributed learning scenario, the model parameters are aggregated by the Byzantine fault-tolerant algorithm selection mechanism according to the preset parameters to obtain the model parameters for updating the global model, including the following steps: The preset parameters include a first preset parameter and a second preset parameter, and the gradient vectors corresponding to the model parameters of the plurality of distributed devices corresponding to each server are aggregated by the first aggregation algorithm or the second aggregation algorithm according to the first preset parameter to obtain a plurality of aggregated gradient vectors, and the first preset parameter is used to adapt to the situation that the Byzantine nodes are concentrated in each server corresponding to the plurality of distributed devices. The plurality of aggregated gradient vectors are aggregated by the first aggregation algorithm or the second aggregation algorithm according to the second preset parameter to obtain aggregated model parameters, and the second preset parameter is used to adapt to the situation that the Byzantine nodes are evenly distributed in all servers.

[0011] Preferably, the first preset parameter is a coefficient less than a preset threshold multiplied by the proportion of Byzantine nodes, and the second preset parameter is a coefficient greater than the preset threshold multiplied by the proportion of Byzantine nodes.

[0012] Further, the non-hierarchical structure distributed learning scenario includes a center server and a plurality of distributed devices, and at least one Byzantine node is unevenly distributed in the plurality of distributed devices.

[0013] Further, if the current distributed learning model training scenario is a non-hierarchical structure distributed learning scenario, the model parameters are aggregated by the Byzantine fault-tolerant algorithm selection mechanism to obtain the model parameters for updating the global model, including the following steps: Calculate the distance between the gradient vector corresponding to the model parameters of each distributed device and the gradient vector corresponding to the model parameters of the other distributed devices, and obtain the distance set of the gradient vectors corresponding to each distributed device. Distances greater than a preset value are discarded from the distance set of gradient vectors corresponding to each distributed device, resulting in multiple first sets; The distances of the gradient vectors of each first set are aggregated using the mean aggregation algorithm to obtain the mean distances of the gradient vectors of each first set. The average distances of the gradient vectors of all first sets are aggregated using either a first aggregation algorithm or a second aggregation algorithm to obtain the gradient vector corresponding to the average distances of the gradient vectors of all first sets. The gradient vector corresponding to the average distances of the gradient vectors of all first sets is used to update the model parameters of the global model.

[0014] Furthermore, the first aggregation algorithm includes the following steps:

[0015]

[0016] in,( V 1) , ··· , ( V n ) is the first n gradient vectors, v 1 , ··· ,v d For the first d A multi-dimensional output vector T ( ) represents the first aggregation algorithm, ( V i ) j For the first j The first dimension i gradient vectors, U j For the first j The first set of each dimension.

[0017] Furthermore, the second aggregation algorithm includes the following steps: Calculate the distance between the gradient vector corresponding to the model parameters of each distributed device and the gradient vector corresponding to the model parameters of the other distributed devices; Each distributed device selects a gradient vector that fits within a preset distance range, resulting in a second set; The distributed device selects the gradient vector with the smallest distance from the second set as the corresponding model parameter.

[0018] Furthermore, calculating the distance between the gradient vector corresponding to the model parameters of each distributed device and the gradient vectors corresponding to the model parameters of the other distributed devices includes: d n-1,n =|| V n-1 -V n ||2 in, V n-1 For the first n-1 gradient vectors, V n For the first n gradient vectors, d n-1,n For the first n-1 The gradient vector and the first n The distance between gradient vectors, ||..||2 is the Euclidean distance.

[0019] Compared with the prior art, the beneficial effects of the technical solution of the present invention are: This invention first involves distributed devices acquiring model parameters sent by a central server. These distributed devices then aggregate the model parameters according to a distributed learning model training scenario. This training scenario includes both non-hierarchical and hierarchical distributed learning scenarios. If the current training scenario is non-hierarchical, the model parameters are aggregated using a Byzantine fault tolerance algorithm to obtain parameters for updating the global model. If the current training scenario is hierarchical, the model parameters are aggregated using a Byzantine fault tolerance algorithm based on preset parameters to obtain parameters for updating the global model. This invention proposes data aggregation rules for both non-hierarchical and hierarchical distributed learning scenarios, addressing the problem that existing Byzantine fault tolerance algorithms cannot adapt to the structure of distributed learning and reducing the risk of failure in distributed learning. Attached Figure Description

[0020] Figure 1 This is a flowchart of the data aggregation method based on the Byzantine fault-tolerant algorithm described in this invention; Figure 2 This is a schematic diagram of the polymerization process of the non-hierarchical structure described in this embodiment; Figure 3 This is a schematic diagram of the intra-group aggregation algorithm described in this embodiment; Figure 4 This is a schematic diagram of the inter-group aggregation algorithm described in this embodiment; Figure 5This is a schematic diagram of the hierarchical distributed learning scenario structure described in this invention. Detailed Implementation

[0021] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent. To better illustrate this embodiment, some parts in the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions; It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings.

[0022] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0023] Example 1 A data aggregation method based on the Byzantine fault-tolerant algorithm, such as Figure 1 As shown, it includes the following steps: S1: Obtain the model parameters obtained by aggregating distributed learning model training scenarios; wherein, the distributed learning model training scenarios include non-hierarchical distributed learning scenarios and hierarchical distributed learning scenarios; S2: If the current distributed learning model training scenario is a non-hierarchical distributed learning scenario, the model parameters are aggregated through the Byzantine fault tolerance algorithm selection mechanism to obtain the model parameters used to update the global model; S3: If the current distributed learning model training scenario is a hierarchical distributed learning scenario, the model parameters are aggregated according to the preset parameters through the Byzantine fault tolerance algorithm selection mechanism to obtain the model parameters used to update the global model.

[0024] In the specific implementation process, different algorithms in the Byzantine fault tolerance algorithm selection mechanism are selected to aggregate model parameters according to the non-hierarchical distributed learning scenario and the hierarchical distributed learning scenario, respectively, to obtain model parameters used to update the global model. Among them, if the current distributed learning model training scenario is a hierarchical distributed learning scenario, the model parameters are aggregated by combining the preset parameters and the Byzantine fault tolerance algorithm selection mechanism.

[0025] Example 2 This embodiment, based on Embodiment 1, continues to disclose the following content: The Byzantine fault-tolerant algorithm selection mechanism includes one or more combinations of mean aggregation algorithm, first aggregation algorithm, or second aggregation algorithm.

[0026] In the embodiment, the first aggregation algorithm is the trimmed mean algorithm or the Bulyan algorithm, the first aggregation algorithm is the Krum algorithm or the Bulyan algorithm, the Krum algorithm satisfies (α, f)-Byzantine resilience, and the principle of the algorithm is that a correct vector is closer to other correct vectors than to incorrect vectors. Therefore, it is intuitive to reduce the number of correct nodes to n-f-2 by introducing the number of Byzantine nodes.

[0027] The maximum number of Byzantine nodes that can be resisted by the algorithm is ; The trimmed mean algorithm mainly processes each dimension of the vector; The Bulyan algorithm uses an iterative method to select a benign set; The trimmed mean algorithm is used to aggregate data in the benign set to obtain the final result.

[0028] Embodiment 3 Based on embodiments 1 and 2, the following content is further disclosed: The hierarchical structure of the distributed learning scenario includes a center server, at least one server, and a plurality of distributed devices corresponding to each server, and at least one Byzantine node is unevenly distributed in the plurality of distributed devices.

[0029] In some embodiments, the hierarchical structure has the following three important prerequisites: Whether the Byzantine nodes of each group are determined; Whether the grouping can be changed; Whether the groups can communicate with each other; Based on the above problems, the following assumptions are made for the hierarchical structure: It is assumed that the total number of Byzantine nodes in the distributed system can be mastered; It is assumed that the grouping cannot be dominated, that is, the grouping is already determined; It is assumed that each group has a head node, and the head node is a non-Byzantine node; It is assumed that the groups cannot communicate with each other, that is, the grouping cannot be changed, and the number of Byzantine nodes in the group is uncertain.

[0030] From the above assumptions, the problem model can be obtained: The number of Byzantine nodes in the entire distributed system is clear; Due to the uncertainty of the grouping and the inability to re-group, the number of Byzantine nodes in each group cannot be known; In the case where the number of Byzantine nodes in the group cannot be known, since the existing Byzantine fault-tolerant algorithm requires the number of Byzantine nodes, the existing Byzantine fault-tolerant algorithm cannot be directly used; The distribution of Byzantine nodes among groups is unclear The non-hierarchical distributed learning scenario includes a central server and a plurality of distributed devices, and at least one Byzantine node is unevenly distributed among the plurality of distributed devices.

[0031] In a simplest distributed system, as shown in Figure 5 There is 1 parameter server and n worker nodes, assuming that the data generated by the normal worker nodes is independent and identically distributed data, which is assumed to be Gaussian distribution in this paper, and it is assumed that f Byzantine nodes appear among the n worker nodes.

[0032] Embodiment 4 This embodiment is based on embodiments 1, 2 and 3, and further discloses the following content: If the current distributed learning model training scenario is a non-hierarchical distributed learning scenario, the model parameters are aggregated by the Byzantine fault-tolerant algorithm selection mechanism to obtain the model parameters for updating the global model, including the following steps: The distance between the gradient vector corresponding to the model parameters of each distributed device and the gradient vector corresponding to the model parameters of the remaining distributed devices is calculated to obtain a distance set of the gradient vector corresponding to each distributed device; Discard the distances greater than the preset value in the distance set of the gradient vector corresponding to each distributed device to obtain a plurality of first sets; The distances of the gradient vectors of each first set are aggregated by a mean aggregation algorithm to obtain the mean of the distances of the gradient vectors of the first set, respectively. The mean of the distances of the gradient vectors of all first sets is aggregated by a first aggregation algorithm or a second aggregation algorithm to obtain a gradient vector corresponding to the mean of the distances of the gradient vectors of all first sets, and the gradient vector corresponding to the mean of the distances of the gradient vectors of all first sets is used to update the model parameters of the global model.

[0033] In the specific implementation process, as shown in Figure 2 The sum of the distances between each vector and the other n-f vectors closest to the vector is calculated, which is denoted as D set; According to the D set calculated in the first step, the f vectors with the largest distance are discarded; The remaining vectors in the second step are grouped and aggregated among groups and within groups.

[0034] The method mainly imitates some characteristics of the Krum algorithm and the Bulyan algorithm. The algorithm uses an intuition of the Krum algorithm that a correct vector is closer to other correct vectors than to incorrect vectors, and uses a characteristic of the Bulyan algorithm that can combine multiple aggregation rules, and further uses two different aggregation rules.

[0035] In Figure 2 lines 7 and 9, two different symbol marked aggregation rules and different numbers of Byzantine nodes are used for data aggregation, and in actual use, the two aggregation rules can be different aggregation rules or the same aggregation rule, and the number of Byzantine nodes can also be flexibly changed.

[0036] The most critical points of the algorithm are as follows: Selection of two aggregation rules; Setting of the number of Byzantine nodes required by the aggregation rule.

[0037] To solve the above problems, the following scheme is proposed: In Figure 2 lines 3 and 4, most of the Byzantine nodes have been excluded, so the aggregation rule A selects the mean aggregation algorithm for aggregation when intra-group aggregation is performed. The mean aggregation algorithm does not need to set the number of Byzantine nodes, so f' in line 7 does not need to be set. Although it is expected to exclude most of the Byzantine nodes, some Byzantine nodes will inevitably remain, so the inter-group aggregation rule B can select Krum or Trimmed Mean for aggregation, and the number of Byzantine nodes can be set relatively conservatively, slightly larger.

[0038] Since the time complexity of the algorithm is related to the selection of the aggregation rule, this section analyzes the time complexity by using the analysis results in the above, that is, the aggregation rule A is mean, and the aggregation rule B is Krum or Trimmed Mean.

[0039] In a non-hierarchical structure and when v 1 , … ,v n is d dimensional, the time complexity of the Byzantine fault-tolerant algorithm selection mechanism proposed by the application is O ( n 2 · d ) when Krum and Trimmed Mean are used.

[0040] The time complexity of this part is O ( n 2 d ) when using the fast selection algorithm consistent with the Krum algorithm, and the total iteration n times). The time complexity of lines 3-4 of the algorithm is O ( n ). Assuming that the data is divided into k groups, each group has l data, the time complexity of lines 6-8 is O ( k l ), n = k l , so the time complexity is O ( n ). The time complexity of line 9 is divided into the cases of Krum and Trimmed Mean. When it is Krum, the time complexity is O ( k 2 d ), and when it is Trimmed Mean, the time complexity is O ( k d ), and since k < n , the total time complexity is O ( n 2 d ).

[0041] Embodiment 5 This embodiment is based on embodiments 1, 2, 3 and 4, and further discloses the following content: If the current distributed learning model training scenario is a hierarchical distributed learning scenario, the model parameters are aggregated by a Byzantine fault-tolerant algorithm selection mechanism to obtain model parameters for updating the global model, including the following steps: As shown in Figure 3 , the gradient vectors corresponding to the model parameters of each server corresponding to multiple distributed devices are aggregated according to a first preset parameter through a first aggregation algorithm or a second aggregation algorithm to obtain multiple aggregated gradient vectors, the first preset parameter being used to adapt to the situation of the Byzantine nodes being distributed in each server corresponding to multiple distributed devices; As shown in Figure 4 , the multiple aggregated gradient vectors are aggregated according to a second preset parameter through a first aggregation algorithm or a second aggregation algorithm to obtain aggregated model parameters, the second preset parameter being used to adapt to the situation of the Byzantine nodes being uniformly distributed in all servers. ​​​​​​

[0042] Although the exact number of Byzantine nodes in each group is unknown, the total number of Byzantine nodes in the entire distributed system is known. f That is, the probability that a node is a Byzantine node can be calculated. Therefore, in order to apply existing Byzantine fault-tolerant algorithms, we can base them on probability. p To obtain the number of Byzantine nodes in the group.

[0043] However, based on the second point of the main problem mentioned above, if we directly use probability... p Applying existing Byzantine fault-tolerant algorithms may significantly reduce the effectiveness, because the probability of a node being a Byzantine node in different groups is highly likely to be different from the probability of... p This is not consistent. To resolve this issue, parameters can be used to adjust the probabilities. p Adjustments can be made to design different probabilities for within-group and between-group scenarios. λp , λ ′ p To perform Byzantine fault-tolerant data aggregation, intra-group aggregation and inter-group aggregation algorithms are proposed. Simultaneously, parameters can be adjusted. λ , λ Configure the parameters for the hierarchical structure as shown in Table 1.

[0044] Table 1

[0045] The first aggregation algorithm includes the following steps:

[0046]

[0047] in,( V 1) , ··· , ( V n ) is the first n gradient vectors, v 1 , ··· ,v d For the first d A multi-dimensional output vector T ( ) represents the first aggregation algorithm, ( V i ) j For the first j The first dimension i gradient vectors, U j For the first j The first set of each dimension.

[0048] The second aggregation algorithm comprises the following steps: calculating the distance between the gradient vector corresponding to the model parameters of each distributed device and the gradient vector corresponding to the model parameters of the remaining distributed devices; each distributed device selects the gradient vector that meets the preset distance range to obtain a second set; the distributed device selects the gradient vector with the smallest distance in the second set as the corresponding model parameter.

[0049] Embodiment 6 On the basis of embodiments 1, 2, 3, 4, and 5, the following content is further disclosed: the first preset parameter is a coefficient less than the threshold value multiplied by the proportion of the Byzantine node, and the second preset parameter is a coefficient greater than the threshold value multiplied by the proportion of the Byzantine node.

[0050] According to different parameters λ , λ , the proportions of the Byzantine nodes in the group and between the groups are designed, and the following two extreme cases are considered in this embodiment: Case one: the Byzantine nodes are concentrated in some groups; Case two: the Byzantine nodes are evenly distributed between the groups.

[0051] In case one, it can be predicted that the proportion of the Byzantine nodes in most groups will be less than the global proportion p . Therefore, in order to make the parameter setting more representative, it is necessary to set it for the case of most groups. Therefore, the parameter should be set a little smaller in this case, that is, λ 1. However, in order to prevent the aggregation effect of some groups from being poor, the parameter between the groups should be set a little larger at this time, that is, λ ′ > 1.

[0052] In case two, since the Byzantine nodes are distributed relatively evenly, the parameters of the Byzantine nodes in the group and between the groups can be set close to 1.

[0053] However, in actual application, the distribution of the Byzantine nodes cannot be predicted, so the worst case is assumed. In the worst case, the distribution of the Byzantine nodes in the group cannot be predicted, so the two extreme cases mentioned above need to be considered comprehensively. On the basis of the first extreme case, if the parameter setting of the first extreme case is applied to the second extreme case, the data aggregation in the group may lead to poor data aggregation effect in the group due to the smaller parameter than the actual proportion of the Byzantine nodes in the group. In this case, it is reasonable to set a larger parameter between the groups. Therefore, the parameter setting of the first extreme case can be applied to the second extreme case.

[0054] In summary, the parameters within the group can be set a little smaller, and the parameters between the groups can be set a little larger, that is λ 1、 λ ′ > 1。

[0055] The time complexity of the method model under the hierarchical structure is also related to the selection mechanism of the Byzantine fault-tolerant algorithm. The time complexity of the intra-group aggregation algorithm and the inter-group aggregation algorithm is analyzed respectively.

[0056] Suppose there are g groups, and the data v 1 , ··· ,v n is d dimensional. When the intra-group aggregation algorithm is Krum, the time complexity of the intra-group aggregation algorithm is O ( g 2 · d ), and when the intra-group aggregation algorithm is TrimmedMean, the time complexity of the intra-group aggregation algorithm is O(g · d); Similarly, the time complexity of the inter-group aggregation algorithm is O ( l 2 · d ) when the aggregation rule is Krum, and O ( l · d ) when the aggregation rule is TrimmedMean, where l is the number of intra-group nodes.

[0057] The distance between the gradient vector corresponding to the model parameter of each distributed device and the gradient vector corresponding to the model parameter of the remaining distributed device is calculated, including: d n-1,n =|| V n-1 -V n ||2 wherein, V n-1 is the n-1 gradient vector, V n is the n gradient vector, d n-1,n is the distance between the n-1 gradient vector and the n gradient vector, and ||..||2 is the Euclidean distance.

[0058] The same or similar reference numerals mean the same or similar components; The terms describing the positional relationship in the drawings are used only for illustrative purposes and should not be construed as limiting the present patent; Obviously, the above-mentioned embodiments of the present application are merely exemplary and are not intended to limit the implementation of the present application. Based on the above description, other different forms of changes or variations can be made by those skilled in the art. Here, it is not necessary or possible to exhaust all the implementations. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application should be included in the protection scope of the claims of the present application.

Claims

1. A data aggregation method based on a Byzantine fault-tolerant algorithm, characterized in that, The method comprises the following steps: obtaining model parameters aggregated by a distributed learning model training scenario, wherein the distributed learning model training scenario comprises a non-hierarchical distributed learning scenario and a hierarchical distributed learning scenario; if the current distributed learning model training scenario is the non-hierarchical distributed learning scenario, aggregating the model parameters by a Byzantine fault-tolerant algorithm selection mechanism to obtain model parameters for updating a global model; if the current distributed learning model training scenario is the hierarchical distributed learning scenario, aggregating the model parameters by the Byzantine fault-tolerant algorithm selection mechanism according to preset parameters to obtain model parameters for updating the global model.

2. The data aggregation method based on the Byzantine fault-tolerant algorithm according to claim 1, wherein the Byzantine fault-tolerant algorithm selection mechanism comprises any one or a combination of multiple of a mean aggregation algorithm, a first aggregation algorithm or a second aggregation algorithm.

3. The data aggregation method based on the Byzantine fault-tolerant algorithm according to claim 2, wherein the hierarchical distributed learning scenario comprises a center server, at least one server and a plurality of distributed devices corresponding to each server, and at least one Byzantine node is unevenly distributed in the plurality of distributed devices.

4. The data aggregation method based on the Byzantine fault-tolerant algorithm according to claim 3, wherein if the current distributed learning model training scenario is the hierarchical distributed learning scenario, aggregating the model parameters by the Byzantine fault-tolerant algorithm selection mechanism according to preset parameters to obtain model parameters for updating the global model comprises the following steps: the preset parameters comprise a first preset parameter and a second preset parameter, and a plurality of aggregated gradient vectors are obtained by aggregating gradient vectors corresponding to model parameters of the plurality of distributed devices corresponding to each server according to the first preset parameter by the first aggregation algorithm or the second aggregation algorithm, wherein the first preset parameter is used to adapt to the situation that Byzantine nodes are concentrated in each server corresponding to the plurality of distributed devices; aggregating the plurality of aggregated gradient vectors by the first aggregation algorithm or the second aggregation algorithm according to the second preset parameter to obtain aggregated model parameters, wherein the second preset parameter is used to adapt to the situation that Byzantine nodes are evenly distributed in all servers.

5. The data aggregation method based on the Byzantine fault-tolerant algorithm according to claim 4, wherein the first preset parameter is a coefficient smaller than a preset threshold multiplied by a Byzantine node ratio, and the second preset parameter is a coefficient larger than the preset threshold multiplied by the Byzantine node ratio.

6. The data aggregation method based on the Byzantine fault-tolerant algorithm according to claim 2, wherein the non-hierarchical distributed learning scenario comprises a center server and a plurality of distributed devices, and at least one Byzantine node is unevenly distributed in the plurality of distributed devices.

7. The data aggregation method based on the Byzantine fault-tolerant algorithm according to claim 6, wherein If the current distributed learning model training scene is a non-hierarchical structure distributed learning scene, the model parameters are aggregated by a Byzantine fault-tolerant algorithm selection mechanism to obtain model parameters for updating the global model, including the following steps: The distance between the gradient vector corresponding to the model parameters of each distributed device and the gradient vector corresponding to the model parameters of the remaining distributed devices is calculated to obtain a distance set of the gradient vector corresponding to each distributed device; In the distance set of the gradient vector corresponding to each distributed device, distances greater than a preset value are discarded to obtain a plurality of first sets; The distances of the gradient vectors in each first set are aggregated by a mean aggregation algorithm to obtain the mean of the distances of the gradient vectors in the first set; The means of the distances of the gradient vectors in all first sets are aggregated by a first aggregation algorithm or a second aggregation algorithm to obtain a gradient vector corresponding to the mean of the distances of the gradient vectors in all first sets, and the gradient vector corresponding to the mean of the distances of the gradient vectors in all first sets is used to update the model parameters of the global model.

8. The data aggregation method based on the Byzantine fault-tolerant algorithm according to claim 2, characterized in that, the first aggregation algorithm includes the following steps: in,( V 1) , ··· , ( V n ) is the first n gradient vectors, v 1 , ··· ,v d For the first d A multi-dimensional output vector T ( ) represents the first aggregation algorithm, ( V i ) j For the first j The first dimension i gradient vectors, U j For the first j The first set of each dimension.

9. The data aggregation method based on the Byzantine fault-tolerant algorithm according to claim 2, characterized in that, the second aggregation algorithm includes the following steps: The distance between the gradient vector corresponding to the model parameters of each distributed device and the gradient vector corresponding to the model parameters of the remaining distributed devices is calculated. Each distributed device selects a gradient vector that meets a preset distance range to obtain a second set. The distributed device selects the gradient vector with the smallest distance in the second set as the corresponding model parameters. 10.The data aggregation method based on a Byzantine fault tolerance algorithm according to claim 8, characterized in that, The distance between the gradient vector corresponding to the model parameters of each distributed device and the gradient vector corresponding to the model parameters of the remaining distributed devices is calculated, including: d n-1,n =|| V n-1 -V n ||2 wherein V n-1 is the first gradient vector, n-1 V n is the first gradient vector, n d n-1,n is the distance between the first gradient vector and the second gradient vector, n-1 n is the Euclidean distance.​​​

Images