Raft election strategy-based stable federal learning method and related equipment

Abstract

The invention provides a Raft election strategy-based stable federated learning method and related equipment, and relates to the field of federated learning. The method comprises the following steps: dividing all equipment participating in federated learning into a plurality of clusters, and electing proxy equipment in each cluster based on a Raft algorithm; judging whether the proxy equipment is online or not, if yes, continuing training, and if not, electing the proxy equipment again based on the Raft algorithm; the server issues the global model obtained by the last round of aggregation to the proxy device; the proxy device distributes the received global model to each device in the cluster, and each device in the cluster performs local training and uploads a local training result to the proxy device; the proxy device aggregates local training results in the cluster and uploads a cluster model obtained by aggregation to a server; the server aggregates the cluster models of the clusters to generate a global model; and repeating the training process until a stop condition is reached. The method can effectively improve the stability and efficiency of the federal learning process.

Description

Technical Field

[0001] The present invention relates to the field of federated learning technology, and in particular to a stable federated learning method based on a Raft election strategy and related equipment. Background Art

[0002] From smartphones to industrial sensors, various terminal devices (clients) continuously generate a large amount of data. This data contains rich information such as user behavior, device status, and environmental conditions, and has become a key resource for promoting technological innovation and business development. However, as the value of data becomes increasingly prominent, data security and privacy issues have also attracted attention. In recent years, data breaches have occurred frequently, which not only damages the privacy of users, but also has a significant impact on the reputation and operations of enterprises. How to protect the security and privacy of data has become an urgent problem that needs to be solved immediately.

[0003] Federated Learning (FL), as an emerging distributed machine learning paradigm, provides a possible solution to this problem. Specifically, federated learning allows multiple devices to jointly train a model without centralizing the data on a server. This means that the data of each device can remain local and does not need to be shared with other devices, thereby reducing the risk of data leakage. Although federated learning has great potential in data privacy protection, there are still major challenges in implementing an efficient and robust FL system. First, the devices involved in training may have different computing capabilities. Second, the network environment and device status are dynamically changing. The above factors will cause instability in the devices involved in training, which in turn affects the stability and efficiency of FL training.

[0004] There have been many studies on improving the training efficiency and stability in federated learning. For example, Xiong et al. proposed a method in the paper "Feddm: Iterative Distribution Matching for Communication-Efficient Federated Learning" to perform distribution matching by constructing a synthetic dataset locally on the client. This method can reduce the number of communications while improving the quality of the model. Marnissi O proposed a device selection strategy based on gradient importance in the paper "Client Selection in Federated Learning Based on Gradients Importance", which improves learning efficiency by selecting the device with the highest gradient value in each round of communication. However, these existing methods still lack effective application in complex network environments, especially in dynamically changing network environments and diversified device performance conditions. Summary of the invention

[0005] In view of the problem of poor stability in federated learning, the present invention provides a stable federated learning method based on the Raft election strategy to adapt to the dynamic device network environment and improve the efficiency and reliability of federated training while ensuring data privacy.

[0006] In a first aspect, the present invention provides a stable federated learning method based on a Raft election strategy, comprising:

[0007] Step 1: Divide all devices participating in federated learning into multiple clusters, and elect proxy devices in each cluster based on the Raft algorithm;

[0008] Step 2: Determine whether the proxy device is online. If it is online, continue training. If it is offline, re-elect the proxy device based on the Raft algorithm.

[0009] Step 3: The server sends the global model obtained from the previous round of aggregation to the proxy device;

[0010] Step 4: The proxy device distributes the received global model to each device in the cluster. Each device in the cluster performs local training and uploads the local training results to the proxy device.

[0011] Step 5: The proxy device aggregates the local training results in the cluster and uploads the aggregated cluster model to the server;

[0012] Step 6: The server aggregates the cluster models of each cluster to generate a global model;

[0013] Step 7: Determine whether the stopping condition is met. If so, output the global model. Otherwise, repeat steps 2 to 6 for the next round of training.

[0014] Furthermore, in step 1, the proxy device in each cluster is elected based on the Raft algorithm, which specifically includes:

[0015] According to the computing power and idle time of the devices in the cluster, the device u is calculated using the Mahalanobis distance as follows: i The performance weight W i ;

[0016]

[0017] Among them, v j represents other devices in the cluster, MS represents the Mahalanobis distance function, and K represents the number of devices participating in the training in the cluster;

[0018] Voting is initiated in turn according to the performance weight value, and the proxy device is elected using the Raft algorithm.

[0019] Furthermore, in step 5, the proxy device aggregates the local training results in the cluster, specifically including:

[0020]

[0021] Among them, ω s represents the cluster model obtained by aggregation, ω i represents the i-th local training result received by the proxy device, and N represents the number of local training results received by the proxy device.

[0022] Furthermore, step 6 specifically includes:

[0023]

[0024] Among them, ω T represents the generated global model, ω j represents the jth cluster model received by the server, and M represents the number of cluster models received by the server.

[0025] In a second aspect, the present invention provides a stable federated learning system based on the Raft election strategy, comprising:

[0026] The grouping module is used to divide all devices participating in federated learning into multiple clusters and elect proxy devices in each cluster;

[0027] The first judgment module is used to judge whether the proxy device is online. If it is online, the training continues; if it is offline, the proxy device is re-elected;

[0028] The communication module is used for the server to send the global model obtained by the previous round of aggregation to the proxy device; the proxy device to distribute the received global model to each device in the cluster, and each device in the cluster to upload the local training results to the proxy device; the proxy device to upload the cluster model obtained by aggregation to the server;

[0029] The training module is used to enable each device in the cluster to perform local training;

[0030] A first aggregation module, used for the proxy device to aggregate local training results in the cluster;

[0031] The second aggregation module is used for the server to aggregate the cluster models of each cluster to generate a global model;

[0032] The second judgment module is used to judge whether the training stop condition is reached.

[0033] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program.

[0034] In a fourth aspect, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in the first aspect.

[0035] Beneficial effects of the present invention:

[0036] (1) The present invention provides a stable federated learning method based on the Raft election strategy. The Raft algorithm is introduced to elect proxy devices, and the network status of the proxy devices is detected. The proxy devices initially aggregate the local training results in their clusters, and then the proxy devices upload the aggregated cluster model to the server. The server aggregates the cluster model again. This can avoid the impact of offline proxy devices on the stability of the entire federated learning process and reduce the number of communication rounds, thereby significantly improving the stability and efficiency of federated training.

[0037] (2) The present invention introduces the Mahalanobis distance function to calculate the performance weight value of the devices participating in the training, and improves the traditional Raft algorithm based on the performance weight value of the devices, thereby ensuring that the elected proxy devices have higher computing power and further improving the efficiency of federated learning.

[0038] (3) Through simulation experiments on public data sets, it is proved that the scheme of the present invention can effectively reduce the number of communication rounds, accelerate model convergence, and improve the stability of the entire system, so that the scheme of the present invention can perform efficient and reliable federated learning in complex dynamic scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 One of the flow charts of a stable federated learning method based on the Raft election strategy provided by an embodiment of the present invention;

[0040] Figure 2 A second flow chart of a stable federated learning method based on the Raft election strategy provided in an embodiment of the present invention;

[0041] Figure 3 A schematic diagram of a primary election proxy device provided by an embodiment of the present invention;

[0042] Figure 4 A schematic diagram of re-electing a proxy device when a proxy device is offline provided by an embodiment of the present invention;

[0043] Figure 5 An accuracy trend chart provided by an embodiment of the present invention when the offline rate is 100% and the number of participating devices is 50;

[0044] Figure 6An accuracy trend chart provided by an embodiment of the present invention when the offline rate is 100% and the number of device participants is 100;

[0045] Figure 7 This is an accuracy trend chart provided by an embodiment of the present invention when the offline rate is 50% and the number of participating devices is 50;

[0046] Figure 8 An accuracy trend chart provided by an embodiment of the present invention when the offline rate is 50% and the number of device participants is 100;

[0047] Fig. 9 This is a structural block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] like Figure 1 As shown, the embodiment of the present invention provides a stable federated learning method based on the Raft election strategy, comprising the following steps:

[0050] S101: Divide all devices participating in federated learning into multiple clusters, and elect proxy devices in each cluster based on the Raft algorithm;

[0051] S102: Determine whether the proxy device is online. If it is online, continue training. If it is offline, re-elect the proxy device based on the Raft algorithm. Specifically, if the proxy device is offline for a period of time greater than the set threshold, the election operation is triggered. The re-selected proxy device replaces the offline proxy device to perform its duties, thereby ensuring that the cluster training can continue to be executed and will not be separated from the overall federation training.

[0052] S103: The server sends the global model obtained by the previous round of aggregation to the proxy device; it can be understood that the global model of the first round needs to be obtained through initialization. That is, in the first round of training, the server initializes the global model ω 0 Send it to the proxy device C in each cluster agent ; When the iteration round is greater than or equal to 2, the server sends the global model ω of the previous round of aggregation to the proxy device C in each cluster agent , proceed to the next round of training.

[0053] S104: The proxy device distributes the received global model to each device in the cluster. Each device in the cluster performs local training and uploads the local training results to the proxy device. That is, in the first round of training, the initialized global model ω 0 Distribute to each device in the cluster; when the iteration round is greater than or equal to 2, the proxy device sends the global model ω aggregated in the previous round to the devices in each cluster for local training.

[0054] It is understandable that the computing power of each device in the cluster is not uniform, and the computing power of some devices may not be sufficient for local training. Therefore, devices with computing power perform local training and upload the results to the proxy device, and devices without computing power can upload local data to the proxy device for training.

[0055] S105: The proxy device aggregates local training results in the cluster and uploads the aggregated cluster model to the server;

[0056] S106: The server aggregates cluster models of each cluster to generate a global model;

[0057] S107: Determine whether the stopping condition is met. If so, output the global model. Otherwise, repeat steps S102 to S106 to perform the next round of training.

[0058] Specifically, the stopping condition can be set in advance, for example, the training is stopped when the number of iterations reaches a set total number of iterations; or the training is stopped when the global model has converged at this time.

[0059] The embodiment of the present invention provides a stable federated learning method based on the Raft election strategy. All devices participating in the federated training have introduced the Raft election algorithm. Before the training starts, the cluster is initialized and the proxy devices of each cluster are elected to construct a cloud-edge federated training architecture.

[0060] In addition, considering that during the training process, due to factors such as the dynamics of the device or network, there will be scenarios where the proxy device in the cluster is offline, the introduced Raft algorithm can detect whether the device is offline through heartbeat. When the proxy device is offline, in order to ensure the stability of the overall federated training, a new round of proxy device election will be started among the candidates to select a new proxy device. In this way, the architecture not only ensures the stability of the overall federated training, but also makes full use of resources to improve training efficiency. It can effectively cope with the challenges of unstable learning environments such as sudden device offline, limited device computing power, and device heterogeneity, and can also accelerate model training and improve model aggregation speed and performance.

[0061] In one embodiment, the Mahalanobis distance is combined with the Raft algorithm to elect a proxy device. The specific process is as follows: Based on the computing power and idle time of the devices in the cluster, the Mahalanobis distance is used to calculate the number of devices u according to the following formula: i performance weights;

[0062]

[0063] Among them, v j represents other devices in the cluster, K represents the number of devices participating in the training in the cluster; MS represents the Mahalanobis distance function, which is calculated as follows:

[0064]

[0065] Among them, x and y represent the comprehensive performance values ​​corresponding to two different devices. In this embodiment, when allocating the comprehensive performance value to each device, the computing power and idle time of the device are mainly considered. Specifically, the stronger the computing power and the longer the idle time, the higher the comprehensive performance value, indicating that the performance of the device is higher. The comprehensive performance values ​​of the two devices can be expressed in the form of and Represents two devices u i and v j The computing power (the number of CPU cores is used to represent the computing power in this embodiment), and Represents two devices u i and v j Idle time (in this embodiment, daily idle time is used for statistics). -1 (xy) represents two devices u i and v j The covariance between .

[0066] Among them, for a single device u i For example, let a and b represent the computing power and idle time of the device respectively. Then the formula for calculating the covariance between the computing power and idle time of the device is:

[0067]

[0068] Among them, μ a =E(a) represents the device u i The average computing power of the cluster, μ b =E(b) represents device u i The average idle time of the cluster.

[0069] Voting is initiated in order of performance weight values ​​(the device with a larger performance weight value initiates voting earlier and is more likely to be selected as a proxy device). The proxy device is elected using the Raft algorithm. The election process can be expressed as follows:

[0070] C agent =Raft(c i ,c j ), i≠j;

[0071] Among them, C i is the performance weight value calculated by the Mahalanobis distance algorithm in each cluster, Raft(c i ,c j ) represents the election process of each device based on the Raft algorithm through performance weight value, C agent Indicates the proxy device elected by the cluster.

[0072] It is understandable that the computing power and idle time of the device are dynamically changing. Therefore, each time a proxy device needs to be re-elected, the performance weight of the device needs to be recalculated.

[0073] The embodiment of the present invention provides a stable federated learning method based on the Raft election strategy. In a terminal device grouping cluster, the computing power and idle time of the devices in each cluster are used to calculate the performance weight value of each device through the Mahalanobis distance algorithm. The larger the performance weight value, the higher the corresponding performance of the device. This means that devices with higher performance have a higher priority to be elected as proxy devices, so that a more stable cloud-edge federated learning architecture can be constructed.

[0074] The Raft election algorithm is introduced for all terminal devices participating in federated training. Before the training starts, the cluster is initialized, and the proxy device is elected through the calculated weight value of the terminal device, thereby building a cloud-edge federated training architecture. During the training process, due to factors such as the dynamics of the device or network, there will be scenarios where the proxy device is offline in the cluster. On the one hand, the weight value of each device is calculated based on factors such as the computing power and idle time of each device, and the initial value of the proxy device is elected based on the weight value. On the other hand, when the proxy device is offline, in order to ensure the stability of the overall federated training, the introduced Raft algorithm detects the device offline through the heartbeat, and starts a new round of elections among the candidates with calculated weights to select a new proxy device leader. In this way, the architecture not only ensures the stability of the overall federated training, but also makes full use of resources to improve training efficiency.

[0075] In one embodiment, in step S105, the proxy device aggregates local training results in the cluster, specifically including:

[0076]

[0077] Among them, ω s represents the cluster model obtained by aggregation, ω i represents the i-th local training result received by the proxy device, and N represents the number of local training results received by the proxy device.

[0078] It is understandable that the proxy device may also use other aggregation methods to aggregate local training results.

[0079] In one embodiment, step S106 specifically includes:

[0080]

[0081] Among them, ω T represents the generated global model, ω j represents the jth cluster model received by the server, and M represents the number of cluster models received by the server.

[0082] It is understandable that the server may also use other aggregation methods to aggregate the cluster model.

[0083] Corresponding to the above method, an embodiment of the present invention also provides a stable federated learning system based on the Raft election strategy, including a grouping module, a first judgment module, a communication module, a training module, a first aggregation module, a second aggregation module and a second judgment module.

[0084] Among them, the grouping module is used to divide all devices participating in federated learning into multiple clusters and elect proxy devices in each cluster; the first judgment module is used to determine whether the proxy device is online. If it is online, training continues; if it is offline, the proxy device is re-elected; the communication module is used for the server to send the global model obtained by the previous round of aggregation to the proxy device; the proxy device distributes the received global model to each device in the cluster, and each device in the cluster uploads the local training results to the proxy device; the proxy device uploads the aggregated cluster model to the server; the training module is used to enable each device in the cluster to perform local training; the first aggregation module is used for the proxy device to aggregate the local training results in the cluster; the second aggregation module is used for the server to aggregate the cluster models of each cluster to generate a global model; the second judgment module is used to determine whether the training stop condition is met.

[0085] It should be noted that the stable federated learning system based on the Raft election strategy provided in the embodiment of the present invention is to implement the above method. Its specific functions can be referred to the above method embodiments, which will not be described in detail here.

[0086] In order to verify the effect of the scheme of the present invention (hereinafter referred to as FedPRE method), the following experiment was conducted.

[0087] 1. Experimental setup

[0088] The experimental environment is: the experimental process is run on a machine with a CPU of Intel Core i7-1165G7@2.80GHz, a GPU of NVIDIA GeForceMX450, and a RAM of 16GB. The software environment information is Python 3.9. The public dataset MNIST is used for verification. The dataset contains 6,000 training image data for input into the federated training experiment, and it is assumed that there are no malicious nodes in the participating devices of the experiment.

[0089] According to the method of the present invention, in terms of cluster grouping, each cluster is set to consist of 5 terminal devices. When performing local training, each iteration time is 0.5 seconds, and the basic communication time constant is 0.02 seconds. After 100 rounds of training, the designated proxy device is offline, and a terminal device is added to the cluster to simulate the dynamic proxy situation, where the weight value of the added device is higher than that of other devices. Through the normal federal training process, 50% and 100% offline situations of proxy devices in all clusters are simulated, and through experimental comparison, it is verified that the method of the present invention can effectively improve the stability of federal training.

[0090] (II) Analysis of experimental results

[0091] Experiment 1: Device offline rate 100%

[0092] First, by changing the number of terminal devices participating in the training and the scenario where the proxy devices are 100% offline during the 100th iteration, the experiment is conducted. The number of terminal devices is set to 50. The FedPRE algorithm proposed in the present invention is compared with FedSGD, FedAVG, FedAE without the introduction of the Raft algorithm, and FL_Raft without the introduction of the election weight algorithm (i.e. Figure 5 The FedPRE-UNELECT algorithm in the paper is used for experimental comparison, as shown in the attached figure. Figure 5 shown.

[0093] The experimental results show that the training algorithm without the introduction of the Raft election algorithm, such as the blue, red and yellow in the figure, after 100 iterations, all proxy devices are offline, and the accuracy of the federated training results will tend to balance, and there is no trend of updating. This is also in line with the previous prediction results. The proxy device is offline, which will cause the training results in the group cluster to be unable to participate in the update of the model. Compared with the purple line, the experimental result is the experimental algorithm FL_Raft that does not introduce the weight election strategy after the introduction of Raft. Since the traditional Raft algorithm cannot elect a higher-performance proxy device, it will affect the training efficiency of the overall training results, resulting in the accuracy of the model after aggregation. The algorithm is not better than the algorithm that introduces the weight strategy. The green result is the FedPRE algorithm proposed by the present invention. When all proxy devices of all group clusters are offline at 100 training iterations, the FedPRE algorithm can elect new proxy devices to participate in the federated training, ensuring that the training results can be produced normally.

[0094] Under the premise that the offline rate remains unchanged at 100%, when the number of participating terminals increases to 100, the effectiveness of the algorithm designed by the present invention is verified. Figure 6 As shown in the figure, after 100 rounds of training, 100% of the proxy devices in the home cluster are set to be offline. The accuracy of the comparison algorithm will not improve, and thus tends to be stable, while the accuracy of the algorithm FedPRE proposed in the present invention steadily increases. This is because the algorithm proposed in the present invention solves the problem that the terminal devices in the home cannot participate in the training when the proxy devices are offline. In summary, the algorithm FedPRE proposed in the present invention is superior to other algorithms in solving the problem of proxy devices being offline.

[0095] Experiment 2: Device offline rate 50%

[0096] The present invention analyzes the experimental results by changing the offline rate of the group cluster proxy devices to 50%. First, the number of participating smart home terminals is set to 50 for verification, in which the comparison experiment algorithm and other conditions of the experiment remain unchanged, thereby verifying the performance results of the proxy devices when not all are offline, as shown in the attached figure. Figure 7 shown.

[0097] The experimental results of reducing the offline rate of proxy devices show that the training algorithm without the introduction of the Raft election algorithm, such as the blue, red and yellow in the figure, will not have the same training results as when all are offline after 100 iterations, and will tend to balance. The accuracy will still be improved, but the degree of improvement will be significantly reduced before and after 100 rounds. Through analysis, it can be concluded that this is because some devices are not offline because of the proxy devices. The terminal devices participating in the training can update the trained model to participate in the aggregation process of the global model, thereby improving the accuracy of the global model. It can be concluded that whether the proxy device is offline can affect the results of the overall federated training. The FedRE algorithm proposed in the present invention, when the proxy device is 50% offline, the performance result is consistent with the offline rate of 100%. After 100 rounds of training, the accuracy of the federated learning results can still rise steadily, so it can be shown that the FedPRE algorithm elects new proxy devices in the home cluster grouping participating in the training, which helps to steadily improve the accuracy of the training results.

[0098] In order to verify the effectiveness of the FedPRE algorithm with different numbers of participating terminals, the present invention increases the number of participating terminals to conduct experiments when the proxy devices of 50% of the households are offline. The results are shown in the attached figure. Figure 8 As shown, when the number of participating terminals is 100, the accuracy of the training results is consistent with the trend when the number of participating terminals is 50. It can be concluded that the algorithm proposed in the present invention is effective for different numbers of participating terminals and different offline levels of proxy devices, and FedPRE can provide a stable training process for the federated training process.

[0099] Fig. 9 An example of a physical structure diagram of an electronic device is shown in FIG. Fig. 9As shown, the electronic device may include: a processor (processor) 901, a communication interface (Communications Interface) 902, a memory (memory) 903 and a communication bus 904, wherein the processor 901, the communication interface 902, and the memory 903 communicate with each other through the communication bus 904. The processor 901 can call the logic instructions in the memory 903 to execute a stable federated learning method based on the Raft election strategy, which includes: step 1: all devices participating in federated learning are divided into multiple clusters, and the proxy device in each cluster is elected based on the Raft algorithm; step 2: determine whether the proxy device is online, if it is online, continue training, if it is offline, re-elect the proxy device based on the Raft algorithm; step 3: the server sends the global model obtained by the previous round of aggregation to the proxy device; step 4: the proxy device distributes the received global model to each device in the cluster, each device in the cluster performs local training, and uploads the local training results to the proxy device; step 5: the proxy device aggregates the local training results in the cluster, and uploads the aggregated cluster model to the server; step 6: the server aggregates the cluster models of each cluster to generate a global model; step 7: determine whether the stop condition is met, if so, output the global model, otherwise, repeat steps 2 to 6 for the next round of training.

[0100] In addition, when the logic instructions in the above-mentioned memory 903 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0101] An embodiment of the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the methods provided by the above-mentioned method embodiments, for example, including: Step 1: All devices participating in federated learning are divided into multiple clusters, and proxy devices in each cluster are elected based on the Raft algorithm; Step 2: Determine whether the proxy device is online, if online, continue training, if offline, re-elect the proxy device based on the Raft algorithm; Step 3: The server sends the global model obtained by the previous round of aggregation to the proxy device; Step 4: The proxy device distributes the received global model to each device in the cluster, and each device in the cluster performs local training and uploads the local training results to the proxy device; Step 5: The proxy device aggregates the local training results in the cluster, and uploads the aggregated cluster model to the server; Step 6: The server aggregates the cluster models of each cluster to generate a global model; Step 7: Determine whether the stop condition is met, if so, output the global model, otherwise, repeat steps 2 to 6 for the next round of training.

[0102] An embodiment of the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the methods provided by the above-mentioned method embodiments are implemented, for example, including: Step 1: All devices participating in federated learning are divided into multiple clusters, and proxy devices in each cluster are elected based on the Raft algorithm; Step 2: Determine whether the proxy device is online, if online, continue training, if offline, re-elect the proxy device based on the Raft algorithm; Step 3: The server sends the global model obtained by the previous round of aggregation to the proxy device; Step 4: The proxy device distributes the received global model to each device in the cluster, and each device in the cluster performs local training and uploads the local training results to the proxy device; Step 5: The proxy device aggregates the local training results in the cluster, and uploads the aggregated cluster model to the server; Step 6: The server aggregates the cluster models of each cluster to generate a global model; Step 7: Determine whether the stop condition is met. If so, output the global model, otherwise, repeat steps 2 to 6 for the next round of training.

[0103] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A stable federated learning method based on Raft election strategy, characterized in that: include: Step 1: Divide all devices participating in federated learning into multiple clusters, and elect proxy devices in each cluster based on the Raft algorithm; Step 2: Determine whether the proxy device is online. If it is online, continue training. If it is offline, re-elect the proxy device based on the Raft algorithm. Step 3: The server sends the global model obtained from the previous round of aggregation to the proxy device; Step 4: The proxy device distributes the received global model to each device in the cluster. Each device in the cluster performs local training and uploads the local training results to the proxy device. Step 5: The proxy device aggregates the local training results in the cluster and uploads the aggregated cluster model to the server; Step 6: The server aggregates the cluster models of each cluster to generate a global model; Step 7: Determine whether the stopping condition is met. If so, output the global model. Otherwise, repeat steps 2 to 6 for the next round of training.

2. According to claim 1, a stable federated learning method based on Raft election strategy is characterized in that: In step 1, the proxy device in each cluster is elected based on the Raft algorithm, including: According to the computing power and idle time of the devices in the cluster, the performance weight of the device UI is calculated using the Mahalanobis distance according to the following formula Wi ; Where vj represents other devices in the cluster, MS represents the Mahalanobis distance function, and K represents the number of devices participating in the training in the cluster; Voting is initiated in turn according to the performance weight value, and the proxy device is elected using the Raft algorithm.

3. According to claim 1, a stable federated learning method based on Raft election strategy is characterized in that: In step 5, the proxy device aggregates the local training results in the cluster, including: Among them, ω s represents the cluster model obtained by aggregation, ω i represents the i-th local training result received by the proxy device, and N represents the number of local training results received by the proxy device.

4. According to claim 1, a stable federated learning method based on Raft election strategy is characterized in that: Step 6 specifically includes: Among them, ω T represents the generated global model, ω j represents the jth cluster model received by the server, and M represents the number of cluster models received by the server.

5. A stable federated learning system based on Raft election strategy, characterized in that: include: The grouping module is used to divide all devices participating in federated learning into multiple clusters and elect proxy devices in each cluster; The first judgment module is used to judge whether the proxy device is online. If it is online, the training continues; if it is offline, the proxy device is re-elected; The communication module is used for the server to send the global model obtained by the previous round of aggregation to the proxy device; the proxy device to distribute the received global model to each device in the cluster, and each device in the cluster to upload the local training results to the proxy device; the proxy device to upload the cluster model obtained by aggregation to the server; The training module is used to enable each device in the cluster to perform local training; A first aggregation module, used for the proxy device to aggregate local training results in the cluster; The second aggregation module is used for the server to aggregate the cluster models of each cluster to generate a global model; The second judgment module is used to judge whether the training stop condition is reached.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 4 is implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images