Edge device data transmission method based on hybrid federated learning framework

Abstract

The invention relates to an edge device data transmission method based on a hybrid federated learning framework. The method comprises the following steps: constructing a hybrid federal learning framework; the hybrid federal learning framework comprises an edge client and a server; the server comprises a client management module, a classifier and a coordinator, and the classifier classifies the edge clients into a lagging-behind person, a normal person and a leader according to the training speed of each edge client; the coordinator combines a plurality of leaders into centralized federated learning to maintain fast model iteration. The method comprises the following steps of: intermittently returning model parameters to a connected starter in local decentralized federated learning by a laggard to ensure model convergence, converting data Non-IID and a client heterogeneous degree into matching measurement of a leader and the laggard by a coordinator, and adjusting the coordinator and a classifier in real time by a hybrid federated learning framework to adapt to a complex and changeable network environment. By adopting the method, the data transmission efficiency of the edge equipment can be improved.

Description

Technical Field

[0001] The present application relates to the field of Internet of Things technology, and in particular to an edge device data transmission method based on a hybrid federated learning framework. Background Art

[0002] In the industrial Internet of Things scenario, edge devices in many factories, such as sensors and smart controllers, have limited computing power and energy resources. These devices need to participate in federated learning to optimize production processes, predict equipment failures, etc., but complex model training and aggregation processes may overwhelm the devices. Different edge clients (such as sensors, smart controllers, etc.) have different hardware configurations, computing power, network environments, and other factors, resulting in client heterogeneity. This difference makes some clients train quickly and become leading clients during the federated learning process, while others may become lagging clients due to their own limitations, just like the lengths of the boards in a wooden barrel are different. In the complex aggregation process, the efficiency of the entire federated learning often depends on the slowest lagging client, just like how much water a wooden barrel can hold depends on the shortest board. Accelerating federated learning training can reduce the time and resource consumption of equipment in training and data transmission. For example, in the training of equipment failure prediction models, edge devices can complete local training and parameter upload faster, reduce equipment energy consumption, improve production efficiency, reduce downtime caused by equipment failure, and improve the stability and economic benefits of industrial production. Summary of the invention

[0003] Based on this, it is necessary to provide an edge device data transmission method based on a hybrid federated learning framework that can improve the data transmission efficiency of edge devices in response to the above technical problems.

[0004] An edge device data transmission method based on a hybrid federated learning framework, the method comprising: Construct a hybrid federated learning framework; the hybrid federated learning framework includes an edge client and a server; the server includes a client management module, a classifier, and a coordinator; obtain a data set generated or collected by an edge client; Initialize the internal parameters and equipment failure prediction model of the edge client and server. After the edge client submits a registration request to the client management module, it downloads the configuration data from the server and trains the equipment failure prediction model for the current round based on the configuration data and data set, and uploads the updated model parameters to the server; The server globally aggregates the updated models from edge clients according to the aggregation strategy, and uses the classifier to classify edge clients into stragglers, normal clients, and leaders according to their training speeds; The coordinator receives the classification results of the edge client from the classifier and transforms the matching problem of the leader and the stragglers into the maximum weight problem in the bipartite graph to generate the optimal matching solution. The optimal matching solution is sent to the edge client through the client management module to build local decentralized federated learning for the locally aggregated leaders and stragglers, and the stragglers are temporarily removed from the central aggregation in the centralized federated learning to complete the current round of model training. Other edge clients resubmit registration applications to participate in a new round of training, and distribute new training parameters to edge clients participating in this round of training through centralized federated learning and aggregators in local decentralized federated learning until the device fault prediction model converges or reaches the number of training rounds, and a trained device fault prediction model is obtained.

[0005] The above-mentioned edge device data transmission method based on the hybrid federated learning framework, this application constructs a hybrid federated learning framework, uses a classifier to divide edge clients into stragglers, normal ones and leaders according to the training speed of edge clients, can quickly identify devices of different performance levels, and then uses a coordinator to accept the classification results of edge clients from the classifier and convert the matching problem of leaders and stragglers into the maximum weight problem in the bipartite graph to generate the optimal matching solution, and sends the optimal matching solution to the edge client through the client management module to build local decentralized federated learning for the leaders and stragglers of local aggregation, and temporarily remove the stragglers from the central aggregation in the centralized federated learning to complete the current round of model training, and in the centralized federated learning link, temporarily remove the stragglers from the central aggregation, so that normal ones and leaders can quickly advance global aggregation, greatly reducing waiting time. At the same time, using the idle time when the stragglers are removed, with the help of local decentralized federated learning, they can start learning locally and improve their own capabilities. In addition, the coordinator transforms the matching of leaders and stragglers into the maximum weight problem in a bipartite graph, and generates the optimal matching solution. The leader can accurately help stragglers according to the optimal matching solution, reduce invalid transmission caused by poor coordination, and improve the accuracy of data transmission. Finally, after the edge device completes a round of training, the devices that do not meet the standards reapply and participate in a new round of training. The aggregator distributes the new parameters to the participating devices in a timely manner. The devices can continuously obtain the latest model parameters, avoid repeated training and invalid transmission, and ensure that data transmission always serves efficient model training. Ultimately, the overall data transmission efficiency of edge devices is improved, and local training and parameter upload are completed faster, reducing equipment energy consumption, improving production efficiency, reducing downtime caused by equipment failure, and improving the stability of industrial production. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 It is a flowchart of an edge device data transmission method based on a hybrid federated learning framework in one embodiment; Figure 2A schematic diagram of a hybrid federated learning framework in one embodiment; Figure 3 A schematic diagram of a matching strategy optimization target in an embodiment; Figure 4 It is a schematic diagram of the influence of different data sets and data distribution under the synchronous aggregation strategy in another embodiment; Figure 5 This is an impact diagram of different data sets and data distributions under an asynchronous aggregation strategy in one embodiment. DETAILED DESCRIPTION

[0007] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0008] In one embodiment, Figure 1 As shown, a method for edge device data transmission based on a hybrid federated learning framework is provided, comprising the following steps: Step 102, construct a hybrid federated learning framework; the hybrid federated learning framework includes an edge client and a server; the server includes a client management module, a classifier and a coordinator; obtain a data set generated or collected by the edge client.

[0009] Constructing a hybrid federated learning framework (JHFed) Figure 2 As shown in the figure, JHFed is built on two components: the classifier classifies clients into stragglers, normals, and leaders based on the training speed of each client; the coordinator combines multiple leaders into centralized federated learning (CFL) to maintain fast model iteration. The stragglers intermittently return model parameters to the connected starters in the local decentralized federated learning (DFL) to ensure model convergence. In addition, the coordinator converts data Non-IID (non-independent and identically distributed) and client heterogeneity into leader and stragglers matching metrics. JHFed adjusts the coordinator and classifier in real time to adapt to the complex and changing industrial network environment.

[0010] Step 104, initialize the internal parameters and equipment fault prediction model of the edge client and server. After submitting a registration request to the client management module, the edge client downloads the configuration data from the server and trains the equipment fault prediction model for the current round based on the configuration data and data set, and uploads the updated model parameters to the server.

[0011] The edge client and server initialize internal parameters and models. Subsequently, the edge client submits a registration request to the client management on the server side. If the request is not accepted, the edge client will wait for the next round of registration. If the request is accepted, the edge client will download configuration data from the server, including the number of training rounds, training model, batch size, etc.

[0012] The edge client performs model training on the current local data set and uploads the updated model parameters to the server. The server globally aggregates the updated models from the edge clients according to the aggregation strategy (asynchronous aggregation, synchronous aggregation, and secure aggregation, etc.).

[0013] Step 106 , the server globally aggregates the updated models from the edge clients according to the aggregation strategy, and uses a classifier to classify the edge clients into stragglers, normal clients, and leaders according to the training speeds of the edge clients.

[0014] When the waiting problem of the "barrel effect" occurs, the classifier will maintain a sequence of stragglers and a sequence of leaders based on the local training time and the number of uploaded parameters of the edge client, and divide the edge clients into leaders, stragglers, and normal ones. This classification can quickly identify devices of different performance levels, laying the foundation for subsequent differentiated processing. Stragglers slow down the overall progress due to limitations in computing power, network conditions, etc. Accurately identifying them can solve the problem in a targeted manner, avoid letting fast devices wait in vain for a long time, and thus ensure a smooth data transmission process.

[0015] Step 108, using the coordinator to accept the edge client classification results from the classifier and convert the matching problem of the leader and the stragglers into the maximum weight problem in the bipartite graph to generate the optimal matching solution, and sending the optimal matching solution to the edge client through the client management module to build local decentralized federated learning for the locally aggregated leaders and stragglers, and temporarily remove the stragglers from the central aggregation in the centralized federated learning to complete the current round of model training.

[0016] This application proposes a strategy that combines DFL (local decentralized federated learning) with CFL (centralized federated learning), which is cleverly integrated into the entire model training process. In the centralized federated learning link, the stragglers are temporarily removed from the central aggregation, so that the normal clients and leading clients can quickly advance the global aggregation, greatly reducing the waiting time. The coordinator will use the waiting time caused by complex aggregation to build DFL (local decentralized federated learning). In this process, the coordinator temporarily removes the stragglers from CFL (centralized federated learning). This move is to avoid the burden of stragglers on the global aggregation, while also making full use of the waiting time, allowing the stragglers to learn locally instead of letting other normal and leading clients wait, thereby improving the overall time utilization efficiency.

[0017] The coordinator transforms the matching problem of leaders and stragglers into the maximum weight problem in a bipartite graph, and combines the client classification information provided by the classifier to ensure that each straggler can be matched with a suitable leader. Then, the optimal matching solution is sent to the edge client through the client management module in the server. The classifier provides a classification basis, and the coordinator generates and transmits matching solutions based on this basis, so that the edge client can build local decentralized federated learning according to the solution for the leaders and stragglers that are locally aggregated. At the same time, in centralized federated learning, the stragglers are temporarily removed from the central aggregation to complete the current round of model training. Through the optimal matching solution, the leader can accurately help the stragglers according to the solution, reduce invalid transmission caused by poor coordination, and improve model training efficiency.

[0018] In step 110, other edge clients resubmit registration applications to participate in a new round of training, and distribute new training parameters to edge clients participating in this round of training through aggregators in centralized federated learning and local decentralized federated learning until the device fault prediction model converges or reaches the number of training rounds, thereby obtaining a trained device fault prediction model.

[0019] After the edge client completes a round of training, the devices that do not meet the standards reapply and participate in a new round of training. The aggregator distributes the new parameters to the participating devices in a timely manner, and the devices can continuously obtain the latest model parameters. This iterative training and timely parameter distribution mechanism ensures that the edge devices can obtain the latest model parameters in a timely manner, reduces repeated training and invalid data transmission caused by untimely parameter updates, improves the effectiveness and efficiency of data transmission, makes the entire training process smoother and more efficient, and ultimately improves the overall data transmission efficiency of edge devices.

[0020] The above-mentioned edge device data transmission method based on the hybrid federated learning framework, this application constructs a hybrid federated learning framework, uses a classifier to divide edge clients into stragglers, normal ones and leaders according to the training speed of edge clients, can quickly identify devices of different performance levels, and then uses a coordinator to accept the classification results of edge clients from the classifier and convert the matching problem of leaders and stragglers into the maximum weight problem in the bipartite graph to generate the optimal matching solution, and sends the optimal matching solution to the edge client through the client management module to build local decentralized federated learning for the leaders and stragglers of local aggregation, and temporarily remove the stragglers from the central aggregation in the centralized federated learning to complete the current round of model training, and in the centralized federated learning link, temporarily remove the stragglers from the central aggregation, so that normal ones and leaders can quickly advance global aggregation, greatly reducing waiting time. At the same time, using the idle time when the stragglers are removed, with the help of local decentralized federated learning, they can start learning locally and improve their own capabilities. In addition, the coordinator transforms the matching of leaders and stragglers into the maximum weight problem in a bipartite graph, and generates the optimal matching solution. The leader can accurately help stragglers according to the optimal matching solution, reduce invalid transmission caused by poor coordination, and improve the accuracy of data transmission. Finally, after the edge device completes a round of training, the devices that do not meet the standards reapply and participate in a new round of training. The aggregator distributes the new parameters to the participating devices in a timely manner. The devices can continuously obtain the latest model parameters, avoid repeated training and invalid transmission, and ensure that data transmission always serves efficient model training. Ultimately, the overall data transmission efficiency of edge devices is improved, and local training and parameter upload are completed faster, reducing equipment energy consumption, improving production efficiency, reducing downtime caused by equipment failure, and improving the stability of industrial production.

[0021] In one embodiment, a process of modeling a hybrid federated learning framework includes: Assume that the computing resources of the edge client and storage resources Sufficient for local model training, with It is sufficient for model transfer. Since the computing resources on the server are sufficient compared to the edge clients, they are not discussed in this paper. It is worth noting that the dataset of the edge client may change dynamically over time, but in a single round of training, the input of the model is constant. In a distributed network, the edge client dataset may change dynamically over time, and the input of the model is constrained to be constant in a single training round. Define the classes contained in the dataset as . Then use To define the Non-IID data.

[0022]

[0023] When the edge client registers with the FL training framework, it uploads the available resources (computation, storage, and bandwidth) and task requirements (task type and deadline) to the server. The time taken for the above registration is recorded as:

[0024] in Indicates that from the edge client Information uploaded to the server. Indicates the registration processing time on the server. Indicates that the server distributes initialization parameters to edge clients Then, the edge client Receive the global initial model and local iteration number After that, the model is trained to generate model parameters . Define a round of customer training time as:

[0025] in Represents the size of the machine learning task and the dataset is related to the training model. Therefore, The local iteration of the round is:

[0026] When using the synchronous aggregation method, the server waits for all edge clients to upload parameters before starting global aggregation. The transmission time to the server is defined as:.

[0027] in surface The size of the training parameters, express The bandwidth at time t.

[0028] Considering the diversity of aggregation methods, the aggregation time is generalized as After the global model aggregation is completed, the updated model is distributed from the server to the edge clients. This transmission time is defined as:

[0029] When using asynchronous aggregation methods, the server can wait Each edge client uploads parameters, and then enables global aggregation. To express the waiting time of the aggregation:

[0030] Compared with synchronous aggregation, the number of edge clients participating in asynchronous aggregation is becomes , The above formula represents the waiting time of synchronization aggregation. Therefore, the synchronization aggregation time is approximately defined as .

[0031] In asynchronous aggregation, the updated model will only be distributed to edge clients participating in the aggregation.

[0032] definition:

[0033] is a binary Representation parameters From the edge client Passed to the server. Then you can get:

[0034] In DFL, edge clients can receive parameters updated by a neighbor to initiate model aggregation. Similar to CFL, the asynchronous aggregation computing method of CFL can be used to obtain training time, aggregation time, and transmission time. Therefore, the JHFed problem is formulated as follows:

[0035]

[0036] Among them, the first constraint indicates that the loss value of the loss function can converge, the second constraint ensures the resource constraint of consuming internal available resources, the third constraint limits the number of aggregated edge clients, and the fourth constraint ensures that edge clients participate in aggregation. In the above optimization problem, is the unknown variable to be solved.

[0037] In one embodiment, the configuration data includes the number of training rounds, the training model, and the batch size.

[0038] In one embodiment, a classifier is used to classify edge clients into stragglers, normal clients, and leaders according to the training speed of the edge clients, including: The classifier sets the training interval according to the average and long-term level of the edge client's historical upload time; if the current edge client's upload time to upload available resources and task requirements to the server is greater than the training interval, the edge client is defined as a leader and added to the leader sequence; If the time it takes for the current edge client to upload available resources and task requirements to the server is less than the training interval, the edge client is defined as a straggler and added to the stragglers sequence; If the time it takes for the current edge client to upload available resources and task requirements to the server is equal to the training interval, the edge client is defined as normal.

[0039] In a specific embodiment, the clients are divided into three categories according to the number of training and uploading times of the clients: Stragglers: Since edge clients inevitably generate a round of updates at different times, stragglers are common in scenarios such as mobile edge computing, unstable network connections, or edge client failures. In synchronous aggregation, since aggregation requires parameters from all edge clients, the server must wait for stragglers to complete before starting aggregation. In asynchronous aggregation, since the updated parameters from stragglers are relatively old, the convergence efficiency of FL is reduced.

[0040] Leader: This application proposes the concept of leader as the antonym of stragglers. As the global aggregation algorithm becomes more complex, its running time will be significantly extended, which means that edge clients may have to wait for the server to complete the global aggregation. Leaders usually have sufficient computing and bandwidth resources or smaller data sets, allowing them to train models and upload at a faster speed. In synchronous aggregation, leaders need to wait for stragglers. In asynchronous aggregation, leaders tend to be updated more frequently.

[0041] Normals: Normals are between leaders and stragglers, representing the average level of edge clients. Due to changes in datasets or resource availability, the identity of normals may shift to other categories.

[0042] Specifically, the classifier maintains a list on the server to record the upload time:

[0043] in Indicates the time when the server distributes the model to the edge client, Upload the updated model, Indicates the updated time. In JHFed, the classification results of the edge client will be sent to the coordinator for DFL formulation and CFL adjustment.

[0044] Considering the communication channel limitations on the server side, even if the edge clients have similar resources, the updated parameters will arrive in a certain order. Frequently starting the classifier due to a small time difference will result in a huge overhead. In addition, please note that in the case of disconnection or bandwidth reduction, the delay of communication may be extended, resulting in the possibility that the classifier will mistakenly identify the edge client as a straggle. In order to reduce the probability of such events, this application defines:

[0045] in Is a positive integer representing the tolerance value. is the average of the historical upload time of edge clients, Indicates the long-term level. It is the upload time of the edge client in the current round, which represents the short-term level. When the upload time When the interval is greater than this, this application defines the edge client as a leader and adds it to the leader sequence When uploading time When the interval is less than this, the application defines the edge client as a straggler and adds it to the stragglers sequence. In order to avoid frequent activation of the classifier, this application proposes a threshold with a difference of only When the value is greater than this threshold, the classifier will be triggered.

[0046] In one embodiment, the classifier sets the training interval to be based on the average value of the edge client's historical upload time and the long-term level.

[0047] in, is a positive integer representing the tolerance value, is the average of the client's historical upload time, Indicates the long-term level.

[0048] In one embodiment, the leader and straggler matching problem is transformed into a maximum weight problem in a bipartite graph, including Construct an xy coordinate system and use the x-axis to distinguish between leaders and laggards. On the x-axis, the stragglers Below the x-axis, A scaling value representing the number of client model updates. express and The vector sum of The length of approximates the leader and stragglers matching metric, express and The vector sum of The projection on the y-axis approximately represents the degree to which the data is not independent and identically distributed, and the projection on the x-axis approximately represents the number of times the client has uploaded the model; According to the relationship between each vector in the xy coordinate system, the matching strategy optimization target is adopted to transform the matching problem of leaders and stragglers into the maximum weight problem in the bipartite graph.

[0049] In a specific embodiment, after the classifier determines the client category, the coordinator removes the stragglers from the CFL and then connects the stragglers with the leaders to build the DFL. When there are leaders but no stragglers, JHFed establishes connections between the leaders but does not remove them from the CFL. Figure 3 As shown, there are various DFL schemes for leaders and stragglers. Considering that Non-IID and heterogeneous clients mainly affect the convergence efficiency, this application adopts a matching strategy to find a solution that is more conducive to global training convergence. Figure 3 middle is the drift angle, which is calculated by the difference between the updated client and global parameter vectors. It is scaled from the values ​​updated in client 4. Use the x-axis to distinguish between leaders and stragglers. On the x-axis, the stragglers Below the x-axis, A scaling value representing the number of client model updates. express and The vector sum of The length of approximates the leader and stragglers matching metric, express and The vector sum of The projection on the y-axis approximately represents the degree to which the data is not independent and identically distributed, and the projection on the x-axis approximately represents the number of model uploads by the client. Inspired by graph theory, the above problem is transformed into the maximum weight problem in a bipartite graph. In actual scenarios, the number of clients at the edge of the network will increase from hundreds to thousands. Existing methods, such as the KM algorithm, need to traverse all nodes to find the optimal solution, so any brute force-based search method is prohibited. In order to solve the above dilemma, this application adopts an approximate calculation scheme to reduce the computational complexity. This application uses a simulated annealing algorithm to approximate the optimal solution.

[0050] In one embodiment, in order to analyze the feasibility of the proposed model training framework, the convergence of the proposed JHFed method is introduced. There is a set of standard routines for convergence analysis of federated learning, so we follow it and take the following four widely used assumptions: Assumption 1: (L-smooth)Assume that the loss function f on the client is and for L− smooth , L>0 :

[0051] Assumption 2. (Strong Convexity) Assume that the loss function f on the client is θ-convex, $\theta>0$:

[0052] Assumption 3. (Bounded Gradient) The squared norm of the stochastic gradient is uniformly bounded.

[0053]

[0054] Where ξ represents a sample of the client's local dataset.

[0055] Assumption 4. (Global Optimum Exists) Assume that the loss function has a global minimum .

[0056] In the synchronous JHFed scheme, when leaders and stragglers emerge, the stragglers are removed from the CFL and merged with the leaders to form the DFL. JHFed passes the information of stragglers to the leaders, compensates the slower updates with faster updates, and converges back to the CFL after several rounds.

[0057] Theorem 1. Assume that the client uses an unbiased sampling scheme. , represents the learning rate. According to the above assumptions, After rounds of iterations, we have:

[0058] in , express The model parameters after round of update, are the initial model parameters.

[0059] Proof: Without loss of generality, for each global round in synchronous JHFed, , in represents the number of clients participating in the aggregation. ,use Smoothness and Strong convexity, we have

[0060] in

[0061] In asynchronous JHFed, the server receives After the model of each client updates its parameters, global aggregation can be performed. ,The optimality gap of asynchronous JHFed is similar to that of synchronous JHFed, but . Note that JHFed converges to the bounds With Non-IID data Spend, Clients and rounds With increase, As the value of increases, the convergence bounds become looser. Bigger and When is smaller, the bound of Theorem 1 decreases.

[0062] Remark 1. Theorem 1 shows that for θ-convex, L-smooth and -Non-IID data, JHFed The rate convergence reaches the global optimal value after T rounds of iterations. The time spent in each round mainly includes training, transmission and aggregation. Due to the complexity of aggregation, client heterogeneity, data Non-IID, etc., the above three serial steps may be blocked and cause additional waiting time. Based on the above intuition, in synchronous aggregation, JHFed performs more local client aggregations in a fixed time, thereby reducing the time required to converge to the same boundary. In asynchronous aggregation, JHFed operates clients to work together for aggregation during the waiting time, completing T rounds of iterations faster to accelerate convergence.

[0063] In the specific simulation experiments, three classic open data sets are used, including MNIST, FMNIST and CIFAR-10. MNIST contains 70,000 28×28 grayscale handwritten photos, divided into 10 classes, with 60,000 samples in the training set and 10,000 samples in the test set. FashionMNIST has the same number of samples, classes and training / test splits as MNIST. However, images of clothing items are used instead of handwritten numbers. CIFAR-10 contains 60,000 32×32 color images, marked into 10 classes, of which 50,000 samples are used for training and 10,000 samples are used for testing.

[0064] Two classic models are used to train the dataset. One is the deep convolutional neural network LeNet-5 for the MNIST dataset. LeNet-5 consists of two 5 × 5 convolutional layers, a fully connected layer (120 units), a fully connected layer (84 units), and a softmax output layer (10 units). LeNet-55 uses the backpropagation algorithm to calculate the gradient and optimizes the weights and biases of the network by minimizing the error function. The other is ResNet-18 for FMNIST and CIFAR-10. ResNet-18 contains 5 convolution groups and 18 weight layers. This paper introduces SkipConnections to solve the degradation problem, which greatly alleviates the difficulty of training deep neural networks.

[0065] The number of aggregation rounds on the server is set to 100. In addition, malicious client identification is added to simulate complex aggregation scenarios. Assume that the number of local aggregation rounds on each client is 2, the batch size of the dataset is 128, the learning rate is 0.1, and it stops decaying after 50 rounds. In the IID experiment, all datasets are evenly distributed among clients. In the Non-IID case, the dataset is partitioned according to the degree of Non-IID distribution. During training, the client uses cross entropy (CE) as the loss function and SGD momentum as the optimization method. In order to better evaluate the performance of JHFed, we conducted comparative experiments using the following three types of benchmark algorithms: FedAvg, MIFA, and JHFed-R.

[0066] Numerical results: 1) Synchronous Aggregation: Six sets of experiments are conducted to compare the performance of the proposed method with the baseline. Table 1 shows the accuracy comparison of synchronous FedAvg and our JHFed on 3 datasets (with both IID and Non-IID data partitions). The best results are indicated in bold. Since MIFA is an asynchronous aggregation scheme, it is excluded here. In Table 1, Accuracy represents the classification accuracy of the final model on the test set, that is, the ratio of the number of correctly classified samples to the total number of samples. Time represents the entire time from scheduling the model to the end of training, including aggregation, local training, data loading, etc. Speedup is measured for each global clock and normalized by the runtime of FedAvg.

[0067] JHFed shows the best performance in the synchronous aggregation scheme. On the three IID datasets, JHFed has a higher accuracy than FedAvg. However, there is no obvious gap between JHFed and FedAvg because the advantage of JHFed's mechanism considering Non-IID is not reflected, and it is difficult to avoid the impact of client heterogeneity under the synchronous aggregation scheme. In contrast, JHFed shows better performance in the Non-IID setting, and its accuracy on the CIFAR-10 dataset is 10% higher than that of FedAvg. Compared with JHFed, JHFed-R adopts a random matching generation strategy, which leads to large fluctuations in accuracy among the three datasets. On Non-IID CIFAR-10, JHFed-R has a higher accuracy than FedAvg, while on IID CIFAR-10 it is lower. The experimental results demonstrate the stability of JHFed and the superiority of the approximate optimal matching strategy.

[0068] Table 1

[0069] Since the servers use a synchronous aggregation scheme, the runtime speedup of JEDFED is not significant. On the Non-IID CIFAR-10 dataset, JEDFED achieves a 1.07× speedup compared to FedAvg, while the random match generation strategy results in JEDFED-R having a similar training time to FedAvg. Figure 4 The accuracy of the algorithms on the three datasets is shown in detail. In the synchronous regime, all converged with similar efficiency, but when they finally stabilized, the accuracy of JHFed was better. Unbalanced datasets can provide misleading results compared to IID data. Therefore, NonIID data showed larger fluctuations during convergence.

[0070] 2) Asynchronous Aggregation: Table 2 shows the accuracy comparison of asynchronous FedAvg and our JHFed on 3 datasets (with both IID and Non-IID data partitions). The number of asynchronies is set to 1 so that the server can start global aggregation after receiving an update from one client. The differences in accuracy and runtime among the four algorithms are more significant in the asynchronous aggregation scheme. On IID data, JHFed achieves nearly 3% higher accuracy than FedAvg. On the Non-IID CIFAR10 dataset, JHFed shows a better accuracy advantage, leading FedAvg by 4.5% and MIFA by 2%, respectively. In the FedAvg scheme, some stragglers are removed, resulting in a lack of data during training, which leads to lower accuracy on the test set. JHFed-R becomes more unstable in the asynchronous aggregation scheme under the random match generation strategy. Its accuracy is lower than FedAvg on other datasets except Non-IID MNIST, but its runtime is shorter. MIFA corrects the backward bias and outperforms FedAvg and JHFed-R in terms of accuracy and running time, but is inferior to JHFed. In asynchronous aggregation, JHFed shows the best convergence efficiency performance, with a speedup of 1.57× compared to FedAvg.

[0071] Table 2

[0072] Figure 5 shows the training process of the algorithm on three datasets in detail. Compared with the synchronous scheme, the fluctuations in the convergence process are more obvious due to the increase in the number of Non-IID data. When stragglers act as providers of parameter updates in a round, the uploaded parameters may contain part of the stragglers, which exacerbates the fluctuations. Overall, the classifier and coordinator matching strategy in JHFed shows better performance, and the training results tend to be stable in the post-iteration stage. In a distributed scenario with Non-IID data and client heterogeneity, the testbed experiments show that JHFed can achieve better results in terms of accuracy and training efficiency, verifying the theoretical results.

[0073] 3) Non-IID degree: The effect of data heterogeneity on CIFAR-10 is studied by changing the Non-IID degree parameter κ. κ is the 0-1 normalized form of Γ. For larger κ, the data non-IID will be more unbalanced. The results are shown in Table 3. JHFed always has the best accuracy in the three imbalance levels. When the level of data non-IID decreases, JHFed still outperforms fedag and MIFA, and the advantage in accuracy is more obvious. The experiments prove the effectiveness and robustness of the method.

[0074] Table 3

[0075] 4) Aggregation time: To demonstrate the impact of different levels of complex aggregation, different global aggregation schemes were tried on IID CIFAR-10. Specifically, two different aggregation schemes were tried: i) Malicious Detection; ii) Malicious Detection + Security Aggregation. JHFed outperforms FedAvg in the number of client aggregations in the same time. In more complex aggregations, JHFed takes less time than FedAvg to aggregate the same number of times. The experiments show that JHFed has excellent acceleration performance under the same number of aggregations.

[0076] 5) Convergence performance: In this application, the key to achieving acceleration is to classify clients and perform local client connections. It would be an interesting starting point to study the impact of dynamic network topology and architecture on the final performance. Figure 4 and Figure 5 , the convergence curve of JHFed can be further analyzed. It can be observed that in the early stages of training, the proposed JHFed exhibits lower accuracy and stronger oscillations compared to FedAvg. However, as the number of communication rounds increases, JHFed eventually achieves similar performance to FedAvg. This is consistent with the intuition that the significant difference between dynamic and static network topologies decreases when the algorithm converges.

[0077] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0078] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0079] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for edge device data transmission based on a hybrid federated learning framework, characterized in that: The method comprises: Constructing a hybrid federated learning framework; the hybrid federated learning framework includes an edge client and a server; the server includes a client management module, a classifier, and a coordinator; obtaining a data set generated or collected by the edge client; Initialize internal parameters and a device fault prediction model for the edge client and the server, the edge client downloads configuration data from the server after submitting a registration request to the client management module, performs a current round of training on the device fault prediction model according to the configuration data and the data set, and uploads the updated model parameters to the server; The server globally aggregates the update models from the edge clients according to the aggregation strategy, and uses the classifier to classify the edge clients into stragglers, normal clients, and leaders according to the training speed of the edge clients; The coordinator receives the classification results of the edge client from the classifier and transforms the matching problem of the leader and the stragglers into the maximum weight problem in the bipartite graph to generate the optimal matching solution. The optimal matching solution is sent to the edge client through the client management module to build local decentralized federated learning for the locally aggregated leaders and stragglers, and the stragglers are temporarily removed from the central aggregation in the centralized federated learning to complete the current round of model training. Other edge clients resubmit registration applications to participate in a new round of training, and distribute new training parameters to edge clients participating in this round of training through centralized federated learning and aggregators in local decentralized federated learning until the device fault prediction model converges or reaches the number of training rounds, and a trained device fault prediction model is obtained.

2. The method according to claim 1, characterized in that The configuration data includes the number of training rounds, training model and batch size.

3. The method according to claim 1, characterized in that Using the classifier to classify the edge clients into stragglers, normal clients, and leaders according to the training speed of the edge clients, including: The classifier sets a training interval according to the average and long-term level of the historical upload time of the edge client; if the upload time of the current edge client uploading the available resources and task requirements to the server is greater than the training interval, the edge client is defined as a leader and added to the leader sequence; If the time taken by the current edge client to upload available resources and task requirements to the server is less than the training interval, the edge client is defined as a straggler and added to the stragglers sequence; If the time for the current edge client to upload available resources and task requirements to the server is equal to the training interval, the edge client is defined as normal.

4. The method according to claim 2, characterized in that: The classifier sets the training interval to be based on the average value of the edge client's historical upload time and the long-term level Where tol is a positive integer representing the tolerance value, t avg is the average historical upload time of the client, t now Indicates the long-term level.

5. The method according to claim 1, characterized in that The matching problem of leaders and stragglers is transformed into a maximum weight problem in a bipartite graph to generate the optimal matching solution, including: The matching problem between leaders and stragglers is transformed into a maximum weight problem in a bipartite graph, and the maximum weight problem is solved using a simulated annealing algorithm to obtain an optimal matching solution.

6. The method according to claim 4, characterized in that The matching problem of leaders and stragglers is transformed into the maximum weight problem in a bipartite graph, including Construct an xy coordinate system and use the x-axis to distinguish between leaders and laggards. On the x-axis, the stragglers Below the x-axis, A scaling value representing the number of client model updates. express and The vector sum of The length of approximates the leader and stragglers matching metric, express and The vector sum of The projection on the y-axis approximately represents the degree to which the data is not independent and identically distributed, and the projection on the x-axis approximately represents the number of times the client has uploaded the model; According to the relationship between each vector in the xy coordinate system, a matching strategy optimization target is adopted to transform the matching problem of leaders and stragglers into a maximum weight problem in a bipartite graph.

Images