Edge device data transmission method based on hybrid federated learning framework

Abstract

The present application relates to a method for edge device data transmission based on a hybrid federated learning framework. The method comprises: constructing a hybrid federated learning framework; the hybrid federated learning framework comprises an edge client and a server; the server comprises a client management module, a classifier and a coordinator, the classifier classifies the edge clients into stragglers, normal clients and leaders according to the training speed of each edge client; the coordinator combines multiple leaders into centralized federated learning to maintain fast model iteration. The stragglers intermittently return model parameters to the connected starters in the local decentralized federated learning to ensure model convergence, the coordinator converts the data Non-IID and the degree of client heterogeneity into leader and stragglers matching metrics, and the hybrid federated learning framework adjusts the coordinator and classifier in real time to adapt to complex and changing network environments. The use of this method can improve the efficiency of edge device data transmission.

Description

Technical Field

[0001] The present application relates to the field of Internet of Things technology, and in particular to an edge device data transmission method based on a hybrid federated learning framework. Background Art

[0002] In the Industrial Internet of Things (IIoT) scenario, edge devices in many factories, such as sensors and smart controllers, have limited computing power and energy resources. These devices need to participate in federated learning to optimize production processes and predict equipment failures, but the complex model training and aggregation processes can overwhelm these devices. Different edge clients (such as sensors and smart controllers) vary in hardware configuration, computing power, and network environment, resulting in client heterogeneity. This variability can lead to some clients training quickly and becoming leading clients in federated learning, while others, due to their limitations, may become lagging clients, much like the varying lengths of the planks in a barrel. During complex aggregation, the efficiency of federated learning often depends on the slowest-training lagging client, just as the water capacity of a barrel depends on the shortest plank. Accelerating federated learning training can reduce the time and resource consumption of training and data transmission. For example, in training equipment failure prediction models, edge devices can complete local training and parameter upload faster, reducing energy consumption, improving production efficiency, and reducing downtime caused by equipment failures, thereby enhancing the stability and economic benefits of industrial production. Summary of the Invention

[0003] Based on this, it is necessary to provide an edge device data transmission method based on a hybrid federated learning framework that can improve the data transmission efficiency of edge devices in response to the above technical problems.

[0004] A method for transmitting data from an edge device based on a hybrid federated learning framework, the method comprising:

[0005] Build a hybrid federated learning framework; the hybrid federated learning framework includes edge clients and servers; the server includes a client management module, a classifier, and a coordinator; obtain the data sets generated or collected by the edge clients;

[0006] Initialize the internal parameters and device fault prediction model of the edge client and server. After the edge client submits a registration request to the client management module, it downloads the configuration data from the server and trains the device fault prediction model for the current round based on the configuration data and dataset, and uploads the updated model parameters to the server.

[0007] The server globally aggregates the updated models from edge clients according to the aggregation strategy, and uses a classifier to classify edge clients into stragglers, normal clients, and leaders based on their training speed.

[0008] The coordinator receives the classification results from the edge client of the classifier and transforms the matching problem of leaders and stragglers into a maximum weight problem in a bipartite graph to generate the optimal matching solution. The client management module sends the optimal matching solution to the edge client to build local decentralized federated learning for the locally aggregated leaders and stragglers. In the centralized federated learning, the stragglers are temporarily removed from the central aggregation to complete the current round of model training.

[0009] Other edge clients resubmit their registration applications to participate in a new round of training, and distribute the new training parameters to the edge clients participating in this round of training through the aggregator in centralized federated learning and local decentralized federated learning until the device fault prediction model converges or the number of training rounds is reached, and a trained device fault prediction model is obtained.

[0010] The above-mentioned edge device data transmission method based on the hybrid federated learning framework, this application constructs a hybrid federated learning framework, uses a classifier to divide edge clients into stragglers, normal ones and leaders according to the training speed of edge clients, can quickly identify devices of different performance levels, and then uses a coordinator to accept the edge client classification results from the classifier and convert the matching problem of leaders and stragglers into a maximum weight problem in a bipartite graph to generate the optimal matching solution, and sends the optimal matching solution to the edge client through the client management module to build local decentralized federated learning for the locally aggregated leaders and stragglers, and temporarily remove the stragglers from the central aggregation in the centralized federated learning to complete the current round of model training. In the centralized federated learning link, the stragglers are temporarily removed from the central aggregation, so that normal ones and leaders can quickly advance global aggregation, greatly reducing waiting time. At the same time, by utilizing the idle period when the stragglers are removed, they can start learning locally with the help of local decentralized federated learning to improve their own capabilities. In addition, the coordinator transforms the matching of leaders and stragglers into a maximum weight problem in a bipartite graph, generating an optimal matching solution. The leader can then precisely assist stragglers based on the optimal matching solution, reducing invalid transmissions caused by poor coordination and improving data transmission accuracy. Finally, after the edge devices complete a round of training, devices that do not meet the standards reapply and participate in a new round of training. The aggregator promptly distributes the new parameters to the participating devices, allowing them to continuously obtain the latest model parameters, avoiding repeated training and invalid transmissions, ensuring that data transmission always serves efficient model training, and ultimately improving the overall data transmission efficiency of edge devices. This allows for faster completion of local training and parameter uploads, reducing equipment energy consumption, improving production efficiency, reducing downtime due to equipment failures, and improving the stability of industrial production. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 1. A flow chart of an edge device data transmission method based on a hybrid federated learning framework in one embodiment;

[0012] Figure 2 A schematic diagram of a hybrid federated learning framework in one embodiment;

[0013] Figure 3 A schematic diagram of a matching strategy optimization target in one embodiment;

[0014] Figure 4 This is a schematic diagram of the impact of different data sets and data distribution under the synchronous aggregation strategy in another embodiment;

[0015] Figure 5 FIG. 4 is an impact diagram of different data sets and data distributions under the asynchronous aggregation strategy in one embodiment. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0017] In one embodiment, Figure 1 As shown, a method for edge device data transmission based on a hybrid federated learning framework is provided, including the following steps:

[0018] Step 102: construct a hybrid federated learning framework; the hybrid federated learning framework includes an edge client and a server; the server includes a client management module, a classifier, and a coordinator; and obtain a data set generated or collected by the edge client.

[0019] Constructing a hybrid federated learning framework (JHFed) Figure 2 As shown, JHFed is built on two components: a classifier that categorizes clients into stragglers, runners, and leaders based on their training speed; and a coordinator that combines multiple leaders into a centralized federated learning (CFL) to maintain rapid model iteration. Stragglers intermittently return model parameters to connected starters in a local decentralized federated learning (DFL) to ensure model convergence. Furthermore, the coordinator converts data non-IID (non-independent and identically distributed) data and client heterogeneity into a leader-stragglers matching metric. JHFed adjusts the coordinator and classifier in real time to adapt to complex and changing industrial network environments.

[0020] In step 104, the internal parameters and the device failure prediction model of the edge client and the server are initialized. After the edge client submits a registration request to the client management module, it downloads the configuration data from the server and trains the device failure prediction model for the current round based on the configuration data and the data set, and uploads the updated model parameters to the server.

[0021] The edge client and server initialize their internal parameters and models. Subsequently, the edge client submits a registration request to the client management on the server. If the request is not accepted, the edge client waits for the next registration round. If the request is accepted, the edge client downloads configuration data from the server, including the number of training rounds, training model, batch size, and more.

[0022] The edge client performs model training on the current local dataset and uploads the updated model parameters to the server. The server globally aggregates the updated models from the edge clients according to the aggregation strategy (asynchronous aggregation, synchronous aggregation, and secure aggregation, etc.).

[0023] In step 106 , the server globally aggregates the updated models from the edge clients according to the aggregation strategy, and uses a classifier to classify the edge clients into stragglers, normal clients, and leaders according to their training speeds.

[0024] When the "barrel effect" waiting problem arises, the classifier maintains a sequence of stragglers and a sequence of leaders based on the edge client's local training time and the number of uploaded parameters, dividing the edge clients into leaders, stragglers, and normal clients. This classification quickly identifies devices of varying performance levels, laying the foundation for subsequent differentiated processing. Stragglers, due to limitations in computing power and network conditions, slow down overall progress. Accurately identifying them allows for targeted solutions, preventing fast devices from waiting in vain for extended periods and ensuring smooth data transmission.

[0025] In step 108, the coordinator receives the classification results of the edge client from the classifier and converts the matching problem of the leader and the stragglers into a maximum weight problem in a bipartite graph to generate the optimal matching solution. The optimal matching solution is sent to the edge client through the client management module to build local decentralized federated learning for the locally aggregated leaders and stragglers, and temporarily remove the stragglers from the central aggregation in the centralized federated learning to complete the current round of model training.

[0026] This application proposes a strategy that combines DFL (decentralized federated learning) with CFL (centralized federated learning), cleverly integrating it into the entire model training process. During the centralized federated learning phase, stragglers are temporarily removed from the central aggregation, allowing normal and leading clients to quickly advance global convergence, significantly reducing wait time. The coordinator utilizes the wait time caused by complex convergence to build the DFL (decentralized federated learning). During this process, the coordinator temporarily removes stragglers from the CFL (centralized federated learning). This measure is intended to prevent stragglers from burdening the global convergence while also fully utilizing wait time, allowing stragglers to learn locally rather than forcing other normal and leading clients to wait, thereby improving overall time efficiency.

[0027] The coordinator transforms the leader-straggler matching problem into a maximum weight problem in a bipartite graph. Combined with the client classification information provided by the classifier, it ensures that each straggler is matched with an appropriate leader. This optimal matching solution is then sent to the edge client through the client management module in the server. The classifier provides the classification basis, and the coordinator generates and delivers matching solutions based on this basis, allowing edge clients to build local decentralized federated learning based on the locally aggregated leaders and stragglers in the solution. Meanwhile, in centralized federated learning, stragglers are temporarily removed from the central aggregation to complete the current round of model training. Through the optimal matching solution, the leader can precisely assist stragglers according to the solution, reducing ineffective transmission caused by poor coordination and improving model training efficiency.

[0028] In step 110, other edge clients resubmit registration applications to participate in a new round of training, and distribute new training parameters to edge clients participating in this round of training through the aggregator in centralized federated learning and local decentralized federated learning until the device fault prediction model converges or reaches the number of training rounds, and a trained device fault prediction model is obtained.

[0029] After the edge client completes a round of training, the devices that do not meet the standards reapply and participate in a new round of training. The aggregator distributes the new parameters to the participating devices in a timely manner, and the devices can continuously obtain the latest model parameters. This iterative training and timely parameter distribution mechanism ensures that the edge devices can obtain the latest model parameters in a timely manner, reduces repeated training and invalid data transmission caused by untimely parameter updates, improves the effectiveness and efficiency of data transmission, makes the entire training process smoother and more efficient, and ultimately improves the overall data transmission efficiency of edge devices.

[0030] The above-mentioned edge device data transmission method based on the hybrid federated learning framework, this application constructs a hybrid federated learning framework, uses a classifier to divide edge clients into stragglers, normal ones and leaders according to the training speed of edge clients, can quickly identify devices of different performance levels, and then uses a coordinator to accept the edge client classification results from the classifier and convert the matching problem of leaders and stragglers into a maximum weight problem in a bipartite graph to generate the optimal matching solution, and sends the optimal matching solution to the edge client through the client management module to build local decentralized federated learning for the locally aggregated leaders and stragglers, and temporarily remove the stragglers from the central aggregation in the centralized federated learning to complete the current round of model training. In the centralized federated learning link, the stragglers are temporarily removed from the central aggregation, so that normal ones and leaders can quickly advance global aggregation, greatly reducing waiting time. At the same time, by utilizing the idle period when the stragglers are removed, they can start learning locally with the help of local decentralized federated learning to improve their own capabilities. In addition, the coordinator transforms the matching of leaders and stragglers into a maximum weight problem in a bipartite graph, generating an optimal matching solution. The leader can then precisely assist stragglers based on the optimal matching solution, reducing invalid transmissions caused by poor coordination and improving data transmission accuracy. Finally, after the edge devices complete a round of training, devices that do not meet the standards reapply and participate in a new round of training. The aggregator promptly distributes the new parameters to the participating devices, allowing them to continuously obtain the latest model parameters, avoiding repeated training and invalid transmissions, ensuring that data transmission always serves efficient model training, and ultimately improving the overall data transmission efficiency of edge devices. This allows for faster completion of local training and parameter uploads, reducing equipment energy consumption, improving production efficiency, reducing downtime due to equipment failures, and improving the stability of industrial production.

[0031] In one embodiment, the process of modeling a hybrid federated learning framework includes:

[0032] Assuming the computing resources of the edge client and storage resources Sufficient for local model training, with Sufficient for model transfer. Since the computing resources on the server are sufficient compared to the edge clients, they are not discussed in this paper. It is worth noting that the dataset of the edge client may change dynamically over time, but in a single round of training, the input of the model is constant. In a distributed network, the edge client dataset may change dynamically over time, and in a single training round, the input of the model is constrained to be constant. Define the classes contained in the dataset as Then use To define the Non-IID data.

[0033] .

[0034] When an edge client registers with the FL training framework, it uploads the available resources (computation, storage, and bandwidth) and task requirements (task type and deadline) to the server. The time taken for the above registration is recorded as:

[0035] ;

[0036] in Indicates that from the edge client Information uploaded to the server. Indicates the registration processing time on the server. Indicates that the server distributes initialization parameters to edge clients Then, the edge client Receive the global initial model and local iteration number After that, the model is trained to generate model parameters A round of customer training time is defined as:

[0037] ;

[0038] in Indicates the size of the machine learning task and the dataset Related to the training model. Therefore, The local iteration of the round is:

[0039] .

[0040] When using the synchronous aggregation method, the server waits for all edge clients to upload parameters before starting global aggregation. The transmission time to the server is defined as:

[0041] ;

[0042] in surface The size of the training parameters, express The bandwidth at time t.

[0043] Considering the diversity of aggregation methods, the aggregation time is generalized as After the global model aggregation is completed, the updated model is distributed from the server to the edge clients. This transmission time is defined as:

[0044] ;

[0045] When using asynchronous aggregation methods, the server can wait Each edge client uploads parameters, and then enables global aggregation. To express the waiting time of the aggregation:

[0046] ;

[0047] Compared with synchronous aggregation, the number of edge clients participating in asynchronous aggregation is becomes , When , the above formula represents the waiting time of synchronization aggregation. Therefore, the synchronization aggregation time is approximately defined as .

[0048] In asynchronous aggregation, the updated model will only be distributed to edge clients participating in the aggregation.

[0049] definition:

[0050] ;

[0051] is a binary Representation parameters From the edge client Passed to the server. Then you can get:

[0052] .

[0053] In DFL, edge clients can receive updated parameters from a neighbor to initiate model aggregation. Similar to CFL, using CFL's asynchronous aggregation computation method can reduce training time, aggregation time, and transmission time. Therefore, the JHFed problem is formulated as follows:

[0054] ;

[0055] ;

[0056] Among them, the first constraint indicates that the loss value of the loss function can converge, the second constraint ensures the resource constraint of consuming internal available resources, the third constraint limits the number of edge clients aggregated, and the fourth constraint ensures that edge clients participate in aggregation. In the above optimization problem, is the unknown variable to be solved.

[0057] In one embodiment, the configuration data includes the number of training rounds, the training model, and the batch size.

[0058] In one embodiment, a classifier is used to classify edge clients into stragglers, normal clients, and leaders based on their training speeds, including:

[0059] The classifier sets a training interval based on the average and long-term level of the edge client's historical upload time. If the current edge client's upload time to upload available resources and task requirements to the server is greater than the training interval, the edge client is defined as a leader and added to the leader sequence.

[0060] If the time it takes for the current edge client to upload available resources and task requirements to the server is less than the training interval, the edge client is defined as a straggler and added to the stragglers sequence;

[0061] If the time it takes for the current edge client to upload available resources and task requirements to the server is equal to the training interval, the edge client is defined as normal.

[0062] In a specific embodiment, clients are divided into three categories according to the number of training and upload times of the clients:

[0063] Stragglers: Since edge clients inevitably generate updates at different times, stragglers are common in scenarios such as mobile edge computing, unstable network connections, or edge client failures. In synchronous aggregation, since aggregation requires parameters from all edge clients, the server must wait for stragglers to complete before starting aggregation. In asynchronous aggregation, the updated parameters from stragglers are relatively old, which reduces the convergence efficiency of FL.

[0064] Leaders: This application proposes the concept of leaders as the opposite of stragglers. As global aggregation algorithms become more complex, their runtime can significantly increase, meaning edge clients may have to wait for the server to complete global aggregation. Leaders typically have sufficient computing and bandwidth resources or smaller datasets, enabling them to train models and upload faster. In synchronous aggregation, leaders need to wait for stragglers. In asynchronous aggregation, leaders tend to update more frequently.

[0065] Normals: Normals fall between leaders and stragglers, representing the average level of edge clients. Due to changes in datasets or resource availability, the status of normals may shift to other categories.

[0066] Specifically, the classifier maintains a list on the server to record the upload time:

[0067] ;

[0068] in Indicates the time when the server distributes the model to the edge client, Upload the updated model, Indicates the updated time. In JHFed, the classification results of the edge client will be sent to the coordinator for the formulation of DFL and adjustment of CFL.

[0069] Given the limitations of the communication channel on the server side, even if edge clients have similar resources, the updated parameters will arrive in a certain order. Frequently starting the classifier due to small time differences will result in significant overhead. Furthermore, please note that in the event of a disconnection or bandwidth drop, the communication delay may be extended, causing the classifier to mistakenly identify the edge client as a straggler. To reduce the probability of such events, this application defines:

[0070] ;

[0071] in Is a positive integer representing the tolerance value. is the average historical upload time of edge clients, Indicates the long-term level. It is the upload time of the edge client in the current round, which represents the short-term level. When the interval is greater than this, this application defines the edge client as a leader and adds it to the leader sequence. When uploading time When the interval is less than this, the application defines the edge client as a straggler and adds it to the stragglers sequence. In order to avoid frequent classifier startup, this application proposes a threshold with a difference of only the number of updates. When the value is greater than this threshold, the classifier will be triggered.

[0072] In one embodiment, the classifier sets the training interval based on the average and long-term level of the historical upload time of the edge client:

[0073] ;

[0074] in, is a positive integer representing the tolerance value, is the average of the client's historical upload time, Indicates the long-term level.

[0075] In one embodiment, the leader and straggler matching problem is transformed into a maximum weight problem in a bipartite graph, including

[0076] Construct an xy coordinate system and use the x-axis to distinguish between leaders and laggards. Above the x-axis, the stragglers Below the x-axis, A scaling value representing the number of client model updates, express and The vector sum of The length of approximately represents the leader and straggler matching metric, express and The vector sum of The projection on the y-axis approximately represents the degree to which the data is not independent and identically distributed, and the projection on the x-axis approximately represents the number of times the client has uploaded the model;

[0077] According to the relationship between each vector in the xy coordinate system, the matching strategy optimization objective is adopted to transform the matching problem of leaders and stragglers into the maximum weight problem in a bipartite graph.

[0078] In a specific embodiment, after the classifier determines the client category, the coordinator removes the stragglers from the CFL and then connects the stragglers with the leaders to build the DFL. When there is a leader but no stragglers, JHFed establishes a connection between the leaders but does not remove them from the CFL. Figure 3 As shown in Figure 3, there are various DFL schemes for leaders and stragglers. Considering that Non-IID and heterogeneous clients mainly affect the convergence efficiency, this application adopts a matching strategy to find a solution that is more conducive to global training convergence. Figure 3 middle is the drift angle, which is calculated by the difference between the updated client and global parameter vectors. It is scaled from the updated values ​​in client 4. The x-axis is used to distinguish between leaders and stragglers. Above the x-axis, the stragglers Below the x-axis, A scaling value representing the number of client model updates, express and The vector sum of The length of approximately represents the leader and straggler matching metric, express and The vector sum of The projection on the y-axis approximately represents the degree to which the data is not independent and identically distributed, and the projection on the x-axis approximately represents the number of model uploads by the client. Inspired by graph theory, the above problem is transformed into a maximum weight problem in a bipartite graph. In actual scenarios, the number of clients at the edge of the network will increase from hundreds to thousands. Existing methods, such as the KM algorithm, need to traverse all nodes to find the optimal solution, so any brute force-based search method is prohibited. In order to solve the above dilemma, this application uses an approximate calculation scheme to reduce the computational complexity. This application uses a simulated annealing algorithm to approximate the optimal solution.

[0079] In one embodiment, in order to analyze the feasibility of the proposed model training framework, the convergence of the proposed JHFed method is introduced. There is a set of standard routines for convergence analysis of federated learning, so we follow it and make the following four widely used assumptions:

[0080] Assumption 1. ) Assume that the loss function f on the client is and for , :

[0081] ;

[0082] Assumption 2. (Strong Convexity) Assume that the loss function f on the client is , $\theta>0$ :

[0083] ;

[0084] Assumption 3. (Bounded Gradient) The squared norm of the stochastic gradient is uniformly bounded.

[0085] ;

[0086] where ξ represents a sample of the client's local dataset.

[0087] Assumption 4. (Global Optimum Exists) Assume that the loss function has a global minimum .

[0088] In the synchronous JHFed scheme, when a leader and a stragglers emerge, the stragglers are removed from the CFL and merged with the leader to form the DFL. JHFed passes the information of the stragglers to the leader, compensates the slower updates with faster updates, and converges back to the CFL after several rounds.

[0089] Theorem 1. Assume that the client uses an unbiased sampling scheme. , represents the learning rate. According to the above assumptions, After rounds of iterations, we have:

[0090] ;

[0091] in , express The model parameters after round of update, are the initial model parameters.

[0092] Proof: Without loss of generality, for each global round in synchronous JHFed, ,

[0093] in Denotes the number of clients participating in the aggregation. Let ,use Smoothness and Strong convexity, we have:

[0094] ;

[0095] in .

[0096] In asynchronous JHFed, the server receives After the model parameters of each client are updated, global aggregation can be performed. ,The optimality gap of asynchronous JHFed is similar to that of synchronous JHFed, but . Note that JHFed converges to bounds and Non-IID data Spend, Clients and rounds With The increase, Increasing the value of leads to looser convergence bounds. Bigger and When is smaller, the bound of Theorem 1 decreases.

[0097] Remark 1. Theorem 1 shows that for θ-convex, L-smooth and -Non-IID data, JHFed The convergence rate reaches the global optimum after T rounds of iterations. The time spent in each round primarily consists of training, transmission, and aggregation. Due to complex aggregation, client heterogeneity, and non-IID data, these three sequential steps may block and incur additional waiting time. Based on this intuition, in synchronous aggregation, JHFed performs more local client aggregations within a fixed timeframe, reducing the time required to converge to the same boundary. In asynchronous aggregation, JHFed manipulates clients to jointly aggregate during the waiting time, completing T rounds of iterations faster and accelerating convergence.

[0098] Simulation experiments were conducted on three classic open datasets: MNIST, FMNIST, and CIFAR-10. MNIST consists of 70,000 28×28 grayscale handwritten images, divided into 10 classes, with a training set of 60,000 samples and a test set of 10,000 samples. FashionMNIST has the same number of samples, classes, and training / test splits as MNIST, but uses images of clothing items instead of handwritten digits. CIFAR-10 consists of 60,000 32×32 color images, labeled into 10 classes, with 50,000 samples used for training and 10,000 for testing.

[0099] Two classic models were used to train the dataset. One is LeNet-5, a deep convolutional neural network for the MNIST dataset. LeNet-5 consists of two 5 × 5 convolutional layers, a fully connected layer (120 units), a fully connected layer (84 units), and a softmax output layer (10 units). LeNet-55 uses the backpropagation algorithm to calculate gradients and optimize the network weights and biases by minimizing the error function. The other is ResNet-18 for FMNIST and CIFAR-10. ResNet-18 consists of 5 convolutional groups and 18 weight layers. This paper introduces SkipConnections to address degradation issues, significantly easing the difficulty of training deep neural networks.

[0100] The number of aggregation rounds on the server was set to 100. Furthermore, malicious client identification was added to simulate complex aggregation scenarios. Assuming the number of local aggregation rounds on each client was 2, the batch size of the dataset was 128, the learning rate was 0.1, and decay stopped after 50 rounds. In the IID experiments, all datasets were evenly distributed across clients. In the Non-IID case, the datasets were partitioned based on the degree of Non-IID distribution. During training, clients used cross-entropy (CE) as the loss function and SGD with momentum as the optimization method. To better evaluate the performance of JHFed, we conducted comparative experiments using the following three types of benchmark algorithms: FedAvg, MIFA, and JHFed-R.

[0101] Numerical results:

[0102] 1) Synchronous Aggregation: Six sets of experiments were conducted to compare the performance of the proposed method with the baseline. Table 1 shows the accuracy comparison of synchronous FedAvg and our JHFed on three datasets (with both IID and non-IID data partitions). The best results are indicated in bold. Since MIFA is an asynchronous aggregation scheme, it is excluded here. In Table 1, Accuracy represents the classification accuracy of the final model on the test set, that is, the ratio of the number of correctly classified samples to the total number of samples. Time represents the total time from scheduling the model to the end of training, including aggregation, local training, data loading, etc. Speedup is measured per global clock and normalized by the runtime of FedAvg.

[0103] JHFed exhibits the best performance in the synchronous aggregation scheme. Across the three IID datasets, JHFed achieves higher accuracy than FedAvg. However, there is no significant difference between JHFed and FedAvg because JHFed's mechanism, which takes into account the advantages of Non-IID, fails to materialize, and the impact of client heterogeneity is difficult to avoid under the synchronous aggregation scheme. In contrast, JHFed demonstrates better performance in the Non-IID setting, achieving 10% higher accuracy than FedAvg on the CIFAR-10 dataset. Compared to JHFed, JHFed-R employs a random matching generation strategy, resulting in greater accuracy fluctuations across the three datasets. On the Non-IID CIFAR-10, JHFed-R achieves higher accuracy than FedAvg, but lower on the IID CIFAR-10. These experimental results demonstrate JHFed's stability and the superiority of its near-optimal matching strategy.

[0104] Table 1

[0105]

[0106] Due to the synchronous aggregation scheme used by the server, JEDFED's runtime speedup is not significant. On the Non-IID CIFAR-10 dataset, JEDFED achieves a 1.07× speedup compared to FedAvg, while the random match generation strategy results in JEDFED-R with a similar training time to FedAvg. Figure 4 The accuracy of the algorithms on three datasets is shown in detail. In the synchronous scenario, all converged with similar efficiency, but when they eventually stabilized, JHFed achieved better accuracy. Unbalanced datasets can provide misleading results compared to IID data. Consequently, the NonIID data exhibited greater fluctuations during convergence.

[0107] 2) Asynchronous Aggregation: Table 2 shows the accuracy comparison between the asynchronous FedAvg and our JHFed on three datasets with both IID and non-IID data partitions. Setting the number of asynchronous joins to 1 allows the server to initiate global aggregation after receiving an update from a client. The differences in accuracy and runtime among the four algorithms are more significant in the asynchronous aggregation scheme. On IID data, JHFed achieves a nearly 3% improvement in accuracy over FedAvg. On the non-IID CIFAR10 dataset, JHFed demonstrates an even greater accuracy advantage, leading FedAvg by 4.5% and MIFA by 2%, respectively. In the FedAvg scheme, some stragglers are removed, resulting in a lack of data during training and, consequently, lower accuracy on the test set. JHFed-R, under the random match generation strategy, becomes more unstable in the asynchronous aggregation scheme. Its accuracy is lower than FedAvg on all datasets except the non-IID MNIST dataset, but its runtime is faster. MIFA corrects the backward bias and outperforms FedAvg and JHFed-R in terms of accuracy and runtime, but is inferior to JHFed. In asynchronous aggregation, JHFed shows the best convergence efficiency performance, with a speedup of 1.57× compared to FedAvg.

[0108] Table 2

[0109]

[0110] Figure 5 details the algorithm's training process on three datasets. Compared to the synchronous approach, fluctuations during convergence are more pronounced due to the increased amount of non-IID data. When stragglers act as parameter update providers in a round, the uploaded parameters may contain a portion of the stragglers, exacerbating fluctuations. Overall, the classifier and coordinator matching strategy in JHFed demonstrates superior performance, with training results stabilizing in the post-iteration phase. In a distributed scenario with non-IID data and client heterogeneity, testbed experiments demonstrate that JHFed can achieve superior results in terms of accuracy and training efficiency, validating the theoretical results.

[0111] 3) Non-IID Degree: We studied the impact of data heterogeneity on CIFAR-10 by varying the non-IID degree parameter κ. κ is the 0-1 normalized version of Γ. A larger κ results in greater data non-IID imbalance. The results are shown in Table 3. JHFed consistently achieves the best accuracy across the three imbalance levels. When the level of data non-IID decreases, JHFed continues to outperform fedag and MIFA, with a more pronounced advantage in accuracy. Experiments demonstrate the effectiveness and robustness of this approach.

[0112] Table 3

[0113]

[0114] 4) Aggregation Time: To demonstrate the impact of varying aggregation complexity, we experimented with different global aggregation schemes on the IID CIFAR-10 dataset. Specifically, we tested two different aggregation schemes: i) malicious detection; ii) malicious detection plus secure aggregation. JHFed outperformed FedAvg in the number of client aggregations achieved in the same amount of time. For more complex aggregations, JHFed achieved the same number of aggregations in less time than FedAvg. Experiments show that JHFed offers superior acceleration performance when the number of aggregations is the same.

[0115] 5) Convergence performance: In this application, the key to achieving acceleration is to classify clients and perform local client connections. Studying the impact of dynamic network topology and architecture on the final performance will be an interesting starting point. Figure 4 and Figure 5 , we can further analyze the convergence curve of JHFed. We can observe that in the early stages of training, the proposed JHFed exhibits lower accuracy and stronger oscillations compared to FedAvg. However, as the number of communication rounds increases, JHFed eventually achieves similar performance to FedAvg. This is consistent with the intuition that the significant difference between dynamic and static network topologies decreases as the algorithm converges.

[0116] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0117] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0118] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and such modifications and improvements are intended to fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for edge device data transmission based on a hybrid federated learning framework, characterized in that: The method comprises: Constructing a hybrid federated learning framework; the hybrid federated learning framework includes an edge client and a server; the server includes a client management module, a classifier, and a coordinator; obtaining a data set generated or collected by the edge client; Initializing internal parameters and a device failure prediction model for the edge client and server, the edge client submitting a registration request to the client management module, downloading configuration data from the server, training the device failure prediction model for a current round based on the configuration data and the data set, and uploading the updated model parameters to the server; The server globally aggregates the updated models from the edge clients according to the aggregation strategy, and uses the classifier to classify the edge clients into stragglers, normal clients, and leaders according to the training speed of the edge clients; The coordinator receives the classification results from the edge client of the classifier and transforms the matching problem of leaders and stragglers into a maximum weight problem in a bipartite graph to generate the optimal matching solution. The client management module sends the optimal matching solution to the edge client to build local decentralized federated learning for the locally aggregated leaders and stragglers. In the centralized federated learning, the stragglers are temporarily removed from the central aggregation to complete the current round of model training. Other edge clients resubmit registration applications to participate in a new round of training. The new training parameters are distributed to the edge clients participating in this round of training through the aggregator in centralized federated learning and local decentralized federated learning until the device fault prediction model converges or the number of training rounds is reached, resulting in a trained device fault prediction model. The matching problem of leaders and stragglers is transformed into a maximum weight problem in a bipartite graph, including Construct an xy coordinate system and use the x-axis to distinguish between leaders and laggards. Above the x-axis, the stragglers Below the x-axis, A scaling value representing the number of client model updates, express and The vector sum of The length of approximately represents the leader and straggler matching metric, express and The vector sum of The projection on the y-axis approximately represents the degree to which the data is not independent and identically distributed, and the projection on the x-axis approximately represents the number of times the client has uploaded the model; According to the relationship between each vector in the xy coordinate system, a matching strategy optimization objective is adopted to transform the matching problem of leaders and stragglers into a maximum weight problem in a bipartite graph.

2. The method according to claim 1, characterized in that The configuration data includes the number of training rounds, training model, and batch size.

3. The method according to claim 1, characterized in that Using the classifier to classify the edge clients into stragglers, normal clients, and leaders according to the training speed of the edge clients, including: The classifier sets a training interval based on the average and long-term level of the edge client's historical upload time; if the upload time of the current edge client to upload available resources and task requirements to the server is greater than the training interval, the edge client is defined as a leader and added to the leader sequence; If the time it takes for the current edge client to upload available resources and task requirements to the server is less than the training interval, the edge client is defined as a straggler and added to a stragglers sequence; If the time taken by the current edge client to upload available resources and task requirements to the server is equal to the training interval, the edge client is defined as normal.

4. The method according to claim 2, characterized in that The classifier sets the training interval to be based on the average value and long-term level of the historical upload time of the edge client. in, is a positive integer representing the tolerance value, is the average of the client's historical upload time, Indicates the long-term level.

5. The method according to claim 1, wherein The leader and straggler matching problem is transformed into a maximum weight problem in a bipartite graph to generate the optimal matching solution, including: The matching problem between leaders and stragglers is transformed into a maximum weight problem in a bipartite graph, and the maximum weight problem is solved using a simulated annealing algorithm to obtain the optimal matching solution.

Images