Federal learning convergence acceleration method under unbalanced data distribution of multiple terminal devices

Abstract

The invention discloses a federated learning convergence acceleration method under unbalanced data distribution of multiple terminal devices, and relates to the technical field of federated learning convergence acceleration of the multiple terminal devices, comprising the following steps: collecting the real-time data volume of each terminal device sample by using a network probe; the method comprises the following steps: acquiring a scheduling timestamp and an updating global model timestamp of data of a central server through Zabbix software, distributing training strategies of a terminal equipment large sample and a combined sample according to a data volume level of a terminal equipment sample, selecting a global model updating mode according to waiting delay time of the central server, and updating the global model according to the waiting delay time of the central server. According to the method, a network probe acquisition technology, a Zabbix software acquisition technology, a data enhancement technology, a generative adversarial network technology and a random forest algorithm and Bagging algorithm fusion technology are closely combined with a modern information technology, and the intelligent degree in the process of the federated learning convergence acceleration method of multiple terminal devices is remarkably enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of accelerating the convergence of federated learning for multi-terminal devices, and specifically relates to a method for accelerating the convergence of federated learning under the imbalance of data distribution among multi-terminal devices. Background Art

[0002] In today's digital age, the amount of data generated by multi-terminal devices has increased explosively. These devices include smartphones, Internet of Things sensors, smart wearable devices, etc. However, there are significant differences in the quantity, characteristics, and distribution of data collected by different terminal devices, that is, the data distribution is unbalanced. As an emerging distributed machine learning technology, federated learning allows each terminal device to jointly train a model without sharing the original data. However, when faced with the situation of unbalanced data distribution among multi-terminal devices, traditional federated learning algorithms are prone to problems such as slow convergence speed and low model accuracy. In order to give full play to the advantages of federated learning and improve the model training efficiency, it is particularly important to develop a method for accelerating the convergence of federated learning under the unbalanced data distribution of multi-terminal devices;

[0003] Although there have been great progresses in the direction of methods for accelerating the convergence of federated learning under the unbalanced data distribution of multi-terminal devices in the prior art, there are still some problems to be optimized. The data of terminal devices is unbalanced in terms of data volume. The traditional federated average algorithm cannot allocate resources according to the differences in the data volume of multi-terminal devices. Moreover, the differences in the allocation of data volume resources of terminal devices cause inconsistent waiting delay times of the central server, which in turn leads to a slow global model update speed and affects the acceleration of federated learning convergence. Summary of the Invention

[0004] To achieve the above objectives, the present invention is realized through the following technical solutions: A method for accelerating the convergence of federated learning under the unbalanced data distribution of multi-terminal devices, including the following steps:

[0005] Step 1: Use network probes to collect the real-time data volume of each terminal device sample, and collect the scheduling timestamp and the global model update timestamp of the central server data through Zabbix software, providing data support for the subsequent steps;

[0006] Step 2: Set a data volume threshold for the terminal device samples, and divide the data volume levels of each terminal device sample. Among them, the divided levels are large samples and small samples, providing a guarantee for the subsequent allocation of training strategies for the samples in the terminal devices;

[0007] Step 3: Adopt data augmentation technology to initially process the real-time data volume of the terminal device samples, and obtain synthetic samples similar to the small samples through a generative adversarial network, and preprocess the synthetic samples and the large samples;

[0008] Step 4: Allocate the training strategies for large samples and combined samples of the terminal device according to the data volume level of the terminal device sample, which solves the problem that the traditional federated averaging algorithm cannot allocate resources according to the data volume differences of multiple terminal devices;

[0009] Step 5: Establish an MEC heterogeneous data system model, set asynchronous thread scheduling for the central server, calculate the waiting delay time of the central server, set the waiting delay time threshold of the central server, calculate the waiting time exceeding the expectation of the central server, which lays a foundation for subsequent selection of the global model update method;

[0010] Step 6: Select the global model update method according to the waiting delay time of the central server;

[0011] Step 7: Respectively construct a data volume scheduling model and a waiting delay time scheduling model through the random forest algorithm, combine the random forest algorithm with the Bagging algorithm, fuse the data volume scheduling model and the waiting delay time scheduling model, and obtain a federated learning convergence acceleration task allocation model, which solves the problem that the inconsistent waiting delay time of the central server leads to a slow global model update speed.

[0012] A further improvement of the technical solution of the present invention lies in: in the said Step 1, the process of collecting the real-time data volume of each terminal device sample and the scheduling timestamp and global model update timestamp of the central server data includes:

[0013] Extract samples from each terminal device, and use a network probe to collect the real-time data volume of each terminal device sample; download Zabbix software on the central server to measure the scheduling timestamp and global model update timestamp of the central server.

[0014] A further improvement of the technical solution of the present invention lies in: in the said Step 2, the process of setting the data volume threshold of the terminal device sample and dividing the data volume levels of each terminal device sample includes:

[0015] Set the data volume threshold of the terminal device sample. According to the Pareto principle, when the real-time data volume of the terminal device sample is greater than the data volume threshold of the terminal device sample, the data volume of the terminal device sample at this moment is classified as a large sample, and when the real-time data volume of the terminal device sample is less than the data volume threshold of the terminal device sample, the data volume of the terminal device sample at this moment is classified as a small sample.

[0016] A further improvement of the technical solution of the present invention lies in: in the said Step 3, adopt a data augmentation technique to preprocess the real-time data volume of the terminal device sample, and obtain a combined sample through a generative adversarial network. The process of preprocessing the combined sample and the large sample includes:

[0017] Set data augmentation technical parameters, select the data augmentation technology corresponding to the data augmentation technical parameters, and perform preliminary processing on the real-time data volume of the terminal device samples;

[0018] The generative adversarial network consists of a generator, a discriminator, and a loss function. Initialize the neural network parameters of the generator and the discriminator. The generator synthesizes synthetic samples similar to the small samples, inputs the small samples and the synthetic samples into the discriminator, calculates the loss function of the discriminator, obtains the determination result of the synthetic samples, and updates the discriminator parameters using the backpropagation algorithm; According to the judgment result of the synthetic samples, calculate the loss function of the generator, and update the generator parameters using the backpropagation algorithm. Repeat the alternating training of the generator and the discriminator. Through multiple rounds of iteration, obtain synthetic samples similar to the small samples, and combine the small samples and the synthetic samples to obtain combined samples;

[0019] Perform data cleaning on the combined samples and the large samples, remove missing values, outliers, and duplicate values in the combined samples and the large samples, and perform standardization processing on the combined samples and the large samples after data cleaning.

[0020] A further improvement of the technical solution of the present invention lies in: In the fourth step, the process of allocating the training strategies of the terminal device large samples and the combined samples according to the data volume level of the terminal device samples includes:

[0021] Allocate the training strategies of the terminal device large samples and the combined samples according to the data volume level of the terminal device samples. For the combined samples, use the FedAvg algorithm to perform local gradient descent on the combined samples; For the large samples, based on the MEC network of the FedAvg algorithm, select the combined samples for training, perform stochastic gradient descent calculation on the combined samples, evaluate the improvement of the combined samples on the target loss function, and gradually increase the combined samples according to the improvement of the combined samples on the target loss function until the gradient descent condition of the large samples is met, and then use the FedAvg algorithm to perform local gradient descent on the large samples.

[0022] A further improvement of the technical solution of the present invention lies in: In the fourth step, the principle of the FedAvg algorithm includes:

[0023] The FedAvg algorithm uses the parameter ω to perform gradient descent calculation on the prediction function, uses the terminal device sample dataset to establish a subset of terminal device samples, selects the size of the small samples in the subset of terminal device samples, and the process of calculating the minimization of the target loss function is as follows:

[0024]

[0025] Among them, is the target loss function, is the size of the small samples in the subset of terminal device samples, is a subset of terminal device samples, is a prediction function calculated by gradient descent through parameter ω;

[0026] Determine the gradient descent condition and evaluate the improvement of the minimized objective loss function. If the improvement remains unchanged, select terminal device samples with the same sample size as to perform the next iteration; if the improvement changes, reselect samples larger than and perform the next iteration until the gradient descent condition is met.

[0027] A further improvement of the technical solution of the present invention lies in: in step five, establish an MEC heterogeneous data system model, set asynchronous thread scheduling for the central server, calculate the waiting delay time of the central server, and set the waiting delay time threshold of the central server. The process of calculating the waiting time exceeding the expected waiting time of the central server includes:

[0028] A1. The MEC heterogeneous data system model consists of a central server and k MEC terminal nodes. Among them, each MEC terminal node includes a terminal device and its sample data set, and a federated learning training model is deployed on each MEC terminal node;

[0029] A2. The central server consists of a scheduling thread, an update thread, and a coordinator. The tasks of the scheduling thread and the update thread are controlled by two asynchronous parallel threads. Among them, the coordinator sends scheduling instructions and the global model to the MEC terminal devices at the same time interval. When the terminal device finishes its work, it uploads the updated global model parameters, obtains the upload queue of the global model, and updates the global model according to the upload queue of the global model;

[0030] A3. The process of calculating the waiting delay time of the central server is as follows:

[0031]

[0032] Among them, is the waiting delay time of the central server, is the timestamp of the central server updating the global model, is the scheduling timestamp of the central server;

[0033] A4. Set the waiting delay time threshold of the central server. When the waiting delay time of the central server is lower than the waiting delay time threshold of the central server, obtain the waiting time exceeding the expected waiting time of the central server by calculating the difference between the waiting delay time of the central server and the waiting delay time threshold of the central server.

[0034] A further improvement of the technical solution of the present invention lies in: in the sixth step, the process of selecting the global model update method according to the waiting delay time of the central server includes:

[0035] The global model update methods include the time weighted average method and the weighted asynchronous aggregation method;

[0036] When the waiting delay time of the central server is greater than or equal to the waiting delay time threshold of the central server, the waiting delay time of the central server is long, and the time weighted average method is used to calculate the mixed hyperparameters of the t-th round of iteration, combined with the weighted asynchronous aggregation method, to obtain the global model;

[0037] When the waiting delay time of the central server is lower than the waiting delay time threshold of the central server, the waiting delay time of the central server is short, and the global model is obtained through the time weighted average method;

[0038] A further improvement of the technical solution of the present invention lies in: in the sixth step, the principles of obtaining the global model by the time weighted average method and the weighted asynchronous aggregation method include:

[0039] B1. Using the hinge function to represent the delay function, and adopting the time weighted average method, the process of calculating the mixed hyperparameters of the t-th round of iteration is as follows:

[0040]

[0041]

[0042] Wherein, is the delay function, is the mixed hyperparameter of the t-th round of iteration, a and b are constants, and a>0, b>0, is the waiting time exceeding the expectation of the central server, is the waiting delay time threshold of the central server;

[0043] In the t-th round of iteration, the central server receives the local training model parameters fed back by the terminal device, and uses the mixed hyperparameters of the t-th round of iteration and the local training model parameters fed back by the terminal device received by the central server to calculate the global model as follows:

[0044]

[0045] Wherein, is the global model obtained by the time weighted average method in the t-th round of iteration; is the global model obtained by the time weighted average method in the (t-1)-th round of iteration, is the mixed hyperparameter of the t-th round of iteration, is the local training model parameter fed back by the terminal device received by the central server;

[0046] B2. Based on the weighted asynchronous aggregation method, set the weight parameters of each terminal device scheduled in the t-th round of iteration. The process of calculating the weighted asynchronous aggregation to update the global model is as follows:

[0047]

[0048]

[0049] Among them, is the global model obtained by the weighted asynchronous aggregation method in the t-th round of iteration, is the global model obtained by the weighted asynchronous aggregation method in the (t - 1)-th round of iteration, is the weight hyperparameter in the t-th round of iteration, is the weight parameter of each terminal device scheduled in the t-th round of iteration, is the hybrid hyperparameter in the t-th round of iteration, is the local training model parameter fed back by the terminal device received by the central server.

[0050] A further improvement of the technical solution of the present invention lies in: in step seven, the process of constructing a data volume scheduling model and a waiting delay time scheduling model, and fusing the data volume scheduling model and the waiting delay time scheduling model to obtain a federated learning convergence acceleration task allocation model includes:

[0051] S1. Use the random forest algorithm, set the random forest model parameters, construct three random forest models, and assign the numbers A model, B model, and C model to the three random forest models;

[0052] S2. Take the data volume of the terminal device samples and their corresponding training strategies for large samples and synthetic samples of the terminal devices as a data set, divide it into a training set and a test set according to a ratio of 7:3, use the training set data to iteratively train the A model, obtain the non-linear relationships between the data volume of the terminal device samples and the training strategies for large samples and synthetic samples respectively, so that when inputting the data volume of the terminal device samples, the A model outputs the training strategies for large samples and synthetic samples respectively. Input the test set data into the A model, compare the training strategies for large samples and synthetic samples output by the A model with the actual training strategies for large samples and synthetic samples, evaluate the performance of the trained A model, adjust the A model parameters, optimize the A model, and obtain the data volume scheduling model;

[0053] S3. Use the central server waiting delay time and its corresponding global model update method as a data set, divide it into a training set and a test set according to a ratio of 7:3, use the training set data to iteratively train the B model, obtain the non-linear relationship between the central server waiting delay time and its corresponding global model update method, realize inputting the central server waiting delay time, and the B model outputs the global model update method corresponding to the central server waiting delay time. Input the test set data into the B model, compare the global model update method corresponding to the central server waiting delay time output by the B model with the global model update method corresponding to the actual central server waiting delay time, evaluate the performance of the trained B model, adjust the parameters of the B model, optimize the B model, and obtain the waiting delay time scheduling model;

[0054] S4. Define the model fusion environment of the data volume scheduling model and the waiting delay time scheduling model. Through the interaction of the data volume scheduling model and the waiting delay time scheduling model fusion environment, generate Bagging data, divide the Bagging data into Bagging training data and Bagging test data, use the Bagging training data to train the data volume scheduling model and the waiting delay time scheduling model respectively, obtain the Bagging base model. In the test stage, input the Bagging test data into the Bagging base model to obtain the prediction results of the Bagging base model, fuse all the prediction results of the Bagging base models to obtain the fused Bagging base model prediction results, and calculate the average value of the fused Bagging base model prediction results;

[0055] Use the Bagging data and the average value of the corresponding fused Bagging base model prediction results as a data set, divide it into a training set and a test set according to a ratio of 7:3, use the training set data to iteratively train the C model, obtain the non-linear relationship between the Bagging data and the average value of the fused Bagging base model prediction results, realize inputting the Bagging data, and the C model outputs the average value of the fused Bagging base model prediction results corresponding to the Bagging data. Input the test set data into the C model, compare the average value of the fused Bagging base model prediction results corresponding to the Bagging data output by the C model with the average value of the fused Bagging base model prediction results corresponding to the actual Bagging data, evaluate the performance of the trained C model, adjust the parameters of the C model, optimize the C model, and obtain the federated learning convergence acceleration task allocation model.

[0056] The beneficial effects of the present invention are as follows: For the method for accelerating the convergence of federated learning under unbalanced data distribution of multi-terminal devices, compared with the traditional method for accelerating the convergence of federated learning under unbalanced data distribution of multi-terminal devices, the network probe acquisition technology and Zabbix software acquisition technology in the method of the present invention are closely combined with modern information technology to accurately capture the real-time data volume of terminal device samples and the scheduling timestamp and global model update timestamp data of the central server data. Combining data augmentation technology, generative adversarial network technology and data preprocessing technology, the collected data is processed, the data volume threshold of terminal device samples is set, the data volume levels of each terminal device sample are divided, and according to the data volume levels of terminal device samples, the training strategies of terminal device large samples and combined samples are allocated, achieving real-time and comprehensive monitoring of the data volume and its training strategies in the process of federated learning. The waiting delay time threshold of the central server is set, the waiting time exceeding the expectation of the central server is calculated, and according to the waiting delay time of the central server, the global model update method is selected. By adopting the fusion technology of random forest algorithm and Bagging algorithm, real-time and comprehensive monitoring of the impact of waiting delay time on model update in the process of federated learning is achieved. It solves the problems that the data of terminal devices is unevenly distributed in terms of data volume, the traditional federated average algorithm cannot allocate resources according to the data volume differences of multi-terminal devices, and due to the differences in data volume resource allocation of terminal devices, the waiting delay time of the central server is inconsistent, resulting in a slow global model update speed and affecting the acceleration of federated learning convergence. It ensures that the method in the present invention can refine the dynamic monitoring standard for the method of accelerating the convergence of federated learning under unbalanced data distribution of multi-terminal devices within a more accurate range, making the monitored data more accurate indicators under the same conditions. The research and application of this method significantly enhance the degree of intelligence in the process of the method for accelerating the convergence of federated learning of multi-terminal devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings described below are only some embodiments recorded in the present invention, and those of ordinary skill in the art can also obtain other drawings based on these drawings.

[0058] Figure 1 It is a flowchart of the method for accelerating the convergence of federated learning under unbalanced data distribution of multi-terminal devices of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0060] As Figure 1 shown, the present invention provides a method for accelerating the convergence of federated learning under unbalanced data distribution of multi-terminal devices, which consists of the following steps:

[0061] Step 1: Use network probes to collect the real-time data volume of each terminal device sample, and collect the scheduling timestamp and global model update timestamp of the central server data through Zabbix software, providing data support for the subsequent steps;

[0062] Step 2: Set the data volume threshold of the terminal device sample, and divide the data volume level of each terminal device sample. Among them, the divided levels are large samples and small samples, providing a guarantee for the subsequent distribution of the training strategies of the samples in the terminal device;

[0063] Step 3: Adopt data augmentation technology to initially process the real-time data volume of the terminal device sample. Through a generative adversarial network, obtain synthetic samples similar to the small samples, and preprocess the synthetic samples and large samples;

[0064] Step 4: According to the data volume level of the terminal device sample, allocate the training strategies of the large samples and combined samples of the terminal device, solving the problem that the traditional federated averaging algorithm cannot allocate resources according to the data volume difference of multi-terminal devices;

[0065] Step 5: Establish a MEC heterogeneous data system model, set asynchronous thread scheduling for the central server, calculate the waiting delay time of the central server, set the waiting delay time threshold of the central server, and calculate the waiting time exceeding the expectation of the central server, laying a foundation for the subsequent selection of the global model update method;

[0066] Step 6: Select the global model update method according to the waiting delay time of the central server;

[0067] Step 7: Through the random forest algorithm, construct a data volume scheduling model and a waiting delay time scheduling model respectively. Combine the random forest algorithm with the Bagging algorithm, and fuse the data volume scheduling model and the waiting delay time scheduling model to obtain a federated learning convergence acceleration task allocation model, solving the problem that the inconsistent waiting delay time of the central server leads to a slow global model update speed.

[0068] Preferably, in step one, the process of collecting the real-time data volume of each terminal device sample and the scheduling timestamp and global model update timestamp of the central server data includes:

[0069] Extract samples from each terminal device, and use a network probe to collect the real-time data volume of each terminal device sample; download Zabbix software on the central server to measure the scheduling timestamp and global model update timestamp of the central server.

[0070] Preferably, in step two, the process of setting the data volume threshold of the terminal device sample and dividing the data volume levels of each terminal device sample includes:

[0071] Set the data volume threshold of the terminal device sample. According to the Pareto principle, when the real-time data volume of the terminal device sample is greater than the data volume threshold of the terminal device sample, the data volume of the terminal device sample at that moment is classified as a large sample; when the real-time data volume of the terminal device sample is less than the data volume threshold of the terminal device sample, the data volume of the terminal device sample at that moment is classified as a small sample.

[0072] Preferably, in step three, the process of using data augmentation techniques to preprocess the real-time data volume of the terminal device sample, obtaining combined samples through a generative adversarial network, and preprocessing the combined samples and large samples includes:

[0073] Set the data augmentation technique parameters, select the data augmentation technique corresponding to the data augmentation technique parameters, and preprocess the real-time data volume of the terminal device sample;

[0074] The generative adversarial network consists of a generator, a discriminator, and a loss function. Initialize the neural network parameters of the generator and the discriminator. The generator synthesizes synthetic samples similar to the small samples, inputs the small samples and the synthetic samples into the discriminator, calculates the loss function of the discriminator, obtains the judgment result of the synthetic samples, and updates the discriminator parameters using the backpropagation algorithm; according to the judgment result of the synthetic samples, calculate the loss function of the generator, and update the generator parameters using the backpropagation algorithm. Repeat the alternating training of the generator and the discriminator, and through multiple rounds of iteration, obtain synthetic samples similar to the small samples, and combine the small samples and the synthetic samples to obtain combined samples;

[0075] Perform data cleaning on the combined samples and large samples to remove missing values, outliers, and duplicate values in the combined samples and large samples, and perform standardization processing on the combined samples and large samples after data cleaning.

[0076] Preferably, in step four, the process of allocating the training strategies for the large samples and combined samples of the terminal device according to the data volume levels of the terminal device samples includes:

[0077] According to the data volume level of the terminal device samples, allocate the training strategies for the large samples and combined samples of the terminal devices. For the combined samples, use the FedAvg algorithm to perform local gradient descent on the combined samples; for the large samples, based on the MEC network of the FedAvg algorithm, select the combined samples for training, perform stochastic gradient descent calculation on the combined samples, evaluate the improvement of the combined samples on the target loss function, and gradually increase the combined samples according to the improvement of the combined samples on the target loss function until the gradient descent condition of the large samples is satisfied, and then use the FedAvg algorithm to perform local gradient descent on the large samples.

[0078] Preferably, in step four, the principle of the FedAvg algorithm includes:

[0079] The FedAvg algorithm uses the parameter ω to perform gradient descent calculation on the prediction function, uses the terminal device sample data set to establish a subset of the terminal device samples, selects the size of the small samples in the subset of the terminal device samples, and the process of calculating the minimization of the target loss function is as follows:

[0080]

[0081] Among them, is the target loss function, is the size of the small samples in the subset of the terminal device samples, is the subset of the terminal device samples, is the prediction function for gradient descent calculation through the parameter ω;

[0082] Determine the gradient descent condition and evaluate the improvement of the minimization of the target loss function. If the improvement remains unchanged, select the terminal device samples with the same sample size as to perform the next iteration; if the improvement changes, re-select the samples larger than to perform the next iteration until the gradient descent condition is satisfied.

[0083] Preferably, in step five, establish a MEC heterogeneous data system model, set asynchronous thread scheduling for the central server, calculate the waiting delay time of the central server, and set the threshold of the waiting delay time of the central server. The process of calculating the waiting time exceeding the expected waiting time of the central server includes:

[0084] A1. The MEC heterogeneous data system model consists of a central server and k MEC terminal nodes. Among them, each MEC terminal node includes a terminal device and its sample data set, and a federated learning training model is deployed on each MEC terminal node;

[0085] A2. The central server consists of a scheduling thread, an update thread, and a coordinator. The tasks of the scheduling thread and the update thread are controlled by two asynchronous parallel threads. Among them, the coordinator sends scheduling instructions and the global model to the MEC terminal devices at the same time interval. When the terminal devices complete their work, they upload and update the global model parameters, obtain the upload queue of the global model, and update the global model according to the upload queue of the global model.

[0086] A3. The process of the central server waiting for the delay time is as follows:

[0087]

[0088] Among them, is the waiting delay time of the central server, is the timestamp of the central server updating the global model, is the scheduling timestamp of the central server;

[0089] A4. Set the waiting delay time threshold of the central server. When the waiting delay time of the central server is lower than the waiting delay time threshold of the central server, by calculating the difference between the waiting delay time of the central server and the waiting delay time threshold of the central server, obtain the waiting time exceeding the expectation of the central server.

[0090] Preferably, in step six, the process of selecting the global model update method according to the waiting delay time of the central server includes:

[0091] Among them, the global model update methods include the time weighted average method and the weighted asynchronous aggregation method;

[0092] When the waiting delay time of the central server is greater than or equal to the waiting delay time threshold of the central server, the waiting delay time of the central server is long. Use the time weighted average method to calculate the hybrid hyperparameters of the t-th round of iteration, and combine the weighted asynchronous aggregation method to obtain the global model;

[0093] When the waiting delay time of the central server is lower than the waiting delay time threshold of the central server, the waiting delay time of the central server is short. Obtain the global model through the time weighted average method;

[0094] Preferably, in step six, the principles of obtaining the global model by the time weighted average method and the weighted asynchronous aggregation method include:

[0095] B1. Use the hinge function to represent the delay function. The process of calculating the hybrid hyperparameters of the t-th round of iteration by the time weighted average method is as follows:

[0096]

[0097]

[0098] Among them, is the delay function, is the mixed hyperparameter of the t-th round of iteration, a and b are constants, and a > 0, b > 0, is the waiting time exceeding the expectation of the central server, is the waiting delay time threshold of the central server;

[0099] In the t-th round of iteration, the central server receives the local training model parameters fed back by the terminal device, and uses the mixed hyperparameter of the t-th round of iteration and the local training model parameters fed back by the terminal device received by the central server to calculate the global model as follows:

[0100]

[0101] Among them, is the global model obtained by the time-weighted average method in the t-th round of iteration; is the global model obtained by the time-weighted average method in the (t - 1)-th round of iteration, is the mixed hyperparameter of the t-th round of iteration, is the local training model parameter fed back by the terminal device received by the central server;

[0102] B2. Based on the weighted asynchronous aggregation method, set the weight parameter of each terminal device scheduled in the t-th round of iteration, and calculate the process of weighted asynchronous aggregation to update the global model as follows:

[0103]

[0104]

[0105] Among them, is the global model obtained by the weighted asynchronous aggregation method in the t-th round of iteration, is the global model obtained by the weighted asynchronous aggregation method in the (t - 1)-th round of iteration, is the weight hyperparameter in the t-th round of iteration, is the weight parameter of each terminal device scheduled in the t-th round of iteration, is the mixed hyperparameter of the t-th round of iteration, is the local training model parameter fed back by the terminal device received by the central server.

[0106] Preferably, in step seven, the process of constructing the data volume scheduling model and the waiting delay time scheduling model, and fusing the data volume scheduling model and the waiting delay time scheduling model to obtain the federated learning convergence acceleration task allocation model includes:

[0107] S1. Using the random forest algorithm, set the random forest model parameters, construct three random forest models, and assign the numbers A model, B model, and C model to the three random forest models;

[0108] S2. Take the data volume of the terminal device samples and their corresponding training strategies for the large terminal device samples and synthetic samples as the data set, divide it into a training set and a test set according to the ratio of 7:3, use the training set data to iteratively train the A model, obtain the non-linear relationships between the data volume of the terminal device samples and the training strategies for the large samples and synthetic samples respectively, realize that when inputting the data volume of the terminal device samples, the A model outputs the training strategies for the large samples and synthetic samples respectively, input the test set data into the A model, compare the training strategies for the large samples and synthetic samples output by the A model with the actual training strategies for the large samples and synthetic samples, evaluate the performance of the trained A model, adjust the A model parameters, optimize the A model, and obtain the data volume scheduling model;

[0109] S3. Take the waiting delay time of the central server and its corresponding global model update method as the data set, divide it into a training set and a test set according to the ratio of 7:3, use the training set data to iteratively train the B model, obtain the non-linear relationship between the waiting delay time of the central server and its corresponding global model update method, realize that when inputting the waiting delay time of the central server, the B model outputs the global model update method corresponding to the waiting delay time of the central server, input the test set data into the B model, compare the global model update method output by the B model corresponding to the waiting delay time of the central server with the actual global model update method corresponding to the waiting delay time of the central server, evaluate the performance of the trained B model, adjust the B model parameters, optimize the B model, and obtain the waiting delay time scheduling model;

[0110] S4. Define the model fusion environment of the data volume scheduling model and the waiting delay time scheduling model. Through the interaction of the model fusion environment of the data volume scheduling model and the waiting delay time scheduling model, generate Bagging data, divide the Bagging data into Bagging training data and Bagging test data, use the Bagging training data to train the data volume scheduling model and the waiting delay time scheduling model respectively to obtain the Bagging base models. In the test stage, input the Bagging test data into the Bagging base models to obtain the prediction results of the Bagging base models, fuse all the prediction results of the Bagging base models to obtain the fused prediction results of the Bagging base models, and calculate the average value of the fused prediction results of the Bagging base models;

[0111] Take the Bagging data and the average of the predicted results of its corresponding fused Bagging base models as the dataset, divide it into a training set and a test set according to the ratio of 7:3, use the training set data to iteratively train the C model, obtain the non-linear relationship between the Bagging data and the average of the predicted results of the fused Bagging base models, realize inputting the Bagging data, and the C model outputs the average of the predicted results of the fused Bagging base models corresponding to the Bagging data. Input the test set data into the C model, compare the average of the predicted results of the fused Bagging base models corresponding to the Bagging data output by the C model with the average of the predicted results of the fused Bagging base models corresponding to the actual Bagging data, evaluate the performance of the trained C model, adjust the parameters of the C model, optimize the C model, and obtain the federated learning convergence acceleration task allocation model.

[0112] As described above, it is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed by this application can easily think of changes or substitutions, which should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A method for accelerating the convergence of federated learning under unbalanced data distribution of multi-terminal devices, characterized in that: It includes the following steps: Step 1: Use a network probe to collect the real-time data volume of each terminal device sample, and collect the scheduling timestamp of the central server data and the global model update timestamp through Zabbix software; Step 2: Set the data volume threshold of the terminal device sample, and divide the data volume level of each terminal device sample. Among them, the divided levels are large samples and small samples; Step 3: Adopt data augmentation technology to preprocess the real-time data volume of the terminal device sample. Through a generative adversarial network, obtain synthetic samples similar to the small samples, and preprocess the synthetic samples and large samples; Step 4: According to the data volume level of the terminal device sample, allocate the training strategies for the large samples and combined samples of the terminal device; Step 5: Establish a MEC heterogeneous data system model, set asynchronous thread scheduling for the central server, calculate the waiting delay time of the central server, set the waiting delay time threshold of the central server, and calculate the waiting time exceeding the expectation of the central server; Step 6: Select the global model update method according to the waiting delay time of the central server; Step 7: Through the random forest algorithm, construct a data volume scheduling model and a waiting delay time scheduling model respectively. Combine the random forest algorithm and the Bagging algorithm, and fuse the data volume scheduling model and the waiting delay time scheduling model to obtain a federated learning convergence acceleration task allocation model.

2. The method for accelerating the convergence of federated learning under the unbalanced data distribution of multi-terminal devices according to claim 1, wherein: In the above Step 1, the process of collecting the real-time data volume of each terminal device sample and the scheduling timestamp of the central server data and the global model update timestamp includes: Extract samples from each terminal device, and use a network probe to collect the real-time data volume of each terminal device sample; Download Zabbix software on the central server to measure the scheduling timestamp and the global model update timestamp of the central server.

3. The federated learning convergence acceleration method under unbalanced data distribution of multi-terminal devices according to claim 2, wherein: In the above Step 2, the process of setting the data volume threshold of the terminal device sample and dividing the data volume level of each terminal device sample includes: Set the data volume threshold of the terminal device sample. According to the Pareto principle, when the real-time data volume of the terminal device sample is greater than the data volume threshold of the terminal device sample, the data volume of the terminal device sample at this moment is divided into large samples. When the real-time data volume of the terminal device sample is less than the data volume threshold of the terminal device sample, the data volume of the terminal device sample at this moment is divided into small samples.

4. The method for accelerating the convergence of federated learning under the unbalanced data distribution of multi-terminal devices according to claim 3, characterized in that: In the above Step 3, the process of adopting data augmentation technology to preprocess the real-time data volume of the terminal device sample, obtaining combined samples through a generative adversarial network, and preprocessing the combined samples and large samples includes: Set the data augmentation technology parameters, select the data augmentation technology corresponding to the data augmentation technology parameters, and preprocess the real-time data volume of the terminal device sample; The generative adversarial network consists of a generator, a discriminator, and a loss function. Initialize the neural network parameters of the generator and the discriminator. The generator synthesizes synthetic samples similar to the small samples, input the small samples and the synthetic samples into the discriminator, calculate the loss function of the discriminator, obtain the determination result of the synthetic samples, and update the discriminator parameters using the backpropagation algorithm; According to the judgment result of the synthetic samples, calculate the loss function of the generator, and use the backpropagation algorithm to update the generator parameters. Repeat the alternating training of the generator and the discriminator. Through multiple rounds of iteration, obtain synthetic samples similar to the small samples, and combine the small samples and the synthetic samples to obtain combined samples; Perform data cleaning on the combined samples and the large samples to remove missing values, outliers, and duplicate values in the combined samples and the large samples, and perform standardization processing on the combined samples and the large samples after data cleaning.

5. The method for accelerating the convergence of federated learning under the unbalanced data distribution of multi-terminal devices according to claim 4, characterized in that: In step 4, the process of allocating the training strategies for the large samples and the combined samples of the terminal device according to the data volume level of the terminal device samples includes: Allocate the training strategies for the large samples and the combined samples of the terminal device according to the data volume level of the terminal device samples. For the combined samples, use the FedAvg algorithm to perform local gradient descent on the combined samples; for the large samples, based on the MEC network of the FedAvg algorithm, select the combined samples for training, perform stochastic gradient descent calculation on the combined samples, evaluate the improvement of the combined samples on the target loss function, and gradually increase the combined samples according to the improvement of the combined samples on the target loss function until the gradient descent condition of the large samples is met, and then use the FedAvg algorithm to perform local gradient descent on the large samples.

6. The method for accelerating the convergence of federated learning under the unbalanced data distribution of multi-terminal devices according to claim 5, wherein: In step 4, the principle of the FedAvg algorithm includes: The FedAvg algorithm uses the parameter ω to perform gradient descent calculation on the prediction function, uses the terminal device sample dataset to establish a subset of terminal device samples, selects the size of the small samples in the subset of terminal device samples, and the process of calculating the minimization of the target loss function is as follows: ； Among them, is the target loss function, is the size of the small samples in the subset of terminal device samples, is the subset of terminal device samples, is the prediction function calculated by gradient descent through the parameter ω; Determine the gradient descent condition and evaluate the improvement of minimizing the objective loss function. If the improvement remains unchanged, select the terminal device samples with the same sample size as to perform the next iteration; if the improvement changes, re-select samples larger than and perform the next iteration until the gradient descent condition is satisfied.

7. The method for accelerating the convergence of federated learning under the unbalanced data distribution of multi-terminal devices according to claim 6, characterized in that: In step 5, establish an MEC heterogeneous data system model, set asynchronous thread scheduling for the central server, calculate the waiting delay time of the central server, and set the waiting delay time threshold of the central server. The process of calculating the waiting time exceeding the expectation of the central server includes: A1. The MEC heterogeneous data system model consists of a central server and k MEC terminal nodes. Among them, each MEC terminal node includes a terminal device and its sample dataset, and a federated learning training model is deployed on each MEC terminal node; A2. The central server consists of a scheduling thread, an update thread, and a coordinator. The tasks of the scheduling thread and the update thread are controlled by two asynchronous parallel threads. Among them, the coordinator sends scheduling instructions and the global model to the MEC terminal devices at the same time interval. When the terminal device finishes its work, it uploads the updated global model parameters, obtains the upload queue of the global model, and updates the global model according to the upload queue of the global model; A3. The process of calculating the waiting delay time of the central server is as follows: ； Among them, is the waiting delay time of the central server, is the global model timestamp updated by the central server, is the scheduling timestamp of the central server; A4. Set the waiting delay time threshold of the central server. When the waiting delay time of the central server is lower than the waiting delay time threshold of the central server, obtain the waiting time exceeding the expectation of the central server by calculating the difference between the waiting delay time of the central server and the waiting delay time threshold of the central server.

8. The method for accelerating the convergence of federated learning under the uneven distribution of multi-terminal device data according to claim 7, wherein: In step 6, the process of selecting the global model update method according to the waiting delay time of the central server includes: The global model update methods include the time-weighted average method and the asynchronous weight aggregation method; When the waiting delay time of the central server is greater than or equal to the waiting delay time threshold of the central server, the waiting delay time of the central server is long. The time-weighted average method is used to calculate the hybrid hyperparameters of the t-th round of iteration, and combined with the asynchronous weight aggregation method, the global model is obtained; When the waiting delay time of the central server is lower than the waiting delay time threshold of the central server, the waiting delay time of the central server is short. The global model is obtained through the time-weighted average method.

9. The federated learning convergence acceleration method under uneven data distribution of multi-terminal devices according to claim 8, wherein: In the sixth step, the principles of obtaining the global model by the time-weighted average method and the asynchronous weight aggregation method include: B1. Using the hinge function to represent the delay function, the process of calculating the hybrid hyperparameters of the t-th round of iteration by the time-weighted average method is as follows: ； ； Among them, is a delay function, is the mixed hyperparameter of the t-th round of iteration, a and b are constants, and a > 0, b > 0, is the waiting time exceeding the expectation of the central server, is the waiting delay time threshold of the central server; In the t-th round of iteration, the central server receives the local training model parameters fed back by the terminal devices. The process of calculating the global model using the hybrid hyperparameters of the t-th round of iteration and the local training model parameters fed back by the terminal devices received by the central server is as follows: ；； Among them, is the global model obtained by the time-weighted average method in the t-th round of iteration; is the global model obtained by the time-weighted average method in the (t - 1)-th round of iteration, is the hybrid hyperparameter in the t-th round of iteration, is the local training model parameters fed back by the terminal device received by the central server; B2. Based on the asynchronous weight aggregation method, set the weight parameters of each terminal device scheduled in the t-th round of iteration. The process of calculating the asynchronous weight aggregation to update the global model is as follows: ； ； Among them, is the global model obtained by the weighted asynchronous aggregation method in the t-th iteration, is the global model obtained by the weighted asynchronous aggregation method in the (t - 1)-th iteration, is the weight hyperparameter in the t-th iteration, is the weight parameter of each terminal device scheduled in the t-th iteration, is the hybrid hyperparameter in the t-th iteration, is the local training model parameter fed back by the terminal device received by the central server.

10. The federated learning convergence acceleration method under the unbalanced data distribution of multi-terminal devices according to claim 9, characterized in that: In the seventh step, the process of constructing the data volume scheduling model and the waiting delay time scheduling model, and fusing the data volume scheduling model and the waiting delay time scheduling model to obtain the federated learning convergence acceleration task allocation model includes: S1. Using the random forest algorithm, set the random forest model parameters, construct three random forest models, and assign the numbers A model, B model, and C model to the three random forest models; S2. Using the data volume of the terminal device samples and their corresponding training strategies for large samples and synthetic samples of the terminal devices as the data set, divide it into a training set and a test set according to a ratio of 7:

3. Use the training set data to iteratively train the A model to obtain the non-linear relationships between the data volume of the terminal device samples and the training strategies for large samples and synthetic samples respectively, so that when the data volume of the input terminal device samples is input, the A model outputs the training strategies for large samples and synthetic samples respectively. Input the test set data into the A model, compare the training strategies for large samples and synthetic samples output by the A model with the actual training strategies for large samples and synthetic samples, evaluate the performance of the trained A model, adjust the A model parameters, optimize the A model, and obtain the data volume scheduling model; S3. Take the waiting delay time of the central server and its corresponding global model update method as a data set, divide it into a training set and a test set according to a ratio of 7:3, use the training set data to iteratively train the B model, obtain the non-linear relationship between the waiting delay time of the central server and its corresponding global model update method, realize inputting the waiting delay time of the central server, and the B model outputs the global model update method corresponding to the waiting delay time of the central server. Input the test set data into the B model, compare the global model update method corresponding to the waiting delay time of the central server output by the B model with the global model update method corresponding to the actual waiting delay time of the central server, evaluate the performance of the trained B model, adjust the parameters of the B model, optimize the B model, and obtain the waiting delay time scheduling model; S4. Define the model fusion environment of the data volume scheduling model and the waiting delay time scheduling model. Through the interaction of the data volume scheduling model and the waiting delay time scheduling model fusion environment, generate Bagging data, divide the Bagging data into Bagging training data and Bagging test data, use the Bagging training data to train the data volume scheduling model and the waiting delay time scheduling model respectively, obtain the Bagging base model. In the test stage, input the Bagging test data into the Bagging base model, obtain the prediction results of the Bagging base model, fuse all the prediction results of the Bagging base model, obtain the fused prediction results of the Bagging base model, and calculate the average value of the fused prediction results of the Bagging base model; Take the Bagging data and the average value of its corresponding fused prediction results of the Bagging base model as a data set, divide it into a training set and a test set according to a ratio of 7:3, use the training set data to iteratively train the C model, obtain the non-linear relationship between the Bagging data and the average value of the fused prediction results of the Bagging base model, realize inputting the Bagging data, and the C model outputs the average value of the fused prediction results of the Bagging base model corresponding to the Bagging data. Input the test set data into the C model, compare the average value of the fused prediction results of the Bagging base model corresponding to the Bagging data output by the C model with the average value of the fused prediction results of the Bagging base model corresponding to the actual Bagging data, evaluate the performance of the trained C model, adjust the parameters of the C model, optimize the C model, and obtain the federated learning convergence acceleration task allocation model.

Images