MEC federated learning model aggregation optimization method based on non-independent identically distributed data

Abstract

The invention discloses an MEC federated learning model aggregation optimization method based on non-independent identically distributed data, and relates to the technical field of edge computing, and the method comprises the following steps: a distributed mobile terminal collects local data, records performance parameters, carries out power law distribution classification after preprocessing, sets a dynamic threshold value, and optimizes local training by using a dynamic sampling algorithm; the asynchronous transmission global model is fused with weight optimization updating and timestamp monitoring technologies to provide heterogeneous network state feedback, and efficient federated learning model aggregation is achieved. According to the method, the efficiency and precision of MEC federated learning under non-independent identically distributed data are remarkably improved, resource allocation is effectively balanced through dynamic threshold adjustment and sampling optimization, the communication overhead is reduced, the model generalization ability is enhanced through asynchronous model updating and weight fusion strategies, real-time feedback of the network state is ensured through the timestamp monitoring technology, and the method has the advantages of being high in reliability and high in reliability. And an efficient and reliable solution is provided for federated learning in the field of edge computing.

Description

Technical Field

[0001] The present invention relates to the technical field of edge computing, and particularly to an optimization method for aggregating MEC federated learning models based on non-independent and identically distributed data. Background Art

[0002] MEC, namely multi-access edge computing, is an innovative computing technology. It creates a high-performance, low-latency, and high-bandwidth telecom-level service environment by providing the IT services and cloud computing functions required by telecom users near the wireless access network. The MEC technology aims to push the computing and storage resources from the traditional cloud computing data center to the edge location of the user access network to reduce the latency and cost of data transmission, enhance the value of the mobile network bandwidth, and achieve deep integration with mobile Internet and Internet of Things services. The characteristics of MEC are mainly reflected in high performance, flexibility, security, and green energy conservation. It supports multiple network access methods and application scenarios, improves data security and privacy protection through the deployment and management of edge computing nodes, and realizes green and sustainable development by optimizing resource utilization and reducing transmission energy consumption. In short, as a cutting-edge technology, MEC is attracting more and more attention with its unique advantages and broad application prospects, and has become an important force in promoting digital transformation and intelligent development.

[0003] To solve the problems of data heterogeneity and device heterogeneity in MEC, the existing technology adopts the method of centralized data processing and unified model training. However, there will still be situations of non-independent and identically distributed data and significant differences in device performance, which will lead to problems such as low model training efficiency and limited model generalization ability. To overcome these challenges, an optimization method for aggregating MEC federated learning models based on non-independent and identically distributed data is proposed. Summary of the Invention

[0004] The purpose of the present invention is to provide an optimization method for aggregating MEC federated learning models based on non-independent and identically distributed data to solve the problems raised in the above background art.

[0005] To solve the above technical problems, the technical solution adopted by the present invention is: an optimization method for aggregating MEC federated learning models based on non-independent and identically distributed data, including the following steps:

[0006] S1. Collect local sample data volume through distributed mobile terminal devices, and record the device computing rate and network bandwidth;

[0007] S2. Preprocess the collected local data, set the local sample data volume threshold through the power-law distribution classification technology, and classify the devices. Introduce an adaptive dynamic threshold adjustment mechanism to dynamically update the threshold according to the real-time sample data volume distribution;

[0008] S3. Use the dynamic sampling algorithm to calculate the dynamic sample size of large-sample devices, realize local training optimization driven by gradient variance, set the initial number of iterations according to the size of the local sample data set of each terminal device, and dynamically adjust the number of iterations;

[0009] S4. Through the wireless channel and the scheduling thread, realize the global model distribution between the server and the terminal device and the asynchronous transmission of updated parameters, fuse the delay weighting and the data volume weight, and optimize the global model update;

[0010] S5. Through the timestamp monitoring technology, display the device delay time and the network bandwidth utilization rate in real time, and provide visual feedback on the heterogeneous network status.

[0011] A further improvement of the technical solution of the present invention lies in: in the S1, the process of collecting the local sample data volume through the distributed mobile terminal device and recording the device calculation rate and network bandwidth includes:

[0012] Deploy the terminal devices in the same network environment and connect them to the MEC server. The terminal devices include smart phones, tablets and sensor nodes. The terminal devices obtain the total amount of data stored locally through the file system interface, record its size, and form the local sample data set of the device. , estimate the calculation rate by monitoring the CPU utilization rate and the number of currently executing tasks, monitor the current upload and download rates through the network interface, and take their average value as the network bandwidth. Report the local sample data volume, device calculation rate and network bandwidth data to the MEC server before each iteration.

[0013] A further improvement of the technical solution of the present invention lies in: in the S2, the process of preprocessing the collected local data includes:

[0014] Adopt the 3σ principle to remove the noise data in the local sample data set size, device calculation rate and network bandwidth parameters, use the linear interpolation method to fill in the missing values in the local sample data set, record the calculation rate and network bandwidth of each terminal device, check whether they are within a reasonable range, if there are data points deviating from the expected value, use statistical methods to identify and remove the outliers, and fill in the corresponding missing data with the average value of the terminal device calculation rate and network bandwidth.

[0015] A further improvement of the technical solution of the present invention lies in: in the S2, the process of setting the local sample data volume threshold and classifying the devices through the power-law distribution classification technology includes:

[0016] Based on the Pareto principle, set the sample data volume size threshold as , calculate the cumulative distribution function of the data set size, and set is the 80th percentile of the sample dataset size. If , then the device belongs to a large sample. If , then the device belongs to a small sample.

[0017] A further improvement of the technical solution of the present invention lies in: in S2, an adaptive dynamic threshold adjustment mechanism is introduced. The process of dynamically updating the threshold according to the real-time sample data volume distribution includes:

[0018] After each iteration ends, the MEC server calculates the cumulative distribution function according to the latest local sample data volume reported by the terminal device. According to the Pareto principle, the local sample data volumes of the terminal devices are arranged in ascending order to obtain an ordered sequence, calculate the cumulative ratio of each element in the ordered sequence, find the smallest serial number that makes the cumulative ratio reach 0.8, and set the corresponding sample data volume as the new threshold. According to the updated threshold, the MEC server regularly sends the latest threshold and classification results to the terminal device to classify the device.

[0019] A further improvement of the technical solution of the present invention lies in: in S3, using a dynamic sampling algorithm to calculate the dynamic sample volume of large sample devices and realizing the process of local training optimization driven by gradient variance includes:

[0020] Select an initial sample volume for each large sample device. In each iteration, the terminal device calculates the gradient and estimates its variance using the current sample volume. According to the change of the gradient variance, adaptively adjust the sample volume in the next iteration. If the gradient variance under the current sample volume is large, increase the sample volume to reduce the variance. If the gradient variance under the current sample volume is small, reduce the sample volume to improve the efficiency;

[0021] In each iteration, the terminal device selects samples from the local sample dataset according to the dynamic sample volume and calculates the gradient using the selected samples. Update the local model parameters of the terminal device according to the calculated gradient, and upload the updated model parameters to the MEC server.

[0022] A further improvement of the technical solution of the present invention lies in: in S3, setting the initial number of iterations according to the size of the local sample dataset of each terminal device and the process of dynamically adjusting the number of iterations includes:

[0023] Monitor the change rate of the loss function in each round of training to adjust the remaining number of iterations of each terminal device. Set the global maximum number of iterations, and calculate the initial number of iterations according to the size of the local sample dataset of each device and the average value of the sample data volume , after each round of training, calculate the change rate of the loss function , if , then reduce the remaining number of iterations , if , the remaining number of iterations is increased , the remaining number of iterations is updated, and a change threshold of the loss function is set. When the change of the loss function is less than the change threshold of the loss function within 5 consecutive rounds, the training is stopped, where represents the average value of the sample data set size of the acquisition terminal device and are adjustment coefficients

[0024] A further improvement of the technical solution of the present invention lies in: in the S4, the process of realizing the asynchronous transmission of the global model distribution and updated parameters between the server and the terminal device through the wireless channel and the scheduling thread includes:

[0025] The asynchronous communication mechanism is executed in parallel through the scheduling thread and the update thread, so that the device uploads the updated model parameters after completing local training at any time without waiting for other devices to synchronize. The scheduling thread is responsible for regularly sending the global shared model and timestamp to the terminal device to trigger the device to perform local training tasks. The update thread receives the local model parameters uploaded by each device and updates the global model. If the device cannot complete the training on time due to resource limitations, the server does not wait for the device and continues to process the feedback of other devices.

[0026] A further improvement of the technical solution of the present invention lies in: in the S4, the process of optimizing the global model update by fusing delay weighting and data volume weight includes:

[0027] Using the method of delay weighting and data volume weight, the server calculates the hybrid hyperparameter according to the delay time of each device, and controls the influence of the device with a larger delay on the global model update through the hinge function, calculates the weight according to the size of the sample data volume of the device, and the server uses the weight and the hybrid hyperparameter to update the global model by the weighted average method.

[0028] A further improvement of the technical solution of the present invention lies in: in the S5, the process of real-time displaying the device delay time and network bandwidth utilization rate and providing visual feedback of the heterogeneous network state through the timestamp monitoring technology includes:

[0029] Record the local training start time and the model parameter upload completion time of each terminal device, calculate the device delay time, monitor the upload rate and download rate of the device using the network interface, and take their average value as the network bandwidth utilization rate;

[0030] Dynamically display the delay time and network bandwidth utilization rate of each device through the graphical interface, generate a cumulative distribution function to evaluate the overall distribution of device delays in the system, and calculate the average network bandwidth utilization rate of the terminal device.

[0031] Due to the above technical solution, the technical progress achieved by the present invention compared with the prior art is as follows:

[0032] 1. The present invention provides an optimization method for aggregating MEC federated learning models based on non-independent and identically distributed data. By collecting local sample data through distributed mobile terminal devices and recording device performance parameters, it effectively utilizes the computing power and data resources of edge devices, significantly improving the efficiency of model training and data utilization rate, and solving the problems of high latency and resource waste caused by centralized data processing in traditional methods.

[0033] 2. The present invention provides an optimization method for aggregating MEC federated learning models based on non-independent and identically distributed data. By introducing an adaptive dynamic threshold adjustment mechanism and a dynamic sampling algorithm, it can flexibly handle uneven data distribution and device performance differences, realizing local training optimization driven by gradient variance, improving the generalization ability and training accuracy of the model, and reducing the model bias caused by non-independent and identically distributed data.

[0034] 3. The present invention provides an optimization method for aggregating MEC federated learning models based on non-independent and identically distributed data. Through the asynchronous transmission mechanism of wireless channels and scheduling threads, and a global model update strategy that combines delay weighting and data volume weights, it optimizes the aggregation process of the global model, reduces transmission latency and resource consumption. At the same time, the timestamp monitoring technology provides visual feedback on the heterogeneous network state, facilitating system monitoring and optimization, and enhancing the stability and reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings.

[0036] Figure 1 It is a flowchart of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present invention fall within the scope of protection of the present invention.

[0038] Embodiments are as follows Figure 1As shown in the figure, the present invention provides an aggregation optimization method for an MEC federated learning model based on non-independent and identically distributed data, including the following steps:

[0039] S1. Collect the local sample data volume through distributed mobile terminal devices, record the device computing rate and network bandwidth, deploy the terminal devices in the same network environment and connect them to the MEC server. The terminal devices include smart phones, tablets and sensor nodes. The terminal devices obtain the total data volume stored locally through the file system interface and record its size to form the local sample data set of the device. Estimate the computing rate by monitoring the CPU utilization rate and the number of currently executing tasks, monitor the current upload and download rates using the network interface, and take their average value as the network bandwidth. Before each iteration, report the local sample data volume, device computing rate and network bandwidth data to the MEC server.

[0040] S2. Preprocess the collected local data. Through the power-law distribution classification technology, set the local sample data volume threshold and classify the devices. Introduce an adaptive dynamic threshold adjustment mechanism to dynamically update the threshold according to the real-time sample data volume distribution. Use the 3σ principle to remove the noise data in the local sample data set size, device computing rate and network bandwidth parameters. Use the linear interpolation method to fill in the missing values in the local sample data set. Record the computing rate and network bandwidth of each terminal device, check whether they are within a reasonable range. If there are data points deviating from the expected value, use statistical methods to identify and remove the outliers, and fill in the corresponding missing data with the mean values of the terminal device computing rate and network bandwidth. Based on the Pareto principle, set the sample data volume size threshold as , calculate the cumulative distribution function of the data set size, and set as the 80th percentile of the sample data set size. If , then the device belongs to the large sample. If , then the device belongs to the small sample. After each iteration, the MEC server calculates the cumulative distribution function according to the latest local sample data volume reported by the terminal device. According to the Pareto principle, arrange the local sample data volumes of the terminal devices in ascending order to obtain an ordered sequence, calculate the cumulative proportion of each element in the ordered sequence, find the smallest serial number that makes the cumulative proportion reach 0.8, and set the corresponding sample data volume as the new threshold. According to the updated threshold, the MEC server regularly sends the latest threshold and classification results to the terminal devices to classify the devices.

[0041] S3. Use the dynamic sampling algorithm to calculate the dynamic sample size of large-sample devices, achieve local training optimization driven by gradient variance, set the initial number of iterations according to the size of the local sample dataset of each terminal device, dynamically adjust the number of iterations, select the initial sample size for each large-sample device. In each iteration, the terminal device calculates the gradient and estimates its variance using the current sample size. According to the change of the gradient variance, adaptively adjust the sample size in the next iteration. If the gradient variance under the current sample size is large, increase the sample size to reduce the variance. If the gradient variance under the current sample size is small, reduce the sample size to improve efficiency. In each iteration, the terminal device selects samples from the local sample dataset according to the dynamic sample size and calculates the gradient using the selected samples. Update the local model parameters of the terminal device according to the calculated gradient, upload the updated model parameters to the MEC server, monitor the change rate of the loss function in each round of training to adjust the remaining number of iterations of each terminal device, set the global maximum number of iterations, and calculate the initial number of iterations according to the size of the local sample dataset and the average value of the sample data volume of each device. After each round of training, calculate the change rate of the loss function. If then reduce the remaining number of iterations. If then increase the remaining number of iterations. Update the remaining number of iterations, set the loss function change threshold. When the change of the loss function is less than this loss function change threshold for 5 consecutive rounds, stop training. Among them, represents the average value of the sample dataset sizes collected by the terminal devices, and are adjustment coefficients;

[0042] S4. Through the wireless channel and the scheduling thread, achieve the asynchronous transmission of the global model distribution between the server and the terminal devices and the updated parameters, fuse the delay weighting and the data volume weight, optimize the global model update. The asynchronous communication mechanism is executed in parallel by the scheduling thread and the update thread, enabling the device to upload the updated model parameters after completing local training at any time without waiting for other devices to synchronize. The scheduling thread is responsible for regularly sending the global shared model and timestamp to the terminal devices to trigger the local training tasks of the devices. The update thread receives the local model parameters uploaded by each device and updates the global model. If a device cannot complete training on time due to resource limitations, the server does not wait for this device and continues to process the feedback from other devices. Using the method of delay weighting and data volume weight, the server calculates the hybrid hyperparameter according to the delay time of each device and controls the impact of the device with a larger delay on the global model update through the hinge function. Calculate the weight according to the size of the sample data volume of the device. The server uses the weight and the hybrid hyperparameter to update the global model by the weighted average method;

[0043] S5. Through timestamp monitoring technology, the device delay time and network bandwidth utilization rate are displayed in real time, providing visual feedback on the heterogeneous network status. Record the local training start time and the model parameter upload completion time of each terminal device, calculate the device delay time, monitor the upload rate and download rate of the device using the network interface, and take their average as the network bandwidth utilization rate. Dynamically display the delay time and network bandwidth utilization rate of each device through a graphical interface, generate a cumulative distribution function to evaluate the overall distribution of device delays in the system, and calculate the average network bandwidth utilization rate of the terminal devices.

[0044] As described above, it is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed by this application can easily think of changes or substitutions, which should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. An optimization method for aggregating MEC federated learning models based on non-independent and identically distributed data, characterized in that It includes the following steps: S1. Collect the local sample data volume through distributed mobile terminal devices, and record the device computing rate and network bandwidth; S2. Preprocess the collected local data. Through the power-law distribution classification technology, set the local sample data volume threshold, and conduct device classification. Introduce an adaptive dynamic threshold adjustment mechanism to dynamically update the threshold according to the real-time sample data volume distribution; S3. Use the dynamic sampling algorithm to calculate the dynamic sample volume of large-sample devices, realize the local training optimization driven by gradient variance, set the initial number of iterations according to the size of the local sample data set of each terminal device, and dynamically adjust the number of iterations; S4. Through the wireless channel and the scheduling thread, realize the global model distribution between the server and the terminal device and the asynchronous transmission of updated parameters, and fuse the delay weight and the data volume weight to optimize the global model update; S5. Through the timestamp monitoring technology, display the device delay time and network bandwidth utilization rate in real time, and provide visual feedback on the heterogeneous network status.

2. The method for aggregating and optimizing the MEC federated learning model based on non-independent and identically distributed data according to claim 1, wherein: In S1, the process of collecting the local sample data volume through distributed mobile terminal devices and recording the device computing rate and network bandwidth includes: Deploy the terminal devices in the same network environment and connect them to the MEC server. The terminal devices include smart phones, tablets, and sensor nodes. The terminal devices obtain the total amount of data stored locally through the file system interface, record its size, and form the local sample data set of the device. , estimate the computing rate by monitoring the CPU utilization rate and the number of tasks currently being executed, monitor the current upload and download rates using the network interface, and take their average value as the network bandwidth. Before each iteration, report the local sample data volume, device computing rate, and network bandwidth data to the MEC server.

3. The method for aggregating and optimizing the MEC federated learning model based on non-independent and identically distributed data according to claim 2, wherein: In S2, the process of preprocessing the collected local data includes: Adopt the 3σ principle to remove the noise data in the local sample data set size, device computing rate and network bandwidth parameters, use the linear interpolation method to fill in the missing values in the local sample data set, record the computing rate and network bandwidth of each terminal device, check whether they are within a reasonable range. If there are data points deviating from the expected value, use statistical methods to identify and remove the outliers, and fill in the corresponding missing data with the mean value of the terminal device computing rate and network bandwidth.

4. The method for aggregating and optimizing the MEC federated learning model based on non-independent and identically distributed data according to claim 3, wherein: In S2, the process of setting the local sample data volume threshold and conducting device classification through the power-law distribution classification technology includes: Set the threshold of the sample data volume size based on the Pareto principle as , calculate the cumulative distribution function of the data set size, and set as the 80th percentile of the sample data set size. If , then the device belongs to a large sample. If , then the device belongs to a small sample.

5. The method for aggregating and optimizing the MEC federated learning model based on non-independent and identically distributed data according to claim 4, wherein: In S2, the process of introducing the adaptive dynamic threshold adjustment mechanism to dynamically update the threshold according to the real-time sample data volume distribution includes: After each iteration, the MEC server calculates the cumulative distribution function according to the latest local sample data volume reported by the terminal device. According to the Pareto principle, arrange the local sample data volume of the terminal devices in ascending order to obtain an ordered sequence, calculate the cumulative proportion of each element in the ordered sequence, find the smallest serial number that makes the cumulative proportion reach 0.8, and set the corresponding sample data volume as the new threshold. According to the updated threshold, the MEC server regularly sends the latest threshold and classification results to the terminal devices to classify the devices.

6. The method for aggregating and optimizing the MEC federated learning model based on non-independent and identically distributed data according to claim 5, wherein: In S3, the process of using the dynamic sampling algorithm to calculate the dynamic sample volume of large-sample devices and realizing the local training optimization driven by gradient variance includes: Select an initial sample volume for each large-sample device. In each iteration, the terminal device calculates the gradient and estimates its variance using the current sample volume. According to the change of the gradient variance, adaptively adjust the sample volume in the next iteration. If the gradient variance under the current sample volume is large, increase the sample volume to reduce the variance. If the gradient variance under the current sample volume is small, reduce the sample volume to improve the efficiency; In each iteration, the terminal device selects samples from the local sample dataset according to the dynamically determined sample size, calculates gradients using the selected samples, updates the local model parameters of the terminal device based on the calculated gradients, and uploads the updated model parameters to the MEC server.

7. The method for aggregating and optimizing the MEC federated learning model based on non-independent and identically distributed data according to claim 6, wherein: In S3, the initial number of iterations is set according to the size of the local sample dataset of each terminal device. The process of dynamically adjusting the number of iterations includes: Monitor the change rate of the loss function in each round of training to adjust the remaining number of iterations for each terminal device. Set the global maximum number of iterations, and calculate the initial number of iterations based on the size of the local sample dataset of each device and the average value of the sample data volume. After each round of training, calculate the change rate of the loss function. If , then reduce the remaining number of iterations. If , then increase the remaining number of iterations. , update the remaining number of iterations, set the loss function change threshold, and stop training when the change of the loss function is less than this loss function change threshold for 5 consecutive rounds. Among them, represents the average value of the sizes of the sample datasets of the acquisition terminal devices, and are adjustment coefficients.

8. The method for aggregating and optimizing the MEC federated learning model based on non-independent and identically distributed data according to claim 7, wherein: In S4, the process of implementing the asynchronous transmission of the global model distribution and updated parameters between the server and the terminal device through the wireless channel and the scheduling thread includes: The asynchronous communication mechanism is executed in parallel by the scheduling thread and the update thread, enabling the device to upload the updated model parameters at any time after completing local training without waiting for other devices to synchronize. The scheduling thread is responsible for periodically sending the globally shared model and timestamp to the terminal device to trigger the local training task. The update thread receives the local model parameters uploaded by each device and updates the global model. If a device fails to complete training on time due to resource constraints, the server does not wait for this device and continues to process the feedback from other devices.

9. The method for aggregating and optimizing the MEC federated learning model based on non-independent and identically distributed data according to claim 8, wherein: In S4, the process of optimizing the global model update by integrating delay weighting and data volume weighting includes: Using the method of delay weighting and data volume weighting, the server calculates the hybrid hyperparameter according to the delay time of each device, and controls the impact of devices with larger delays on the global model update through the hinge function. The weight is calculated according to the size of the sample data volume of the device. The server uses the weight and the hybrid hyperparameter to update the global model by the weighted average method.

10. The method for aggregating and optimizing the MEC federated learning model based on non-independent and identically distributed data according to claim 9, wherein: In S5, the process of providing visual feedback on the heterogeneous network status by monitoring the device delay time and network bandwidth utilization in real time through the timestamp monitoring technology includes: Record the start time of local training and the completion time of uploading model parameters of each terminal device, calculate the device delay time, monitor the upload rate and download rate of the device using the network interface, and take their average value as the network bandwidth utilization; Dynamically display the delay time and network bandwidth utilization of each device through a graphical interface, generate a cumulative distribution function to evaluate the overall distribution of device delays in the system, and calculate the average network bandwidth utilization of the terminal device.

Images