A time-efficient method for designing a scheduling strategy for federated edge learning

Abstract

The present invention discloses a time-efficient method for designing a scheduling strategy for federated edge learning. First, the relationship between optimization variables and convergence performance is theoretically derived. Based on the convergence analysis, an optimization problem of minimizing the weighted sum of convergence error and time under the constraints of average energy and gradient age of each device is proposed. Secondly, the Lyapunov optimization method is used to transform the long-term random optimization problem into an online optimization problem so that it can be solved online in each round of training. Then, an adaptive data-assisted scheduling strategy is proposed, and it is proved that the algorithm has a polynomial algorithm complexity while maintaining asymptotic optimality. Finally, through experimental observation, the impact of different heterogeneity of devices on training efficiency is studied. The results show that the present invention can comprehensively consider factors such as device selection, amount of training data and gradient quantization level, and improve the training efficiency of federated edge learning under the triple heterogeneity of statistics, computing resources and communication resources of the device.

Description

Technical Field

[0001] The present invention relates to the field of data communication technology, and in particular to a high-efficiency federated edge learning scheduling strategy design method. Background Art

[0002] Artificial intelligence (AI) and machine learning (ML) are key technologies shaping the future of the internet and the world, significantly changing how data is collected and analyzed. However, this unprecedented volume of data presents significant challenges for traditional centralized learning, including significant data communication overhead, task latency, and privacy concerns. Fortunately, the development of mobile edge computing technology has enabled mobile devices to be equipped with high-performance central processing units (CPUs) or graphics processing units (GPUs), enabling edge devices to train small AI models locally. Consequently, distributed machine learning has emerged, with the most popular and widely used being the federated learning architecture. In federated learning, multiple edge users leverage distributed computing to train local data and upload model parameters to a central server in a privacy-preserving manner, resulting in a shared model.

[0003] Due to the diversity of services and the need for device personalization, the data generated by devices is not independent and identically distributed. This poses a significant challenge to training a universal model for federated learning, which can reduce model accuracy. Therefore, studying how to improve model accuracy in the presence of device statistical heterogeneity is an urgent issue and has attracted widespread attention from scholars.

[0004] In federated learning with device statistical heterogeneity, combining federated learning with other personalized learning methods is a common approach to addressing device statistical heterogeneity. Federated transfer learning utilizes the federated learning update mechanism to first obtain a universal global model. This global model is then fine-tuned locally on the device based on personalized data to obtain a personalized model adapted to each device's data. Federated meta-learning strategies utilize meta-learning mechanisms. The learning goal is no longer to obtain a global model that minimizes the weighted sum of the loss functions of all devices. Instead, the goal is to obtain an optimal global model after one or several local gradient descents. This allows for rapid on-device personalization, better adaptation to the device's personalized data, and improved accuracy. Federated multi-task learning is also an effective approach to personalized learning. It calculates a correlation matrix to capture the similarities between models on different devices and uses heterogeneous data as different training targets to conduct multi-task learning.

[0005] While federated learning training strategies that account for device statistical heterogeneity can improve model accuracy and better adapt to device data, they fail to consider the training cost and latency of running federated learning on wireless networks. Many emerging edge intelligence applications (such as autonomous driving and virtual reality devices) require real-time services. In wireless networks, devices participating in federated learning training often have limited computing power, battery capacity, and wireless spectrum resources. Training large amounts of data and transmitting high-dimensional models will result in significant training latency, impacting training effectiveness. Therefore, while ensuring model training accuracy, optimizing training time is crucial.

[0006] Wireless communication also plays a crucial role in federated learning systems. Because devices and base stations require constant interaction, the heterogeneity and scarcity of wireless resources significantly impact learning performance. Therefore, wireless resource optimization in federated learning is a current research hotspot. When federated learning meets wireless networks, simply improving model accuracy is often insufficient; training costs must also be considered. For example, in federated learning with a single base station, wireless resources are allocated to devices and transmission power is controlled to minimize training latency and energy consumption while ensuring model accuracy. Over-the-air computation methods are also commonly used to improve communication efficiency. These methods simulate local gradients or models and exploit the superposition characteristics of wireless multi-access channels for over-the-air summation. Furthermore, in federated learning with multiple base stations, hierarchical federated learning architectures have been widely studied, minimizing training costs by jointly optimizing edge access strategies and heterogeneous resource allocation. As can be seen, most resource optimization work focuses on the impact of wireless communication on training performance. However, in each training round, in addition to model transmission, the global model must undergo further local updates and iterations until convergence. Existing research has overlooked the impact of the computational process and the coupling between computation and communication on model performance. Even though some work has considered computation, it primarily optimizes computing power from the perspective of reducing training costs, such as designing local CPU frequency, without theoretically studying how computation affects learning performance. Therefore, given heterogeneous system computing and communication resources, the present invention requires jointly optimizing the computation and communication processes in federated learning to train models quickly and accurately.

[0007] Due to the scarcity of spectrum resources, only a subset of devices can upload local models in each training round. In synchronous federated learning, the latency of each round is the latency of the slowest training device, so device scheduling is a key issue in optimizing latency. In traditional federated learning, devices are randomly scheduled for training. Due to the heterogeneity of device systems, computing power, and channel conditions, devices with better resources are selected to minimize training costs. However, given the statistical heterogeneity of devices, the contribution of data from different devices to convergence varies. To improve convergence performance and accelerate convergence, the norm of the device's local gradient is used to measure the importance of the device to convergence. Devices that contribute significantly to convergence should be scheduled for training to improve convergence performance. Existing device scheduling strategies consider the importance of device system resources and model updates separately. However, due to the mismatch between data distribution and system resource distribution, devices with better resources may not cover all training data categories. Similarly, devices that contribute to convergence may experience extended training time due to poor resource conditions. Therefore, existing device scheduling schemes cannot simultaneously optimize convergence performance and training time and cannot be directly applied. Research on device scheduling methods that achieve fast and accurate model training is crucial. Furthermore, while some studies have considered both the device's channel conditions and gradient importance, they have neglected the device's computational resources. Devices with good channel conditions may have poor computational capabilities, which also increases training latency. Therefore, in wireless federated learning, it is necessary to jointly consider factors such as data importance, computational power, and channel conditions when scheduling devices to improve training efficiency. Summary of the Invention

[0008] In response to the shortcomings of existing technologies, this paper studies the interaction between model accuracy and training time under computing, communication, and statistical heterogeneity, and proposes a time-efficient method for designing a federated edge learning scheduling strategy. Taking into account factors such as device scheduling, training batch data size, and gradient quantization level, this paper proposes a trade-off optimization problem between accuracy and time under the constraints of average energy and gradient age of each device, and theoretically derives the relationship between control variables and the upper bound of convergence. Based on the convergence analysis, the Lyapunov optimization method is used to transform the long-term stochastic optimization problem into an online optimization problem so that it can be solved online in each round of training. Then, an effective scheduling algorithm is further proposed, which is proved to have polynomial algorithmic complexity and maintain asymptotic optimality. Finally, through experimental observation, the impact of different device heterogeneity on the efficiency of federated edge learning training is studied.

[0009] In order to achieve the above object, the present invention provides the following technical solutions:

[0010] The present invention provides a time-efficient federated edge learning scheduling strategy design method, comprising the following steps:

[0011] S1, each participating device randomly collects data from the local dataset to calculate the local gradient;

[0012] S2. Each participating device adaptively quantizes the local gradient and transmits the quantized gradient to the base station;

[0013] S3. Under the constraints of average energy consumption and average gradient age of each device, the optimization problem is to minimize the weighted sum of convergence error and training delay; the gradient age represents the number of communication intervals between the current training round and the last time the local gradient was uploaded;

[0014] S4. Deriving the relationship between control variables and the upper bound of the convergence rate, and using the Lyapunov optimization method to decouple the long-term stochastic optimization problem in step S3 into a deterministic online optimization problem; the control variables include the selection of participating devices, the amount of training data, and the gradient quantization level;

[0015] S5. In each round of training, a data volume adaptive scheduling strategy is proposed to solve the optimization problem. The algorithm has polynomial algorithm complexity and is asymptotically optimal.

[0016] S6. Through experiments, we observe the impact of device statistical heterogeneity, computing resource heterogeneity, and communication resource heterogeneity on training efficiency and provide a scheduling strategy.

[0017] Furthermore, in step S1, each participating device obtains The random sampling size is d n (r) dataset And satisfy:

[0018]

[0019] In round r, device n uses local data to calculate the local gradient, which is expressed as:

[0020]

[0021] Furthermore, in step S2, due to limited spectrum resources, the present invention uses gradient quantization technology to reduce the number of transmitted bits and lower the communication cost. qn (g n (r)) represents the gradient g n (r) quantization function, q n is the gradient quantization level of device n, Expressed as:

[0022]

[0023] Among them, ξ i (g n (r), qn (r)) are independent random variables.

[0024] Furthermore, in step S3, the gradient age gradually increases with the training round r at a rate of 1 until device n is selected to participate in training. If device n is scheduled in the rth round, the gradient age of device n is reduced to zero. Otherwise, the age of device n is increased by 1, which is expressed as:

[0025]

[0026] Among them, A n (r) represents the gradient age of device n during the rth round of training, Represents a device collection, a n (r) is a binary variable, indicating whether device n is scheduled in round r. If a n (r) = 1, indicating that device n participates in training in round r; otherwise, a n (r) = 0.

[0027] Furthermore, the optimization problem P1 of step S3 is expressed as follows:

[0028]

[0029] The constraints are:

[0030]

[0031]

[0032]

[0033]

[0034]

[0035] Among them, F(θ(r)) is the loss function, θ(r) is the global model of the base station in the rth round, u is the weight parameter for adjusting the trade-off between training performance and energy cost; T(r) is the training time of each round, E n (r) is the energy consumption of device n, including the energy consumption of local gradient calculation and gradient upload energy consumption and is the threshold, q n is the gradient quantization level of device n; d n (r) Each scheduled device uses the local dataset The amount of randomly collected data.

[0036] Furthermore, the delay calculation process of the device in step S3 is as follows:

[0037] The training delay T(r) of each round is equal to the delay of the slowest device in the current round, expressed as:

[0038]

[0039] Among them, T n (r) is the training time of each round of the device, including the local gradient calculation time and gradient upload time;

[0040] The gradient computation time for device n is expressed as:

[0041]

[0042] Among them, c n The number of CPU cycles required to process one data sample, f n is the CPU cycle frequency, l n is the number of iterations of local update, d n (r) is the amount of training data;

[0043] For the local gradient upload process of device n, the present invention adopts the orthogonal frequency division multiple access (OFDMA) communication system. The edge base station evenly distributes the total bandwidth B to all scheduling devices. n The transmission rate of device n is expressed as:

[0044]

[0045] Among them, B n Indicates the upload bandwidth allocated to device n, h n represents the channel gain between device n and the edge server, p n represents the model upload power of device n, and N0 represents the power spectral density of Gaussian noise;

[0046] Assume that the local gradient contains m n elements, the size of the local model is S n Expressed as:

[0047]

[0048] The transmission delay of device n uploading the local gradient to the edge base station is expressed as:

[0049]

[0050] In round r, the training delay of device n includes computation delay and communication delay, which can be expressed as:

[0051]

[0052] Furthermore, the calculation process of the device energy consumption in step S3 is:

[0053] Energy consumption E of device n in round r n (r) includes the energy consumption of local gradient calculation and gradient upload energy consumption E n (r) is expressed as:

[0054]

[0055] For the local gradient calculation process of device n, α n represents the effective capacitance coefficient, and the energy consumption of gradient calculation is expressed as:

[0056]

[0057] For the gradient upload process of device n, the energy of gradient upload depends on the upload time and transmission power. Expressed as:

[0058]

[0059] Furthermore, the relationship between the control variable in step S4 and the upper bound of the convergence rate is expressed as:

[0060]

[0061] in,

[0062]

[0063] in is the global variance, M 2 is the local gradient variance bound, G 2 is the local gradient bound, η is the learning rate, L is a constant, m n is the number of elements contained in the local gradient, θ * is the optimal global model parameter. The first term of the convergence rate reflects the difference from the initial weight to the optimal weight. As the number of training rounds tends to infinity, the first term eventually tends to 0. The second term C(r) is regarded as the convergence error bound and decreases with the increase of the number of scheduling devices, the amount of training data, and the gradient quantization level.

[0064] Replace the loss function in the optimization problem of step S3 with the convergence error bound, and the optimization problem is transformed into P2:

[0065]

[0066] Constraints:

[0067]

[0068]

[0069]

[0070]

[0071]

[0072] Among them, C max and T max is a normalization constant to eliminate the effects of different orders of magnitude.

[0073] Furthermore, in step S4, the process of converting the long-term stochastic optimization problem P2 into a deterministic online optimization problem in each round using the Lyapunov optimization method is as follows:

[0074] For long-term energy consumption constraints, construct N energy queues to store backlogs. To express the difference between the current energy consumption and the energy threshold, it is expressed as:

[0075]

[0076] For gradient age limits, construct a gradient age queue backlog To express the gap between the current gradient age and the age threshold, it is expressed as:

[0077]

[0078] Let Z(r) = {Y(r), Q(r)} be the total queue backlog, where The total queue backlog is expressed as:

[0079] Z n (r+1)=max{Q n (r+1), Y n (r+1)}

[0080] The Lyapunov function L(Z(r)) is introduced to represent the congestion state of the network. The Lyapunov function is expressed as:

[0081]

[0082] Let Δ(Z(r)) be expressed as the conditional Lyapunov drift function, defined as the difference between the Lyapunov functions between two consecutive rounds of training. The conditional Lyapunov drift function is expressed as:

[0083]

[0084] Minimize the drift-plus-penalty function Δ according to Lyapunov control V (Z(r)), expressed as:

[0085]

[0086] Among them, C max and T max To normalize the constant to eliminate the effects of different orders of magnitude, the non-negative weight parameter V is a tuning parameter of the objective function and the average virtual queue backlog;

[0087] Given the virtual queue backlog Z(r), and the optimization decision made in round r, the Lyapunov drift-plus-penalty function satisfies the following inequality:

[0088]

[0089] Where B1 and B2 are constants:

[0090]

[0091]

[0092] Convert the optimization problem P2 into a deterministic optimization problem P3 for online decision making:

[0093]

[0094] Constraints:

[0095]

[0096]

[0097]

[0098]

[0099]

[0100] in, and are the upper bounds of age and energy consumption, respectively.

[0101] Furthermore, step S5 proposes a data volume adaptive assisted scheduling strategy to solve the optimization problem P3. First, the scheduling strategy is initialized by selecting all devices, and then the local training batch data size (d n (r)) and gradient quantization level (q n(r)), the device scheduling strategy is given next; the specific process is:

[0102] In order to solve the Min-Max problem, an auxiliary variable τ(r) is introduced so that:

[0103]

[0104] The optimization problem is transformed into P4:

[0105]

[0106] Constraints:

[0107]

[0108]

[0109]

[0110]

[0111]

[0112]

[0113] The amount of training data d n (r) and gradient quantization level q n (r) Relax the integer variables into continuous variables, use the convex-concave programming algorithm to transform the non-convex problem into a convex optimization problem, and express the coupled variables as the sum of a convex function and a concave function (difference-of-convex):

[0114]

[0115] in, It is a concave function. Convert the concave function into a convex function in the following form:

[0116]

[0117]

[0118] The optimization problem P4 is transformed into a convex problem and solved using standard convex optimization methods;

[0119] For a given number of scheduled devices, amount of training data, and gradient quantization level, based on the objective function of problem (P4), and taking into account both the data distribution and resource distribution of the devices, devices with fast training speed and significant gradients contribute more to training efficiency. The device selection scheme is updated by eliminating the device with the smallest contribution, resulting in a new device scheduling strategy. In round r, the contribution of device n to training efficiency is defined as:

[0120]

[0121] The contribution of device n to convergence is:

[0122]

[0123] The above process is iterated continuously by iterating the possible participating devices n∈{1,…N} until the optimal number of participating devices that minimizes the objective function is found.

[0124] Compared with the prior art, the present invention has the following beneficial effects:

[0125] First, this paper derives the convergence bounds of model training and examines the impact of statistical heterogeneity, device scheduling decisions, the amount of training data used for each scheduled device, and the gradient quantization level on convergence performance, providing a theoretical perspective for designing device scheduling strategies. Based on the convergence analysis, the problem of optimizing training efficiency is modeled as an integer nonlinear programming problem (INLP) with the goal of minimizing the weighted sum of convergence error and training delay, taking into account the long-term energy consumption constraints of the devices and the age limit of the gradients.

[0126] Secondly, to efficiently solve the proposed problem, the present invention utilizes Lyapunov optimization techniques to transform the long-term optimization problem into a deterministic optimization problem for each training round. A low-complexity device scheduling scheme is then designed to rapidly make decisions regarding device selection, training data volume, and gradient quantization level. The present invention theoretically demonstrates the asymptotic optimality of the proposed algorithm.

[0127] Finally, experiments were conducted to investigate how heterogeneity in communication, computation, and statistics affects the training efficiency of wireless federated learning. Based on the varying degrees of device heterogeneity, this paper proposes a user scheduling principle to improve training efficiency. Results show that the proposed strategy balances computation and communication time, leveraging device-side computing power to compensate for poor channel conditions, resulting in improved efficiency compared to existing model training strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0128] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments described in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0129] Figure 1 This is a diagram of the federated edge learning system architecture provided for an embodiment of the present invention.

[0130] Figure 2A diagram showing the training efficiency of federated edge learning under different scheduling strategies provided by an embodiment of the present invention.

[0131] Figure 3 The embodiment of the present invention provides training efficiency under four different scheduling strategies when the number of scheduled devices increases from 2 to 10.

[0132] Figure 4 The embodiments of the present invention provide loss functions under four different scheduling strategies when the number of scheduled devices increases from 2 to 10. DETAILED DESCRIPTION

[0133] In order to better understand the present technical solution, the following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solution in the embodiments of the present invention. Obviously, the examples described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field based on this application fall within the scope of protection of the present invention.

[0134] In federated edge learning, multiple edge devices train local models using local data, upload local model gradients to the base station, and aggregate all local gradients to update the global model, which is then broadcast. However, in wireless networks, devices have limited communication resources, computing power, and battery capacity. Computing large amounts of data and transmitting high-dimensional models incurs significant training costs. Given the heterogeneity of computing resources, communication resources, and statistics across devices, designing an efficient scheduling strategy to achieve fast and accurate model training is crucial.

[0135] In order to solve this problem, the present invention considers the following aspects:

[0136] First, determine the number of scheduled devices. On the one hand, when devices are statistically heterogeneous, the system tends to schedule more devices to improve model accuracy, increasing data diversity, reducing convergence error, and improving training performance. On the other hand, due to limited total bandwidth, scheduling more devices increases the training time per round. Therefore, the present invention carefully designs the number of scheduled devices to balance model accuracy and training latency.

[0137] Secondly, determine which devices to schedule for training. Due to the heterogeneity of computing and communication resources between devices, the present invention requires scheduling devices with good resources to accelerate the training process. However, the data distribution and resource distribution of devices are not consistent. Greedily selecting the fastest training device will reduce the contribution of slow training devices to the global model, resulting in a biased global model. Therefore, under the dual heterogeneity of system and statistics, "fast federated edge learning" and "high-precision model training" should not be confused. When scheduling devices, the resource distribution and data distribution of the device should be considered together.

[0138] Finally, the model calculation and gradient transmission process is designed for the scheduling device. Computing large amounts of data and losslessly transmitting model gradients will improve model accuracy, but this also incurs significant training costs. Therefore, the present invention should jointly consider model accuracy and training costs when designing the training process. Furthermore, there is no theoretical conclusion on the impact of training data volume and gradient quantization level on model accuracy. The present invention requires theoretically characterizing the impact of device statistical heterogeneity, device scheduling, training data size, and gradient quantization level on convergence performance, and rationally designing a method to balance model accuracy and training time.

[0139] This paper addresses these issues by minimizing the weighted sum of convergence error and training latency under the constraints of device energy consumption and gradient age. Furthermore, it jointly optimizes device scheduling, training data size, and gradient quantization level in the presence of heterogeneous device statistics, computing resources, and communication resources, thereby improving training efficiency. Details are provided below.

[0140] 1 System Model

[0141] like Figure 1 As shown in FIG, the overall system architecture of the present invention adopts a wireless federated learning architecture, which consists of an edge base station and N edge devices. Represents a collection of devices. is the number of devices. For each device Collect and store training datasets where x ni is the i-th stored sample of device n, y ni is x ni The corresponding label, is the amount of training data for device n. This paper assumes that each device can only access a subset of the data, and that the training data across devices is not independent and identically distributed. Due to scarce spectrum resources, only a subset of devices can be selected to upload model gradients in each round of training.

[0142] Figure 2The training efficiency (achieving high accuracy in a short time) under different scheduling strategies is presented. Given heterogeneous device communication, computation, and statistics, the strategies that optimize training time and convergence performance, ignoring data distribution and training costs, perform poorly in terms of accuracy and training latency, respectively, resulting in low training efficiency. Therefore, in order to achieve fast and accurate model training, a balance between accuracy and training time must be struck in system design.

[0143] 1.1 Wireless Federated Learning Training Process

[0144] In wireless federated learning, the training goal is to learn a unified model to adapt to heterogeneous data on devices:

[0145]

[0146] Among them, f n (θ,x ni ,y ni ) is the local loss function of device n, which represents the global model parameter θ in mapping the input-output data pair (x ni ,y ni ) error.

[0147] The training process of wireless federated learning includes the following steps:

[0148] 1) Device Scheduling and Global Model Broadcasting: In each round of training, the base station first determines the number of participating devices and the devices to be trained. Let a n (r) is a binary variable, indicating whether device n is scheduled in round r. n (r) = 1, indicating that device n participates in training in round r; otherwise, a n (r) = 0. Let is the number of scheduled devices in round r. The base station broadcasts the global model θ(r) of the current round to participating devices.

[0149] 2) Local gradient calculation: Each scheduled device calculates the gradient from the local dataset. The random sampling size is d n (r) dataset And satisfy:

[0150]

[0151] In round r, device n uses local data to calculate the local gradient, which can be expressed as:

[0152]

[0153] 3) Gradient quantization and uploading: Due to limited spectrum resources, the present invention uses gradient quantization technology to reduce the number of transmitted bits and lower communication costs. represents the gradient g n (r) quantization function, q n is the gradient quantization level of device n, It can be expressed as:

[0154]

[0155] Among them, ξ i (g n (r), q n (r)) are independent random variables that satisfy:

[0156]

[0157] in, is an integer such that:

[0158]

[0159] ξ i (g n (r), q n (r)) expectation satisfaction:

[0160]

[0161] The stochastic gradient quantization method used above is unbiased, i.e. And has bounded variance:

[0162]

[0163] The scheduled device uploads the quantized gradient to the base station.

[0164] 4) Global gradient aggregation: The base station aggregates local gradients and updates the global model, which can be expressed as:

[0165]

[0166] The base station broadcasts the new global model to the new round of scheduling devices as the initial model for their new round of updates. The above process iterates between the base station and devices until the global model converges.

[0167] 1.2 Training Time Model

[0168] The training time for each round includes the local gradient calculation time and the gradient upload time. Note that since the base station has sufficient computing power and downlink spectrum, the global model update and broadcast time can be ignored.

[0169] For the local gradient calculation process of device n, c n Indicates the number of CPU cycles required to process one data sample, f n Indicates the CPU cycle frequency, l n Indicates the number of iterations of local update. For a given training data size d n (r), the computation time of device n can be expressed as follows:

[0170]

[0171] For the local gradient upload process of device n, the present invention adopts the orthogonal frequency division multiple access (OFDMA) communication system, which can also be easily extended to other communication systems. The edge base station evenly distributes the total bandwidth B to all scheduling devices. n The transmission rate of device n can be expressed as:

[0172]

[0173] Among them, B n Indicates the upload bandwidth allocated to device n, h n represents the channel gain between device n and the edge server, p n represents the model upload power of device n, and N0 represents the power spectral density of Gaussian noise.

[0174] Assume that the local gradient contains m n elements, the size of the local model is S n It can be expressed as:

[0175]

[0176] The transmission delay of device n uploading the local gradient to the edge base station can be expressed as follows:

[0177]

[0178] In round r, the training latency of device n includes computational latency and communication latency, which can be expressed as:

[0179]

[0180] Since the present invention is synchronous federated learning, the training delay of each round is equal to the delay of the slowest device. Therefore, the training delay of the rth round can be expressed as:

[0181]

[0182] 1.3 Training Energy Consumption Model

[0183] In each round, the energy consumption E of device n n (r) includes the energy consumption of local gradient calculation and gradient upload energy consumption E n (r) can be expressed as:

[0184]

[0185] For the local gradient calculation process of device n, α n represents the effective capacitance coefficient, and the energy consumption of gradient calculation can be expressed as:

[0186]

[0187] For the gradient upload process of device n, the energy of gradient upload depends on the transmission time and transmission power. It can be expressed as:

[0188]

[0189] Since the available energy of the device is limited, the present invention assumes that the average energy consumption of device n is less than the threshold It can be expressed as:

[0190]

[0191] 1.4 Gradient Age Model

[0192] To reduce training time, devices with fast training speeds are more likely to be scheduled. However, when the training data across devices is not independent and identically distributed, a device scheduling strategy that only considers training speed can lead to biased global models and significantly reduce the generalization ability of federated learning. Therefore, device scheduling should strike a balance between training speed and gradient importance. This paper introduces the concept of gradient age to quantify gradient importance.

[0193] A n (r) represents the gradient age of device n during the rth round of training, which indicates the number of communication intervals between the current round and the last time the local gradient was uploaded.

[0194] Gradient age gradually increases with rate 1 over training rounds r until device n is selected to participate in training. More specifically, if device n is scheduled in round r (a n (r) = 1), then the gradient age of device n is reduced to zero, otherwise the age of device n is increased by 1, which can be expressed as:

[0195]

[0196] The older the gradient age, the more likely it is that the device's data has not contributed to the global model for many rounds, which may cause the global model to lack information about the device. To ensure data fairness and improve model accuracy, the average gradient age of each device needs to be less than a threshold, which can be expressed as:

[0197]

[0198] 1.5 System Goals

[0199] In this paper, the goal is to improve training efficiency under the constraints of energy consumption and gradient age. Maximizing training efficiency is equivalent to obtaining the best loss function value with the least training time on the device. The optimization problem can be expressed as follows:

[0200] (P1):

[0201]

[0202] Constraints:

[0203]

[0204]

[0205]

[0206]

[0207]

[0208] In the objective function, u is a weight parameter that adjusts the trade-off between training performance and energy cost.

[0209] In this problem, due to the lack of interpretability of the internal mechanism of the neural network, the network training process is complex and changeable, and the loss function has no explicit closed expression. Therefore, the present invention needs to derive a closed approximate solution to the global loss. In addition, due to the long-term energy and gradient age constraints, problem (P1) is a long-term stochastic optimization problem. The optimal solution to the problem requires obtaining all information of all rounds in advance (such as channel conditions and model gradients), which is impossible. Therefore, the present invention needs to further decouple the problem into a deterministic optimization problem and solve it online in each round of training.

[0210] 2 Convergence Analysis and Scheduling Optimization

[0211] 2.1 Impact of scheduling variables on convergence performance

[0212] This paper analyzes the convergence rate on heterogeneous devices, mainly to study how scheduling variables affect training performance. To facilitate the convergence analysis, this paper makes the following assumptions.

[0213] Assumption 1: Loss function f1(θ), ..., f N (θ) is a μ-strongly convex function, that is, for satisfy:

[0214]

[0215] Assumption 2: The global update of local gradients is bounded, i.e., for satisfy:

[0216]

[0217] Based on the above assumptions and the federated learning update mechanism, this paper first presents an importance theorem for the upper bound of the convergence rate. The convergence performance analysis of this paper jointly considers the statistical heterogeneity of the data distribution, the device scheduling strategy, the batch size, and the gradient quantization of the scheduling device.

[0218] Theorem 1: Given the initial global model parameters (θ(0)) of the first round of training and the optimization variables of each round of training, after R rounds of training, the upper bound of the convergence rate of wireless federated learning can be expressed as:

[0219]

[0220] in,

[0221]

[0222] The proof of Theorem 1 is as follows:

[0223] Since the loss function is strongly convex, it can be expressed as:

[0224]

[0225] Next, for convenience, we will Abbreviated as Will Abbreviated as Will Abbreviated as The present invention solves the first term on the right side of the equation:

[0226]

[0227]

[0228] The present invention solves the second term on the right side of the equation:

[0229]

[0230] Bringing back the original formula, we get:

[0231]

[0232] In order to quantify the average value of the squared gradient norm, the present invention makes the following two assumptions: Assumption 3: The variance of the local gradient is bounded, that is, So that:

[0233]

[0234] Assumption 4: The local gradient is bounded, that is So that:

[0235]

[0236] The gradient Abbreviated as The average value of the squared gradient norm is bounded, with an upper bound of:

[0237]

[0238] Therefore, the present invention can obtain:

[0239]

[0240] in,

[0241] By summing the above inequalities from r=1 to r=R, the present invention has:

[0242]

[0243] The above inequality can be rewritten as follows:

[0244]

[0245]

[0246] Among them, θ * are the optimal global model parameters.

[0247] In Theorem 1, the expectation of the gradient norm is used as a convergence indicator. The first term in the convergence rate reflects the difference between the initial weights and the optimal weights, and it eventually approaches 0 as the number of training rounds approaches infinity. The second term, C(r), can be considered the convergence error bound; minimizing C(r) improves convergence performance.

[0248] From Theorem 1, the present invention can observe two facts:

[0249] 1) Impact of Statistical Heterogeneity: The first term of the convergence error C(r) is the global variance due to the statistical heterogeneity of device data. It decreases as the number of scheduled devices increases. This is intuitive because the global model can gain more information from the local data of more devices, which mitigates the impact of statistical heterogeneity on the convergence error.

[0250] 2) Impact of data volume and gradient quantization for local training on the device: From Theorem 1, we can observe that the convergence error decreases with the increase of data volume and gradient quantization level, that is, both accurate local updates and lossless gradient transfer can reduce the convergence error.

[0251] In order to make the optimization problem easier to handle, according to Theorem 1, the present invention replaces the loss function with the convergence error bound, and the problem (P1) can be reformulated as:

[0252] (P2):

[0253]

[0254] Constraints:

[0255]

[0256]

[0257]

[0258]

[0259]

[0260] Among them, C max and T max is a normalization constant to eliminate the effects of different orders of magnitude.

[0261] 2.2 Problem Transformation Based on Lyapunov Optimization Technique

[0262] In order to deal with long-term constraints, the present invention applies the Lyapunov optimization method to transform the long-term stochastic optimization problem (P2) into a deterministic optimization problem in each round.

[0263] For long-term energy consumption limitation, the present invention constructs N energy queues to store energy. To express the difference between the current energy consumption and the energy threshold, it can be expressed as:

[0264]

[0265] For the gradient age limit, the present invention also constructs a gradient age queue backlog To express the gap between the current gradient age and the age threshold, it can be expressed as:

[0266]

[0267] Let Z(r) = {Y(r), Q(r)} be the total queue backlog, where It can be expressed as:

[0268] Z n (r+1)=max{Q n (r+1), Y n (r+1)}

[0269] Then, the present invention introduces the Lyapunov function L(Z(r)) to represent the congestion state of the network. The larger the function value, the more unstable the queue of at least one device. The Lyapunov function can be expressed as:

[0270]

[0271] In order to push the queue backlog towards a lower congestion state, Δ(Z(r)) is expressed as a conditional Lyapunov drift function, defined as the difference between the Lyapunov functions between two consecutive training rounds, which can be expressed as:

[0272]

[0273] According to Lyapunov control, the present invention minimizes the drift-plus-penalty function Δ of the dual objective V (Z(r)), which can be expressed as:

[0274]

[0275] The non-negative weight parameter V is a tuning parameter for the objective function and the average virtual queue backlog. By adjusting the control parameter V, the system can flexibly choose between various trade-offs. Note that the drift penalty function is no longer a long-term constraint and can be solved online during the current training round. This means that the original optimization problem (P2) with long-term constraints has now been transformed into an online, real-time problem. This enables the present invention to make online decisions based on the current network state without any prior information.

[0276] Lemma 1: Given the virtual queue backlog Z(r) and the optimization decision made in round r, the upper bound of the drift penalty function can be expressed as:

[0277]

[0278] Where B1 and B2 are constants:

[0279]

[0280]

[0281] The proof of Lemma 1 is as follows:

[0282] Given the virtual queue backlog Z(r), and the optimization decision made in round r, the drift penalty function can be expressed as:

[0283]

[0284] Next, the present invention solves Δ V The upper bound of (Q(r)) can be expressed as:

[0285]

[0286] in, is a constant.

[0287] At the same time, the present invention solves Δ V Upper bound of (Y(r)):

[0288]

[0289] in, is a constant.

[0290] Therefore, the present invention can obtain:

[0291]

[0292] To better understand the problem, according to the upper bound given by Lemma 1, the present invention transforms problem (P2) into a deterministic optimization problem for online decision making:

[0293] (P3):

[0294]

[0295] Constraints:

[0296]

[0297]

[0298]

[0299]

[0300]

[0301] When the queue backlog is small and queue stability is met, the system will focus more on optimizing the weighted sum of convergence error and training time, which temporarily dominates the overall goal. Therefore, the system tends to select devices with good system resources, thereby achieving a smaller convergence error within the same training time. However, frequent selection of these devices will increase the queue backlog of energy and age, making queue stability increasingly important in system design. In order to maintain queue stability, the system will schedule devices with poor resources and infrequent updates, which increases training time. Intuitively, by dynamically repeating the above process, the online optimization framework of the present invention can adaptively minimize the optimization objective and stabilize the queue.

[0302] 2.3 Data Volume Adaptive Assisted Scheduling Strategy In order to solve the Min-Max problem, an auxiliary variable τ(r) is introduced, so that:

[0303]

[0304] The optimization problem is transformed into P4:

[0305]

[0306] Constraints:

[0307]

[0308]

[0309]

[0310]

[0311]

[0312]

[0313] Considering the coupling relationship between device selection and other control strategies, this paper proposes an iterative algorithm for heterogeneous devices that balances model accuracy and training cost. This paper further proposes an iterative algorithm to solve problem (P4). The process can be summarized as follows.

[0314] First, the present invention initializes the scheduling strategy by selecting all devices. Then, the present invention optimizes other control variables, namely the amount of local training data (d n (r)) and gradient quantization level (q n (r)).

[0315] The present invention will n (r) and q n(r) is relaxed from integer variables to continuous variables, transforming the problem into a more manageable form. However, due to the coupling between computation and communication during training, this problem remains difficult to solve. To effectively address this issue, the present invention utilizes a concave-convex programming algorithm to transform the non-convex problem into a convex optimization problem.

[0316] The present invention expresses the coupling variable as the sum of a convex function and a concave function. The present invention has:

[0317]

[0318] in, is a concave function, and the first-order Taylor expansion of a concave function at any point is its upper bound. Therefore, a concave function can be transformed into a convex function of the following form:

[0319]

[0320] Similarly, the present invention processes the following concave function in the same way:

[0321]

[0322] According to the above process, problem (P3) is transformed into a convex problem, and the present invention adopts standard convex optimization methods (such as CVXPY) to solve it. It is worth noting that the d n (r) and q n (r) is a continuous value, and the present invention further uses rounding technology to convert it into an integer value.

[0323] Next, the present invention proposes a device scheduling strategy. For a given number of scheduled devices, training batch size, and gradient quantization level, the present invention extracts the device selection variable from the objective function of problem (P4) and obtains the contribution of each device to training efficiency:

[0324]

[0325] The contribution of device n to convergence is:

[0326]

[0327] The device contribution metric reflects the present invention's goal of balancing model accuracy and training cost by jointly considering device resources and data distribution. The present invention sorts devices in descending order based on the device selection principle. The candidate device list is updated by removing the device with the lowest contribution, thereby obtaining a new device scheduling strategy. The present invention iterates over the possible number of devices n∈{1,…N} until the optimal number of devices is found that minimizes the objective function.

[0328] 2.4 Algorithm Complexity Analysis

[0329] By analyzing the number of iterations required for each stage of the algorithm and the complexity of each iteration, the present invention shows that the proposed algorithm has a polynomial time complexity.

[0330] The present invention first analyzes the complexity of the resource allocation algorithm. The algorithm requires at most K iterations. For each iteration of the algorithm, the present invention uses a complexity of The interior point method is used to solve the convex optimization problem. Therefore, under a given device selection strategy, the complexity of the resource allocation algorithm is Then, the complexity of the device scheduling strategy is analyzed. For a given number of devices, the present invention sorts the candidate devices according to a sorting algorithm (such as the Mergesort method). The worst-case algorithm complexity is Finally, the present invention analyzes the total number of iterations. The present invention iterates the possible number of planned devices n∈{1,…N} until the optimal number of devices that minimizes the objective function is found. Therefore, the number of iterations of the present invention does not exceed In summary, the algorithm of the present invention has Polynomial time complexity.

[0331] 2.5 Optimality Analysis

[0332] The present invention compares the proposed algorithm with an offline algorithm, where all network states are known in advance, i.e. the channel conditions of each training round are known. and is the offline optimal solution to the above problem, Γ opt (r) is the offline optimal objective function value corresponding to the rth round of training. The present invention has the following lemma:

[0333] Lemma 2: Assume that the time-varying channel is a steady-state process and is independent and identically distributed in each round. Therefore, for any feasible solution to the problem, it satisfies:

[0334]

[0335]

[0336] Define (C^(r)+τ^(r)) as the asymptotically optimal solution obtained by the proposed online algorithm, which satisfies the following theorem.

[0337] Theorem 2: The average efficiency of all devices has an upper bound, which can be expressed as:

[0338]

[0339] in,

[0340] The proof of Theorem 2 is as follows:

[0341] The upper bound of the drift penalty function can be expressed as:

[0342]

[0343] The last inequality above follows from Lemma 2.

[0344] Accumulating the above inequalities from r=1 to r=R, the present invention has:

[0345]

[0346] in,

[0347] Theorem 2 provides a strong theoretical guarantee for the algorithm to obtain the asymptotically optimal solution. The algorithm can make adaptive dynamic decisions in a time-varying environment and The offline optimal solution is obtained within the specified range. This means that when the value of parameter V is increased sufficiently, the online results of the present invention can approach the offline optimal solution. The weight parameter V can balance the objectives and constraints. In practical applications, the present invention should carefully select V to improve system efficiency within the constraints.

[0348] 3 Experimental observation and design principles

[0349] This paper experimentally observed the impact of communication resource heterogeneity, computing resource heterogeneity, and statistical heterogeneity on training efficiency. It found that training efficiency exhibits a non-monotonic trend with increasing number of scheduled devices, suggesting an optimal number of scheduled devices. Simulations were performed using a CNN model on the MINIST dataset. Assuming a system with 10 devices, after 300 rounds of training, the performance of several strategies was compared for 2, 4, 6, 8, and 10 scheduled users.

[0350] First, the present invention uses a random device scheduling strategy as a baseline (labeled as “random”), which randomly selects devices to participate in training, and each scheduled device uses the same gradient quantization level.

[0351] Then, the present invention considers the heterogeneity of communication resources (labeled as "communication resource heterogeneity"). This strategy selects the device with the best instantaneous channel gain to participate in training, and each scheduled device determines the gradient quantization level according to its own channel conditions.

[0352] Furthermore, the present invention incorporates considerations of data statistical heterogeneity (labeled as “communication-statistical heterogeneity”). This strategy schedules devices by comprehensively considering channel conditions and gradient timeliness. Each scheduled device determines the parameter quantization level based on its own channel conditions.

[0353] In the aforementioned strategies, the amount of data trained locally on the device is fixed. However, the hardware conditions of devices in the Internet of Things (IoT), such as smart factories and smart transportation, are highly heterogeneous. Therefore, this paper further considers the computing power of the device, allowing the device to dynamically select the amount of local training data (labeled as "communication-statistics-computation heterogeneity") to observe whether the device-side computing power can compensate for poor channel conditions by opening up the freedom of computing.

[0354] like Figure 3 As shown in the figure, as the number of devices scheduled per round increases, performance first decreases and then increases, indicating that training efficiency first increases and then decreases. Under each strategy, there is an optimal number of scheduled devices (marked with asterisks). The training process can be divided into three phases: the statistical heterogeneity-limited phase, the communication-limited and computation-assisted phase, and the communication-limited phase. Let's analyze each of these phases below.

[0355] 3.1 Impact of Device Heterogeneity on Training Efficiency

[0356] 1) Statistical heterogeneity restriction stage:

[0357] When the number of devices scheduled per round is small, the training efficiency is mainly limited by the statistical heterogeneity of the device data. Increasing the number of scheduled devices can cover more diverse training data, thereby improving training efficiency. Figure 4 As shown in the figure, the loss function decreases as the number of scheduled devices increases. Therefore, the strategies that consider statistical heterogeneity ("communication-statistical heterogeneity" and "communication-statistical-computation heterogeneity") have better training efficiency than other strategies. The performance gain of considering data statistical heterogeneity is the purple shaded area.

[0358] 2) Communication-limited-computation-assisted phase:

[0359] As the number of scheduled devices increases, the delay of uploading gradients to each device increases due to the limited total bandwidth. In order to shorten the delay of each round of training, gradient quantization technology is usually used to shorten the gradient upload delay. Figure 4 As shown in the figure, gradient quantization will lead to gradient accuracy loss, and the loss function value increases with the number of users.

[0360] It is worth noting that the device can effectively compensate for the quantization error by using more data for local training. Figure 3 In the example, after the number of users exceeds four, the strategy that considers computing heterogeneity ("communication-statistics-computation heterogeneity") balances device computing latency and gradient transmission latency. By allowing devices with strong computing power to use more data for local training, training efficiency can continue to improve. However, the training efficiency of other strategies begins to deteriorate due to bandwidth limitations. The performance gain brought about by considering computing heterogeneity is shown in the yellow shaded area.

[0361] 3) Communication restriction stage:

[0362] As the number of scheduled devices increases further, gradient upload latency and gradient quantization error dominate the training efficiency, and the training efficiency under all strategies deteriorates.

[0363] 3.2 Design Principles

[0364] Based on the above experimental observations, the present invention finds that: in the case of heterogeneous devices, by utilizing the heterogeneity of computing resources and jointly optimizing the amount of training data, quantization errors can be compensated and training efficiency can be improved.

[0365] When the computing power of the terminal devices participating in model training is similar and the computing time cost between devices is similar, training efficiency cannot be further improved by utilizing computing resources. Therefore, a "communication-statistical heterogeneity" strategy can be adopted to select devices with large channel gain and gradient age to participate in training.

[0366] However, in the networked machine intelligence scenarios that this invention focuses on (such as smart factories), the computing capabilities between devices are often highly heterogeneous, and this invention should make full use of computing gains (such as Figure 3 The gray shaded area below improves training efficiency by enabling devices with strong computing power to train more data.

[0367] In summary, the method of the present invention has the following advantages:

[0368] a) Due to the lack of interpretability of the internal mechanisms of neural networks, the network training process is complex and changeable, and the model evolution process is difficult to clearly express. This paper derives a closed-form approximate solution to the loss function expectation and characterizes the relationship between the convergence error and the statistical heterogeneity of the equipment, equipment scheduling, the amount of training data, and the gradient quantization level, providing a theoretical basis for optimizing model accuracy.

[0369] b) This paper studies the trade-off between model accuracy and training latency, jointly optimizes device scheduling, the amount of data in the computational process, and the gradient quantization level of the communication process, and improves system efficiency by scheduling important devices and replacing computational and communication costs based on the dynamic resource status.

[0370] c) Due to the long-term constraints of energy consumption and gradient age, the proposed problem is a long-term optimization problem. This paper uses Lyapunov optimization techniques to transform the long-term optimization problem into a deterministic optimization problem for each round, which can be solved online without requiring future network states. Furthermore, due to the coupled nature of computation and communication during federated learning training, this paper uses a concave-convex programming algorithm to transform the non-convex problem into a convex one, which is then solved using a low-complexity convex optimization method. Finally, this paper demonstrates that the proposed algorithm has polynomial complexity and is asymptotically optimal.

[0371] d) Through experimental observations, this paper studies the impact of device statistical heterogeneity, computing resource heterogeneity, and communication resource heterogeneity on training efficiency. The results show that training efficiency first improves and then deteriorates as the number of scheduled devices increases. When scheduling devices, it is necessary to consider all three types of heterogeneity simultaneously. The proposed strategy balances computation and communication time, using device-side computing power to compensate for poor channel conditions. This improves efficiency compared to existing model training strategies.

[0372] e) The present invention provides device scheduling recommendations. When the computing capabilities of the terminal devices participating in model training are similar, only statistical heterogeneity and communication resource heterogeneity need to be considered. When the degree of computing capability heterogeneity between devices is high, such as in connected machine intelligence scenarios (such as smart factories), the present invention should fully consider computing resource heterogeneity and improve training efficiency by enabling devices with strong computing capabilities to train more data.

[0373] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention are included in the scope of protection of the present invention.

Claims

1. A time-efficient federated edge learning scheduling strategy design method, characterized by: The following steps are involved: S1, each participating device randomly collects data from the local dataset to calculate the local gradient; S2. Each participating device adaptively quantizes the local gradient and transmits the quantized gradient to the base station; S3. Under the constraints of average energy consumption and average gradient age of each device, the optimization problem is to minimize the weighted sum of convergence error and training delay. The gradient age represents the number of communication intervals between the current training round and the last time the local gradient was uploaded. In step S3, the gradient age gradually increases at a rate of 1 with each training round r until device n is selected for training. If device n is scheduled in round r, the gradient age of device n is reduced to zero. Otherwise, the age of device n is increased by 1, which is expressed as: Among them, A n (r) represents the gradient age of device n during the rth round of training, Represents a device collection, a n (r) is a binary variable, indicating whether device n is scheduled in round r. If a n (r) = 1, indicating that device n participates in training in round r; otherwise, a n (r) = 0; The optimization problem P1 is expressed as follows: The constraints are: Among them, F(θ(r)) is the loss function, θ(r) is the global model of the base station in the rth round, u is the weight parameter for adjusting the trade-off between training performance and energy cost; T(r) is the training time of each round, E n (r) is the energy consumption of device n, including the energy consumption of local gradient calculation and gradient upload energy consumption and is the threshold, q n is the gradient quantization level of device n; d n (r) Each scheduled device uses the local dataset The amount of randomly collected data; S4. Derive the relationship between the control variables and the upper bound of the convergence rate, and use the Lyapunov optimization method to decouple the long-term stochastic optimization problem in step S3 into a deterministic online optimization problem; the control variables include the selection of participating devices, the amount of training data, and the gradient quantization level; the relationship between the control variables in step S4 and the upper bound of the convergence rate is expressed as: in, in, is the global variance, M 2 is the local gradient variance bound, G 2 is the local gradient bound, η is the learning rate, L is a constant, m n is the number of elements contained in the local gradient, θ* is the optimal global model parameter, and the first term of the convergence rate reflects the difference from the initial weight to the optimal weight. As the number of training rounds tends to infinity, the first term eventually tends to 0. The second term C(r) is regarded as the convergence error bound and decreases with the increase of the number of scheduling devices, the amount of training data, and the gradient quantization level. S5. In each round of training, a data volume adaptive scheduling strategy is proposed to solve the optimization problem. The algorithm has polynomial algorithm complexity and is asymptotically optimal. S6. Through experiments, we observe the impact of device statistical heterogeneity, computing resource heterogeneity, and communication resource heterogeneity on training efficiency and provide a scheduling strategy.

2. The high-efficiency federated edge learning scheduling strategy design method according to claim 1 is characterized in that: In step S1, each participating device obtains The random sampling size is d n (r) dataset And satisfy: In round r, device n uses local data to calculate the local gradient, which is expressed as:

3. The high-efficiency federated edge learning scheduling strategy design method according to claim 2 is characterized in that: In step S2, the gradient quantization technology is used to reduce the number of transmitted bits and reduce the communication cost. represents the gradient g n (r) quantization function, q n is the gradient quantization level of device n, Expressed as: Among them, ξ i (g n (r),q n (r)) are independent random variables.

4. The high-efficiency federated edge learning scheduling strategy design method according to claim 3 is characterized in that: The training delay calculation process of step S3 is: The training delay T(r) of each round is equal to the delay of the slowest device in the current round, expressed as: Among them, T n (r) is the training time of each round of the device, including the local gradient calculation time and gradient upload time; The gradient computation time for device n is expressed as: Among them, c n The number of CPU cycles required to process one data sample, f n is the CPU cycle frequency, l n is the number of iterations of local update, d n (r) is the amount of training data; For the local gradient upload process of device n, the orthogonal frequency division multiple access communication system is adopted, and the edge base station evenly distributes the total bandwidth B to all scheduling devices, R n The transmission rate of device n is expressed as: Among them, B n Indicates the upload bandwidth allocated to device n, h n represents the channel gain between device n and the edge server, p n represents the model upload power of device n, and N0 represents the power spectral density of Gaussian noise; Assume that the local gradient contains m n elements, the size of the local model is S n It can be expressed as: The transmission delay of device n uploading the local gradient to the edge base station is expressed as: In round r, the training delay of device n includes computation delay and communication delay, which can be expressed as:

5. The high-efficiency federated edge learning scheduling strategy design method according to claim 4 is characterized in that: The calculation process of the equipment energy consumption in step S3 is: Energy consumption E of device n in round r n (r) includes the energy consumption of local gradient calculation and gradient upload energy consumption E n (r) is expressed as: For the local gradient calculation process of device n, α n represents the effective capacitance coefficient, and the energy consumption of gradient calculation is expressed as: For the gradient upload process of device n, the energy of gradient upload depends on the upload time and transmission power. Expressed as:

6. The high-efficiency federated edge learning scheduling strategy design method according to claim 5 is characterized in that: The relationship between the control variable in step S4 and the upper bound of the convergence rate is expressed as: in, in, is the global variance, M 2 is the local gradient variance bound, G 2 is the local gradient bound, η is the learning rate, L is a constant, m n is the number of elements contained in the local gradient, θ* is the optimal global model parameter, and the first term of the convergence rate reflects the difference from the initial weight to the optimal weight. As the number of training rounds tends to infinity, the first term eventually tends to 0. The second term C(r) is regarded as the convergence error bound, which decreases with the increase of the number of scheduling devices, the amount of training data, and the gradient quantization level.

7. The high-efficiency federated edge learning scheduling strategy design method according to claim 6 is characterized in that: Replace the loss function in the optimization problem of step S3 with the convergence error bound, and the optimization problem is transformed into P2: Constraints: Among them, C max and T max is a normalization constant to eliminate the effects of different orders of magnitude; The process of converting the long-term stochastic optimization problem P2 into a deterministic online optimization problem in each round using the Lyapunov optimization method is as follows: For long-term energy consumption constraints, construct N energy queues to store backlogs. To express the difference between the current energy consumption and the energy threshold, it is expressed as: For gradient age limits, construct a gradient age queue backlog To express the gap between the current gradient age and the age threshold, it is expressed as: Let Z(r) = {Y(r), Q(r)} be the total queue backlog, where The total queue backlog is expressed as: Z n (r+1)=max{Q n (r+1),Y n (r+1)} The Lyapunov function L(Z(r)) is introduced to represent the congestion state of the network. The Lyapunov function is expressed as: Let Δ(Z(r)) be expressed as the conditional Lyapunov drift function, defined as the difference between the Lyapunov functions between two consecutive rounds of training. The conditional Lyapunov drift function is expressed as: Minimize the drift-plus-penalty function Δ according to Lyapunov control V (Z(r)), expressed as: Among them, C max and T max To normalize the constant to eliminate the effects of different orders of magnitude, the non-negative weight parameter V is a tuning parameter of the objective function and the average virtual queue backlog; Given the virtual queue backlog Z(r) and the optimization decision made in round r, the Lyapunov function satisfies the following inequality: Where B1 and B2 are constants: Convert the optimization problem P2 into a deterministic optimization problem P3 for online decision making: Constraints: in, and are the upper bounds of age and energy consumption, respectively.

8. The high-efficiency federated edge learning scheduling strategy design method according to claim 7 is characterized in that: Step S5 proposes a data volume adaptive assisted scheduling strategy to solve the optimization problem P3. First, the scheduling strategy is initialized by selecting all devices, and then the local training batch data size d is optimized. n (r) and gradient quantization level q n (r), the device scheduling strategy is given next.

9. The high-efficiency federated edge learning scheduling strategy design method according to claim 8 is characterized in that: The specific process of step S5 proposing a data volume adaptive auxiliary scheduling strategy to solve the optimization problem P3 is as follows: In order to solve the Min-Max problem, an auxiliary variable τ(r) is introduced so that: The optimization problem is transformed into P4: Constraints: The amount of training data d n (r) and gradient quantization level q n (r) Relax the integer variables into continuous variables, use the concave-convex programming algorithm to transform the non-convex problem into a convex optimization problem, and express the coupled variables as the sum of a convex function and a concave function: in, It is a concave function. Convert the concave function into a convex function of the following form: The optimization problem P4 is transformed into a convex problem and solved using standard convex optimization methods; For a given number of scheduled devices, amount of training data, and gradient quantization level, based on the objective function of problem (P4), and taking into account both the data distribution and resource distribution of the devices, devices with fast training speed and significant gradients contribute more to training efficiency. The device selection scheme is updated by eliminating the device with the smallest contribution, resulting in a new device scheduling strategy. In round r, the contribution of device n to training efficiency is defined as: The contribution of device n to convergence is: The above process is iterated continuously by iterating the possible participating devices n∈{1,…N} until the optimal number of participating devices that minimizes the objective function is found.

Images