A device scheduling method based on combining channel and local model update

Abstract

The present invention discloses a device scheduling method based on the combination of channel and local model update, which relates to the fields of federated learning and user scheduling. In the process of local model aggregation, over-the-air computing is used to improve communication efficiency. The aggregation error generated during the over-the-air computing process is reduced by optimizing the beamforming vector at the receiving end. Then, a device scheduling method that considers both channel and local model updates is proposed. When the number of scheduled devices is fixed, compared with the existing technology, the present invention can reduce computing and communication overhead, improve model training speed, and thus achieve better learning performance.

Description

Technical Field

[0001] The present invention relates to the field of wireless communications, and in particular to a device scheduling method based on combining channel and local model updating. Background Art

[0002] In recent years, breakthroughs in machine learning technology, increased computing power, and the explosive growth of data have made AI applications such as autonomous driving and virtual reality increasingly a reality. Typical machine learning techniques typically perform model training in a centralized manner, where raw data generated by smart mobile devices is sent directly to a centralized cloud data center. This large amount of data transmission inevitably causes network congestion and high latency, and the transmission of raw data can also lead to privacy leaks. Meanwhile, the increasing computing power of base stations and edge mobile devices such as mobile phones and tablets has made it possible to perform computations at the edge of the network. Based on this, the federated learning training framework has been proposed. In federated learning, only the trained model or model gradient is transmitted, while private data remains on the edge mobile device, thus protecting privacy and conserving communication resources.

[0003] To further improve the utilization of communication spectrum resources during federated learning, researchers are combining over-the-air computing with federated learning. Over-the-air computing leverages the superposition properties of channels to aggregate gradients during federated learning training. Because some computation is performed during transmission, it can also reduce latency.

[0004] User scheduling in federated learning is a hot research topic. Typically, a large number of edge devices are connected to the parameter server, but due to communication load and device energy consumption, only a subset of these devices interact with the parameter server in each training round. In federated learning based on over-the-air computing, user scheduling typically considers two factors: the channel and local model update. Traditionally, combining the channel and local model update approaches involves first selecting devices based on channel gain, performing local training on all selected devices, and then scheduling a smaller number of devices based on the local training results. This results in some devices undergoing local training but not being selected, wasting their energy. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a device scheduling method based on the combination of channel and local model update. This method is aimed at a federated learning system based on air computing, and improves the model training effect by optimizing the scheduling of edge mobile devices.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] Step 1: Build a federated learning system

[0008] In an edge intelligence scenario, there are K single-antenna edge mobile devices, denoted as K = {1, 2, ..., K} and a parameter server with M antennas. Each edge mobile device k∈K has a local dataset D k ,|D k | represents the dataset D k In federated learning, the parameter server and edge mobile devices jointly train a global model ω. The federated learning training process is a cyclic process. Each cycle is called a training round. Each training round obtains a new global model. t represents the global model obtained in the t-th training round, ω 0 represents the initial global model that has not been trained by federated learning, ω t-1 Represents the global model obtained in the previous training round, forming a federated learning system;

[0009] Step 2: The two parameters are updated based on the channel and local model, and the parameter server schedules the edge mobile devices to participate in the training;

[0010] Step 3: The parameter server takes the global model ω obtained in the previous cycle t-1 Send it to all scheduled edge mobile devices;

[0011] Step 4: The scheduled edge mobile devices use the stochastic gradient descent algorithm to perform local training to obtain their respective local gradients;

[0012] Step 5: The scheduled edge mobile device uploads the obtained local gradient to the parameter server, updates the global model, and obtains ω t ,The process of uploading local gradients adopts air calculation and optimizes the air calculation;

[0013] Steps 2-5 are repeated until the global model ω t convergence.

[0014] Furthermore, the step 2 specifically includes:

[0015] Channel parameters: In the tth round of training, there are different channel gain vectors h between the parameter server and each edge mobile device k k,t Each edge mobile device sends a pilot sequence to the parameter server, and the parameter server estimates the channel gain vector h between each edge mobile device and the parameter server based on the received pilot sequence k,t ; Use the channel gain vector h k,t The l2 norm of represents the channel parameter between the parameter server and the edge mobile device k, where Represents vector h k,tThe absolute value of the i-th component. Since there are M antennas in the parameter server in total, the channel gain vector h k,t has M components in total;

[0016] Local model update parameter: In the t-th training round, after local training, the edge mobile device k obtains the local gradient g k,t , and uses the l2 norm ||g k,t || of the local gradient g k,t || to represent the local model update parameter of the edge mobile device k; The larger the value of ||g k,t ||, the greater the improvement of the local training result of device k on the effect of federated learning training;

[0017] Since the edge mobile device k has not performed local training before being scheduled, and the result g of local training is required during scheduling k,t , calculate the l2 norm ||g k,t [[ID=二十]] of g k,t || to guide the scheduling. Therefore, it is necessary to estimate the l2 norm ||g k,t || of the local gradient obtained by local training;

[0018] Combining the channel with the local model update to determine the scheduling priority of the device. The definition of the device scheduling priority is as follows:

[0019] I k,t = c||h k,t ||+(l-c)||g k,t || (1)

[0020] where I k,t represents the scheduling priority of the edge mobile device k in the t-th training round. c∈[0,1] is a hyperparameter used to control the influence weights of the two scheduling parameters;

[0021] Sort the scheduling priorities I k,t of all edge mobile devices k in descending order. When the number of scheduled devices is fixed at N, 0<N<K, select the first N I k,t with the largest values to participate in the federated learning training in the t-th round; The set of edge mobile devices scheduled in the t-th training round is:

[0022]

[0023] Furthermore, the estimation of the l2 norm ||g k,t || of the local gradient obtained by local training is specifically:

[0024] Since the local gradients calculated by the edge mobile device in each training round have strong temporal correlation, the l2 norm of the gradient in the most recent training round is used to estimate the l2 norm of the gradient in the current training round. Specifically, in the tth training round, the estimated local gradient l2 norm is:

[0025]

[0026] where t k Represents the last scheduled federated learning training round of device k;

[0027] In the first round of federated learning training, all edge mobile devices need to participate in the training and upload the local gradients of all edge mobile devices so that the parameter server can estimate the gradient l2 norm of subsequent training rounds; when the edge mobile device k is scheduled in the tth round of training, the parameter server uses the local gradient g after local training k,t renew

[0028] Furthermore, the step 4 specifically includes:

[0029] The scheduled edge mobile devices use the stochastic gradient descent algorithm for local training. The stochastic gradient descent algorithm first constructs a loss function and then uses the loss function to calculate the local gradient.

[0030] The global loss function of the system is in Represents the sample capacity of the entire federated learning system, F k (ω t-1 ) is the global model ω obtained in the previous cycle t-1 The local loss function on the edge mobile device k, the global loss function is the weighted average of the local loss functions of all edge mobile devices; the local loss function on the edge mobile device k Where (x k ,y k ) represents the data sample on device k, f(x k ,y k ;ω t-1 ) is a single sample (x k ,y k ) corresponding to the loss function, f(x k ,y k ;ω t-1 ) measures the global model ω t-1 For the sample (x k ,y k )’s matching performance;

[0031] Construct the formula for obtaining the local gradient by performing local training on edge mobile device k in the tth training round:

[0032]

[0033] Where ▽f(x k ,y k ;ω t-1 ) represents the derivative of the loss function, L k,t ∈D k is the number of training rounds from the dataset D k The mini-batch sample set obtained by random selection, L b =|L k,t | is the mini-batch sample set L k,t The number of samples in g; k,t is the local gradient obtained by local training.

[0034] Furthermore, the step 5 specifically includes:

[0035] The local gradient g obtained by the scheduled edge mobile device k,t Upload to the parameter server and update the global model. First, build the formula for the parameter server to update the global model:

[0036]

[0037] where η t is the learning rate in the tth training round, |S t | is the number of devices in the scheduling device set;

[0038] The process of uploading gradients adopts air calculation. Air calculation refers to the use of the superposition characteristics of the channel to realize the aggregation of local gradients during the transmission of local gradient signals, that is, to complete the (5) formula during the transmission of local gradient signals. Part of the calculation; the specific implementation process of the aerial calculation is as follows:

[0039] In the tth round of training, the scheduled edge mobile devices simultaneously upload the calculated local gradient g k,t , all the sent local gradients are aggregated in the air, and the aggregation signal received by the parameter server is:

[0040]

[0041] where p k,t is the transmitter scalar of device k in the tth training round; n is a scalar with mean 0 and variance σ 2 Gaussian white noise vector;

[0042] The parameter server assigns a beamforming vector to the received aggregate signal. The aggregate signal processed by the parameter server is:

[0043]

[0044] where m is the receiver beamforming vector, and the superscript H represents the transpose. That is, local gradient signal aggregation completed by air calculation;

[0045] The aggregation of local gradients achieved by air calculations is affected by channel fading and noise, so air calculations are optimized;

[0046] In an ideal channel without channel fading and noise interference, the ideal aggregate signal is:

[0047]

[0048] The error between the ideal aggregate signal and the actual aggregate signal is expressed as the mean square error, which is expressed as follows:

[0049]

[0050] Where E is the symbol for finding mathematical expectation;

[0051] In order to reduce the influence of channel gain and noise during the over-the-air calculation process and improve the performance of over-the-air calculation, it is necessary to design the transceiver according to the mean square error minimization criterion to minimize the mean square error between the ideal aggregate signal and the actual aggregate signal.

[0052] Furthermore, the designing of the transceiver according to the mean square error minimization criterion to minimize the mean square error between the ideal aggregate signal and the actual aggregate signal specifically includes:

[0053] Designing a transceiver means determining the transmitter scalar p k,t and the receiver beamforming vector m;

[0054] The transmitter scalar is designed as:

[0055]

[0056] Where μ is the transmit power control factor, |p k,t | 2 ≤P0, P0 is the maximum transmit power, symbol || || 2 is to find the square of the l2 norm of its middle vector;

[0057] The transmit power control factor is designed as:

[0058]

[0059] Substituting equations (10) and (11) into equation (7), the actual aggregate signal is simplified to:

[0060]

[0061] The mean square error between the actual aggregate signal and the ideal aggregate signal is further expressed as:

[0062]

[0063] The optimization problem is constructed with the goal of minimizing the mean square error:

[0064]

[0065] Introducing a dummy variable The optimization problem (14) is transformed into the following form

[0066]

[0067] Reassign m, We obtain the equivalent optimization problem:

[0068]

[0069] Solve the optimization problem in equation (16) to obtain the receiver beamforming vector m.

[0070] Furthermore, solving the optimization problem in formula (16) specifically includes:

[0071] The semi-definite relaxation method SDR is used to find the initial solution of the above problem, and then the continuous convex approximation algorithm SCA is used to optimize the initial solution; the initial solution of the SDR method is as follows:

[0072] Let A = mm H , A * =min A tr(A), where tr(A) represents the trace of matrix A, and λ1 is A * The largest eigenvalue, u1 is the eigenvector corresponding to λ1;

[0073] If A * The rank of is 1, then is the optimal solution to the optimization problem;

[0074] If A * The rank is not 1, and the initial solution is Optimize;

[0075] The specific steps of SCA method to optimize the initial solution are as follows:

[0076] In the optimization problem of formula (16), the non-convex restriction is ||mH h k,t || 2 ≥1, Introduce auxiliary variable c k,t =[Re(m H h k,t ),Im(m H h k,t )], formula (16) is converted to:

[0077]

[0078] The non-convex constraint in (17) is ||c k,t || 2 ≥1, Use ||c k,t || 2 ≥||c k,t (l) || 2 +2(c k,t (l) ) T (c k,t -c k,t (l) )≥1, Iterative relaxation of linear constraints performs a convex approximation to the non-convex constraints; where c k,t (l) is the solution after l iterations of optimization,

[0079] Replacing the non-convex constraint in Equation (17) with the above convex constraint, Equation (17) can be rewritten as:

[0080]

[0081] make Iterate and solve equation (18) until Where ε represents the solution accuracy, and the corresponding optimal solution is the receiver beamforming vector m.

[0082] The beneficial effects of the present invention are:

[0083] This paper proposes a device scheduling method based on a combination of channel and local model updates. When scheduling edge mobile devices in a federated learning system, gradient estimation is used to estimate local model update parameters before local training. Channel-related parameters and local model update-related parameters are then added together by assigning different weights. The results are then compared to schedule the edge mobile devices. Compared to traditional approaches that first select devices based on channel gain, locally train all selected devices, and then schedule a smaller number of devices based on the local training results, this approach selects edge mobile devices for scheduling in a single scheduling step, reduces unnecessary local training, and conserves energy consumption for edge mobile devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Figure 1 Schematic diagram of the federated learning system model of the present invention;

[0085] Figure 2 Flowchart of a specific embodiment of the present invention;

[0086] Figure 3 It is a flow chart of the specific scheduling scheme in the present invention. DETAILED DESCRIPTION

[0087] In order to better understand the purpose, structure and function of the present invention, the device scheduling method based on the combination of channel and local model update of the present invention is further described in detail below with reference to the accompanying drawings.

[0088] The present invention specifically comprises the following steps:

[0089] Step 1: Build a federated learning system

[0090] See Figure 1 In an edge intelligence scenario, there are K single-antenna edge mobile devices, denoted as K = {1, 2, ..., K} and a parameter server with M antennas. Each edge mobile device k∈K has a local dataset D k ,|D k | represents the dataset D k In federated learning, the parameter server and edge mobile devices jointly train a global model ω. The federated learning training process is a cyclic process. Each cycle is called a training round. Each training round obtains a new global model. t represents the global model obtained in the t-th training round, ω 0 represents the initial global model that has not been trained by federated learning, ω t-1 Represents the global model obtained in the previous training round, forming a federated learning system;

[0091] Step 2: Combine the channel and local model to update the two parameters, and the parameter server schedules the edge mobile devices to participate in the training

[0092] Channel parameters: In the tth round of training, there are different channel gain vectors h between the parameter server and each edge mobile device k k,t Each edge mobile device sends a pilot sequence to the parameter server, and the parameter server estimates the channel gain vector h between each edge mobile device and the parameter server based on the received pilot sequence k,t ; Use the channel gain vector h k,t The l2 norm of represents the channel parameter between the parameter server and the edge mobile device k, where Represents vector h k,t The absolute value of the i-th component. Since the parameter server has a total of M antennas, the channel gain vector h k,t There are M components in total;

[0093] Local model update parameters: In the tth round of training, the edge mobile device k performs local training and obtains the local gradient g k,t , using the local gradient g k,t The l2 norm of ||g k,t || represents the local model update parameter of edge mobile device k; || g k,t The larger the value of ||, the greater the improvement in the effect of local training on device k on federated learning training.

[0094] Since the edge mobile device k does not undergo local training before being scheduled, and the local training result g is used during scheduling k,t , calculate g k,t The l2 norm of ||g k,t || guides scheduling, so it is necessary to perform the l2 norm of the local gradient obtained by local training ||g k,t ||Make an estimate;

[0095] The channel is combined with the local model update to determine the priority of device scheduling. The device scheduling priority is defined as follows:

[0096] I k,t =c||h k,t ||+(1-c)||g k,t || (1)

[0097] Among them I k,t represents the scheduling priority of edge mobile device k in the tth training round, c∈[0,1] is a hyperparameter used to control the influence weight of the two scheduling parameters;

[0098] The scheduling priority I for all edge mobile devices k k,t is sorted in descending order. When the number of scheduled devices is fixed at N, where 0 < N < K, the first N I k,t with the largest values are selected to participate in the t-th round of federated learning training; the set of edge mobile devices scheduled in the t-th training round is:

[0099]

[0100] The specific scheduling scheme for edge mobile devices is as Figure 3 shown;

[0101] The method for estimating the l2 norm ||g k,t || of the local gradient obtained from local training is as follows: Since the local gradients calculated by edge mobile devices in each round of training have strong temporal correlation, the l2 norm of the gradient in the most recent training round is used to estimate the l2 norm of the gradient in the current training round. Specifically, in the t-th training round, the estimated l2 norm of the local gradient is:

[0102]

[0103] where t k represents the round of federated learning training in which device k was last scheduled;

[0104] In the first round of federated learning training, all edge mobile devices are required to participate in training and upload the local gradients g k,1 from the first round of training of all edge mobile devices, so that the parameter server can estimate the l2 norm of the gradients in subsequent training rounds; when edge mobile device k is scheduled in the t-th training round, the parameter server updates with the local gradient g k,t after local training

[0105] Step 3: The parameter server distributes the global model ω t-1 obtained in the previous cycle to all scheduled edge mobile devices

[0106] The parameter server distributes the global model ω t-1 obtained in the previous cycle to all scheduled edge mobile devices via a wireless channel. Since the parameter server has sufficient energy and bandwidth compared to edge mobile devices, it is assumed that the distribution process of the global model ω t-1 is an error-free transmission process.

[0107] Step 4: The scheduled edge mobile devices perform local training using the stochastic gradient descent algorithm to obtain their respective local gradients

[0108] The stochastic gradient descent algorithm first constructs the loss function, and then uses the loss function to calculate the local gradient;

[0109] The scheduled edge mobile device receives the global model ω obtained in the previous training round t-1 , construct a global loss function for the entire federated learning system, in Represents the sample capacity of the entire federated learning system, F k (ω t-1 ) is the global model ω obtained in the previous cycle t-1 The local loss function on the edge mobile device k, the global loss function is the weighted average of the local loss functions of all edge mobile devices; the local loss function on the edge mobile device k Where (x k ,y k ) represents the data sample on device k, f(x k ,y k ;ω t-1 ) is a single sample (x k ,y k ) corresponding to the loss function, f(x k ,y k ;ω t-1 ) measures the global model ω t-1 For the sample (x k ,y k )’s matching performance;

[0110] Construct the formula for obtaining the local gradient by performing local training on edge mobile device k in the tth training round:

[0111]

[0112] Where ▽f(x k ,y k ;ω t-1 ) represents the derivative of the loss function, L k,t ∈D k is the number of training rounds from the dataset D k The mini-batch sample set obtained by random selection, L b =|L k,t | is the mini-batch sample set L k,t The number of samples in g; k,t is the local gradient obtained by local training.

[0113] Step 5: The scheduled edge mobile device uploads the obtained local gradient to the parameter server, updates the global model, and obtains ω t ,The process of uploading local gradients adopts air calculation and optimizes the air calculation;

[0114] The local gradient g obtained by the scheduled edge mobile device k,t Upload to the parameter server and update the global model. First, build the formula for the parameter server to update the global model:

[0115]

[0116] where η t is the learning rate in the tth training round, |S t | is the number of devices in the scheduling device set;

[0117] The process of uploading gradients adopts air calculation. Air calculation refers to the use of the superposition characteristics of the channel to realize the aggregation of local gradients during the transmission of local gradient signals, that is, to complete the (5) formula during the transmission of local gradient signals. Part of the calculation; the specific implementation process of the aerial calculation is as follows:

[0118] In the tth round of training, the scheduled edge mobile devices simultaneously upload the calculated local gradient g k,t , all the sent local gradients are aggregated in the air, and the aggregation signal received by the parameter server is:

[0119]

[0120] where p k,t is the transmitter scalar of device k in the tth training round; n is a scalar with mean 0 and variance σ 2 Gaussian white noise vector;

[0121] The parameter server assigns a beamforming vector to the received aggregate signal. The aggregate signal processed by the parameter server is:

[0122]

[0123] where m is the receiver beamforming vector, and the superscript H represents the transpose. That is, local gradient signal aggregation completed by air calculation;

[0124] The aggregation of local gradients achieved by air calculations is affected by channel fading and noise, so air calculations are optimized;

[0125] In an ideal channel without channel fading and noise interference, the ideal aggregate signal is:

[0126]

[0127] The error between the ideal aggregate signal and the actual aggregate signal is expressed as the mean square error, which is expressed as follows:

[0128]

[0129] Where E is the symbol for finding mathematical expectation;

[0130] To reduce the impact of channel gain and noise during over-the-air computation and improve its performance, the transceiver must be designed based on the mean square error (MSE) minimization criterion. This minimizes the mean square error (MSE) between the ideal aggregate signal and the actual aggregate signal. Specifically, the following factors must be considered:

[0131] Designing a transceiver means determining the transmitter scalar p k,t and the receiver beamforming vector m;

[0132] The transmitter scalar is designed as:

[0133]

[0134] Where μ is the transmit power control factor, |p k,t | 2 ≤P0, P0 is the maximum transmit power, symbol || 2 is to find the square of the l2 norm of its middle vector;

[0135] The transmit power control factor is designed as:

[0136]

[0137] Substituting equations (10) and (11) into equation (7), the actual aggregate signal is simplified to:

[0138]

[0139] The mean square error between the actual aggregate signal and the ideal aggregate signal is further expressed as:

[0140]

[0141] The optimization problem is constructed with the goal of minimizing the mean square error:

[0142]

[0143] Introducing a dummy variable The optimization problem (14) is transformed into the following form

[0144]

[0145] Reassign m, We obtain the equivalent optimization problem:

[0146]

[0147] Solve the optimization problem in equation (16) to obtain the receiver beamforming vector m, which specifically includes:

[0148] Use the semi-definite relaxation method SDR to find the initial solution of the above problem, and then use the continuous convex approximation algorithm SCA to optimize the initial solution;

[0149] The SDR method finds the initial solution as follows:

[0150] Let A = mm H , A * =min A tr(A), where tr(A) represents the trace of matrix A, and λ1 is A * The largest eigenvalue, u1 is the eigenvector corresponding to λ1;

[0151] If A * The rank of is 1, then is the optimal solution to the optimization problem;

[0152] If A * The rank is not 1, and the initial solution is Optimize;

[0153] The specific steps of SCA method to optimize the initial solution are as follows:

[0154] In the optimization problem of formula (16), the non-convex restriction is ||m H h k,t || 2 ≥1, Introduce auxiliary variable c k,t =[Re(m H h k,t ),Im(m H h k,t )], formula (16) is converted to:

[0155]

[0156] The non-convex constraint in (17) is ||c k,t || 2 ≥1, Use ||c k,t || 2 ≥||c k,t (l) || 2 +2(c k,t (l) ) T (c k,t -c k,t (l) )≥1, Iterative relaxation of linear constraints performs a convex approximation to the non-convex constraints; where ck,t (l) is the solution after l iterations of optimization,

[0157] Replacing the non-convex constraint in Equation (17) with the above convex constraint, Equation (17) can be rewritten as:

[0158]

[0159] make Iterate and solve equation (18) until Where ε represents the solution accuracy, and the corresponding optimal solution is the receiver beamforming vector m.

[0160] Steps 2-5 are repeated until the global model ω t convergence.

[0161] The flowchart of the entire specific implementation method is as follows Figure 2 shown.

[0162] Those skilled in the art will understand that the foregoing descriptions are merely preferred embodiments of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art will still be able to modify the technical solutions described in the foregoing examples or substitute equivalent features for some of the technical features. Any modifications, equivalent substitutions, etc. made within the spirit and principles of the invention shall be included within the scope of protection of the invention.

Claims

1. A device scheduling method based on combining channel and local model update, characterized in that: The method comprises the following steps: Step 1: Build a federated learning system In an edge intelligence scenario, there are K single-antenna edge mobile devices, represented as and a parameter server with M antennas, each edge mobile device Both have local datasets Representation dataset In federated learning, the parameter server and edge mobile devices jointly train a global model ω. The federated learning training process is a cyclic process. Each cycle is called a training round. Each training round obtains a new global model. t represents the global model obtained in the t-th training round, ω 0 represents the initial global model that has not been trained by federated learning, ω t-1 Represents the global model obtained in the previous training round, forming a federated learning system; Step 2: Combining the channel and local model update parameters, the parameter server schedules the edge mobile devices participating in the training. Combining the channel and local model updates determines the device scheduling priority. The device scheduling priority is defined as follows: I k,t =c||h k,t ||+(1-c)||g k,t || (1) Among them I k,t represents the scheduling priority of edge mobile device k in the tth training round, c∈[0,1] is a hyperparameter used to control the influence weight of the two scheduling parameters; g k,t represents the local gradient; h k,t represents the channel gain vector; Step 3: The parameter server takes the global model ω obtained in the previous cycle t-1 Send it to all scheduled edge mobile devices; Step 4: The scheduled edge mobile devices use the stochastic gradient descent algorithm to perform local training to obtain their respective local gradients; Step 5: The scheduled edge mobile device uploads the obtained local gradient to the parameter server, updates the global model, and obtains ω t ,The process of uploading local gradients adopts air calculation and optimizes the air calculation; Steps 2-5 are repeated until the global model ω t convergence.

2. The device scheduling method based on combining channel and local model update according to claim 1, characterized in that: The step 2 specifically includes: Channel parameters: In the tth round of training, there are different channel gain vectors h between the parameter server and each edge mobile device k k,t Each edge mobile device sends a pilot sequence to the parameter server, and the parameter server estimates the channel gain vector h between each edge mobile device and the parameter server based on the received pilot sequence k,t ; Use the channel gain vector h k,t The l2 norm of represents the channel parameter between the parameter server and the edge mobile device k, where Represents vector h k,t The absolute value of the i-th component. Since the parameter server has a total of M antennas, the channel gain vector h k,t There are M components in total; Local model update parameters: In the tth round of training, the edge mobile device k performs local training and obtains the local gradient g k,t , using the local gradient g k,t The l2 norm of ||g k,t || represents the local model update parameter of edge mobile device k; || g k,t The larger the value of ||, the greater the improvement in the effect of local training on device k on federated learning training. Since the edge mobile device k does not undergo local training before being scheduled, and the local training result g is used during scheduling k,t , calculate g k,t The l2 norm of ||g k,t || guides scheduling, so it is necessary to perform the l2 norm of the local gradient obtained by local training ||g k,t ||Make an estimate; I k,t Represents the scheduling priority of the edge mobile device k in the t-th training round. The scheduling priorities I of all edge mobile devices k k,t are sorted in descending order. When the number of scheduled devices is fixed at N, where 0 < N < K, select the first N I k,t with the largest values to participate in the federated learning training in the t-th round; The set of edge mobile devices scheduled in the t-th training round is:

3. The device scheduling method based on combining channel and local model update according to claim 2, characterized in that: The l2 norm of the local gradient obtained by local training || g k,t || Make an estimate, specifically: Since the local gradients calculated by the edge mobile device in each training round have strong temporal correlation, the l2 norm of the gradient in the most recent training round is used to estimate the l2 norm of the gradient in the current training round. Specifically, in the tth training round, the estimated local gradient l2 norm is: where t k Represents the last scheduled federated learning training round of device k; In the first round of federated learning training, all edge mobile devices need to participate in the training and upload the local gradients of all edge mobile devices so that the parameter server can estimate the gradient l2 norm of subsequent training rounds; when the edge mobile device k is scheduled in the tth round of training, the parameter server uses the local gradient g after local training k,t renew 4. The device scheduling method based on combining channel and local model update according to claim 1, characterized in that: The step 4 specifically includes: The scheduled edge mobile devices use the stochastic gradient descent algorithm for local training. The stochastic gradient descent algorithm first constructs a loss function and then uses the loss function to calculate the local gradient. The scheduled edge mobile device receives the global model ω obtained in the previous training round t-1 , construct a global loss function for the entire federated learning system, in Represents the sample capacity of the entire federated learning system, F k (ω t-1 ) is the global model ω obtained in the previous cycle t-1 The local loss function on the edge mobile device k, the global loss function is the weighted average of the local loss functions of all edge mobile devices; the local loss function on the edge mobile device k Where (x k ,y k ) represents the data sample on device k, f(x k ,y k ;ω t-1 ) is a single sample (x k ,y k ) corresponding to the loss function, f(x k ,y k ;ω t-1 ) measures the global model ω t-1 For the sample (x k ,y k )’s matching performance; Construct the formula for obtaining the local gradient by performing local training on edge mobile device k in the tth training round: in represents the derivative of the loss function, is the number of training rounds from the dataset A small batch of samples is obtained by random selection in is a small batch of samples The number of samples in g; k,t is the local gradient obtained by local training.

5. The device scheduling method based on combining channel and local model update according to claim 1, characterized in that: The step 5 specifically includes: The local gradient g obtained by the scheduled edge mobile device k,t Upload to the parameter server and update the global model. First, build the formula for the parameter server to update the global model: where η t is the learning rate in the tth training round, is the number of devices in the scheduling device set; The process of uploading gradients adopts air calculation. Air calculation refers to the use of the superposition characteristics of the channel to realize the aggregation of local gradients during the transmission of local gradient signals, that is, to complete the calculation in formula (5) during the transmission of local gradient signals. Part of the calculation; the specific implementation process of the aerial calculation is as follows: In the tth round of training, the scheduled edge mobile devices simultaneously upload the calculated local gradient g k,t , all the sent local gradients are aggregated in the air, and the aggregation signal received by the parameter server is: where p k,t is the transmitter scalar of device k in the tth training round; n is a scalar with mean 0 and variance σ 2 Gaussian white noise vector; The parameter server assigns a beamforming vector to the received aggregate signal. The aggregate signal processed by the parameter server is: where m is the receiver beamforming vector, and the superscript H represents the transpose. That is, local gradient signal aggregation completed by air calculation; The aggregation of local gradients achieved by air calculations is affected by channel fading and noise, so air calculations are optimized; In an ideal channel without channel fading and noise interference, the ideal aggregate signal is: The error between the ideal aggregate signal and the actual aggregate signal is expressed as the mean square error, which is expressed as follows: Where E is the symbol for finding mathematical expectation; In order to reduce the influence of channel gain and noise during the over-the-air calculation process and improve the performance of over-the-air calculation, it is necessary to design the transceiver according to the mean square error minimization criterion to minimize the mean square error between the ideal aggregate signal and the actual aggregate signal.

6. The device scheduling method based on combining channel and local model update according to claim 5, characterized in that: The designing of the transceiver according to the mean square error minimization criterion to minimize the mean square error between the ideal aggregate signal and the actual aggregate signal specifically includes: Designing a transceiver means determining the transmitter scalar p k,t and the receiver beamforming vector m; The transmitter scalar is designed as: Where μ is the transmit power control factor, |p k,t | 2 ≤P0, P0 is the maximum transmit power, symbol || || 2 is to find the square of the l2 norm of its middle vector; The transmit power control factor is designed as: Substituting equations (10) and (11) into equation (7), the actual aggregate signal is simplified to: The mean square error between the actual aggregate signal and the ideal aggregate signal is further expressed as: The optimization problem is constructed with the goal of minimizing the mean square error: Introducing a dummy variable The optimization problem (14) is transformed into the following form Reassign m, We obtain the equivalent optimization problem: Solve the optimization problem in equation (16) to obtain the receiver beamforming vector m.

7. The device scheduling method based on combining channel and local model update according to claim 6, characterized in that: The optimization problem in formula (16) is solved, specifically including: Use the semi-definite relaxation method SDR to find the initial solution of the above problem, and then use the continuous convex approximation algorithm SCA to optimize the initial solution; The SDR method finds the initial solution as follows: Let A = mm H , A * =min A tr(A), where tr(A) represents the trace of matrix A, and λ1 is A * The largest eigenvalue, u1 is the eigenvector corresponding to λ1; If A * The rank of is 1, then is the optimal solution to the optimization problem; If A * The rank is not 1, and the initial solution is Optimize; The specific steps of SCA method to optimize the initial solution are as follows: In the optimization problem of formula (16), the non-convex restriction is Introduce auxiliary variable c k,t =[Re(m H h k,t ),Im(m H h k,t )], formula (16) is converted to: The non-convex constraint in (17) is use Iterative relaxation of linear constraints performs a convex approximation to the non-convex constraints; where c k,t (l) is the solution after l iterations of optimization, Replacing the non-convex constraint in Equation (17) with the above convex constraint, Equation (17) can be rewritten as: make Iterate and solve equation (18) until Where ε represents the solution accuracy, and the corresponding optimal solution is the receiver beamforming vector m.

Images