Device scheduling method in multi-task federated learning system

Abstract

The application discloses a device scheduling method in a multitask federated learning system, and specific steps of the method are as follows: S1, constructing a system model of multitask federated learning; S2, establishing an optimization problem with the goal of minimizing the time of a multitask federated learning process; S3, scheduling devices to participate in a federated learning task training process; S4, converting the device scheduling process into a multi-armed bandit and a matching process; and S5, designing a device scheduling algorithm. The application is suitable for scheduling the most suitable device for each task in federated learning, thereby minimizing the time delay of the multitask federated learning process.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of federated learning, and particularly relates to a device scheduling method in a multi-task federated learning system. BACKGROUND

[0002] In the traditional cloud-centric approach, the data collected by mobile devices is uploaded and processed in a cloud-based server or data center. However, due to the increasing awareness of privacy protection, this approach is no longer advantageous. In order to protect the privacy of training data and promote collaborative machine learning (ML) of complex models among distributed devices, McMahan et al. introduced a distributed ML method called federated learning (FL) in the article "Communication-efficient learning of deep networks from decentralized data" published in AISTATS. The advantages of efficient use of network bandwidth, protection of privacy, and solution to the "data island" problem make federated learning more advantageous than traditional methods. However, when the number of devices participating in the training process of federated learning is large, the latency caused by wireless transmission and local computation at the device end is still a bottleneck of federated learning. Therefore, how to reduce the latency of federated learning has become a research hotspot in wireless communication, and client scheduling (CS) is an important direction to reduce the training latency of federated learning.

[0003] In 2019, Jiawen Kang et al. published the article "Incentive mechanism for reliable federated learning: A joint optimization approach to combining reputation and contract theory" in IEEE Internet of Things Journal, which introduced a multi-weight subjective logic model to calculate reputation values to schedule reliable devices for federated learning, and designed an effective incentive mechanism to encourage devices with high reputation values to participate in the federated learning training process.

[0004] In 2021, Wenqi Shi et al. published "Joint Device Scheduling and Resource Allocation for Latency Constrained Wireless Federated Learning" in IEEE Transactions on Wireless Communications, which uses a greedy device scheduling algorithm to select devices with small local update times to maximize model accuracy within a given training time and latency-constrained wireless federated learning framework.

[0005] In 2021, Jie Xu et al. published "Client Selection and Bandwidth Allocation in Wireless Federated Learning Networks: A Long-Term Perspective" in IEEE Transactions on Wireless Communications, which presents a new perspective on the allocation of network resources over multiple federated learning training rounds, proposing that learning rounds are not only interdependent in time, but also have different significance for the final learning result.

[0006] In 2021, Mohammad Mohammadi Amiri et al. published "Convergence of Update Aware Device Scheduling for Federated Learning at the Wireless Edge" in IEEE Transactions on Wireless Communications, which proposes to determine the subset of devices to be scheduled in each round and how to allocate resources among participating devices based on the channel conditions of the devices and the importance of the local model updates of the devices.

[0007] The above works are all based on the assumption that the prior information such as wireless channel state information and device computing resource usage is known for device scheduling. However, in actual federated learning scenarios, the server cannot accurately know the channel state, client computing power, and dataset, so the device scheduling problem under the uncertainty of federated learning communication and computing resources is a bottleneck that restricts the performance of federated learning. SUMMARY

[0008] The application aims to solve the problems in the prior art. The application considers the case where prior information such as a wireless channel state and client computing resources is unknown, and proposes a device scheduling method in a multi-task federated learning system. The device scheduling is performed by using a multi-armed bandit and a matching method, so as to minimize the time of a multi-task federated learning process.

[0009] The application adopts the following technical solutions to achieve the above application purpose.

[0010] A device scheduling method in a multi-task federated learning system, and specific steps of the method are as follows:

[0011] S1, constructing a system model of multi-task federated learning;

[0012] S2, establishing an optimization problem with the goal of minimizing the time of a multi-task federated learning process;

[0013] S3, scheduling devices to participate in a federated learning task training process;

[0014] S4, converting the device scheduling process into a multi-armed bandit and a matching process;

[0015] S5, designing a device scheduling algorithm.

[0016] Further, the system model in S1 includes M mobile edge computing servers and K terminal devices. Each server has a task to be trained, that is, a total of M tasks need to be trained. The optimization problem is expressed as:

[0017]

[0018]

[0019]

[0020]

[0021] Further, each server needs to schedule devices from a set of available devices:

[0022]

[0023] wherein, denotes the task m a set of devices scheduled in the tthround, denotes the set of available devices of the task m , and N denotes the number of channels.

[0024] Further, the availability constraint of the device needs to be considered when scheduling the device for each task, and the optimization problem can be expressed as:

[0025]

[0026]

[0027]

[0028]

[0029]

[0030] Furthermore, in the device scheduling process of multi-task federated learning, the prior information of the devices is unknown. In this case, the multi-armed bandit approach is used for device scheduling. The corresponding optimization problem can be restated as finding the optimal action for each task according to a specific strategy to maximize the cumulative reward of multi-task federated learning:

[0031]

[0032]

[0033]

[0034] Furthermore, the update method of the feedback reward value of each "rocker arm" in the multi-arm bandit is:

[0035]

[0036] Furthermore, the update method for the number of times each "rocker arm" is selected in the multi-arm bandit is:

[0037]

[0038] Furthermore, in the process of using multi-armed bandit machines for device scheduling, a matching method is introduced, that is, each server builds a preference list of available devices based on the reward value fed back by the UCB algorithm and the price of the device:

[0039]

[0040] Denote each server's preference list for all available devices as It is to Sorted in descending order, each server schedules devices according to its preference list.

[0041] Furthermore, considering that a device can only train one task in the same round, and there is a situation where a device is scheduled by multiple servers, the "price" of the device is added in the matching process. When a device is scheduled by multiple servers at the same time, the device will increase its price ρ accordingly.m,i , servers that select the same device will recalculate their preference lists. If the "price" of the device is too high, some servers will abandon it. This process continues until there is no situation where a device is selected by multiple servers.

[0042] The beneficial effect of the present invention is that the present invention performs device scheduling by using a multi-armed bandit and a matching method, thereby minimizing the time of a multi-task federated learning process. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is the system model of the present invention;

[0044] Figure 2 is a flow chart of the method of the present invention;

[0045] Figure 3(a), Figure 3(b), and Figure 3(c) are simulation results showing the performance gap between different tasks as the number of communication rounds changes.

[0046] Figure 4(a), Figure 4(b), and Figure 4(c) are simulation comparison diagrams showing how the accuracy of different tasks varies with the number of communication rounds of the present invention.

[0047] FIG5(a), FIG5(b), and FIG5(c) are simulation comparison diagrams showing how the cumulative number of failed devices for different tasks of the present invention changes with the number of communication rounds. DETAILED DESCRIPTION

[0048] The technical solution of the present invention is described in detail below with reference to the accompanying drawings.

[0049] The multi-task federated learning device scheduling method disclosed in the present invention has a system model including: M servers, K devices, and the set of servers is represented as The collection of devices is represented as Each server m, Each task needs to be trained, that is, there are M tasks in total. The goal of the federated learning training process is to schedule the most appropriate device for each task, thereby minimizing the latency of the multi-task federated learning process. The optimization problem is expressed as:

[0050]

[0051]

[0052]

[0053]

[0054]

[0055] Due to the lack of prior information, solving the problem It is not easy. In order to solve the problem The present application solves the problem Modeling as a multi-armed bandit problem, the corresponding optimization problem can be explained as finding the action To maximize the cumulative reward, that is:

[0056]

[0057]

[0058]

[0059] The present application uses upper confidence bound (UCB) and matching algorithm to solve the problem Each server constructs a preference list of devices according to the reward value fed back by the UCB algorithm, and selects the top N devices in its preference list, and the device i feeds back to the task m in the tth round, The calculation method of reward value is:

[0060]

[0061] In this case, there is a situation where a device is selected by multiple servers, however, the present application assumes that a device can only train one task in the same round, therefore, the present application considers the price of the device in the device scheduling process, when a device is selected by multiple servers, the device will correspondingly increase the price: p m,i = p m,i + Δp i , Therefore, each task calculates its preference value for all available devices, and constructs a preference list of devices for each task according to the preference value. The servers that select the same device will recalculate their preference values based on the consideration of the price of the device, and form a preference list, until there is no situation where a device is selected by multiple servers.

[0062] For task m, The set of devices scheduled in the tth round is represented by In order to minimize the time of multi-task federated learning process, the present application proposes a multi-task federated learning device scheduling method based on multi-armed bandit and matching theory, the specific implementation steps are as follows:

[0063] Step 1: initialization

[0064] Initialization Where φ m represents the matching result of task m, and Ω represents the conflict device set selected by multiple tasks. For any task m, Temporarily match any device i, Observe the reward value that each device gives back for the task.

[0065] Step 2: Build a Preference List

[0066] Each task constructs a preference list for devices based on the reward value reported by the device. This preference list is constructed assuming that some prior information, such as channel state information, device computing capabilities, and dataset statistical characteristics, is unknown. Task m calculates its preference value for device i based on the UCB algorithm and the device's price:

[0067]

[0068] In formula (2), ρ m,i It represents the cost that task m needs to pay when selecting device i.

[0069] Set task m, For all available devices The preference list is represented as It is to Sort in descending order.

[0070] Step 3: Match

[0071] (1) When When, if Then m, From its preference list Select your favorite i to send a connection request.

[0072] For any available device i, if it is only selected by one task, such as m, then they are directly matched, that is: φ m =i.

[0073] (2) If Each device i,i∈Ω will increase its corresponding price ρ according to the following formula m,i :

[0074]

[0075] In formula (3), Δρ m,i represents the price that device i increases for task m.

[0076] (2.1) All tasks that have selected device i recalculate their preference values ​​for device i according to formula (2) and update their selection strategies accordingly. If the cost of device i is too high, some tasks will give up selecting this device and choose other devices instead.

[0077] (2.2) Update y according to formula (4) and formula (5) m,i (t) and zm,i (t).

[0078]

[0079]

[0080] (2.3) Repeat the above process.

[0081] Until

[0082] Finally, each task m is based on the derived m Scheduling devices. When the scheduling process of the above devices is completed, the task issuing, training and uploading process is carried out.

[0083] The beneficial effects brought by the method of the application can be further illustrated by the following simulation.

[0084] I. Simulation conditions

[0085] The number of servers M = 3, the number of devices K = 30, the channel gain of the uplink and downlink is composed of small-scale fading and large-scale fading, wherein the small-scale fading is uniformly distributed Rayleigh distribution, and the large-scale fading is generated by the path loss model: PL [dB] = 128.1 + 37.6log 10 (d), the noise power is σ 2 = -107 dB, and the bandwidth of the orthogonal channel is The transmission power of the uplink and downlink is The size of the local model and the global model is The longest communication time is τ m,max = 0.5 s, For handwritten digit recognition, each device uses the mnist dataset to train a FNN network composed of 50 neurons. Due to the large propagation delay or processing delay, some devices selected in some rounds may not be able to send their results to the server for aggregation. Define as the cumulative performance gap, wherein π1 can be the proposed algorithm or the introduced baseline algorithm. Note that due to the large propagation delay or calculation delay, some devices selected in a certain round may not be able to transmit their results to the server for aggregation. The device i that satisfies τ m,i (t) ≥ τ m,max is called a failed device, and the cumulative number of failed devices is counted.

[0086] II. Simulation content and simulation results

[0087] Simulation 1: Simulation comparison of performance gap between different communication rounds by using the method of the application and the random scheduling method respectively.

[0088] Simulation results: As shown in FIGS. 3(a), (b), and (c), the device scheduling algorithm considered in the present application has better performance than the random device scheduling algorithm, and the performance gap increases with the increase of the communication round number, because the proposed device scheduling algorithm continuously learns the data information of each device in the process of performing multi-task federated learning, and balances between the selected devices and the unselected devices.

[0089] Simulation 2: The proposed method and the random scheduling method are respectively used to test the accuracy change with the communication round number on the mnist dataset.

[0090] Simulation results: As shown in FIGS. 4(a), (b), and (c), with the increase of the communication round number, the accuracy first increases and then remains unchanged. At the same time, the convergence speed of the test accuracy of the proposed method with the change of the communication round number is close to the convergence speed of the baseline.

[0091] Simulation 3: The proposed method and the random scheduling method are respectively used to simulate the number of failed devices between different communication round numbers.

[0092] Simulation results: As shown in FIGS. 5(a), (b), and (c), with the increase of the communication round number, the proposed algorithm has fewer failed devices, which indicates that more devices can participate in the learning process and participate in more federated learning communication round processes within a given time interval.

[0093] The above embodiments only illustrate the technical idea of the present application, and cannot limit the protection scope of the present application. Any modification made according to the technical idea of the present application on the basis of the technical scheme falls within the protection scope of the present application.

Claims

1. A method for device scheduling in a multi-task federated learning system, the method comprising: Specific steps of the method are: S1, constructing a system model of multi-task federated learning; S2, establishing an optimization problem with the goal of minimizing the time of the multi-task federated learning process; S3, scheduling devices to participate in the federated learning task training process; S4, converting the device scheduling process into a multi-armed bandit and matching process; S5, designing a device scheduling algorithm; The update mode of the reward value fed back by each "swing arm" in the multi-armed bandit is: The update mode of the number of times each "swing arm" in the multi-armed bandit is selected is: In the process of using the multi-armed bandit for device scheduling, the matching method is introduced, that is, each server constructs a preference list of available devices according to the reward value fed back by the UCB algorithm and the price of the device: representing the preference list of each server for all available devices is to be sorted in descending order, each server schedules devices according to its preference list; M mobile edge computing servers; denotes the set of available devices for task m, N denotes the number of channels; Consider a device can only train one task in the same round, there is a case that one device is scheduled by multiple servers, in the process of matching, add the "price" of the device, when there is a case that one device is scheduled by multiple servers, the device will increase its own price ρ m,i The servers that select the same device will recalculate their preference list until there is no case that one device is selected by multiple servers.

2. The method of claim 1, wherein, The system model in S1 includes M mobile edge computing servers, K terminal devices, each server has a task to be trained, that is, there are M tasks to be trained in total, and the optimization problem is represented as:

3. The method of claim 2, wherein, Each server needs to schedule devices from the set of available devices: wherein, denotes the set of devices scheduled for task m in the tth round, denotes the set of available devices for task m, N denotes the number of channels.

4. The method of claim 1, wherein, When scheduling devices for each task, the availability constraints of the devices need to be considered, and the optimization problem can be represented as:

5. The method of claim 4, wherein, In the device scheduling process of multi-task federated learning, the prior information of the devices is unknown, in this case, the multi-armed bandit method is used for device scheduling, and the corresponding optimization problem can be restated as finding the optimal action for each task according to a specific strategy to maximize the cumulative reward of multi-task federated learning:

Images