Asynchronous cluster personalized federal learning method assisted by multifunctional unmanned aerial vehicle

Abstract

The invention relates to an asynchronous cluster personalized federal learning method assisted by a multifunctional unmanned aerial vehicle, and belongs to the technical field of wireless communication. The method comprises the following steps: S1, establishing an FL network system model assisted by an unmanned aerial vehicle; s2, establishing an unmanned aerial vehicle logistics transportation and auxiliary FL training model; s3, establishing a communication time delay model, a model training time model and an uplink time delay model between the unmanned aerial vehicle and the sub-server; s4, establishing a personalized federated learning mechanism, solving a personalized model by an inner layer, solving a global model by an outer layer, and training the personalized model through asynchronous double-layer parallel optimization and an intra-group synchronous federated average mechanism; s5, optimizing the flight path of the unmanned aerial vehicle by taking the obsolessness of the group model as an optimization variable; and S6, adjusting an optimization variable according to a relationship between model obsolessness and user equipment training times, and proposing an unmanned aerial vehicle path dynamic optimization algorithm based on deep reinforcement learning. According to the multifunctional unmanned aerial vehicle path optimization algorithm provided by the invention, the overall communication efficiency is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of wireless communications and relates to a multifunctional unmanned aerial vehicle-assisted asynchronous cluster personalized federated learning method. Background Art

[0002] Federated learning (FL) is an emerging distributed machine learning paradigm that has attracted widespread attention from industry and academia because it can train global models for all participating devices while protecting user data privacy. However, in practical applications, traditional FL faces challenges in device data heterogeneity and communication efficiency.

[0003] Data heterogeneity stems from differences in device attributes, preferences, and data collection patterns. To address this challenge, personalized federated learning (PFL) was proposed, which aims to develop personalized solutions for each device, pay attention to data differences, and improve model performance. The communication efficiency is challenged because traditional FL needs to wait for all devices to complete training and upload models before entering the next round of global training. This causes devices with fast training speeds to wait for slower devices, resulting in a waste of time. To solve this problem, asynchronous federated learning (AFL) came into being. AFL allows devices to enter the next round immediately after completing training, greatly reducing training time. However, AFL also brings about the problem of model staleness, that is, the time when devices upload models is different, causing some devices to update based on outdated global models, which in turn affects the model aggregation effect and communication efficiency. Existing research mostly starts with adjusting the model update strategy to indirectly solve this problem, rather than directly improving communication efficiency.

[0004] Drones are widely used in wireless communications, cargo delivery, data collection, and edge computing due to their high maneuverability and LoS communication links. To address the above issues, drones can be used to improve the communication efficiency between users and servers, accelerate model transmission, and reduce communication latency. However, existing drone-assisted AFL solutions do not study how to reduce the impact of model staleness. In order to improve the performance of AFL in data heterogeneous scenarios, some work has proposed APFL, but these studies mainly focus on designing personalized solutions and improving communication efficiency. There is a lack of research on the impact of model staleness, and most of them remain at the individual level, without involving the challenges of group data heterogeneity. In addition, with the development of science and technology and the maturity of drone technology, drones with single functions have limitations in performing multiple tasks, resource utilization, and cost-effectiveness, and are difficult to meet diverse needs. Summary of the invention

[0005] In view of this, the object of the present invention is to provide an asynchronous cluster personalized federated learning method assisted by a multifunctional UAV, aiming to utilize the high maneuverability of the multifunctional UAV to improve the communication efficiency of the FL system and reduce the average staleness of the model at the group level.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A multifunctional drone-assisted asynchronous cluster personalized federated learning method includes the following steps:

[0008] S1: Establish a multifunctional UAV-assisted FL network system model, which includes a multifunctional UAV equipped with communication, computing modules and cargo compartment as the edge server of the network system, a ground base station and a server forming a sub-server, and user devices associated with the same sub-server forming a group;

[0009] S2: Establish a multifunctional UAV model that includes logistics transportation and assisting ground user groups to conduct FL training. At the same time, establish a LoS wireless communication channel model between the UAV and the sub-server and a simplified wireless communication model between the sub-server and the associated user group;

[0010] S3: Combined with S2, a model of downlink communication delay, model training time and uplink communication delay between the multifunctional UAV and the sub-server is established;

[0011] S4: Establish a federated learning mechanism to asynchronously train personalized models for user groups in response to data heterogeneity at the user group level; Based on a two-layer parallel optimization problem in which the inner layer solves the personalized model and the outer layer solves the global model, use asynchronous two-layer parallel optimization between groups and synchronous federated averaging mechanism within the group to train personalized models for the group;

[0012] S5: Under the constraints of cargo delivery, UAV flight and training time slots, the staleness of the group model is used as the optimization variable to construct an optimization problem and optimize the flight path of the multifunctional UAV;

[0013] S6: According to the relationship between model staleness and the number of times the user device completes the training task, the optimization variables in S5 are modified, and the constraints are kept unchanged. The transformed optimization problem is an integer nonlinear optimization problem. A UAV path dynamic optimization algorithm based on deep reinforcement learning DRL is proposed.

[0014] Furthermore, in S1, a multifunctional UAV-assisted FL network system model is established, including the following processes:

[0015] The FL network system model includes a UAV equipped with communication, computing modules and cargo compartment, M ground base stations and sub-servers composed of servers, and N users. The set of sub-servers is represented as The set of users is represented as The ML task is performed in a FL network assisted by a multifunctional UAV. The multifunctional UAV acts as a mobile base station and edge server, connected to the sub-server on the ground through a wireless channel; the multifunctional UAV maintains a safe height H sf , fly from the starting point to the end point, and deliver the goods to O receiving points, The coordinates of in Represents the collection of these receiving points; each subserver are all located on the horizontal plane, and their coordinates are Each user Equipped with an antenna located on a horizontal plane; each sub-server A group of users is served through a wireless channel. This group of users constitutes a user group. in Use a binary random variable X m,n ∈{0,1} represents the user in group m The state of X m,n =1 means that user device n can participate in model training, X m,n =0 means that user device n cannot participate in model training; define the user in group m The probability of participating in model training is Groups User devices in The dataset used is represented as Thus the total data set of group m is expressed as And the data distribution between different groups m is non-IID at the group level.

[0016] Furthermore, in S2, a multifunctional UAV model including logistics transportation and assisting ground user groups in FL training is established, including the following processes:

[0017] The flight time of the multi-functional UAV is divided into K time slots, using represents the index of the flight time slot, the length of each flight time slot is τ; the maximum flight distance of each time slot of the multi-functional UAV is D max , the horizontal projection of the flight area is Since the cargo delivery task has success or failure, a binary random variable is used Indicates the delivery point The delivery status of goods, including Indicates that the goods have been successfully delivered. Indicates that the goods have not been delivered; assuming that the drone descends vertically from the air above the receiving point o when delivering goods, and the time consumed in the middle is not included in the flight time slot, the multi-functional drone needs to complete the delivery of all goods before the end of the flight. The cargo delivery status in the Kth flight time slot should meet the following constraints:

[0018]

[0019] The flight path length of the multi-function UAV is limited by its own battery capacity. Assume that the maximum flight path length of the UAV is Q max time slot; the drone is in The horizontal projection coordinates of the time slots are Indicates that the trajectory is express.

[0020] Furthermore, in S2, a LoS wireless communication channel model between the multifunctional UAV and the sub-server is established, which includes the following processes:

[0021] In each time slot k, the sub-server accesses the multifunctional UAV through orthogonal frequency division multiplexing, with a total bandwidth of B, and each sub-server gets an average bandwidth of B m =B / N; Assuming that the wireless communication channel between the multifunctional UAV and the sub-server is a LoS channel, and the channel gain at 1 meter is β0, the channel gain between the multifunctional UAV and the sub-server m is as follows:

[0022]

[0023] Furthermore, in S2, the wireless communication model between the sub-server and the associated user group includes the following process:

[0024] Assume that in each group iteration, the sum of the downlink communication delay between the subserver and the user, the user's local update time, and the uplink communication delay is less than τ e .

[0025] Furthermore, in S3, a downlink communication delay, model training time and uplink communication delay model between the multifunctional UAV and the sub-server are established, including the following processes:

[0026] Use p u [k] represents the transmission power of the multi-functional UAV, p m [k] represents the transmission power of the sub-server, N0 represents the noise power, and combined with S2, the downlink communication delay, model training time, and uplink communication delay between the multi-functional UAV and the sub-server are modeled, including the following sub-steps:

[0027] S31: Establish a downlink communication delay model, the content of which is as follows:

[0028] At the beginning of each training time slot k, the multi-function UAV broadcasts the latest global model to all sub-servers; assuming that the bandwidth of the broadcast channel is B broadcast , then in the training time slot k, the downlink communication data rate between the multi-function UAV and the sub-server m is:

[0029]

[0030] Assuming that the number of parameters of the ML model is s, where each parameter is stored as a b-bit floating point number, the downlink communication delay is:

[0031]

[0032] S32: Establish a model training time model, the content is as follows:

[0033] Assume that the time required for each group iteration is τ e , then in the training time slot k, the model training time required for each global iteration of the idle user group and the associated sub-server m is:

[0034]

[0035] Where E is the number of group iterations;

[0036] S33: Establish an uplink communication delay model, the content of which is as follows:

[0037] In training time slot k, the uplink communication rate of subserver m is:

[0038]

[0039] The uplink communication delay is:

[0040]

[0041] Furthermore, in S4, a federated learning mechanism is established to asynchronously train personalized models for user groups in view of data heterogeneity at the user group level, including the following steps:

[0042] S41: Establish a two-level parallel optimization problem for group asynchronous personalized federated learning assisted by multifunctional drones, the content is as follows:

[0043] The following optimization problem P1 is established to find its optimal solution w * :

[0044] P1:

[0045] Among them, u m represents the personalized model of group m, w represents the global model, and F m(w) is defined as the following optimization problem P2:

[0046] P2:

[0047] Among them, λ is a regularization parameter used to control the personalized model u m The difference with the global model w; f m (u m ) represents the training loss of all users in group m The sum of n (u m ) represents the personalized model u m Locally private datasets The loss function on , and F m The definition of (w) is related to the Moreau envelope;

[0048] In the two-layer parallel optimization problem represented by optimization problems P1 and P2, P1 is the outer optimization problem for solving the global model w, and P2 is the outer optimization problem for solving the personalized model u. m The inner optimization problem of the two optimization problems is decoupled;

[0049] S42: The group model and the personalized model are synchronously updated to obtain a personalized model and a group model, the contents of which are as follows:

[0050] Use e∈E to denote the index of E group iterations, where is the index set of E group iterations; at the beginning of each group iteration, subserver m broadcasts the latest group model to all members in group m who can participate in training The user n participating in the training calculates the user model by performing gradient descent on the optimization problem P2 When all users participating in the training have completed the training, sub-server m aggregates the user models of all users participating in the training. To generate a personalized model; after the e∈Eth group iteration, the group model calculates the updated group model by performing gradient descent on the optimization problem P1; finally, after E group iterations, the obtained group model is uploaded to the multi-functional UAV to complete the group training task after a training time slot update;

[0051] S43: Global model asynchronous update, the content is as follows:

[0052] Before performing asynchronous update of the global model, the staleness of the group model uploaded by the sub-server is first analyzed. In the example, we use a binary random variable π m [k]∈{0,1} represents a group Whether the training task is completed in this time slot, the training task status of the group is updated in the multi-functional drone, where π m [k] = 1 means that group m has completed this task and is ready for subsequent training tasks, while π m [k] = 0 means other situations; sequence Contains group m in all training time slots The state of; the set of user groups that can participate in model training in training time slot k Including all π m [k-1] = 1 group; use γ m [k] represents the group model uploaded by sub-server m in training time slot k If the group model does not exist, then its corresponding γ m [k] does not exist, group model The obsolescence of m [k] is updated as follows:

[0053]

[0054] Among them, k m is the index of the global model used by the collaborative training model of sub-server m and the associated user group m, N / A indicates staleness γ m [k] does not exist, it is not difficult to find that if π m [k] = 1, then γ m [k] represents the sequence π m The time slot k is the same as the previous π m = the number of zeros between time slots of 1; if we define 0×N / A=0, and use the integer ν k ≥0 represents the sequence π m The number of trailing zeros gives the following formula:

[0055]

[0056] Combined with the above formula, the average staleness of the local model uploaded by group m is Satisfies the following inequality:

[0057]

[0058] The larger the value of The smaller it is, that is, the more times group m completes training tasks, the smaller the average staleness of the local model uploaded by group m; combining the downlink communication delay, model training time and uplink communication delay model in S3 to update the group training task completion state sequence π m ; Finally, the global model is obtained by aggregation update through the following formula:

[0059]

[0060] Among them, g(γ m [k]) is a dynamic adaptive discount factor used to reduce the impact of stale group models on the global model. The function g(·) is defined as g(x) = (x+1) -a , a is a positive constant; parameter β>0 is used to control the model aggregation to the global model w k The smoothing coefficient of the influence of .

[0061] Furthermore, in S5, under the constraints of the number of cargo delivery, UAV flight and training time slots, an optimization problem of minimizing the obsolescence of the group model is constructed to optimize the flight path of the multifunctional UAV, including the following processes:

[0062] When the multifunctional UAV is in different locations, the uplink and downlink communication delays between it and the sub-server are different, that is, the time required for the group to complete a training task is inconsistent; a multifunctional UAV path optimization problem that minimizes the average staleness of the group model is constructed. By optimizing the flight path of the multifunctional UAV, the average staleness of all group models is minimized while completing the cargo delivery task. The mean

[0063] Further, in S6, according to the average obsolescence of the model in S4 and the number of group training tasks The relationship between the optimization variables will be minimized, that is, the average staleness of all group models Modify it to maximize the number of group training task completions and keep the constraints of the original optimization problem unchanged.

[0064] Furthermore, in S6, a UAV path dynamic optimization algorithm based on deep reinforcement learning (DRL) is proposed to solve the modified optimization problem, which includes the following steps:

[0065] A1: To implement the DRL method, the flight actions of the multi-functional UAV need to be simplified and discretized. Modeled as a A square grid map, where the size of each grid is D max , flight area in The multi-function drone can only fly in four directions: "east", "south", "west" and "north", and each flight distance is D max , that is, each time slot can only fly from one grid to four adjacent grids;

[0066] A2: Establish the DRL model action space;

[0067] A3: Establish a state space consisting of environmental information and multi-function UAV state, where the environmental information includes the starting point coordinates q start , end point coordinate q end And multi-function drones to the border The state of the multi-function UAV includes the position q[k] of the current time slot, the number of time slots flown k, the position two time slots ago, and the cargo delivery status

[0068] A3: Construct the reward function of the DRL model to ensure that the multi-functional drone completes the delivery of all goods and the path length is within the specified range. The reward function should include the round reward R episode 、Safety Reward R safe , Mobile Reward R move , Group Training Task Reward R π and cargo delivery incentives;

[0069] A4: Use the DQN algorithm to solve the flight path of the multi-functional UAV. In each round of each iteration, it includes three parts: the interaction between the multi-functional UAV and the environment, experience playback, and the DQN algorithm with the target Q network. The process of the third part will be iterated multiple times until the end of the round or the maximum number of iterations is reached.

[0070] The beneficial effects of the present invention are:

[0071] (1) The present invention breaks the inherent personalized design that is only for the individual level, and proposes a PFL solution for the group level, which is more suitable for actual task requirements. At the same time, the traditional PFL solution does not consider reducing the obsolescence of user training or reducing the impact of obsolescence. Therefore, from this perspective, the present invention proposes an asynchronous cluster personalized federated learning method that takes into account communication efficiency and data heterogeneity;

[0072] (2) In the asynchronous cluster personalized federated learning method assisted by a multifunctional UAV proposed in the present invention, a multifunctional UAV is used to assist the ground user group and the sub-server base station to train the FL network. Compared with the fixed cloud server, the multifunctional UAV is more flexible and can adapt to the changing communication channel. At the same time, it can also complete multiple tasks and improve the utilization efficiency of resources.

[0073] (3) In the asynchronous cluster personalized federated learning method assisted by a multi-functional drone proposed in the present invention, the proposed path optimization algorithm designs a reward function composed of multiple parts, among which the group training task reward is mainly used to motivate drones to find paths with lower staleness, and reduce model staleness by improving the average uplink communication rate between sub-services, thereby effectively improving communication efficiency.

[0074] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0076] Figure 1 A detailed flowchart of the personalized federated learning method for asynchronous clusters assisted by multifunctional drones;

[0077] Figure 2 A practical application scenario diagram of the asynchronous cluster personalized federated learning method assisted by multifunctional drones;

[0078] Figure 3 A grid diagram of the UAV flight paths generated by two comparative path optimization algorithms and the algorithm proposed in the present invention;

[0079] Figure 4 for Figure 3 Scatter plot of the obsolescence of paths generated by the comparison algorithm in and the algorithm proposed in the present invention;

[0080] Figure 5 As the time slot increases, the drone moves along Figure 3 The average uplink transmission rate diagram of the user when the path generated by the comparison algorithm and the algorithm proposed in the present invention is flying;

[0081] Figure 6 for Figure 3 The reward curves of the three path optimization algorithms;

[0082] Figure 7 The average test accuracy of the proposed method and the comparative method under different device availability when the number of devices is fixed, the device groups are divided, and the device data is non-independent and identically distributed (non-IID). DETAILED DESCRIPTION

[0083] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0084] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0085] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0086] like Figure 1 and Figure 2 As shown, a multifunctional drone-assisted asynchronous cluster personalized federated learning method includes the following steps:

[0087] S1: Establish a multifunctional UAV-assisted FL network system model, which includes a multifunctional UAV equipped with communication, computing modules and cargo compartment as the edge server of the network system, a ground base station and a server forming a sub-server, and user devices associated with the same sub-server forming a group;

[0088] S2: Establish a multifunctional UAV model that includes logistics transportation and assisting ground user groups to conduct FL training. At the same time, establish a LoS wireless communication channel model between the UAV and the sub-server and a simplified wireless communication model between the sub-server and the associated user group;

[0089] S3: Combined with S2, a model of downlink communication delay, model training time and uplink communication delay between the multifunctional UAV and the sub-server is established;

[0090] S4: Establish a federated learning mechanism to asynchronously train personalized models for user groups in response to data heterogeneity at the user group level. This mechanism is based on a two-layer parallel optimization problem in which the inner layer solves the personalized model and the outer layer solves the global model. It uses asynchronous two-layer parallel optimization between groups and synchronous federated averaging mechanism within groups to train personalized models for groups.

[0091] S5: Under the constraints of cargo delivery, UAV flight and training time slots, the staleness of the group model is used as the optimization variable to construct an optimization problem and optimize the flight path of the multifunctional UAV;

[0092] S6: According to the relationship between the model staleness and the number of times the user device completes the training task, the optimization variables in S5 are modified, and the constraints are kept unchanged. The transformed optimization problem is an integer nonlinear optimization problem. This step will analyze its structural characteristics and propose a UAV path dynamic optimization algorithm based on deep reinforcement learning (DRL);

[0093] Specifically, a multifunctional UAV-assisted FL network system model is established in S1, including the following processes:

[0094] The network consists of a drone equipped with communication, computing modules and cargo compartments, M ground base stations and sub-servers consisting of servers, and N users. The set of sub-servers is represented by The set of users is represented as The ML task is performed in a FL network assisted by a multifunctional UAV. The multifunctional UAV acts as a mobile base station and edge server, connected to the sub-server on the ground through a wireless channel; the multifunctional UAV maintains a safe height H sf , fly from the starting point to the end point, and deliver the goods to O receiving points, The coordinates of in Represents the collection of these receiving points; each subserver are all located on the horizontal plane, and their coordinates are Each user Equipped with an antenna located on a horizontal plane; each sub-server A group of users is served through a wireless channel. This group of users constitutes a user group. in Due to unstable network connection, device calls or low battery, users in each group are not always available to participate in model training. A binary random variable X m,n ∈{0,1} represents the user in group m The state, specifically, X m,n=1 means that user device n can participate in model training, X m,n =0 means that user device n cannot participate in model training; define the user in group m The probability of participating in model training is User devices in group m The dataset used is represented as Thus the total data set of group m can be expressed as And the data distribution between different groups m is non-independent and identically distributed (non-IID) at the group level;

[0095] Specifically, a multifunctional UAV model including logistics transportation and assisting ground user groups in FL training is established in S2, including the following processes:

[0096] The flight time of the multi-functional UAV is divided into K time slots, using represents the index of the flight time slot, the length of each flight time slot is τ; the maximum flight distance of each time slot of the multi-functional UAV is D max , the horizontal projection of the flight area is Since the cargo delivery task has success or failure, a binary random variable is used Indicates the delivery point The delivery status of goods, including Indicates that the goods have been successfully delivered. Indicates that the goods have not been delivered; assuming that the drone descends vertically from the air above the receiving point o when delivering goods, and the time consumed in the middle is not included in the flight time slot, the multi-functional drone needs to complete the delivery of all goods before the end of the flight. The cargo delivery status in the Kth flight time slot should meet the following constraints:

[0097]

[0098] Because the flight path length of the multi-function UAV is limited by its own battery capacity, it is assumed that the UAV can only fly Q max time slot; the drone is in The horizontal projection coordinates of the time slots are Indicates that the trajectory is It is expressed that the flight constraints of the multifunctional UAV can be written as follows:

[0099]

[0100] C3:q start =q[0]

[0101] C4:q end =q[K]

[0102]

[0103] Among them, q in constraints C3 and C4 start and q end denote the projection coordinates of the starting point and the end point respectively; assuming that the ML model is trained to deliver O from a multi-functional drone min The kth delivery point after the goods start >0 flight slots start and last for K train flight time slots; define the drone delivery time slot min The time slot of the goods at the receiving point is k min , the index set of training time slots is expressed as In the training time slot The idle group set is

[0104] Specifically, a LoS wireless communication channel model between the multifunctional UAV and the sub-server is established in S2, which includes the following processes:

[0105] In each time slot k, the sub-server accesses the multifunctional UAV through orthogonal frequency division multiplexing, with a total bandwidth of B, and each sub-server gets an average bandwidth of B m =B / N; Assuming that the wireless communication channel between the multifunctional UAV and the sub-server is a LoS channel, and the channel gain at 1 meter is β0, the channel gain between the multifunctional UAV and the sub-server m is as follows:

[0106]

[0107] Specifically, the wireless communication model between the S2 neutron server and the associated user group includes the following processes:

[0108] Since the communication distance between the sub-server and the associated user group is shorter than that between the multi-function drone and the sub-server, this step will simplify the wireless communication model between the sub-server and the associated user group; it is assumed that in each group iteration, the sum of the downlink communication delay between the sub-server and the user, the user local update time and the uplink communication delay is less than τ e ;

[0109] Specifically, S3 establishes the downlink communication delay, model training time, and uplink communication delay models between the multifunctional drone and the sub-server, including the following processes:

[0110] This step uses p u [k] represents the transmission power of the multi-functional UAV, p m [k] represents the transmission power of the sub-server, N0 represents the noise power, and combined with S2, the downlink communication delay, model training time, and uplink communication delay between the multi-functional UAV and the sub-server are modeled, including the following sub-steps:

[0111] S31: Establish a downlink communication delay model, the content of which is as follows:

[0112] At the beginning of each training time slot k, the multi-function UAV broadcasts the latest global model to all sub-servers; assuming that the bandwidth of the broadcast channel is B broadcast , then in the training time slot k, the downlink communication data rate between the multi-function UAV and the sub-server m is:

[0113]

[0114] Assuming that the number of parameters of the ML model is s, where each parameter is stored as a b-bit floating point number, the downlink communication delay is:

[0115]

[0116] S32: Establish a model training time model, the content is as follows:

[0117] Assume that the time required for each group iteration is τ e , then in the training time slot k, the model training time required for each global iteration of the idle user group and the associated sub-server m is:

[0118]

[0119] Where E is the number of group iterations;

[0120] S33: Establish an uplink communication delay model, the content of which is as follows:

[0121] In training time slot k, the uplink communication rate of subserver m is:

[0122]

[0123] The uplink communication delay is:

[0124]

[0125] Specifically, S4 establishes a federated learning mechanism for asynchronously training personalized models for user groups based on data heterogeneity at the user group level, which includes the following steps:

[0126] S41: Establish a two-layer parallel optimization problem of group asynchronous personalized federated learning assisted by multifunctional drones. The specific contents are as follows:

[0127] The following optimization problem P1 is established to find its optimal solution w * :

[0128] P1:

[0129] Among them, u m represents the personalized model of group m, w represents the global model, and F m (w) is defined as the following optimization problem P2:

[0130] P2:

[0131] Among them, λ is a regularization parameter used to control the personalized model u m The difference with the global model w. m (u m ) represents the training loss of all users in group m The sum of n (u m ) represents the personalized model u m Locally private datasets The loss function on , and F m The definition of (w) is related to the Moreau envelope;

[0132] In the two-layer parallel optimization problem represented by optimization problems P1 and P2, P1 is the outer optimization problem for solving the global model w, and P2 is the outer optimization problem for solving the personalized model u. m The inner optimization problem of the two optimization problems is decoupled;

[0133] S42: The group model and the personalized model are synchronously updated to obtain a personalized model and a group model, the specific contents of which are as follows:

[0134] Use e∈E to denote the index of E group iterations, where is the index set for E group iterations; definition Indicates The global model updated after training time slots, represents the updated group model for the kth training time slot, represents the personalized model updated in the kth training time slot, The group model after the e-th group iteration of training time slots is expressed as Before the e∈Eth group iteration begins, initialize When the group iteration starts, the child server m will update the latest group model Broadcast to all participants in group m The user n participating in the training calculates the user model by performing H steps of gradient descent As shown below:

[0135]

[0136] in, ηt represents the step length, f n (·) represents the user loss function; the sub-server m then aggregates the user models of all users in the associated group that can participate in the training through the following formula To generate a personalized model:

[0137]

[0138] After the e∈Eth group iteration, the group model is calculated by performing gradient descent on the optimization problem P1 to update the group model, as shown in the following formula:

[0139]

[0140] Get the updated group model; finally, after E group iterations, get the personalized model and group models And the group model Upload to a multi-function drone;

[0141] S43: Global model asynchronous update, the specific content is as follows:

[0142] Before performing asynchronous update of the global model, the staleness of the group model uploaded by the sub-server is first analyzed. In the example, we use a binary random variable π m [k]∈{0,1} represents a group Whether the training task is completed in this time slot, the training task status of the group is updated in the multi-functional drone, where π m [k] = 1 means that group m has completed this task and is ready for subsequent training tasks, while π m [k] = 0 means other situations; sequence Contains group m in all training time slots The state of; the set of user groups that can participate in model training in training time slot k Including all π m [k-1] = 1 group; use γ m [k] represents the group model uploaded by sub-server m in training time slot k If the group model does not exist, then its corresponding γ m [k] does not exist, group model The obsolescence of m [k] is updated as follows:

[0143]

[0144] In which, define π m[k star -1]=1 and k m is the index of the global model used by the collaborative training model of sub-server m and the associated user group m, N / A indicates staleness γ m [k] does not exist, it is not difficult to find that if π m [k] = 1, then γ m [k] represents the sequence π m The time slot k is the same as the previous π m = the number of zeros between time slots of 1; if we define 0×N / A=0, and use the integer ν k ≥0 represents the sequence π m The number of trailing zeros can be obtained as follows:

[0145]

[0146] Combined with the above formula, the average staleness of the local model uploaded by group m is Satisfies the following inequality:

[0147]

[0148] From the above formula, we can observe that The larger the value of The smaller it is, that is, the more times group m completes training tasks, the smaller the average staleness of the local model uploaded by sub-server m; combining the downlink communication delay, model training time and uplink communication delay model in S3 to update the group training task completion state sequence π m , including the following:

[0149] Assuming that the length τ of a time slot k is much larger than the downlink communication delay, use represents the time required for user group m to complete model training in training time slot k. Updated by the following formula:

[0150]

[0151] Among them, the definition Indicates that user group m completes model training in training time slot k, and subserver m can upload the obtained group model to the multi-function drone, or subserver m is uploading the group model (if π m [k-1] = 0 and );use represents the remaining data that sub-server m needs to upload in time slot k. In training time slot k, Updated by the following formula:

[0152]

[0153] Among them, the definition represents the uplink communication rate of subserver m in the kth training time slot. Then the subserver m completes the training task in the training time slot k. Therefore, the task state π of user n in the kth training time slot is n [k] is updated by the following formula:

[0154]

[0155] Finally, the global model is obtained by aggregation update through the following formula:

[0156]

[0157] Among them, g(γ m [k]) is a dynamic adaptive discount factor used to reduce the impact of stale group models on the global model. The function g(·) is defined as g(x) = (x+1) -a , a is a positive constant; parameter β>0 is used to control the model aggregation to the global model w k The smoothing coefficient of the influence degree;

[0158] Specifically, S5 constructs an optimization problem to minimize the staleness of the group model under the constraints of the number of cargo delivery, drone flight and training time slots to optimize the flight path of the multi-functional drone, including the following processes:

[0159] When the multifunctional UAV is in different locations, the uplink and downlink communication delays between it and the sub-server are different, that is, the time required for the group to complete a training task is inconsistent. Therefore, the flight path of the multifunctional UAV will not only affect the training task status of the group, but also affect the obsolescence of the group model; this step constructs a multifunctional UAV path optimization problem that minimizes the average obsolescence of the group model. By optimizing the flight path of the multifunctional UAV, the average obsolescence of all group models is minimized while completing the cargo delivery task. The mean The specific optimization problem is as follows:

[0160]

[0161]

[0162]

[0163] The constraints C1, C2, C3, C4, and C5 are the flight constraints of the multifunctional UAV given in S2. is the cargo delivery constraint given in S2, constraint Indicates that the multifunctional UAV is in time slot kstart The remaining flight path length must be greater than the number of training time slots K train ;

[0164] Specifically, S6 specifically includes modifying the optimization variables of the optimization problem established in S5. Since the objective function of the optimization problem established in step 5 is an implicit function of the optimization variables and cannot be solved directly, the average staleness of the model in S4 and the number of group training tasks are used to modify the optimization variables. The relationship between the optimization variables (average staleness of all group models) will be minimized. ) is modified to maximize the number of group training task completions, while keeping the constraints of the original optimization problem unchanged. The modified optimization problem is as follows:

[0165]

[0166]

[0167]

[0168] Specifically, S6 specifically includes proposing a UAV path dynamic optimization algorithm based on deep reinforcement learning (DRL) to solve the modified optimization problem, where the optimization variables are mapped to the action space, the constraints are mapped to the state space, and the reward function is constructed according to the optimization objectives and constraints, which specifically includes the following steps:

[0169] A1: To implement the DRL method, the flight actions of the multi-functional UAV need to be simplified and discretized. Modeled as a A square grid map, where the size of each grid is D max , flight area in The multi-function drone can only fly in four directions: "east", "south", "west" and "north", and each flight distance is D max , that is, each time slot can only fly from one grid to four adjacent grids, so the flight area The bounds of are given by the set defined below:

[0170]

[0171] If the next position of the multi-function drone is Then the multi-function drone will hover in its original position and receive punishment from the environment;

[0172] A2: Establish the DRL model action space, which includes the following:

[0173] In the DRL model, the action space only includes the flight direction of the multifunctional UAV, so the action space is represented by the set It means that the definition is as follows:

[0174]

[0175] A3: Establish a state space consisting of environmental information and multi-function UAV state, where the environmental information includes the starting point coordinates q start , end point coordinate q end And multi-function drones to the border The state of the multi-function UAV includes the position q[k] of the current time slot, the number of time slots flown k, the position two time slots ago, and the cargo delivery status Therefore, the specific state space can be set It is defined as follows:

[0176]

[0177] in,

[0178] A3: Construct the reward function of the DRL model to ensure that the multi-functional drone completes the delivery of all goods and the path length is within the specified range. In which Defined as The reward function should include the round reward R episode 、Safety Reward R safe , Mobile Reward R move , Group Training Task Reward R π and cargo delivery reward R o , so the specific reward function R(k) is obtained by the following formula:

[0179] R(k)=α1R episode [k]+α2R safe [k]+α3R move [k]+α4R π [k]+α5R o [k]

[0180] Among them, α1, α2, α3, α4 and α5 are all positive constants; the round reward R episode [k] is related to the cargo delivery constraints and drone flight constraints in the optimization problem P4. In the last time slot of the round, the round reward can be calculated as follows:

[0181]

[0182] Among them, r success is a large positive number. Other time slots R episode[k] = 0; safety reward R safe [k] is related to the UAV flight constraints in the optimization problem P4. When the next position of the multi-function UAV is on the boundary, that is, Then the multi-function drone will hover in its original position and be punished safe [k]<0; otherwise, p safe [k] = 0, the safety reward can be calculated by the following formula:

[0183]

[0184] Mobile Rewards R move [k] is related to the UAV flight constraints in the optimization problem P4, which is given by a negative heuristic and a non-positive constant Composition, that is The former is when the path length exceeds Q max The latter is the penalty for returning to the position two time slots ago, i.e. where p move <0. The mobile reward can be calculated by the following formula:

[0185]

[0186] where p time is a very small negative number; Group training task reward where r n [k] is the reward for each base station’s task completion status change, which corresponds to the objective function of the optimization problem P4. During the training period, the group training task reward can be calculated as follows:

[0187]

[0188] Note that when When m [k] = 0; Goods delivery reward R o [k] is related to the cargo distribution constraints in the optimization problem P4, which consists of three parts, namely Among them, the first It can be obtained by the following formula:

[0189]

[0190] where p o is a negative constant. If the multifunctional UAV completes the cargo delivery at or before the Kth time slot, that is, but otherwise, Item 2 It is used to reward the multifunctional UAV for completing the delivery of goods to a receiving point. If the multifunctional UAV completes the delivery task in the kth training time slot, then is a positive constant r o , otherwise 0; the last item is a negative heuristic that motivates a multi-purpose drone to fly over a delivery point:

[0191]

[0192] Among them, ω is a positive constant;

[0193] A4: Use the DQN algorithm to solve the flight path of the multifunctional drone. In each round of each iteration, it includes the interaction between the multifunctional drone and the environment, experience playback, and the DQN algorithm with the target Q network. The multifunctional drone inputs the current state s[k] into the Q network to obtain the action a with the largest Q value. max [k], in order to explore the map, the multi-function drone's actions have a certain probability (exploration degree) of being random actions a random [k]; Select a in the multi-function drone max [k] or a random [k] is used as action a[k], and the next state s[k+1] and reward r[k] are obtained according to s[k] and a[k] interacting with the environment; the experience (s[k], a[k], r[k], s[k+1]) obtained from the current interaction is stored in the experience pool; after the number of samples stored in the experience pool reaches the set value, the current parameters of the Q network and the target Q network are defined as w now and Start updating the Q network and Q target network, including the following steps:

[0194] a.1: Perform forward propagation on the Q network and obtain:

[0195] a.2: Perform forward propagation on the target Q network and obtain

[0196] a.3: Calculate TD target Where γ is the discount rate;

[0197] a.4: Calculate TD error

[0198] a.5: Back propagate the Q network to get the gradient

[0199] a.6: Update Q network parameters by gradient descent:

[0200]

[0201] a.7: After a certain number of iterations, use the Q network parameters to update the target Q network parameters;

[0202] The update process of the above Q network and target Q network will be iterated multiple times until the round ends or the maximum number of iterations is reached.

[0203] Furthermore, the present invention uses a multifunctional drone-assisted asynchronous cluster personalized federated learning method to perform simulation analysis:

[0204] Figure 3 The figure is a raster diagram of the flight path of the multifunctional UAV under two comparison algorithms and the algorithm proposed by the present invention; wherein, the comparison algorithm 1 is a random path algorithm, and the comparison algorithm 2 is a shortest path algorithm. Since the coordinates of the starting point, the end point and the delivery point are symmetrical about the diagonal line of the flight area, the shortest path algorithm corresponds to two different paths, indicating the different order of passing through the delivery points; it should be noted that the random path algorithm and the shortest path algorithm are selected as the comparison algorithms of the algorithm proposed by the present invention because the existing latest UAV path optimization algorithm is not suitable for the system constructed by the present invention. Except for the difference in the reward function between the several algorithms and the proposed algorithm, the other simulation parameters are consistent; from Figure 3 The following points can be seen:

[0205] (1) The flight paths found by the algorithm proposed in the present invention are more concentrated near the sub-server than those found by the random path algorithm. The reason is that the reward function of the algorithm proposed in the present invention includes the group training task reward, and the reward is related to the communication distance between the multifunctional UAV and the sub-server. The shorter the communication distance between the multifunctional UAV and the sub-server, the lower the staleness of the group model uploaded by the sub-server, and the greater the group training task reward.

[0206] (2) Multifunctional UAVs in O min = kth time after delivery to 2 receiving points start = 20 flight slots to start cluster asynchronous personalized federated learning training. During this period, the distances between the two paths and the sub-servers are very different, and “path 1” is closer to the sub-server than “path 2”.

[0207] Figure 4 The average model obsolescence scatter plots of the four paths generated by the proposed algorithm and the two comparison algorithms. If the algorithm finds a path that completes the cargo delivery task and reaches the destination in a certain round, there is a horizontal axis representing the round. Figure 4 It can be seen that the average staleness of the paths found by the algorithm proposed in the present invention after convergence is concentrated between 0.15 and 0.2, which is significantly lower than the average staleness of the other two comparison algorithms.

[0208] Figure 5The figure shows the change of the average uplink communication rate of users under the flight paths generated by the algorithm proposed in the present invention and the two comparison algorithms as the time slot increases. It can be seen that in the training time slot, the path corresponding to the algorithm of the present invention significantly improves the average uplink communication rate of users. Since the average uplink communication rate is closely related to the communication distance from the drone to the sub-server, combined with Figure 3 As a result, the flight path of the algorithm of the present invention is more concentrated near the sub-server, so the communication distance between the drone and the sub-server is shorter, which makes the average uplink communication rate of the user higher.

[0209] Figure 6 The return curves corresponding to the four paths generated by the algorithm proposed in the present invention and the two comparison algorithms are shown in Figure 2. Figure 5 It can be seen that the convergence reward value of the algorithm proposed in the present invention is significantly higher than that of the two comparison algorithms. The reason is also that the reward function in the path optimization algorithm proposed in the present invention includes the group training task reward.

[0210] Figure 7 The performance of a multi-functional drone-assisted asynchronous cluster personalized federated learning method in a group-level Non-IID data setting is demonstrated; Figure 6 Comparison scheme 1 is a PFL scheme for solving Non-IID data at the individual level, and comparison scheme 2 is a non-personalized scheme. Figure 6 It can be seen that both the scheme proposed in the present invention and the PFL scheme at the individual level have better performance than the non-personalized scheme, indicating that the personalized algorithm has significant advantages under the Non-IID data setting; the model performance of the scheme proposed in the present invention under this data setting is significantly better than the two comparison schemes, but when the probability of users participating in training decreases, since the amount of data available for training in the group is very close to the amount of data of the user, the performance of the scheme proposed in the present invention is close to that of the PFL scheme at the individual level.

[0211] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A multifunctional drone-assisted asynchronous cluster personalized federated learning method, characterized by: The following steps are involved: S1: Establish a multifunctional UAV-assisted FL network system model, which includes a multifunctional UAV equipped with communication, computing modules and cargo compartment as the edge server of the network system, a ground base station and a server forming a sub-server, and user devices associated with the same sub-server forming a group; S2: Establish a multifunctional UAV model that includes logistics transportation and assisting ground user groups to conduct FL training. At the same time, establish a LoS wireless communication channel model between the UAV and the sub-server and a simplified wireless communication model between the sub-server and the associated user group; S3: Combined with S2, a model of downlink communication delay, model training time and uplink communication delay between the multifunctional UAV and the sub-server is established; S4: Establish a federated learning mechanism to asynchronously train personalized models for user groups based on data heterogeneity at the user group level; Based on a two-layer parallel optimization problem in which the inner layer solves the personalized model and the outer layer solves the global model, the personalized model is trained for the group using asynchronous two-layer parallel optimization between groups and synchronous federated averaging mechanism within the group. S5: Under the constraints of cargo delivery, UAV flight and training time slots, the staleness of the group model is used as the optimization variable to construct an optimization problem and optimize the flight path of the multifunctional UAV; S6: According to the relationship between model staleness and the number of times the user device completes the training task, the optimization variables in S5 are modified, and the constraints are kept unchanged. The transformed optimization problem is an integer nonlinear optimization problem. A UAV path dynamic optimization algorithm based on deep reinforcement learning DRL is proposed.

2. The method of asynchronous cluster personalized federated learning assisted by a multifunctional drone according to claim 1, characterized in that: In S1, a multifunctional UAV-assisted FL network system model is established, including the following processes: The FL network system model includes a UAV equipped with communication, computing modules and cargo compartment, M ground base stations and sub-servers composed of servers, and N users. The set of sub-servers is represented as The set of users is represented as The ML task is performed in a FL network assisted by a multifunctional UAV. The multifunctional UAV acts as a mobile base station and edge server, connected to the sub-server on the ground through a wireless channel; the multifunctional UAV maintains a safe height H sf , fly from the starting point to the end point, and deliver the goods to O receiving points, The coordinates of in Represents the collection of these receiving points; each subserver are all located on the horizontal plane, and their coordinates are Per user Equipped with an antenna located on a horizontal plane; each sub-server A group of users is served through a wireless channel. This group of users constitutes a user group. in Use a binary random variable X m,n ∈{0,1} represents the user in group m The state of X m,n =1 means that user device n can participate in model training, X m,n =0 means that user device n cannot participate in model training; define the user in group m The probability of participating in model training is Groups User devices in The dataset used is represented as Thus the total data set of group m is expressed as And the data distribution between different groups m is non-IID at the group level.

3. The method of asynchronous cluster personalized federated learning assisted by a multifunctional drone according to claim 2, characterized in that: In S2, a multifunctional UAV model including logistics transportation and assisting ground user groups in FL training is established, including the following processes: The flight time of the multi-functional UAV is divided into K time slots, using represents the index of the flight time slot, the length of each flight time slot is τ; the maximum flight distance of each time slot of the multi-functional UAV is D max , the horizontal projection of the flight area is Since the cargo delivery task has success or failure, a binary random variable is used Indicates the delivery point The delivery status of goods, including Indicates that the goods have been successfully delivered. Indicates that the goods have not been delivered; assuming that the drone descends vertically from the air above the receiving point o when delivering goods, and the time consumed in the middle is not included in the flight time slot, the multi-functional drone needs to complete the delivery of all goods before the end of the flight. The cargo delivery status in the Kth flight time slot should meet the following constraints: The flight path length of the multi-function UAV is limited by its own battery capacity. Assume that the maximum flight path length of the UAV is Q max time slot; the drone is in The horizontal projection coordinates of the time slots are Indicates that the trajectory is express.

4. The method of asynchronous cluster personalized federated learning assisted by a multifunctional drone according to claim 3, characterized in that: In S2, a LoS wireless communication channel model between the multifunctional UAV and the sub-server is established, which includes the following processes: In each time slot k, the sub-server accesses the multifunctional UAV through orthogonal frequency division multiplexing, with a total bandwidth of B, and each sub-server gets an average bandwidth of B m =B / N; Assuming that the wireless communication channel between the multifunctional UAV and the sub-server is a LoS channel, and the channel gain at 1 meter is β0, the channel gain between the multifunctional UAV and the sub-server m is as follows:

5. The method of asynchronous cluster personalized federated learning assisted by a multifunctional drone according to claim 4, characterized in that: In S2, the wireless communication model between the sub-server and the associated user group includes the following processes: Assume that in each group iteration, the sum of the downlink communication delay between the subserver and the user, the user's local update time, and the uplink communication delay is less than τ e .

6. The method of asynchronous cluster personalized federated learning assisted by a multifunctional drone according to claim 5, characterized in that: In S3, a downlink communication delay, model training time, and uplink communication delay model between the multifunctional UAV and the sub-server are established, including the following processes: Use p u [k] represents the transmission power of the multi-functional UAV, p m [k] represents the transmission power of the sub-server, N0 represents the noise power, and combined with S2, the downlink communication delay, model training time, and uplink communication delay between the multi-functional UAV and the sub-server are modeled, including the following sub-steps: S31: Establish a downlink communication delay model, the content of which is as follows: At the beginning of each training time slot k, the multi-function UAV broadcasts the latest global model to all sub-servers; assuming that the bandwidth of the broadcast channel is B broadcast , then in the training time slot k, the downlink communication data rate between the multi-function UAV and the sub-server m is: Assuming that the number of parameters of the ML model is s, where each parameter is stored as a b-bit floating point number, the downlink communication delay is: S32: Establish a model training time model, the content is as follows: Assume that the time required for each group iteration is τ e , then in the training time slot k, the model training time required for each global iteration of the idle user group and the associated sub-server m is: Where E is the number of group iterations; S33: Establish an uplink communication delay model, the content of which is as follows: In training time slot k, the uplink communication rate of subserver m is: The uplink communication delay is:

7. The method of asynchronous cluster personalized federated learning assisted by a multifunctional drone according to claim 6, characterized in that: In S4, a federated learning mechanism is established to asynchronously train personalized models for user groups based on data heterogeneity at the user group level, including the following steps: S41: Establish a two-level parallel optimization problem for group asynchronous personalized federated learning assisted by multifunctional drones, the content is as follows: The following optimization problem P1 is established to find its optimal solution w * : P1： Among them, u m represents the personalized model of group m, w represents the global model, and F m (w) is defined as the following optimization problem P2: P2： Among them, λ is a regularization parameter used to control the personalized model u m The difference with the global model w; f m (u m ) represents the training loss of all users in group m The sum of n (u m ) represents the personalized model u m Locally private datasets The loss function on , and F m The definition of (w) is related to the Moreau envelope; In the two-layer parallel optimization problem represented by optimization problems P1 and P2, P1 is the outer optimization problem for solving the global model w, and P2 is the outer optimization problem for solving the personalized model u. m The inner optimization problem of the two optimization problems is decoupled; S42: The group model and the personalized model are synchronously updated to obtain a personalized model and a group model, the contents of which are as follows: Use e∈E to denote the index of E group iterations, where is the index set of E group iterations; at the beginning of each group iteration, subserver m broadcasts the latest group model to all members in group m who can participate in training The user n participating in the training calculates the user model by performing gradient descent on the optimization problem P2 When all users participating in the training have completed the training, sub-server m aggregates the user models of all users participating in the training. To generate a personalized model; after the e∈Eth group iteration, the group model calculates the updated group model by performing gradient descent on the optimization problem P1; finally, after E group iterations, the obtained group model is uploaded to the multi-functional UAV to complete the group training task after a training time slot update; S43: Global model asynchronous update, the content is as follows: Before performing asynchronous update of the global model, the staleness of the group model uploaded by the sub-server is first analyzed. In the example, we use a binary random variable π m [k]∈{0,1} represents a group Whether the training task is completed in this time slot, the training task status of the group is updated in the multi-functional drone, where π m [k] = 1 means that group m has completed this task and is ready for subsequent training tasks, while π m [k] = 0 means other situations; sequence Contains group m in all training time slots The state of; the set of user groups that can participate in model training in training time slot k Including all π m [k-1] = 1 group; use γ m [k] represents the group model uploaded by sub-server m in training time slot k If the group model does not exist, then its corresponding γ m [k] does not exist, group model The obsolescence of m [k] is updated as follows: Among them, k m is the index of the global model used by the collaborative training model of sub-server m and the associated user group m, N / A indicates staleness γ m [k] does not exist, it is not difficult to find that if π m [k] = 1, then γ m [k] represents the sequence π m The time slot k is the same as the previous π m = the number of zeros between time slots of 1; if we define 0×N / A=0, and use the integer ν k ≥0 represents the sequence π m The number of trailing zeros gives the following formula: Combined with the above formula, the average staleness of the local model uploaded by group m is Satisfies the following inequality: The larger the value of The smaller it is, that is, the more times group m completes training tasks, the smaller the average staleness of the local model uploaded by group m; combining the downlink communication delay, model training time and uplink communication delay model in S3 to update the group training task completion state sequence π m ; Finally, the global model is obtained by aggregation update through the following formula: Among them, g(γ m [k]) is a dynamic adaptive discount factor used to reduce the impact of stale group models on the global model. The function g(·) is defined as g(x) = (x+1) -a , a is a positive constant; parameter β>0 is used to control the model aggregation to the global model w k The smoothing coefficient of the influence of .

8. The method of asynchronous cluster personalized federated learning assisted by a multifunctional drone according to claim 7, characterized in that: In S5, under the constraints of the number of cargo delivery, UAV flight and training time slots, an optimization problem of minimizing the staleness of the group model is constructed to optimize the flight path of the multifunctional UAV, including the following processes: When the multifunctional UAV is in different locations, the uplink and downlink communication delays between it and the sub-server are different, that is, the time required for the group to complete a training task is inconsistent; A multifunctional UAV path optimization problem that minimizes the average staleness of the group model is constructed. By optimizing the flight path of the multifunctional UAV, the average staleness of all group models is minimized while completing the cargo delivery task. The mean 9. The method of asynchronous cluster personalized federated learning assisted by a multifunctional drone according to claim 8, characterized in that: In S6, according to the average staleness of the model in S4 and the number of group training tasks The relationship between the optimization variables will be minimized, that is, the average staleness of all group models Modify it to maximize the number of group training task completions and keep the constraints of the original optimization problem unchanged.

10. The method of asynchronous cluster personalized federated learning assisted by a multifunctional drone according to claim 9, characterized in that: In S6, a UAV path dynamic optimization algorithm based on deep reinforcement learning (DRL) is proposed to solve the modified optimization problem, which includes the following steps: A1: To implement the DRL method, the flight actions of the multi-functional UAV need to be simplified and discretized. Modeled as a A square grid map, where the size of each grid is D max , flight area in The multi-function drone can only fly in four directions: "east", "south", "west" and "north", and each flight distance is D max , that is, each time slot can only fly from one grid to four adjacent grids; A2: Establish the DRL model action space; A3: Establish a state space consisting of environmental information and multi-function UAV state, where the environmental information includes the starting point coordinates q start , end point coordinate q end And multi-function drones to the border The state of the multi-function UAV includes the position q[k] of the current time slot, the number of time slots flown k, the position two time slots ago, and the cargo delivery status A3: Construct the reward function of the DRL model to ensure that the multi-functional drone completes the delivery of all goods and the path length is within the specified range. The reward function should include the round reward R episode , Safety Reward R safe , Mobile Reward R move , Group Training Task Reward R π and cargo delivery incentives; A4: Use the DQN algorithm to solve the flight path of the multi-functional UAV. In each round of each iteration, it includes three parts: the interaction between the multi-functional UAV and the environment, experience playback, and the DQN algorithm with the target Q network. The process of the third part will be iterated multiple times until the end of the round or the maximum number of iterations is reached.

Images