Layered federated learning method and device based on unmanned aerial vehicle network, medium and equipment

By using dual-delay deep deterministic strategy gradient algorithm to screen clients and manage drone energy in the layered federated learning of drone networks, the training data loss problem caused by drone power depletion is solved, and training is improved Efficiency and reduce global costs.

CN118917433APending Publication Date: 2024-11-08XIAMEN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410838633.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-26
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

During the stratified federated learning process of drone networks, training data is missing due to drone power depletion, which affects training efficiency.

Method used

By sending a personalized model and an edge model to the to-select client, receiving the training correlation, determining the correlation threshold based on the dual-delay depth deterministic strategy gradient algorithm, filtering the target client, and deciding whether to participate in the next level based on the energy consumption situation Round iterative training to ensure that data is not lost.

Benefits of technology

It effectively avoids the lack of training data caused by drone power depletion, improves the training efficiency of stratified federated learning, and reduces global costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118917433A_ABST
    Figure CN118917433A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a hierarchical federated learning method and device based on an unmanned aerial vehicle network, a medium and equipment. The method comprises the following steps: sending a personalized model and a to-be-trained edge model to each to-be-selected client in a coverage range; receiving the training association degree sent by each to-be-selected client; determining a correlation degree threshold and determining a target client; receiving and aggregating local models trained by each target client to obtain a target edge model of the edge iteration training of the round; according to the energy consumed by the target edge aggregation unmanned aerial vehicle from the beginning of the current round of global iterative training to the end of the current round of edge iterative training, the predicted consumed energy of the next round of edge iterative training and the initial energy of the target edge aggregation unmanned aerial vehicle when the current round of global iterative training is started; and determining whether the target edge aggregation unmanned aerial vehicle can participate in the next round of edge iteration training. According to the technical scheme of the embodiment of the invention, the training process can be prevented from being influenced by power exhaustion of the unmanned aerial vehicle, and the training efficiency of hierarchical federal learning is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to a hierarchical federated learning method, apparatus, medium and equipment based on drone networks. Background Art

[0002] Hierarchical Federated Learning (HFL) is an advanced machine learning method. It is a variant of federated learning. Its basic process is that edge devices train data locally, send updates to intermediate aggregators for integration and synchronization, and finally aggregate models on the central server. Its advantage is that intermediate aggregators are set up to collaborate on model training at three levels, thereby avoiding the huge global cost caused by a large number of edge devices (such as smartphones or drones) communicating directly with the central server. These characteristics make it show great potential and application value in various application scenarios, such as smart transportation, healthcare, and the Internet of Things, and it has also been widely studied.

[0003] Among the current technical solutions, drone-assisted hierarchical federated learning has become a potential solution to improve the scalability and reliability of hierarchical federated learning aggregators. As drones act as intermediate aggregators, they reduce communication overhead and convergence iterations, while improving the transmission performance and coverage of federated learning users. However, in the hierarchical federated learning process based on drone networks, training data cannot be uploaded or saved in time due to the exhaustion of the battery of a drone, resulting in missing training data and greatly affecting the training efficiency. Therefore, how to avoid the influence of the drone battery exhaustion on the training process and thus improve the training efficiency of hierarchical federated learning has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The embodiments of the present application provide a hierarchical federated learning method, apparatus, medium and equipment based on a drone network, which can at least to a certain extent avoid the impact of the training process on the drone's power exhaustion, thereby improving the training efficiency of hierarchical federated learning.

[0005] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0006] According to one aspect of an embodiment of the present application, a hierarchical federated learning method based on a drone network is provided, which is applied to a target edge aggregation drone;

[0007] The method comprises:

[0008] Send its own personalized model and the edge model to be trained to each selected client within the coverage area;

[0009] Receiving a training relevance sent by each of the candidate clients, where the training relevance is determined by the candidate client according to a model similarity between a local model obtained by the candidate client in a previous round of edge iteration training and the personalized model;

[0010] Based on a double-delayed deep deterministic policy gradient algorithm, determining a correlation threshold that maximizes the contribution of the target edge model obtained by iterative training of the target edge aggregation UAV on the historical edge to the global model;

[0011] Determine the candidate client whose training correlation degree is greater than or equal to the correlation degree threshold as the target client of this round of edge iterative training;

[0012] Receiving and aggregating local models obtained by training the edge model to be trained by each target client in each edge iteration to obtain a target edge model for this round of edge iteration training;

[0013] Determine whether the target edge aggregation UAV can participate in the next round of edge iteration training according to the energy consumed by the target edge aggregation UAV from the start of this round of global iteration training to the end of this round of edge iteration training, the predicted consumed energy of the next round of edge iteration training, and its initial energy at the start of this round of global iteration training;

[0014] If the target edge aggregation drone cannot participate in the next round of edge iterative training, the target edge model obtained in this round of edge iterative training is uploaded to the global aggregation drone for global aggregation. Otherwise, the next round of edge iterative training continues.

[0015] According to one aspect of an embodiment of the present application, a hierarchical federated learning device based on a drone network is provided, which is applied to a target edge aggregation drone;

[0016] The device comprises:

[0017] A sending module is used to send its own personalized model and the edge model to be trained to each selected client within the coverage area;

[0018] A receiving module, configured to receive a training relevance sent by each of the candidate clients, wherein the training relevance is determined by the candidate client according to a model similarity between a local model obtained by the candidate client in a previous round of edge iteration training and the personalized model;

[0019] A first determination module is used to determine, based on a double-delayed deep deterministic policy gradient algorithm, a correlation threshold that maximizes the contribution value of the target edge model obtained by iterative training of the target edge aggregation UAV on the historical edge to the global model;

[0020] A second determination module is used to determine the candidate client whose training correlation degree is greater than or equal to the correlation degree threshold as the target client of this round of edge iterative training;

[0021] An aggregation module is used to receive and aggregate local models obtained by training the edge model to be trained by each target client in each edge iteration to obtain a target edge model for this round of edge iteration training;

[0022] A judgment module is used to determine whether the target edge aggregation UAV can participate in the next round of edge iteration training according to the energy consumed by the target edge aggregation UAV from the start of this round of global iteration training to the end of this round of edge iteration training, the predicted consumed energy of the next round of edge iteration training, and its initial energy at the start of this round of global iteration training;

[0023] The processing module is used to upload the target edge model obtained by the current round of edge iterative training to the global aggregation drone for global aggregation when the target edge aggregation drone cannot participate in the next round of edge iterative training; otherwise, continue the next round of edge iterative training.

[0024] According to one aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the hierarchical federated learning method based on the drone network as described in the above embodiment is implemented.

[0025] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the hierarchical federated learning method based on the drone network as described in the above embodiments.

[0026] According to one aspect of an embodiment of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the hierarchical federated learning method based on a drone network provided in the above embodiment.

[0027] In the technical solutions provided in some embodiments of the present application, by sending its own personalized model and the edge model to be trained to each candidate client within the coverage area, receiving the training correlation sent by each candidate client, the training correlation is determined by the candidate client according to the model similarity between the local model obtained by the candidate client in the previous round of edge iterative training and the personalized model, then, based on the double-delay deep deterministic policy gradient algorithm, determine the correlation threshold that maximizes the contribution value of the target edge model obtained by the target edge aggregation drone in the historical edge iterative training to the global model, determine the candidate client whose training correlation is greater than or equal to the correlation threshold as the target client for this round of edge iterative training, and receive and Aggregate the local models obtained by the target clients in each edge iteration to obtain the target edge model of this round of edge iteration training. According to the energy consumed by the target edge aggregation drone from the start of this round of global iteration training to the end of this round of edge iteration training, the predicted energy consumption of the next round of edge iteration training, and its initial energy at the beginning of this round of global iteration training, determine whether the target edge aggregation drone can participate in the next round of edge iteration training. If the target edge aggregation drone cannot participate in the next round of edge iteration training, upload the target edge model obtained in this round of edge iteration training to the global aggregation drone for global aggregation. Otherwise, continue with the next round of edge iteration training. In this way, the loss of training data caused by the exhaustion of power of a drone during training can be prevented, which can avoid affecting the training process, thereby improving the training efficiency of hierarchical federated learning.

[0028] Moreover, the multi-indicator client-UAV association scheme with adaptive threshold combined with the dual-delayed deep deterministic policy gradient algorithm can reduce the global cost in the entire hierarchical federated learning scenario by reducing data heterogeneity and avoiding data missing problems.

[0029] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0031] Figure 1 A schematic diagram of a process of a hierarchical federated learning method based on a drone network according to an embodiment of the present application is shown;

[0032] Figure 2A schematic diagram showing a global cost comparison between a hierarchical federated learning method based on a drone network and other different target client selection methods according to an embodiment of the present application is shown;

[0033] Figure 3 A schematic diagram showing the convergence comparison between the hierarchical federated learning method based on a drone network and other different target client selection methods according to an embodiment of the present application is shown;

[0034] Figure 4 A schematic diagram showing a comparison of the convergence of a hierarchical federated learning method based on a drone network according to an embodiment of the present application and a drone being directly disconnected due to insufficient power;

[0035] Figure 5 A block diagram of a hierarchical federated learning device based on a drone network according to an embodiment of the present application is shown;

[0036] Figure 6 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0037] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0038] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0039] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0040] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0041] Figure 1 A flowchart of a hierarchical federated learning method based on a drone network according to an embodiment of the present application is shown.

[0042] Among them, the drone network may include several edge-aggregated drones with communication connections. The method can be applied to the target edge-aggregated drone in the drone network. It should be noted that the target edge-aggregated drone can be any edge-aggregated drone in the drone network, that is, any edge-aggregated drone in the drone network can implement the hierarchical federated learning method provided in this application.

[0043] Please refer to Figure 1 The hierarchical federated learning method based on the UAV network includes at least steps S110 to S170, which are described in detail as follows:

[0044] In step S110, the personalized model of the client and the edge model to be trained are sent to each selected client within the coverage area.

[0045] In this embodiment, before the hierarchical federated learning starts, the number of drones and clients, data set labels, locations, and movement rules can be initialized, and the global model can be initialized. In addition, the number of local iterations of the client and the accuracy of local training and edge training can also be initialized. The global aggregation count variable i=0 is used to count the number of global aggregations. That is, when a global aggregation is performed due to insufficient power of a drone, the value of the count variable is increased by one, so that the corresponding global aggregation can be known according to the count variable.

[0046] In one example, during the initialization and parameter definition process, the number of initialized drones is M=5, and the number of clients within the coverage area of ​​each drone is M. m =30. In each edge iteration, 30% of the clients in the coverage area of ​​the drone randomly move to the coverage area of ​​another drone, initialize the dataset label of the drone, and give 2-3 dataset labels to each client in the coverage area. Initialize the global model parameter w global The client may include but is not limited to electronic devices with data processing and transmission functions, such as vehicles, mobile terminals, and base stations.

[0047] Before training begins, the central aggregation drone can broadcast the global model w to all clients within the coverage area of ​​the drone swarm. global During the training process, the client can use local data D n Train a local model.

[0048] When the kth round of edge iteration training begins, the target edge aggregation drone m can broadcast its own personalized model and the edge model to be trained to the client set moving within its coverage area. (i.e., a set of clients to be selected), wherein the personalized model can be a model owned by the edge-aggregated drone determined according to the drone data set; the model to be trained can be a target edge model obtained by aggregating the local models of several clients in the previous round of edge iterative training.

[0049] In step S120, a training relevance sent by each of the candidate clients is received, where the training relevance is determined by the candidate client according to the model similarity between the local model obtained by the candidate client in the previous round of edge iteration training and the personalized model.

[0050] In this embodiment, when the selected client receives the personalized model sent by the target edge aggregation drone, it can determine the local model obtained in the previous round of edge iteration training and compare it with the personalized model, so as to determine the model similarity between the two, and then determine the training correlation between the selected client and the next round of edge iteration training task of the target edge aggregation drone. It should be noted that the higher the model similarity, the more concentrated the data features covered by the drone will be, which is not conducive to the rapid convergence of the global model. Therefore, the higher the model similarity corresponding to the selected client, the lower its corresponding training correlation, and the lower the possibility of selecting the selected client for the next round of edge iteration training, thereby making the data distribution more uniform and more conducive to the convergence of the global model.

[0051] In one embodiment of the present application, the candidate client can determine the model similarity R between the local model obtained by the previous round of edge iteration training and the personalized model according to the following formula: k,m,n :

[0052]

[0053] Among them, x j represents the local sample of the jth input, The parameter matrix is The pre-softmax output of the personalized model, the input local sample is x j ; The parameter matrix is The pre-softmax output of the local model of the client, the input local sample is x j , H represents the number of local training times of the client; It is a hyperparameter that can be used to balance the impact of data and model weight distribution on the client.

[0054] In one embodiment of the present application, the candidate client determines the training association degree between itself and the target edge aggregation drone according to the following formula:

[0055]

[0056] Among them, ρ1, ρ2, ρ3 are weight coefficients and ρ1+ρ2+ρ3=1, R k,m,n d k,m,n 、f k,m,n They respectively represent the model similarity between the target edge aggregation drone m and the candidate client n in the kth round of edge iteration training, the distance from the candidate client n to the target edge aggregation drone m, and the CPU clock frequency of the client n, where the CPU clock frequency of the candidate client n can be used to represent the client computing resources.

[0057] Therefore, by combining the model similarity between the local model of the candidate client and the personalized model of the target edge aggregation drone, the distance between the candidate client and the target edge aggregation drone, and the CPU clock frequency of the candidate client to determine the training correlation, the rationality and effectiveness of the determined training correlation are guaranteed, thereby ensuring the accuracy of subsequent target client screening.

[0058] Please continue to refer to Figure 1 In step S130, based on the double-delayed deep deterministic policy gradient algorithm, a correlation threshold is determined that maximizes the contribution value of the target edge model obtained by iterative training of the target edge aggregation drone in the historical edge to the global model.

[0059] In this embodiment, all candidate clients within the coverage of the target edge aggregation drone can communicate briefly with the target edge aggregation drone, and the target edge aggregation drone m calculates the adaptive threshold β according to the reinforcement learning scheme based on double-delay deep deterministic policy gradient k,m (i.e., correlation threshold), different β k,m The selection of will affect the edge training model (i.e., the target edge model), and the target edge model will further affect the training of the global model.

[0060] Therefore, appropriate indicators are needed to observe the contribution of edge models aggregated under different threshold conditions to the global model, and to adjust the correlation threshold accordingly.

[0061] In one example, the change of the global model is predicted by observing the change of the edge model. Specifically, the w of the target edge aggregation drone is selected. k,m With w k-1,m Two indicators of the two edge models on a small batch global test dataset: the training loss value difference ψ1 and the training accuracy difference ψ2, which respectively represent the convergence of the edge model and the improvement of the prediction performance of the edge model.

[0062] ψ1=L(w k-1,m )-L(w k,m )

[0063] ψ2=acc k,m -acc k-1,m

[0064] Among them, L(w k,m ),acc k,m Represent the edge model w k,m The loss value and training accuracy of the indicator in the global test data set. L(w k-1,m ),acc k-1,m Represent the edge model w k-1,m The loss value and training accuracy of the indicator on the global test dataset.

[0065] The contribution of the edge model to the global model can be described by a linear combination of ψ1 and ψ2:

[0066] ε1.ψ1+ε2.ψ2

[0067] Among them, ε1 and ε2 represent corresponding weight coefficients and ε1+ε2=1. It should be noted that different coefficients can represent different experimental environment requirements, such as iid or non-iid.

[0068] It should be understood that effective edge contribution can reduce the global model loss value and / or improve the global model accuracy. Therefore, the purpose of the kth round of edge iteration training can be expressed as selecting a suitable β k,m Maximize the edge contribution, defined as:

[0069]

[0070] However, in order to ensure that we do not excessively pursue accuracy and select clients that are farther away or have less computing resources but contribute more to accuracy and loss rate, we need to limit the energy consumed by the selected client's single-round edge iteration training to be less than the local resources of a given client in a single round. Right now:

[0071]

[0072] This is an implicit Markov decision process in a continuous action space, which can be transformed into a constrained MDP process. The high dimensionality of the state space and the continuous actions can be considered as a continuous space, single agent.

[0073] From this, we analyze its state-action reward-reward model:

[0074] State: A direct indicator for evaluating an edge model is its prediction performance on a small batch of global test sets. Therefore, the state is defined as follows: m (k) = [L k,m ,acc k,m ]

[0075] Action: Once the state is observed, the drone will select and execute an action. Intuitively, the selected β k,m The corresponding action is: a k,m =[β k,m ]

[0076] Reward Function: The reward function specifies the drone's reward in a certain state by selecting an adaptive threshold (action a k,m ) is the contribution of the edge model trained, defined as:

[0077]

[0078] According to the above principle, the double-delayed deep deterministic policy gradient algorithm is iteratively updated to select the β with the largest contribution value. k,m , and the training correlation α with the client feedback k,m,n Determines whether the client participates in this round of edge iteration training.

[0079] Please continue to refer to Figure 1 In step S140, the candidate client whose training correlation degree is greater than or equal to the correlation degree threshold is determined as the target client of this round of edge iteration training.

[0080] In this embodiment, the target edge aggregation drone can compare the determined relevance threshold with the training relevance uploaded by each candidate client, and determine the candidate client whose training relevance is greater than or equal to the relevance threshold as the target client for this round of edge iteration training. It should be understood that the target clients of different rounds of edge iteration training can be the same or different. After determining the target client, the target edge aggregation drone can instruct the target client to perform local training according to the received edge model to be trained.

[0081] In one example, after the target client is determined, the target edge aggregation drone may send an instruction to the target client to instruct the start of training, so that the target client starts a new round of local training after receiving the instruction.

[0082] In step S150, local models obtained by training the edge model to be trained by each target client in each edge iteration are received and aggregated to obtain a target edge model of this round of edge iteration training.

[0083] In this embodiment, when the local training of each target client reaches a predetermined local model accuracy or a certain number of training rounds, the target client can upload the trained local model to the target edge aggregation drone for aggregation. Specifically, the local training process of the target client is as follows:

[0084]

[0085] in, represents the hth local training model of the kth round of edge iterative training, h represents the number of local training rounds of the client, where h = 1, 2, 3...H. k represents the number of edge iterative training rounds of the target edge aggregation drone m, where k = 1, 2, 3...K. represents the data sample obtained from the client in the hth local training, and η is the learning rate.

[0086] When the expected Stop training when

[0087] After receiving the local models uploaded by each target client, the target edge aggregation drone can perform federated averaging according to the following formula to obtain the target edge model of this round of edge iterative training:

[0088]

[0089] in, represents the model parameters uploaded by the target client n under the coverage of the target edge aggregation drone, |D n | represents the dataset size of target client n, Represents the dataset size of all target clients within the coverage area of ​​the target edge aggregation drone m in the kth round of edge iteration training.

[0090] Please continue to refer to Figure 1In step S160, it is determined whether the target edge aggregation UAV can participate in the next round of edge iteration training based on the energy consumed by the target edge aggregation UAV from the start of this round of global iteration training to the end of this round of edge iteration training, the predicted consumed energy of the next round of edge iteration training, and its initial energy at the start of this round of global iteration training.

[0091] In this embodiment, the target edge aggregation drone can check whether its remaining power can support the next round of edge iteration training. Specifically, the target edge aggregation drone can obtain the energy consumed by itself from the start of the i-th round of global aggregation to the k-th round of edge iteration training. The predicted energy consumption of the next round of edge iteration training (i.e., the k+1th round of edge iteration training) And the initial energy of the target edge aggregation drone at the beginning of this round of global iterative training Then, the target edge aggregation drone can compare the relationship between the three to determine whether it can participate in the next round of edge iteration training. It should be understood that a round of global iteration training can include one or more global aggregations of the model, and a global aggregation of a model can include one or more rounds of edge iteration training, and an edge iteration training includes a fixed round of local iteration training.

[0092] In one example, the predicted energy consumption of the next round of edge iteration training is The maximum or average value of the energy consumed in each round of edge iteration training starting from the current round of global iteration training can be used as an estimate.

[0093] In one embodiment of the present application, when the initial energy of the target edge aggregation drone at the beginning of this round of global iterative training is greater than or equal to the energy consumed from the beginning of this round of global iterative training to the end of this round of edge iterative training, and is less than or equal to the sum of the energy consumed from the beginning of this round of global iterative training to the end of this round of edge iterative training and the predicted energy consumption of the next round of edge iterative training (i.e. ), it is determined that the target edge aggregation drone cannot participate in the next round of edge iterative training, otherwise it can participate in the next round of edge iterative training.

[0094] In this way, it can be ensured that the remaining energy of the target edge aggregation drone is sufficient for the next round of edge iteration training, avoiding the impact of data missing on training efficiency.

[0095] Please continue to refer to Figure 1 In step S170, if the target edge aggregation drone cannot participate in the next round of edge iterative training, the target edge model obtained in this round of edge iterative training is uploaded to the global aggregation drone for global aggregation. Otherwise, the next round of edge iterative training is continued.

[0096] In this embodiment, when it is determined that the target edge aggregation drone cannot participate in the next round of edge iterative training, the target edge model obtained by the target edge aggregation drone in this round of edge iterative training can be uploaded to the global aggregation drone. The global aggregation drone can perform global aggregation based on the target edge models uploaded by several edge aggregation drones to obtain a global model.

[0097] Among them, the global aggregation drone can be selected and determined from the edge aggregation drones that can participate in the next round of edge iterative training in the drone network using a greedy algorithm.

[0098] If all drones can participate in the next round of edge iteration training, the edge iteration training will continue until the predetermined edge accuracy is reached.

[0099] Therefore, according to Figure 1 In the embodiment shown, the personalized model and the edge model to be trained are sent to each candidate client within the coverage area, and the training correlation sent by each candidate client is received. The training correlation is determined by the candidate client according to the model similarity between the local model obtained by the candidate client in the previous round of edge iteration training and the personalized model. Then, based on the double-delay deep deterministic policy gradient algorithm, a correlation threshold is determined that maximizes the contribution value of the target edge model obtained by the target edge aggregation drone in the historical edge iteration training to the global model. The candidate client with a training correlation greater than or equal to the correlation threshold is determined as the target client for this round of edge iteration training. The training correlation sent by each candidate client is received and aggregated. The target client treats the local model obtained by training the edge model in each edge iteration to obtain the target edge model of this round of edge iteration training. According to the energy consumed by the target edge aggregation drone from the start of this round of global training to the end of this round of edge iteration training, the predicted energy consumption of the next round of edge iteration training, and its remaining energy at the beginning of this round of global training, it determines whether the target edge aggregation drone can participate in the next round of edge iteration training. If the target edge aggregation drone cannot participate in the next round of edge iteration training, the target edge model obtained by this round of edge iteration training is uploaded to the global aggregation drone for global aggregation. Otherwise, the next round of edge iteration training will continue. In this way, the loss of training data caused by the exhaustion of power of a drone during training can be prevented, which will affect the training process and improve the training efficiency of hierarchical federated learning.

[0100] Moreover, the multi-indicator client-UAV association scheme with adaptive threshold combined with the dual-delayed deep deterministic policy gradient algorithm can reduce the global cost in the entire hierarchical federated learning scenario by reducing data heterogeneity and avoiding data missing problems.

[0101] In one example, the calculation rule of the total global cost in the entire hierarchical federated learning process is as follows:

[0102] In the i-th round of global training, there are a total of Second edge iteration:

[0103] Client time cost and energy cost:

[0104] 1) Local training latency of client n: The amount of local model parameter data sent by client n Required uplink time is the rate at which the client sends data to the drone, so the time cost for each client n to complete a round of edge iteration under drone m is expressed as:

[0105]

[0106] 2) The energy cost consumed by client n to complete one edge iteration: in The energy cost of training the local model, represents the communication energy cost of client n uploading the local model to the associated drone m.

[0107] The time cost and energy cost of drone m as an edge end in a round of global iteration:

[0108] 1) Time cost:

[0109] The first step is to clarify the time cost of drone m in a round of edge iteration in It represents the broadcasting time of the drone broadcasting the edge model to be trained and its own personalized model to the client. Secondly, the total time cost in a round of global iterative training is the sum of the time of the corresponding multiple rounds of edge iterative training:

[0110]

[0111] 2) Energy cost:

[0112] The energy cost of the edge iteration proposed in the present invention is the sum of the energy consumed by the client and the drone. Therefore, in a round of global iterative training, the total energy is the hovering energy cost and the broadcast energy cost of the drone m in each round of edge iteration, that is, Energy cost consumed by clients participating in edge iterations The sum of multiple edge iteration energies of :

[0113]

[0114] When a drone runs out of power during global training, global aggregation is required. This application divides the global training into i global aggregation parts according to the number of global aggregations that occur, where i represents the i-th global aggregation.

[0115] The time cost and energy cost of UAV m uploading model parameters to the global aggregation center UAV m(i) is:

[0116]

[0117] represents the amount of edge model parameter data of drone m after completing the kth round of edge iteration training; represents the transmission rate of drone m to the global aggregation center drone m(i) in the kth round of edge iteration training; Represents the uplink power of UAV m.

[0118] The total time of a single round of global iterative training should be the time cost T incurred by the drone as an edge aggregator in the edge end m , the time cost of sending the target edge model to m(i) m(i) is the time cost of broadcasting the global model Sure.

[0119] The total energy of a single round of global iterative training should be the energy cost E generated by the edge drone end m , the energy cost of uploading the edge model from the online drone to the global aggregation center drone m(i) The hovering cost of the online drone in the process of uploading the edge model and m(i) sending the aggregated global model to all clients and the broadcast energy cost of m(i) broadcasting the aggregated global model to the client constitute.

[0120] In order to take energy costs into consideration to the greatest extent, the client can be required to train locally until it reaches a certain accuracy or converges. Then the time cost and energy cost of a global iteration are:

[0121]

[0122] The hovering cost of all drones during the upload of the target edge model and the download of the global model is mainly composed of the hovering time of the drone and the average power loss during the hovering process. Determine, expressed as:

[0123]

[0124] in, is the set of online drones participating in the i-th global iteration.

[0125] Finally, the global time cost and energy cost of the entire hierarchical federated learning process is the sum of the time cost and energy cost of multiple rounds of global iterations.

[0126]

[0127] in Indicates the number of global aggregations performed in the entire hierarchical federated learning process.

[0128] like Figure 2-3 As shown, based on the above-mentioned global cost calculation rules, the hierarchical federated learning method based on the drone network provided in this application is compared with other algorithms that use different methods to select target clients (such as random selection, selection based on model similarity or distance using a greedy algorithm), which can not only achieve the same accuracy, but also significantly reduce the global cost.

[0129] like Figure 4 As shown, compared with other algorithms, the hierarchical federated learning method based on the drone network provided by the present application performs global aggregation before the drone is disconnected, which can effectively avoid the problem of data loss caused by insufficient power of the drone and reduce the impact on the convergence rate. In addition, using the method provided by the present application, even if 2 or 3 drones are disconnected, the convergence rate is basically the same. Therefore, the method provided by the present application can effectively reduce the impact of the drone disconnection on the convergence rate.

[0130] The following describes an apparatus embodiment of the present application, which can be used to execute the hierarchical federated learning method based on drone networks in the above-mentioned embodiments of the present application. For details not disclosed in the apparatus embodiment of the present application, please refer to the above-mentioned embodiment of the hierarchical federated learning method based on drone networks in the present application.

[0131] Figure 5 A block diagram of a hierarchical federated learning device based on a drone network according to an embodiment of the present application is shown.

[0132] Reference Figure 5 As shown, a hierarchical federated learning device based on a drone network according to an embodiment of the present application is applied to a target edge aggregation drone;

[0133] The device includes:

[0134] A sending module is used to send its own personalized model and the edge model to be trained to each selected client within the coverage area;

[0135] A receiving module, configured to receive a training relevance sent by each of the candidate clients, wherein the training relevance is determined by the candidate client according to a model similarity between a local model obtained by the candidate client in a previous round of edge iteration training and the personalized model;

[0136] A first determination module is used to determine, based on a double-delayed deep deterministic policy gradient algorithm, a correlation threshold that maximizes the contribution value of the target edge model obtained by iterative training of the target edge aggregation UAV on the historical edge to the global model;

[0137] A second determination module is used to determine the candidate client whose training correlation degree is greater than or equal to the correlation degree threshold as the target client of this round of edge iterative training;

[0138] An aggregation module is used to receive and aggregate local models obtained by training the edge model to be trained by each target client in each edge iteration to obtain a target edge model for this round of edge iteration training;

[0139] A judgment module is used to determine whether the target edge aggregation UAV can participate in the next round of edge iteration training according to the energy consumed by the target edge aggregation UAV from the start of this round of global iteration training to the end of this round of edge iteration training, the predicted consumed energy of the next round of edge iteration training, and its initial energy at the start of this round of global iteration training;

[0140] The processing module is used to upload the target edge model obtained by the current round of edge iterative training to the global aggregation drone for global aggregation when the target edge aggregation drone cannot participate in the next round of edge iterative training; otherwise, continue the next round of edge iterative training.

[0141] In one embodiment of the present application, the candidate client determines the model similarity R between the local model obtained by the previous round of edge iteration training and the personalized model according to the following formula: k,m,n :

[0142]

[0143] Among them, x j represents the local sample of the jth input, The parameter matrix is The pre-softmax output of the personalized model, the input local sample is x j ; The parameter matrix is The pre-softmax output of the local model of the client, the input local sample is x j , H represents the number of local training times of the client; is a hyperparameter.

[0144] In one embodiment of the present application, the candidate client determines the training association degree between itself and the target edge aggregation drone according to the following formula:

[0145]

[0146] Among them, ρ1, ρ2, ρ3 are weight coefficients and ρ1+ρ2+ρ3=1, R k,m,n d k,m,n 、f k,m,n They respectively represent the model similarity between the target edge aggregation drone m and the candidate client n in the kth round of edge iteration training, the distance from the candidate client n to the target edge aggregation drone m, and the CPU clock frequency of the client n.

[0147] In one embodiment of the present application, determining whether the target edge aggregation drone can participate in the next round of edge iteration training is performed based on the energy consumed by the target edge aggregation drone from the start of this round of global iteration training to the end of this round of edge iteration training, the predicted consumed energy of the next round of edge iteration training, and its initial energy at the start of this round of global iteration training, including:

[0148] When the initial energy of the target edge aggregation UAV at the beginning of this round of global iterative training is greater than or equal to the energy consumed from the beginning of this round of global iterative training to the end of this round of edge iterative training, and is less than or equal to the sum of the energy consumed from the beginning of this round of global iterative training to the end of this round of edge iterative training and the predicted consumed energy of the next round of edge iterative training, it is determined that the target edge aggregation UAV cannot participate in the next round of edge iterative training, otherwise it can participate in the next round of edge iterative training.

[0149] Figure 6 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown.

[0150] It should be noted that Figure 6 The computer system of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0151] like Figure 6As shown, the computer system includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage part 508 to the random access memory (RAM) 503, such as executing the method described in the above embodiment. In RAM 503, various programs and data required for system operation are also stored. CPU 501, ROM 502 and RAM 503 are connected to each other through bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0152] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed so that a computer program read therefrom is installed into the storage section 508 as needed.

[0153] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 509, and / or installed from a removable medium 511. When the computer program is executed by a central processing unit (CPU) 501, various functions defined in the system of the present application are executed.

[0154] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0155] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0156] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0157] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

[0158] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0159] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0160] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0161] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A hierarchical federated learning method based on drone networks, characterized in that: Application to target edge aggregation drones; The method comprises: Send its own personalized model and the edge model to be trained to each selected client within the coverage area; Receiving a training relevance sent by each of the candidate clients, where the training relevance is determined by the candidate client according to a model similarity between a local model obtained by the candidate client in a previous round of edge iteration training and the personalized model; Based on a double-delayed deep deterministic policy gradient algorithm, determining a correlation threshold that maximizes the contribution of the target edge model obtained by iterative training of the target edge aggregation UAV on the historical edge to the global model; Determine the candidate client whose training correlation degree is greater than or equal to the correlation degree threshold as the target client of this round of edge iterative training; Receiving and aggregating local models obtained by training the edge model to be trained by each target client in each edge iteration to obtain a target edge model for this round of edge iteration training; Determine whether the target edge aggregation UAV can participate in the next round of edge iteration training according to the energy consumed by the target edge aggregation UAV from the start of this round of global iteration training to the end of this round of edge iteration training, the predicted consumed energy of the next round of edge iteration training, and its initial energy at the start of this round of global iteration training; If the target edge aggregation drone cannot participate in the next round of edge iterative training, the target edge model obtained in this round of edge iterative training is uploaded to the global aggregation drone for global aggregation. Otherwise, the next round of edge iterative training continues.

2. The method according to claim 1, characterized in that The candidate client determines the model similarity R between the local model obtained by the previous round of edge iteration training and the personalized model according to the following formula: k,m,n : Among them, x j represents the local sample of the jth input, The parameter matrix is The pre-softmax output of the personalized model, the input local sample is x j ; The parameter matrix is The pre-softmax output of the local model of the client, the input local sample is x j , H represents the number of local training times of the client; is a hyperparameter.

3. The method according to claim 2, characterized in that The candidate client determines the training association degree between itself and the target edge aggregation drone according to the following formula: Among them, ρ1, ρ2, ρ3 are weight coefficients and ρ1+ρ2+ρ3=1, R k,m,n d k,m,n 、f k,m,n They respectively represent the model similarity between the target edge aggregation drone m and the candidate client n in the kth round of edge iteration training, the distance from the candidate client n to the target edge aggregation drone m, and the CPU clock frequency of the candidate client n.

4. The method according to claim 1, characterized in that: Determining whether the target edge aggregation UAV can participate in the next round of edge iteration training according to the energy consumed by the target edge aggregation UAV from the start of this round of global iteration training to the end of this round of edge iteration training, the predicted consumed energy of the next round of edge iteration training, and its initial energy at the start of this round of global iteration training, includes: When the initial energy of the target edge aggregation UAV at the beginning of this round of global iterative training is greater than or equal to the energy consumed from the beginning of this round of global iterative training to the end of this round of edge iterative training, and is less than or equal to the sum of the energy consumed from the beginning of this round of global iterative training to the end of this round of edge iterative training and the predicted consumed energy of the next round of edge iterative training, it is determined that the target edge aggregation UAV cannot participate in the next round of edge iterative training, otherwise it can participate in the next round of edge iterative training.

5. A hierarchical federated learning device based on drone network, characterized in that: Application to target edge aggregation drones; The device comprises: A sending module is used to send its own personalized model and the edge model to be trained to each selected client within the coverage area; A receiving module, configured to receive a training relevance sent by each of the candidate clients, wherein the training relevance is determined by the candidate client according to a model similarity between a local model obtained by the candidate client in a previous round of edge iteration training and the personalized model; A first determination module is used to determine, based on a double-delayed deep deterministic policy gradient algorithm, a correlation threshold that maximizes the contribution value of the target edge model obtained by iterative training of the target edge aggregation UAV on the historical edge to the global model; A second determination module is used to determine the candidate client whose training correlation degree is greater than or equal to the correlation degree threshold as the target client of this round of edge iterative training; An aggregation module is used to receive and aggregate local models obtained by training the edge model to be trained by each target client in each edge iteration to obtain a target edge model for this round of edge iteration training; A judgment module is used to determine whether the target edge aggregation UAV can participate in the next round of edge iteration training according to the energy consumed by the target edge aggregation UAV from the start of this round of global iteration training to the end of this round of edge iteration training, the predicted consumed energy of the next round of edge iteration training, and its initial energy at the start of this round of global iteration training; The processing module is used to upload the target edge model obtained by the current round of edge iterative training to the global aggregation drone for global aggregation when the target edge aggregation drone cannot participate in the next round of edge iterative training; otherwise, continue the next round of edge iterative training.

6. The device according to claim 5, characterized in that The candidate client determines the model similarity R between the local model obtained by the previous round of edge iteration training and the personalized model according to the following formula: k,m,n : Among them, x j represents the local sample of the jth input, The parameter matrix is The pre-softmax output of the personalized model, the input local sample is x j ; The parameter matrix is The pre-softmax output of the local model of the client, the input local sample is x j , H represents the number of local training times of the client; is a hyperparameter.

7. The device according to claim 6, characterized in that The candidate client determines the training association degree between itself and the target edge aggregation drone according to the following formula: Among them, ρ1, ρ2, ρ3 are weight coefficients and ρ1+ρ2+ρ3=1, R k,m,n ,d k,m,n 、f k,m,n They respectively represent the model similarity between the target edge aggregation drone m and the candidate client n in the kth round of edge iteration training, the distance from the candidate client n to the target edge aggregation drone m, and the CPU clock frequency of the candidate client n.

8. The device according to claim 5, characterized in that Determining whether the target edge aggregation UAV can participate in the next round of edge iteration training according to the energy consumed by the target edge aggregation UAV from the start of this round of global iteration training to the end of this round of edge iteration training, the predicted consumed energy of the next round of edge iteration training, and its initial energy at the start of this round of global iteration training, includes: When the initial energy of the target edge aggregation UAV at the beginning of this round of global iterative training is greater than or equal to the energy consumed from the beginning of this round of global iterative training to the end of this round of edge iterative training, and is less than or equal to the sum of the energy consumed from the beginning of this round of global iterative training to the end of this round of edge iterative training and the predicted consumed energy of the next round of edge iterative training, it is determined that the target edge aggregation UAV cannot participate in the next round of edge iterative training, otherwise it can participate in the next round of edge iterative training.

9. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the hierarchical federated learning method based on a drone network as described in any one of claims 1 to 4 is implemented.

10. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the hierarchical federated learning method based on a drone network as described in any one of claims 1 to 4.