Segmentation federal learning method and device, computing unmanned aerial vehicle and storage medium

Abstract

The invention discloses a segmentation federal learning method and device, a computing unmanned aerial vehicle and a storage medium, and relates to the technical field of machine learning. The method comprises the steps of obtaining a cooperative training model sent by an unmanned aerial vehicle coordinator, and determining a segmentation layer, an associated client and a client resource allocation amount of the cooperative training model based on a target strategy optimization algorithm; dividing the cooperative training model into a client sub-model and an unmanned aerial vehicle sub-model according to the segmentation layer, and taking the client sub-model and the unmanned aerial vehicle sub-model as a sub-model pair; the computing resources are allocated to the associated clients through the client resource allocation amount, the client sub-models are issued to the associated clients, and the sub-models are cooperatively trained through the associated clients and the computing unmanned aerial vehicle to obtain an initial cooperative training model; and sending the initial cooperative training model to an unmanned aerial vehicle coordinator, so that the unmanned aerial vehicle coordinator globally aggregates the initial cooperative training model to obtain a target cooperative training model, thereby improving the training efficiency of the target cooperative training model.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, and in particular to a segmented federated learning method, apparatus, computational drone, and storage medium. Background Technology

[0002] Low-Altitude Intelligent Internet of Things (AIoT) combines drone swarms with IoT devices to form an innovative network architecture. Federated Learning (FL) is an emerging distributed learning framework that allows devices to train models locally and upload the model parameters to a central server for aggregation, thus avoiding the challenges of data transmission and privacy protection faced by traditional centralized learning.

[0003] In applications of federated learning, because federated learning only transmits model parameters rather than raw data, it significantly reduces communication overhead and the risk of privacy leaks. However, several challenges remain in current federated learning scenarios: First, edge nodes have limited computing power and energy supply; second, the heterogeneity of network conditions and device computing performance, as well as significant differences in communication latency and bandwidth resources among different devices, all affect training efficiency and model convergence. Therefore, with a large number of users accessing the low-altitude smart IoT, the application of federated learning in this scenario places higher demands on resource scheduling. Thus, how to implement federated learning more effectively has become an urgent problem to be solved. Summary of the Invention

[0004] This invention provides a segmented federated learning method, apparatus, computational drone, and storage medium to address the problem in the prior art where significant differences in communication latency and bandwidth resources among different devices affect training efficiency.

[0005] According to one aspect of the present invention, a segmentation federated learning method is provided, wherein the method is applied to computation of unmanned aerial vehicles, the method comprising:

[0006] Obtain the collaborative training model sent by the UAV coordinator, and determine the segmentation layer, associated clients, and client resource allocation of the collaborative training model based on the objective policy optimization algorithm;

[0007] The collaborative training model is divided into a client sub-model and a drone sub-model according to the segmentation layer, and the client sub-model and the drone sub-model are regarded as a sub-model pair;

[0008] The computing resources are allocated to the associated clients using the client resource allocation amount, the client sub-models are distributed to the associated clients, and the associated clients and computing drones perform collaborative training on the sub-model pairs to obtain an initial collaborative training model;

[0009] The initial collaborative training model is sent to the UAV coordinator, so that the UAV coordinator can globally aggregate the initial collaborative training model to obtain the target collaborative training model.

[0010] According to another aspect of the present invention, a segmented federated learning apparatus is provided, wherein the apparatus comprises:

[0011] The resource scheduling module is used to obtain the collaborative training model sent by the UAV coordinator, and determine the segmentation layer, associated clients, and client resource allocation of the collaborative training model based on the objective policy optimization algorithm.

[0012] The model determination module is used to divide the collaboratively trained model into a client sub-model and a drone sub-model according to the segmentation layer, and to treat the client sub-model and the drone sub-model as a sub-model pair;

[0013] The collaborative training module is used to allocate computing resources to associated clients through the client resource allocation amount, distribute the client sub-models to associated clients, and perform collaborative training on the sub-model pairs through the associated clients and the computing drone to obtain an initial collaborative training model;

[0014] The model sending module is used to send the initial collaborative training model to the UAV coordinator, so that the UAV coordinator can globally aggregate the initial collaborative training model to obtain the target collaborative training model.

[0015] According to another aspect of the present invention, a computing drone is provided, the computing drone comprising:

[0016] At least one processor; and

[0017] A memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform a segmented federated learning method according to any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement a segmented federated learning method according to any embodiment of the present invention.

[0020] The technical solution of this invention obtains the collaborative training model sent by the UAV coordinator, determines the segmentation layer, associated clients, and client resource allocation based on the target policy optimization algorithm, divides the collaborative training model into client sub-models and UAV sub-models according to the segmentation layer, and sets the client sub-models and UAV sub-models as sub-model pairs. Computational resources are allocated to associated clients based on the client resource allocation, and the client sub-models are distributed to the associated clients. Collaborative training is performed between the associated clients and the UAV sub-model pairs to obtain an initial collaborative training model. This initial collaborative training model is then sent to the UAV coordinator, which globally aggregates the initial collaborative training model to obtain the target collaborative training model, achieving efficient allocation of training tasks and resources. Simultaneously, the associated clients only transmit model parameters, not raw data, which significantly reduces communication overhead and the risk of privacy leakage, improving the efficiency of training the target collaborative training model.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of a segmented federated learning method provided in Embodiment 1 of the present invention;

[0024] Figure 2 This is a flowchart of a segmented federated learning method provided in Embodiment 2 of the present invention;

[0025] Figure 3 This is a flowchart of a segmentation federated learning method provided in Embodiment 3 of the present invention;

[0026] Figure 4 This is a diagram of a low-altitude intelligent network architecture provided in Embodiment 4 of the present invention;

[0027] Figure 5 This is a resource allocation diagram provided according to Embodiment 4 of the present invention;

[0028] Figure 6This is a schematic diagram of a drone swarm-assisted federated learning process provided in Embodiment 4 of the present invention;

[0029] Figure 7 This is a schematic diagram of a segmented federated learning structure provided in Embodiment 4 of the present invention;

[0030] Figure 8 This is a reinforcement learning structure diagram provided in Embodiment 4 of the present invention;

[0031] Figure 9 This is a schematic diagram of the structure of a segmented federated learning device according to Embodiment 5 of the present invention;

[0032] Figure 10 This is a schematic diagram of the structure of a computational drone that implements the segmentation federated learning method of this invention. Detailed Implementation

[0033] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0035] Example 1

[0036] Figure 1 This is a flowchart of a segmentation federated learning method according to Embodiment 1 of the present invention. This embodiment is applicable to model training in low-altitude intelligent Internet of Things (IoT) scenarios. The method can be executed by a segmentation federated learning device, which can be implemented in hardware and / or software and can be configured in a computational drone. Figure 1 As shown, the method includes:

[0037] S110. Obtain the collaborative training model sent by the UAV coordinator, and determine the segmentation layer, associated client, and client resource allocation of the collaborative training model based on the objective policy optimization algorithm.

[0038] In this context, the UAV coordinator refers to a pre-configured UAV management organization, such as a UAV Management Agency (UMA), used to command UAVs in the airspace and perform resource allocation and task scheduling. The UAV coordinator can act as a federated learner. The collaborative training model refers to the model to be trained, a model trained collaboratively by multiple parties. It is a global model jointly optimized without relying on centralized data and can be trained based on federated learning algorithms. In one embodiment, the collaborative training model may include, but is not limited to, image recognition models, UAV swarm low-altitude inspection models, and deep learning models. The target policy optimization algorithm refers to a pre-set method for optimizing the segmentation layer, associated clients, and client resource allocation, such as reinforcement learning algorithms like Proximal Policy Optimization with Clip (PPO-clip). The segmentation layer refers to the model's segmentation points, used to split the collaborative training model into client sub-models and UAV sub-models, which perform training tasks through associated clients and computed UAVs, respectively. The segmentation layer dimension selected by each associated client is within the layer of the collaborative training model. Associated clients refer to clients that perform collaborative training with the computational drone. Generally, each computational drone can have one or more associated clients. Client resource allocation refers to the amount of resources, such as computing resources, allocated by the computational drone to the associated clients.

[0039] In this embodiment, a collaborative training model sent by a drone coordinator can be received. A target policy optimization algorithm is used to determine the segmentation layer, associated clients, and client resource allocation for the collaborative training model. In actual operation, associated clients can be determined based on the model's training data, the segmentation layer can be determined based on the heterogeneity of each client, and the resource requirements of each associated client can be determined as the client resource allocation. Alternatively, the PPO-clip algorithm can be used as the target policy optimization algorithm. The current connection status between each client and the drone, the current model segmentation layer, the drone's remaining computing power, the drone's current energy consumption, and the current communication distance between the client and the drone are collected in real time at preset time steps as the current state. This current state is input into the target policy optimization algorithm. The algorithm uses discrete strategies to obtain discrete actions such as segmentation layer adjustment actions and client-drone association matching actions within a pre-set action space, and uses continuous strategies to obtain continuous actions such as resource allocation actions within the pre-set action space. The discrete and continuous actions are executed, adjusting the segmentation layer, the current connection status between the client and the drone, and the resource allocation according to the target actions. The current state after executing the target actions is collected as the state for the next time step. The immediate reward corresponding to the current state is determined, and the current state, target action, immediate reward, and next-time state are stored as a transition sample pair in the experience buffer. The value of the current state is evaluated by the critic network in the target policy optimization algorithm, providing a key basis for updating the policy network (including discrete and continuous policies). The policy network and critic network are updated until the immediate reward converges, the immediate delay is minimized, and the energy consumption is minimized. The current splitting layer corresponding to the immediate reward convergence is taken as the splitting layer, the client and the UAV currently connected client is taken as the associated client, and the current resource allocation amount is taken as the client resource allocation amount.

[0040] S120. Based on the segmentation layer, the collaborative training model is divided into client sub-model and drone sub-model, and the client sub-model and drone sub-model are treated as a sub-model pair.

[0041] In one embodiment, the client sub-model is the first layer to the segmentation layer in the collaborative training model; the drone sub-model is the layer below the segmentation layer to the last layer in the collaborative training model.

[0042] In this embodiment, the co-trained model can be split according to the segmentation layer to obtain a client sub-model and a drone sub-model, and the client sub-model and the drone sub-model can be treated as a pair of sub-models. For example, assume the segmentation layer is {l m ,l m ∈L}. The co-trained model w can be divided into two sub-models according to the split layer. Client Submodel From level 1 to level 1m Layer, drone sub-model from l m +1 layers up to layer L. Where L is the last layer.

[0043] S130. Allocate computing resources to associated clients through client resource allocation, distribute client sub-models to associated clients, and obtain an initial collaborative training model by co-training the sub-models through associated clients and computing drones.

[0044] The initial collaborative training model can be understood as the initial collaborative training model trained by a single computing drone and its corresponding associated client. Since there are multiple computing drones, there can also be multiple initial collaborative training models. The computing drone can be regarded as an edge computing server, with each computing drone serving clients in a certain area. It is mainly responsible for collaborative training with ground clients and communicating with them to collect and aggregate the training parameters of the client sub-models.

[0045] In this embodiment, computing resources can be allocated to associated clients according to their resource allocation, and client sub-models can be distributed to these clients. Federated learning training is then performed on the client sub-models and drone sub-models, respectively, by the associated clients and the computing drones. The trained models are then aggregated by the computing drones to obtain the initial collaborative training model. In actual operation, at the beginning of each training round, all associated clients receive the latest client sub-models from their corresponding computing drones, while the drones themselves retain their own drone sub-models. The associated clients and computing drones collaborate, training the sub-model pairs in parallel. Specifically, each associated client first performs forward propagation on its client sub-model to obtain intermediate data and sends it to its corresponding computing drone. The computing drone performs the remaining forward propagation on its drone sub-model and performs backward gradient calculation to update the drone sub-model, obtaining the gradient at the cut-out layer, and sends it back to the corresponding associated client. The associated client performs the remaining backward gradient calculation from the cut-out layer and updates its client-side sub-model. Generally, the initial collaborative training model can be trained for a preset number of rounds. After completing the preset number of rounds of collaborative training, the associated clients send their updated client sub-models to their respective computing drones. Subsequently, the UAV performs a model aggregation process to obtain an initial collaborative training model. In one embodiment, when there are multiple associated clients, the client sub-models can be sent to all associated clients, and the client sub-models updated by all clients can be aggregated to obtain the initial collaborative training model.

[0046] S140. Send the initial collaborative training model to the UAV coordinator so that the UAV coordinator can globally aggregate the initial collaborative training model to obtain the target collaborative training model.

[0047] In this embodiment, after the computational model determines the initial collaborative training model, it can send the initial collaborative training model to the UAV coordinator. Generally, the computational UAV can transmit the initial collaborative training model to the UAV coordinator for global aggregation after multiple aggregations; alternatively, the computational UAV can transmit the initial collaborative training model to the UAV coordinator for global aggregation after each aggregation. After receiving the initial collaborative training model, the UAV coordinator can generate the target collaborative training model by weighted fusion of parameters, gradients, and other information from multiple initial collaborative training models.

[0048] In this embodiment of the invention, by acquiring the collaborative training model sent by the UAV coordinator, the segmentation layer, associated clients, and client resource allocation of the collaborative training model are determined based on the target policy optimization algorithm. According to the segmentation layer, the collaborative training model is divided into client sub-models and UAV sub-models, which are then used as sub-model pairs. Computational resources are allocated to associated clients based on the client resource allocation, and the client sub-models are distributed to the associated clients. Collaborative training is then performed between the associated clients and the UAV sub-model pairs to obtain an initial collaborative training model. This initial collaborative training model is then sent to the UAV coordinator, which globally aggregates the initial collaborative training model to obtain the target collaborative training model, achieving efficient allocation of training tasks and resources. Simultaneously, the associated clients only transmit model parameters, not raw data, which significantly reduces communication overhead and the risk of privacy leakage, improving the training efficiency of the target collaborative training model.

[0049] In one embodiment, after the UAV coordinator globally aggregates the initial cooperative training model to obtain the target cooperative training model, the method further includes:

[0050] The target collaborative training model is then deployed as a new collaborative training model to the computational drone.

[0051] In this embodiment, after the UAV coordinator globally aggregates the initial collaborative training model to obtain the target collaborative training model, the target collaborative training model can be distributed to the computational UAV as a new collaborative training model. The computational UAV then determines the segmentation layer, associated clients, and client resource allocation of the new collaborative training model based on the target policy optimization algorithm. According to the segmentation layer, the collaborative training model is divided into client sub-models and UAV sub-models, and the client sub-models and UAV sub-models are treated as sub-model pairs. Computational resources are allocated to associated clients through the client resource allocation, and the client sub-models are distributed to associated clients. The associated clients and the computational UAV pair of sub-models are then used for collaborative training to obtain a new initial collaborative training model, until multiple rounds of federated learning training are completed.

[0052] Example 2

[0053] Figure 2 This is a flowchart of a segmentation federated learning method according to Embodiment 2 of the present invention. This embodiment is based on the above-described implementation methods, further optimizing and extending the process of determining the segmentation layer, associated clients, and client resource allocation of the collaborative training model based on the objective policy optimization algorithm, and can be combined with various optional technical solutions in the above-described implementation methods. Figure 2 As shown, the method includes:

[0054] S201. Obtain the collaborative training model sent by the UAV coordinator.

[0055] S202. Collect the current connection status between the client and the drone, the current model cutting layer, the remaining computing power of the drone, the current energy consumption of the drone, and the current communication distance between the client and the drone in real time according to the preset time step as the current status.

[0056] The preset time step is used to decompose a continuous or dynamic process into computable, iterable discrete steps. The current connection status between the client and the drone indicates whether each client is connected to the computational drone. Each client can only connect to one computational drone. The current model slicing layer refers to the slicing layer of the model at the current moment, which can be the initially set slicing layer. The drone's remaining computing power refers to the current remaining computing power. The drone's current energy consumption refers to the current energy consumption value of each computational drone. The client-drone current communication distance records the ground-to-air distance between each client and the computational drone.

[0057] In this embodiment, the environment can be initialized, and the current connection status between the client and the drone, the current model slicing layer, the remaining computing power of the drone, the current energy consumption of the drone, and the current communication distance between the client and the drone can be collected in real time from the environment according to a preset time step. The current connection status between the client and the drone, the current model slicing layer, the remaining computing power of the drone, the current energy consumption of the drone, and the current communication distance between the client and the drone are used as the current status.

[0058] S203. Input the current state into the target policy optimization algorithm, and obtain the target action in the pre-set action space through the actor network in the target policy optimization algorithm.

[0059] The objective policy optimization algorithm includes an actor network and a critic network, with the actor network including discrete and continuous policies. In one embodiment, the current state satisfies preset constraints. Exemplarily, these preset constraints may include at least: ensuring that the total amount of computing resources allocated to the drone does not exceed the computing power of the drone; ensuring that each client can only connect to one drone during each round of cloud aggregation by the drone coordinator; ensuring that the model split point selected by each client is within a layer of the deep learning model; and ensuring that the energy consumption of each drone must be below a fixed threshold, and that the communication distance constraint is less than a preset value.

[0060] The Actor Network is the core component responsible for generating actions. Its core function is to output specific action decisions based on the agent's current environmental state, guiding the agent's interaction with the environment to maximize long-term cumulative rewards. The Critics Network is the core component responsible for evaluating value. Its core role is to provide feedback signals to the Actor Network for policy optimization by estimating the value of states or state-action pairs. Discrete policies and continuous policies correspond to the decision-making needs of discrete and continuous action spaces, respectively. The core function of both is to generate actions based on the current state.

[0061] In this embodiment, the current state can be input into the target policy optimization algorithm, and actions can be obtained as target actions through discrete and continuous policies in the target policy actor network.

[0062] In one embodiment, obtaining a target action in a pre-set action space through the actor network in the target policy optimization algorithm includes: obtaining discrete actions in the pre-set action space through a discrete policy, and obtaining continuous actions in the pre-set action space through a continuous policy, and using the discrete actions and continuous actions as the target actions.

[0063] Discrete actions include at least the cutting layer adjustment action and the association matching action between the client and the drone; continuous actions include at least the resource allocation action.

[0064] Action space refers to the set or range of all possible actions that can be taken when interacting with the environment. Based on the nature of the actions, action space can be divided into discrete action space and continuous action space. Action a t It can be defined as: For each client, the action This determines how to adjust the partitioning layer of the computation task: -1 means moving the partition point closer to the input layer, offloading more computation to the computation drone; 0 means keeping it unchanged; and +1 means moving the partition point closer to the output layer, offloading less computation to the drone. This also determines the matching actions between the client and the computation drone. Each client is limited to associating with only one drone. Once the associated client is determined, this action specifies the amount of computing resources allocated to the associated client from the drone: Where Δ max It is the largest adjustment step size.

[0065] In this embodiment, discrete actions such as cutting layer adjustment actions and client-drone association matching actions can be obtained in a pre-set action space through a discrete strategy, and continuous actions such as resource allocation actions can be obtained in a pre-set action space through a continuous strategy, with discrete actions and continuous actions as target actions.

[0066] S204. Execute the target action in the current environment, adjust the current connection status and resource allocation of the segmentation layer, client and drone according to the target action, and collect the current state after executing the target action as the state at the next moment.

[0067] In the embodiment, a target action can be executed to adjust the current environment's segmentation layer, the current connection status between the client and the drone, and the resource allocation. After adjusting the target action, the current connection status between the client and the drone, the current model segmentation layer, the drone's remaining computing power, the drone's current energy consumption, and the current communication distance between the client and the drone are collected again as the state at the next moment.

[0068] S205. Determine the immediate reward corresponding to the current state according to the preset reward function, and store the current state, target action, immediate reward and the next state as a transition sample pair in the experience buffer.

[0069] Here, the immediate reward refers to the reward value determined by a preset reward function, and the experience buffer refers to a pre-configured storage space, such as a Buffer. In actual operation, the current state, target action, immediate reward, and next-moment state can be bound together, which can be regarded as a vector and stored in the pre-configured experience buffer as a transformation sample pair for subsequent data retrieval.

[0070] In one embodiment, the preset reward function is constructed based on the resource allocation, the current energy consumption of the drone, the computing power of the drone, the current communication distance between the client and the drone, and the aggregation time of the target collaborative training model. For example, the preset reward function can be r t =-T A -λ(Penalty C1 +Penalty C5 +Penalty C6 );in,

[0071] λ is the penalty factor. m,k This refers to the client that connects to the computing drone, i.e., the associated client; This refers to the amount of resources allocated; This refers to the computing power of the drone; This represents the current energy consumption of the drone. The preset energy consumption threshold; d m,k This refers to the current communication distance between the client and the drone; D G2A T represents the maximum communication distance. A The aggregation time for collaborative training models targeting the objective.

[0072] In this embodiment, the immediate reward can be determined by substituting the current state into a preset reward function, and the current state, target action, immediate reward and next moment state can be bound together and stored as a conversion sample pair in the experience buffer.

[0073] S206. The critic network outputs the current state value estimate based on the current state and the next state value estimate based on the next time step state. The target value is determined according to the next state value estimate and the immediate reward.

[0074] In this embodiment, a critic network can be used to determine the current state value estimate corresponding to the current state, and to determine the next state value estimate corresponding to the next time step state. The sum of the next state value estimate and the immediate reward is then used as the target value. Generally, the next state value estimate has a discount factor. The product of the discount factor and the next state value estimate can be determined, and the sum of the product and the next state value estimate is used as the target value.

[0075] S207. Determine the mean square error between the target value and the current state value estimate as the critic loss function, and update the parameters of the critic network according to the critic loss function.

[0076] In this embodiment, the critic loss function is used to measure the difference between the value estimate and the target value, typically employing mean squared error (MSE). Gradient descent is used to minimize the critic loss function, thereby updating the parameters of the critic network.

[0077] S208. Determine the advantage estimate for each time step based on the generalized advantage estimation algorithm. Update the parameters of the actor network based on the advantage estimate and the near-end policy optimization loss function until the immediate reward converges. Take the current segmentation layer corresponding to the immediate reward convergence as the segmentation layer, take the client and the drone's current connection state as the associated client, and take the current resource allocation amount as the client resource allocation amount.

[0078] Among them, the Generalized Advantage Estimation (GAE) algorithm is a method in reinforcement learning used to accurately estimate the advantage function. The proximal policy optimization loss function is the PPO-clip loss function. In one embodiment, the proximal policy optimization loss function can be... and Representing discrete actions and The objective function and follow The calculation method in a represents the probability ratio. t This indicates the selected discrete action. Indicates continuous actions follow The designed objective function and continuous strategy The objective function is

[0079] Among them, continuous strategy The probability ratio is μ represents the loss function. The weight.

[0080] In this embodiment, the advantage estimate for each time step can be determined according to the generalized advantage estimation algorithm, and the loss value is determined and the parameters of the actor network are updated based on the advantage estimate and the loss function is optimized according to the near-end policy until the immediate reward converges. The current segmentation layer corresponding to the immediate reward convergence is taken as the segmentation layer, the client and the drone are currently connected clients as associated clients, and the current resource allocation amount is taken as the client resource allocation amount.

[0081] S209. Based on the segmentation layer, the collaborative training model is divided into client sub-model and drone sub-model, and the client sub-model and drone sub-model are treated as a sub-model pair.

[0082] S210. Allocate computing resources to associated clients through client resource allocation, distribute client sub-models to associated clients, and obtain an initial collaborative training model by co-training the sub-models through associated clients and computing drones.

[0083] S211. Send the initial collaborative training model to the UAV coordinator so that the UAV coordinator can globally aggregate the initial collaborative training model to obtain the target collaborative training model.

[0084] In this embodiment of the invention, the current connection state between the client and the drone, the current model segmentation layer, the drone's remaining computing power, the drone's current energy consumption, and the current communication distance between the client and the drone are collected in real time according to a preset time step as the current state. This current state is then input into a target policy optimization algorithm. The actor network in the target policy optimization algorithm obtains the target action from a pre-set action space. The target policy optimization algorithm includes an actor network and a critic network. The actor network includes discrete and continuous policies. The target action is executed in the current environment. The segmentation layer, the current connection state between the client and the drone, and the resource allocation are adjusted according to the target action. The current state after executing the target action is collected as the next time step state. The immediate reward corresponding to the current state is determined according to a preset reward function. The current state, target action, immediate reward, and next time step state are stored as a transformation sample pair in an experience buffer. The critic network then uses the current state as the basis for the next time step state. The algorithm outputs the current state value estimate and the next state value estimate based on the next time step. It then determines the target value based on the next state value estimate and the immediate reward, using the mean squared error between the target value and the current state value estimate as the critic loss function. The parameters of the critic network are updated according to the critic loss function. A generalized advantage estimation algorithm is used to determine the advantage estimate at each time step. The parameters of the actor network are updated based on the advantage estimate and the near-end policy optimization loss function until the immediate reward converges. The current segmentation layer corresponding to the immediate reward convergence is used as the segmentation layer. Clients currently connected to the drone are considered associated clients, and the current resource allocation is used as the client resource allocation. This addresses the issue of device heterogeneity in low-altitude intelligent IoT by combining federated learning latency and drone energy consumption assessment to dynamically allocate resources, achieving fairness and efficiency in task allocation. It solves the problem that traditional methods struggle to simultaneously optimize multi-dimensional decisions.

[0085] Example 3

[0086] Figure 3 This is a flowchart of a segmentation federated learning method according to Embodiment 2 of the present invention. This embodiment is based on the above-described implementation methods, further optimizing and extending the process of obtaining an initial collaborative training model by collaboratively training sub-model pairs through associated clients and computational drones, and can be combined with various optional technical solutions in the above-described implementation methods. For example... Figure 3 As shown, the method includes:

[0087] S310. Obtain the collaborative training model sent by the UAV coordinator, and determine the segmentation layer, associated client, and client resource allocation of the collaborative training model based on the objective policy optimization algorithm.

[0088] S320. Based on the segmentation layer, the collaborative training model is divided into client sub-model and drone sub-model, and the client sub-model and drone sub-model are treated as a sub-model pair.

[0089] S330. Allocate computing resources to associated clients based on client resource allocation amount, and distribute client sub-models to associated clients.

[0090] S340: By forward propagating the client sub-model according to the local data through the associated client, intermediate data is obtained, and the intermediate data is sent to the corresponding computational drone.

[0091] In this embodiment, a client sub-model can be trained using local data via a connected client, intermediate data can be obtained by forward propagation of the client sub-model, and the intermediate data can be transmitted to the corresponding computational drone. The intermediate data consists of the model parameters obtained by training the client sub-model.

[0092] S350: The drone updates its sub-model by forward propagation based on intermediate data, and then backward calculates the gradient of the segmentation layer, sending the gradient to the associated client. In this embodiment, after receiving the intermediate data, the drone can continue to forward propagate and update the sub-model using the intermediate data. Simultaneously, the gradient of the segmentation layer is determined by backward calculation and sent to the associated client.

[0093] S360 updates the client sub-model backwards according to the gradient through the associated client, obtains the updated client sub-model, and then performs forward propagation on the updated client sub-model through local data until a preset round of collaborative training is performed, and then sends the updated client sub-model to the computational drone.

[0094] In this embodiment, after receiving the gradient, the associated client can perform the remaining backward gradient calculation from the segmentation layer to update the client sub-model. It then continues to perform forward propagation on the updated client sub-model according to the local data to obtain intermediate data. This intermediate data is transmitted to the corresponding computational drone. The computational drone then performs forward propagation on its drone sub-model according to the intermediate data to update the drone sub-model, and backward calculates the gradient of the segmentation layer. This gradient is sent to the associated client, which completes the backward update of its client sub-model according to the gradient, obtaining the updated client sub-model. This process continues until a preset number of collaborative training rounds are completed, at which point the updated client sub-model is sent to the computational drone.

[0095] S370. The computational drone aggregates the updated client sub-model and the updated drone sub-model to obtain an initial collaborative training model. In this embodiment, the computational drone can aggregate the updated client sub-model and the updated drone sub-model to generate an initial collaborative training model.

[0096] S380. Send the initial collaborative training model to the UAV coordinator so that the UAV coordinator can globally aggregate the initial collaborative training model to obtain the target collaborative training model.

[0097] In this embodiment of the invention, intermediate data is obtained by forward propagating the client sub-model using local data from the associated client. This intermediate data is then sent to the corresponding computational drone. The computational drone updates its sub-model using the intermediate data and backward calculates the gradient of the segmentation layer. This gradient is then sent to the associated client, which completes the backward update of the client sub-model using the gradient, resulting in an updated client sub-model. The updated client sub-model is then forward propagated using local data until a preset number of collaborative training rounds are performed. The updated client sub-model is then sent to the computational drone, which aggregates the updated client sub-model and the updated drone sub-model to obtain an initial collaborative training model. This achieves federated learning between the associated client and the computational drone. The local data of the associated client is always stored on the local node, and only the locally trained model parameters need to be uploaded, thus fundamentally avoiding the risk of leakage, theft, or misuse of local data during transmission and storage.

[0098] Example 4

[0099] Figure 4 This is a low-altitude intelligent network architecture diagram provided in Embodiment 4 of the present invention. This embodiment uses a global model as the target collaborative training model and a local model as the initial collaborative training model as an example to further illustrate a segmentation federated learning method. Figure 4 As shown, in this architecture, multiple stationary hovering unmanned aerial vehicles (UAVs) are deployed to provide services to ground clients, assisting them in completing federated learning computational tasks. The UAVs are categorized into a group of computational UAVs. A drone manager (UAV-C) to effectively serve client devices. Computational drones have good computing performance and can be regarded as edge computing servers. Each computational drone serves clients in a certain area. They are mainly responsible for co-training machine learning models with ground clients (assuming the model has L levels) and communicating with them to collect and aggregate client model training parameters. The drone manager acts as a federated learning server to aggregate global model parameters in the edge computing server and generate a brand new global model.

[0100] In one embodiment, Figure 5 This is a resource allocation diagram provided according to Embodiment 4 of the present invention. Figure 5As shown, drone managers can distribute the collaborative training model to each computational drone through associated decisions, and determine the segmentation layer {l} through each computational drone. m ,l m ∈L} and the client resource allocation for each associated client

[0101] In one embodiment, Figure 6 This is a schematic diagram of a drone swarm-assisted federated learning process according to Embodiment 4 of the present invention. Figure 6 As shown, in each round of the federated learning process epochr (where all training samples undergo one forward propagation and one backward propagation in the model, i.e., all training samples are iterated once), the following is used: This indicates whether client m collaborates with computational drone k for model training, i.e., whether client m belongs to the cluster of computational drone k. Here, a cluster refers to a group of computational drones and their associated drones; in the following text, associated clients within a cluster are referred to as clients.

[0102] S1. Calculate the heterogeneity of each client within the cluster when the drone is based on {l}. m ,l m The L layer divides the global model w into two sub-models. Client Submodel From level 1 to level 1 m Layer, UAV side model from l m +1 layers up to L layers. At the start of the r-th round of training, all clients receive the latest client sub-model from drones in the same cluster. The drone itself retains another part of the drone sub-model.

[0103] S2, clients and computing drones within the same cluster collaborate to train sub-model pairs in parallel. Specifically, each client m first trains the sub-model... Perform forward propagation to obtain intermediate data z m .

[0104] S3, the client and the intermediate data z m Send to the computing drones in the cluster.

[0105] S4. Calculate the UAV side-to-side UAV sub-model. Perform the remaining forward propagation.

[0106] S5. Calculate the backward gradient of the UAV to update the UAV sub-model. Obtain the gradient at the cutting layer

[0107] S6, Calculate the gradient of the drone Send it back to the corresponding client within the cluster.

[0108] S7. The client performs the remaining backward gradient calculation from the cut layer and updates the client sub-model.

[0109] S8, Each cluster is executing E e After one round of collaborative training, all clients in the cluster send their updated local models to their respective drones. Subsequently, the drones perform a local model aggregation process within the cluster, executing E... p After the initial local aggregation, the aggregated model is transferred to the UAV-C for global model aggregation. Then, the UAV-C creates the latest global model and sends it back to the computational UAV, repeating S1-S5 until R rounds of federated learning training are completed.

[0110] In one embodiment, Figure 7 This is a schematic diagram of a segmented federated learning structure provided according to Embodiment 4 of the present invention. Figure 7 As shown, segmented federated learning is essentially drone swarm-assisted federated learning. Segmented federated learning is the same as S1-S8 described above.

[0111] In one embodiment, since each federated learning training round is independent, we only analyze one training round. For simplicity, the index r of the training round will be omitted below. The training latency and energy consumption per drone in one round of the federated learning framework include:

[0112] 1) Client sub-model download and upload latency: When selecting the lth... m When layer m is used as a model partitioning layer for client m, the size of the client sub-model parameters can be expressed as: (in bits), the download and upload latency of the client sub-model parameters can be calculated using the following formulas: in, These represent the downlink and uplink transmission rates from the client to the computing drone, respectively. The total system bandwidth is represented by B, and is evenly distributed among each client. and These represent the uplink and downlink transmission power of the client and the computing drone within the same cluster, respectively. Maximum transmit power for each client, This indicates the average power allocated to each client within the cluster by the drone. To average ground-to-air (G2A) path loss, Represents Gaussian white noise power.

[0113] 2) Client-side forward propagation delay: To train the client sub-model, define... This indicates the first l steps of processing each data point during the forward propagation on the client side. m The computational workload required to split the layer (in FLOPs). The forward propagation computation delay for client m is expressed as: Where |D m | indicates the size of the local dataset owned by client m. Let m be the frequency (cycles / s) of the Central Processing Unit (CPU) of the client, and n be the frequency of the CPU. m The number of CPU flops per cycle.

[0114] 3) Intermediate data transmission delay: Let Represents the client sub-model at the lth m The data size (in bits) output by the layer, and the intermediate data transmission delay of client m can be expressed as:

[0115] 4) UAV-side model computation delay: Let The computational workload required for the latter l layers in the forward propagation and backward gradient calculation of each data point on the UAV end can be represented by the following formula: in, To calculate the CPU frequency (cycle / s) allocated to client m by drone k, n k Let flop be the number of CPU flops per cycle for drone k.

[0116] 5) Intermediate data gradient transmission delay: Let Let represent the size (in bits) of the gradient data output by the UAV sub-model at layer l. The intermediate data gradient transmission delay of the UAV can be expressed as:

[0117] 6) Client-side sub-model backward gradient calculation latency: Definition This represents the first l of each data point processed in the client-side backward gradient. m The computational workload required for splitting the layer (in FLOPs). The backward gradient computation latency for client m can be expressed as:

[0118] 7) Communication latency of uploading the UAV-side aggregated model to UAV-C: Let ε k Indicates that drone k is in its E p The communication latency of UAV k uploading to UAV-C after the second edge aggregation is expressed as: in, The total bandwidth of UAV-C is represented by b, and it is evenly distributed among each UAV. It is the transmit power of the drone k that sends the local aggregation model to the UAV-C. This represents the path loss between UAV-C and drone k. The noise power between the UAV-C and the drone k.

[0119] Since the proposed federated learning framework is a synchronous framework, each training phase needs to be completed synchronously before proceeding to the next training phase. Based on 1)–7), the cloud aggregation time for UAV-C under the federated learning architecture can be expressed as:

[0120] (1) The time T for each round of cloud gathering A The last one to complete E p The drones at the secondary edge aggregation point were identified. A It can be represented as:

[0121] (2) Add T k This indicates that the calculation of drone k completes E p The time for the second edge aggregation, which is determined by the last completed E e The client for in-round cluster model co-training is determined. k It can be represented as: in, This indicates the transmission time for the UAV k to send model parameters to the UAV-C.

[0122] (3) Client m completes E e The training time for each round of user-UAV collaboration is: in, This indicates the training time for the client-side and the drone sub-models within the cluster. This indicates the communication time between the client and the drones within its cluster.

[0123] Based on 1)-7), the total energy consumption of UAV k in one round of cloud aggregation training in UAV-C can be expressed as:

[0124] in, This represents the hovering energy consumption of drone k. Let η be the minimum hovering power of the drone k, and η represent the power efficiency.

[0125] The optimization problem P that minimizes training latency in one round includes: The goal of this patent is to formulate an association matching strategy between drones and ground clients within a federated learning framework. m,k}, using drones to allocate computing resources to ground clients in each cluster. Used for collaborative model training, and to select appropriate model split points based on the uneven computing power caused by the heterogeneity of each client. m Finally, a load balance is achieved to maximize UAV-C cloud aggregation, minimize time, and minimize UAV energy consumption. The optimization problem under consideration is formally expressed as P:

[0126]

[0127] Among them, constraint C1 ensures that the total amount of computing resources allocated to the drone does not exceed the drone's computing power. Constraint C2 is a decision variable indicating whether client m is associated with computational drone k. Constraint C3 states that each client can only connect to one computational drone in each round of cloud aggregation in UAV-C. Constraint C4 ensures that the model split point selected by each client is within a layer of the global model. Constraint C5 ensures that the energy consumption of each computational drone must be below a fixed threshold. Constraint C6 represents the G2A communication distance constraint.

[0128] Clearly, the objective function and constraints of the formulaic problem P are non-convex and constitute an NP-hard problem. First, the problem is inevitably entangled with a mixed action space consisting of discrete and continuous actions, which are highly correlated. Second, as a collaborative model training problem, the system must consider all clients and drones to explore an optimal policy that minimizes the objective function and the application of the drones by coordinating all resources involved.

[0129] Based on this, this patent uses action space parameterization to solve the model training problem for drone collaborative clients in federated learning systems. It models problem P as a parameterized action Markov decision process. Furthermore, it proposes a hybrid action space reinforcement learning algorithm based on proximal policy optimization to further address problem P.

[0130] In one embodiment, the Markov decision process includes:

[0131] 1) State space: state s t It can be represented as: s t ={s1,s2,s3,s4,s5}, according to the method requirements, it is necessary to collect the current connection status between the client and the drone. Current model segmentation layer position on the client The remaining computing power of the drone Current power consumption status of drones Communication distance between the client and the drone

[0132] 2) Action space: Action a t It can be defined as: For each client m, the action This determines how to adjust the partitioning layer of the computational task: -1 means moving the partition point closer to the input layer, offloading more computation to the drone; 0 means keeping it unchanged; and +1 means moving the partition point closer to the output layer, offloading less computation to the drone. The matching action between the client m and the drone... Each client is limited to associating with only one computing drone. Within each drone cluster, this action specifies the amount of computing resources allocated to each client within the cluster: Where Δ max It is the largest adjustment step size.

[0133] 3) Reward Function: The reward function quantifies the desirability of an agent's actions in a specific state, providing immediate feedback on the agent's behavior at each time step. The reward function is defined as:

[0134] r t =-T A -λ(Penalty C1 +Penalty C5 +Penalty C6 )

[0135] λ is the penalty factor, s m,k This refers to the client that connects to the computing drone, i.e., the associated client; This refers to the amount of resources allocated; This refers to the computing power of the drone; This represents the current energy consumption of the drone. The preset energy consumption threshold; d m,k This refers to the current communication distance between the client and the drone; D G2A T represents the maximum communication distance. A The aggregation time for collaborative training models targeting the objective.

[0136] The hybrid action-space reinforcement learning algorithm proposed in this patent uses the PPO-clip algorithm as the objective policy optimization algorithm. The PPO-clip algorithm is currently the default algorithm in the OpenAI Baselines reinforcement learning benchmark algorithm set. It improves the data sampling efficiency and robustness of the algorithm and reduces the complexity of hyperparameter selection by selecting a conservative lower bound for the objective function through a simple clipping mechanism. Specifically, in the implementation of the PPO-clip algorithm, we maintain two policy networks, namely the current policy π... θ and previous strategies used for sample collection that have not yet been updated. Then, the network parameters are updated by maximizing the objective function, thereby optimizing the policy π. θ (a t |s t The objective function of the PPO-clip algorithm can be expressed as:

[0137]

[0138] in, This represents the probability ratio, used to measure the difference between two strategies. T represents the time steps of the execution strategy. The function clip(r) t (θ), 1-∈, 1+∈) are used to ensure that the difference between the current policy and the old policy is kept within the range [1-∈, 1+∈]. Probability ratios exceeding the range [1-∈, 1+∈] will cause the advantage function to fail. It was cut off.

[0139] In the deep reinforcement learning algorithm proposed in this patent, the PPO-clip algorithm requires optimization of discrete strategies separately. and continuous strategy These two strategies are optimized independently and do not interfere with each other; discrete strategy. The objective function is in a represents the probability ratio. t This represents the selected discrete action. Similarly, for continuous policies... The objective function is Among them, continuous strategy The probability ratio is

[0140] Therefore, the overall loss function L of the actor network Actor It can be deduced as and Representing discrete actions and The objective function and follow The calculation method Indicates continuous actions follow The designed objective function, μ, represents the loss function. The weight.

[0141] The above analysis details how to optimize discrete and continuous policies, and based on this, we propose a deep reinforcement learning algorithm based on proximal policy optimization to solve the collaborative training problem of federated learning models. In one embodiment, Figure 8 This is a reinforcement learning structure diagram provided according to Embodiment 4 of the present invention. Figure 8As shown, the environment and relay buffer can be initialized. Initialize the actor network and and critics' network For each episode execution, initialize the observation state S. For executions from t=1 to T: obtain the current state s from the environment. t From discrete strategy and continuous strategy Get action a from t And execute, calculate the immediate reward r t And observe the state s at the next moment. t+1 , set the current state s t Action a t Instant rewards t and the next state s t+1 Convert to transformed sample pairs t ,a t ,r t ,s t+1 >Store to From critics' networks. Calculate the advantage estimate use Update the actor network and use L in PPO-clip Critic Update the commentator network in a certain way, and output θ. c (old)←θ c θ d (old)←θ d Until the instant reward converges.

[0142] This application proposes an innovative framework based on segmentation-based federated learning for low-altitude intelligent IoT architectures. By dynamically adjusting model segmentation points, it achieves efficient allocation of computational tasks and communication loads between UAVs and client devices. This mechanism significantly reduces the computational burden on resource-constrained devices and optimizes communication efficiency. Furthermore, addressing the issue of device heterogeneity in low-altitude intelligent IoT, this application proposes a UAV-client association matching strategy. Combining a comprehensive evaluation of federated learning latency and UAV energy consumption, it dynamically allocates resources, achieving fairness and efficiency in task allocation. Moreover, this application employs a reinforcement learning method based on the Proximal Policy Optimization (PPO-clip) algorithm, jointly optimizing discrete actions (such as task allocation) and continuous actions (such as resource allocation), thus solving the problem that traditional methods struggle to simultaneously optimize multi-dimensional decisions.

[0143] Example 5

[0144] Figure 9 This is a schematic diagram of a segmented federated learning device according to Embodiment 5 of the present invention.​ Figure 9 As shown, the device includes: a resource scheduling module 91, a model determination module 92, a collaborative training module 93, and a model sending module 94.

[0145] Among them, the resource scheduling module 91 is used to obtain the collaborative training model sent by the UAV coordinator, and determine the segmentation layer, associated client and client resource allocation of the collaborative training model based on the target policy optimization algorithm.

[0146] The model determination module 92 is used to divide the co-trained model into client sub-models and drone sub-models according to the segmentation layer, and to treat the client sub-models and drone sub-models as sub-model pairs;

[0147] The collaborative training module 93 is used to allocate computing resources to associated clients through client resource allocation, distribute client sub-models to associated clients, and obtain an initial collaborative training model by performing collaborative training on the sub-model pairs through associated clients and computing drones.

[0148] The model sending module 94 is used to send the initial collaborative training model to the UAV coordinator, so that the UAV coordinator can perform global aggregation of the initial collaborative training model to obtain the target collaborative training model.

[0149] The technical solution of this invention involves obtaining the collaborative training model sent by the UAV coordinator through a resource scheduling module, determining the segmentation layer, associated clients, and client resource allocation based on a target policy optimization algorithm, dividing the collaborative training model into client sub-models and UAV sub-models according to the segmentation layer, and treating the client sub-models and UAV sub-models as sub-model pairs, and allocating computing resources to associated clients through the client resource allocation module, and sending the client sub-models to associated clients. The initial collaborative training model is obtained through collaborative training between the associated clients and the UAV sub-model pairs, and the model sending module sends the initial collaborative training model to the UAV coordinator, enabling the UAV coordinator to globally aggregate the initial collaborative training model to obtain the target collaborative training model, thus achieving efficient allocation of training tasks and resources. Simultaneously, the associated clients only transmit model parameters rather than raw data, which significantly reduces communication overhead and the risk of privacy leakage, and improves the training efficiency of the target collaborative training model.

[0150] In one embodiment, the resource scheduling module 91 includes:

[0151] The current state determination unit is used to collect the current connection state between the client and the drone, the current model slicing layer, the remaining computing power of the drone, the current energy consumption of the drone, and the current communication distance between the client and the drone in real time according to a preset time step as the current state; the policy determination unit is used to input the current state into the target policy optimization algorithm, and obtain the target action in the pre-set action space through the actor network in the target policy optimization algorithm; wherein, the target policy optimization algorithm includes an actor network and a critic network, and the actor network includes discrete policies and continuous policies;

[0152] The state determination unit is used to execute the target action in the current environment, adjust the current connection state and resource allocation of the segmentation layer, client and drone according to the target action, and collect the current state after executing the target action as the state of the next moment; the storage unit is used to determine the immediate reward corresponding to the current state according to the preset reward function, and store the current state, target action, immediate reward and the state of the next moment as a transformation sample pair in the experience buffer; the value determination unit is used to output the current state value estimate based on the current state through the critic network, and output the next state value estimate based on the next moment state, and determine the target value according to the next state value estimate and the immediate reward.

[0153] The parameter determination unit is used to determine the mean squared error between the target value and the current state value estimate as the critic loss function, and update the parameters of the critic network according to the critic loss function. The resource determination unit is used to determine the advantage estimate at each time step based on the generalized advantage estimation algorithm, and update the parameters of the actor network based on the advantage estimate and the near-end policy optimization loss function until the immediate reward converges. The current segmentation layer corresponding to the immediate reward convergence is taken as the segmentation layer, the client and the drone are currently connected clients as associated clients, and the current resource allocation amount is taken as the client resource allocation amount.

[0154] In one embodiment, the strategy determination unit is specifically used for:

[0155] Discrete actions are obtained from a pre-defined action space using a discrete strategy, and continuous actions are obtained from a pre-defined action space using a continuous strategy. The discrete and continuous actions are used as target actions. The discrete actions include at least the cutting layer adjustment action and the association matching action between the client and the drone. The continuous actions include at least the resource allocation action.

[0156] In one embodiment, the preset reward function is constructed based on the resource allocation, the current energy consumption of the UAV, the computing power of the UAV, the current communication distance between the client and the UAV, and the aggregation time of the target collaborative training model; the current state satisfies the preset constraints.

[0157] In one embodiment, the client sub-model is the first layer to the segmentation layer in the collaborative training model; the drone sub-model is the layer below the segmentation layer to the last layer in the collaborative training model.

[0158] In one embodiment, the collaborative training module 93 includes:

[0159] The intermediate data determination unit is used to obtain intermediate data by forward propagating the client sub-model according to local data through the associated client, and then sending the intermediate data to the corresponding computational drone; the gradient determination unit is used to update the drone sub-model by forward propagating the drone sub-model according to the intermediate data through the computational drone, and then backward calculate the gradient of the segmentation layer, and send the gradient to the associated client; the model transmission unit is used to complete the backward update of the client sub-model according to the gradient through the associated client, obtain the updated client sub-model, and then forward propagate the updated client sub-model according to local data until a preset number of rounds of collaborative training are performed, and then send the updated client sub-model to the computational drone; the model aggregation unit is used to aggregate the updated client sub-model and the updated drone sub-model through the computational drone to obtain the initial collaborative training model.

[0160] In one embodiment, a segmented federated learning apparatus further includes:

[0161] The model upload module is used to send the target co-training model as a new co-training model to the computational drone.

[0162] The segmented federated learning device provided in this embodiment of the invention can execute a segmented federated learning method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0163] Example 6

[0164] Figure 10 This is a schematic diagram of the structure of a computational unmanned aerial vehicle (UAV) implementing the segmentation federated learning method of this invention. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0165] like Figure 10As shown, the computing drone 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 and a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer programs stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the computing drone 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0166] Multiple components in the computational drone 10 are connected to I / O interface 15, including: input unit 16, such as a keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as a disk, optical disk, etc.; and communication unit 19, such as a network card, modem, wireless transceiver, etc. Communication unit 19 allows the computational drone 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks. Processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, central processing unit (CPU), graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as a segmented federated learning method.

[0167] In some embodiments, a segmentation federated learning method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the computing drone 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the segmentation federated learning method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform a segmentation federated learning method by any other suitable means (e.g., by means of firmware).

[0168] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0169] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, partially on a machine and partially on a remote machine as a standalone software package, or entirely on a remote machine or server. In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. Computer-readable storage media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0170] To provide interaction with a user, the systems and techniques described herein can be implemented on a computational drone, which includes: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computational drone. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0171] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0172] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0173] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0174] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A segmented federated learning method, characterized in that, Applications to computing drones include: Obtain the collaborative training model sent by the UAV coordinator, and determine the segmentation layer, associated clients, and client resource allocation of the collaborative training model based on the objective policy optimization algorithm; The collaborative training model is divided into a client sub-model and a drone sub-model according to the segmentation layer, and the client sub-model and the drone sub-model are regarded as a sub-model pair; The computing resources are allocated to the associated clients using the client resource allocation amount, the client sub-models are distributed to the associated clients, and the associated clients and computing drones perform collaborative training on the sub-model pairs to obtain an initial collaborative training model; The initial collaborative training model is sent to the UAV coordinator, so that the UAV coordinator can globally aggregate the initial collaborative training model to obtain the target collaborative training model.

2. The method according to claim 1, characterized in that, The determination of the segmentation layer, associated clients, and client resource allocation for the collaborative training model based on the objective policy optimization algorithm includes: The current status is collected in real time according to the preset time steps, including the current connection status between the client and the drone, the current model cutting layer, the remaining computing power of the drone, the current energy consumption of the drone, and the current communication distance between the client and the drone. The current state is input into the target policy optimization algorithm, and the target action is obtained in the pre-set action space through the actor network in the target policy optimization algorithm; wherein, the target policy optimization algorithm includes an actor network and a critic network, and the actor network includes discrete policies and continuous policies; The target action is executed in the current environment. The current connection status and resource allocation of the segmentation layer, client and drone are adjusted according to the target action. The current state after the target action is executed is collected as the state at the next moment. The immediate reward corresponding to the current state is determined according to a preset reward function, and the current state, the target action, the immediate reward, and the next state are stored as a transition sample pair in the experience buffer. The critic network outputs a current state value estimate based on the current state and a next state value estimate based on the next time step state. The target value is determined according to the next state value estimate and the immediate reward. The mean squared error between the target value and the current state value estimate is determined as the critic loss function, and the parameters of the critic network are updated according to the critic loss function. The generalized advantage estimation algorithm is used to determine the advantage estimate at each time step. The parameters of the actor network are updated based on the advantage estimate and the near-end policy optimization loss function until the immediate reward converges. The current segmentation layer corresponding to the immediate reward convergence is taken as the segmentation layer. The client and the drone are currently connected. The client is taken as the associated client. The current resource allocation amount is taken as the client resource allocation amount.

3. The method according to claim 2, characterized in that, The step of obtaining the target action from a pre-defined action space through the actor network in the target policy optimization algorithm includes: Discrete actions are obtained in a pre-set action space through a discrete strategy, and continuous actions are obtained in a pre-set action space through a continuous strategy. The discrete actions and the continuous actions are used as target actions. The discrete actions include at least the cutting layer adjustment action and the association matching action between the client and the drone; the continuous actions include at least the resource allocation action.

4. The method according to claim 2, characterized in that, The preset reward function is constructed based on the resource allocation, the current energy consumption of the drone, the computing power of the drone, the current communication distance between the client and the drone, and the aggregation time of the target collaborative training model. The current state satisfies the preset constraints.

5. The method according to claim 1, characterized in that, The client sub-model consists of the first layer to the segmentation layer in the collaborative training model; the drone sub-model consists of the layer below the segmentation layer to the last layer in the collaborative training model.

6. The method according to claim 1, characterized in that, The step of obtaining an initial collaborative training model by collaboratively training the sub-model pair through the associated client and the computing drone includes: Intermediate data is obtained by forward propagating the client sub-model according to the local data through the associated client, and the intermediate data is sent to the corresponding computing drone; The computational drone updates the drone sub-model by forward propagation based on intermediate data, and then calculates the gradient of the segmentation layer backward, sending the gradient to the associated client. The client sub-model is updated backward by the associated client according to the gradient to obtain the updated client sub-model. Then, the updated client sub-model is forward propagated through local data until a preset number of rounds of collaborative training are performed. Finally, the updated client sub-model is sent to the computational drone. The computational drone is aggregated according to the updated client sub-model and the updated drone sub-model to obtain the initial collaborative training model.

7. The method according to claim 1, characterized in that, After the UAV coordinator globally aggregates the initial collaborative training model to obtain the target collaborative training model, the following steps are also included: The target collaborative training model is then distributed to the computational drone as a new collaborative training model.

8. A segmented federated learning device, characterized in that, include: The resource scheduling module is used to obtain the collaborative training model sent by the UAV coordinator, and determine the segmentation layer, associated clients, and client resource allocation of the collaborative training model based on the objective policy optimization algorithm. The model determination module is used to divide the collaboratively trained model into a client sub-model and a drone sub-model according to the segmentation layer, and to treat the client sub-model and the drone sub-model as a sub-model pair; The collaborative training module is used to allocate computing resources to associated clients through the client resource allocation amount, distribute the client sub-models to associated clients, and perform collaborative training on the sub-model pairs through the associated clients and the computing drone to obtain an initial collaborative training model; The model sending module is used to send the initial collaborative training model to the UAV coordinator, so that the UAV coordinator can globally aggregate the initial collaborative training model to obtain the target collaborative training model.

9. A computational unmanned aerial vehicle (UAV), characterized in that, The computing drone includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform a segmented federated learning method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute and implement a segmented federated learning method according to any one of claims 1-7.

Images