A multi-task modeling method, device, equipment and medium for drone swarm

Abstract

The present application provides a multi-task modeling method, device, equipment and medium for a drone swarm, the method comprising the steps of: constructing a multi-task initial model; issuing global initial parameters; iteratively training personalized initial sub-models and global initial sub-models to determine model pruning strategies; pruning and iteratively training global training sub-models; uploading and aggregating global pruning parameters; repeating the above operations until the target loss function converges to generate a multi-task application model. The present application constructs global initial sub-models and personalized initial sub-models to cope with different but interrelated tasks between drones, achieve knowledge sharing, and help each drone complete specific tasks. In view of the frequent model parameter exchanges and bandwidth constraints of wireless spectrum resources, model pruning strategies and bandwidth allocation strategies are provided to increase the transmission rate when model parameters are exchanged; ultimately, a high-precision multi-task application model is obtained to achieve a variety of tasks with strong adaptability.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle technology, and in particular to a multi-task modeling method, device, equipment and medium for a group of unmanned aerial vehicles. Background Art

[0002] Drones have the advantages of fast deployment, controllable action, and flexible network, and are widely used in various fields. For example, drones can be used in the field of task processing.

[0003] With the widespread application of various machine learning networks in various fields, when executing pending tasks based on drones, federated learning can generally be used to improve the accuracy of task processing. Federated learning can support multiple distributed data holders to collaboratively train specific neural network models without sharing local data. Compared with traditional centralized network model training, local raw data in federated learning does not need to leave the local storage of each distributed participant (ie, data holder). Therefore, the risk of data leakage can be greatly reduced and the rights and interests of each participant can be better protected. Secondly, the training of network models in federated learning is carried out in parallel at each participant, and no longer relies too much on the computing and storage capabilities of the central server. Therefore, the computing efficiency is also greatly improved.

[0004] However, most previous federated learning requires all participants to have the same network model structure. When the local data of each participant is not independent and identically distributed, this data heterogeneity will cause the federated learning training process to be extremely unstable, which in turn causes the global model to have low generalization ability and cannot be well applied to the specific applications of each participant. In addition, participants with limited computing power will limit the complexity of the trainable network model, thereby affecting the overall learning and reasoning ability of the network model, as well as the real-time online update ability of the network model. Furthermore, in the process of federated learning, each participant needs to frequently exchange network model parameters with the parameter server, which poses a challenge to how to efficiently utilize the scarce wireless spectrum resources in reality. Summary of the invention

[0005] The present invention provides a multi-task modeling method, device, equipment and medium for a drone swarm, which solves the problem that when a drone swarm is modeled to handle different tasks, the previous training process of federated learning is extremely unstable, resulting in a global model with low generalization ability, limiting the complexity of the network model, affecting its overall learning and reasoning capabilities, and real-time online update capabilities; at the same time, frequent network model parameter exchanges lead to excessive delays in drone calculations and communications.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is to provide a multi-task modeling method for a drone swarm, comprising the following steps: a first step, obtaining a plurality of different tasks to be processed of the drone swarm, and constructing a multi-task initial model according to the tasks to be processed; wherein the multi-task initial model includes a global initial sub-model and a plurality of different personalized initial sub-models; a second step, broadcasting and sending the global initial parameters in the global initial sub-model to each drone; a third step, iteratively training and pruning the global initial sub-model of each drone based on the global initial parameters, the local training data set of each drone, the personalized initial sub-model and the model pruning strategy, respectively, to obtain the global pruning parameters corresponding to each drone; a fourth step, after uploading all the global pruning parameters, performing aggregation processing to obtain the global aggregation parameters; a fifth step, repeatedly performing the same operations as the second step to the fourth step until the target loss function converges, obtaining the global application parameters, and broadcasting and sending the global application parameters to each drone, respectively generating the multi-task application model corresponding to each drone; wherein the multi-task application model includes a global application sub-model and a personalized application sub-model.

[0007] In some embodiments, the third step specifically includes: based on the global initial parameters and the local training data set of each of the UAVs, iteratively training the personalized initial sub-model corresponding to each of the UAVs to obtain a personalized training sub-model; according to the global initial parameters, the local training data set of each of the UAVs and the personalized training sub-model, iteratively training the global initial sub-model of each of the UAVs to obtain a global training sub-model; using the model pruning strategy, pruning and iteratively training the global training sub-model of each of the UAVs to obtain a global pruned sub-model, and extracting the global pruning parameters in the global pruned sub-model corresponding to each of the UAVs.

[0008] In some embodiments, the model pruning strategy is used to perform iterative training after pruning the global training sub-model of each of the drones, including: calculating the global parameter change based on the global initial parameters of the global initial sub-model and the global training parameters of the global training sub-model; calculating the pruning identification vector according to the model pruning rate and the global parameter change; pruning the global training sub-model according to the pruning identification vector, and iteratively training the pruned global training sub-model; wherein the model pruning strategy includes the global parameter change, the model pruning rate and the pruning identification vector.

[0009] In some embodiments, the method for calculating the model pruning rate includes the steps of: transforming the target loss function according to the calculation and communication delay constraints of the drone swarm, and calculating the value range of the model pruning rate; sequentially using the convex optimization problem analysis method, the Lagrange multiplier method, and the KKT conditional analysis method for mathematical modeling, solving the transformed target loss function, and determining the model pruning rate according to the value range of the model pruning rate.

[0010] In some embodiments, in the fourth step, the method for uploading the global pruning parameters includes the steps of: allocating a corresponding upload bandwidth to the global pruning parameters corresponding to each of the drones based on a bandwidth allocation factor and a total bandwidth; and uploading all of the global pruning parameters according to the corresponding upload bandwidth through an orthogonal frequency division multiple access transmission method.

[0011] In some embodiments, the method for calculating the bandwidth allocation factor includes the steps of: transforming the target loss function according to the calculation and communication delay constraints and bandwidth constraints of the drone swarm; using convex optimization problem analysis method, Lagrange multiplier method and KKT condition analysis method in sequence for mathematical modeling, solving the transformed target loss function, and determining the bandwidth allocation factor.

[0012] In some embodiments, the method for constructing the target loss function includes: obtaining a federal loss function, and constructing the federal loss function as an initial loss function based on the global initial sub-model, the personalized initial sub-model of each of the drones, and a local training data set; performing an equivalent transformation on the initial loss function according to the computing and communication delay constraints, bandwidth constraints, and model pruning constraints of the drone swarm to obtain the target loss function.

[0013] The present invention also provides a multi-task modeling device for a drone swarm, comprising: an acquisition and construction unit, used to acquire a plurality of different tasks to be processed of the drone swarm, and construct a multi-task initial model according to the tasks to be processed; wherein the multi-task initial model includes a global initial sub-model and a plurality of different personalized initial sub-models; a distribution unit, used to broadcast and distribute the global initial parameters in the global initial sub-model to each drone; and also used to broadcast and distribute the global application parameters to each drone; a training and pruning unit, used to iteratively train and prune the global initial sub-model of each drone based on the global initial parameters, the local training data set of each drone, the personalized initial sub-model and the model pruning strategy, and obtain the global pruning parameters corresponding to each drone; an uploading unit, used to upload all the global pruning parameters; an aggregation unit, used to aggregate all the global pruning parameters to obtain global aggregation parameters; a generating unit, used to generate the multi-task application model corresponding to each drone; wherein the multi-task application model includes a global application sub-model and a personalized application sub-model.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method when executing the computer program.

[0015] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described above are implemented.

[0016] The beneficial effects of the present invention are as follows: the present invention discloses a multi-task modeling method, device, equipment and medium for a drone swarm, the method comprising the following steps: the first step is to construct a multi-task initial model; the second step is to send the global initial parameters to each drone; the third step is to iteratively train and prune the global initial sub-model; the fourth step is to perform aggregation processing after uploading all the global pruning parameters to obtain the global aggregation parameters; the fifth step is to repeat the same operations as the second to fourth steps until the target loss function converges, send the global application parameters to each drone, and generate a multi-task application model corresponding to each drone. The present application can effectively cope with different but interrelated tasks to be processed between drones by constructing a global initial sub-model and a plurality of different personalized initial sub-models, and while realizing knowledge sharing through the global initial sub-model, it also helps each drone complete specific diversified tasks through the personalized initial sub-model. In response to the frequent model parameter exchanges and bandwidth constraints of wireless spectrum resources faced by drone swarms, a model pruning strategy is provided to prune model parameters, effectively reducing the burden of model parameter exchange; and an optimized bandwidth allocation strategy is provided to greatly improve the transmission rate and utilization rate during model parameter exchange, reducing calculation and communication delays; ultimately, a high-precision, strong generalization capability multi-task application model is obtained to realize various tasks between each drone, with strong adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flowchart of a multi-task modeling method of a drone swarm in the present application;

[0018] Figure 2 It is a schematic diagram of the architecture of a multi-task modeling method of a drone swarm in the present application;

[0019] Figure 3 It is an iterative schematic diagram of a multi-task initial model in a multi-task modeling method for a drone swarm in the present application;

[0020] Figure 4 It is a schematic diagram of iterative training of a model corresponding to one of the drones in a multi-task modeling method of a drone swarm in the present application;

[0021] Figure 5 It is a schematic diagram of solving the target loss function, bandwidth allocation factor and model pruning rate in a multi-task modeling method of a drone swarm in the present application;

[0022] Figure 6 This is a schematic diagram of the composition of a multi-task modeling device for a drone swarm in the present application;

[0023] Figure 7 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present application;

[0024] Figure 8 It is a schematic block diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0025] In order to facilitate the understanding of the present invention, the present invention is described in more detail below in conjunction with the accompanying drawings and specific embodiments. Preferred embodiments of the present invention are provided in the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described in this specification. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.

[0026] It should be noted that, unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used in this specification includes any and all combinations of one or more related listed items.

[0027] like Figure 1 As shown, the present invention provides a multi-task modeling method for a drone swarm, the method comprising the steps of:

[0028] S1: Acquire multiple different pending tasks of the drone swarm, and construct a multi-task initial model according to the pending tasks; wherein the multi-task initial model includes a global initial sub-model and multiple different personalized initial sub-models.

[0029] S2: Broadcast the global initial parameters in the global initial sub-model to each UAV.

[0030] S3: Based on the global initial parameters, the local training dataset of each UAV, the personalized initial sub-model and the model pruning strategy, the global initial sub-model of each UAV is iteratively trained and pruned to obtain the global pruning parameters corresponding to each UAV.

[0031] S4: After uploading all global pruning parameters, aggregation processing is performed to obtain global aggregation parameters.

[0032] S5: Repeat the same operations as S2 to S4 until the target loss function converges, obtain the global application parameters, and broadcast the global application parameters to each drone to generate a multi-task application model corresponding to each drone; wherein the multi-task application model includes a global application sub-model and a personalized application sub-model.

[0033] This application can effectively deal with the different but interrelated tasks to be processed between drones by constructing a global initial sub-model and multiple different personalized initial sub-models. While realizing knowledge sharing through the global initial sub-model, it also helps each drone complete specific diversified tasks through personalized initial sub-models. Further, in response to the frequent model parameter exchanges and bandwidth constraints of wireless spectrum resources faced by drone groups, a model pruning strategy is provided to prune model parameters, effectively reducing the burden of model parameter exchange; and an optimized bandwidth allocation strategy is provided to greatly improve the transmission rate and utilization rate when exchanging model parameters, reducing computing and communication delays; ultimately, a high-precision, strong generalization capability multi-task application model is obtained to realize various tasks between drones, with strong adaptability.

[0034] Combine the following Figures 1 to 8 , the present application is further described in detail with specific embodiments.

[0035] Figure 1 A schematic diagram of the process of the multi-task modeling method of the drone swarm provided in the embodiment of the present application is shown, and the details are as follows:

[0036] S1: Acquire multiple different pending tasks of the drone swarm, and construct a multi-task initial model according to the pending tasks; wherein the multi-task initial model includes a global initial sub-model and multiple different personalized initial sub-models.

[0037] It should be noted that the tasks to be processed for each drone in the drone swarm in this application are different but related to each other.

[0038] First, a multi-task initial model is constructed according to multiple different tasks to be processed. The multi-task initial model includes a global initial sub-model and multiple different personalized initial sub-models. Each task to be processed (or each drone) corresponds to a global initial sub-model and a personalized initial sub-model; therefore, each drone has its own different local training data set for its global initial sub-model and personalized initial sub-model. The global initial sub-models of all drones have the same network model architecture to achieve efficient knowledge sharing among drones; the personalized initial sub-model of each drone is privately owned by each drone, is different, and is designed according to the specific task to be completed by the drone (i.e., the task to be processed).

[0039] In this application, a drone swarm is set to include K drones, and one of the drones carries an edge computing device, and the drone carrying the edge computing device is set as a parameter server for the federated learning process.

[0040] Furthermore, the set of K drones is defined as: K = {1, 2, ..., K}. Then the local training data set of drone k∈K can be expressed as:

[0041]

[0042] Among them, D k represents the local training dataset of drone k, x i represents the i-th sample, y i Represents the corresponding i-th label (i.e., sample x i The real result), D k Representative D k The number of samples in .

[0043] Then, the local training data set of all drones can be expressed as:

[0044]

[0045] Where D represents the local training dataset of all drones, D k represents the local training dataset of one of the drones k, and K represents the total number of K drones.

[0046] Then, the number of samples of all drones can be expressed as:

[0047]

[0048] Where D represents the number of samples of all drones, D k Represents the number of samples of drone k.

[0049] Furthermore, from the above, we can see that the global initial sub-model of all drones can be defined as u, and the personalized initial sub-model of drone k can be defined as v k Then in the g-th model iteration, the global initial submodel is represented by u g , the global initial sub-model of UAV k can also be expressed as The personalized initial sub-model of drone k is expressed as

[0050] Furthermore, in order to evaluate the performance of the global initial sub-model and the personalized initial sub-model during the model iteration of the present application, the present application constructs a target loss function.

[0051] Specific, combined Figure 5 As shown, the method for constructing the target loss function includes:

[0052] Obtain a federated loss function, and construct the federated loss function as an initial loss function based on the global initial sub-model, the personalized initial sub-model of each drone, and the local training dataset.

[0053] In this embodiment, the obtained federated loss function is expressed as:

[0054]

[0055] Where D represents the number of samples in the local training dataset D of all drones, D k Represents the local training dataset D of drone k k The number of samples, f k (x i ,y i ,w) represents one of the samples x i The corresponding loss function value, F k (w) represents the federated loss function of drone k, F(w) represents the federated loss function of all drones, and w represents the federated model.

[0056] Based on the aforementioned global initial sub-model, the personalized initial sub-model of each drone, and the local training dataset, the above-mentioned federated loss function is constructed as the initial loss function and expressed as the following formula:

[0057]

[0058] Among them, w=[u,v k ], u represents the global initial sub-model, v k represents the personalized initial sub-model of drone k, D represents the number of samples in the local training dataset D of all drones, and D k Represents the local training dataset D of drone k k The number of samples, f k (x i ,y i ,u,v k ) represents one of the samples x i The corresponding loss function value, F k (u,v k ) represents the initial loss function of drone k, Represents the initial loss function of all drones.

[0059] Furthermore, according to the computation and communication delay constraints, bandwidth constraints, and model pruning constraints of the drone swarm, the above initial loss function is equivalently transformed to obtain the target loss function.

[0060] Specifically, the above process can be expressed by the following formula:

[0061]

[0062] Among them, st represents that T g≤T th represents the computation and communication delay constraints, and represents the bandwidth constraint, represents the model pruning constraint; T g represents the time required for the g-th iteration of the global initial sub-model in the parameter server of this application, T th Represents the maximum time constraint for iterative updating of the global initial sub-model in the parameter server. represents the bandwidth allocation factor corresponding to the g-th iteration of the global initial sub-model of drone k, represents the model pruning rate corresponding to the g-th iteration of the global initial sub-model of drone k, G represents the total number of iterations of the global initial sub-model in the parameter server, Represents the target loss function.

[0063] S2: Broadcast the global initial parameters in the global initial sub-model to each UAV.

[0064] Combination Figure 2 As shown in the figure, the first model iteration begins, and parameter server 1 (i.e., the drone with edge computing equipment) sends the global initial sub-model u g The global initial parameters in are broadcasted to each drone. Among them, g represents the global initial sub-model u in parameter server 1 g The g-th iteration of , the current value of g is 1.

[0065] Since the downlink has sufficient channel bandwidth, the transmission rate is usually high. Therefore, the time delay of parameter server 1 broadcasting can be ignored.

[0066] S3: Based on the global initial parameters, the local training dataset of each UAV, the personalized initial sub-model and the model pruning strategy, the global initial sub-model of each UAV is iteratively trained and pruned to obtain the global pruning parameters corresponding to each UAV.

[0067] Combination Figure 3 , Figure 4 As shown, specifically, firstly, based on the global initial parameters and the local training data set of each UAV, the personalized initial sub-model corresponding to each UAV is iteratively trained to obtain a personalized training sub-model.

[0068] In this embodiment, it is set to iteratively train the personalized initial sub-model corresponding to each drone τ v times; set a personalized initial sub-model for one of the drones k The number of personalized parameters (i.e., the number of network weights) is The CPU frequency of drone k is f k, update the personalized initial sub-model A parameter requires the CPU of drone k to run C at a time k times. Then the personalized initial sub-model of one of the drones k is Iterative training τ v After that, it can be expressed by the following formula:

[0069]

[0070] in, Represents iterative training τ v After that, the personalized training sub-model of drone k is Represents iterative training τ v -1 personalized initial sub-model of drone k after time η v represents the learning rate, represents the gradient, Represents the local training dataset D of drone k selected by stochastic gradient descent k Sample data in u g Represents the current global initial submodel.

[0071] Furthermore, based on the CPU frequency f of drone k k , the number of CPU operations required to update a personalized parameter once C k , and the personalized initial sub-model of drone k The number of personalized parameters The personalized initial sub-model can be calculated Iterative training τ v The time required is expressed by the following formula:

[0072]

[0073] in, Represents personalized initial sub-model Iterative training τ v The time required.

[0074] Furthermore, according to the global initial parameters, the local training data set of each UAV and the personalized training sub-model, the global initial sub-model of each UAV is iteratively trained to obtain the global training sub-model.

[0075] Combination Figure 3 , Figure 4 As shown, in this embodiment, it is set to iteratively train the global initial sub-model corresponding to each drone times; set the global initial sub-model of one of the drones k The number of global initial parameters (i.e. the number of network weights) is Update the global initial submodel A global initial parameter requires the CPU of drone k to run C once k times. Then the global initial sub-model of one of the drones k is Iterative training After that, it can be expressed by the following formula:

[0076]

[0077] in, Represents iterative training After that, the global training sub-model of drone k is Represents iterative training The global initial sub-model of drone k after the second time, η u represents the learning rate, represents the gradient, Represents the local training dataset D of drone k selected by stochastic gradient descent k The sample data in Represents the personalized training sub-model.

[0078] Furthermore, based on the CPU frequency f of drone k k , the number of CPU operations required to update a global initial parameter once C k , and the global initial sub-model of UAV k The number of global initial parameters The global initial sub-model can be calculated Iterative training The time required is expressed by the following formula:

[0079]

[0080] in, Represents the global initial submodel Iterative training The time required.

[0081] Furthermore, the model pruning strategy is used to prune the global training sub-model of each UAV and then iteratively train it to obtain a global pruned sub-model.

[0082] The model pruning strategy includes the global parameter change, the model pruning rate and the pruning identification vector.

[0083] Specific, combined Figure 3 , Figure 4As shown, firstly, based on the global initial parameters of the global initial sub-model and the global training parameters of the global training sub-model, the global parameter change is calculated. The above process can be expressed by the following formula:

[0084]

[0085] in, Represents the global initial submodel and global training sub-model The corresponding global parameter change of the jth parameter, Represents the global training sub-model The jth parameter value of Represents the global initial submodel The jth parameter value of .

[0086] Furthermore, according to the current model pruning rate Global initial submodel and global training sub-model Global parameter delta for all parameters Calculate the trimming identification vector in, represents the pruning identity vector of the jth parameter, and like Indicates that the jth parameter is retained, if Indicates that the jth parameter is pruned.

[0087] Further, according to the trimming identification vector Train sub-models globally The global training parameters are pruned, and the pruned global training sub-model is iteratively trained to obtain the global pruned sub-model. The above process can be expressed by the following formula:

[0088]

[0089] where ⊙ represents the element-wise product, Represents iterative training τ u The global pruning sub-model of drone k after the second time, Represents iterative training τ u -1 times after the global training sub-model of drone k, η u represents the learning rate, represents the gradient, Represents the local training dataset D of drone k selected by stochastic gradient descent k The sample data in Represents the personalized training sub-model.

[0090] Furthermore, based on the CPU frequency f of drone kk , the number of CPU operations required to update a global training parameter once C k , the global initial sub-model of UAV k The number of global initial parameters And the model pruning rate The pruned global training sub-model can be calculated to iterate the training τ u The time required is expressed by the following formula:

[0091]

[0092] in, Represents the pruned global training sub-model iterative training τ u The time required.

[0093] In summary, the global initial sub-model of drone k in S3 The time required for the g-th iteration training (the current value of g is 1) is:

[0094]

[0095] in, Represents personalized initial sub-model Iterative training τ v The time required, Represents the global initial submodel Iterative training The time required, Represents the pruned global training sub-model iterative training τ u The time required.

[0096] Furthermore, the global pruning parameters in the global pruning sub-model corresponding to each UAV are extracted.

[0097] S4: After uploading all global pruning parameters, aggregation processing is performed to obtain global aggregation parameters.

[0098] Combination Figure 2 , Figure 3 As shown, each UAV extracts the global pruning parameters in its corresponding global pruning sub-model and uploads them to the parameter server 1 respectively.

[0099] Specifically, the method for uploading global pruning parameters includes the following steps:

[0100] Based on the bandwidth allocation factor and the total bandwidth, the corresponding upload bandwidth is allocated to the global pruning parameters corresponding to each drone. Through the orthogonal frequency division multiple access transmission method, all global pruning parameters are uploaded according to the corresponding upload bandwidth. Among them, the orthogonal frequency division multiple access transmission method can ensure that the transmission channels of multiple drones will not interfere with each other when uploading.

[0101] In this embodiment, the transmission power of one of the drones k is set to p k , the uplink transmission gain during uploading is The number of bits required to upload each global pruning parameter is The background noise power is σ 2 , the total bandwidth is B, and the bandwidth allocation factor corresponding to drone k is (i.e. the proportion of the total bandwidth B), then and According to the above settings, the uplink transmission rate during uploading can be calculated and expressed as follows:

[0102]

[0103] in, represents the uplink transmission rate when UAV k uploads its global pruning parameters.

[0104] Furthermore, the time required for UAV k to upload its global pruning parameters is calculated using the following formula:

[0105]

[0106] in, represents the uplink transmission rate when UAV k uploads its global pruning parameters, represents the model pruning rate, represents all global pruning parameters of UAV k, represents the time required for UAV k to upload its global pruning parameters.

[0107] Furthermore, after each UAV uploads its own global pruning parameters to the parameter server 1, the parameter server 1 aggregates all global pruning parameters to obtain global aggregate parameters. And further, the global initial sub-model u in the parameter server 1 is aggregated according to the global aggregate parameters. g The updated global initial sub-model is expressed as follows:

[0108]

[0109] Among them, u g+1 represents the updated global initial sub-model in parameter server 1, Kg represents the set of all drones in the g-th iteration, |K g |Represents K g The number of Represents the global initial sub-model of UAV k.

[0110] In summary, the time required for an iterative update of the global initial sub-model in parameter server 1 is:

[0111]

[0112] in, Represents the personalized initial sub-model of drone k Iterative training τ v The time required, Represents the global initial sub-model of UAV k Iterative training The time required, Represents the iterative training τ of the global training sub-model after drone k pruning u The time required, represents the time required for UAV k to upload its global pruning parameters, T g represents the time required for one iteration update of the global initial sub-model in parameter server 1, Represents the time required for one of the UAVs k to perform the g-th iteration.

[0113] S5: Repeat the same operations as S2 to S4 until the target loss function converges, obtain the global application parameters, and broadcast the global application parameters to each drone to generate a multi-task application model corresponding to each drone; wherein the multi-task application model includes a global application sub-model and a personalized application sub-model.

[0114] Specifically, after the parameter server obtains the global aggregation parameters, it starts its next iterative update and continues to perform the same operations as S2 to S4 (such as broadcasting, iterative training, pruning, uploading, aggregation, etc.) until the target loss function constructed in S1 converges. Finally, the parameter server obtains the aggregated global application parameters and broadcasts the global application parameters to each drone. Each drone updates the parameters of its own model according to the global application parameters, and finally obtains its corresponding multi-task application model. Among them, the multi-task application model includes a global application sub-model and a personalized application sub-model. The global application sub-model of each drone is the same, while the personalized application sub-models are different. Furthermore, according to the control of the parameter server, each different task to be processed is executed through its own multi-task application model.

[0115] In summary, it should be noted that if Figure 5As shown, the calculation method of the bandwidth allocation factor in S4 of the present application includes the steps of:

[0116] According to the computation and communication delay constraints and bandwidth constraints of the drone swarm, the target loss function is transformed. The convex optimization problem analysis method, Lagrange multiplier method and KKT condition analysis method are used in sequence for mathematical modeling, the transformed target loss function is solved, and the bandwidth allocation factor is determined.

[0117] Specifically, according to the computation and communication delay constraints T of the drone swarm in S1 g ≤T th , and the time T required for one iteration update of the global initial sub-model in the parameter server obtained in S4 above g , the transformation can be obtained:

[0118]

[0119] Further, Further transformation and calculation of the above formula can be obtained:

[0120]

[0121] in, Represents the range of values ​​for the model pruning rate.

[0122] Furthermore, based on the range of the pruning rate of the above model and the target loss function constructed in S1, the formula of the converted target loss function is:

[0123]

[0124] Furthermore, the convex optimization problem analysis method, Lagrange multiplier method and KKT condition analysis method are used in turn for mathematical modeling, the converted target loss function is solved, and the bandwidth allocation factor is determined, which is expressed as the following formula:

[0125]

[0126] in, represents the bandwidth allocation factor and λ represents the Lagrange multiplier.

[0127] Furthermore, according to the above bandwidth allocation factor Similarly, the method for calculating the model pruning rate in S3 of the present application includes the following steps:

[0128] According to the calculation and communication delay constraints of the drone group, the target loss function is transformed and the range of the model pruning rate is calculated. The convex optimization problem analysis method, Lagrange multiplier method and KKT condition analysis method are used in sequence for mathematical modeling, the transformed target loss function is solved, and the model pruning rate is determined according to the range of the model pruning rate (for details, please refer to the bandwidth allocation factor b mentioned above). k g The calculation process is not repeated here), and is expressed by the following formula:

[0129]

[0130] in, Represents the model pruning rate.

[0131] Corresponding to the multi-task modeling method of the drone swarm in the above embodiment, Figure 6 A schematic diagram of the composition of a multi-task modeling device for a drone swarm provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0132] refer to Figure 6 , the multi-task modeling device of the drone swarm includes:

[0133] The acquisition construction unit 101 is used to obtain multiple different tasks to be processed of the drone group and construct a multi-task initial model according to the tasks to be processed; wherein the multi-task initial model includes a global initial sub-model and multiple different personalized initial sub-models.

[0134] The sending unit 102 is used to broadcast and send the global initial parameters in the global initial sub-model to each drone; and is also used to broadcast and send the global application parameters to each drone.

[0135] The training and pruning unit 103 is used to iteratively train and prune the global initial sub-model of each drone based on the global initial parameters, the local training data set of each drone, the personalized initial sub-model and the model pruning strategy, so as to obtain the global pruning parameters corresponding to each drone.

[0136] The uploading unit 104 is used to upload all global pruning parameters.

[0137] The aggregation unit 105 is used to aggregate all global pruning parameters to obtain global aggregation parameters.

[0138] The generating unit 106 is used to generate a multi-task application model corresponding to each drone respectively; wherein the multi-task application model includes a global application sub-model and a personalized application sub-model.

[0139] It should be noted that since the multi-task modeling device of the above-mentioned drone swarm is based on the same concept as the embodiment of the method of the present application, its specific functions, the process of each module realizing its respective function and the technical effects brought about can be specifically referred to the description of the embodiment shown in the multi-task modeling method of the above-mentioned drone swarm, and will not be repeated here.

[0140] Based on the same inventive concept, the present application also provides an electronic device, which includes a processor, a memory and a communication circuit, and the processor is connected to the memory and the communication circuit respectively; wherein the communication circuit is used for communication connection, the memory is used to store a computer program, and the processor is used to execute the computer program to implement the above-mentioned multi-task modeling method of the drone swarm.

[0141] See also Figure 7 The electronic device described in the embodiment of the present application may specifically include a processor 210 and a memory 220. The memory 220 is coupled to the processor 210.

[0142] The processor 210 is used to control the operation of the electronic device. The processor 210 may also be referred to as a CPU (Central Processing Unit). The processor 210 may be an integrated circuit chip having the ability to process signals. The processor 210 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. A general-purpose processor may be a microprocessor or the processor 210 may also be any conventional processor, etc.

[0143] The memory 220 is used to store computer programs, and may be a RAM, a ROM, or other types of storage terminals. Specifically, the memory 220 may include one or more computer-readable storage media, which may be non-transitory or transient. The memory 220 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage terminals, flash memory storage terminals. In some embodiments, the non-transitory computer-readable storage medium in the memory 220 is used to store at least one program code.

[0144] The processor 210 is used to execute the computer program stored in the memory 220 to implement the methods described in the various method embodiments of the present application.

[0145] In some embodiments, the electronic device may further include: a peripheral terminal interface 230 and at least one peripheral terminal. The processor 210, the memory 220 and the peripheral terminal interface 230 may be connected via a bus or a signal line. Each peripheral terminal may be connected to the peripheral terminal interface 230 via a bus, a signal line or a circuit board. Specifically, the peripheral terminal includes: at least one of a radio frequency circuit 240, a display screen 250, an audio circuit 260 and a power supply 270.

[0146] The peripheral terminal interface 230 may be used to connect at least one peripheral terminal related to I / O (Input / output) to the processor 210 and the memory 220. In some embodiments, the processor 210, the memory 220, and the peripheral terminal interface 230 are integrated on the same chip or circuit board; in some other implementations, any one or two of the processor 210, the memory 220, and the peripheral terminal interface 230 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0147] The radio frequency circuit 240 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 240 communicates with the communication network and other IoT devices through electromagnetic signals, and the radio frequency circuit 240 is the communication circuit of the electronic device. The radio frequency circuit 240 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 240 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, an operator identity module card, and the like. The radio frequency circuit 240 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 240 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0148] The display screen 250 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 250 is a touch display screen, the display screen 250 also has the ability to collect touch signals on the surface or above the surface of the display screen 250. The touch signal can be input to the processor 210 as a control signal for processing. At this time, the display screen 250 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 250 can be one, set on the front panel of the electronic device; in other embodiments, the display screen 250 can be at least two, respectively set on different surfaces of the electronic device or in a folding design; in other embodiments, the display screen 250 can be a flexible display screen, set on a curved surface or a folding surface of the electronic device. Even, the display screen 250 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 250 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0149] The audio circuit 260 may include a microphone and a speaker. The microphone is used to collect sound waves from the operator and the environment, and convert the sound waves into electrical signals and input them into the processor 210 for processing, or input them into the radio frequency circuit 240 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged in different parts of the electronic device. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 210 or the radio frequency circuit 240 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 260 may also include a headphone jack.

[0150] The power supply 270 is used to power various components in the electronic device. The power supply 270 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 270 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged through a wired line, and a wireless rechargeable battery is a battery that is charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0151] For a detailed description of the functions and execution processes of each functional module or component in the electronic device embodiment of the present application, reference can be made to the description in the above-mentioned method embodiments of the present application, which will not be repeated here.

[0152] In the several embodiments provided in the present application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the various embodiments of the electronic devices described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0153] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0154] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0155] Based on the same inventive concept, the present application also provides a computer-readable storage medium storing a computer program, which can be executed by a processor to implement the multi-task modeling method of the above-mentioned drone swarm.

[0156] See also Figure 8 , if the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium 300. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions / computer programs to enable an IoT device (which can be a personal computer, server, or network terminal, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks or optical disks, and electronic terminals such as computers, mobile phones, laptops, tablet computers, cameras, etc. having the above-mentioned storage media.

[0157] The description of the execution process of the program data in the computer-readable storage medium can refer to the description in the above-mentioned method embodiments of the present application, and will not be repeated here.

[0158] It can be seen that the present invention discloses a multi-task modeling method, device, equipment and medium for a drone swarm, the method comprising the following steps: the first step is to construct a multi-task initial model; the second step is to send the global initial parameters to each drone; the third step is to iteratively train and prune the global initial sub-model; the fourth step is to perform aggregation processing after uploading all the global pruning parameters to obtain the global aggregation parameters; the fifth step is to repeat the same operations as the second to fourth steps until the target loss function converges, send the global application parameters to each drone, and generate a multi-task application model corresponding to each drone. By constructing a global initial sub-model and a plurality of different personalized initial sub-models, the present application can effectively cope with different but interrelated tasks to be processed between drones, and while realizing knowledge sharing through the global initial sub-model, it also helps each drone complete specific diversified tasks through the personalized initial sub-model. In response to the frequent model parameter exchanges and bandwidth constraints of wireless spectrum resources faced by drone swarms, a model pruning strategy is provided to prune model parameters, effectively reducing the burden of model parameter exchange; and an optimized bandwidth allocation strategy is provided to greatly improve the transmission rate and utilization rate during model parameter exchange, reducing calculation and communication delays; ultimately, a high-precision, strong generalization capability multi-task application model is obtained to realize various tasks between each drone, with strong adaptability.

[0159] The above are only embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A multi-task modeling method for drone swarms, characterized in that: Includes steps: The first step is to obtain a plurality of different tasks to be processed of the drone group, and construct a multi-task initial model according to the tasks to be processed; wherein the multi-task initial model includes a global initial sub-model and a plurality of different personalized initial sub-models; The second step is to broadcast the global initial parameters in the global initial sub-model to each drone; The third step is to iteratively train and prune the global initial sub-model of each drone based on the global initial parameters, the local training data set of each drone, the personalized initial sub-model and the model pruning strategy to obtain the global pruning parameters corresponding to each drone; Step 4: After uploading all the global pruning parameters, perform aggregation processing to obtain global aggregation parameters; In the fifth step, the same operations as those in the second step to the fourth step are repeated until the target loss function converges, and the global application parameters are obtained. The global application parameters are broadcast and distributed to each of the drones, and a multi-task application model corresponding to each drone is generated respectively; wherein the multi-task application model includes a global application sub-model and a personalized application sub-model.

2. The multi-task modeling method of drone swarm according to claim 1, characterized in that: The third step specifically includes: Based on the global initial parameters and the local training data set of each of the drones, iteratively training the personalized initial sub-model corresponding to each of the drones to obtain a personalized training sub-model; According to the global initial parameters, the local training data set of each of the drones and the personalized training sub-model, iteratively training the global initial sub-model of each of the drones is performed to obtain a global training sub-model; The model pruning strategy is used to prune and iteratively train the global training sub-model of each UAV to obtain a global pruned sub-model, and the global pruning parameters in the global pruned sub-model corresponding to each UAV are extracted.

3. The multi-task modeling method of drone swarm according to claim 2, characterized in that: The method of using the model pruning strategy to perform pruning and iterative training on the global training sub-model of each drone includes: Based on the global initial parameters of the global initial sub-model and the global training parameters of the global training sub-model, a global parameter change amount is calculated; Calculate and obtain a pruning identification vector according to the model pruning rate and the global parameter change; Pruning the global training sub-model according to the pruning identification vector, and iteratively training the pruned global training sub-model; The model pruning strategy includes the global parameter change, the model pruning rate and the pruning identification vector.

4. The multi-task modeling method of drone swarm according to claim 3, characterized in that: The method for calculating the model pruning rate comprises the steps of: According to the computation and communication delay constraints of the drone swarm, the target loss function is transformed to calculate the value range of the model pruning rate; The convex optimization problem analysis method, the Lagrange multiplier method and the KKT condition analysis method are used in sequence for mathematical modeling, the transformed target loss function is solved, and the model pruning rate is determined according to the value range of the model pruning rate.

5. The multi-task modeling method of drone swarm according to claim 1, characterized in that: In the fourth step, the method for uploading the global pruning parameters includes the steps of: Based on the bandwidth allocation factor and the total bandwidth, allocating corresponding upload bandwidth to the global pruning parameters corresponding to each of the drones; Through an orthogonal frequency division multiple access transmission mode, all the global pruning parameters are uploaded respectively according to the corresponding upload bandwidth.

6. The multi-task modeling method of drone swarm according to claim 5, characterized in that: The method for calculating the bandwidth allocation factor comprises the steps of: Transforming the target loss function according to the computation and communication delay constraints and bandwidth constraints of the drone swarm; The convex optimization problem analysis method, the Lagrange multiplier method and the KKT condition analysis method are used in sequence to perform mathematical modeling, solve the converted target loss function, and determine the bandwidth allocation factor.

7. The multi-task modeling method of drone swarm according to claim 1, characterized in that: The method for constructing the target loss function includes: Obtaining a federated loss function, and constructing the federated loss function as an initial loss function based on the global initial sub-model, the personalized initial sub-model of each of the drones, and a local training data set; According to the computation and communication delay constraints, bandwidth constraints, and model pruning constraints of the drone swarm, the initial loss function is equivalently transformed to obtain the target loss function.

8. A multi-task modeling device for a drone swarm, characterized in that: include: An acquisition construction unit is used to acquire a plurality of different tasks to be processed of the drone group, and construct a multi-task initial model according to the tasks to be processed; wherein the multi-task initial model includes a global initial sub-model and a plurality of different personalized initial sub-models; A sending unit, used for broadcasting and sending the global initial parameters in the global initial sub-model to each drone; and also used for broadcasting and sending the global application parameters to each drone; A training and pruning unit, configured to iteratively train and prune the global initial sub-model of each drone based on the global initial parameters, the local training data set of each drone, the personalized initial sub-model and the model pruning strategy, to obtain global pruning parameters corresponding to each drone; An uploading unit, used for uploading all the global pruning parameters; an aggregation unit, configured to aggregate all the global pruning parameters to obtain a global aggregation parameter; A generating unit is used to generate a multi-task application model corresponding to each of the drones respectively; wherein the multi-task application model includes a global application sub-model and a personalized application sub-model.

9. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images