Unmanned aerial vehicle cluster lightweight and complex neural network dynamic collaborative reasoning method for multiple service flows

Abstract

The invention discloses an unmanned aerial vehicle cluster lightweight and complex neural network dynamic collaborative reasoning method for multiple service flows. The method comprises the steps that each task unmanned aerial vehicle is provided with a transmission link connected with a ground station through a plurality of relay unmanned aerial vehicles; after each task unmanned aerial vehicle receives a reasoning task request, a lightweight CNN neural network or a complex CNN neural network is selected through a model predictive control (MPC) method, if the lightweight CNN neural network is selected, a task is executed locally, and a reasoning result is transmitted to a ground station through a link, and if the complex CNN neural network is selected, the task is executed locally. And if yes, dividing the network model into a plurality of sub-models according to a layer structure through a multi-agent deep reinforcement learning method based on MADDPG according to the available resources, allocating the sub-models to unmanned aerial vehicles meeting requirements for execution, and transmitting a reasoning result to a ground station. According to the invention, the efficiency of multi-service-flow-oriented unmanned aerial vehicle cluster lightweight and complex neural network dynamic cooperative reasoning can be effectively improved under the condition that the task precision is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of collaborative reasoning of drone clusters, and in particular to a lightweight and complex neural network dynamic collaborative reasoning method for drone clusters oriented to multiple business flows. Background Art

[0002] Drones have been widely used to perform tasks such as military reconnaissance, environmental monitoring, rescue operations, and border patrols due to their flexibility and cost-saving. At the same time, with the development of artificial intelligence and machine learning, it has become an inevitable trend to use complex machine learning algorithms to achieve the above applications. Deploying advanced CNN neural networks on drone clusters can significantly enhance the perception, understanding, planning, and decision-making capabilities of drones, enabling them to undertake more complex and demanding tasks. However, there are still many problems and challenges in deploying advanced CNN neural networks on drone clusters. As a resource-constrained platform, drones have difficulty in efficiently inferring these CNN neural networks that require a lot of computing and storage resources.

[0003] To solve the above problems, there are currently two traditional methods. The first method is model compression, which compresses complex models into lightweight models that drones can process efficiently. However, this method inevitably sacrifices the accuracy of the model to some extent. The second method is collaborative edge intelligence (CoEI), which mainly offloads part of the computational load of complex models to the cloud or edge servers. Although this method can maintain high accuracy, it will cause significant communication delays. In order to solve the shortcomings of these two methods, some work has proposed to achieve a balance between latency and accuracy by dynamically switching between complex CNN neural networks and lightweight CNN neural networks. For example, the invention with patent publication number CN116451576A discloses a dynamic collaborative reasoning system and method for drone clusters based on large and small model switching. The drone cluster is composed of multiple heterogeneous drones with computing capabilities. Each drone includes: a large model, a small model, an environment perception module and an information feedback module deployed on each drone. The environment perception module is connected to the information feedback module, the information feedback module is connected to the drone, and the drone is connected to the large model and the small model respectively; the environment perception module is used to perceive environmental information; the information feedback module is used to receive the environmental information perceived by the environment perception module, and is used for communication and data transmission with the drone; the large model is used to perform drone collaborative reasoning tasks based on the environmental information received by the information feedback module; the small model is used to perform drone single-machine reasoning tasks based on the environmental information received by the information feedback module, and the reasoning tasks are better completed by switching between large and small models. However, these works only target a single business flow, which limits the application scenarios to a certain extent. For example, a drone swarm used to monitor a specific area often requires multiple mission drones equipped with sensors to collaboratively cover the entire area, and the business flow of each sensor needs to be processed in real time. Summary of the invention

[0004] The purpose of the present invention is to provide a method for dynamic collaborative reasoning of lightweight and complex neural networks of drone clusters for multiple business flows. By optimizing the scenarios for multiple business flows, the application scenarios of the method for dynamic collaborative reasoning of lightweight and complex neural networks of drone clusters for multiple business flows are expanded, which has great practical significance.

[0005] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:

[0006] A method for dynamic collaborative reasoning of lightweight and complex neural networks of drone clusters for multiple business flows, the method comprising the following steps:

[0007] Obtain the trajectory, memory, computing power, inference task distribution information and accuracy requirements of each business flow of the drone cluster; the drone cluster consists of N heterogeneous drones with computing power, M of which are used to obtain input information are defined as mission drones, and the rest are defined as relay drones; lightweight CNN neural networks and complex CNN neural networks are deployed on the mission drones at the same time, and only complex CNN neural networks are deployed on the relay drones;

[0008] According to the constraints of drone memory and mission accuracy, a dynamic collaborative reasoning problem model of lightweight and complex neural networks for drone clusters facing multiple business flows, which is composed of a combination of lightweight CNN neural networks and complex CNN neural networks, is established; the dynamic collaborative reasoning problem model is divided into the first sub-problem and the second sub-problem, where the first sub-problem is the selection problem of lightweight CNN neural networks and complex CNN neural networks, and the second sub-problem is the segmentation problem of complex CNN neural networks when selecting complex CNN neural networks; the number of parameters of lightweight CNN neural networks is greater than that of complex CNN neural networks;

[0009] Each inference task is obtained according to the inference task distribution information, and minimizing the inference delay of all tasks is taken as the optimization goal. Based on the computing power of the drone cluster, the accuracy constraint of each business flow, and the communication rate obtained from the trajectory of the drone cluster, the first sub-problem is solved by the model predictive control method; when the solution result of the first sub-problem is to select a lightweight CNN neural network, the task is executed locally and the inference result is transmitted to the ground station through the link; when the solution result of the first sub-problem is to select a complex CNN neural network, the second sub-problem is solved by the deep reinforcement learning method, the network model is divided into multiple sub-models according to the layer structure, the sub-models are assigned to drones that meet the requirements for execution and the inference results are transmitted to the ground station.

[0010] As a preferred example, the process of solving the first sub-problem by the model predictive control method includes the following steps:

[0011] A1, initialize the initial accuracy of each service flow;

[0012] A2, determine whether the current task is the last task. If not, jump to step A3 for execution; otherwise, return to the best model selection solution for all tasks and end the process;

[0013] A3, generate a model selection scheme for the [R, R+H-1]th task including the current task according to the accuracy constraint of each business flow;

[0014] A4, estimates the latency of all model selection schemes when a complex CNN neural network is evenly split by layers;

[0015] A5: Select the model selection scheme with the lowest delay among all model selection schemes as the best model selection scheme, and return to step A2.

[0016] As a preferred example, the process of solving the second sub-problem by deep reinforcement learning method includes the following steps:

[0017] B1, define each mission drone as an intelligent agent;

[0018] B2, obtain the current state based on the drone currently executing the task in each business flow and its computing power, the action performed by the agent in the previous step, the actions performed by other agents in the previous step, and the communication rate between drones;

[0019] B3, based on the observation of each step of each business flow, the next action, i.e., the number of layers executed by the drone, is obtained;

[0020] B4, initialize weights, network parameters and replay experience pool;

[0021] B5, initialize the environment and obtain the initial state;

[0022] B6, determine whether all task flows are completed. If not, jump to step B7 for execution; otherwise, end the training;

[0023] B7, obtain the probability of each action according to the action network, and use the action mask to mask illegal actions according to the action restrictions and sample the executed actions through the Gumbel-Softmax method;

[0024] B8, each agent performs action A and obtains the corresponding reward and next state;

[0025] B9, stores the current state, selected action, reward and next state as a set of data in the experience replay pool, and randomly extracts multiple sets of data from the experience replay pool;

[0026] B10, training and optimizing the Critic network and the Actor network according to multiple sets of training data extracted from the experience replay pool, repeatedly optimizing the Critic network and the Actor network until the network converges; return to step B6.

[0027] As a preferred example, in step B10, the value loss function of the Critic network is:

[0028]

[0029] in is the loss function, φ i is the updated parameter, is the target Q value, γ is the attenuation factor, Qi ′ is the target Q value, and is the target network is the current Q value, S j is the state of sample j, are the actions of c agents in sample j.

[0030] As a preferred example, in step B10, the update target of the Actor network is:

[0031]

[0032] in represents the gradient, J is the strategy update target, θ i is the updated parameter, is the gradient of the Actor network, It is the Critic network for action A i gradient.

[0033] As a preferred example, the parameter amount of the lightweight CNN neural network is greater than or equal to 10M, and the parameter amount of the complex CNN neural network is less than 10M.

[0034] As a preferred example, the following formula is used to derive the communication rate from the trajectory of the drone cluster:

[0035]

[0036] In the formula, B i,k represents the bandwidth between UAV i and UAV k; P represents the transmission power; represents the distance between drones at time t; α is the path loss exponent; g0 is the channel coefficient following the complex normal distribution CN(0,1); N0 is the additive Gaussian white noise.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The lightweight and complex neural network dynamic collaborative reasoning method for drone clusters with multiple business flows of the present invention can obtain a collaborative reasoning solution with low latency for multiple business flows according to given accuracy constraints under the memory constraints of the drone, thereby effectively improving the reasoning efficiency of the dynamic collaborative reasoning system of the drone cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flow chart of a method for dynamic collaborative reasoning of lightweight and complex neural networks of drone clusters for multiple business flows according to an embodiment of the present invention;

[0040] Figure 2 A scenario diagram for collaborative reasoning of lightweight and complex neural networks dynamically switching between the drone clusters of an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings.

[0042] An embodiment of the present invention provides a method for dynamic collaborative reasoning of lightweight and complex neural networks for drone clusters for multiple business flows, which obtains the trajectory, memory, computing power, reasoning task distribution information of the drone cluster and the accuracy requirements of each business flow; the drone cluster is composed of N heterogeneous drones with computing capabilities, of which M drones used to obtain input information are defined as task drones, and the rest are relay drones; based on the drone memory and task accuracy constraints, a problem model for dynamic collaborative reasoning of lightweight and complex neural networks for drone clusters for multiple business flows is established, which is composed of a combination of lightweight CNN neural networks and complex CNN neural networks, and the model is further divided into a first sub-problem and a second sub-problem, wherein the first sub-problem is the selection problem between lightweight CNN neural networks and complex CNN neural networks, and the second sub-problem is the segmentation problem of complex CNN neural networks when selecting complex CNN neural networks. The optimization goal of this model is to minimize the reasoning delay of all tasks; such as Figure 1 As shown, first, multiple task drones obtain each reasoning task according to the reasoning task distribution information, and solve the first sub-problem through the model predictive control method to select a lightweight CNN neural network or a complex CNN neural network according to the computing power of the drone cluster, the accuracy constraint of each business flow, and the communication rate obtained from the trajectory of the drone cluster. If the result of the first sub-problem is to select a complex CNN neural network, the second sub-problem is solved by a deep reinforcement learning method based on MADDPG.

[0043] In this embodiment, the parameter amount of the lightweight CNN neural network is greater than that of the complex CNN neural network. For example, it can be set as follows: the parameter amount of the lightweight CNN neural network is greater than or equal to 10M, and the parameter amount of the complex CNN neural network is less than 10M.

[0044] As a preferred example, the process of solving the first sub-problem using the model predictive control method includes the following sub-steps:

[0045] A1, initialize the initial accuracy of each business flow, the current task is R.

[0046] A2, determine whether the current task is the last task, if not, jump to step A3 to execute; otherwise, return the best model selection solution for all tasks.

[0047] A3, based on the accuracy constraints of each business flow, generates a model selection scheme for the [R, R+H-1]th task including the current task.

[0048] A4, estimates the latency of all model selection schemes when a complex CNN neural network is evenly split by layers.

[0049] A5: Select the model selection scheme with the lowest delay among all model selection schemes as the best model selection scheme.

[0050] As a preferred example, the process of solving the second sub-problem using a multi-agent deep reinforcement learning method based on MADDPG includes the following sub-steps:

[0051] B1, define each mission drone as an intelligent agent.

[0052] B2, obtain the current state based on the drone currently executing the task in each business flow and its computing power, the action performed by the agent in the previous step, the actions performed by other agents in the previous step, and the communication rate between drones.

[0053] B3, based on the observation of each step of each business flow, the next action, i.e., the number of layers executed by the drone, is obtained.

[0054] B4, initialize weights, network parameters and replay experience pool.

[0055] B5, initialize the environment and obtain the initial state.

[0056] B6, determine whether all task flows are completed. If not, jump to step B7 for execution; otherwise, end the training.

[0057] B7, obtain the probability of each action according to the action network, use the action mask to mask illegal actions according to the action restrictions, and sample the executed actions through the Gumbel-Softmax method; the Gumbel-Softmax method in step B7 makes the process of sampling from a discrete distribution differentiable, thereby allowing the model parameters to be updated with gradients during back propagation.

[0058] B8, each agent performs action A and obtains the corresponding reward and next state.

[0059] B9, stores the current state, selected action, reward and next state as a set of data in the experience replay pool, and randomly extracts multiple sets of data from the experience replay pool.

[0060] B10, train and optimize the Critic network and the Actor network based on multiple sets of training data extracted from the experience replay pool, and repeatedly optimize the Critic network and the Actor network until the network converges.

[0061] In step B10, the Critic network value loss function is as follows:

[0062]

[0063] in is the loss function, φ i is the updated parameter, is the target Q value, γ is the attenuation factor, Q i ′ is the target Q value, and is the target network is the current Q value, S j is the state of sample j, are the actions of c agents in sample j.

[0064] The Actor network update targets are as follows:

[0065]

[0066] in represents the gradient, J is the strategy update target, θ i is the updated parameter, is the gradient of the Actor network, It is the Critic network for action A i gradient.

[0067] As a preferred example, the communication rate calculation process obtained from the trajectory of the drone cluster is as follows:

[0068]

[0069] In the formula, B i,k represents the bandwidth between UAV i and UAV k; P represents the transmission power; represents the distance between drones at time t; α is the path loss exponent; g0 is the channel coefficient following the complex normal distribution CN(0,1); N0 is the additive Gaussian white noise.

[0070] For example, Figure 2 As shown in the figure, three mission drones have obtained a reasoning task. First, the model prediction control method is used to solve subproblem 1 to obtain the model selection scheme of the reasoning task. Mission drone 1 and mission drone 2 execute complex CNN neural network, and mission drone 3 executes lightweight CNN network. Then, mission drone 1 and mission drone 2 infer the trained multi-agent reinforcement learning model based on MADDPG to obtain the complex CNN neural network model segmentation scheme. After that, each drone performs reasoning based on the model selection scheme and model segmentation scheme obtained by decision making.

[0071] The lightweight and complex neural network dynamic collaborative reasoning method for drone clusters with multiple business flows of the present invention can obtain a collaborative reasoning solution with low latency for multiple business flows according to given accuracy constraints under the memory constraints of the drone, thereby effectively improving the reasoning efficiency of the dynamic collaborative reasoning system of the drone cluster.

[0072] For example, suppose there are three mission drones and five relay drones, where the accuracy constraints of the business flows of the three mission drones are 75%, 80%, and 85% respectively. Assuming that the lightweight model can achieve 70% accuracy, the complex model has 16 layers and can achieve 90% accuracy. If each business flow has 5 tasks, then the proposed scheme can give a model selection scheme and a complex model segmentation scheme based on the above conditions and the memory limit of each drone.

[0073] For example, the proposed scheme provides the following solutions for model selection of three business flows:

[0074] Model selection scheme: Each row represents the model selection scheme for each task flow for 5 tasks, 1 represents the selection of complex CNN neural network, and 0 represents the selection of lightweight CNN neural network. And the model selection scheme represented by each row meets the accuracy constraints of its business flow. Taking business flow 1 as an example, its tasks 1, 4, and 5 use lightweight CNN neural network, and tasks 2 and 3 use complex CNN neural network. The average accuracy is: (3*70%+2*90%) / 5=78%.

[0075] According to the model selection scheme, tasks with a value of 1 all represent the selection of complex CNN neural networks. Therefore, it is necessary to segment the model of the complex CNN neural network. Taking task flow 1 as an example, the second and third tasks need to use complex models, so the complex model segmentation scheme is:

[0076] Task 2 model segmentation scheme: [3 5 4 0 3 1]

[0077] Task 3 model segmentation scheme: [6 2 0 0 5 3]

[0078] The first column represents the task drone, and the remaining five columns represent the five relay drones in turn. The values ​​represent the number of layers of the complex CNN neural network executed by each drone. Taking the model segmentation scheme of Task 2 as an example, Task drone 1 executes three layers, and then transmits the intermediate results to relay drone 1. Relay drone 1 executes five layers, and then transmits the intermediate results to relay drone 2. Relay drone 2 executes four layers, and then transmits the intermediate results to relay drone 3. This is executed in sequence until relay drone 5 obtains the inference result, and then transmits the inference result to the ground station. At the same time, the scheme meets the memory constraints of the drones, that is, the memory occupied by the three-layer model allocated to task drone 1 is less than the maximum memory of drone 1, the memory occupied by the five-layer model allocated to relay drone 1 is less than the maximum memory of relay drone 1, and so on for other drones.

[0079] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0080] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0081] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0082] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0083] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0084] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A lightweight and complex neural network dynamic collaborative reasoning method for drone clusters with multiple business flows, characterized by: The dynamic collaborative reasoning method comprises the following steps: Obtain the trajectory, memory, computing power, inference task distribution information and accuracy requirements of each business flow of the drone cluster; the drone cluster consists of N heterogeneous drones with computing power, M of which are used to obtain input information are defined as mission drones, and the rest are defined as relay drones; lightweight CNN neural networks and complex CNN neural networks are deployed on the mission drones at the same time, and only complex CNN neural networks are deployed on the relay drones; According to the constraints of drone memory and mission accuracy, a dynamic collaborative reasoning problem model of lightweight and complex neural networks for drone clusters facing multiple business flows, which is composed of a combination of lightweight CNN neural networks and complex CNN neural networks, is established; the dynamic collaborative reasoning problem model is divided into the first sub-problem and the second sub-problem, where the first sub-problem is the selection problem of lightweight CNN neural networks and complex CNN neural networks, and the second sub-problem is the segmentation problem of complex CNN neural networks when selecting complex CNN neural networks; the number of parameters of lightweight CNN neural networks is greater than that of complex CNN neural networks; Each inference task is obtained according to the inference task distribution information, and minimizing the inference delay of all tasks is taken as the optimization goal. Based on the computing power of the drone cluster, the accuracy constraint of each business flow, and the communication rate obtained from the trajectory of the drone cluster, the first sub-problem is solved by the model predictive control method; when the solution result of the first sub-problem is to select a lightweight CNN neural network, the task is executed locally and the inference result is transmitted to the ground station through the link; when the solution result of the first sub-problem is to select a complex CNN neural network, the second sub-problem is solved by the deep reinforcement learning method, the network model is divided into multiple sub-models according to the layer structure, the sub-models are assigned to drones that meet the requirements for execution and the inference results are transmitted to the ground station.

2. The method for dynamic collaborative reasoning of lightweight and complex neural networks for multi-service flows of drone clusters according to claim 1 is characterized in that: The process of solving the first sub-problem by the model predictive control method includes the following steps: A1, initialize the initial accuracy of each service flow; A2, determine whether the current task is the last task. If not, jump to step A3 for execution; otherwise, return to the best model selection solution for all tasks and end the process; A3, generate a model selection scheme for the [R, R+H-1]th task including the current task according to the accuracy constraint of each business flow; A4, estimates the latency of all model selection schemes when a complex CNN neural network is evenly split by layers; A5: Select the model selection scheme with the lowest delay among all model selection schemes as the best model selection scheme, and return to step A2.

3. The method for dynamic collaborative reasoning of lightweight and complex neural networks for multi-service flows of drone clusters according to claim 1 is characterized in that: The process of solving the second sub-problem by deep reinforcement learning method includes the following steps: B1, define each mission drone as an intelligent agent; B2, obtain the current state based on the drone currently executing the task in each business flow and its computing power, the action performed by the agent in the previous step, the actions performed by other agents in the previous step, and the communication rate between drones; B3, based on the observation of each step of each business flow, the next action, i.e., the number of layers executed by the drone, is obtained; B4, initialize weights, network parameters and replay experience pool; B5, initialize the environment and obtain the initial state; B6, determine whether all task flows are completed. If not, jump to step B7 for execution; otherwise, end the training; B7, obtain the probability of each action according to the action network, and use the action mask to mask illegal actions according to the action restrictions and sample the executed actions through the Gumbel-Softmax method; B8, each agent performs action A and obtains the corresponding reward and next state; B9, stores the current state, selected action, reward and next state as a set of data in the experience replay pool, and randomly extracts multiple sets of data from the experience replay pool; B10, training and optimizing the Critic network and the Actor network according to multiple sets of training data extracted from the experience replay pool, repeatedly optimizing the Critic network and the Actor network until the network converges; return to step B6.

4. The method for dynamic collaborative reasoning of lightweight and complex neural networks for multi-service flows of drone clusters according to claim 3 is characterized in that: In step B10, the value loss function of the Critic network is: in is the loss function, φ i is the updated parameter, is the target Q value, γ is the attenuation factor, Q i ′ is the target Q value, and is the target network is the current Q value, s k is the state of sample j, are the actions of c agents in sample j.

5. The method for dynamic collaborative reasoning of lightweight and complex neural networks for multi-service flows of drone clusters according to claim 3 is characterized in that: In step B10, the update target of the Actor network is: in represents the gradient, J is the strategy update target, θ i is the updated parameter, is the gradient of the Actor network, It is the Critic network for action A i gradient.

6. The method for dynamic collaborative reasoning of lightweight and complex neural networks for multi-service flows of drone clusters according to claim 1 is characterized in that: The parameter amount of the lightweight CNN neural network is greater than or equal to 10M, and the parameter amount of the complex CNN neural network is less than 10M.

7. The method for dynamic collaborative reasoning of lightweight and complex neural networks for multi-service flows of drone clusters according to claim 1 is characterized in that: The communication rate is derived from the trajectory of the drone cluster using the following formula: In the formula, B i,k represents the bandwidth between UAV i and UAV k; P represents the transmission power; represents the distance between drones at time t; α is the path loss exponent; g0 is the channel coefficient following the complex normal distribution CN(0,1); N0 is the additive Gaussian white noise.

Images