A multi-service flow-oriented dynamic collaborative inference method of a lightweight and complex neural network of a drone cluster

Abstract

The application discloses a kind of unmanned aerial vehicle cluster light and complex neural network dynamic collaborative inference methods for multi-service flow, comprising: each task unmanned aerial vehicle has a transmission link connected to ground station by multiple relay unmanned aerial vehicles;After each task unmanned aerial vehicle receives inference task request, it will select light CNN neural network or complex CNN neural network by model predictive control (MPC) method, if selecting light CNN neural network, execute task locally and transmit inference result to ground station through link, if selecting complex CNN neural network, according to available resources, by MADDPG-based multi-agent deep reinforcement learning method, network model is divided into multiple sub-models according to the structure of layer, and the sub-model is distributed to the unmanned aerial vehicle that meets the requirement to execute and transmit the inference result to ground station.The application can effectively improve the efficiency of light and complex neural network dynamic collaborative inference of unmanned aerial vehicle cluster for multi-service flow under the condition of ensuring task accuracy.

Description

Technical Field

[0001] This invention relates to the field of collaborative reasoning technology for unmanned aerial vehicle (UAV) swarms, and specifically to a lightweight and complex neural network dynamic collaborative reasoning method for UAV swarms with multiple service flows. Background Technology

[0002] Unmanned aerial vehicles (UAVs) have been widely used for missions such as military reconnaissance, environmental monitoring, rescue operations, and border patrols due to their flexibility and cost-effectiveness. Meanwhile, with the development of artificial intelligence and machine learning, utilizing complex machine learning algorithms to achieve these applications has become an inevitable trend. Deploying advanced CNN neural networks on UAV swarms can significantly enhance their perception, understanding, planning, and decision-making capabilities, enabling them to undertake more complex and demanding tasks. However, deploying advanced CNN neural networks on UAV swarms still faces many problems and challenges. As a resource-constrained platform, UAVs struggle to efficiently infer these CNN neural networks that require substantial computational and storage resources.

[0003] To address these issues, two traditional approaches exist. The first is model compression, which compresses complex models into lightweight models that drones can efficiently process. However, this approach inevitably sacrifices some accuracy. The second approach is Collaborative Edge Intelligence (CoEI), which primarily offloads a portion of the computational load of complex models to cloud or edge servers. While this approach maintains high accuracy, it results in significant communication latency. To address the shortcomings of these two methods, some works have proposed achieving a balance between latency and accuracy by dynamically switching between complex and lightweight CNN neural networks. For example, patent publication number CN116451576A discloses a dynamic collaborative inference system and method for UAV swarms based on switching between large and small models. The UAV swarm consists of multiple heterogeneous UAVs with computing capabilities. Each UAV includes a large model, a small model, an environment perception module, and an information feedback module deployed on it. The environment perception module is connected to the information feedback module, and the information feedback module is connected to the UAV. The UAV is connected to both the large and small models. The environment perception module is used to perceive environmental information. The information feedback module is used to receive the environmental information perceived by the environment perception module and is used for communication and data transmission with the UAVs. The large model is used to perform collaborative inference tasks based on the environmental information received by the information feedback module. The small model is used to perform individual inference tasks based on the environmental information received by the information feedback module. By switching between large and small models, the inference tasks can be completed more effectively. However, these tasks only target a single business flow, which limits the application scenarios to some extent. For example, drone swarms used to monitor a specific area often require multiple task drones equipped with sensors to work together to cover the entire area, and the business flow of each sensor needs to be processed in real time. Summary of the Invention

[0004] The purpose of this invention is to provide a lightweight and complex neural network dynamic collaborative reasoning method for UAV swarms with multiple service flows. By optimizing for multi-service flow scenarios, the application scenarios of the lightweight and complex neural network dynamic collaborative reasoning method for UAV swarms with multiple service flows are expanded, which has great practical significance.

[0005] To achieve the above-mentioned technical objectives, the technical solution adopted by the present invention is as follows:

[0006] A lightweight and complex neural network dynamic collaborative reasoning method for UAV swarms with multiple service flows, the dynamic collaborative reasoning method includes the following steps:

[0007] The system acquires information on the trajectory, memory, computing power, inference task distribution, and accuracy requirements of each business flow of the drone swarm. The drone swarm consists of N heterogeneous drones with computing capabilities. M of these drones, used to acquire input information, are defined as mission drones, and the rest are defined as relay drones. Both lightweight CNN neural networks and complex CNN neural networks are deployed on the mission drones, while only complex CNN neural networks are deployed on the relay drones.

[0008] Based on the constraints of UAV memory and mission accuracy, a dynamic collaborative reasoning problem model for UAV swarms oriented towards multiple service flows is established, consisting of a combination of lightweight and complex CNN neural networks. The dynamic collaborative reasoning problem model is divided into a first subproblem and a second subproblem. The first subproblem is the selection problem between lightweight and complex CNN neural networks, and the second subproblem is the segmentation problem of the complex CNN neural network when selecting it. The number of parameters of the lightweight CNN neural network is smaller than that of the complex CNN neural network.

[0009] Each inference task is obtained based on the inference task distribution information. Minimizing the inference latency of all tasks is taken as the optimization objective. Based on the computing power of the UAV swarm, the accuracy constraints of each service flow, and the communication rate derived from the trajectory of the UAV swarm, the first sub-problem is solved using model predictive control. If the solution to the first sub-problem is to select a lightweight CNN neural network, the task is executed locally and the inference result is transmitted to the ground station via a link. If the solution to the first sub-problem is to select a complex CNN neural network, the second sub-problem is solved using deep reinforcement learning. The network model is divided into multiple sub-models according to the layer structure. The sub-models are assigned to UAVs that meet the requirements for execution and the inference results are transmitted to the ground station.

[0010] As a preferred example, the process of solving the first subproblem using the model predictive control method includes the following steps:

[0011] A1, initialize the initial precision for each business flow;

[0012] A2. Determine if the current task is the last task. If not, proceed to step A3. Otherwise, return to the best model selection scheme for all tasks and end the process.

[0013] A3, select a scheme for generating the model of the [R,R+H-1]th task, including the current task, based on the precision constraints of each business flow;

[0014] A4, estimate the latency of all model selection schemes when the complex CNN neural network is split by layer average;

[0015] A5. Select the model selection scheme with the lowest latency among all model selection schemes as the best model selection scheme, and return to step A2.

[0016] As a preferred example, the process of solving the second sub-problem using deep reinforcement learning methods includes the following steps:

[0017] B1 defines each mission drone as an intelligent agent;

[0018] B2, obtains the current state based on the drone currently executing the task in each business flow and its computing power, the action performed by the agent in the previous step, the actions performed by other agents in the previous step, and the communication rate between drones;

[0019] B3, based on observations at each step of each business flow, determines the next action, i.e., the layer to be executed by the drone;

[0020] B4, initialize weights, network parameters, and replay experience pool;

[0021] B5, initialize the environment and obtain the initial state;

[0022] B6. Determine if all task flows are complete. If not, proceed to step B7; otherwise, end the training.

[0023] B7 obtains the probability of each action based on the action network, and uses an action mask to block illegal actions according to the action restrictions, and samples the executed actions using the Gumbel-Softmax method;

[0024] B8, each agent performs action A and obtains the corresponding reward and the next state;

[0025] B9 stores the current state, selected action, reward, and next state as a set of data in the experience replay pool, and randomly extracts multiple sets of data from the experience replay pool.

[0026] B10, Train and optimize the Critic network and Actor network based on multiple sets of training data extracted from the experience replay pool, and repeat the optimization of the Critic network and Actor network until the network converges; return to step B6.

[0027] As a preferred example, in step B10, the value loss function of the Critic network is:

[0028]

[0029] in It is a loss function. These are updated parameters. For the target Q value, As the attenuation factor, The target Q value is denoted as , and the target network is denoted as . The current Q value, Let j be the state of sample j. Let c be the actions of each agent in sample j.

[0030] As a preferred example, in step B10, the update objective of the Actor network is:

[0031]

[0032] in This indicates calculating the gradient. Update the target for the strategy. These are updated parameters. It is the gradient of the Actor network. It is the Critic network for actions The gradient.

[0033] As a preferred example, the number of parameters of the lightweight CNN neural network is greater than or equal to 10M, while the number of parameters of the complex CNN neural network is less than 10M.

[0034] As a preferred example, the communication rate is derived from the trajectory of the drone swarm using the following formula:

[0035]

[0036] In the formula, P represents the bandwidth between drone i and drone k; P represents the transmission power. The distance between the drones at time t is represented by α, which is the path loss exponent. These are channel coefficients that follow a complex normal distribution CN(0,1); It is additive white Gaussian noise.

[0037] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0038] The present invention provides a lightweight and complex neural network dynamic collaborative reasoning method for UAV swarms oriented to multiple service flows. Under the memory constraints of UAVs and given accuracy constraints, it can obtain a low-latency collaborative reasoning scheme for multiple service flows, thereby effectively improving the reasoning efficiency of the UAV swarm dynamic collaborative reasoning system. Attached Figure Description

[0039] Figure 1 This is a flowchart of a lightweight and complex neural network dynamic collaborative reasoning method for UAV swarms oriented towards multiple service flows, according to an embodiment of the present invention.

[0040] Figure 2This is a scenario diagram illustrating the dynamic switching between lightweight and complex neural networks for collaborative inference in a drone swarm, as described in this embodiment of the invention. Detailed Implementation

[0041] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0042] This invention provides a method for dynamic collaborative inference using lightweight and complex neural networks in a multi-service flow-oriented UAV swarm. It acquires information on the trajectory, memory, computing power, inference task distribution, and accuracy requirements of each service flow within the UAV swarm. The UAV swarm consists of N heterogeneous UAVs with computing capabilities, where M UAVs used to acquire input information are defined as mission UAVs, and the rest are relay UAVs. Based on UAV memory and mission accuracy constraints, a dynamic collaborative inference problem model for a multi-service flow-oriented UAV swarm using lightweight and complex neural networks is established, composed of a combination of lightweight and complex CNN neural networks. This model is further divided into a first sub-problem and a second sub-problem. The first sub-problem is the selection problem between lightweight and complex CNN neural networks, and the second sub-problem is the segmentation problem of the complex CNN neural network when selecting it. The optimization objective of this model is to minimize the inference latency of all tasks. Figure 1 As shown, firstly, multiple task drones acquire each inference task based on the inference task distribution information. Then, based on the computing power of the drone swarm, the accuracy constraints of each service flow, and the communication rate derived from the trajectory of the drone swarm, the model predictive control method is used to solve the first sub-problem to select either a lightweight CNN neural network or a complex CNN neural network. If the result of the first sub-problem is to select a complex CNN neural network, then the second sub-problem is solved using a deep reinforcement learning method based on MADDPG.

[0043] In this embodiment, the number of parameters of a lightweight CNN neural network is smaller than that of a complex CNN neural network. For example, it can be set as follows: the number of parameters of a complex CNN neural network is greater than or equal to 10M, and the number of parameters of a lightweight CNN neural network is less than 10M.

[0044] As a preferred example, the process of solving the first subproblem using the model predictive control method includes the following sub-steps:

[0045] A1 initializes the initial precision of each business flow, with the current task being R.

[0046] A2. Determine if the current task is the last task. If not, proceed to step A3; otherwise, return the best model selection scheme for all tasks.

[0047] A3. Based on the precision constraints of each business flow, select a scheme for the generation model of the [R,R+H-1]th task, including the current task.

[0048] A4 estimates the latency of all model selection schemes when the complex CNN neural network is split by layer on an average basis.

[0049] A5. Select the model selection scheme with the lowest latency among all model selection schemes as the best model selection scheme.

[0050] As a preferred example, the process of solving the second sub-problem using a MADDPG-based multi-agent deep reinforcement learning method includes the following sub-steps;

[0051] B1 defines each mission drone as an intelligent agent.

[0052] B2 obtains the current state based on the drone currently performing the task in each business flow, its computing power, the action performed by the agent in the previous step, the actions performed by other agents in the previous step, and the communication rate between drones.

[0053] B3 determines the next action, i.e., the layer to be executed by the drone, based on observations at each step of each business flow.

[0054] B4, initialize weights, network parameters, and replay experience pool.

[0055] B5 initializes the environment and obtains the initial state.

[0056] B6. Determine if all task flows are complete. If not, proceed to step B7; otherwise, end the training.

[0057] B7. Obtain the probability of each action from the action network, and use an action mask to block illegal actions according to the action restrictions, and sample the executed actions using the Gumbel-Softmax method; the Gumbel-Softmax method in step B7 makes the sampling process from the discrete distribution differentiable, thereby allowing the model parameters to be updated with gradients during backpropagation.

[0058] B8, each agent performs action A and obtains the corresponding reward and the next state.

[0059] B9 stores the current state, selected action, reward, and next state as a set of data in the experience replay pool, and randomly selects multiple sets of data from the experience replay pool.

[0060] B10: Train and optimize the Critic and Actor networks using multiple sets of training data extracted from the experience replay pool. Repeat the optimization of the Critic and Actor networks until the networks converge.

[0061] In step B10, the Critic network value loss function is as follows:

[0062]

[0063] in It is a loss function. These are updated parameters. For the target Q value, As the attenuation factor, The target Q value is denoted as , and the target network is denoted as . The current Q value, Let j be the state of sample j. Let c be the actions of each agent in sample j.

[0064] The Actor network update target is as follows:

[0065]

[0066] in This indicates calculating the gradient. Update the target for the strategy. These are updated parameters. It is the gradient of the Actor network. It is the Critic network for actions The gradient.

[0067] As a preferred example, the communication rate calculation process derived from the trajectory of the drone swarm is as follows:

[0068]

[0069] In the formula, P represents the bandwidth between drone i and drone k; P represents the transmission power. The distance between the drones at time t is represented by α, which is the path loss exponent. These are channel coefficients that follow a complex normal distribution CN(0,1); It is additive white Gaussian noise.

[0070] For example, such as Figure 2As shown, three UAVs acquire a reasoning task. First, the model selection scheme for this reasoning task is obtained by solving sub-problem one using model predictive control. This results in UAV 1 and UAV 2 executing a complex CNN neural network, while UAV 3 executes a lightweight CNN network. Then, UAV 1 and UAV 2 use the MADDPG-based multi-agent reinforcement learning model to infer the complex CNN neural network model segmentation scheme. Finally, each UAV performs reasoning based on the model selection scheme and model segmentation scheme obtained from the decision-making process.

[0071] The present invention provides a lightweight and complex neural network dynamic collaborative reasoning method for UAV swarms oriented to multiple service flows. Under the memory constraints of UAVs and given accuracy constraints, it can obtain a low-latency collaborative reasoning scheme for multiple service flows, thereby effectively improving the reasoning efficiency of the UAV swarm dynamic collaborative reasoning system.

[0072] For example, suppose we have three task drones and five relay drones. The accuracy constraints for the workflows of the three task drones are 75%, 80%, and 85%, respectively. We assume a lightweight model can achieve 70% accuracy, while a complex model with 16 layers can achieve 90% accuracy. Assume each workflow has five tasks. Then, the proposed solution can provide a model selection scheme and a complex model partitioning scheme based on the above conditions and the memory limitations of each drone.

[0073] For example, the proposed solution offers the following options for model selection for the three business flows:

[0074] Model selection scheme: Each row represents the model selection scheme for each of the five tasks in each task flow. 1 represents the selection of a complex CNN neural network, and 0 represents the selection of a lightweight CNN neural network. Furthermore, the model selection scheme represented by each row satisfies the accuracy constraints of its respective task flow. Taking task flow one as an example, tasks one, four, and five use lightweight CNN neural networks, while tasks two and three use complex CNN neural networks, with an average accuracy of (3*70%+2*90%) / 5=78%.

[0075] According to the model selection scheme, tasks with a value of 1 represent the selection of complex CNN neural networks. Therefore, it is necessary to perform model segmentation on complex CNN neural networks. Taking task flow one as an example, the second and third tasks require the use of complex models. The complex model segmentation scheme is as follows:

[0076] Task 2 Model Segmentation Scheme:

[0077] Task 3 Model Segmentation Scheme:

[0078] The first column represents the mission drone, and the remaining five columns represent the five relay drones, with the values ​​indicating the number of layers in the complex CNN neural network executed by each drone. Taking the model segmentation scheme for Mission 2 as an example, Mission Drone 1 executes three layers, then transmits the intermediate results to Relay Drone 1. Relay Drone 1 executes five layers, then transmits the intermediate results to Relay Drone 2. Relay Drone 2 executes four layers, then transmits the intermediate results to Relay Drone 3, and so on, until Relay Drone 5 obtains the inference result, which is then transmitted to the ground station. Simultaneously, the scheme satisfies the drone's memory constraints: the memory occupied by the three-layer model allocated to Mission Drone 1 is less than the maximum memory of Drone 1, the memory occupied by the five-layer model allocated to Relay Drone 1 is less than the maximum memory of Relay Drone 1, and so on for the other drones.

[0079] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0080] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0081] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0082] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment, causing a series of operational steps to be executed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that run on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0083] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0084] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A lightweight and complex neural network dynamic collaborative reasoning method for UAV swarms with multiple service flows, characterized in that, The dynamic collaborative reasoning method includes the following steps: The system acquires information on the trajectory, memory, computing power, inference task distribution, and accuracy requirements of each business flow of the drone swarm. The drone swarm consists of N heterogeneous drones with computing capabilities. M of these drones, used to acquire input information, are defined as mission drones, and the rest are defined as relay drones. Both lightweight CNN neural networks and complex CNN neural networks are deployed on the mission drones, while only complex CNN neural networks are deployed on the relay drones. Based on the constraints of UAV memory and mission accuracy, a dynamic collaborative reasoning problem model for UAV swarms oriented towards multiple service flows is established, consisting of a combination of lightweight and complex CNN neural networks. The dynamic collaborative reasoning problem model is divided into a first subproblem and a second subproblem. The first subproblem is the selection problem between lightweight and complex CNN neural networks, and the second subproblem is the segmentation problem of the complex CNN neural network when selecting it. The number of parameters of the lightweight CNN neural network is smaller than that of the complex CNN neural network. Each inference task is obtained based on the inference task distribution information. Minimizing the inference latency of all tasks is taken as the optimization objective. Based on the computing power of the UAV swarm, the accuracy constraints of each service flow, and the communication rate derived from the trajectory of the UAV swarm, the first sub-problem is solved using model predictive control. If the solution to the first sub-problem is to select a lightweight CNN neural network, the task is executed locally and the inference result is transmitted to the ground station via a link. If the solution to the first sub-problem is to select a complex CNN neural network, the second sub-problem is solved using deep reinforcement learning. The network model is divided into multiple sub-models according to the layer structure. The sub-models are assigned to UAVs that meet the requirements for execution and the inference results are transmitted to the ground station.

2. The lightweight and complex neural network dynamic collaborative reasoning method for UAV swarms oriented towards multiple service flows as described in claim 1, characterized in that, The process of solving the first subproblem using model predictive control methods includes the following steps: A1, initialize the initial precision of each business flow, the current task is R; A2. Determine if the current task is the last task. If not, proceed to step A3. Otherwise, return to the best model selection scheme for all tasks and end the process. A3, select a scheme for generating the model of the [R,R+H-1]th task, including the current task, based on the precision constraints of each business flow; A4, estimate the latency of all model selection schemes when the complex CNN neural network is split by layer average; A5. Select the model selection scheme with the lowest latency among all model selection schemes as the best model selection scheme, and return to step A2.

3. The lightweight and complex neural network dynamic collaborative reasoning method for UAV swarms oriented towards multiple service flows as described in claim 1, characterized in that, The process of solving the second subproblem using deep reinforcement learning methods includes the following steps: B1 defines each mission drone and relay drone as an intelligent agent; B2, obtains the current state based on the drone currently executing the task in each business flow and its computing power, the action performed by the agent in the previous step, the actions performed by other agents in the previous step, and the communication rate between drones; B3, based on observations at each step of each business flow, determines the next action, i.e., the layer to be executed by the drone; B4, initialize weights, network parameters, and replay experience pool; B5, initialize the environment and obtain the initial state; B6. Determine if all task flows are complete. If not, proceed to step B7; otherwise, end the training. B7 obtains the probability of each action based on the action network, and uses an action mask to block illegal actions according to the action restrictions, and samples the executed actions using the Gumbel-Softmax method; B8, each agent performs action A and obtains the corresponding reward and the next state; B9 stores the current state, selected action, reward, and next state as a set of data in the experience replay pool, and randomly extracts multiple sets of data from the experience replay pool. B10: Train and optimize the Critic network and Actor network based on multiple sets of training data extracted from the experience replay pool. Repeat the optimization of the Critic network and Actor network until the network converges; return to step B6.

4. The lightweight and complex neural network dynamic collaborative reasoning method for UAV swarms oriented towards multiple service flows as described in claim 3, characterized in that, In step B10, the value loss function of the Critic network is: ； in It is a loss function. These are updated parameters. For the target Q value, As the attenuation factor, The target Q value is denoted as , and the target network is denoted as . The current Q value, Let j be the state of sample j. Let c be the actions of each agent in sample j.

5. The lightweight and complex neural network dynamic collaborative reasoning method for UAV swarms oriented towards multiple service flows as described in claim 3, characterized in that, In step B10, the update objective of the Actor network is: ； in This indicates calculating the gradient. Update the target for the strategy. These are updated parameters. It is the gradient of the Actor network. It is the Critic network for actions The gradient.

6. The lightweight and complex neural network dynamic collaborative inference method for UAV swarms oriented towards multiple service flows as described in claim 1, characterized in that, The complex CNN neural network has a parameter count greater than or equal to 10M, while the lightweight CNN neural network has a parameter count less than 10M.

7. The lightweight and complex neural network dynamic collaborative inference method for UAV swarms oriented towards multiple service flows as described in claim 1, characterized in that, The communication rate is derived from the trajectory of the drone swarm using the following formula: ； In the formula, P represents the bandwidth between drone i and drone k; P represents the transmission power. The distance between the drones at time t is represented by α, which is the path loss exponent. These are channel coefficients that follow a complex normal distribution CN(0,1); It is additive white Gaussian noise.

Images