Air-ground dynamic collaborative reasoning method and device for low-altitude Internet of Things

Abstract

The invention discloses an air-ground dynamic collaborative reasoning method and device for a low-altitude Internet of Things. The method comprises the steps that a trained small lightweight deep neural network model and a trained large complex deep neural network model are deployed on a single unmanned aerial vehicle and a ground station respectively; the method comprises the following steps: taking an unmanned aerial vehicle as an air node, taking a ground station as a ground node, and after receiving a reasoning task request, carrying out statistical evaluation on own state and available resources by the unmanned aerial vehicle, and sensing the channel condition of air-ground transmission by the unmanned aerial vehicle; and selecting whether the unmanned aerial vehicle only uses the small lightweight deep neural network model to perform reasoning independently, if not, dividing the reasoning task into two sub-tasks according to the layer structure and the layer number of the large complex deep neural network model, and allocating the sub-tasks to the unmanned aerial vehicle and the ground station for common execution. According to the method, a flexible and efficient adaptive reasoning strategy can be provided for air-ground cooperative calculation, and the calculation efficiency of the low-altitude Internet of Things is greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle (UAV) air-ground collaborative reasoning technology, and in particular to an air-ground dynamic collaborative reasoning method and device for a low-altitude intelligent network. Background Art

[0002] The development needs of the low-altitude economy have led to the emergence of the low-altitude intelligent network, which is an important infrastructure for realizing the intelligent interconnection of "man-machine-object" in the low-altitude airspace (3km / 1km). It can help realize the physical network space of networking, digitalization and intelligent operation of low-altitude business. As a key low-altitude new infrastructure, the low-altitude intelligent network is an essential basic information network facility to support the development of the low-altitude economy.

[0003] Drones are often used as the main application carriers of low-altitude intelligent networks due to their flexibility, real-time data collection, and ability to access hard-to-reach areas. Equipped with sensors such as cameras, infrared sensors, and GPS (Global Positioning System), drones are ideal for performing low-altitude missions in challenging environments. In order to handle some complex tasks at low altitudes, it is essential to use accurate and efficient deep neural network (DNN) models to process airborne sensory data in real time. However, since DNN tasks are usually computationally intensive, it is challenging to perform complex DNN inference on drones with limited onboard resources. Although it is feasible to transmit large amounts of airborne sensory data directly to resource-rich ground, this may result in significant transmission delays and high bandwidth consumption given the limited and historical air-ground communication resources.

[0004] In order to solve the inevitable conflict between the high resource requirements of DNNs and the limited onboard resources, and to improve the computational efficiency in low-altitude intelligent networks, existing work mainly has the following two methods: one method is to compress large and complex DNNs into small and lightweight DNNs through pruning, quantization, and knowledge distillation techniques; in this way, the compressed model can be executed directly on a single drone, thereby reducing execution latency at the expense of significant accuracy loss. Another method is to use drones and ground stations to collaboratively execute complex DNN models to ensure high accuracy, while the air-to-ground communications involved may still introduce relatively large inference delays, especially considering the dynamic air-to-ground communications caused by drone motion. In summary, neither of the above two methods can maintain high accuracy and low latency at the same time.

[0005] Inventions with publication numbers CN116451576A, CN117669741A, and CN115934310A disclose task reasoning methods based on the collaboration of complex neural network models and lightweight neural network models of drone swarms. However, these studies focus on drone swarms and rarely involve air-ground collaborative computing.

[0006] The invention with publication number CN118865148A discloses a target recognition method for satellite Internet, including a satellite Internet end-edge-cloud collaborative target recognition system architecture, a drone cluster collaborative recognition method, and a satellite-assisted edge collaborative recognition method. Among them, the end-edge-cloud collaborative target recognition architecture is divided into three parts: terminal layer, satellite edge layer, and ground cloud layer. The drone cluster collaborative recognition method improves the collaborative reasoning performance in terms of delay and energy consumption. When selecting a satellite, the satellite-assisted edge collaborative recognition method selects the satellite with the maximum elevation angle to access the terminal according to the maximum elevation angle criterion. The invention can meet the needs of video analysis in unmanned areas, and can effectively utilize the communication and computing resources of drones and satellites while maintaining detection accuracy. However, the focus of the invention is to use the resources of satellites, drones, and ground stations for collaborative target recognition, emphasizing the selection of satellites with the maximum elevation angle to access the terminal to improve the collaborative efficiency of target recognition, and does not involve the efficient utilization of computing resources during air-ground collaborative reasoning of low-altitude intelligent networks. Summary of the invention

[0007] The purpose of the present invention is to disclose a method and device for dynamic air-ground collaborative reasoning for a low-altitude intelligent network, which can provide a flexible and efficient adaptive reasoning strategy for air-ground collaborative computing, greatly improving the computing efficiency of the low-altitude intelligent network; drones equipped with the dynamic collaborative reasoning method can not only realize fast and efficient air-ground collaborative computing in the low-altitude intelligent network, but also are expected to provide more computing solutions for various industry applications in the low-altitude economy.

[0008] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:

[0009] In a first aspect, an air-ground dynamic collaborative reasoning method for a low-altitude intelligent network is provided, the method comprising the following steps:

[0010] Deploy a small and lightweight deep neural network model and a large and complex deep neural network model on a single drone and ground station respectively;

[0011] The UAV is regarded as an aerial node and the ground station as a ground node. After receiving the inference task request, the UAV evaluates its own status and available computing resources, selects the time to execute the next task based on the time when the task arrives and the current task is completed, and substitutes the distance from the UAV to the ground station to calculate the channel transmission rate, thus completing the perception of the channel conditions for air-to-ground transmission.

[0012] Combining the statistical evaluation results and the perception results, the reasoning tasks are divided into the number of model layers and the dynamic task flow, and the optimization decision is obtained through the deep reinforcement learning algorithm search. The total task delay of the entire task flow under different decision sets under the channel conditions at the current location is evaluated; the reward value corresponding to the total delay under different value conditions is used to train the deep reinforcement learning DQN network, and finally make it converge. The participating node types and the allocated model calculation amount are adjusted according to the decision set of all tasks obtained; if local UAV reasoning is selected, the small and lightweight deep neural network model pre-deployed locally on the UAV will perform reasoning alone. If collaborative processing with the ground station is selected, the reasoning task will be divided into two subtasks according to the number of layers according to the decision, and the subtasks will be assigned to the UAV and the ground station for joint execution.

[0013] Furthermore, when a small and lightweight deep neural network model is selected to perform task reasoning alone, the sum of the computational delay and transmission delay of the drone performing the reasoning task is calculated according to the following formula:

[0014]

[0015] Among them, t small In the case where the UAV performs the reasoning task alone, the UAV that performs n tasks receives the information from the 1st layer to the Lth layer. smax The sum of the delays of the layer reasoning tasks, smax Layer is the maximum number of layers for small lightweight models; C sj The amount of computation required for the UAV to perform the j-th layer reasoning task; P a is the computing power of the drone, k=1,2,…,n.

[0016] Furthermore, when the UAV and the ground station are selected to jointly perform the reasoning task, the target layer of the deep learning model is the smallest subtask execution unit, and the sum of the computational delay and transmission delay of the UAV performing the target layer reasoning task is calculated according to the following formula:

[0017]

[0018] Among them, t large In the case of air-ground collaborative execution of large and complex model reasoning, the UAV executes the first β k The delay of the layer reasoning task is related to the ground station receiving the βth k The sum of the output data delays of the +1 layer, the βth k Layer is the target layer; C lj The amount of computation required for the UAV to perform the j-th layer reasoning task; P a The computing power of the drone; lj To implement the β k The data size of the layer reasoning task output; ρ gis the signal transmission rate of the drone; and They represent the computational delay of the UAV to perform the inference task and the transmission delay of the ground station to receive the input data. g In the formula, B g It is expressed as the transmission bandwidth of the air-ground channel; D is the distance between the UAV and the ground station, α′ is the path loss exponent; g0 is the channel gain; N0 is the noise power.

[0019] Furthermore, the process of adjusting the participating node types and the allocated model computing amount according to the obtained decision set of all tasks includes the following steps:

[0020] Design two decision variables: α and β; α is used to decide whether to perform task reasoning alone, and β is used to decide the model layer of the split point; when a small and lightweight deep neural network model is selected to perform task reasoning alone, α=0, β=0; when the UAV and the ground station are selected to perform the reasoning task together, α=1, β≠0, and β represents the model layer of the split point; the value set of the two decision variables (α, β) is used to represent the result of the system's reasoning decision in a dynamic environment;

[0021] The drone status is perceived in real time. For the arrival time of different tasks in the task flow, the task delay of different decision sets (α, β) is evaluated under the channel conditions of the current position by substituting the position of the drone, evaluating the arrival and completion of the previous task, and considering the limitations of memory and precision. The DQN network of deep reinforcement learning is trained by the reward value corresponding to the total delay under different value conditions, and finally converged. The participating node type α and the allocated model calculation amount β are adjusted according to the decision set of all tasks, and low-latency collaborative reasoning is performed under precision constraints.

[0022] Furthermore, the method further comprises:

[0023] For the arrival of n tasks, the total system delay for reasoning is denoted as t = {t1, t2, ..., t k ..., t n}; For the kth task, the task completion time T k , task arrival time F k and calculate the delay C k It is expressed as the following relationship:

[0024]

[0025] Taking minimizing the total delay of multiple task executions as the optimization goal, the objective function of the optimization problem is defined as:

[0026]

[0027] Among them, α k and β k Denotes the two decision variables for the kth task, the total task delay t k It is calculated by the calculation delay, task arrival time and calculation delay;

[0028] Build constraints:

[0029]

[0030] In the formula, A k is the accuracy of the kth task, A set is the minimum threshold of accuracy for all tasks; m j is the memory requirement of the j-th layer inference task, m a Is the amount of memory available to the system.

[0031] In a second aspect, the present invention discloses an air-ground dynamic collaborative reasoning device for a low-altitude intelligent network, the device comprising:

[0032] The model deployment module is used to deploy a small and lightweight deep neural network model and a large and complex deep neural network model trained on a single drone and a ground station respectively;

[0033] The strategy evaluation module is used to detect and collect low-altitude inference tasks, and after receiving the push task request, it evaluates the system status, available resources, and channel conditions of air-to-ground transmission;

[0034] The task decision module selects the model reasoning method based on the evaluation results. The model reasoning methods include the UAV using a small and lightweight deep neural network model to perform reasoning alone, and the task is assigned to the UAV and the ground station to perform together.

[0035] The task execution module executes the task reasoning process according to the selected model reasoning method.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] First, the air-ground dynamic collaborative reasoning method and device for the low-altitude intelligent network of the present invention are different from the traditional data return reasoning mode and the single-machine local reasoning mode. The method proposed in the present invention improves the task execution efficiency by deploying a deep learning model in the air-ground system and through air-ground collaborative reasoning.

[0038] Second, the air-ground dynamic collaborative reasoning method and device for the low-altitude intelligent network of the present invention is based on the real-time perception of the UAV and task processing status. Through the deep reinforcement learning algorithm, the participating node types and the allocated model layer calculation amount are adjusted, and the optimization decision is executed under the precision limit. Ultimately, low-latency collaborative reasoning is realized, so that the system can complete the reasoning task more efficiently. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A flow chart of an air-ground dynamic collaborative reasoning method for a low-altitude intelligent network provided by an embodiment of the present invention;

[0040] Figure 2 An air-ground dynamic collaborative reasoning scene diagram for a low-altitude intelligent network provided by an embodiment of the present invention;

[0041] Figure 3 A performance comparison result diagram of algorithms on devices with different computing capabilities when using the AlexNet and VGG16 models provided in an embodiment of the present invention;

[0042] Figure 4 A performance comparison result diagram of algorithms on devices with different computing capabilities when using the AlexNet and ResNet152 models provided in an embodiment of the present invention;

[0043] Figure 5 A comparison chart of algorithm performance under different task numbers when using the AlexNet and VGG16 models provided in an embodiment of the present invention;

[0044] Figure 6 A comparison chart of algorithm performance under different task numbers when using the AlexNet and ResNet152 models provided in an embodiment of the present invention;

[0045] Figure 7 A comparison chart of algorithm performance under different precision limits when using the AlexNet and VGG16 models provided in an embodiment of the present invention;

[0046] Figure 8 A comparison chart of algorithm performance under different precision limits when using the AlexNet and ResNet152 models provided in an embodiment of the present invention;

[0047] Fig. 9 A structural diagram of an air-ground dynamic collaborative reasoning system for a low-altitude intelligent network provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings.

[0049] In one embodiment, if Figure 1 As shown, an embodiment of the present invention provides an air-ground dynamic collaborative reasoning method for a low-altitude intelligent network, including:

[0050] Step 101, deploy the same trained deep learning models, one large and one small, on the UAV and ground station in the air-ground system respectively.

[0051] like Figure 2 As shown, an embodiment of the present invention consists of a drone and a ground station. The drone handles onboard computations, while the ground station supports more complex tasks, both of which perform reasoning using DNN models. The system supports two model types: for small, lightweight tasks that require lower accuracy, and for large, complex models for higher accuracy but with increased latency. We pre-deploy a small model and a large model on the drone. The small model runs independently on the drone, while the large model is partitioned and collaboratively executed by the drone and the ground station based on mission requirements.

[0052] Step 102, after receiving the reasoning task request, the UAV evaluates the system status, available resources, and channel conditions of air-to-ground transmission.

[0053] Specifically, the UAV is used as an aerial node and the ground station as a ground node. After receiving an inference task request, the UAV evaluates its own status and available computing resources, and selects the time to execute the next task based on the time when the task arrives and the current task is completed. At the same time, the distance from the UAV to the ground station is substituted to calculate the channel transmission rate, thus completing the perception of the channel conditions for air-to-ground transmission.

[0054] Step 103, based on the evaluation results, it is selected whether the UAV uses a small lightweight deep neural network model to perform reasoning alone, or whether the task is assigned to the UAV and the ground station to perform together. In step 103, the deep learning model is a deep neural network, including an input layer, a hidden layer, and an output layer; the layer in this embodiment refers to the layer structure in the deep neural network.

[0055] Combining the statistical evaluation results and the perception results, the reasoning tasks are divided into the number of model layers and the dynamic task flow, and the optimization decision is obtained through the deep reinforcement learning algorithm search. The total task delay of the entire task flow under different decision sets under the channel conditions at the current location is evaluated; the reward value corresponding to the total delay under different value conditions is used to train the DQN network of deep reinforcement learning, and finally make it converge. According to the decision set of all tasks obtained in this way, the participating node types and the allocated model calculation amount are adjusted; if local reasoning of the drone is selected, there is no need to involve model division. If collaborative processing with the ground station is required, the reasoning task is divided into two subtasks according to the number of layers according to the decision, and the subtasks are assigned to the drone and the ground station for joint execution.

[0056] For ease of expression, the present invention designs two decision variables: α and β. Among them, when a small and lightweight deep neural network model is selected to perform task reasoning alone, α=0, and the reasoning of a large and complex model is not involved at this time, so β ​​also takes a value of 0; when the drone and the ground station are selected to perform the reasoning task together, α=1, at this time β≠0, and β represents the layer of the model at the split point. The value set (α, β) of these two decision variables is used to represent the result of the system's reasoning decision in a dynamic environment.

[0057] The drone status is perceived in real time. For the arrival time of different tasks in the task flow, the task delay of different decision sets (α, β) under the channel conditions of the current position is evaluated by substituting the position of the drone, evaluating the arrival and completion of the previous task, and considering the limitations of memory and accuracy. The DQN network of deep reinforcement learning is trained by the reward value corresponding to the total delay under different values, and finally converged. According to the decision set of all tasks obtained in this way, the participating node type (α) and the allocated model calculation amount (β) are adjusted to perform low-latency collaborative reasoning under accuracy constraints.

[0058] Specifically, for the arrival of n tasks, the total system delay for reasoning is expressed as t = {t1, t2, ..., t k ..., t n}; For the kth task, the task completion time T k , task arrival time F k and calculate the delay C k It is expressed as the following relationship:

[0059]

[0060] The objective function of the optimization problem is defined as:

[0061]

[0062] This formula represents the objective function in the optimization problem, which aims to minimize the total delay of multiple task executions. k It is calculated by combining the computation delay, task arrival time and computation delay. The goal of this formula is to minimize this delay in all tasks while making efficient decisions based on the requirements of the tasks. The constraints are constructed as follows:

[0063] The first constraint:

[0064] The second constraint:

[0065] The first constraint ensures that the overall accuracy of the system remains above a certain threshold. Specifically, the accuracy of all tasks A k Sum and average the results, and enforce that the sum meets the minimum acceptance value A. set , thereby ensuring that the accuracy of the system does not drop excessively due to delay optimization.

[0066] The second constraint ensures that the system has enough memory to handle the computational requirements of all possible model inference scenarios. The memory requirement for all inference tasks is m j The sum cannot exceed the system's available memory capacity m a .

[0067] The first case: When a small lightweight model is selected to perform task reasoning alone, the drone performs the reasoning task alone. The sum of the computational delay and transmission delay of the drone performing the reasoning task is calculated according to the following formula:

[0068]

[0069] Among them, t small In the case where the UAV performs the reasoning task alone, it performs n tasks (k is the number of tasks) and receives the data from the 1st layer to the Lth layer. smax The sum of the delays of the layer reasoning tasks, smax Layer is the maximum number of layers for small lightweight models; C sj The amount of computation required for the UAV to perform the j-th layer reasoning task; ρ a is the computing power of the drone (the number of addition and multiplication operations per second).

[0070] The second case: When the accuracy of the model reasoning performed by the UAV alone cannot meet the requirements, the reasoning task needs to be divided into two subtasks according to the layer structure and number of layers of the deep learning model, and the subtasks are assigned to the UAV and the ground station for joint execution, including:

[0071] When the target layer of the deep learning model is the smallest subtask execution unit, the sum of the computational delay and transmission delay of the drone performing the target layer reasoning task is calculated according to the following formula:

[0072]

[0073] Among them, t large In the case of air-ground collaborative execution of large and complex model reasoning, the UAV executes the first β k The delay of the layer reasoning task is related to the ground station receiving the βth k The sum of the output data delays of the +1 layer, the βth k Layer is the target layer; C lj The amount of computation required for the UAV to perform the j-th layer reasoning task; P a is the computing power of the drone (number of addition and multiplication operations per second); lj To implement the β k The data size of the layer reasoning task output; ρ g is the signal transmission rate of the drone; and They represent the computational delay of the UAV to perform the inference task and the transmission delay of the ground station to receive the input data. g In the formula, B g It is expressed as the transmission bandwidth of the air-to-ground channel; D is the distance between the UAV and the ground station, α is the path loss index; g0 is the channel gain; N0 is the noise power. In this embodiment, only one of the first and second situations will occur, that is, the first and second situations will not occur at the same time.

[0074] Figure 3 , Figure 4 This is a performance comparison result diagram of an embodiment of the present invention on devices with different computing capabilities and different algorithms, where the horizontal axis is the device type and the vertical axis is the total inference latency. Figure 3 , Figure 4 Three benchmark algorithms are used to perform inference tasks in an air-ground system involving three drones with different computing capabilities. First, the inference task of AlexNet and VGG-16 is performed in collaboration, and then AlexNet and ResNet152 are performed in collaboration. Figure 3 It can be seen that despite the different complexities and computational workloads of the three neural network models, the embodiments of the present invention consistently achieve the lowest latency compared to the four benchmark algorithms, especially when using Raspberry Pi 3B+, where the latency is reduced by 34.19% and 29.03% compared to the other algorithms, highlighting the benefits of higher computing power. Figure 4 The total processing time of tasks on the three devices is Figure 2Similarly, the large model VGG16 always takes more total time than ResNet152 to process the same number of tasks, mainly due to its larger number of layers and more complex architecture.

[0075] Figure 5 , Figure 6 This is a comparison chart of algorithm performance under different task numbers provided by the embodiment of the present invention. In order to ensure the consistency of experimental conditions, the requirements for equipment and accuracy are consistent to ensure that all algorithms are uniformly evaluated. Figure 5 In Figure 2, results for a scenario using the highly complex model VGG16 are shown. Figure 6 Results are given for a scenario where the complex model is ResNet152. In both cases, as the number of tasks increases, the total task latency also increases; however, the latency of the embodiment of the present invention remains stable and is the lowest among all algorithms. Specifically, Figure 5 In the example, compared with the three benchmark algorithms, the total delay of the embodiment of the present invention is reduced by 42.21%, 57.46% and 63.65% respectively. Figure 6 Among them, the declines were 19.3%, 39.63% and 30.3% respectively.

[0076] Figure 7 , Figure 8 A comparison chart of algorithm performance under different precision limits provided by embodiments of the present invention. The results show that higher precision requirements lead to increased task latency due to the need for more complex models. Due to the small number of tasks, the latency optimization effect of GC is slightly more obvious. In general, embodiments of the present invention consistently identify favorable decisions at different precision levels while maintaining stable processing efficiency.

[0077] Step 104, executing the task reasoning process according to the selected model reasoning method.

[0078] The UAV air-ground collaborative reasoning method based on adaptive switching of lightweight and complex models proposed in the embodiment of the present invention selects the model reasoning mode by evaluating the resources and communication status of the UAV and the ground station, and selects the lightweight model for separate reasoning or divides the high-precision complex neural network model for collaborative reasoning with the ground station, thereby reducing the high processing delay of the complex model and the insufficient precision of the lightweight model, and also improving the efficiency of the system in performing reasoning tasks. The method provided in the embodiment of the present invention also uses a deep reinforcement learning algorithm, which can quickly collect the optimal solution, so that the air-ground system can dynamically select the appropriate strategy to perform computing tasks according to its own status and application requirements, thereby achieving high-energy-efficiency and low-latency collaborative reasoning.

[0079] Based on the same inventive concept, an embodiment of the present invention also provides an air-ground dynamic collaborative reasoning method system for low-altitude intelligent network. Since the principle of solving the problem by this system is similar to that of the air-ground dynamic collaborative reasoning method for low-altitude intelligent network, the implementation of this system can refer to the implementation of the UAV air-ground collaborative reasoning method based on adaptive switching of lightweight and complex models, and the repeated parts will not be repeated.

[0080] In another embodiment, the air-ground dynamic collaborative reasoning method system for low-altitude intelligent network provided by the embodiment of the present invention is as follows: Fig. 9 As shown, including:

[0081] A model deployment module 10 is used to deploy the same two trained specified deep learning models on the UAV and the ground station in the air-ground system;

[0082] Strategy evaluation module 20, the drone is used as a task source to detect and collect low-altitude reasoning tasks. After receiving the push task request, the drone evaluates the system status, available resources, and channel conditions of air-to-ground transmission;

[0083] The task decision module 30 selects, based on the evaluation result, whether the UAV uses a small and lightweight deep neural network model to perform reasoning alone, or whether the task is assigned to the UAV and the ground station to perform together;

[0084] The task execution module 40 executes the task reasoning process according to the selected model reasoning method.

[0085] The task division module includes:

[0086] The first case is used when a small lightweight model is selected to perform task reasoning alone, and the UAV performs the reasoning task alone. The sum of the computational delay and transmission delay of the UAV performing the reasoning task is calculated according to the following formula:

[0087]

[0088] Among them, t small In the case where the UAV performs the reasoning task alone, the UAV receives the data from layer 1 to layer L. smax The sum of the delays of the layer reasoning tasks, smax Layer is the maximum number of layers for small lightweight models; C sj The amount of computation required for the UAV to perform the j-th layer reasoning task; P a is the computing power of the drone (the number of addition and multiplication operations per second).

[0089] The second case is when the accuracy of the model reasoning performed by the UAV alone cannot meet the requirements. In this case, the reasoning task needs to be divided into two subtasks according to the layer structure and number of layers of the deep learning model, and the subtasks are assigned to the UAV and the ground station for joint execution, including:

[0090] When the target layer of the deep learning model is the smallest subtask execution unit, the sum of the computational delay and transmission delay of the drone performing the target layer reasoning task is calculated according to the following formula:

[0091]

[0092] Among them, t large In the case of air-ground collaborative execution of large and complex model reasoning, the UAV executes the first β k The delay of the layer reasoning task is related to the ground station receiving the βth k The sum of the output data delays of the +1 layer, the βth k Layer is the target layer; C lj The amount of computation required for the UAV to perform the j-th layer reasoning task; P a is the computing power of the drone (number of addition and multiplication operations per second); lj To implement the β k The data size of the layer reasoning task output; ρ g is the signal transmission rate of the drone; and They represent the computational delay of the UAV performing the reasoning task and the transmission delay of the ground station receiving the input data. In this embodiment, only one of the first and second situations will occur, that is, the first and second situations will not occur at the same time.

[0093] For more specific working processes of the above modules, please refer to the corresponding contents disclosed in the above embodiments, which will not be repeated here.

[0094] In another embodiment, the present invention provides a computer device, including a processor and a memory; wherein, when the processor executes a computer program stored in the memory, the steps of the above-mentioned drone swarm collaborative reasoning method based on model segmentation are implemented.

[0095] For more specific processes of the above method, please refer to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0096] In another embodiment, the present invention provides a computer-readable storage medium for storing a computer program; when the computer program is executed by a processor, the steps of the above-mentioned air-ground dynamic collaborative reasoning method for the low-altitude intelligent network are implemented.

[0097] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0098] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0099] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0100] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0101] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0102] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for dynamic air-ground collaborative reasoning for low-altitude intelligent networking, characterized in that: The method comprises the following steps: Deploy a small and lightweight deep neural network model and a large and complex deep neural network model on a single drone and ground station respectively; The UAV is regarded as an aerial node and the ground station as a ground node. After receiving the inference task request, the UAV evaluates its own status and available computing resources, selects the time to execute the next task based on the time when the task arrives and the current task is completed, and substitutes the distance from the UAV to the ground station to calculate the channel transmission rate, thus completing the perception of the channel conditions for air-to-ground transmission. Combining the statistical evaluation results and the perception results, the reasoning tasks are divided into the number of model layers and the dynamic task flow, and the optimization decision is obtained through the deep reinforcement learning algorithm search. The total task delay of the entire task flow under different decision sets under the channel conditions at the current location is evaluated; the reward value corresponding to the total delay under different value conditions is used to train the deep reinforcement learning DQN network, and finally make it converge. The participating node types and the allocated model calculation amount are adjusted according to the decision set of all tasks obtained; if local UAV reasoning is selected, the small and lightweight deep neural network model pre-deployed locally on the UAV will perform reasoning alone. If collaborative processing with the ground station is selected, the reasoning task will be divided into two subtasks according to the number of layers according to the decision, and the subtasks will be assigned to the UAV and the ground station for joint execution.

2. The air-ground dynamic collaborative reasoning method for low-altitude intelligent network according to claim 1 is characterized in that: When a small and lightweight deep neural network model is selected to perform task reasoning alone, the sum of the computational delay and transmission delay of the drone performing the reasoning task is calculated according to the following formula: Among them, t small In the case where the UAV performs the reasoning task alone, the UAV that performs n tasks receives the information from the 1st layer to the Lth layer. smax The sum of the delays of the layer reasoning tasks, smax Layer is the maximum number of layers for small lightweight models; C sj The amount of computation required for the UAV to perform the j-th layer reasoning task; P a is the computing power of the drone, k=1,2,…,n.

3. The air-ground dynamic collaborative reasoning method for low-altitude intelligent network according to claim 1 is characterized in that: When the UAV and the ground station are selected to jointly perform the inference task, the target layer of the deep learning model is the smallest subtask execution unit. The sum of the computational delay and transmission delay of the UAV performing the target layer inference task is calculated according to the following formula: Among them, t large In the case of air-ground collaborative execution of large and complex model reasoning, the UAV executes the first β k The delay of the layer reasoning task is related to the ground station receiving the βth k The sum of the output data delays of the +1 layer, the βth k Layer is the target layer; C lj The amount of computation required for the UAV to perform the j-th layer reasoning task; P a The computing power of the drone; lj To implement the β k The data size of the layer reasoning task output; ρ g is the signal transmission rate of the drone; and They represent the computational delay of the UAV to perform the inference task and the transmission delay of the ground station to receive the input data. g In the formula, B g It is expressed as the transmission bandwidth of the air-ground channel; D is the distance between the UAV and the ground station, α' is the path loss exponent; g0 is the channel gain; N0 is the noise power.

4. The air-ground dynamic collaborative reasoning method for low-altitude intelligent network according to claim 1 is characterized in that: The process of adjusting the participating node types and the allocated model computation amount according to the obtained decision set of all tasks includes the following steps: Design two decision variables: α and β; α is used to decide whether to perform task reasoning alone, and β is used to decide the model layer of the split point; when a small and lightweight deep neural network model is selected to perform task reasoning alone, α=0, β=0; when the UAV and the ground station are selected to perform the reasoning task together, α=1, β≠0, and β represents the model layer of the split point; the value set of the two decision variables (α, β) is used to represent the result of the system's reasoning decision in a dynamic environment; The drone status is perceived in real time. For the arrival time of different tasks in the task flow, the task delay of different decision sets (α, β) is evaluated under the channel conditions of the current position by substituting the position of the drone, evaluating the arrival and completion of the previous task, and considering the limitations of memory and precision. The DQN network of deep reinforcement learning is trained by the reward value corresponding to the total delay under different value conditions, and finally converged. The participating node type α and the allocated model calculation amount β are adjusted according to the decision set of all tasks, and low-latency collaborative reasoning is performed under precision constraints.

5. The air-ground dynamic collaborative reasoning method for low-altitude intelligent network according to claim 4 is characterized in that: The method further comprises: For the arrival of n tasks, the total system delay for reasoning is denoted as t = {t1, t2, ..., t k ..., t n }; For the kth task, the task completion time T k , task arrival time F k and calculate the delay C k It is expressed as the following relationship: The optimization goal is to minimize the total delay of all tasks in the entire task flow, and define the objective function of the optimization problem as: Among them, α k and β k Denotes the two decision variables for the kth task, the total task delay t k It is calculated by the calculation delay, task arrival time and calculation delay; Build constraints: In the formula, A k is the accuracy of the kth task, A set is the minimum threshold of accuracy for all tasks; m j is the memory requirement of the j-th layer inference task, m a Is the amount of memory available to the system.

6. An air-ground dynamic collaborative reasoning device for low-altitude intelligent networking based on any one of claims 1-5, characterized in that: The system comprises: The model deployment module is used to deploy a small and lightweight deep neural network model and a large and complex deep neural network model trained on a single drone and a ground station respectively; The strategy evaluation module is used to detect and collect low-altitude inference tasks, and after receiving the push task request, it evaluates the system status, available resources, and channel conditions of air-to-ground transmission; The task decision module selects the model reasoning method based on the evaluation results. The model reasoning methods include the UAV using a small and lightweight deep neural network model to perform reasoning alone, and the task is assigned to the UAV and the ground station to perform together. The task execution module executes the task reasoning process according to the selected model reasoning method.

7. A computer device comprising a processor and a memory; characterized in that: When the processor executes the computer program stored in the memory, the steps of the drone swarm collaborative reasoning method based on model segmentation as described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium for storing a computer program; characterized in that: When the computer program is executed by a processor, the steps of the air-ground dynamic collaborative reasoning method for low-altitude intelligent network as described in any one of claims 1 to 5 are implemented.

Images