Method for optimizing unmanned aerial vehicle cooperative reasoning and time delay control based on lyapunov theory

Abstract

The application belongs to the technical field of unmanned aerial vehicle cluster control, and particularly relates to an unmanned aerial vehicle cooperative reasoning and time delay control optimization method based on Lyapunov theory, which converts long-term average constraints into queue stability problems, designs suitable reward functions and algorithm architectures, and realizes effective balance among delay, energy consumption and system stability. First, an instant cost function is defined, and long-term average delay and energy consumption constraints are converted into instant delay and energy consumption of each time slot. Then, three queue models are constructed to describe the operation characteristics of the system: an actual task queue, a virtual delay queue and a virtual energy consumption queue. Finally, online optimization is carried out based on the Lyapunov algorithm, so as to minimize instant delay and energy consumption while ensuring system stability.

Description

Technical Field

[0001] This invention belongs to the field of intelligent collaborative computing and deep neural network inference optimization for UAV swarms. Specifically, it relates to a real-time scheduling method for collaborative DNN (Deep Neural Network) inference of UAV swarms based on Lyapunov optimization and PPO (Proximal Policy Optimization) reinforcement learning. It is applicable to real-time task inference scheduling in multi-UAV systems with heterogeneous resources, weak communication environment and dynamic task flow. Background Technology

[0002] With the integration of deep learning technology and UAV swarm applications, multiple UAVs are increasingly undertaking complex visual recognition and inference computation tasks in missions such as intelligent sensing, search and rescue, environmental monitoring, and aerial security. Due to the limited resources of the UAVs themselves, single-UAV inference modes are prone to high latency and high energy consumption in complex mission scenarios. Therefore, multi-UAV collaborative distributed DNN inference is widely considered a feasible solution. By having different UAVs share the inference layers of the deep neural network, task balance can be achieved and overall latency reduced.

[0003] Most existing distributed inference methods rely on static hierarchical allocation strategies or centralized scheduling centers. Static hierarchical methods typically pre-divide the DNN model into fixed layers and allocate them to different nodes, lacking the ability to adapt to dynamic changes in tasks and topology, and struggling to cope with the heterogeneity of UAV resources and the suddenness of tasks. Centralized scheduling relies on a central node for global optimization; however, in weak communication environments, the central node can easily become a bottleneck, and the system has poor robustness.

[0004] In recent years, distributed reinforcement learning methods have been gradually applied to UAV mission scheduling, enabling task migration and adaptive resource allocation through autonomous learning. However, traditional reinforcement learning methods lack theoretical constraints on the long-term stability of the system (such as latency stability and controllable energy consumption), and their training results may show unstable resource queues, leading to UAV swarms getting stuck in task backlog or even failing under high-load scenarios.

[0005] On the other hand, Lyapunov optimization, as a theoretical framework for solving long-term constrained optimization problems, offers clear convergence guarantees in terms of task stability and energy consumption control. However, it requires integration with a decision-making strategy mechanism driven by scheduling. Using Lyapunov optimization alone to solve dynamic multi-UAV inference problems presents the challenge of parsing highly coupled states. Therefore, how to integrate Lyapunov stability constraints with deep reinforcement learning strategy optimization to construct a theoretically stable UAV swarm scheduling strategy that supports heterogeneous resources and dynamic task inference is a core problem that current technology needs to address.

[0006] To address the following issues in existing UAV swarm collaborative DNN inference methods: Static allocation lacks adaptability: Traditional fixed inference layer partitioning methods cannot perform real-time scheduling based on the drone's computing power, energy consumption, and dynamic topology changes; Centralized scheduling is susceptible to communication limitations: scheduling modes that rely on a central node experience performance degradation and poor system robustness in weak communication environments. Reinforcement learning methods lack stability constraints: Existing reinforcement learning scheduling strategies are unable to ensure the stability of the task queue, which may lead to a sharp increase in inference latency or excessive energy consumption. It is difficult to achieve both latency and energy consumption optimization: traditional methods often prioritize inference completion speed while ignoring energy consumption control, making it impossible to operate stably for a long time in resource-constrained networks. Summary of the Invention

[0007] Purpose of the invention: This invention aims to provide an optimization method for UAV cooperative inference and latency control based on Lyapunov theory. This method is a real-time scheduling method for UAV swarm cooperative DNN inference that combines Lyapunov optimization and PPO reinforcement learning. It can achieve reduced inference latency, energy consumption control and queue stability guarantee in dynamic environments.

[0008] Technical solution: An optimization method for UAV cooperative reasoning and time delay control based on Lyapunov theory, the implementation steps of which include: (1) Construct a collaborative reasoning system model for unmanned aerial vehicle (UAV) swarms, including building each UAV into an intelligent agent with computing power, storage capacity, communication bandwidth and remaining energy, dividing the DNN task into layers, and supporting multi-UAV segmented execution; (2) Constructing the actual task queue and virtual queue model: the actual inference layer residual queue A virtual delay queue is constructed based on the average delay threshold and average energy consumption constraints. With virtual energy consumption queue ; The actual task queue is used to characterize the system's real-time load status. Long-term instability indicates that the system is overloaded and unable to complete the task within the available time. Virtual delay queue This is used to transform long-term average latency into a queue stability problem, constraining the average task latency to not exceed a preset threshold. Virtual energy consumption queue This is used to transform long-term average energy consumption into a queue stability problem, constraining energy consumption per unit time to not exceed a set average value; (3) Based on the Lyapunov drift-penalty function, construct the scheduling optimization objective and transform the long-term average delay and energy consumption constraints in step (2) into a time-slot-level solvable problem; (4) Construct a state space that includes local drones and neighboring drone resources and queue states; (5) Define the action as either executing the DNN layer locally or migrating to the action space of a specified neighboring UAV; (6) Construct a reward function based on the Lyapunov drift term, instantaneous delay term, and instantaneous energy consumption term; (7) The scheduling policy network is trained using the PPO reinforcement learning algorithm; (8) Perform online task scheduling based on the real-time system status to complete the collaborative DNN inference of the UAV swarm.

[0009] Furthermore, the actual task queue in step (2) is used to record the number of inference layers waiting to be processed, the virtual delay queue is used to constrain the average task delay to not exceed a preset threshold, and the virtual energy consumption queue is used to constrain the energy consumption per unit time to not exceed a set average value.

[0010] Furthermore, the Lyapunov drift-penalty function in step (3) is constructed as follows: The Lyapunov function is defined as half the sum of the squares of all task queues, virtual delay queues, and virtual energy queues, and the optimization objective is to construct a weighted average of the drift term for the instantaneous delay and energy term.

[0011] Furthermore, in step (4), the state space is fused in matrix form with the computing power, storage capacity, communication bandwidth, remaining energy, remaining inference layers of the task, and the delay cost and energy consumption cost of the current time slot of the local UAV and several neighboring UAVs.

[0012] Furthermore, the action space described in step (5) includes the following decision results: actions This indicates that the task will be executed locally. , This indicates that the task will be migrated to the next stage. A neighbor's drone.

[0013] Furthermore, the reward function in step (6) is a negative weighted combination, which includes an instant delay term, an instant energy consumption term, and a Lyapunov drift term.

[0014] Furthermore, in step (7), the PPO algorithm is used to update the policy network, and the shear ratio is used to constrain the policy update range to improve training stability.

[0015] Furthermore, in step (8), an online reasoning mechanism is adopted to adaptively generate scheduling decisions when the drone swarm changes dynamically, thereby realizing multi-hop hierarchical collaborative reasoning.

[0016] On the other hand, this invention provides a UAV swarm cooperative reasoning scheduling system based on Lyapunov optimization and PPO reinforcement learning. This system is used to execute the optimization method for UAV cooperative reasoning and delay control based on Lyapunov theory as described above. The system includes: The system modeling module is used to abstract the UAV swarm into a multi-agent system and to hierarchically classify the DNN tasks; The queue management module is used to maintain the actual task queue, the virtual delay queue, and the virtual energy consumption queue. Optimize the target module to construct the Lyapunov drift-penalty function; The State and Action module is used to generate the state space and action space. The policy training module is used to train scheduling policies based on the PPO algorithm and Lyapunov constraints. The online execution module is used to dynamically execute tasks locally or make migration decisions and update queue status. It supports multi-hop hierarchical collaborative reasoning and maintains system stability in environments with weak communication, dynamic topology and heterogeneous resources.

[0017] Beneficial effects: Compared with the prior art, the present invention has the following substantial features and significant progress: 1. This invention realizes distributed adaptive scheduling of UAV swarms without the need for centralized control nodes or static model partitioning: each UAV acts as an independent intelligent agent, relying only on local and neighbor observation information to make real-time decisions on task migration and load balancing, effectively coping with communication interruptions, node failures and dynamic topology changes, and significantly improving system robustness (it can still operate stably in intermittent communication environments). 2. This invention provides a strict theoretical guarantee of global queue stability through the Lyapunov drift-penalty mechanism: it transforms long-term average latency and energy consumption constraints into real-time queue control, avoiding task backlog and latency accumulation. Even in the context of sudden high-concurrency task flows and heterogeneous resource environments, it can still ensure long-term stable operation of the system, solving the problem of queue instability in traditional reinforcement learning methods. 3. This invention integrates PPO reinforcement learning to achieve efficient policy search and dynamic task transfer: it supports single-machine multi-layer continuous execution and multi-hop collaborative inference. The policy network incorporates the loss function through shear ratio constraints and Lyapunov drift terms to achieve fast convergence and stable optimization, which is significantly better than single-agent PPO and greedy / random benchmark methods. 4. This invention supports dual-objective optimization of inference latency and energy consumption, and is suitable for energy-constrained task environments; 5. This invention can cope with the heterogeneous computing capabilities and dynamic topology changes of UAVs, and has high adaptability and scalability. It can adapt to UAV nodes with different computing capabilities, memory, bandwidth and remaining energy, and supports seamless expansion from small rescue clusters to large-scale aerial intelligent platforms. 6. This invention has extremely high engineering feasibility and practical value, and is suitable for deployment in intelligent sensing tasks of UAV swarms in environments with scarce infrastructure or extreme conditions, such as emergency rescue, disaster monitoring, smart agriculture, security patrol, and border reconnaissance. It provides a new option for the practical application of edge intelligence in weak communication scenarios with low latency and high robustness. Attached Figure Description

[0018] Figure 1 This is the implementation architecture of the algorithm corresponding to the method described in this invention; Figure 2 This refers to the throughput of the four algorithms in the example on an 8-layer AlexNet task; Figure 3 This represents the throughput of four algorithms on a 16-layer VGGNet task. Figure 4 This is the average latency of the four algorithms on four UAVs and an 8-layer AlexNet task; Figure 5 This is the average latency of the four algorithms on six UAVs and an eight-layer AlexNet task; Figure 6 This is the average latency of the four algorithms on eight UAVs and an eight-layer AlexNet task; Figure 7 This represents the average latency of the four algorithms across four UAVs and a 16-layer VGGNet task. Figure 8 This represents the average latency of the four algorithms across six UAVs and a 16-layer VGGNet task. Figure 9 This represents the average latency of the four algorithms on a task involving eight UAVs and a 16-layer VGGNet. Figure 10 This is the average energy consumption of the four algorithms on an 8-layer AlexNet task using four drones; Figure 11 This is the average energy consumption of the four algorithms on an 8-layer AlexNet task with six drones; Figure 12 This is the average energy consumption of the four algorithms on an 8-layer AlexNet task using 8 drones; Figure 13 This represents the average energy consumption of the four algorithms on a 16-layer VGGNet task using four drones. Figure 14 This is the average energy consumption of the four algorithms on a 16-layer VGGNet task with six drones; Figure 15 This represents the average energy consumption of four algorithms on a 16-layer VGGNet task using eight drones. Detailed Implementation

[0019] To illustrate the technical solution provided by this invention in detail, further description is provided below with reference to the accompanying drawings.

[0020] Combination Figure 1 The algorithm framework shown in this invention mainly includes the following implementation steps: (1) System Model Construction: This invention abstracts the UAV swarm into a multi-agent system, where each UAV possesses computing, storage, communication, and energy characteristics, and the task DNN can be decomposed and executed layer by layer. The UAVs establish a temporary network using the IEEE 802.11 protocol and use a line-of-sight (LOS) transmission model.

[0021] To accurately represent the performance of each drone's hardware, a quaternion is introduced.

[0022] in It represents computing power (FLOP / s), which determines the amount of computation completed per unit of time; Represents available memory (MB), which affects the size of loadable models and the ability to store intermediate features; This represents the communication bandwidth (Mbps), which affects the speed of task unloading and intermediate result transmission. This indicates remaining energy and determines the duration for which the drone can continuously perform its mission. Drones are functionally categorized into data acquisition drones. With reasoning drones . The drones in the mission are equipped with high-resolution cameras and sensors to acquire raw image data and perform preliminary preprocessing within the mission area. The human-machine interface is equipped with an embedded computing unit, namely Jetson Nano, to perform partial or complete deep neural network inference tasks.

[0023] (2) Constructing the actual task queue and virtual queue model: This invention constructs an actual inference layer residual queue for each UAV scenario. A virtual delay queue is constructed based on the average delay threshold and average energy consumption constraints. With virtual energy consumption queue .

[0024]

[0025] Actual task queue express The remaining task load at time t, i.e., the number of inference layers to be processed. express Is it at time t? Perform the task. Indicate whether any new tasks have arrived. .

[0026] The actual task queue reflects the real-time load status of the system. Long-term instability indicates that the system is overloaded and unable to complete the task within the available time.

[0027]

[0028] Virtual delay queue This is used to transform long-term average latency into a queue stability problem, constraining the average task latency to not exceed a preset threshold. This represents the instantaneous delay of time slot t. This represents the upper limit of the allowed average latency.

[0029]

[0030] Virtual energy consumption queue This is used to transform long-term average energy consumption into a queue stability problem, constraining energy consumption per unit time to not exceed a set average value. This represents the instantaneous energy consumption of time slot t. This represents the upper limit of the allowable average energy consumption. By constructing these three queue models, task schedulability and long-term performance constraints can be transformed into queue stability problems, providing a foundation for subsequent distributed scheduling algorithms optimized based on Lyapunov.

[0031] (3) Constructing a Lyapunov drift-penalty optimization framework: The actual task queue ensures the schedulability of the system, while the virtual latency and energy queue ensure long-term performance constraints. To uniformly measure the stability of the system, the following Lyapunov function is introduced. The Lyapunov function is defined as half of the sum of squares of the above queues, and the sum of drift and task latency energy consumption penalty terms is constructed as the optimization objective.

[0032]

[0033] Drift - Penalty:

[0034] Where V>0 is the equilibrium parameter. α and βThese are the weights for latency and energy consumption, and their relevant impact on the queue is implied in the drift term. middle.

[0035] (4) State Space Design: The state space described in this invention considers the computational load, remaining energy, communication rate, queue length, and delay / energy consumption contribution of the current time slot for both the UAV's local and neighboring nodes. The state is represented as an (N+1)×7 matrix. , where N is the number of neighboring drones, and "+1" represents the drone itself. Each row of the matrix corresponds to a complete state description of a drone (itself or a neighbor). The system variables at time t are presented in the mathematical representation of the matrix as follows:

[0036] In the formula, This represents the computational resources of the drone. , , These represent the drone's memory, communication bandwidth, and energy, respectively. The last three columns represent the current status of the task to be unloaded, mainly including: the number of remaining inference layers, task delay, and energy consumption.

[0037] (5) Action Space Design: This invention defines an action as whether to continue executing the next DNN layer locally or to unload the intermediate inference results to a neighboring UAV. The action space is a discrete space, in which actions... 0 indicates local execution. Indicates uninstallation to the 1st A neighbor's drone.

[0038] (6) Reward Function Design: The reward function is a weighted negative sum of instantaneous delay, energy consumption, and Lyapunov drift. This invention utilizes the Lyapunov drift term... A reward function is introduced to implement real-time constraints on queue stability.

[0039]

[0040] in For immediate delay, For immediate energy consumption, Let α, β, and γ be the Lyapunov drift term, and α, β, and γ be the weight parameters. The reward function design follows the "minimize drift plus penalty" principle in Lyapunov optimization.

[0041] (7) Policy training using PPO reinforcement learning algorithm: Stable policy update is achieved by shearing policy ratio constraint. This invention is the first to incorporate Lyapunov constraint term. By incorporating the PPO loss function, stable convergence of distributed multi-agent strategies can be achieved.

[0042]

[0043] This section describes the core cutoff policy loss of PPO, including the value function loss, entropy regularization term, and Lyapunov constraint term. The total loss function of LPPO is as follows:

[0044] in For the shearing strategy loss, For the value function loss, For entropy regularization, For Lyapunov constraints based on drift penalty, and All are correlation coefficients.

[0045] (8) Online scheduling and execution: In actual operation, the task execution path is dynamically decided based on the current queue status and UAV resource status, realizing hierarchical collaborative reasoning across multiple UAVs.

[0046] Step (8) employs a distributed online scheduling method for UAV bee colonies based on LPPO, including the following steps: 1) Neighbor information exchange and state matrix Build; 2) PPO strategy network sampling action ; 3) Task execution and queue update; 4) Experience replay and policy gradient update; 5) Periodic model federal average synchronization.

[0047] Example: This invention was simulated in the MATLAB environment, using three heterogeneous UAVs: an 8-layer AlexNet and a 16-layer VGGNet. Task arrival followed a Poisson distribution, and the UAV cluster sizes were 4, 6, and 8 units, respectively. Table 1 shows the UAV hardware parameter settings. Table 2 shows the hyperparameter settings.

[0048] Table 1. UAV Hardware Parameter Settings

[0049] Table 2. Algorithm Hyperparameter Settings

[0050] DNN models: 8-layer AlexNet (5 convolutional layers and 3 fully connected layers) and 16-layer VGGNet (13 convolutional layers and 3 fully connected layers), representing medium and high complexity tasks, respectively.

[0051] Input: For AlexNet, input a 32×32×3 RGB image; for VGGNet, input a 224×224×3 RGB image.

[0052] Task area: 100m×100m, divided into 25 20m×20m grids.

[0053] UAV cluster sizes: 4, 6, 8.

[0054] Task arrived: Poisson distribution.

[0055] The following performance evaluation metrics are used in this embodiment: Total throughput (number of completed tasks), average inference latency (in milliseconds), and average energy consumption (in j). Baseline comparison methods: PPO2: Dhuheir et al. (2023) Single-agent PPO, with only single-layer assignment and route planning. GREEDY: Always selects the UAV with the strongest computing power among its neighbors. RANDOM: Randomly selects idle neighbor UAVs.

[0056] (1) Results regarding throughput: AlexNet task: LPPO achieves the highest throughput with 6–8 UAVs; VGGNet tasks: In more complex tasks, LPPO has a more obvious advantage and more stable throughput.

[0057] Combination Figure 2 and Figure 3 It is understood that the present invention reduces the average inference latency by 23.5%, increases throughput by 18.7%, and improves resource utilization by 31.2%, with the most significant advantages under intermittent communication and dynamic load.

[0058] (2) Results regarding average latency: In the six sets of experiments (4 / 6 / 8 UAVs + AlexNet / VGGNet), the LPPO curve consistently showed the lowest latency and fastest convergence. Combined with... Figure 4-9 It is understood that the key advantage of this invention lies in the ability of a single machine to execute multiple layers (PPO2 is a single layer) and a multi-agent architecture (PPO2 is a single agent).

[0059] (3) Results regarding average energy consumption: combined with Figure 10-15 It can be seen that LPPO has the lowest energy consumption in all scenarios, and its growth rate is the slowest under complex VGGNet tasks, which demonstrates the superiority of the method described in this invention in energy-constrained scenarios.

Claims

1. An optimization method for UAV cooperative reasoning and time-delay control based on Lyapunov theory, characterized in that, The implementation steps include: (1) Construct a collaborative reasoning system model for unmanned aerial vehicle (UAV) swarms, including building each UAV into an intelligent agent with computing power, storage capacity, communication bandwidth and remaining energy, dividing the DNN task into layers, and supporting multi-UAV segmented execution; (2) Constructing the actual task queue and virtual queue model: the actual inference layer residual queue A virtual delay queue is constructed based on the average delay threshold and average energy consumption constraints. With virtual energy consumption queue ; The actual task queue is used to characterize the system's real-time load status. Long-term instability indicates that the system is overloaded and unable to complete the task within the available time. Virtual delay queue This is used to transform long-term average latency into a queue stability problem, constraining the average task latency to not exceed a preset threshold. Virtual energy consumption queue This is used to transform long-term average energy consumption into a queue stability problem, constraining energy consumption per unit time to not exceed a set average value; (3) Based on the Lyapunov drift-penalty function, construct the scheduling optimization objective and transform the long-term average delay and energy consumption constraints in step (2) into a time-slot-level solvable problem; (4) Construct a state space that includes local drones and neighboring drone resources and queue states; (5) Define the action as either executing the DNN layer locally or migrating to the action space of a specified neighboring UAV; (6) Construct a reward function based on the Lyapunov drift term, instantaneous delay term, and instantaneous energy consumption term; (7) The scheduling policy network is trained using the PPO reinforcement learning algorithm; (8) Perform online task scheduling based on the real-time system status to complete the collaborative DNN inference of the UAV swarm.

2. The optimization method for UAV cooperative reasoning and time delay control based on Lyapunov theory according to claim 1, characterized in that, Step (1) The drone swarm is abstracted into a multi-agent system, and a quadruple is introduced. To indicate the performance of each drone's hardware; in, Indicates computing power. Represents available memory. Indicates communication bandwidth. Indicates remaining energy; This step involves dividing the drones into data acquisition drones and inference drones. The data acquisition drones are equipped with cameras and sensors to collect raw image data within the mission area and perform preliminary preprocessing. The inference drones are equipped with embedded computing units to perform partial or complete deep neural network inference tasks.

3. The optimization method for UAV cooperative reasoning and time-delay control based on Lyapunov theory according to claim 1, characterized in that, Step (2) includes the following mathematical representation: Actual task queue express The remaining task load at time t, i.e., the number of inference layers that need to be processed. express Is it at time t? Perform the task. Indicate whether any new tasks have arrived. , Indicates the number of drones; in, Represents a virtual delay queue. Indicates the instantaneous delay of time t. The upper limit of the allowed average latency; in This represents the instantaneous energy consumption of time slot t. This represents the upper limit of the allowable average energy consumption.

4. The optimization method for UAV cooperative reasoning and time delay control based on Lyapunov theory according to claim 1, characterized in that, The Lyapunov function mentioned in step (3) is defined as half the sum of the squares of the actual task queue, the virtual delay queue, and the virtual energy consumption queue, mathematically expressed as: In the formula, This represents a Lyapunov function. Indicates the first A drone in The actual task queue at any given moment. Indicates the first A drone in Virtual delay queue for each moment Indicates the first A drone in A virtual energy consumption queue at any given moment; The Lyapunov drift-penalty function The sum of the drift term and the weighted instantaneous delay and instantaneous energy consumption is mathematically represented as: In the formula, It is the equilibrium constant. For Lyapunov drift, Let be the system state vector. α and β These are the weights for latency and energy consumption, and their relevant impact on the queue is implied in the drift term. middle, As a penalty item, Indicates the first A drone in Time delay index Indicates the first A drone in Energy consumption indicators at any given time.

5. The optimization method for UAV cooperative reasoning and time delay control based on Lyapunov theory according to claim 1, characterized in that, The state space described in step (4) integrates the computing power, storage capacity, communication bandwidth, remaining energy, remaining inference layers of the task, instantaneous latency cost, and instantaneous energy consumption cost of the local and neighboring UAVs in matrix form, and is represented as an (N+1)×7 state matrix. , where N is the number of neighboring drones, "+1" represents the drone itself, and each row of the matrix corresponds to a complete state description of a drone.

6. The optimization method for UAV cooperative reasoning and time delay control based on Lyapunov theory according to claim 1, characterized in that, The action space mentioned in step (5) is a discrete space: action This indicates that the task will be executed locally. , This indicates that the task will be migrated to the next stage. A neighbor's drone.

7. The optimization method for UAV cooperative reasoning and time delay control based on Lyapunov theory according to claim 1, characterized in that, The reward function described in step (6) is a negatively weighted combination, comprising an instantaneous delay term, an instantaneous energy consumption term, and a Lyapunov drift term: In the formula, For immediate delay, For immediate energy consumption, For Lyapunov drift term, α , β , γ These are the weight parameters.

8. The optimization method for UAV cooperative reasoning and time delay control based on Lyapunov theory according to claim 1, characterized in that, In step (7), the range is updated using a shear ratio constraint strategy, and the Lyapunov constraint term is incorporated into the PPO loss function to improve training stability.

9. The optimization method for UAV cooperative reasoning and time delay control based on Lyapunov theory according to claim 1, characterized in that, Step (8) employs a distributed online inference mechanism to dynamically determine the task execution path based on the current queue status and UAV resource availability, achieving hierarchical collaborative inference across multiple UAVs. The inference process includes: 1) Neighbor information exchange and state matrix Build; 2) PPO strategy network sampling action ; 3) Task execution and queue update; 4) Experience replay and policy gradient update; 5) Periodic model federal average synchronization.

10. A UAV swarm cooperative reasoning scheduling system based on Lyapunov optimization and PPO reinforcement learning constructed according to any one of claims 1-9, characterized in that, include: The system modeling module is used to abstract the UAV swarm into a multi-agent system and hierarchically classify the DNN tasks; The queue management module is used to maintain the actual task queue, the virtual delay queue, and the virtual energy consumption queue. Optimize the target module to construct the Lyapunov drift-penalty function; The State and Action module is used to generate the state space and action space. The policy training module is used to train scheduling policies based on the PPO algorithm and Lyapunov constraints. The online execution module is used to dynamically execute tasks locally or make migration decisions and update queue status. It supports multi-hop hierarchical collaborative reasoning and maintains system stability in environments with weak communication, dynamic topology and heterogeneous resources.

Images