Unmanned aerial vehicle path planning method and system based on GNN and high-order security constraint

Abstract

The invention relates to an unmanned aerial vehicle path planning method and system based on GNN and high-order security constraints. The method comprises the steps that a navigation scene where an unmanned aerial vehicle is located is represented as a heterogeneous directed graph, message passing and feature updating are conducted through the GNN, a risk-aware attention mechanism is introduced, and an interpretable decision result is output; a differentiable HoCBF-QP optimization layer is introduced, an original control command output by a strategy network is used as input, quadratic programming with high-order control barrier function constraints is solved online, minimum-amplitude safety correction is carried out on the control command, and an actuator control command meeting safety constraints is output; starting a HoCBF safety shield during operation so as to strictly ensure that all safety constraints are met before execution; a calculation task of the whole control cycle is modeled into a directed acyclic graph form, and parallel execution is carried out on heterogeneous multiple cores by utilizing a real-time scheduling strategy. The problem that unmanned aerial vehicle navigation control is not effective and unified in three aspects of structure, safety and scheduling is solved.

Description

Technical Field

[0001] This invention relates to an autonomous navigation and intelligent control technology for unmanned aerial vehicles (UAVs), and particularly to a real-time safe path planning system and method for UAVs. Background Technology

[0002] When unmanned aerial vehicles (UAVs) perform autonomous flight missions in dynamic and complex environments, they need to react under strict time-delay constraints and ensure flight safety. Traditional UAV navigation and control systems typically employ a phased pipeline structure of "perception → mapping → planning → control." This classic architecture offers good interpretability, but the sequential execution of each stage introduces cumulative latency, making it difficult to meet the real-time requirements of high-speed scenarios. In recent years, some end-to-end one-stage planning methods have emerged, such as the YOPO method inspired by object detection YOLO. It directly regresses flight action anchor points from sensor inputs through a single neural network, thereby reducing latency. However, this type of "one-stop" planning lacks hard safety constraints and interpretability guarantees, and there are potential risks of unsafe actions during flight.

[0003] Another approach is to combine reinforcement learning or end-to-end control policies with safety filters. For example, training an RL control policy while introducing a control barrier function (CBF) as a safety layer, i.e., the RL+CBF framework. This method guarantees safety to some extent, but often treats safety correction as a post-processing step independent of policy learning. The policy itself may not fully consider safety constraints, potentially leading to overly aggressive or discontinuous control outputs. Furthermore, running deep learning controllers at high frequencies on embedded platforms presents significant challenges, requiring meticulous scheduling of the execution of perception, planning, and control modules. The default scheduling mechanism of commonly used robot middleware ROS struggles to provide strict real-time guarantees. In practice, engineers typically reduce latency by setting control threads to FIFO real-time priority, binding threads to specific CPU cores, and using lock-free queues to transfer perception results. However, these manual optimization methods lack a systematic model and are not easily scalable to complex tasks to guarantee optimal resource utilization.

[0004] Control barrier functions (CBFs), as a safety-assured control method, have received widespread attention in recent years. CBFs ensure that the system state remains within the safe set by imposing inequality constraints on the control inputs, achieving "minimal intervention" safety constraints—that is, making minimal corrections to the control command only when approaching a violation of safety conditions. Standard CBFs typically address constraints of relative order 1. For problems with higher relative orders, such as obstacle avoidance in unmanned aerial vehicles (UAVs) (e.g., position safety constraints have a second-order relationship with acceleration control inputs), it is necessary to utilize extended methods using higher-order control barrier functions (HoCBFs). HoCBFs, by introducing higher-order derivative terms and corresponding gain coefficients, can handle safety constraint problems in highly dynamic systems like UAVs. For example, for a defined safety function... (For example, representing obstacle distance margin), its zeroth-order control barrier function can be taken as .like If the order of the relative control input is 2, then the first derivative function can be defined. And further second-order control barrier functions ,in , A coefficient greater than 0. Through constraints. This ensures the original safety conditions. The positive invariance in a closed-loop system means that once the system state is within a safe set, it will remain safe under control. The above... Through system dynamics Expand as ;in, This is the state vector of the UAV (including state variables such as position and velocity). This is the control input vector for the UAV. This refers to the system's drift term (the natural dynamics without control input). For control input matrix (describes how control inputs affect the state). For safety functions Lie derivative along vector field f, For safety functions The Lie derivative along the vector field g. Therefore, the security constraints. Equivalent to linear inequality constraint form This constraint form of CBF facilitates integration with optimal control, allowing for direct correction of the control input through online constraint optimization.

[0005] In summary, existing technologies have not yet formed a unified solution in terms of structure, security, and scheduling: traditional hierarchical planning has structure and interpretability but lacks real-time performance; end-to-end planning is efficient but security cannot be guaranteed; security filtering methods help with post-event correction but are not deeply integrated with policy planning; and there is a lack of scheduling frameworks for embedded real-time control. Summary of the Invention

[0006] To address the lack of effective unification in the structure, safety, and scheduling aspects of existing UAV navigation and control systems, this paper proposes a UAV path planning method and system based on GNN and high-order safety constraints. The method utilizes graph structure strategies to achieve global planning and interpretability, combines an embedded high-order CBF optimization layer to ensure safety constraints, and fully utilizes heterogeneous computing resources through DAG task scheduling in the control cycle to achieve real-time safety control of 100 times per second on an embedded platform.

[0007] The technical solution of this invention is as follows: A UAV path planning method based on GNN and high-order safety constraints includes: Step 1: Heterogeneous Risk Graph Modeling and Graph Neural Network Planning; First, the navigation scenario of the UAV is represented as a heterogeneous directed graph; the nodes of the graph include task nodes, obstacle nodes, and constraint nodes. For each node, a feature vector containing a risk descriptor with a high-order control barrier function is designed. The edges of the graph represent the relationships between nodes, and edge attributes are added for subsequent message passing; based on this, a graph neural network (GNN) is used to perform L-layer message passing and feature updating on the heterogeneous graph; a risk-aware attention mechanism is introduced in the message passing stage; through L-layer iterative message passing and state updating, the graph neural network extracts global semantics and risk information, and finally outputs interpretable decision results in the readout stage; Step 2: Embed a high-order control barrier function (QP) layer; For the original control command output by the graph neural network, a differentiable HoCBF-QP optimization layer is introduced between the output of the policy network and the actual control command executed by the UAV actuator; The HoCBF-QP optimization layer takes the original control command output by the policy network as input, solves the quadratic programming with high-order control barrier function constraints online, performs minimum-amplitude safety correction on the control command, and outputs actuator control commands that satisfy the safety constraints, which are used to correct the control command in real time to meet the safety constraints; Step 3: Enable runtime HoCBF safety shield; set up a lightweight runtime HoCBF-QP safety shield before final control execution; the difference between this safety shield and embedded QP is that it operates on the final output stage to strictly ensure that all safety constraints are satisfied before execution; the runtime safety shield uses the actuator control instructions output by embedded QP as reference input to solve a QP projection problem, so that a final control command that satisfies all safety constraints can be found in each control cycle. ； When the QP is feasible in a certain state, the control barrier function theory can guarantee that the UAV state will not deviate from the safe set, thus completely preventing accidents; once the final control command is received... ， It is sent to the flight controller for execution, thereby completing the safety control output for the current control cycle; Step 4: Control Cycle DAG Scheduling and Real-Time Execution; The computational task of the entire control cycle is modeled as a directed acyclic graph and executed in parallel on heterogeneous multi-core systems using a real-time scheduling strategy; The graph strategy planning module, the HoCBF-QP safety optimization module, and the real-time scheduling and resource management module work together: the scheduling ensures that the deep learning inference and optimization solution layers are executed at a high frequency, while the safety layer ensures that the output of the planning strategy is always within the safe and feasible domain, thus jointly achieving real-time and safe UAV navigation control.

[0008] Furthermore, step 1 specifically involves: Interpretable decision results include: the UAV's next anchor point trajectory segment, the mission priority DAG, and the control intent map; The attention mechanism is specifically as follows: in the first... In the layer, for nodes Each neighbor node Calculate attention score ;in, Indicates the first Layer time center node eigenvectors, Indicates the first Layer-time neighbor nodes eigenvectors, Indicates that by node Pointing to node The edge attribute is used to encode the topological or geometric relationship between the two. A function that combines node features and edge attributes to measure the performance of a node. For nodes Influence weights; then influence on neighbors The influence is normalized using softmax to obtain the attention weights. Where k represents the index of the attention head, Represents a node The set of neighbors, that is, the set of nodes in the graph structure. There exist all nodes directly connected by edges, where, This indicates that in the graph neural network... In the layer, there are nodes Point to its neighbor node The edge feature vectors are used to encode the geometric or topological relationship between the two and serve as input for attention calculation and message passing in this layer.

[0009] Furthermore, steps 2 and 3 are as follows: Step 2: Actuator refers to the actuator on the UAV body; actuator command refers to the low-level control quantity sent to the aforementioned actuator through the flight control system; this QP layer is based on the aforementioned higher-order control barrier function condition. A constrained optimization problem is constructed with the objective of minimizing the modifications made to the original command, thereby achieving minimally invasive security assurance; let... , ,in, Let U be the state vector of the UAV. This is the control input vector for the UAV. For the system's drift term, To control the input matrix, For safety functions Lie derivative along vector field f, For safety functions Lie derivative along vector field g, This indicates that the second-order auxiliary barrier function, obtained recursively from the higher-order control barrier function, is nonnegative. It is used to characterize a system of relative order 2 where the state remains within the safe set throughout two integration steps; For the corresponding class The gain parameter, used to adjust the contraction speed and conservatism of the second-order safety constraint, is determined by the current state. The calculated coefficients allow the safety constraints to be expressed as linear inequalities. In each control cycle, solve the following quadratic programming problem to obtain the corrected safety control command. :

[0010] in The objective function in the above equation is the correction amount for the original control input. It ensures that it is as close as possible to the original control commands. Constraints This ensures that the aforementioned HoCBF security inequality holds, and the modified security control command... Not exceeding the actuator's permissible control range For cases with a small number of constraints, the above QP has an analytical KKT solution, and the optimal correction Δu can be directly calculated, thus quickly obtaining the desired result. For cases with multiple constraints, a differentiable QP solver is used to obtain the solution and allow the gradient to propagate through the layer to penalize unsafe initial control command preferences during the policy training phase, thereby guiding the policy output to be biased towards the safe and feasible region. Step 3: Runtime security shield outputs security control commands via embedded QP Using this as a reference input, solve a small-scale QP projection problem:

[0011] in, This represents the optimal solution to the QP projection problem, i.e., the HoCBF-QP shield in its current state during runtime. The final safety control command obtained is then sent to the UAV flight control system for execution. Indicates the current state The calculated linear constraint coefficient matrix associated with all HoCBF safety constraints, This is a set of indices for HoCBF safety inequality constraints that are active or need to be checked in the current state. Each row of this matrix corresponds to a safety constraint pair with the control input. The gradient; Indicates the first The HoCBF safety inequality constraint in the current state The constant term / right-hand term below is used to characterize the "residual margin" of the constraint when no control input is applied; This represents the set of all HoCBF safety inequality constraints that need to be satisfied in the current state.

[0012] Furthermore, step 4 specifically involves: A control cycle includes several tasks such as sensor data acquisition and preprocessing, graph construction, GNN inference computation, QP solving, security checks, and sending control commands for execution. A Directed Acyclic Graph (DAG) is built based on the dependencies between these tasks. Using this DAG, tasks without direct dependencies are scheduled for parallel execution, thus fully utilizing the computing resources of the CPU and GPU. To reduce resource contention and scheduling overhead, thread binding and asynchronous execution techniques are employed: critical threads are fixed to run on designated CPU cores, avoiding scheduling and switching between multiple cores; lock-free producer-consumer queues are used to transfer data between GPU inference threads and CPU control threads, reducing synchronization blocking; and CUDA asynchronous streaming and event mechanisms are used to overlap GPU computation and data copying with CPU computation.

[0013] A UAV path planning system based on GNN and high-order safety constraints includes: a graph construction module for constructing a graph model required for path planning based on environmental information; a graph neural network inference module for performing path planning calculations on the graph model to predict the optimal or suboptimal safe flight route; a safety constraint module for applying high-order flight safety constraints to ensure that the planned path meets obstacle avoidance and flight specification requirements; a control command generation module for converting the planned route into control commands executable by the UAV at its underlying level; and a real-time scheduling module for coordinating the continuous operation of each module according to the control cycle on an embedded platform to achieve real-time closed-loop control of path planning. Each module is deployed on an embedded computing platform on the UAV and interacts with the external environment and UAV actuators through sensor data input and flight control command output. System Architecture: This system adopts a modular, layered design. The UAV is equipped with an embedded AI computing platform, which integrates a high-performance GPU and a multi-core CPU to run deep learning models and process sensor data in real time at the edge. The platform is equipped with environmental perception sensors, an inertial navigation unit (IMU), and a satellite positioning module to acquire maps of the UAV's surrounding environment, obstacle information, and its own attitude and position. The UAV path planning and control system framework is implemented in software on this embedded platform, divided into modules for graph construction, graph neural network inference, safety constraints, control command generation, and real-time scheduling. Each module is connected through a predefined data interface: sensor data is preprocessed and then transmitted to... The graph construction module generates the environmental graph structure; the graph neural network inference module obtains graph data and the current mission objective from the graph construction module, and outputs the planned path or the next waypoint; the safety constraint module verifies and adjusts the planning results; the control command generation module calculates the control quantity based on the final path and sends it to the UAV flight control system; the real-time scheduling module acts as the brain of the system, triggering the sequential execution of each module according to the preset control cycle, and monitoring the operating status of each module to ensure real-time performance and reliability; the software architecture of the entire platform follows a real-time multi-threaded design, with each functional module being decoupled from each other while working collaboratively under the control of the scheduling module, thereby ensuring that the UAV path planning operates stably in a dynamic environment.

[0014] Furthermore, the graph construction module specifically includes: The graph construction module is responsible for abstracting the UAV's flight space into a graph structure model; first, based on the map information and obstacle distribution provided by the environmental perception sensors, it defines the set of vertices of the graph. Sum of edges Regular grids, spatial grid points, or key waypoints are used as graph nodes. An edge is established when adjacent nodes are connected and there are no obstacles blocking the way. Each node is associated with attribute information; each edge is assigned a weight to quantify the cost of the drone flying along that edge; the edge weights are defined by comprehensively considering flight distance, energy consumption, and safety risk factors. The weight is the Euclidean distance between the two nodes. Weighted sum of security risk costs:

[0015] in Represents a node and The distance between, Indicates along the edge The risks and costs of flying The weighting coefficients are used to balance the impact of distance and risk on the overall cost; In this way, if a path passes through a high-risk area, the weight of the corresponding edge will be significantly increased, so that it will be automatically avoided in subsequent path planning or only selected when there is no other way. The graph construction module is also responsible for dynamically adjusting the graph structure according to the real-time updated sensor data: when new obstacles appear in the environment or the positions of existing obstacles change, graph nodes are added or deleted or edge weights are updated accordingly; if the target position changes or the current position of the UAV is updated, the information of the start or end node in the graph is modified. This dynamic graph model enables this module to reflect the changes in the environment in which the UAV is located in real time, providing an accurate and up-to-date search space for subsequent graph neural network inference.

[0016] Furthermore, the graph neural network inference module specifically comprises: The graph neural network (GNN) inference module is used to efficiently calculate the optimal path for a UAV on a constructed environment graph. This module employs a pre-trained GNN model, transforming the path planning problem into a neural network inference process on the graph. The GNN model takes the graph structure and related features as input: the initial feature vector of each node includes its coordinates, altitude, indication of whether it is a target node, and local risk value; the edge features include the aforementioned weights. And possible path constraint labels; the model uses the entire graph As the input structure of the neural network, the hidden state of the nodes is iteratively updated through multi-layer message passing and neighbor information aggregation; let... Indicates the first After layer iteration, the node Given the hidden vector representation, a message-passing layer of a GNN can be described as follows:

[0017] in For nodes The set of neighboring nodes, For the edge eigenvectors, For trainable weight matrix, It is a non-linear activation function. Used to contact neighbors The state and edge features are weighted and transformed; The multi-layered graph neural network structure makes the final representation of each node... It integrates graph information from its k-hop neighborhood; when the number of network layers and the receptive field are large enough, the starting node can perceive the global information of the target node; after several iterations, the GNN output layer maps each node to a path evaluation value or a next action preference; the GNN model learns approximate reasoning ability about the shortest path on the graph through offline training, which is equivalent to using a learning method to approximate the solution of the Bellman optimization equation; the cost function of the optimal path. The Bellman equation should be satisfied:

[0018] in , Let represent the target node in the graph, and define the cost of this node as 0. The cost is 0; the graph neural network approximates the above recursive relationship through a finite layer of message passing: initially, the cost of the target node is set to the minimum, and the low cost is propagated from the target to the neighboring nodes layer by layer, and the cost value assessment of each node is gradually corrected; finally, the output of the starting node will give the minimum cumulative cost estimate from the starting point to the target, and the output of the neighboring nodes will indicate the direction selection for the next step; once the GNN inference module has completed the path evaluation in the current environment, starting from the starting point, it greedily selects the descent gradient direction to gradually approach the target based on the evaluation value of the neighboring nodes, and obtains an approximately optimal path sequence; or, the output of the GNN directly gives the decision of the neighboring node to go to next, and the drone moves towards the target step by step and re-evaluates the path at each step.

[0019] Furthermore, the safety constraint module specifically includes: The safety constraint module serves as the central safety control unit throughout the entire process. It comprehensively utilizes task rules, airspace restrictions, obstacle information, and UAV platform performance constraints to provide unified safety management for each stage, including graph construction, GNN inference, and path output. On one hand, this module filters, reweights, and masks nodes and edges at the layer level to preemptively eliminate obviously infeasible or high-risk routes. On the other hand, it checks candidate paths point by point at the trajectory level, triggering adaptive replanning when safety constraints are violated. Furthermore, it works in conjunction with the HoCBF-QP safety optimization module to further reduce path-level constraints to control-level constraints, thereby ensuring flight safety at both the path planning and underlying control levels. In the pre-planning stage, the safety constraint module preprocesses the environmental graph generated by the graph construction module according to the flight mission and airspace regulations: it removes all nodes located in no-fly zones or outside the flight airspace, and removes these inaccessible nodes from the node set; for edges that cross dangerous areas or do not meet flight performance restrictions, they are directly prohibited from use, that is, removed from the edge set, or their weights are set to maximum values ​​to avoid being selected by the GNN. During the planning process, the safety constraint module maintains a series of flight state constraint parameters and explicitly integrates these constraints into the graph structure by dynamically adjusting edge weights or node states: On the one hand, if the shortest distance between a certain path and an obstacle is less than the safe distance... If the risk cost of the corresponding edge is significantly increased, the edge will be considered impassable. On the other hand, for maneuvers that involve the performance limits of the UAV, the system will automatically increase the cost of the corresponding edge of the maneuver or break it down into multiple actions to meet the flight state constraints. In the post-planning phase, after the GNN inference module generates the initial path, the safety constraint module performs a global review of the entire candidate path, checking point by point for violations of airspace rules, obstacle safety distances, or platform performance limitations. If a waypoint in the path is found to be located at the boundary of a no-fly zone or too close to an obstacle, the module will trigger replanning: by increasing the weight penalty for the violation area or tightening the relevant constraint parameters, the GNN inference module is called again to calculate the path until a new path that meets the predetermined safety conditions is obtained. To ensure obstacle avoidance safety, the distance between any point on the UAV's trajectory and any obstacle in the environment must not be less than the safety distance; otherwise, the path is deemed unqualified and automatically adjusted or replanned. The safety constraint module ensures, through the aforementioned constraints, that the planned flight path is sufficiently isolated from no-fly zones and obstacles in space, conforms to the platform performance limitations of the UAV in kinematics, and meets flight rules and safety redundancy at the mission level. The safety constraint module also integrates a dynamic safety mechanism: it continuously monitors changes in the surrounding environment during the UAV's flight, updates the constraints in real time, and notifies the scheduling module to trigger path replanning, thereby promptly correcting the flight path to avoid newly emerging threats, enabling the UAV to complete its mission in compliance with flight safety regulations in complex environments.

[0020] Furthermore, the control command generation module specifically comprises: Control command generation module: This module is responsible for converting the planned path into a sequence of low-level commands executable by the UAV flight control system, thereby putting the path planning results into practice. The path output by the graph policy network is a set of spatially discrete waypoints. ,in Starting point For the target point, the middle This indicates a path inflection point or a point indicating the next target. The control command generation module first smooths and interpolates the path points appropriately to avoid overly abrupt turns that could lead to flight instability. Then, based on the UAV's current state and the location of the next target point, it calculates the required control output. Simultaneously, based on the current position and... The azimuth difference determines the desired heading angle And climb / descent rates; for quadcopter UAVs, these high-level motion commands are further calculated by the flight control system into specific motor speeds and control surface deflections; the control command generation module also uses PID control and model predictive control algorithms to convert position deviations into attitude and throttle adjustments; control command generation is performed in real time online: in each control cycle, the control output is updated according to the latest planned path and flight status, thus possessing the ability to adapt to environmental changes; whenever the UAV reaches the vicinity of the current target path point, the module will switch to the next path point, continuously guiding the UAV toward the final target; the control command generation module is controlled by the real-time scheduling and resource management module according to a fixed control cycle. T Periodic Triggering of Online Operation: In each control cycle, the control command generation module reads the nearest reference waypoint on the current planned path from the graph strategy planning module, obtains the real-time flight status of the UAV from the sensing and control interface module, and then calculates new flight control commands according to the preset trajectory tracking control law and sends them to the UAV flight control system. When a path replanning is triggered due to environmental changes during flight, the graph strategy planning module updates the current valid path in shared memory. The control command generation module automatically reads the latest path in the next control cycle and calculates new control commands accordingly, achieving seamless connection and smooth transition between the old and new paths. Through the above-mentioned periodic online calculation and closed-loop update mechanism, the control command generation module establishes a direct connection between the algorithm planning layer and the physical control layer, enabling the planning results to be reflected in the actual flight actions of the UAV in a stable and controllable manner, and to generate adaptive responses to environmental changes.

[0021] Furthermore, the control command generation module specifically comprises: Real-time scheduling module: The real-time scheduling module is the central hub of the system, responsible for coordinating the execution of various functional modules on the embedded platform at a predetermined frequency, forming a closed-loop real-time path planning and control process. This module employs a real-time multi-task scheduling mechanism to ensure that critical tasks are completed within strict timeframes. During operation, the scheduling module triggers the following steps in a periodic loop: First, at the beginning of each cycle, a sensor data acquisition and preprocessing thread is started to update the latest information on the UAV's current position, attitude, and surrounding environment. Next, the graph construction module is called to integrate the new sensor data into the environmental graph model, updating node and edge information. Then, the graph policy network is triggered to perform path planning calculations on the updated graph, obtaining a suggested route or next action decision for the UAV from its current state to the target point. Subsequently, the safety constraint module is run to review the planning results and make necessary corrections. Finally, the control command generation module is called to calculate control commands based on the final confirmed path and send them to the UAV for execution via the flight control interface. The real-time scheduling module pre-sets strict time slices and sequences for the above processes to ensure that the entire loop occurs within a defined control cycle. The scheduling module ensures that each module does not time out through fine-grained time management: if a module fails to complete within the specified time in a certain cycle, the scheduling module discards the planning results of that cycle or simplifies the processing to avoid blocking the execution of subsequent cycles; at the same time, the parallel execution of multi-core CPUs and GPUs, with GNN inference accelerated by GPUs, shortens the overall cycle latency; the data dependencies between modules are strictly maintained by the scheduling module: subsequent GNN inference will only start after the graph construction module produces the latest graph data; the safety verification of the planned path must be completed before sending new control commands; the real-time scheduling module is also responsible for monitoring the system's operating status, and will promptly issue alarms or adjust the cycle to ensure flight safety once an anomaly is detected; through the above scheduling strategies, a closed-loop real-time operation from environmental perception, path planning to control execution is achieved.

[0022] The beneficial effects of this invention are as follows: This invention organically combines learning planning, formal security assurance, and real-time scheduling. It provides a security-aware graph strategy and a DAG scheduling method for control cycles, used for real-time autonomous navigation control of unmanned aerial vehicles (UAVs). This method models the UAV navigation task as a heterogeneous graph structure, performs global planning and decision-making through a graph neural network, embeds high-order control barrier functions during control generation, and achieves high-frequency real-time control on an embedded multi-core heterogeneous computing platform using DAG task scheduling per cycle. It has the following beneficial effects: (1) Improved mission success rate and safety: By embedding high-order CBF constraints in policy planning and combining them with runtime safety shields, the UAV can always meet safety conditions during mission execution, greatly reducing the frequency of dangerous situations such as collisions. According to experimental results, compared with baseline methods that only use end-to-end planning or simple RL+CBF posterior correction, this invention significantly improves the overall mission success rate and significantly reduces the number of safety interventions.

[0023] (2) Smooth trajectory and continuous control: Because the HoCBF safety layer optimizes the original control output with minimal amplitude, it avoids abrupt control cutoff. As a result, the speed and acceleration curves of the UAV are smoother, reducing high-frequency jitter and impact on the actuators. Compared with the control strategy without safety filtering, the change in control quantity after applying HoCBF is more continuous and gentle, such as a significant reduction in the total change of speed trajectory.

[0024] (3) Real-time performance and embedded deployment reliability: The control cycle DAG scheduling scheme of this invention fully utilizes the parallel capabilities of heterogeneous multi-core hardware, achieving a control frequency of up to 100Hz on resource-constrained embedded platforms, meeting the real-time requirements of high-speed flight. The computational latency at each stage is strictly controlled, and the end-to-end system latency remains within 10–20ms with small fluctuations, ensuring the determinism and stability of the control loop. Compared to systems lacking dedicated scheduling optimization, this invention significantly reduces CPU / GPU idle and waiting time, improves the utilization of computing units, and ensures no timeouts even under complex task loads.

[0025] (4) Decision Interpretability and System Extensibility: The task priority DAG and control intent graph output by this invention intuitively demonstrate the task relationships and environmental threat factors behind the UAV's current decision, making the autonomous control strategy transparent and easy for humans to understand and verify. This graph structure strategy also provides the possibility for future expansion to multi-UAV collaboration (exchanging task intent graphs), distributed task allocation, etc. The graph-based message passing framework makes the algorithm robust and extensible to newly added obstacles or tasks in the environment, and the graph structure can be easily adjusted to adapt to new scenario requirements.

[0026] In summary, this invention organically integrates three innovations: heterogeneous graph neural network planning, embedded high-order control barrier function QP optimization, and DAG scheduling within the control cycle, realizing a real-time UAV safety navigation method suitable for dynamic scenarios. This method combines theoretical rigor with engineering practicality, significantly improving the speed and reliability of UAV autonomous navigation while ensuring safety, and has broad application prospects in practical embedded autonomous flight missions. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the architecture of the UAV real-time path planning system of the present invention. Detailed Implementation

[0028] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0029] A UAV path planning method based on graph neural network structure and high-order safety constraints includes: Step 1: Heterogeneous Risk Graph Modeling and Graph Neural Network Planning. First, the navigation scenario of the UAV is represented as a heterogeneous directed graph. The graph nodes include task nodes, obstacle nodes, and constraint nodes. Different types of nodes carry different semantic information; for example, task nodes represent waypoints or sub-targets, and obstacle nodes represent dynamic or static obstacles. For each node, a feature vector containing a risk descriptor of a higher-order control barrier function is designed, such as the zeroth, first, and second-order values ​​of the aforementioned safety function. The risk feature component of a node is used to characterize the safety margin of the node's related constraints. The edges of the graph represent the relationships between nodes, such as the relative distance and orientation between the drone and obstacles, and the order of tasks, with additional edge attributes (such as distance, collision time prediction, and deadline margin) for subsequent message passing. Based on this, a graph neural network (GNN) is used for L-layer message passing and feature updating of this heterogeneous graph. To highlight the influence of high-risk neighbors on decision-making, this invention introduces a risk-aware attention mechanism in the message passing stage. Specifically, in the first... In the layer, for nodes Each neighbor node Calculate attention score ;in, Indicates the first Layer time center node eigenvectors, Indicates the first Layer-time neighbor nodes eigenvectors, Indicates that by node Pointing to node Edge attributes (such as relative positions, distances, or other structural / semantic information between nodes) are used to encode the topological or geometric relationships between them. A function that combines node features and edge attributes to measure the performance of a node. For nodes The influence weight of neighbors. The influence is normalized using softmax to obtain the attention weights. Where k represents the index of the attention head, Represents a node The set of neighbors, that is, the set of nodes in the graph structure. There exist all nodes directly connected by edges, where, This indicates that in the graph neural network... In the layer, there are nodes Point to its neighbor node The edge feature vectors are used to encode the geometric or topological relationship between the two nodes (such as relative position, distance, and risk-related attributes), and serve as input for the attention calculation and message passing of this layer. The risk-aware attention mechanism enables attention to be triggered when a neighboring node is at high risk (e.g., its...). When the value is very small (indicating a low safety margin), the neighbor receives a larger [safety margin]. Weights. Through L-layer iterative message passing and state updates, the graph neural network extracts global semantics and risk information, ultimately outputting interpretable decision results in the readout phase, including: (i) the UAV's next anchor point trajectory segment (a target sub-trajectory on a short time scale), (ii) a task priority DAG (a directed acyclic graph representing the order and importance of each sub-task), and (iii) a control intent graph (reflecting the edges that have the greatest impact on the current decision, used to explain why the UAV takes a certain action). These outputs provide a reference trajectory for the current decision and reveal the task relationships and risk factors considered in the strategy, enhancing the transparency and interpretability of the decision.

[0030] Step 2: Embed a high-order control barrier function (QP) layer. This is for the original control commands output by the graph neural network. (i.e., the velocity or acceleration reference quantity given by the graph neural network inference module / graph policy network in step 1, hereinafter referred to as "policy network output"), for example, the velocity or acceleration command of the UAV. This invention introduces a differentiable HoCBF-QP optimization layer between the policy network output and the actual control command executed by the UAV actuator. Here, the policy network refers to the graph policy network described in step 1, responsible for outputting the UAV's reference control command based on the heterogeneous risk graph; the actuator refers to the motors, control surfaces, and other actuators on the UAV body; the actuator command refers to the underlying control quantity (such as desired thrust, attitude, or angular velocity command) issued to the aforementioned actuator through the flight control system. The HoCBF-QP optimization layer takes the original control command output by the policy network as input, solves a quadratic programming problem with higher-order control barrier function constraints online, performs minimum-amplitude safety corrections on the control command, and outputs actuator control commands that satisfy the safety constraints, used to correct the control command in real time to meet the safety constraints. This QP layer is based on the aforementioned higher-order control barrier function conditions. A constrained optimization problem is constructed with the objective of minimizing the modifications to the original command, thereby achieving minimally invasive security guarantees. Specifically, let... , ,in, This is the state vector of the UAV (including state variables such as position and velocity). This is the control input vector for the UAV. This refers to the system's drift term (the natural dynamics without control input). For control input matrix (describes how control inputs affect the state). For safety functions Lie derivative along vector field f, For safety functions Lie derivative along vector field g, This indicates that the second-order auxiliary barrier function, obtained recursively from the higher-order control barrier function, is nonnegative. It is used to characterize a system of relative order 2 where the state remains within the safe set throughout two integration steps; For the corresponding class The gain parameter (design constant) is used to adjust the contraction rate and conservatism of the second-order safety constraint. The larger the value, the stronger the force that pulls the system back to the safe set boundary, and the more conservative the constraints, thus affecting the current state. The calculated coefficients allow the safety constraints to be expressed as linear inequalities. In each control cycle, the following quadratic programming problem is solved to obtain the corrected safety control command. :

[0031] in The objective function in the above equation is the correction amount for the original control input. It ensures that the original control commands and constraints are as close as possible to the actual control conditions. This ensures that the aforementioned HoCBF security inequality holds, and the modified security control command... Not exceeding the actuator's permissible control range The optimization problem is a quadratic form with linear constraints, which can be solved efficiently online and is easily integrated into the computation graph of a graph policy network for gradient backpropagation training. For cases with a small number of constraints (e.g., when only a single constraint is active), the above QP has an analytical KKT solution (Kuhn-Tucker theorem), and the optimal correction Δu can be directly calculated, thus quickly obtaining the desired result. For cases with multiple constraints, a differentiable QP solver is used to obtain the solution and allow gradients to propagate through the layer. This penalizes unsafe initial control command preferences during the policy training phase, thereby guiding the policy output towards the safe and feasible region.

[0032] Step 3: Enable the runtime HoCBF security shield. In addition to the embedded QP security layer, this invention also sets up a lightweight runtime HoCBF-QP security shield before final control execution. This shield is similar to the embedded QP form described above, but operates in the final output stage to strictly ensure that all security constraints are met before execution. Specifically, the runtime security shield is implemented as an embedded QP output... Using this as a reference input, solve a small-scale QP projection problem:

[0033] in, This represents the optimal solution to the QP projection problem, i.e., the HoCBF-QP shield in its current state during runtime. The final safety control command obtained is then sent to the UAV flight control system for execution. Indicates the current state The calculated linear constraint coefficient matrix associated with all HoCBF safety constraints, This is a set of indices for HoCBF safety inequality constraints that are active or need to be checked in the current state. Each row of this matrix corresponds to a safety constraint pair with the control input. The gradient (e.g., by) , (composed of equal items) (or the whole thing is called) ) indicates the first The HoCBF safety inequality constraint in the current state The constant term / right-hand term is determined by the higher-order control barrier function and its Lie derivative, class The function parameters are calculated and used to characterize the "residual margin" of the constraint when no control input is applied; This represents the set of all HoCBF safety inequality constraints that need to be satisfied in the current state (e.g., multiple obstacles correspond to multiple constraints). Furthermore, this optimization can incorporate other constraints such as upper and lower limits for control inputs. Due to the dimensionality of UAV control... Typically lower (e.g.) The QP problem described above is small in scale and can be solved quickly on the onboard CPU, thus allowing the finding of a final control command that satisfies all safety constraints in each control cycle. 。 When the QP is feasible in a certain state, the control barrier function theory can guarantee that the UAV state will not leave the safe set (forward invariance), thus completely preventing accidents. Once obtained... ， It is sent to the flight controller for execution, thereby completing the safety control output of the current control cycle.

[0034] Step 4: Control Cycle DAG Scheduling and Real-Time Execution. To ensure the stable high-frequency operation of the graph policy planning module (graph policy network), HoCBF-QP safety optimization module, and sensing and control interface module on the embedded platform, this invention models the computational tasks of the entire control cycle as a directed acyclic graph (DAG) and executes them in parallel on heterogeneous multi-core processors using a real-time scheduling strategy. Specifically, a control cycle includes several tasks such as sensor data acquisition and preprocessing, graph construction, GNN inference computation, QP solving, safety checks, and sending control commands for execution. A DAG graph is established based on the sequential dependencies between these tasks. For example, after the sensing task is completed, its output is provided to the planning task via a lock-free queue; after the planning (GNN inference) obtains the result, QP solving is triggered, etc. Using this directed acyclic task graph, tasks without direct dependencies can be arranged for parallel execution, thereby fully utilizing the computing resources of the CPU and GPU. For example, within a cycle, when the GPU begins executing the GNN forward inference of the current cycle, the CPU can simultaneously process the QP solution left over from the previous cycle or the sensing data reception of the current cycle, thereby achieving pipeline overlap between the GPU and CPU. To reduce resource contention and scheduling overhead, this invention employs thread binding and asynchronous execution techniques: key threads are fixed to run on designated CPU cores, avoiding scheduling switches between multiple cores; lock-free producer-consumer queues are used to transfer data between the GPU inference thread and the CPU control thread, reducing synchronization blocking; and CUDA asynchronous streaming and event mechanisms are used to allow computation and data copying on the GPU to overlap with CPU computation. Through the above scheduling optimizations, the entire control system can stably achieve a control frequency of 100 times per second on embedded SoC platforms (such as the NVIDIA Jetson series), while ensuring that the worst-case execution time of each task within the cycle is controlled, without deadline omissions, meeting hard real-time requirements. The graph strategy planning module, the HoCBF-QP safety optimization module, and the real-time scheduling and resource management module work together: scheduling ensures that the deep learning inference and optimization solution layer can be executed at a high frequency, while the safety layer ensures that the output of the planning strategy is always within the safe and feasible domain, thus jointly achieving real-time, safe and reliable UAV navigation and control.

[0035] like Figure 1As shown, the architecture of a real-time path planning system for unmanned aerial vehicles (UAVs) based on graph neural network structure and high-order safety constraints proposed in this invention is illustrated. It includes the following functional modules and their interactions: (1) a graph construction module, used to construct the graph model required for path planning based on environmental information; (2) a graph neural network inference module, used to perform path planning calculations on the graph model and predict the optimal or suboptimal safe flight route; (3) a safety constraint module, used to apply high-order flight safety constraints to ensure that the planned path meets obstacle avoidance and flight specification requirements; (4) a control command generation module, used to convert the planned route into executable control commands at the UAV's underlying level; and (5) a real-time scheduling module, used to coordinate the continuous operation of each module on the embedded platform according to the control cycle to achieve real-time closed-loop control of path planning. Each module is deployed on the UAV's onboard embedded computing platform (e.g., NVIDIA Jetson Xavier NX), and interacts with the external environment and UAV actuators through sensor data input and flight control command output. The following describes each module and process.

[0036] (1) System Architecture: This system adopts a modular and layered design, and the overall architecture is as follows: Figure 1 As shown. The UAV is equipped with an embedded AI computing platform (such as Jetson Xavier NX), which integrates a high-performance GPU and a multi-core CPU, enabling real-time execution of deep learning models and sensor data processing at the edge. The platform is configured with environmental perception sensors (such as LiDAR, cameras, ultrasonic radar, etc.), an inertial navigation unit (IMU), and a satellite positioning module to acquire maps of the UAV's surrounding environment, obstacle information, and its own attitude and position. The UAV path planning and control system framework of this invention is implemented in software on this embedded platform, divided into functional modules such as graph construction, graph neural network inference, safety constraints, control command generation, and real-time scheduling. Each module is connected through a predefined data interface: sensor-perceived data is preprocessed and transmitted to the graph construction module to generate an environmental graph structure; the graph neural network inference module obtains graph data and the current mission objective from the graph construction module, outputting the planned path or next waypoint; the safety constraint module verifies and adjusts the planning results; the control command generation module calculates control quantities based on the final path and sends them to the UAV flight control system; the real-time scheduling module acts as the system's brain, triggering the sequential execution of each module according to a preset control cycle and monitoring the operating status of each module to ensure real-time performance and reliability. The software architecture of the entire platform follows a real-time multi-threaded design. Each functional module is decoupled from the others but works collaboratively under the control of the scheduling module, thereby ensuring that UAV path planning operates efficiently and stably in a dynamic environment.

[0037] (2) Graph Construction Module: The graph construction module is responsible for abstracting the UAV's flight space into a graph structure model. First, based on the map information and obstacle distribution provided by the environmental perception sensors, the set of vertices of the graph is defined. Sum of edges Typically, regular grids, spatial grid points, or key waypoints can be used as graph nodes. An edge is established when adjacent nodes are connected and there are no obstacles blocking the way. Each node includes attribute information, such as spatial coordinates, altitude, and current wind risk value; each edge is assigned a weight to quantify the cost of the drone flying along that edge. The edge weights are set by comprehensively considering factors such as flight distance, energy consumption, and safety risks. For example, edge weights can be defined... The weight is the Euclidean distance between the two nodes. Weighted sum of security risk costs:

[0038] in Represents a node and The distance between, Indicates along the edge The risks and costs of flying (such as penalties for approaching obstacles or entering no-fly zones). The weighting coefficients are used to balance the impact of distance and risk on the overall cost.

[0039] By employing the above method, if a path passes through a high-risk area (such as an area with dense obstacles or the edge of a no-fly zone), the weight of the corresponding edge will be significantly increased, thus automatically avoiding it in subsequent path planning or only selecting it when there is no other option. The graph construction module is also responsible for dynamically adjusting the graph structure based on real-time updated sensor data: when new obstacles appear in the environment or the positions of existing obstacles change, graph nodes are added or deleted or edge weights are updated accordingly; if the target position changes or the UAV's current position is updated (e.g., navigation correction), the information of the start or end node is modified in the graph. This dynamic graph model enables the present invention to reflect changes in the UAV's environment in real time, providing an accurate and up-to-date search space for subsequent graph neural network inference.

[0040] (3) Graph Neural Network Inference Module (hereinafter referred to as Graph Policy Network): The Graph Neural Network (GNN) inference module is the core of this invention, used to efficiently calculate the optimal path of the UAV on the constructed environment graph. This module uses a pre-trained graph neural network model to transform the path planning problem into a neural network inference process on the graph. Specifically, the GNN model takes the graph structure and related features as input: the initial feature vector of each node may include its coordinates, altitude, indication of whether it is a target node, and local risk value; the edge features include the aforementioned weights. And possible path constraint labels (such as flags indicating whether an edge crosses a dangerous region). The model uses the entire graph. As the input structure of the neural network, the hidden state of the nodes is iteratively updated through multi-layer message passing and neighbor information aggregation. Formally, let... Indicates the first After layer iteration, the node Given the hidden vector representation, a message-passing layer of a GNN can be described as follows:

[0041] in For nodes The set of neighboring nodes, For the edge eigenvectors, For trainable weight matrix, It is a non-linear activation function. Used to contact neighbors The state and edge features are weighted and transformed (e.g., this can be implemented as follows). (The sum of the product of the edge weights).

[0042] The multi-layered graph neural network structure makes the final representation of each node... It incorporates graph information from its k-hop neighborhood. When the number of network layers and the receptive field are large enough, the starting node can perceive global information about the target node. After several iterations, the GNN output layer maps each node to a path evaluation value or a next action preference. For example, the network can output the estimated residual cost from each node to the target. This approximates the heuristic cost of traditional pathfinding. In this invention, the GNN model learns approximate reasoning ability for the shortest path on a graph through offline training. Its principle is equivalent to using a learning method to approximate the solution of the Bellman optimization equation. Ideally, the cost function of the optimal path... The Bellman equation should be satisfied:

[0043] in , Let represent the target node (terminus node) in the graph, and define the cost of this node as 0. The cost is 0. Graph Neural Networks (GNNs) approximate the above recursive relationship through a finite number of message passing layers: initially, the cost of the target node is set to the minimum; then, the low cost is propagated layer by layer from the target to neighboring nodes, and the cost assessment of each node is gradually adjusted. Finally, the output of the starting node provides the minimum cumulative cost estimate from the starting point to the target, while the outputs of neighboring nodes indicate the next direction selection. This parallel iterative computation based on GNNs can fully utilize GPU acceleration, achieving higher speeds than traditional single-source shortest path algorithms (such as Dijkstra's algorithm or A*). ） Real-time solutions on large-scale graphs are more efficient. Once the GNN inference module completes the path evaluation for the current environment, this invention can start from the starting point and greedily select the descent gradient direction based on the evaluation values ​​of neighboring nodes to gradually approach the target, obtaining an approximately optimal path sequence. Alternatively, the GNN output can directly provide the decision of the next neighboring node to go to, and the UAV can move towards the target step by step and re-evaluate the path at each step. Due to the fast speed of GNN inference, it can be frequently executed during UAV flight, thus achieving on-flight planning. Deploying an optimized GNN inference engine on an embedded platform (utilizing the GPU acceleration parallel computing capabilities of Jetson NX) makes each path planning calculation only take tens of milliseconds, meeting real-time requirements. In summary, through graph policy networks, UAVs can quickly calculate flight paths that meet safety constraints in a learning-based intelligent manner, which is particularly suitable for online path planning decisions in dynamically changing environments.

[0044] The safety constraint module, serving as the central safety control unit throughout the entire process, comprehensively utilizes task rules, airspace restrictions, obstacle information, and UAV platform performance constraints to provide unified safety management across all stages, including graph construction, GNN inference, and path output, demonstrating the high-order safety constraint features of this invention. This module filters, reweights, and masks nodes and edges at the layer level, preemptively eliminating obviously infeasible or high-risk routes. Simultaneously, it performs point-by-point checks on candidate paths at the trajectory level, triggering adaptive replanning when safety constraints are violated. Furthermore, it can collaborate with the HoCBF-QP safety optimization module to further reduce path-level constraints to control-level constraints, thereby ensuring flight safety at both the path planning and underlying control levels.

[0045] In the pre-planning stage, the safety constraint module can preprocess the environment graph generated by the graph construction module according to the flight mission and airspace regulations: remove all nodes located in no-fly zones or outside the flight airspace, and remove these inaccessible nodes from the node set; for edges that cross dangerous areas or do not meet flight performance limits (such as edges with too small a turning radius or too high a climb rate), they are directly prohibited from use, that is, removed from the edge set, or their weights are set to maximum values ​​to avoid being selected by the GNN.

[0046] During the planning process, the safety constraint module maintains a series of flight state constraint parameters, such as the minimum safe flight altitude. Maximum tilt angle Minimum obstacle distance And so on, and explicitly incorporate these constraints into the graph structure by dynamically adjusting edge weights or node states: on the one hand, if the shortest distance between a certain path and an obstacle is less than the safe distance. If the risk cost of the corresponding edge is significantly increased, it will be considered impassable until that edge is deemed untravelable. On the other hand, for maneuvers that involve the limits of UAV performance (such as sharp turns or steep climbs), the system will automatically increase the cost of the corresponding edge for that maneuver, or break it down into multiple milder actions to meet the requirements. , Constraints, etc.

[0047] In the post-planning phase, after the GNN inference module generates the initial path, the safety constraint module performs a global review of the entire candidate path, checking point by point for violations of airspace rules, obstacle safety distances, or platform performance limitations. If a waypoint in the path is found to be located on a no-fly zone boundary or too close to an obstacle, the module will trigger replanning: by increasing the weight penalty for the violation area or tightening relevant constraint parameters, the GNN inference module is called again to calculate the path until a new path that meets the predetermined safety conditions is obtained. For example, to ensure obstacle avoidance safety, this invention requires that the distance between any point on the UAV trajectory and any obstacle in the environment is not less than a safe distance. Otherwise, the path will be deemed unqualified and will be automatically adjusted or replanned.

[0048] In addition, the safety constraint module maintains a series of flight state constraint parameters, such as the minimum safe flight altitude. Maximum tilt angle Minimum obstacle distance During path planning, these constraints can be reflected by dynamically adjusting edge weights or node states: on the one hand, if a certain path approaches an obstacle within a safe distance... Then the risk cost of the corresponding edge The system may increase the cost of a particular maneuver, potentially rendering it impassable. Conversely, for maneuvers exceeding the drone's performance limits (such as sharp turns or steep climbs), the system will tend to increase the cost of the maneuver or break it down into multiple gentler maneuvers to meet the requirements. Restrictions are imposed. After the GNN inference module generates an initial path, the safety constraint module reviews the entire candidate path, checking for violations point by point. If a waypoint in the path is found to be located on a no-fly zone boundary or too close to an obstacle, the module will trigger replanning: this can be done by adjusting relevant constraint parameters (such as further increasing the weight penalty for violating areas) and recalculating using the GNN module until a new path that meets the safety conditions is obtained. For example, to ensure obstacle avoidance safety, this invention requires that the distance between any point on the UAV trajectory and any obstacle in the environment is not less than a safe distance. :

[0049] in A collection of obstacles. The set of drone trajectory points obtained from the planning, Represents trajectory points and obstacles The distance. For example, for no-fly zones containing civil aviation routes, requirements... It does not contain any points that fall within this airspace.

[0050] The safety constraint module ensures, through the aforementioned constraints, that the planned flight path is sufficiently isolated from no-fly zones and obstacles in space, conforms to the platform performance limitations of the UAV kinematically, and meets flight rules and safety redundancy at the mission level (e.g., considering whether the remaining battery power is sufficient to complete the path and reserving a return margin). This module can also integrate a dynamic safety mechanism: continuously monitoring changes in the surrounding environment during UAV flight, if a new dynamic obstacle (other aircraft, etc.) suddenly appears, the constraints are updated in real time, and the scheduling module is notified to trigger path replanning, thereby promptly correcting the flight path to avoid the newly emerging threat. Through the multi-layered safeguards of the safety constraint module, this invention significantly improves the safety and reliability of the path planning results, enabling UAVs to complete missions in complex environments while adhering to flight safety regulations.

[0051] (5) Control Command Generation Module: The control command generation module is responsible for converting the planned path into a sequence of low-level commands executable by the UAV flight control system, thereby putting the path planning results into practice. Generally, the path output by the graph policy network is a set of spatially discrete waypoints. (in Starting point For the target point, the middle (a path inflection point), or an indication of the next target point. The control command generation module first smooths and interpolates the path points appropriately to avoid overly abrupt turns that could lead to flight instability. Then, based on the UAV's current state (position, velocity, attitude) and the location of the next target point, it calculates the required control output. For example, it can calculate the UAV's current position pointing towards the next path point. The unit direction vector, combined with the desired flight speed Generate speed command Meanwhile, based on the current location and The azimuth difference determines the desired heading angle And climb / descent rates. For quadcopter UAVs, these high-level motion commands are further calculated by the flight control system into specific motor speeds and control surface deflections. The control command generation module can also employ classic control algorithms such as PID control and model predictive control to convert position deviations into attitude and throttle adjustments. For example, through altitude control loops and horizontal position control loops, it gradually approaches the path point to maintain the planned trajectory. Whenever the UAV reaches the vicinity of the current target path point, the module will switch to the next path point, continuously guiding the UAV toward the final target. It is worth noting that the control command generation of this invention is performed online in real time: within each control cycle, the control output is updated according to the latest planned path and flight status, thus possessing the ability to adapt to environmental changes. Whenever the UAV reaches the vicinity of the current target path point, the module will switch to the next path point, continuously guiding the UAV toward the final target. In this invention, the control command generation module is controlled by the real-time scheduling and resource management module according to a fixed control cycle. T Periodic Triggering Online Operation: Within each control cycle, the control command generation module reads the nearest reference waypoint on the current planned path from the graph strategy planning module, and obtains the real-time flight status of the UAV (including position, speed, and attitude) from the sensing and control interface module. Then, it calculates new flight control commands based on the preset trajectory tracking control law (such as position-speed-attitude graded PID control or equivalent nonlinear attitude control law) and sends them to the UAV flight control system.

[0052] When a path replanning is triggered due to environmental changes during flight, the graph strategy planning module updates the currently valid path in shared memory. The control command generation module automatically reads the latest path in the next control cycle and calculates new control commands accordingly, achieving a seamless connection and smooth transition between the old and new paths. Through this periodic online calculation and closed-loop update mechanism, the control command generation module establishes a direct connection between the algorithm planning layer and the physical control layer, enabling the planning results to be reflected in the actual flight maneuvers of the UAV in a stable and controllable manner, and to adaptively respond to environmental changes. If the path changes due to replanning during flight, the control command generation module will seamlessly connect to the new path and calculate new control commands, ensuring a smooth transition for the UAV to the new route. Through this module, a direct connection is established between the algorithm planning layer and the physical control layer, ensuring that the planning results are reflected in the actual flight maneuvers of the UAV in a stable and controllable manner.

[0053] (6) Real-time scheduling module: The real-time scheduling module is the central hub of the system, responsible for coordinating the execution of various functional modules at a predetermined frequency on the embedded platform to form a closed-loop real-time path planning and control process. This module typically adopts a real-time multi-task scheduling mechanism (e.g., based on thread priority or real-time operating system kernel) to ensure that critical tasks (such as obstacle avoidance planning and control output) are completed within a strict time limit. During operation, the scheduling module triggers the following steps in a periodic loop: First, at the beginning of each cycle, the sensor data acquisition and preprocessing thread is started to update the latest information on the UAV's current position, attitude, and surrounding environment; then, the graph construction module is called to integrate the new sensor data into the environmental graph model and update the information of nodes and edges; then, the graph policy network is triggered to perform path planning calculations on the updated graph to obtain the suggested route (or next action decision) for the UAV from the current state to the target point; then, the safety constraint module is run to review the planning results and make necessary corrections; finally, the control command generation module is called to calculate control commands based on the finally confirmed path and send them to the UAV for execution through the flight control interface. The scheduling module of this invention pre-sets strict time slices and sequences for the above process to ensure that the entire loop is completed within a limited control cycle. Completed within the timeframe. For example, the control cycle can be... The timeframe is set to 100ms (i.e., a 10Hz refresh rate): sensor updates and graph construction may take 20ms, GNN inference computation 50ms, and safety checks and instruction generation a total of 20ms, leaving approximately 10ms as a scheduling margin. The scheduling module ensures that no module times out through fine-grained time management: if a module fails to complete within the specified time in a given cycle, the scheduling module can take measures such as discarding the planning results of that cycle or simplifying the processing to avoid blocking the execution of subsequent cycles. Simultaneously, the parallel capabilities of multi-core CPUs and GPUs are fully utilized—for example, sensor data acquisition and preprocessing can be executed in parallel by CPU threads, while GNN inference is accelerated by the GPU, thereby shortening the overall cycle latency. Data dependencies between modules are strictly maintained by the scheduling module: subsequent GNN inference will only begin after the graph construction module has produced the latest graph data; similarly, safety checks on the planned path must be completed before sending new control instructions. This synchronization mechanism ensures system state consistency and avoids decision-making errors caused by asynchronous updates. The real-time scheduling module is also responsible for monitoring system operation, such as CPU / GPU utilization, battery level, and flight control feedback. If any anomalies are detected (e.g., excessive computational latency or hardware overload), it will promptly issue an alarm or adjust the cycle to ensure flight safety. Through this scheduling strategy, this invention achieves closed-loop real-time operation from environmental perception and path planning to control execution: the UAV can continuously perceive environmental changes and adjust its flight path on the embedded platform at millisecond intervals, truly achieving "real-time planning and instant obstacle avoidance." This is particularly important for scenarios requiring high dynamic response (such as low-altitude flight in urban areas to avoid pedestrians and buildings), ensuring the safe and autonomous flight of the UAV in complex environments.

[0054] In summary, this invention, through the organic coordination of the aforementioned modules, realizes a real-time path planning system deployable on an UAV embedded platform. This system utilizes the parallel computing and learning capabilities of graph neural networks to efficiently solve for near-optimal paths in dynamic environments, while simultaneously employing high-order safety constraints to strictly ensure the safety and compliance of flight paths. The entire planning-control process runs in real-time closed-loop mode on embedded hardware such as Jetson NX, enabling UAVs to autonomously complete path planning and obstacle avoidance in complex environments without relying on ground stations. Experimental tests show that the proposed solution can achieve millisecond-level path updates with minimal computational overhead, promptly adjusting routes when the environment changes, successfully avoiding collisions and intrusions into prohibited areas; compared to traditional planning methods, it significantly improves the safety and real-time performance of UAV mission execution. Therefore, the UAV real-time path planning system and method provided by this invention possess good robustness and practical value, and can be widely applied in fields requiring autonomous obstacle avoidance and dynamic route planning, such as UAV urban low-altitude logistics delivery, emergency rescue reconnaissance, and inspection patrols.

[0055] The above-described embodiments are merely one implementation of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention should be determined by the appended claims.

Claims

1. A UAV path planning method based on GNN and high-order safety constraints, characterized in that, include: Step 1: Heterogeneous risk graph modeling and graph neural network planning; First, the navigation scenario of the UAV is represented as a heterogeneous directed graph. The nodes of the graph include task nodes, obstacle nodes, and constraint nodes. A feature vector containing a risk descriptor of a high-order control barrier function is designed for each node. The edges of the graph represent the relationships between nodes, and edge attributes are attached for subsequent message passing. Based on this, a graph neural network (GNN) is used to perform L-layer message passing and feature updating on the heterogeneous graph. A risk-aware attention mechanism is introduced in the message passing stage. Through L-layer iterative message passing and state updating, the graph neural network extracts global semantics and risk information, and finally outputs interpretable decision results in the readout stage. Step 2: Embed a high-order control barrier function (QP) layer; For the original control command output by the graph neural network, a differentiable HoCBF-QP optimization layer is introduced between the output of the policy network and the actual control command executed by the UAV actuator; The HoCBF-QP optimization layer takes the original control command output by the policy network as input, solves the quadratic programming with high-order control barrier function constraints online, performs minimum-amplitude safety correction on the control command, and outputs actuator control commands that satisfy the safety constraints, which are used to correct the control command in real time to meet the safety constraints; Step 3: Enable runtime HoCBF security shield; A lightweight runtime HoCBF-QP safety shield is set up before the final control execution. This shield differs from the embedded QP in that it operates at the final output stage to strictly ensure that all safety constraints are satisfied before execution. The runtime safety shield uses the actuator control instructions output by the embedded QP as reference input to solve a QP projection problem, ensuring that a final control command satisfying all safety constraints can be found in each control cycle. ； When the QP is feasible in a certain state, the control barrier function theory can guarantee that the UAV state will not deviate from the safe set, thus completely preventing accidents; once the final control command is received... ， It is sent to the flight controller for execution, thereby completing the safety control output for the current control cycle; Step 4: Control cycle DAG scheduling and real-time execution; The computational task of the entire control loop is modeled as a directed acyclic graph and executed in parallel on heterogeneous multi-core processors using a real-time scheduling strategy. The graph strategy planning module, the HoCBF-QP safety optimization module, and the real-time scheduling and resource management module work together to achieve real-time and safe UAV navigation control. The scheduling ensures that the deep learning inference and optimization solution layers are executed at a high frequency, while the safety layer ensures that the output of the planning strategy is always within the safe and feasible domain.

2. The UAV path planning method based on GNN and high-order safety constraints according to claim 1, characterized in that, Step 1 is as follows: Interpretable decision results include: the UAV's next anchor point trajectory segment, the mission priority DAG, and the control intent map; The attention mechanism is specifically as follows: in the first... In the layer, for nodes Each neighbor node Calculate attention score ;in, Indicates the first Layer time center node eigenvectors, Indicates the first Layer-time neighbor nodes eigenvectors, Indicates that by node Pointing to node The edge attribute is used to encode the topological or geometric relationship between the two. A function that combines node features and edge attributes to measure the performance of a node. For nodes Influence weights; then influence on neighbors The influence is normalized using softmax to obtain the attention weights. Where k represents the index of the attention head, Represents a node The set of neighbors, that is, the set of nodes in the graph structure. There exist all nodes directly connected by edges, where, This indicates that in the graph neural network... In the layer, there are nodes Point to its neighbor node The edge feature vectors are used to encode the geometric or topological relationship between the two and serve as input for attention calculation and message passing in this layer.

3. The UAV path planning method based on GNN and high-order safety constraints according to claim 1, characterized in that, Steps 2 and 3 are as follows: Step 2: The actuator refers to the execution mechanism on the UAV body; the actuator command refers to the low-level control quantity sent to the aforementioned actuator through the flight control system; this QP layer is based on the aforementioned higher-order control barrier function condition. A constrained optimization problem is constructed with the objective of minimizing the modifications to the original command, thereby achieving minimally invasive security assurance; let... , ,in, Let U be the state vector of the UAV. This is the control input vector for the UAV. For the system's drift term, To control the input matrix, For safety functions Lie derivative along vector field f, For safety functions Lie derivative along vector field g, This indicates the non-negativity of the second-order auxiliary barrier function obtained recursively from the higher-order control barrier function. It is used to characterize a system of relative order 2 where the state remains within the safe set throughout two integration steps; For the corresponding class The gain parameter, used to adjust the contraction speed and conservatism of the second-order security constraint, is determined by the current state. The calculated coefficients allow the safety constraints to be expressed as linear inequalities. In each control cycle, solve the following quadratic programming problem to obtain the corrected safety control command. : in The objective function in the above equation is the correction amount for the original control input. It ensures that it is as close as possible to the original control commands. Constraints This ensures that the aforementioned HoCBF security inequality holds, and the modified security control command... Not exceeding the actuator's permissible control range For cases with a small number of constraints, the above QP has an analytical KKT solution, and the optimal correction Δu can be directly calculated, thus quickly obtaining the desired result. For cases with multiple constraints, a differentiable QP solver is used to obtain the solution and allow gradients to propagate through the layer to penalize unsafe initial control command preferences during the policy training phase, thereby guiding the policy output to be biased toward the safe and feasible region. Step 3: Runtime security shield outputs security control commands via embedded QP Using this as a reference input, solve a small-scale QP projection problem: in, This represents the optimal solution to the QP projection problem, i.e., the HoCBF-QP shield in its current state during runtime. The final safety control command obtained is then sent to the UAV flight control system for execution. Indicates the current state The calculated linear constraint coefficient matrix associated with all HoCBF safety constraints, This is a set of indices for HoCBF safety inequality constraints that are active or need to be checked in the current state. Each row of this matrix corresponds to a safety constraint pair with the control input. The gradient; Indicates the first The HoCBF safety inequality constraint in the current state The constant term / right-hand term below is used to characterize the "residual margin" of the constraint when no control input is applied; This represents the set of all HoCBF safety inequality constraints that need to be satisfied in the current state.

4. The UAV path planning method based on GNN and high-order safety constraints according to claim 1, characterized in that, Step 4 is as follows: A control cycle includes several tasks such as sensor data acquisition and preprocessing, graph construction, GNN inference computation, QP solving, security checks, and sending control commands for execution. A Directed Acyclic Graph (DAG) is built based on the dependencies between these tasks. Using this DAG, tasks without direct dependencies are scheduled for parallel execution, thus fully utilizing the computing resources of the CPU and GPU. To reduce resource contention and scheduling overhead, thread binding and asynchronous execution techniques are employed: critical threads are fixed to run on designated CPU cores, avoiding scheduling and switching between multiple cores; lock-free producer-consumer queues are used to transfer data between GPU inference threads and CPU control threads, reducing synchronization blocking; and CUDA asynchronous streaming and event mechanisms are used to overlap GPU computation and data copying with CPU computation.

5. A UAV path planning system based on GNN and high-order safety constraints, characterized in that, include: The graph construction module is used to build the graph model required for path planning based on environmental information. The graph neural network inference module is used to perform path planning calculations on the graph model and predict the optimal or suboptimal safe flight route. The safety constraint module is used to apply high-order flight safety constraints to ensure that the planned path meets obstacle avoidance and flight specification requirements; the control command generation module is used to convert the planned route into control commands that can be executed by the UAV at the underlying level. The real-time scheduling module is used to coordinate the continuous operation of each module according to the control cycle on the embedded platform to realize real-time closed-loop control of path planning; each module is deployed on the embedded computing platform on the UAV and realizes interaction with the external environment and UAV actuators through sensor data input and flight control command output. System Architecture: This system adopts a modular, layered design. The UAV is equipped with an embedded AI computing platform, which integrates a high-performance GPU and a multi-core CPU to run deep learning models and process sensor data in real time at the edge. The platform is equipped with environmental perception sensors, an inertial navigation unit (IMU), and a satellite positioning module to acquire maps of the UAV's surrounding environment, obstacle information, and its own attitude and position. The UAV path planning and control system framework is implemented in software on this embedded platform, divided into modules for graph construction, graph neural network inference, safety constraints, control command generation, and real-time scheduling. Each module is connected through a predefined data interface: sensor data is preprocessed and then transmitted to... The graph construction module generates the environmental graph structure; the graph neural network inference module obtains graph data and the current mission objective from the graph construction module, and outputs the planned path or the next waypoint; the safety constraint module verifies and adjusts the planning results; the control command generation module calculates the control quantity based on the final path and sends it to the UAV flight control system; the real-time scheduling module acts as the brain of the system, triggering the sequential execution of each module according to the preset control cycle, and monitoring the operating status of each module to ensure real-time performance and reliability; the software architecture of the entire platform follows a real-time multi-threaded design, with each functional module being decoupled from each other while working collaboratively under the control of the scheduling module, thereby ensuring that the UAV path planning operates stably in a dynamic environment.

6. The UAV path planning system based on GNN and high-order safety constraints according to claim 5, characterized in that, The graph construction module is specifically as follows: The graph construction module is responsible for abstracting the UAV's flight space into a graph structure model; first, based on the map information and obstacle distribution provided by the environmental perception sensors, it defines the set of vertices of the graph. Sum of edges Regular grids, spatial grid points, or key waypoints are used as graph nodes. An edge is established when adjacent nodes are connected and there are no obstacles blocking the way. Each node is associated with attribute information; each edge is assigned a weight to quantify the cost of the drone flying along that edge; the edge weights are defined by comprehensively considering flight distance, energy consumption, and safety risk factors. The weight is the Euclidean distance between the two nodes. Weighted sum of security risk costs: in Represents a node and The distance between, Indicates along the edge The risks and costs of flying The weighting coefficients are used to balance the impact of distance and risk on the overall cost; In this way, if a path passes through a high-risk area, the weight of the corresponding edge will be significantly increased, so that it will be automatically avoided in subsequent path planning or only selected when there is no other way; the graph construction module is also responsible for dynamically adjusting the graph structure according to the real-time updated sensor data: when new obstacles appear in the environment or the position of existing obstacles changes, graph nodes are added or deleted or edge weights are updated accordingly. If the target location changes or the drone's current location is updated, the information of the starting or ending node in the graph is modified. This dynamic graph model enables this module to reflect changes in the drone's environment in real time, providing an accurate and up-to-date search space for subsequent graph neural network inference.

7. The UAV path planning system based on GNN and high-order safety constraints according to claim 5, characterized in that, The graph neural network inference module is specifically as follows: The graph neural network (GNN) inference module is used to efficiently calculate the optimal path for a UAV on a constructed environment graph. This module employs a pre-trained GNN model, transforming the path planning problem into a neural network inference process on the graph. The GNN model takes the graph structure and related features as input: the initial feature vector of each node includes its coordinates, altitude, indication of whether it is a target node, and local risk value; the edge features include the aforementioned weights. And possible path constraint labels; the model uses the entire graph As the input structure of the neural network, the hidden state of the nodes is iteratively updated through multi-layer message passing and neighbor information aggregation; let... Indicates the first After layer iteration, the node Given the hidden vector representation, a message-passing layer of a GNN can be described as follows: in For nodes The set of neighboring nodes, For the edge eigenvectors, For trainable weight matrix, It is a non-linear activation function. Used to tell neighbors The state and edge features are weighted and transformed; The multi-layered graph neural network structure makes the final representation of each node... It integrates graph information within its k-hop neighborhood; when the number of network layers and the receptive field are large enough, the starting node can perceive the global information of the target node; After several iterations, the GNN output layer maps each node to a path evaluation value or a next action preference. GNN models learn to approximate shortest paths on a graph through offline training. This is essentially equivalent to using a learning method to approximate the solution of the Bellman optimization equation; the cost function of the optimal path... The Bellman equation should be satisfied: in , Let represent the target node in the graph, and define the cost of this node as 0. The cost is 0; the graph neural network approximately implements the above recursive relationship through a finite layer of message passing: initially, the cost of the target node is set to the minimum, the low cost is propagated from the target to the neighboring nodes layer by layer, and the cost value assessment of each node is gradually corrected; finally, the output of the starting node will give the minimum cumulative cost estimate from the starting point to the target, and the output of the neighboring nodes will indicate the direction selection of the next step. Once the GNN inference module completes the path evaluation for the current environment, starting from the origin, it greedily selects the descent gradient direction based on the evaluation values ​​of neighboring nodes to gradually approach the target, obtaining an approximately optimal path sequence; or, the output of the GNN directly gives the decision of the next neighboring node to go to, and the UAV moves towards the target step by step accordingly and re-evaluates the path at each step.

8. The UAV path planning system based on GNN and high-order safety constraints according to claim 5, characterized in that, The safety constraint module specifically includes: The safety constraint module serves as the central safety control unit throughout the entire process. It comprehensively utilizes task rules, airspace restrictions, obstacle information, and UAV platform performance constraints to provide unified safety management for each stage, including graph construction, GNN inference, and path output. On one hand, this module filters, reweights, and masks nodes and edges at the layer level to preemptively eliminate obviously infeasible or high-risk routes. On the other hand, it checks candidate paths point by point at the trajectory level, triggering adaptive replanning when safety constraints are violated. Furthermore, it works in conjunction with the HoCBF-QP safety optimization module to further reduce path-level constraints to control-level constraints, thereby ensuring flight safety at both the path planning and underlying control levels. In the pre-planning stage, the safety constraint module preprocesses the environmental graph generated by the graph construction module according to the flight mission and airspace regulations: it removes all nodes located in no-fly zones or outside the flight airspace, and removes these inaccessible nodes from the node set; for edges that cross dangerous areas or do not meet flight performance restrictions, they are directly prohibited from use, that is, removed from the edge set, or their weights are set to maximum values ​​to avoid being selected by the GNN. During the planning process, the safety constraint module maintains a series of flight state constraint parameters and explicitly integrates these constraints into the graph structure by dynamically adjusting edge weights or node states: On the one hand, if the shortest distance between a certain path and an obstacle is less than the safe distance... If the risk cost of the corresponding edge is significantly increased, the edge will be considered impassable. On the other hand, for maneuvers that involve the performance limits of the UAV, the system will automatically increase the cost of the corresponding edge of the maneuver or break it down into multiple actions to meet the flight state constraints. In the post-planning phase, after the GNN inference module generates the initial path, the safety constraint module performs a global review of the entire candidate path, checking point by point for violations of airspace rules, obstacle safety distances, or platform performance limitations. If a waypoint in the path is found to be located on the boundary of a no-fly zone or too close to an obstacle, the module will trigger replanning: by increasing the weight penalty for the violation area or tightening the relevant constraint parameters, the GNN inference module is called again to calculate the path until a new path that meets the predetermined safety conditions is obtained. To ensure obstacle avoidance safety, the distance between any point on the drone's trajectory and any obstacle in the environment must not be less than a safe distance; otherwise, the path is deemed unqualified and will be automatically adjusted or replanned. The safety constraint module ensures, through the aforementioned constraints, that the planned flight path is sufficiently isolated from no-fly zones and obstacles in space, conforms to the platform performance limitations of the UAV in kinematics, and meets flight rules and safety redundancy at the mission level. The safety constraint module also integrates a dynamic safety mechanism: it continuously monitors changes in the surrounding environment during the UAV's flight, updates the constraints in real time, and notifies the scheduling module to trigger path replanning, thereby promptly correcting the flight path to avoid newly emerging threats, enabling the UAV to complete its mission in compliance with flight safety regulations in complex environments.

9. The UAV path planning system based on GNN and high-order safety constraints according to claim 5, characterized in that, The control command generation module is specifically as follows: Control command generation module: This module is responsible for converting the planned path into a sequence of low-level commands executable by the UAV flight control system, thereby putting the path planning results into practice. The path output by the graph policy network is a set of spatially discrete waypoints. ,in Starting point For the target point, the middle This indicates a path inflection point or a point indicating the next target. The control command generation module first smooths and interpolates the path points appropriately to avoid overly abrupt turns that could lead to flight instability. Then, based on the UAV's current state and the location of the next target point, it calculates the required control output. Simultaneously, based on the current position and... The azimuth difference determines the desired heading angle And climb / descent rates; for quadcopter UAVs, these high-level motion commands are further calculated by the flight control system into specific motor speeds and control surface deflections; the control command generation module also uses PID control and model predictive control algorithms to convert position deviations into attitude and throttle adjustments; control command generation is performed in real time online: in each control cycle, the control output is updated according to the latest planned path and flight status, thus possessing the ability to adapt to environmental changes; whenever the UAV reaches the vicinity of the current target path point, the module will switch to the next path point, continuously guiding the UAV toward the final target; the control command generation module is controlled by the real-time scheduling and resource management module according to a fixed control cycle. T Periodic Triggering of Online Operation: In each control cycle, the control command generation module reads the nearest reference waypoint on the current planned path from the graph strategy planning module, obtains the real-time flight status of the UAV from the sensing and control interface module, and then calculates new flight control commands according to the preset trajectory tracking control law and sends them to the UAV flight control system. When a path replanning is triggered due to environmental changes during flight, the graph strategy planning module updates the current valid path in shared memory. The control command generation module automatically reads the latest path in the next control cycle and calculates new control commands accordingly, achieving seamless connection and smooth transition between the old and new paths. Through the above-mentioned periodic online calculation and closed-loop update mechanism, the control command generation module establishes a direct connection between the algorithm planning layer and the physical control layer, enabling the planning results to be reflected in the actual flight actions of the UAV in a stable and controllable manner, and to generate adaptive responses to environmental changes.

10. The UAV path planning system based on GNN and high-order safety constraints according to claim 5, characterized in that, The control command generation module is specifically as follows: Real-time scheduling module: The real-time scheduling module is the central hub of the system, responsible for coordinating the execution of various functional modules on the embedded platform at a predetermined frequency, forming a closed-loop real-time path planning and control process. This real-time scheduling module adopts a real-time multi-task scheduling mechanism to ensure that critical tasks are completed within strict time limits. During operation, the scheduling module triggers the following steps in a periodic loop: First, at the beginning of each cycle, the sensor data acquisition and preprocessing thread is started to update the latest information on the UAV's current position, attitude, and surrounding environment; then, the graph construction module is called to fuse the new sensor data into the environmental graph model and update the information of nodes and edges. Then the graph policy network is triggered to perform path planning calculations on the updated graph and obtain the suggested route or next action decision for the drone from the current state to the target point. The safety constraint module is then run to review the planning results and make necessary corrections. Finally, the control command generation module is called to calculate control commands based on the final confirmed path and send them to the UAV for execution via the flight control interface. The real-time scheduling module pre-sets strict time slices and sequences for the above processes to ensure that the entire loop operates within a defined control period. The scheduling module ensures that each module does not time out through fine-grained time management: if a module fails to complete within the specified time in a certain cycle, the scheduling module discards the planning results of that cycle or simplifies the processing to avoid blocking the execution of subsequent cycles; at the same time, the parallel execution of multi-core CPUs and GPUs, with GNN inference accelerated by GPUs, shortens the overall cycle latency; the data dependencies between modules are strictly maintained by the scheduling module: subsequent GNN inference will only start after the graph construction module produces the latest graph data; the safety verification of the planned path must be completed before sending new control commands; the real-time scheduling module is also responsible for monitoring the system's operating status, and will promptly issue alarms or adjust the cycle to ensure flight safety once an anomaly is detected; through the above scheduling strategies, a closed-loop real-time operation from environmental perception, path planning to control execution is achieved.

Images