Shielded UAV Path Planning for Urban Collision Constraints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current UAV path planning methods, particularly reinforcement learning algorithms, struggle to guarantee the safety of UAV operations in urban airspace due to the complexity of urban environments and the lack of hard constraints to ensure air collision and ground impact risks are minimized.
Innovation Solution
The development of a safe reinforcement learning algorithm called shield-DDPG, which incorporates a shield module based on linear temporal logic to verify the safety of actions and ensure compliance with safety specifications, thereby preventing air collisions and ground impacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If reinforcement learning algorithms (DQN, DDPG) are used for UAV path planning, then the algorithm is highly versatile and can interact with the environment to obtain optimal actions, but it is difficult to guarantee the safety of the final algorithm output with a mathematically proven solution
Solution Approach 1:
The patent introduces a shield module as an intermediary component between the reinforcement learning algorithm and the environment. This shield module acts as a mediator that verifies the safety of actions generated by the RL algorithm before execution, providing mathematical safety guarantees while preserving the versatility of the original algorithm. The shield module checks whether actions satisfy safety specifications (avoiding collisions with buildings, staying within no-fly zones, maintaining safe distances) and blocks unsafe actions, thus resolving the contradiction between algorithm versatility and safety reliability.
2Productivity
If linear programming algorithms (MILP) are used, then the computation is relatively simple and efficient, but the algorithm cannot quickly deal with the problem of a large number of decision variables
Solution Approach 1:
The patent segments the path planning problem into two distinct parts: (1) the reinforcement learning agent that handles high-dimensional decision variables and learns optimal policies through interaction, and (2) the shield module that handles safety verification of individual actions. This segmentation allows the RL algorithm to efficiently handle complex urban environments with many decision variables, while the shield module ensures safety constraints are met, thus resolving the contradiction between computation efficiency and handling complexity.
3Manufacturing precision
If graph search algorithms (Dijkstra, A*, RRT) are used, then the algorithm is suitable for solving the shortest path planning problem, but the efficiency decreases as the number of nodes traversed by the algorithms increases
Solution Approach 1:
The patent replaces traditional graph search algorithms (which systematically traverse nodes) with a reinforcement learning approach that learns optimal paths through environmental interaction and reward-based learning. The shield module then substitutes the role of graph search safety checks by verifying RL-generated actions against safety specifications. This substitution maintains path planning accuracy while significantly improving efficiency in complex urban environments with numerous nodes.
4Ease of manufacture
If intelligent optimization algorithms (PSO, ACO, GA) are used, then the algorithm is widespread and convenient for parallel processing, but the algorithm is prone to fall into local-optimum in some complex situations
Solution Approach 1:
The patent implements a feedback mechanism where the shield module continuously monitors and verifies the safety of actions generated by the reinforcement learning algorithm. This feedback loop ensures that even if the RL algorithm explores suboptimal paths during learning, the safety constraints are always satisfied. The feedback from the environment (rewards and penalties) guides the RL algorithm toward optimal solutions while the shield provides additional safety feedback, resolving the contradiction between parallel processing convenience and solution optimality.
Data Source
AI summary
The present invention discloses a method for UAV path planning in urban airspace based on a safe reinforcement learning (RL) algorithm called shield-DDPG, which combines a shield model with a DDPG algorithm and pertains to the field of UAV technologies. The method takes an attractive force from the destination point into account when an action is selected, which improves the convergence speed of the algorithm and also improve the efficiency of UAV path planning. More importantly, the method provided by the present invention can effectively verify safety of an action in terms of the air collision risk and the ground impact risk, and ensure that a final output of the algorithm is a safe optimal solution. Therefore, the present invention can effectively solve the problem that when the RL algorithm is used for UAV path planning, it is difficult to ensure the safety of the learning or execution process due to the lack of hard constraints.