A conflict-free route planning method based on airspace grid

Abstract

The present invention proposes a conflict-free route planning method based on an airspace grid, which relates to the field of path planning. Specifically, the following steps are as follows: first, an aircraft deploys an airborne platform to collect four-dimensional trajectory data and send the flight start point, end point, and obstacle information to a route planning module; the route planning module constructs a four-dimensional space-time airspace grid based on the coordinate relationships of different three-dimensional spaces on each time slice. Then, based on the airspace grid, a four-dimensional space-time transit value graph is iteratively calculated using reinforcement learning; a conflict-free path with the lowest cost is selected from the transit value graph and transmitted to a path smoothing module; a route smoothing algorithm smoothes the sequence of route control points based on the physical and dynamic characteristics of the aircraft, and finally the smoothed result is sent to the airborne platform, which controls the flight of the aircraft according to the route. The present invention derives the transit value graph through a value iteration network, greatly reducing system operation time and improving effectiveness.

Description

Technical Field

[0001] The present invention relates to the field of path planning, and in particular to a conflict-free route planning method based on airspace grids. Background Art

[0002] Route planning means: by using various path planning algorithms, after a given starting point and end point, a flight route with short distance, low time consumption and no conflicts is planned for the aircraft. Traditional path planning methods such as Dijkstra algorithm, ant colony algorithm or A* algorithm focus on planning the optimal path with the shortest flight distance and the least time consumption; however, when there are many obstacles in the airspace and the obstacles change dynamically over time, the above path planning algorithms are not very good and may even plan conflicting paths; and traditional path planning methods do not take into account the importance of path smoothness for route planning. When the route has too sharp corners, it will seriously affect the riding experience of the people in the aircraft and even affect air traffic safety. Therefore, it is very necessary to invent a method that can adapt to the dynamic changes of obstacles and can plan smooth and conflict-free routes.

[0003] In the existing technology, there is a low-altitude rescue aircraft trajectory planning method based on a three-dimensional airspace grid, such as application number 201510250318.X, which solves the conflict-free trajectory planning of multiple aircraft in low-altitude complex terrain, weather distribution and aircraft performance under aviation rescue, and provides a basis for the aircraft flight plan arrangement of the aviation emergency rescue command system. However, when planning the path, the constraints considered by this method only include fixed obstacles such as terrain and weather, and do not take into account factors when obstacles change dynamically over time; that is, the path planning is only performed in the topological structure of the static map of the obstacle position at the current moment. When the obstacle position changes at the next moment, the result of the previous round of path planning becomes invalid and needs to be re-planned.

[0004] Application number 201710255262.6 describes a joint search method for multi-objective path planning for UAVs in urban low-altitude environments. The method first plans a minimally costed path from a starting point to a destination offline. Then, when an unknown obstacle is detected, it uses online search to plan a modified trajectory to avoid dynamic obstacles. This online search method, with its smaller search space, enables rapid online replanning of a safe path for the UAV, thus meeting the real-time requirements for path planning. While this method takes into account the impact of dynamic obstacles, it only performs online search and planning of a new path when a dynamic obstacle is detected in real time, significantly compromising real-time performance and failing to consider the optimal path from a global perspective. For path planning around dynamically changing obstacles, a new path is replanned online after the change is detected. Therefore, the resulting path planning by this technology is not globally optimal and may even result in unnecessary starts, stops, and avoidance maneuvers, reducing driving efficiency and the overall passenger experience.

[0005] A cross-domain heterogeneous cluster path planning method based on reinforcement learning with application number 202110391296.4 constructs a Markov decision process and reward function for cross-domain heterogeneous cluster maneuvers, and uses the MADDPG algorithm to solve the cluster individual maneuvering strategy, thereby realizing path planning for cross-domain heterogeneous clusters. As long as the cluster individuals have relevant sensors, path planning can be achieved through the method of the present invention. Under this method, different aircraft from airspace, sea area, land area and other domains can jointly perform path planning, thereby realizing "cross-domain heterogeneous path planning". However, this method uses a fully connected neural network for path planning, so the model needs to be retrained for different application scenarios, and cannot adapt to environments with relatively high real-time requirements. Summary of the Invention

[0006] To address the above issues, the present invention discloses a conflict-free route planning method based on an airspace grid. After receiving data from an airborne platform, the method constructs a four-dimensional spacetime airspace grid by adding a time dimension to the three-dimensional space. The method then calculates a four-dimensional spacetime traffic value graph, constructs an optimal conflict-free path based on the value graph, and finally outputs the final trajectory planning result after processing by a path smoothing system. The present invention comprehensively considers obstacle information at each time step, performs value iteration in four-dimensional spacetime, and obtains a globally optimal conflict-free path in a low-cost manner. Furthermore, the method adds path smoothing processing to make the aircraft's flight trajectory more consistent with physical characteristics and dynamic properties.

[0007] The specific steps are as follows:

[0008] Step 1: The aircraft deploys the airborne platform to collect 4D trajectory data, processes the flight start point, end point and obstacle information into a dynamic point sequence, and sends it to the route planning module;

[0009] Step 2: The route planning module constructs a four-dimensional spacetime airspace grid based on the coordinate relationship of different three-dimensional spaces on each time slice.

[0010] The specific construction process is:

[0011] The starting and ending points of the aircraft are used as boundaries to generate a three-dimensional target airspace, and then the three-dimensional target airspace is divided into several four-dimensional grids. The length, width and height of each four-dimensional grid unit are set according to the standard interval distance of the aircraft, and the aircraft's track and the position of the obstacle are mapped to the set four-dimensional grid according to the time slice.

[0012] Step 3: Based on the airspace grid, the four-dimensional space-time access value map is iteratively calculated according to reinforcement learning; and a conflict-free path with the minimum cost is selected from the access value map and sent to the path smoothing module.

[0013] The specific process is:

[0014] Step 301, define the value function v(s) and reward function r(s) of state s;

[0015] The value function v(s) is defined as the expected reward obtained by the aircraft starting from state s and reaching the target position along the path selected by the policy π; the policy π is defined as the mapping from the state s to the probability of any possible action a;

[0016] The reward function r(s) is defined as:

[0017]

[0018] Among them S obstacle is the set of grid positions occupied by other elements (obstacles) in the environment except the current agent; S goal is the grid position of the target;

[0019] Step 302: Based on the four-dimensional spacetime airspace grid map, traverse the path from each location to the destination of the aircraft, use the reward function to calculate the reward, i.e., the travel cost, of each location, and store it in the reward map;

[0020] For any state s = (t, i, j, k), the value in the reward graph represents the reward value given when the state is at the grid position (i, j, k) at time t.

[0021] Step 303: Calculate the rewards at each moment using a value iteration algorithm based on the reward graph. Through multiple rounds of iterations, calculate the four-dimensional space-time pass value graph that converges at time T.

[0022] First, in the initialization phase of the iterative algorithm, the values ​​of all positions except the target position at all times are set to -∞, and the value of the target position at all times is set to 0; N is the maximum number of input frames of the value iteration algorithm.

[0023] Then, during the calculation, the three-dimensional space value iteration algorithm is used in the form of a static map to calculate the pass value map at time T using the last frame of the grid map;

[0024] When the transit value graph at the last T moments converges, the transit value graphs at moments T-1, T-2,…, 0 are calculated in sequence using the four-dimensional space-time value iteration method, and the transit values ​​of all positions are eventually updated.

[0025] The spatiotemporal value function is updated according to the following formula:

[0026] Step 304: When the value of each state in the transit value graph is the optimal value, the optimal flight route path is selected;

[0027] The path is a sequence of flight route control points composed of grid coordinates.

[0028] Step 4: After the route smoothing module obtains the conflict-free path with the lowest cost, the route smoothing algorithm smoothes the route control point sequence according to the physical characteristics and dynamic characteristics of the aircraft. Finally, the smoothed result is sent to the airborne platform, which controls the flight of the aircraft according to the route.

[0029] Data smoothing uses linear interpolation method.

[0030] The advantages of the present invention are:

[0031] 1) The present invention provides a conflict-free route planning method based on an airspace grid. Compared with the existing technology that only relies on the topological structure on a static map of the obstacle position at the current moment for path planning, this method constructs a four-dimensional space-time airspace grid and a four-dimensional space-time value map. It can be applied to scenarios where obstacles change dynamically, and can also plan an optimal conflict-free path when obstacles change dynamically.

[0032] 2) The present invention proposes a conflict-free route planning method based on airspace grids. Compared with the existing technology that only uses reinforcement learning and neural networks for path planning, this method derives the traffic value graph through a value iteration network, avoiding the lengthy model training process, greatly reducing the system operation time, and improving effectiveness.

[0033] 3) The present invention provides a conflict-free route planning method based on airspace grids, and also utilizes a path smoothing algorithm to smooth the route control point sequence according to the physical characteristics and dynamic characteristics of the aircraft, ensuring that the final planned trajectory is smooth and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is the architecture diagram of the route planning system of the present invention;

[0035] Figure 2 This is a flow chart of a conflict-free route planning method based on airspace grids of the present invention. DETAILED DESCRIPTION

[0036] In order to facilitate the understanding and implementation of the present invention by those skilled in the art, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. In particular, the present invention does not limit the types of swarm intelligence optimization and traditional optimization algorithms in the technical solution. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work should fall within the scope of protection of the present invention.

[0037] The present invention discloses a method for conflict-free route planning based on airspace grid, a system architecture of a conflict-free route planning system based on airspace grid, such as Figure 1 As shown in the figure, the system architecture includes: airborne platform, path planning system, and path smoothing system:

[0038] 1) Airborne platform

[0039] The airborne platform deployed on the aircraft can send information such as the starting point, end point, and obstacles to the route planning module. After calculation and processing, the route planning module plans a conflict-free route. The airborne platform can choose to control the aircraft to fly along this route based on the route data.

[0040] 2) Path planning system

[0041] After obtaining information about the starting point, destination, and dynamic obstacles, the path planning system constructs a four-dimensional spacetime grid based on the coordinate relationships of different three-dimensional spaces at each time slice. Based on this grid, the system iteratively calculates a cost-to-travel graph, which represents the cost of traveling from any location to the destination. Based on this cost-to-travel graph, the system then selects an optimal path with the lowest cost. This path is a sequence of flight path control points composed of grid coordinates.

[0042] 3) Path smoothing system

[0043] After obtaining the path processed by the path planning system, the route smoothing system uses the route smoothing algorithm to smooth the route control point sequence according to the physical characteristics and dynamic characteristics of the aircraft, and finally sends the smoothed result to the airborne platform.

[0044] like Figure 2 The specific steps are as follows:

[0045] Step 1: The aircraft deploys the airborne platform to collect 4D trajectory data, processes the flight start point, end point and obstacle information into a dynamic point sequence, and sends it to the route planning module;

[0046] like Figure 1 As shown, the airborne platform S11 is responsible for collecting four-dimensional track data (longitude, latitude, altitude, time) and airborne radar data, processing the obstacle data into a dynamic point sequence, and sending the flight start point, end point and dynamic obstacle data to the path planning system S12.

[0047] In addition, the airborne platform S11 is also responsible for ultimately receiving the planned trajectory data and controlling the aircraft's conflict-free flight based on this trajectory data.

[0048] Step 2: The route planning module constructs a four-dimensional spacetime airspace grid based on the coordinate relationship of different three-dimensional spaces on each time slice.

[0049] When the environment changes dynamically, a 3D grid map cannot fully describe the movement of obstacles. Therefore, the 3D grid map needs to be expanded along the time axis into a 4D grid map. After obtaining the starting point, end point, and dynamic obstacle information from the airborne platform S11, the path planning system S12 first needs to construct a 4D spacetime grid based on the coordinate relationships of the different 3D spaces at each time slice.

[0050] The specific construction process is:

[0051] The starting point and end point of the aircraft are used as boundaries to generate a three-dimensional target airspace, and then the three-dimensional target airspace is divided into several four-dimensional grids. The length, width and height of each four-dimensional grid unit are set according to the standard interval distance of the aircraft, and the tracks of the aircraft and obstacles are mapped to the set four-dimensional grids according to time slices.

[0052] Step 3: Based on the airspace grid, the four-dimensional space-time access value map is iteratively calculated according to reinforcement learning; and a conflict-free path with the minimum cost is selected from the access value map and sent to the path smoothing module.

[0053] Based on the airspace grid map of four-dimensional spacetime, the aircraft traverses from each location to the destination and calculates the corresponding travel cost of each location; all travel costs constitute the four-dimensional spacetime travel value map;

[0054] After constructing the spatial grid of four-dimensional space-time, the pass value map of four-dimensional space-time can be calculated based on reinforcement learning.

[0055] In reinforcement learning, strategy π is defined as a mapping from state s to the probability of any possible action a; there are many choices for strategy π.

[0056] In reinforcement learning, the value v(s) of a state s is defined as the expected reward obtained by the vehicle from state s along the path selected by policy π to reach the target location. The reward r(s) is the feedback given by the environment when the agent chooses action a in state s, which encourages or inhibits the action.

[0057] In the four-dimensional space-time access value graph, the state s = (t, i, j, k) is described by the time t and the grid position (i, j, k). The optimal path planned by the access value graph is also described by a state sequence consisting of a time-position quadruple:

[0058] (t0,i0,j0,k0)→(t1,i1,j1,k1)→…→(t N ,i goal ,j goal ,k goal ).

[0059] Since time flows in a positive direction, in a path sequence, the distance between two adjacent path points s t and s t+1 In addition to the adjacent positions, the time also increases in sequence: (t0,i0,j0,k0)→(t0+1,i1,j1,k1)→…→(t0+N,i goal ,j goal ,k goal )

[0060] Therefore, in four-dimensional spacetime, action a Δi,Δj,Δk The resulting state transition is defined as:

[0061] s'=trans(s,a Δi,Δj,Δk )=<s.t+1,s.i+Δi,s.j+Δj,s.k+Δk>

[0062] That is, the state moves to the adjacent position in the direction of increasing time in four-dimensional space-time.

[0063] The reward function r(s) is defined as:

[0064]

[0065] Among them S obstacleA set of grid positions occupied by other elements (obstacles) in the environment besides the current agent

[0066] {<i,j,k> obstacle1 ,<i,j,k> obstacle2 ,...}, since the grid map changes over time, the state set S occupied by the obstacle obstacle It also changes with time. Therefore, when calculating the reward of state s, we need to select the set of obstacles in the grid map at the time st where s is located.

[0067] First, a 4D reward graph is calculated based on the 4D grid map. The value in the reward graph represents the reward value given when the grid position (i, j, k) is reached at time t. Then, the value of the pass value graph is iterated based on the reward graph.

[0068] During the value iteration process, for any state s = (t, i, j, k), the adjacent state {t+1, i+Δi, j+Δj, k+Δk} at the next moment is selected, where Δi, Δj, Δk∈{-1, 0, 1}. Then, the transit value of the state (t, i, j, k) is updated.

[0069] In the initialization phase of the iterative algorithm, the values ​​of all positions except the target position at all times need to be set to -∞, and the value of the target position at all times is set to 0. Let N be the maximum number of input frames of the value iteration algorithm.

[0070] During the calculation, the final moment's row value graph is first calculated using the last frame of the grid map as a static map using a 3D spatial value iteration algorithm. Once the final moment's pass value graph converges, the pass value graphs for subsequent moments are then calculated using a 4D spatiotemporal value iteration method. Since the final frame's pass value graph has already been calculated and converged using the 3D spatial value iteration algorithm, the pass value of all locations in the pass value graphs for subsequent moments, which use this as the basis for calculation, are ultimately updated.

[0071] When the value of each state in the pass value graph is the optimal value, an optimal path can be selected according to the pass value graph, and finally the path is sent to the route smoothing system S13.

[0072] In this step, based on the defined state function r(s) and reward function v(s), the reward function is used to calculate the reward for each location based on the input grid track data, the 4D spacetime airspace grid map, and the flight origin and destination data. This reward is stored in the return graph, and the value graph is initialized. Then, based on the reward graph, a value iteration algorithm is used to calculate the reward at each moment. Through multiple rounds of iteration, a converged 3D spacetime pass value graph is calculated at time T. Based on the 3D spacetime pass value graph at time T, 4D spacetime pass value graphs for times T-1, T-2, …, 0 are generated through a series of iterations, ultimately updating the pass value for all locations.

[0073] This method uses a neural convolutional neural network to implement the spatiotemporal value iteration algorithm, and the value function is updated according to the following formula:

[0074] After obtaining the pass value map, the optimal flight route can be calculated based on the pass value map.

[0075] Step 4: After the route smoothing module obtains the conflict-free path with the lowest cost, the route smoothing algorithm smoothes the route control point sequence according to the physical characteristics and dynamic characteristics of the aircraft. Finally, the smoothed result is sent to the airborne platform, which controls the flight of the aircraft according to the route.

[0076] The calculation of the optimal flight path is divided into two parts: path control point selection and path smoothing:

[0077] The route smoothing system S13 first obtains the optimal path processed by the path planning system S12. The path is a sequence of flight route control points composed of grid coordinates. The route smoothing algorithm then smoothes the sequence of route control points based on the physical characteristics and dynamic characteristics of the aircraft. Data smoothing uses the linear interpolation method.

[0078] When selecting the optimal path based on the pass value graph, starting from the state s0 = (t0, i0, j0, k0) representing the current position at the current moment, subsequent states are selected as route control points according to the following formula until the target state is reached:

[0079]

[0080] Where s'∈trans(s,a) is the adjacent state of state s.

[0081] Although the route smoothing algorithm smoothes the sequence of route control points based on the agent's physical and dynamic characteristics, if the control points have overly sharp corners, the smoothed flight path may deviate far from the control points, causing the agent to be unable to strictly follow the flight path planned by the route control points, affecting traffic safety. Sharp corners in the route also seriously affect the agent's riding experience.

[0082] Therefore, the corner transformation angle and the pass value can be used together as the basis for selecting the route control point. The route control point selection formula is rewritten as:

[0083] where θ is the orientation at state s. The orientation of the initial state s0 is determined by the agent's current orientation. The orientation of the adjacent state s' is calculated based on the orientation of s and the relative positions of s and s'.

[0084] Therefore, the following formula is used to calculate the orientation in state s':

[0085] Function f α Calculate the impact of the angle change caused by the transition from state s to state s' on the pass value. Function f α It should be determined according to the agent model and the passenger's flight preference. When the agent has better cornering performance, it is less sensitive to corner changes, and when the passenger's value preference is more conservative, it is more sensitive to corner changes.

[0086] After being processed by the route smoothing algorithm, the route smoothing system S13 can send the smoothed track data to the airborne platform S11. After the airborne platform obtains the planned track data, it can further control the flight of the aircraft.

[0087] The core modules of this application proposal are the path planning system and the path smoothing system:

[0088] 1) Path planning system

[0089] The purpose of path planning for an aircraft is to obtain the optimal conflict-free navigation path. When the environment changes dramatically, there are many obstacles in the airspace, and the obstacles change dynamically over time, it is very difficult to plan a flight route that is short in distance, takes less time, and is conflict-free from the start point to the end point. Since the three-dimensional grid map cannot fully describe the movement of obstacles in the environment, this system expands the three-dimensional grid map along the time axis into a four-dimensional space-time airspace grid, and calculates the four-dimensional space-time passage value map based on the four-dimensional space-time data. The movement status of each obstacle in the current environment and other information are processed in the form of a value map, and based on this value map, an optimal conflict-free route is planned from a global perspective.

[0090] 2) Path smoothing system

[0091] Path smoothing is crucial for route planning. Sharp corners can severely impact the passenger experience and even air traffic safety. After obtaining the planned path sequence, the system smoothes the control point sequence based on the aircraft's physical and dynamic characteristics to ensure a smooth and reliable trajectory.

Claims

1. A conflict-free route planning method based on airspace grid, characterized in that: The specific steps are as follows: Step 1: The aircraft deploys the airborne platform to collect 4D trajectory data, processes the flight start point, end point and obstacle information into a dynamic point sequence, and sends it to the route planning module; Step 2: The route planning module constructs a four-dimensional spacetime grid based on the coordinate relationship of the different three-dimensional spaces on each time slice; Step 3: Based on the spatial grid, a four-dimensional space-time pass value map is iteratively calculated using reinforcement learning. A conflict-free path with the lowest cost is selected from the pass value map and sent to the path smoothing module. The specific process is: Step 301, define the value function v(s) and reward function r(s) of state s; The value function v(s) is defined as the expected reward obtained by the aircraft starting from state s and reaching the target position along the path selected by the policy π; the policy π is defined as the mapping from the state s to the probability of any possible action a; The reward function r(s) is defined as: Among them S obstacle is the set of grid positions occupied by other elements in the environment except the current agent; S goal is the grid position of the target; Step 302: Based on the four-dimensional spacetime airspace grid map, traverse the path from each location to the destination of the aircraft, use the reward function to calculate the reward, i.e., the travel cost, of each location, and store it in the reward map; For any state s = (t, i, j, k), the value in the reward graph represents the reward value given when it is at the grid position (i, j, k) at time t; Step 303: Calculate the rewards at each moment using a value iteration algorithm based on the reward graph. Through multiple rounds of iterations, calculate the four-dimensional space-time pass value graph that converges at time T. First, in the initialization phase of the iterative algorithm, the values ​​of all positions except the target position at all times are set to -∞, and the value of the target position at all times is set to 0; N is the maximum number of input frames of the value iteration algorithm; Then, during the calculation, the three-dimensional space value iteration algorithm is used in the form of a static map to calculate the pass value map at time T using the last frame of the grid map; When the pass value graph at the last T moment converges, the four-dimensional space-time value iteration method is used to sequentially calculate the pass value graphs at time T-1, T-2, ..., 0, and the pass value of all positions is finally updated; The spatiotemporal value function is updated according to the following formula: Step 304: When the value of each state in the transit value graph is the optimal value, the optimal flight route path is selected; The path is a sequence of flight route control points consisting of grid coordinates; Step 4: After the route smoothing module obtains the conflict-free path with the lowest cost, the route smoothing algorithm smoothes the route control point sequence according to the physical characteristics and dynamic characteristics of the aircraft. Finally, the smoothed result is sent to the airborne platform, which controls the flight of the aircraft according to the route.

2. The method for conflict-free route planning based on airspace grid according to claim 1, characterized in that: The specific construction process of step 2 is: The starting and ending points of the aircraft are used as boundaries to generate a three-dimensional target airspace, and then the three-dimensional target airspace is divided into several four-dimensional grids. The length, width and height of each four-dimensional grid unit are set according to the standard interval distance of the aircraft, and the aircraft's track and the position of the obstacle are mapped to the set four-dimensional grid according to the time slice.

3. The method for conflict-free route planning based on airspace grid according to claim 1, characterized in that: In the step 4, the data is smoothed using a linear interpolation method.

Images