Controlling of air vehicle to move along air corridor based on trained air corridor model

Abstract

To explain a use of an air corridor model to enable an air vehicle to move through an air corridor while maintaining a safe separation distance from other vehicles.SOLUTION: A trained air corridor model 106 associated with an air corridor is received by an air vehicle 102. The trained air corridor model is configured to maintain a safe separation distance between vehicles within the air corridor. A current position of the air vehicle within the air corridor is specified and positions of other agents 122 within the air corridor are received. A next position of the air vehicle is specified based on the trained corridor model, the current position and agent positions. The air vehicle is controlled to move from its current position to the determined next position.SELECTED DRAWING: Figure 1

Description

[Background technology]

[0001] The advent of Advanced Air Mobility (AAM), also known as Urban Air Mobility (UAM), is making air taxis and cargo aircraft (CAVs) a reality, and new challenges arise when they share the same airspace as conventional aircraft and smaller unmanned aerial vehicle systems (UAS) that operate at ground level (AGL) up to 2,500 feet. This simultaneous use of airspace by UAS traffic management (UTM), conventional air traffic management (ATM), and AAM operations requires an AAM system capable of accommodating a wide variety and vast volume of aircraft vehicles in an efficient and scalable manner, supported by the development of safe control and separation rules and regulations. [Overview of the project]

[0002] This summary is provided to introduce selected concepts that take on simplified forms, which are further described in the detailed explanation below. This summary is not intended to identify the main or essential features of the subject matter of the claims, nor is it intended to be used as an aid in determining the scope of the subject matter of the claims.

[0003] Aspects of the present disclosure enable an aircraft vehicle to efficiently travel through an air corridor while maintaining a safe separation distance, by having the aircraft vehicle's processor receive at least a trained air corridor model associated with an air corridor representing the airspace between a departure point, a destination, and the airspace between the departure point and the destination. In this case, the trained air corridor model is configured to maintain a safe separation distance between vehicles in the air corridor, and by the processor to determine the current position data representing the current position of the aircraft vehicle in the air corridor, by the processor to receive agent position data associated with the position of at least one agent vehicle in the air corridor, and by the processor to determine the next position of the aircraft vehicle in the air corridor based on the trained air corridor model, the current position data, and the agent position data.

[0004] The description in this disclosure can be better understood by referring to the attached drawings and the detailed description below. [Brief explanation of the drawing]

[0005] [Figure 1] This block diagram shows a system, according to one embodiment, for enabling an aerial vehicle to move within an aerial corridor while maintaining safe separation from other vehicles. [Figure 2] This diagram shows an aerial corridor from a departure point to a destination, according to one embodiment of the design. [Figure 3] This figure shows the dynamic adaptation of an aerial corridor according to one embodiment. [Figure 4A] This figure shows how to determine the next cube in an aerial corridor from the cube at the current position, according to one embodiment. [Figure 4B] This figure shows how to determine the next cube in an aerial corridor from the cube at the current position, according to one embodiment. [Figure 5]This flowchart shows a process for controlling an aerial vehicle to move within an aerial corridor while maintaining safe separation from other agent vehicles, according to one embodiment. [Figure 6] This is a flowchart showing the process for training an aerial corridor model according to one embodiment. [Figure 7] A computing device according to one embodiment is shown as a functional block diagram. [Modes for carrying out the invention]

[0006] Corresponding reference numerals indicate the corresponding parts throughout the drawings. Figures 1 through 7 show the system as a schematic diagram. Drawings may not be to scale.

[0007] A system and method for controlling an aerial vehicle to move within an aerial corridor based on a trained aerial corridor model is described. The system and method described involves training a decision model using machine learning techniques with a simulated aerial corridor and a simulated vehicle moving within the simulated aerial corridor. Furthermore, the model is embedded in or otherwise installed within an aerial vehicle, configured to use the model to determine a path to travel through the associated aerial corridor while maintaining a safe isolation distance from other vehicles in the associated aerial corridor. More specifically, the described advanced aerial mobility (AAM) system is based on using multi-agent reinforcement learning (MARL) to efficiently plan the movements of a large number of safely isolated pilot-operated and autonomous aerial vehicles within a defined aerial corridor. The MARL-based model described enables the AAM system to manage a diverse set of multiple aerial vehicles within a defined aerial corridor up to a corridor saturation level.

[0008] The methods and systems described use a trained air corridor model associated with an air corridor. The model is received by each aircraft vehicle. In that case, the trained air corridor model is configured to maintain a safe separation distance between vehicles within the air corridor. The current position of each aircraft vehicle within the air corridor is identified, and the positions of other vehicles within the air corridor are received by each aircraft vehicle. The next position of each aircraft vehicle is determined based on the trained air corridor model, the current position, and the agent position. The aircraft vehicle is controlled to move from the current position to the determined next position. Thereby, the aircraft vehicles can efficiently use the airspace of the air corridor while maintaining a safe distance from each other.

[0009] The present disclosure enables the integration of a large number of diverse AAM-based vehicles into the same airspace as conventional aircraft and UASs, which are respectively managed by air traffic management (ATM) and unmanned aircraft system traffic management (UTM). The AAM system described supports the requirements of various sizes and types of aircraft and provides an expandable and flexible dynamic system that enables those aircraft to move efficiently between locations without interfering with other types of air traffic. The present disclosure operates in a different way by training a MARL-based model using a wide variety of simulated vehicles and situations and then installing that model on each vehicle. Thereby, the vehicles having that model can move safely and efficiently through the associated air corridor.

[0010] The air corridor on which the model is based is adaptable such that the space of the corridor is divided into a plurality of spatio-temporal data cubes. Each of the data cubes is recursively dividable into sub-cubes, providing an adaptable granularity. The data cubes of the air corridor are used by the trained model to provide a discrete decision space for a Markov decision process (MDP) for each vehicle within the air corridor.

[0011] The data cubes used by each vehicle may be sized to fit the minimum safe separation distance specific to each vehicle. Thereby, larger vehicles occupy larger data cubes within the air corridor, and smaller vehicles occupy smaller data cubes within the air corridor. By optimizing the size of the data cube for each vehicle, the efficiency of the use of the air corridor is improved. This is because the vehicle does not occupy an unnecessarily large space within the air corridor.

[0012] By using reinforcement training techniques on a wide variety and amount of simulated vehicles, the present disclosure enables the identification of the precise saturation limits of the associated air corridors. Thereby, the use of the airspace is optimized while maintaining safety. Further, processing operations including the data cubes of the air corridors described are efficiently performed using an octree data structure, representing a hierarchical structure adaptable to the data cubes as described herein.

[0013] FIG. 1 is a block diagram showing a system 100 that enables an aerial vehicle 102 to move within an air corridor while maintaining a safe separation from other vehicles (e.g., other agents 122) according to one embodiment. In some examples, the aerial vehicle 102 obtains or otherwise receives a trained air corridor model 106 from an air corridor model trainer 104. The air corridor model 106 is trained by an air corridor model trainer 104 based on a specifically defined air corridor (e.g., an airspace corridor from an airport location to a downtown location including altitude boundaries) from a starting point described by starting point data 108 to a destination described by destination data 110. Upon obtaining the air corridor model 106, the aerial vehicle 102 is configured to move along the air corridor associated with the air corridor model 106 as described herein.

[0014] In some embodiments, an aerial vehicle 102 is configured to move along an aerial corridor from one location to the next based on the output of an aerial corridor model 106. The aerial vehicle 102 may be configured to collect or otherwise acquire current location data 114 (e.g., altitude data, speed data, Global Positioning System (GPS) data, origin, destination, or relative location data associated with another location) via a location sensor interface 112 and to provide the current location data 114 as input to the aerial corridor model 106. The aerial corridor model 106 is trained to process the current location data 114 and determine the next location data 124 based on that data 114.

[0015] Furthermore, or alternatively, the aircraft vehicle 102 may be configured to receive or otherwise acquire other agent location data 118 via an inter-agent communication interface 116. The inter-agent communication interface 116 is configured to allow the aircraft vehicle 102 to communicate with other agents 122 in the air corridor. The other agent location data 118 may also be provided as input to the air corridor model 106, thereby allowing the next location data 124 determined or generated by the air corridor model 106 to be based on the other agent location data 118. In many embodiments, the aircraft vehicle 102 provides the air corridor model 106 with both the current location data 114 and the other agent location data 118 in order to generate the next location data 124, as described herein.

[0016] In some embodiments, the following position data 124 is provided to the vehicle controller 126 of the aerial vehicle 102. The vehicle controller 126 includes hardware, firmware, and / or software configured to control the movement and / or steering of the aerial vehicle 102 as it moves through the air. This control includes controlling its speed and direction. Further or alternatively, the vehicle controller 126 is configured to interpret the following position data 124 and to automatically control the aerial vehicle 102 to move to the position associated with the following position data 124 (for example, by changing the altitude, speed, and / or direction of movement).

[0017] Furthermore, or alternatively, in embodiments in which the aircraft vehicle 102 is pilot-operated at least partially manually or otherwise controlled by a pilot, the following position data 124 may be displayed or otherwise provided to the pilot of the aircraft vehicle 102. The pilot will then be able to control the aircraft vehicle 102 to move to the following position indicated by the following position data 124.

[0018] Furthermore, in some embodiments, the aircraft vehicle 102 is configured to share current position data 114 with other agents 122 in the aerial corridor, which take the form of current position data 120. In such embodiments, each of the other agents 122 also uses an aerial corridor model to safely travel through the aerial corridor, and these agents 122 may use the current position data 120 provided by the aircraft vehicle 102 as input to their model. Furthermore, in such embodiments, some or all of the other agents 122 may be configured as aircraft vehicles 102, and to those agents 122, the aircraft vehicle 102 is treated as one of the other agents 122.

[0019] Sharing current position data 120 between other agents 122 and the aerial vehicle 102 may further be based on a communication range or buffer range. Thereafter, agents in the aerial corridor share their current position data 120 with other agents within that range (for example, an agent near the departure point and an agent near the destination may be too far apart to communicate their current position data 120 to each other, and such data sharing would be unnecessary for both agents to determine safe movement within the aerial corridor). Such a buffer range may be defined based on the agents' communication capabilities (e.g., the effective communication range of the agents' hardware, firmware, or software) and / or based on the communication range required to enable agents to maintain safe separation (e.g., the minimum range between agents that may be required for an agent to change course to maintain a safe separation distance by responding to the received current position data 120). Other aspects of defining the buffer range may include the speed at which the agents are moving and the agents' maneuverability. Both of these may affect the agents' ability to respond to the presence of other agents in the aerial corridor.

[0020] The aerial corridor model 106 is trained to direct or otherwise enable the aerial vehicle 102 and other agents 122 to move along the associated aerial corridor and maintain safe control and safe separation from other agents within the corridor. In some embodiments, the aerial corridor model 106 is configured as a adaptable aerial corridor model, as described herein with respect to Figure 2.

[0021] The aircraft vehicle 102 and other agents 122 described herein may include any type of unmanned aerial vehicle (UAV) and / or any type of manned aircraft. For example, in one embodiment, the aircraft vehicle 102 is a UAV, while the other agents 122 using the same air corridor may include both manned and unmanned aircraft. In other embodiments, other combinations of unmanned and manned aircraft, including all unmanned aircraft or all manned aircraft, may be guided through the air corridor without departing from the present disclosure.

[0022] Figure 2 shows an aerial corridor 200 from a departure point 202 to a destination 204 according to one embodiment. The aerial corridor 200 may be configured as a joined set of airspace taking the form of a spatiotemporal data cube 206 between the departure point 202 and the destination 204 for use in an aerial corridor model (e.g., model 106). In some embodiments, such a model may be trained in particular for specific departure-destination pairs. Thereafter, the model and associated aerial corridor 200 enable an aircraft vehicle (e.g., aircraft vehicle 102, other agents 122a-n) to travel safely between a departure point and a destination (in both directions) (e.g., the departure point is one or more vertiports / heliports at an airport, and the destination is one or more vertiports / heliports within the downtown area of ​​a city). However, it is not trained for other departure-destination pairs. Although two other agents 122a and 122n are illustrated, in other embodiments more, fewer, or different agents may move within the aerial corridor 200 without departing from the present disclosure.

[0023] More formally, given a reference system R, i.e., a 3D grid network, formed by a set of fixed spatial positions in three dimensions (3D), the centerline 208 of a adaptable aerial corridor 200 is AC ━ =[r1, r2, ..., rn], where ri∈R. The cubes 206 are arranged around reference points along the centerline 208 of the aerial corridor, constructing an aerial corridor 200 composed of spacetime data cubes 206.

[0024] In some embodiments, the aerial corridor 200 includes each cube 206 which is thought to have uniform weather parameters over a period of time. When arranged in this manner, the cubes 206 can be considered as segmented aerial corridor segments.

[0025] Figure 3 shows Figure 300 illustrating the dynamic adaptation of an aerial corridor (e.g., aerial corridor 200) according to one embodiment. Figure 300 includes a spacetime cube (e.g., of the spacetime cube 206 of aerial corridor 200). The spacetime cube is subdivided into a smaller cube 312, which is further subdivided into an even smaller cube 314. The aerial corridor 200 described is dynamically adaptable in that the cube 206 of aerial corridor 200 can be expanded, subdivided, and subdivided as shown in Figure 300 to accommodate aerial vehicles of varying quantities, types, and / or sizes (e.g., during the operation of the associated aerial corridor model 106).

[0026] In such embodiments, the aerial corridor model 106 associated with the aerial corridor 200 is configured to operate using the rule that only one aircraft vehicle can occupy a cube (e.g., cubes 310, 312, and / or 314) at a given time. The cubes 206 of the aerial corridor 200 may then be adapted to subdivide the aerial corridor 200 into an amount of cubes that allows multiple aircraft vehicles to travel safely through the aerial corridor substantially simultaneously, while maintaining a cube size for each aircraft vehicle defined to maintain a safe separation distance around the aircraft vehicle (e.g., based on the size of the aircraft vehicle and / or other factors associated with safe separation).

[0027] More formally, each 3D segment of the aerial corridor 200 (e.g., cube 206) may be partitioned and / or resized as shown with respect to cubes 310, 312, and 314 in Figure 3. This is done by recursively subdividing each 3D segment of the aerial corridor 200 into eight quadrants until a level of fine-grainedness is reached for the minimum separation value for the AAM system (e.g., a measurement of the smallest safe cube size of (1 or more)) (e.g., which may be associated with the size of the aircraft vehicle that can use the aerial corridor). The resulting cubes of the aerial corridor 200 are then used to define a limited set of states for a Markov decision process (MDP) to be used in training and / or operating the associated aerial corridor model 106. In other embodiments, other methods may be employed to train and use the model.

[0028] In some embodiments, the artifact of the adaptation process is an aerial corridor 200 defined as a discrete set of cubes that can be modeled using an octree data structure. In such embodiments, using an octree data structure to represent the set of cubes of the aerial corridor enables efficient retrieval and processing of data associated with these cubes, including identifying safe travel paths and / or conflicts within the cubes of the aerial corridor. In other embodiments, other methods for partitioning the aerial corridor 200 and / or training an associated aerial corridor model 106 based on these partitions may be used without departing from the present disclosure.

[0029] Referring to Figure 1, the air corridor trainer 104 includes hardware, firmware, and / or software configured to train an air corridor model 106 to maintain and use an air corridor such as an air corridor 200. The air corridor 200 represents an actual portion of the airspace between a departure point, as indicated by departure point data 108, and a destination, as indicated by destination data 110. In some embodiments, the air corridor model 106 is trained to predict optimal trajectories for aircraft vehicles and / or agents. Thereafter, they can move forward within the air corridor in an efficient manner, including maintaining safe distances between vehicles and / or between vehicles and other obstacles, and in an efficient manner by maximizing the use of adaptable air corridor space. The aerial corridor model 106 may be trained to receive as input the quantity of aerial vehicles in the aerial corridor, the size of those aerial vehicles and / or other safe separation distance data, the quantity of vertiports, helipads and / or other takeoff and landing areas at the origin and destination of the aerial corridor, and / or other data associated with the safe separation minimum used in the aerial corridor.

[0030] In some embodiments, the aerial corridor model trainer 104 is configured to train the aerial corridor model 106 using reinforcement learning (RL) techniques such as single-agent reinforcement learning (SARL) or multi-agent reinforcement learning (MARL). To perform such training, an aerial corridor associated with the aerial corridor model 106 is simulated, and then a simulated aerial vehicle is guided to move along the simulated aerial corridor. As the simulated aerial vehicle moves, the aerial corridor model 106 determines the next position of the simulated aerial vehicle as trained. In this case, safe decisions may be rewarded, while dangerous decisions (e.g., decisions resulting in collision or violation of safe separation distance) are not rewarded or are punished with respect to the trained model 106. Thus, safe decisions are reinforced, and the model 106 is trained to determine safe next positions for aerial vehicles within the aerial corridor.

[0031] Furthermore, within a set of promising and safe next positions, the next position that allows the aerial vehicle to move more directly or otherwise efficiently along the aerial corridor may be rewarded to a greater extent than the next position that is less direct or less efficient. Thereafter, Model 106 is also trained to optimize the efficiency of the aerial vehicle moving within the aerial corridor. Such a process is described in more detail below with respect to Figures 4A and 4B.

[0032] Figures 4A and 4B are Figures 400A and 400B illustrating, according to one embodiment, the determination of the next position cube from the current position cube 402 within an aerial corridor (e.g., aerial corridor 200). In some embodiments, the determination of the next position cube from the current position cube 402 is performed by or based on a trained aerial corridor model, such as aerial corridor model 106. Such an aerial corridor model may be trained by an aerial corridor model trainer, such as aerial corridor model trainer 104, as described herein. It may be used by an aerial vehicle or other agent, such as aerial vehicle 102 and / or other agent 122. Figure 400A includes the current position cube 402 and a plurality of potential next position cubes 404. The current position cube 402 represents the space of airspace within an associated aerial corridor in which an aerial vehicle 102, controlled or guided by the aerial corridor model, is positioned. The determination that the aircraft vehicle 102 is currently located within the cube 402 of its current location may be based on current location data collected or otherwise acquired by the aircraft vehicle 102, such as telemetry data collected by sensors (e.g., current location data 114), and / or an estimation of the aircraft vehicle 102's position from a previous location.

[0033] As illustrated by arrow A, the aircraft vehicle 102 in the current position cube 402 is moving from left to right. Therefore, the potential next position cube 404 for the aircraft vehicle 102 includes nine cubes located immediately to the right (orthogonal or diagonal) of the current position data 114. In other embodiments, the multiple potential next position cubes 404 may also include cubes surrounding the current position cube 402 in other directions (e.g., cubes above, below, to the side of, or a combination thereof from the current position cube 402). However, when the aircraft vehicle 102 is moving from left to right, the potential next position cubes 404 to the right of cube 402 would be preferable, because they allow the aircraft vehicle 102 to continue moving forward through an aerial corridor (e.g., aerial corridor 200). From the perspective of the aircraft vehicle 102, the cube 404 in the next potential position includes cubes that the aircraft vehicle 102 can enter when it moves straight forward, forward and up, forward and down, forward and right, forward and left, forward and right and up, forward and left and up, forward and right and down, and forward and left and down.

[0034] Figure 400B of Figure 4B shows a subset of potential next-position cubes 404, including the current position cube 402 and cubes 406, 408, and 410. Given the current position of the aircraft vehicle 102 in the air corridor and position data of other agents (e.g., other agent 122), the air corridor model (e.g., air corridor model 106) is configured to select from a plurality of potential next-position cubes 404. In some embodiments, the model is configured to select one of the next-position cubes 404 based on determining or identifying a trained reward or weighted value for each potential next-position cube and selecting the cube associated with the highest value. Reward values ​​may be assigned to each potential next-position cube 404 based on input position data for the aircraft vehicle 102 and any other agents. Thereafter, the cube that is most efficient for traveling through the air corridor, which is also the predicted safe trajectory for the aircraft vehicle 102 with respect to the other agents, receives the highest reward value. For example, if the model determines that only cubes 406, 408, and 410 are predicted to be safe for the aircraft vehicle 102, then the reward value of each cube 406, 408, and 410 is the highest among the cubes in the next potential position. One of those cubes will have a higher reward value than the other two. Thus, that one will be selected over the other two. (For example, cube 408 may be selected over 406 and 410 because it is a safer trajectory that is more direct or efficient to the destination).

[0035] Referring to Figure 1, the simulated aerial vehicle used by the aerial corridor model trainer 104 to train the aerial corridor model 106 may be configured to select a cube for the next potential location to travel within the simulated aerial corridor, as described with respect to Figures 4A and 4B. This process may be performed over a large number of iterations, and the aerial corridor model 106 may be trained and / or tuned based on the results of those iterations, with safe and efficient determination of the next location cube being rewarded. In some embodiments, the training process for the aerial corridor model 106 includes training the model 106 with a minimum number of agents within the aerial corridor, and then increasing the number of agents within the aerial corridor through the training iterations until the aerial corridor is saturated. The aerial corridor may be considered saturated when adding more agents results in an unavoidable collision or otherwise unsafe behavior while traveling within the aerial corridor.

[0036] As previously described, in some embodiments, the trained aerial corridor model 106 is trained and operates according to a distributed approach. Thereafter, each agent in the aerial corridor uses the aerial corridor model 106, and these agents are connected by a time-varying and sparse communication network that allows agents to exchange information without the presence of any central controller. Thus, each agent's policy is independent. At each step or location, each agent performs an individual action based on both available local information, such as current location data 114, and messages sent from other neighboring agents, such as location data 118 of other agents. The common objective of all agents is to maximize the average reward of all agent decisions throughout the aerial corridor network. In a policy gradient approach, this would involve simultaneously optimizing multiple policies from a common reward signal. Such an approach would make learning heterogeneous policies easier.

[0037] In some embodiments, the decentralized approach includes the use of a decentralized actor-critic algorithm based on the policy gradient theorem of MARL. Further, the approach uses softmax temporal consistency, drives a primal-dual decentralized optimization method, and proposes a value propagation algorithm. These approaches present advantages of scalability to a large number of agents, robustness to malicious attacks, and communication efficiency.

[0038] As previously explained, the model 106 may be trained to utilize MDP techniques. An MDP can be described by five tuples S, A, R, P, γ. Here, S is a finite state space, A is a finite action space, P = (P(s′|s, a)) s、s′∈S、a∈A is the transition probability, R = (R(s, a)) s、s′∈S、a∈A is the real-valued immediate reward, γ ∈ (0, 1) is the discount factor. The policy is denoted by π. Here, π(s t 、a t ) is the conditional probability density at a with the policy. V t (s) = max * E[Σ π γ ∞ i=1 R(s t 、a t 、a t )|s0 = s] is the optimal value function, which is the unique fixed point of V(s), i.e., the Bellman optimal operator.

[0039] Furthermore, a networked multi-agent MDP is now described. Let G = (N, E) be an undirected graph with |N| = N agents. Here, E is the set of edges, and (i, j) ∈ E means that agents i and j can communicate with each other via this edge. The networked multi-agent MDP is a tuple (S, {A i} i∈N 、P、{R i}i∈N Characterized by (G, γ), where S is a global state space shared by all agents in N, and A i is the action space of agent i, and A = Π N i=1 A i is the combined action space of all agents, P is the transition probability, and R is the transition probability. i is the local reward function of agent i. In such embodiments, the reward is observed only locally, and the agent is s t Observe and determine a at each time step t =( a t 1, a t 2, ...a t N ) is created. Then, each agent receives its own reward R. i (s t a t ) receives, and the environment changes to a new state s according to the transition probability. t+1 Switch to . Furthermore, since each agent independently makes its own decisions, it is appropriate to assume that the policy π(s, a) is factorized. Due to the fact that actions are performed locally by agents and the critical (value) function is trained locally, this method is entirely distributed, and its objective function is as follows: namely, TIFF2022169449000002.tif52170

[0040] The problem space for a distributed MARL configuration is adapted as follows: each aircraft vehicle in the environment is considered an agent, and there are N connected agents that exist in the environment for a time N t It changes over time. The environment includes adaptable aerial corridors formed by a set of spatiotemporal data cubes as described herein, and global states s t This is maintained at each time step. Individual agent's action space A tThis specifies the location where the agent can arrive at the next time step t (for example, cube 404 at the potential next location in Figure 4A). Each agent, at each time step R t Associated with the reward function in state s, the i-th agent attempts to maximize its own expected discounted return. t+1 The probability of transitioning to this is given by the joining action a t This is the current state s t If a state transition is taken, it is a probability of state transition. Actions are deterministic, but new vehicles become available in various cubes within the aerial corridor at each time, and some currently available vehicles go offline when they reach their destination.

[0041] Using this configuration, the isolation-determining task is designated as fully collaborative (e.g., all vehicles want to move forward efficiently by maximizing the use of the aerial corridor), partially observable (e.g., each vehicle can observe its own state / position), and is a set of multi-agent decision-making problems. This task may be addressed using MARL in a setting where agents are connected over time and potentially via a loose communication network. Each agent's policy is independent in this fully distributed scheme. Specifically, in each state, each vehicle takes action, and these actions together determine the next state of the aerial corridor and the individual rewards for each vehicle. In this collaborative setting, vehicles attempt to maximize the discounted sum of rewards across the entire network.

[0042] In other embodiments, a fully centralized approach or a centralized learning approach may be used when training and using the aerial corridor model 106. In the fully centralized approach, a joined model is used for the actions and observations of all agents in the aerial corridor. A centralized policy maps the joined observations of agents to joined actions. Thus, the problem of a central controller controlling agents in the aerial corridor using information on all available agents reduces to an MDP that can be solved by the SARL algorithm. However, this approach can lead to exponential growth of the observation and action space with a large number of agents, and the need for a central controller to communicate with each agent presents further limitations. Such a centralized approach does not appear to address the scalability provided by the distributed approach described herein.

[0043] While Model 106 is trained using centralized learning, in a centralized learning approach where the use of Model 106 is distributed among agents, the focus is on ensuring that communication between agents is not suppressed during training. Such communication may be performed by a central controller during training. However, during the execution of the trained policy, agents will only be able to communicate over channels with limited bandwidth. As a result, this approach allows the policy of Model 106 to be trained using the experience of all agents simultaneously.

[0044] In some embodiments, the described system is configured to operate within an aerial corridor at a specified altitude. For example, the described system may be configured to operate at altitudes between 400 feet and 2,500 feet. This allows the aircraft vehicle 102 and other agents 122 to operate above the altitude of a specified unmanned aerial vehicle system traffic management (UTM) while remaining below the normal operating altitude of other types of aircraft. In other embodiments, greater, smaller, or different altitude limits may be used without departing from this description.

[0045] In some embodiments, the aircraft operating within the aerial corridor of the described system are configured to perform vertical takeoff and landing (VTOL) and hovering, and these capabilities may be taken into consideration by Model 106 when determining the next position for the vehicle. Furthermore or alternatively, the aircraft operating within the aerial corridor may include fixed-wing vehicles that cannot hover and / or VTOL. Thus, these vehicles result in less flexibility regarding the next position that can be safely achieved at a given time (for example, such vehicles may be less flexible with respect to changes in speed, altitude gain, or changing course more quickly than vehicles that can otherwise hover). Model 106 may also be trained to take such limitations into account, thereby determining the next position for a vehicle within the aerial corridor based on the presence of less maneuverable vehicles within the aerial corridor.

[0046] Furthermore, further learning can be performed in real-world testing via the use of small, remotely located unmanned aerial systems (UAS). Such a process can further enable validation of the simulation-trained model 106 and reveal further real-world considerations that may not be apparent based on the training simulation. Moreover, once model 106 is embedded in an aerial vehicle 102 and / or other agent 122 or otherwise installed, feedback data collected during the operation of those vehicles can be used to further refine and / or improve model 106 according to machine learning techniques such as reinforcement learning as described herein.

[0047] Figure 5 is a flowchart illustrating a process 500 for controlling an aerial vehicle (e.g., aerial vehicle 102) to move within an aerial corridor (e.g., aerial corridor 200) while maintaining safe isolation from other agent vehicles (e.g., other agent 122), according to one embodiment. In some embodiments, the process 500 is performed or otherwise carried out by the aerial vehicle in a system such as system 100, as described herein.

[0048] In 502, a trained air corridor model is received that is associated with the origin, destination, and an air corridor between the origin and destination (e.g., an air corridor in a straight line between the origin and destination in a defined altitude range, such as 400 feet to 2500 feet). In some embodiments, receiving the air corridor model involves the model being embedded in or otherwise installed within the computer system of the aircraft vehicle. Thereafter, the air corridor model can be used by the computer system to influence the control of the aircraft vehicle. Furthermore, the trained air corridor model may be configured to enable the aircraft vehicle to efficiently travel from origin to destination or from destination to origin via the air corridor while maintaining a safe separation distance from other agents moving within or otherwise entering the air corridor (e.g., other conventional aircraft that may interfere with the air corridor when landing on or taking off from a nearby runway).

[0049] In 504, the current position data of the current position of an aircraft vehicle within the aerial corridor is identified. In some embodiments, the aerial corridor is represented by a joined set of spatiotemporal data cubes representing portions of the airspace within the aerial corridor from the origin to the destination. Furthermore, the aerial corridor can be adapted by subdividing or expanding the aerial corridor data cubes (e.g., subdividing a larger data cube into eight quadrants i.e., eight smaller data cubes that fill the space of the larger cube, or merging a group of eight smaller data cubes into a single larger data cube). Such subdivision and / or expansion of the aerial corridor data cubes may be done to accommodate the size and shape of the aircraft vehicle, as well as any other features that affect the minimum safe separation distance of the vehicle. Subdivision and / or expansion may also be based on optimizing the safe and efficient use of the aerial corridor by multiple agent vehicles up to the saturation limit of the aerial corridor. For example, for a particular size of aircraft vehicle, the aerial corridor may be subdivided down to the smallest cube that maintains the safe separation distance of the aircraft vehicle. This ensures that the resulting aerial corridor data cubes are efficiently used by multiple aerial vehicles only when each cube can be occupied by one aerial vehicle. The identified current location data of an aerial vehicle may include the identifier of the data cube in the aerial corridor, as well as other location information such as altitude, latitude, longitude, and distance from origin and / or destination. Other data that may be collected by the aerial vehicle may include speed data and / or direction of travel data.

[0050] In 506, agent location data associated with other agent vehicles within the aerial corridor is received. In some embodiments, agent location data is received via wireless communication between the aerial vehicle and other agent vehicles (e.g., a time-varying loose communication network that allows agent vehicles to exchange information without the presence of any central controller). Furthermore, the aerial vehicle may provide identified current location data to other agent vehicles within the aerial corridor via the same or different communication method. The agent location data may include the same or different information as the identified current location data, such as the identifier of the data cube occupied by the agent vehicle within the aerial corridor, altitude, latitude, longitude, speed of movement, and / or direction of movement. The agent location data may also include the safe separation distance of the other agent vehicles (e.g., if one of the other agent vehicles is larger and occupies a larger data cube, the aerial vehicle needs that information to route its course appropriately to avoid collision with the safe separation distance of the larger vehicle).

[0051] In embodiments where the aerial corridor data cubes are defined and processed using an octree data structure, the identifiers of data cubes occupied by other agents may, due to the nature of the octree data structure, contain sufficient information to maintain a secure isolation distance between the other agents. For example, if an octree data structure is represented by hierarchy data values ​​indicating smaller, finer-grained data cubes for each hierarchy level, then location data containing identifiers for lower-level data cubes indicates that the secure isolation distance of the associated agent may be greater than that of an agent with identifiers for higher-level data cubes (for example, identifier 2.4.6 indicates a three-level hierarchy and shows that the agent is located in the sixth quadrant of the fourth quadrant of the second data cube in the aerial corridor, which is a data cube with a volume of 1 / 64th of the associated complete data cube, while identifier 6.1.2.3 indicates a four-level hierarchy and shows that the agent is located in the third quadrant of the second quadrant of the first quadrant of the sixth data cube in the aerial corridor, which is a data cube with a volume of 1 / 512th of the associated complete data cube, representing a shorter secure isolation distance than identifier 2.4.6).

[0052] Furthermore, in some embodiments, the received agent position data is from a subset of agent vehicles in an aerial corridor that are within a defined buffer range of the aerial vehicle. Such a buffer range may be defined based on requirements regarding the lead time required for the aerial vehicle in the aerial corridor to safely and efficiently avoid collisions, as described herein. Alternatively, the received agent position data may be received from all vehicles in the aerial corridor.

[0053] In 508, the next position of the aircraft vehicle is determined based on a trained air corridor model, current position data, and agent position data. In some embodiments, the air corridor model is configured to take current position data and agent position data as inputs and to provide next position data for the next position as an output. Furthermore, the air corridor model may be trained to determine a set of data cubes for the potential next position based on the current position data (e.g., the identifier and direction of movement of the current data cube), determine a reward value for each of the data cubes for the potential next position based on its current position data and the agent position data of other agents, and select the data cube for the potential next position with the highest reward value as the next position for the aircraft vehicle. Other agent vehicles in the air corridor may also determine their next positions in substantially the same manner, and the aircraft vehicle's trained air corridor model may be trained to determine reward values ​​for the data cubes for the potential next position based at least in part on other agents in the air corridor, which also use such a model to select their next positions.

[0054] In 510, the aircraft vehicle is controlled to move from its current position within the air corridor to a determined next position within the air corridor. In some embodiments, the aircraft vehicle is controlled to move from the data cube of its current position to the data cube of its next position within the set of data cubes that make up the air corridor. Such control of the aircraft vehicle may include one or more of the following: accelerating the aircraft vehicle, decelerating the aircraft vehicle, causing the aircraft vehicle to gain altitude, causing the aircraft vehicle to lose altitude, causing the aircraft vehicle to change direction, i.e., heading, and causing the aircraft vehicle to maintain its current direction, i.e., heading. In many embodiments, the aircraft vehicle is automatically controlled by one or more computing systems, but in other embodiments, control of some parts of the vehicle is manual. In such systems with at least partially manual control, the aircraft vehicle's next position may be communicated to the aircraft vehicle's pilot or other control user. This may include indicating the direction toward the next position and / or indicating one or more controls in order to adjust so that the aircraft vehicle moves to the next position. In such embodiments, the automatically controlled aircraft may be configured to grant priority to manually controlled aircraft within the air corridor, and further strategic planning may be included to ensure that human pilots can perform any necessary maneuvers.

[0055] In step 512, if the aircraft vehicle reaches its destination, the process proceeds to step 514. Alternatively, if the aircraft vehicle does not reach its destination, the process returns to step 504, at which point the processes from 504 to 510 for determining the next position and moving to the next position are executed again. As a result, the aircraft vehicle is controlled to move along the aerial corridor from position to position. Each next position is determined based on the aerial corridor model and the latest position data of nearby agent vehicles that have communicated the aircraft vehicle and its own position data. In step 514, the aircraft vehicle's flight ends (for example, by landing in a helipad, vertiport, or another designated landing area).

[0056] In some embodiments, controlling an aircraft vehicle to take off from and / or land at an aerial corridor's departure or destination may be performed in a different manner before the aircraft vehicle enters the associated aerial corridor. The aerial corridor may be raised above the landing area from the ground. For example, the number of vehicles permitted to take off and land at a particular endpoint of an aerial corridor may be based on the number of helipads or vertiports available at that location. Alternatively or further, if the landing area of ​​an endpoint is located within an aerial corridor, the aerial corridor model may be trained to control aircraft vehicles to take off from and / or land therein in that landing area. Other methods for controlling aircraft vehicle takeoffs and landings at endpoints of an aerial corridor may be used without departing from this description.

[0057] Figure 6 is a flowchart showing a process 600 for training an aerial corridor model (e.g., aerial corridor model 106) according to one embodiment. In some embodiments, as described herein, process 600 is performed by a system such as system 100 and / or components of system 100 such as aerial corridor model trainer 104. In 602, a simulated aerial corridor associated with a origin and destination is defined. In some embodiments, the model may be trained based on a specific origin-destination pair. Defining a simulated aerial corridor may include defining a joined set of spatiotemporal data cubes, as described above.

[0058] In 604, multiple simulated agents are generated, and in 606, these simulated agents are configured to communicate with each other via time-varying coarse communication, thereby simulating the limited communication of real-world systems. In 608, these simulated agents are configured to move through the simulated corridor using a trained aerial corridor model. These simulated agents are then sent into the aerial corridor in a changing configuration, organization, order, and timing.

[0059] In 610, the rewards obtained by the simulated agent as it moves along the aerial corridor are recorded. In some embodiments, as described herein, the simulated agent determines its next position using an MDP based on the aerial corridor model. The aerial corridor model is configured to determine the reward for each simulated agent's potential next position at each point in time, as described herein. Collisions between simulated agents may be assigned very low or even negative rewards to prevent such behavior.

[0060] In 612, the recorded rewards of the simulated agent are used as feedback data to train an aerial corridor model. The model may be tuned to increase the reward for next position decisions that result in the simulated agent successfully reaching its destination, while next position decisions that result in collisions or inefficiencies may have reduced rewards. The aerial corridor model may be trained based on RL techniques such as MARL, as described herein.

[0061] In 614, if the model training is not yet complete, the process returns to 604 to perform another iteration of the described simulation to further refine the aerial corridor model. Alternatively, if the model training is complete in 614, the process may proceed to 616 to perform real-world testing on the model. For example, to verify the model's behavior in the real world and / or to identify any issues not identified during the simulation, an unmanned aerial taxi may be configured to use the trained model and controlled to travel along an aerial corridor in a remote area.

[0062] Exemplary operating environment This disclosure is operable with a computing device according to one embodiment, as shown in the functional block diagram 700 of Figure 7. In one embodiment, components of the computing device 718 may be implemented as parts of an electronic device according to one or more embodiments described herein. The computing device 718 comprises one or more processors 719, which may be microprocessors, controllers, or any other type of processor for processing computer-executable instructional commands to control the operation of the electronic device. Alternatively or further, the processor 719 is any technology capable of executing logic or instructional commands, such as a hardcoded machine. Platform software comprising an operating system 720 or any other suitable platform software may be provided to the device 718, enabling application software 721 to be executed on the device. According to one embodiment, as described herein, controlling an aerial vehicle to move within an aerial corridor while maintaining a safe separation distance using a trained aerial corridor model may be achieved by software, hardware, and / or firmware.

[0063] Computer-executable instructions may be provided using any computer-readable medium accessible by the computing device 718. Computer-readable mediums may include, for example, computer storage media such as memory 722, and communication media. Computer storage media such as memory 722 include volatile and non-volatile removable and non-removable media implemented in any method or technique for storing information such as computer-readable instructions, data structures, and program modules. Computer storage media may include, but are not limited to, RAM, ROM, EPROM, EEPROM, phase-change memory, flash memory, or other memory technologies, CD-ROM, digital multipurpose disk (DVD), or other optical storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices, shingled disk storage, or other magnetic storage devices, or any other non-transmission media that can be used to store information for access by the computing device. In contrast, communication media may embody computer-readable instructions, data structures, program modules, etc., in modulated data signals such as carrier waves or other transport mechanisms. In this specification, computer storage media does not include communication media. Therefore, computer storage media should not be interpreted as the propagating signals themselves. The propagating signals themselves are not an example of computer storage media. Although the computer storage media (memory 722) is shown within the computing device 718, the storage device may be distributed or located remotely and accessed via a network or other communication link (for example, using the communication interface 723).

[0064] The computing device 718 may include an input / output controller 724 configured to output information to one or more output devices 725, for example, a display or speaker that may be separated from or integrated with an electronic device. The input / output controller 724 may also be configured to receive and process input from one or more input devices 726, for example, a keyboard, microphone, or touchpad. In one embodiment, an output device 725 may act as an input device. One embodiment of such a device may be a touch-sensitive display. The input / output controller 724 may output data to a device other than an output device, for example, a locally connected printing device. In one embodiment, a user may provide input to (one or more) input devices 726 and / or receive output from (one or more) output devices 725.

[0065] The functions described herein may be performed at least partially by one or more hardware logic components. According to one embodiment, the computing device 718 is configured to perform the described embodiments of operation and function by program code when executed by the processor 719. Alternatively or further, the functions described herein may be performed at least partially by one or more hardware logic components. For example, non-limitingly, exemplary types of hardware logic components that may be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-chip systems (SOCs), coupled programmable logic circuits (CPLDs), and graphics processing units (GPUs).

[0066] At least a portion of the functionality of various elements within the drawing may be performed by other elements within the drawing or entities not shown within the drawing (e.g., processors, web services, servers, application programs, computing devices, etc.).

[0067] Although described in relation to computing system environments, the embodiments of this disclosure can be implemented using many other general-purpose or dedicated computing system environments, configurations, or devices.

[0068] Examples of well-known computing systems, environments, and / or configurations that may be suitable for use in the manner of this disclosure include, but are not limited to, mobile or portable computing devices (e.g., smartphones), personal computers, server computers, handheld (e.g., tablets) or laptop devices, multiprocessor systems, gaming consoles or controllers, microprocessor-based systems, set-top boxes, programmable home appliances, mobile phones, portable computing and / or communication devices in wearable or accessory form factors (e.g., watches, glasses, headsets, or earphones), network PCs, minicomputers, mainframe computers, distributed computing environments including any of the systems or devices described above, and similar. Generally, this disclosure is operable on any device having processing capabilities, thereby capable of executing instructional commands such as those described herein. Such systems and devices receive input from a user in any way, including from input devices such as keyboards and pointing devices, via gesture input, proximity input (such as hovering), and / or voice input.

[0069] The embodiments of this disclosure are described in the general context of computer-executable instructions, such as program modules, that are executed by one or more computers, or by other devices in software, firmware, or hardware, or a combination thereof. Computer-executable instructions may be structured within one or more computer-executable components or modules. Generally, program modules include, but are not limited to, routines, programs, objects, components, and data structures that perform specific tasks and implement specific abstract data types. Embodiments of this disclosure may be implemented by structuring any number of such components or modules. For example, embodiments of this disclosure are not limited to the specific computer-executable instructions or specific components or modules shown in the drawings and described herein. Other embodiments of this disclosure may include various computer-executable instructions or components having more or fewer functions than those shown and described herein.

[0070] In embodiments including a general-purpose computer, the aspects of the present disclosure convert the general-purpose computer into a dedicated computing device when configured to execute the instructions described herein.

[0071] An exemplary computer system for controlling an aerial vehicle to move within an air corridor while maintaining safe separation from other agent vehicles comprises a processor and a non-transient computer-readable medium storing program code for transferring data to another computer system, the program code causing the processor to: receive a trained air corridor model associated with an air corridor representing a origin, a destination, and the airspace between the origin and the destination, the trained air corridor model configured to maintain safe separation distances between vehicles within the air corridor; identify current position data representing the current position of the aerial vehicle within the air corridor; receive agent position data associated with the position of at least one agent vehicle within the air corridor; and determine the next position of the aerial vehicle within the air corridor based on the trained air corridor model, the current position data, and the agent position data.

[0072] An exemplary computer-controlled method for controlling an aerial vehicle to move within an aerial corridor while maintaining safe separation from other agent vehicles includes: receiving a trained aerial corridor model associated with an aerial corridor representing a departure point, a destination, and the airspace between the departure point and the destination, wherein the trained aerial corridor model is configured to maintain safe separation distances between vehicles within the aerial corridor; the processor identifies current position data representing the current position of the aerial vehicle within the aerial corridor; the processor receives agent position data associated with the position of at least one agent vehicle within the aerial corridor; and the processor determines the next position of the aerial vehicle within the aerial corridor based on the trained aerial corridor model, the current position data, and the agent position data.

[0073] A non-transient computer storage medium storing program code executable by a first computer system at a first site, the program code embodying a method including: receiving a trained air corridor model associated with an air corridor representing a departure point, a destination, and the airspace between the departure point and the destination, the trained air corridor model being configured to maintain a safe separation distance between vehicles in the air corridor; identifying current position data representing the current position of the aircraft vehicle in the air corridor; receiving agent position data associated with the position of at least one agent vehicle in the air corridor; and determining the next position of the aircraft vehicle in the air corridor based on the trained air corridor model, the current position data, and the agent position data.

[0074] In lieu of or in addition to other embodiments described herein, the embodiments include any combination of the following:

[0075] An aerial corridor includes a joined set of spatiotemporal data cubes representing portions of the airspace within the aerial corridor from a point of origin to a destination, where at least one data cube in the joined set of spatiotemporal data cubes is subdivided into multiple data subcubes, the size of which is based on a defined minimum safe separation distance.

[0076] Identifying the current position data of an aerial vehicle within an aerial corridor includes identifying the current data subcube of the aerial corridor where the aerial vehicle is currently located, and determining the next position of the aerial vehicle within the aerial corridor, based on a trained aerial corridor model, current position data, and agent position data, includes determining the next data subcube of the aerial corridor adjacent to the identified current data subcube.

[0077] Determining the next position of an aerial vehicle within an aerial corridor based on a trained aerial corridor model includes adapting the aerial corridor based on the trained aerial corridor model to optimize the use of the aerial corridor, and such adaptation includes at least one of subdividing the aerial corridor data cube into eight quadrant subcubes of the aerial corridor, and merging the aerial corridor subcubes into a larger cube of the aerial corridor.

[0078] The program code further instructs the processor to send the identified current location data to at least one agent vehicle within the aerial corridor.

[0079] The control further includes controlling the aircraft vehicle to move from its current position within the aerial corridor to a determined next position within the aerial corridor. This control includes at least one of accelerating the aircraft vehicle, decelerating the aircraft vehicle, causing the aircraft vehicle to increase altitude, causing the aircraft vehicle to decrease altitude, and causing the aircraft vehicle to change direction.

[0080] The aerial corridor includes airspace at altitudes ranging from 400 feet to 2,500 feet.

[0081] Any range or device value given herein may be extended or modified without loss of the desired effect, as will be obvious to those skilled in the art.

[0082] Personally identifiable information is not recorded by the manner of this disclosure, but the embodiments have been described with reference to data monitored and / or collected from users. In some embodiments, users are notified of data collection (e.g., through a dialog box or preference settings), and users are given the opportunity to give or refuse consent to monitoring and / or collection. Consent may take the form of opt-in consent or opt-out consent.

[0083] While this subject matter has been described in language specific to structural features and / or methodological actions, it should be understood that the subject matter as defined in the attached claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are disclosed as exemplary forms that implement the claims.

[0084] It will be understood that the benefits and advantages described above may relate to one embodiment or to several embodiments. Embodiments are not limited to those that solve any or all of the problems described, or that possess any or all of the benefits and advantages described. It will also be understood that a reference to an item may represent one or more such items.

[0085] Embodiments illustrated and described herein, as well as embodiments not specifically described herein but within the scope of the claims, include exemplary means for receiving a trained air corridor model associated with an air corridor representing a departure point, a destination, and airspace between the departure point and the destination, wherein the trained air corridor model is configured to maintain safe separation distances between vehicles in the air corridor; exemplary means for the processor to identify current position data representing the current position of the aircraft vehicle in the air corridor; exemplary means for the processor to receive agent position data associated with the position of at least one agent vehicle in the air corridor; and exemplary means for the processor to determine the next position of the aircraft vehicle in the air corridor based on the trained air corridor model, the current position data, and the agent position data.

[0086] In this specification, the word “comprising” is used to mean that the following feature(s) or function(s) include (without excluding the presence of one or more additional features or functions).

[0087] In some embodiments, the operations shown in the drawings may be implemented in hardware programmed or designed to perform the operations as software instructions encoded in a computer-readable medium, or both. For example, embodiments of the present disclosure may be implemented as a system-on-a-chip or as other circuits comprising a plurality of interconnected conductive elements.

[0088] The order in which the actions are performed or implemented in the embodiments of the Disclosure shown and described herein is not mandatory unless otherwise specified. That is, the actions can be performed in any order unless otherwise specified, and embodiments of the Disclosure may include additional or fewer actions than those disclosed herein. For example, performing or implementing a particular action before, simultaneously with, or after another action is considered to be within the scope of embodiments of the Disclosure.

[0089] When describing elements of the aspects of this disclosure or examples thereof, the articles “a,” “an,” and “the,” “said” are intended to indicate that one or more of such elements exist. The terms “comprising,” “including,” and “having” are intended to be comprehensive and mean that additional elements other than those listed may exist. The term “exemplary” means “an example of.” The phrase “one or more of A, B, and C” means “at least one of A and / or at least one of B and / or at least one of C.”

[0090] Furthermore, this disclosure includes embodiments according to the following examples.

[0091] Example 1. A computer system for controlling an aerial vehicle to move within an aerial corridor while maintaining safe separation from other agent vehicles, wherein the computer system is: Processor, and A non-transient computer-readable medium storing program code for transferring data to another computer system is provided, and the program code is transmitted to the processor, Receiving a trained air corridor model associated with an air corridor representing a departure point, a destination, and the airspace between the departure point and the destination, wherein the trained air corridor model is configured to maintain a safe separation distance between vehicles within the air corridor. Identifying current position data representing the current position of the aircraft vehicle within the aerial corridor, Receiving agent location data associated with the location of at least one agent vehicle within the aerial corridor, and A computer system that performs the task of determining the next position of the aircraft vehicle within the air corridor based on the trained air corridor model, the current position data, and the agent position data.

[0092] Example 2. The aerial corridor includes a joined set of spatiotemporal data cubes representing the portion of the airspace within the aerial corridor from the departure point to the destination, The computer system according to Embodiment 1, wherein at least one data cube of the spatiotemporal data cube junction set is subdivided into a plurality of data subcubes, the size of which is based on a defined minimum safe separation distance.

[0093] Example 3. Identifying the current position data of the current position of the aircraft vehicle within the aerial corridor includes identifying the current data subcube of the aerial corridor in which the aircraft vehicle is currently located. The computer system according to Embodiment 2, wherein determining the next position of the aircraft vehicle within the air corridor based on the trained air corridor model, the current position data, and the agent position data includes determining the next data subcube in the air corridor adjacent to the identified current data subcube.

[0094] Example 4. The computer system according to Embodiment 2, wherein determining the next position of the aircraft vehicle within the air corridor based on the trained air corridor model includes adapting the air corridor based on the trained air corridor model to optimize the use of the air corridor, wherein the adaptation includes at least one of subdividing the data cube of the air corridor into eight quadrant subcubes of the air corridor, and integrating the subcubes of the air corridor into a larger cube of the air corridor.

[0095] Example 5. The computer system according to Embodiment 1, wherein the program code further causes the processor to send the identified current location data to at least one agent vehicle in the aerial corridor.

[0096] Example 6. The computer system according to Embodiment 1, further comprising controlling the aircraft vehicle to move from the current position in the aerial corridor to a determined next position in the aerial corridor, wherein the control includes at least one of accelerating the aircraft vehicle, decelerating the aircraft vehicle, causing the aircraft vehicle to increase altitude, causing the aircraft vehicle to decrease altitude, and causing the aircraft vehicle to change direction.

[0097] Example 7. The aerial corridor includes airspace at an altitude of 400 feet to 2500 feet, as described in Example 1 of the computer system.

[0098] Example 8. A computer-controlled method for controlling an aerial vehicle to move within an aerial corridor while maintaining safe separation from other agent vehicles, The aircraft vehicle's processor receives a trained air corridor model associated with an air corridor representing a departure point, a destination, and the airspace between the departure point and the destination, wherein the trained air corridor model is configured to maintain a safe separation distance between vehicles within the air corridor. The processor identifies current position data representing the current position of the aircraft vehicle within the aerial corridor. The processor receives agent location data associated with the location of at least one agent vehicle within the aerial corridor, and A computer-controlled method comprising determining the next position of the aerial vehicle within the aerial corridor based on the trained aerial corridor model, the current position data, and the agent position data, using the processor.

[0099] Example 9. The aerial corridor includes a joined set of spatiotemporal data cubes representing the portion of the airspace within the aerial corridor from the departure point to the destination, At least one data cube of the spatiotemporal data cube junction set is subdivided into a plurality of data subcubes, the size of which is determined by the computer-controlled method described in Embodiment 8, based on a defined minimum safe separation distance.

[0100] Example 10. Identifying the current position data of the current position of the aircraft vehicle within the aerial corridor includes identifying the current data subcube of the aerial corridor in which the aircraft vehicle is currently located. The computer-controlled method according to Embodiment 9, wherein determining the next position of the aircraft vehicle within the air corridor based on the trained air corridor model, the current position data, and the agent position data includes determining the next data subcube in the air corridor adjacent to the identified current data subcube.

[0101] Example 11. The computer-controlled method according to Embodiment 9, wherein determining the next position of the aircraft vehicle within the air corridor based on the trained air corridor model includes adapting the air corridor based on the trained air corridor model to optimize the use of the air corridor, wherein the adapting includes at least one of subdividing the data cube of the air corridor into eight quadrant subcubes of the air corridor, and integrating the subcubes of the air corridor into a larger cube of the air corridor.

[0102] Example 12. The computer-controlled method according to Embodiment 8, further comprising sending the current location data identified by the processor to at least one agent vehicle in the aerial corridor.

[0103] Example 13. The computer-controlled method according to Embodiment 8, further comprising controlling the aircraft vehicle to move from the current position in the aerial corridor to a determined next position in the aerial corridor, wherein the control includes at least one of accelerating the aircraft vehicle, decelerating the aircraft vehicle, causing the aircraft vehicle to increase altitude, causing the aircraft vehicle to decrease altitude, and causing the aircraft vehicle to change direction.

[0104] Example 14. The aerial corridor is a computer-controlled method according to Example 8, including airspace at an altitude of 400 feet to 2500 feet.

[0105] Example 15. A non-transient computer storage medium storing program code executable by a first computer system at a first site, wherein the program code embodies the following method, and the method is Receiving a trained air corridor model associated with an air corridor representing a departure point, a destination, and the airspace between the departure point and the destination, wherein the trained air corridor model is configured to maintain a safe separation distance between vehicles within the air corridor. Identifying current position data representing the current position of the aircraft vehicle within the aerial corridor, Receiving agent location data associated with the location of at least one agent vehicle within the aerial corridor, and A non-transient computer storage medium that includes determining the next position of the aircraft vehicle within the air corridor based on the trained air corridor model, the current position data, and the agent position data.

[0106] Example 16. The aerial corridor includes a joined set of spatiotemporal data cubes representing the portion of the airspace within the aerial corridor from the departure point to the destination, At least one data cube of the joined set of spatiotemporal data cubes is subdivided into a plurality of data subcubes, the size of which is based on a defined minimum safe isolation distance, as described in Embodiment 15, for a non-transient computer storage medium.

[0107] Example 17. Identifying the current position data of the current position of the aircraft vehicle within the aerial corridor includes identifying the current data subcube of the aerial corridor in which the aircraft vehicle is currently located. A non-transient computer storage medium according to Embodiment 16, wherein determining the next position of the aircraft vehicle in the air corridor based on the trained air corridor model, the current position data, and the agent position data includes determining the next data subcube in the air corridor adjacent to the identified current data subcube.

[0108] Example 18. Determining the next position of the aircraft vehicle within the air corridor based on the trained air corridor model includes adapting the air corridor based on the trained air corridor model to optimize the use of the air corridor, wherein the adaptation includes at least one of subdividing the data cube of the air corridor into eight quadrant subcubes of the air corridor, and integrating the subcubes of the air corridor into a larger cube of the air corridor, as described in Embodiment 16.

[0109] Example 19. The method embodied by the program code further comprises sending the identified current location data to at least one agent vehicle in the aerial corridor, according to Embodiment 15, for a non-transient computer storage medium.

[0110] Example 20. The method embodied by the program code includes controlling the aircraft vehicle to move from the current position in the aerial corridor to a determined next position in the aerial corridor, wherein the control includes at least one of accelerating the aircraft vehicle, decelerating the aircraft vehicle, causing the aircraft vehicle to increase altitude, causing the aircraft vehicle to decrease altitude, and causing the aircraft vehicle to change direction, as described in Embodiment 15 of the non-transient computer storage medium.

[0111] While aspects of this disclosure have been described in detail, it will be apparent that modifications and changes are possible without departing from the scope of the aspects of this disclosure as defined in the attached claims. Since various modifications can be made to the above structures, products, and methods without departing from the scope of the aspects of this disclosure, all matters included in the above description and illustrated in the attached drawings are intended to be interpreted as illustrative and not limiting.

Claims

1. 1. A computer system (100) for controlling an air vehicle (102) to move within an air corridor (200) while maintaining a safe separation from other agent vehicles (122), the computer system (100) comprising: A processor (719), and a non-transitory computer readable medium having stored thereon a program code (721) for transferring data to another computer system (100), the program code (721) causing the processor (719) to: receiving a trained air corridor model (106) associated with an origin (202), a destination (204), and an air corridor (200) representing an airspace between the origin (202) and the destination (204), the trained air corridor model (106) configured to maintain a safe separation distance between vehicles in the air corridor (200); determining current position data (114) representative of a current position of the air vehicle (102) within the air corridor (200); receiving agent position data (118) associated with a position of at least one agent vehicle (122) within said air corridor (200); determining a next position of the air vehicle (102) within the air corridor (200) based on the trained air corridor model (106), the current position data (114), and the agent position data (118).

2. the air corridor (200) includes a joined set of space-time data cubes (206) representing a portion of airspace within the air corridor (200) from the origin (202) to the destination (204); 2. The computer system of claim 1, wherein at least one data cube of a joined set of the space-time data cube is subdivided into a plurality of data subcubes, the size of the data subcubes being based on a defined minimum safe separation distance.

3. Determining the current position data (114) for the current position of the air vehicle (102) within the air corridor (200) includes determining a current data subcube (310, 312, 314) of the air corridor (200) in which the air vehicle (102) is currently located; 3. The computer system (100) of claim 1 or 2, wherein determining the next position of the air vehicle (102) in the air corridor (200) based on the trained air corridor model (106), the current position data (114), and the agent position data (118) comprises determining a next data subcube (310, 312, 314) of the air corridor (200) adjacent to the identified current data subcube (310, 312, 314).

4. 3. The computer system of claim 2, wherein determining the next position of the air vehicle in the air corridor based on the trained air corridor model includes adapting the air corridor based on the trained air corridor model to optimize use of the air corridor, the adapting including at least one of: subdividing a data cube of the air corridor into subcubes of eight quadrants of the air corridor; and aggregating the subcubes of the air corridor into a larger cube of the air corridor.

5. 2. The computer system (100) of claim 1, further comprising: controlling the air vehicle (102) to move from the current position within the air corridor (200) to the determined next position within the air corridor (200), wherein the controlling comprises at least one of accelerating the air vehicle (102), decelerating the air vehicle (102), causing the air vehicle (102) to increase in altitude, causing the air vehicle (102) to decrease in altitude, and causing the air vehicle (102) to change direction.

6. 1. A computer-controlled method for controlling an air vehicle (102) to travel within an air corridor (200) while maintaining a safe separation from other agent vehicles (122), comprising: receiving, by a processor (719) of the air vehicle (102), a trained air corridor model (106) associated with an origin (202), a destination (204), and an air corridor (200) representing an airspace between the origin (202) and the destination (204), the trained air corridor model (106) configured to maintain a safe separation distance between vehicles in the air corridor (200); determining, by said processor (719), current position data (114) representative of a current position of said air vehicle (102) within said air corridor (200); receiving, by said processor, agent position data associated with a position of at least one agent vehicle within said air corridor; determining, by the processor (719), a next position of the air vehicle (102) within the air corridor (200) based on the trained air corridor model (106), the current position data (114), and the agent position data (118).

7. the air corridor (200) includes a joined set of space-time data cubes (206) representing a portion of airspace within the air corridor (200) from the origin (202) to the destination (204); 7. The computer controlled method of claim 6, wherein at least one data cube (310, 312, 314) of a joined set of the space-time data cube (206) is subdivided into a plurality of data subcubes (310, 312, 314), the size of the data subcubes (310, 312, 314) being based on a defined minimum safe separation distance.

8. Determining the current position data (114) for the current position of the air vehicle (102) within the air corridor (200) includes determining a current data subcube (310, 312, 314) of the air corridor (200) in which the air vehicle (102) is currently located; 8. The computer controlled method of claim 6 or 7, wherein determining the next position of the air vehicle (102) within the air corridor (200) based on the trained air corridor model (106), the current position data (114), and the agent position data (118) comprises determining a next data subcube (310, 312, 314) of the air corridor (200) adjacent to the identified current data subcube (310, 312, 314).

9. 8. The computer controlled method of claim 7, wherein determining the next position of the air vehicle (102) within the air corridor (200) based on the trained air corridor model (106) comprises adapting the air corridor (200) based on the trained air corridor model (106) to optimize use of the air corridor (200), the adapting comprising at least one of: subdividing a data cube (310, 312, 314) of the air corridor (200) into subcubes (310, 312, 314) of eight quadrants of the air corridor (200) and aggregating the subcubes (310, 312, 314) of the air corridor (200) into a larger cube (310, 312, 314) of the air corridor (200).

10. 9. The computer-controlled method of claim 8, further comprising controlling the air vehicle (102) to move from the current position within the air corridor (200) to the determined next position within the air corridor (200), wherein the controlling comprises at least one of accelerating the air vehicle (102), decelerating the air vehicle (102), causing the air vehicle (102) to increase altitude, causing the air vehicle (102) to decrease altitude, and causing the air vehicle (102) to change direction.

Images