Aircraft deployment method based on dynamic airspace network and reinforcement learning

Abstract

The invention discloses an aircraft deployment method based on a dynamic airspace network and reinforcement learning, and relates to the field of air traffic management, and the method comprises the steps: obtaining flight information and task priority weight information of each flight in a historical time period and a current time period; dynamically dividing the airspace into an independent sub-airspace and a shared airspace by using flight information in a historical time period; calculating the safe area of each flight through a space-time buffer zone model, and determining a flight path according to the safe area; and according to the flight information, the airspace division result and the flight path, a release action instruction set including height adjustment, course correction and speed change is generated, and real-time deployment is performed on the flight. The technical effects of the invention lie in that airspace changes can be responded in real time, flight paths of flights are optimized, potential conflicts are reduced, and the airspace utilization rate is improved, so that the air traffic control efficiency and the flight safety are improved.

Description

Technical Field

[0001] This application relates to the field of air traffic management, and in particular to an aircraft allocation method based on a dynamic airspace network and reinforcement learning. Background Art

[0002] The air traffic control system plays a crucial role in ensuring the safe and orderly operation of flights and the rational allocation of airspace resources. It not only guarantees the safety of passengers but also reduces costs by improving efficiency. With the continuous growth of air traffic flow, the efficiency and safety of air traffic control become even more critical.

[0003] Flight simulator training, as the core means of cultivating pilots and enhancing the capabilities of air traffic controllers, plays an irreplaceable role. In the highly simulated environment constructed by the flight simulator, extremely complex and diverse flight scenarios can be simulated. Among them, how to make scientific and reasonable aircraft allocation decisions based on the current situation to assist the decision-making of controllers is the key point and difficulty in training, which is directly related to flight safety and efficiency. However, the existing aircraft allocation methods have certain limitations when dealing with complex and changeable flight simulator training scenarios and are difficult to meet the growing training needs.

[0004] Therefore, in-depth research on aircraft allocation methods based on current situation analysis is of great practical significance for improving the quality of flight simulator training, enhancing the ability of air traffic controllers to handle complex situations, and thus ensuring the safety and smoothness of actual air transportation. Summary of the Invention

[0005] The purpose of this application is to provide an aircraft allocation method based on a dynamic airspace network and reinforcement learning, which can improve air traffic control efficiency, reduce conflict risks, and meet the air transportation needs of multiple flights and multiple aircraft types.

[0006] To achieve the above purpose, this application provides the following solutions: This application provides an aircraft allocation method based on a dynamic airspace network and reinforcement learning, including: Obtain the flight information and task priority weight information of each flight in the historical time period and the current time period; According to the flight information of each flight in the historical time period, divide the airspace units through a dynamic sub-airspace network to obtain an airspace division result; the airspace division result includes independent sub-airspaces and shared airspaces; Input the flight information and task priority weight information of each flight in the current time period into the spatio-temporal buffer model to obtain the current flight path set; the spatio-temporal buffer model is used to calculate the safety area area of each flight according to the flight information, and determine the starting point, passing points, ending point and flight time arrangement of each flight according to the safety area area and task priority weight information of each flight to obtain the current flight path set; According to the flight information of each flight in the current time period, the airspace division result and the current flight path set, obtain the current relief action instruction set; the relief action instruction set includes altitude adjustment instructions, heading correction instructions and speed change instructions; Dispatch each flight in the current time period according to the current relief action instruction set.

[0007] According to the specific embodiments provided by the present application, the present application has the following technical effects: The present application provides an aircraft dispatching method based on a dynamic airspace network and reinforcement learning. By obtaining the flight information and task priority weight information of each flight in the historical time period and the current time period, it can accurately grasp the real-time state and flight conditions of each flight, solve the problem of lagging information update in traditional air traffic control, and realize the real-time monitoring and analysis of flight dynamics; based on the flight information of each flight in the historical time period, divide the airspace through a dynamic sub-airspace network to obtain an airspace division result including independent sub-airspaces and shared airspaces, solve the problems of uneven allocation and low utilization efficiency of airspace resources, and realize the dynamic adjustment and optimal allocation of airspace resources; by inputting the flight information and task priority weight information into the spatio-temporal buffer model, calculate the safety area area of each flight, and determine the flight path set accordingly, solve the problems of unscientific and inaccurate flight path planning of flights, and realize effective path planning based on safety and priority; by generating a relief action instruction set including altitude adjustment, heading correction and speed change according to the flight state information, airspace division result and flight path set, and dispatching each flight, solve the problems of potential conflicts between flights and inflexible dispatching, and realize the precise dispatching and safe flight of flights.

[0008] In summary, the present application not only improves the efficiency and safety of air traffic control, but also enhances the utilization efficiency of airspace resources, meets the air transportation needs of multiple flights and multiple aircraft types, and has significant practical application value and broad development prospects. Brief Description of the Drawings

[0009] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for use in the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0010] Figure 1 This is an application environment diagram of an aircraft allocation method based on a dynamic airspace network and reinforcement learning in an embodiment of the present application.

[0011] Figure 2 This is a schematic flowchart of an aircraft allocation method based on a dynamic airspace network and reinforcement learning provided in an embodiment of the present application.

[0012] Figure 3 This is a schematic flowchart of an aircraft allocation method based on a dynamic airspace network and reinforcement learning provided in another embodiment of the present application.

[0013] Figure 4 This is a schematic structural diagram of a computer device provided in an embodiment of the present application. Detailed implementation manners

[0014] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0015] To make the above objects, features, and advantages of the present application more obvious and understandable, the present application will be further described in detail below with reference to the accompanying drawings and specific implementation manners.

[0016] The aircraft allocation method based on a dynamic airspace network and reinforcement learning provided in the embodiments of the present application can be applied to, for example Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through a network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, placed on the cloud or other servers. The terminal 102 can send the flight information and task priority weight information of each flight in the historical time period and the current time period to the server 104. After receiving the flight information and task priority weight information of each flight in the historical time period and the current time period, the server 104 divides the airspace units through the dynamic sub-airspace network according to the flight information of each flight in the historical time period, and obtains the airspace division result; the airspace division result includes independent sub-airspaces and shared airspaces; input the flight information and task priority weight information of each flight in the current time period into the spatio-temporal buffer zone model to obtain the current flight path set; the spatio-temporal buffer zone model is used to calculate the safety area area of each flight according to the flight information, and determine the starting point, passing points, ending point and flight time arrangement of each flight according to the safety area area of each flight and the task priority weight information to obtain the current flight path set; according to the flight information, airspace division result and current flight path set of each flight in the current time period, obtain the current release action instruction set; the release action instruction set includes altitude adjustment instructions, course correction instructions and speed change instructions; allocate each flight in the current time period according to the current release action instruction set. The server 104 can feedback the obtained current release action instruction set to the terminal 102. In addition, in some embodiments, the aircraft allocation method based on the dynamic airspace network and reinforcement learning can also be implemented separately by the server 104 or the terminal 102. For example, the terminal 102 can directly perform flight allocation processing on the flight information and task priority weight information of each flight in the historical time period and the current time period, or the server 104 can obtain the flight information and task priority weight information of each flight in the historical time period and the current time period from the data storage system, and perform flight allocation processing on the flight information and task priority weight information of each flight in the historical time period and the current time period.

[0017] Among them, the terminal 102 can be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers, and can also be a cloud server.

[0018] In an exemplary embodiment, such as Figure 2As shown, a method for aircraft allocation based on a dynamic airspace network and reinforcement learning is provided. This method is executed by a computer device, which can be specifically executed by a computer device such as a terminal or a server alone, or jointly executed by a terminal and a server. In the embodiments of the present application, taking the application of this method to Figure 1 server 104 in as an example, it includes the following steps 201 to 205. Among them:

[0019] Step 201, obtain the flight information and task priority weight information of each flight in the historical time period and the current time period. The flight information includes flight position information, flight speed information, heading angle information, aircraft type parameter information, and meteorological data information.

[0020] Step 202, divide airspace units through a dynamic sub-airspace network according to the flight information of each flight in the historical time period, and obtain an airspace division result; the airspace division result includes independent sub-airspaces and shared airspaces.

[0021] Step 203, input the flight information and task priority weight information of each flight in the current time period into a spatio-temporal buffer model to obtain a current flight path set; the spatio-temporal buffer model is used to calculate the safety area area of each flight according to the flight information, and determine the starting point, passing points, ending point, and flight time arrangement of each flight according to the safety area area of each flight and the task priority weight information, so as to obtain the current flight path set.

[0022] Step 204, obtain a current release action instruction set according to the flight information, airspace division result, and current flight path set of each flight in the current time period; the release action instruction set includes altitude adjustment instructions, heading correction instructions, and speed change instructions.

[0023] Step 205, allocate each flight in the current time period according to the current release action instruction set.

[0023] By implementing the above steps 201 to 205, the present application can achieve efficient, safe, and automated management of air traffic, optimize airspace resource allocation, improve flight operation efficiency, enhance flight safety, and is particularly suitable for complex air transportation demands with multiple flights and multiple aircraft types.

[0024] In another exemplary embodiment of the present application, step 202 specifically includes: Divide the airspace into multiple cubic grids.

[0025] According to the flight information of each flight in the historical time period, determine the flight density between adjacent cubic grids, and calculate the airspace division modularity through a modularity function based on the flight density.

[0026] Through the greedy algorithm, with the goal of maximizing the increase in the modularity of airspace division, adjacent cubic meshes are merged to obtain the initial airspace unit division result.

[0027] Calculate the flight density between adjacent airspace units in the initial airspace unit division result, and return to the step of "Through the greedy algorithm, with the goal of maximizing the increase in the modularity of airspace division, adjacent cubic meshes are merged to obtain the initial airspace unit division result" until the airspace is divided into a state with only two airspace units, and obtain the airspace division result; among them, a high-traffic airspace unit with a high flight density is used as an independent sub-airspace; another low-traffic airspace unit with a low flight density is used as a shared airspace.

[0028] As an optional implementation manner, the modularity function is: 。

[0029] Among them, represents the modularity of airspace division at the t-th moment; represents the flight density from the i-th cubic mesh to the j-th cubic mesh at the t-th moment; represents the total number of flights flowing in and out of the i-th cubic mesh at the t-th moment; represents the total number of flights flowing in and out of the j-th cubic mesh at the t-th moment; represents the total number of flights flowing in and out of the airspace at the t-th moment; represents the airspace unit to which the i-th cubic mesh belongs at the t-th moment; represents the airspace unit to which the j-th cubic mesh belongs at the t-th moment; represents the first indicator function.

[0030] In another exemplary embodiment of the present application, step 203 specifically includes: The spatio-temporal buffer zone model includes a safety area calculation module and a time window insertion module connected in sequence.

[0031] The safety area calculation module uses the following formula to calculate the safety area of each flight during the current time period: 。

[0032] Among them, represents the safety area of the -th flight at the t-th moment; The task priority weight of the -th flight; represents the -th flight's minimum turning radius of the aircraft; is the control response time; Indicates the maximum speed of the aircraft for the th flight.

[0033] The time window insertion module is used for: Traverse the current flight path set. If the time interval between any two flights arriving at the current path intersection is less than the preset time interval, then according to the task priority weight information, insert a flight with a low priority weight into the preset time window delay until all flights at the current path intersection meet the preset stop condition, and then judge the next path intersection.

[0034] As an alternative implementation, the task priority weight information specifically includes: commercial task priority weight, freight task priority weight, and military task priority weight; among them, the freight task priority weight < commercial task priority weight < military task priority weight.

[0035] As an alternative implementation, the calculation formula for the preset time interval is: .

[0036] Where represents the preset time interval for any th flight and the th flight arriving at the current path intersection at time t; represents the safety area area of the th flight at time t; represents the safety area area of the th flight at time t; represents the speed of the th flight at time t; represents the speed of the th flight at time t; represents the preset minimum time interval.

[0037] In another exemplary embodiment of the present application, step 204 specifically includes: Taking the flight information and airspace division result of each flight within the current time period as the input state information of each flight.

[0038] Based on the input state information of each flight, use the DQN algorithm to adjust the actions of each flight to obtain the current local relief action set.

[0039] According to the current local relief action set, use the genetic algorithm to obtain the current relief action instruction set.

[0040] In another exemplary embodiment of the present application, based on the input status information of each flight, the DQN algorithm is used to adjust the actions of each flight to obtain the current local release action set, which specifically includes: The input status information of each flight is respectively input into the trained deep Q-network model, and the Q-values of all actions are calculated through the following formula: .

[0041] Wherein, represents the Q-value when the -th flight takes the a-th action; represents the task priority weight of the -th flight; , and respectively represent the first weight coefficient, the second weight coefficient and the third weight coefficient; represents the preset minimum interval distance; represents the change in energy consumption after the -th flight takes the a-th action.

[0042] By respectively comparing the Q-values of all actions in each flight and respectively selecting the actions corresponding to the maximum Q-values as the optimal actions of each flight, the local release action set at the current moment is obtained.

[0043] In another exemplary embodiment of the present application, according to the current local release action set, the current release action instruction set is obtained through the genetic algorithm, which specifically includes: Based on the local release action set at the current moment, the influence weight of the -th flight on the -th flight is calculated through the following formula: .

[0044] Wherein, represents the influence weight of the -th flight on the -th flight; and respectively represent the feature vectors corresponding to the optimal actions of the -th flight and the -th flight; represents concatenating the feature vectors corresponding to the optimal actions of the -th flight and the -th flight; represents the activation function, represents the normalization function.

[0045] Based on the influence of the -th flight on the The weight of each flight impact is used to select the action combination that minimizes the number of global conflicts through a genetic algorithm, and the current relief action instruction set is obtained.

[0046] The calculation formula for the number of global conflicts is: 。

[0047] 。

[0048] Among them, represents the number of global conflicts; represents the second indicator function, represents the th flight and the th flight, represents the preset safety distance.

[0049] In another exemplary embodiment of the present application, an aircraft allocation method based on a dynamic airspace network and reinforcement learning is provided. As Figure 3 shown, it specifically includes: Step 1: Design of the dynamic sub-airspace network.

[0050] In actual air traffic, the flight flows in different time periods and regions vary greatly. The static airspace division method in the related art cannot be flexibly adjusted according to the real-time traffic flow, which easily leads to congestion in some airspaces while some airspace resources are idle, reducing the overall utilization rate of airspace resources. Therefore, a method capable of dynamically dividing airspace units according to real-time traffic flow is needed to optimize the allocation and use of airspace resources. For this purpose, a dynamic sub-airspace network is designed, which can dynamically divide airspace units according to real-time traffic flow and optimize the utilization rate of airspace resources.

[0051] Specifically, the input of the dynamic sub-airspace network is the real-time flight position (latitude, longitude, altitude), and the aircraft type parameters (speed, climb rate) and meteorological data (wind speed, turbulence area) can also be added for reference.

[0052] Among them, the real-time flight position can determine the specific position and movement trajectory of the flight in the airspace; the aircraft type parameters affect the flight performance and behavior of the aircraft; and the meteorological data will have an impact on the flight safety and efficiency of the flight.

[0053] The output of the dynamic sub-airspace network is the dynamically adjusted airspace division result (topological structure).

[0054] The specific process is as follows: Step 1.1: Airspace graph modeling.

[0055] The airspace is divided into cubic grids of 1 km × 1 km × 300 m. Each grid is a graph node, and the edge weight is the flight density between adjacent grids. Therefore, each grid has at most 8 edges connected to the surrounding 8 grids. The selection of such grid size comprehensively considers the management accuracy and computational complexity of the airspace. Smaller grids can represent the position and traffic distribution of flights more accurately, but will increase the computational amount; larger grids, although with a smaller computational amount, may not accurately reflect the flight distribution. The size of 1 km × 1 km × 300 m achieves a better balance between the two. Taking each grid as a graph node and setting the edge weight as the flight density between adjacent grids can intuitively reflect the flight traffic relationship between different grids and provide a basis for subsequent airspace division.

[0056] The flight density can be calculated as follows: Using the real-time flight position information, count the flight traffic between each grid and its adjacent grids at each moment within a certain period (such as the historical 12 months). Specifically, it can be calculated by counting the grids passed by each flight: For a certain flight T, according to its historical real-time flight position, the specific position and movement trajectory of this flight in the airspace can be obtained. Then, when it flies out of the airspace grid node i and enters the adjacent grid node j, the weight of i→j is incremented by 1. Finally, the number of flight schedules between adjacent grids is obtained and divided by 12 months, which is the flight density, that is, the traffic between grids. 。

[0057] Step 1.2, Dynamic modularity calculation.

[0058] Define the modularity function: 。

[0059] Among them, represents the airspace partition modularity at the t-th moment; represents the flight density from the i-th cubic grid to the j-th cubic grid at the t-th moment, reflecting the degree of connection tightness between the two grids; represents the total number of flights flowing in and out of the i-th cubic grid at the t-th moment, that is, the sum of the number of flights flowing in and out of grid i, reflecting the importance of the i-th cubic grid in the entire airspace; represents the total number of flights flowing in and out of the j-th cubic grid at the t-th moment; represents the total number of flights flowing in and out of the airspace at the t-th moment; represents the airspace unit to which the i-th cubic grid belongs at the t-th moment; represents the airspace unit to which the j-th cubic grid belongs at the t-th moment; represents the first indicator function.

[0060] The modularity function measures the rationality of the current airspace division by calculating the flow differences between different grids. The larger the modularity Q value, the more reasonable the airspace division is. Indicates whether the i-th cubic grid and the j-th cubic grid belong to the same airspace unit. When At this time, Otherwise . That is, when the two belong to the same airspace unit, it is included in the modularity, otherwise it is not.

[0061] Step 1.3, Real-time optimization.

[0062] Merging high-traffic areas into independent sub-airspaces can reduce interference between different flights and improve flight efficiency; merging low-traffic areas into shared airspaces can avoid waste of resources and improve airspace utilization.

[0063] Maximize the Q value through the greedy algorithm, merge high-traffic areas into independent sub-airspaces, and merge low-traffic areas into shared airspaces. The greedy algorithm is an algorithm that makes the best or optimal (i.e., most favorable) choice at each step of the selection, hoping to lead to a globally optimal result. In this scenario, by continuously selecting operations that can increase the modularity Q value (i.e., merging or splitting airspace units), the optimal airspace division method is gradually found.

[0064] Specifically, taking the grid units divided in Step 1.1 as the initial state of the airspace units, calculate the Q value according to Step 1.2; then traverse all airspace units for merging operations, calculate the change in the Q value under each operation, and select the operation that increases the Q value and has the largest increase amplitude to execute; after executing the operation, update the topological structure of the airspace units and recalculate the Q value, repeat the above operations; until no operation that can increase the Q value is found, the algorithm terminates, and the airspace division method at this time is the result that maximizes the Q value.

[0065] Among them, implementing this implementation method, the result can be updated at a certain time interval. For example, the airspace is divided every hour according to the flight situation within the next hour. Or when a conflict occurs, it is updated according to the new route situation after adjustment and release.

[0066] Step 2, Multi-aircraft type classification and scheduling.

[0067] There are significant differences in the performance of aircraft of different types, such as speed, maneuverability, etc. In air traffic control, if these differences are not considered and a unified scheduling strategy is adopted, it is easy to lead to scheduling conflicts and flight delays. Therefore, it is necessary to design a hierarchical priority queue according to the performance differences of aircraft types to more reasonably arrange the flight paths of flights and reduce scheduling conflicts.

[0068] Therefore, based on the performance differences of aircraft models, a multi-type aircraft scheduling method (MTAS) is designed. By designing a spatio-temporal buffer zone model, scheduling conflicts are reduced.

[0069] The input of step 2 is the aircraft type parameters (maximum speed , minimum turning radius ) in the flight information, and the task urgency (commercial, freight, military, with priority weights set to 1, 0.9, and 1.1 respectively; for commercial aircraft, in this application, the aircraft are divided into 3 categories according to speed and maneuverability, with weights of 1.05, 1, and 0.95 respectively. See step 2.1 for details). Among them, the aircraft type parameters directly affect the flight ability and behavior of the aircraft. The maximum speed determines the upper limit of the flight speed of the aircraft, and the minimum turning radius reflects the maneuverability of the aircraft. The task urgency reflects the importance and urgency of the flight task, and this information can be used to determine the priority and scheduling strategy of the flight. The output is a set of flight paths sorted by priority.

[0070] A spatio-temporal buffer zone model is proposed, which combines queuing theory and kinematic constraints to optimize flight conflicts. Queuing theory can be used to analyze the time relationship of flights during waiting and flight, and kinematic constraints consider the physical characteristics of the aircraft and flight safety requirements. By combining these two, the designed "spatio-temporal buffer zone" model can handle conflicts between flights more accurately and ensure flight safety.

[0071] The specific process is as follows: Step 2.1, Aircraft type classification.

[0072] For commercial aircraft, in this application, the aircraft are divided into 3 categories according to speed and maneuverability, with weights of 1.05, 1, and 0.95 respectively.

[0073] Class A (large airliner): , high priority but poor maneuverability. Large airliners usually carry more passengers, and the importance of their flight tasks is relatively high, so they are given high priority. However, due to their large volume, their maneuverability is relatively poor, and more space and time are required for operation during flight.

[0074] Class B (medium-sized aircraft): , medium priority. Medium-sized aircraft are at an intermediate level in terms of speed and maneuverability, and their priority is set to medium accordingly.

[0075] Class C (small aircraft / drone): , with low priority but high mobility. Small aircraft and drones usually perform some auxiliary or specific tasks, and their importance is relatively low, so the priority is set to low. However, they have strong mobility and can adjust their flight paths more flexibly during flight.

[0076] Step 2.2, Generation of spatio-temporal buffer zone.

[0077] Generate a safety area for each flight. The calculation formula for the area of the safety area is: .

[0078] Among them, represents the area of the safety area of the th flight at the t-th moment; The th flight's task priority weight; represents the th flight's minimum turning radius of the aircraft; is the control response time; represents the th flight's maximum speed of the aircraft. The meaning of this formula is that, combined with its priority weight, taking the minimum turning radius of the th flight's aircraft plus the distance traveled by the aircraft at the maximum speed within the control response time as the radius R of the safety area, that is: , to generate a circular safety area. This safety area ensures that the aircraft has enough space during flight to handle emergencies and avoid collisions with other aircraft.

[0079] Insert a time window at the path intersection: If the time difference between the expected entry times of two flights into the same area minutes, then delay the low-priority flight according to the priority weight to meet the constraint: .

[0080] Among them, represents the preset time interval between any th flight and the th flight arriving at the current path intersection at the t-th moment; represents the area of the safety area of the th flight at the t-th moment; represents the area of the safety area of the th flight at the t-th moment; represents the speed of the th flight at the t-th moment; represents the speed of the th flight at the t-th moment; Indicates the preset minimum time interval. This process first determines the time difference between the expected entry times of two flights into the same area whether it is less than 2 minutes. If minutes, it means the time interval is sufficient and no special processing is required; if minutes, further judgment is needed. Calculate the new time interval . Then determine whether the time difference between the expected entry times of the two flights into the same area is less than the new time interval . If it is less, delay the low-priority flight. 2 minutes is a preset initial safety time interval threshold. Making this judgment first is to quickly screen out flight situations with relatively sufficient time intervals and no risk of conflict in the initial stage, reducing unnecessary complex calculations. For situations where the time difference is less than 2 minutes, further determine whether to delay the flight by precisely calculating the new time interval to balance calculation efficiency and safety guarantee.

[0081] The purpose of this constraint is to ensure that there is sufficient time interval between two flights when entering the same area to avoid conflicts. By calculating the sum of the safety area areas of the two flights divided by the sum of their speeds, a time value is obtained, and the larger value between this time value and 2 minutes is used as the new time interval . If the time difference between the expected entry times of the two flights into the same area is less than the new time interval , delay the low-priority flight, and finally obtain a set of flight paths sorted by priority to meet safety requirements. This set of flight paths is a combination of flight paths obtained after sorting flights of different aircraft types and different mission urgencies according to priority E after obtaining today's flight tasks. In this set, the flight paths of high-priority flights are determined first and are more guaranteed, and the flight paths of low-priority flights are adjusted according to high-priority flights and safety constraints. The representation of this set is a list, where each element represents the flight path information of a flight, including the starting point, passing points, ending point, and time arrangement during the flight, indicating the flight order and path planning of different flights.

[0082] In addition, the size of the time window for inserting the delayed low-priority flight is not fixed. First, a time value is calculated based on the sum of the safety area areas and the sum of the speeds of the two flights, and the larger value between this time value and the preset minimum time interval is taken to obtain the new time interval , and this is used to determine the size of the time window for inserting the delayed low-priority flight.

[0083] Step 3: Conflict prediction and resolution based on multi-agent reinforcement learning.

[0084] For the today's flight plan generated according to Step 2, during the execution process, in order to monitor the flight status of the aircraft to avoid possible dangerous approaches and other conflicts, a conflict prediction and resolution method based on multi-agent reinforcement learning is designed to monitor the aircraft status in real time. In a complex air traffic environment, conflicts between flights may occur at any time. Traditional conflict prediction and resolution methods often rely on fixed rules and models, lacking real-time performance and flexibility. Multi-agent reinforcement learning allows each flight to act as an agent and make autonomous decisions based on its own and the surrounding environment states, thus predicting and resolving conflicts more effectively. For this purpose, a MARL-CDR module is constructed to predict conflicts in real time through distributed agents and generate resolution strategies.

[0085] Specifically, its input flight status is (position (x, y, z), speed v, heading angle ), and whether this position is a shared area (determined according to the DSAN division result designed in Step 1). Among them, flight status information is the basis for the agent to make decisions, used to understand the positions, speeds, and heading angles of itself and surrounding flights, and support the agent to predict potential conflicts. The DSAN division result provides the structural information of the current airspace, and the agent can adjust its decisions according to the division of the airspace.

[0086] Its output is a set of resolution instructions (altitude adjustment , heading correction , speed change ).

[0087] Design a "local-global" double-loop reinforcement learning architecture that can integrate individual decisions and global coordination. This architecture not only considers the local decisions of each flight agent to quickly respond to potential conflicts, but also comprehensively evaluates and adjusts the decisions of all agents through a global coordinator to ensure overall flight safety and efficiency.

[0088] The specific process is as follows: Step 3.1: Construct local agents (Local Agent): Each flight corresponds to an agent, and the DQN algorithm (Deep Q-Network) is used to optimize individual actions. The agent consists of the following two aspects: The state space is the state of itself and neighboring (within 10 km) flights, which is used as input. Selecting 10 km as the neighboring range comprehensively considers the mutual influence range between flights and the computational complexity. Flights within this range have a greater impact on the potential conflicts of the current flight, and at the same time, it will not increase too much computational burden due to considering too much flight information.

[0089] The output action space metrics include climbing 100m, descending 100m, turning left 5°, turning right 5°, accelerating 5%, and decelerating 5%, which can be represented by a vector of length 6. These actions are common ways for an aircraft to adjust its flight state. By choosing appropriate actions, the agent can avoid conflicts with other flights.

[0090] Through the reward function, the agent can learn an action strategy that minimizes energy consumption while ensuring safety. The specific formula for the reward Q function is: 。

[0091] Among them, represents the Q value of the -th flight taking the a-th action; represents the task priority weight of the -th flight; , and represent the first weight coefficient, the second weight coefficient, and the third weight coefficient respectively; represents the preset minimum separation distance, which is obtained by calculating from the state space. That is, for the state space of a certain agent, the Euclidean distance to the nearest aircraft calculated according to its three-dimensional coordinates is used as ; represents the change in energy consumption after the -th flight takes the a-th action, which is related to the size and power of the specific aircraft type, and each aircraft type can obtain it according to its performance indicators. and are used to balance the effects of the minimum separation distance and the change in energy consumption on the reward. represents the impact of the generated adjustment action strategy on the airspace distribution. When the output action causes the aircraft to enter the shared airspace from the independent sub-airspace, = 1.5; when the output action causes the aircraft to enter the independent sub-airspace from the shared airspace, = 0.7; when the output action causes the aircraft to remain in the independent sub-airspace, = 0.9; when the output action causes the aircraft to remain in the shared airspace, = 1.

[0092] In the above way, the model is prompted to use the shared sub-airspace with low traffic to resolve conflicts. The larger the minimum separation distance , the safer the flights are and the higher the reward. The smaller the change in energy consumption , the higher the flight efficiency of the flight and the higher the reward.

[0093] Specifically, build the DQN and the target network, form input-output data pairs from the agent state, actions, etc., and randomly sample for training. Calculate the reward value Q and update the DQN parameters based on Q. After training, input a state space and output a specific action selected from the action space.

[0094] Judge according to Step 2.2 whether it meets the safety standards. For agents within the safety area radius R, input their state space into the DQN to obtain the actions output for the aircraft agent. It can be represented by a 1×6 vector. Among them, a certain bit in the vector being 1 represents selecting a certain action space.

[0095] Step 3.2, Global Coordinator: Suppose there are N flights. Based on the actions for each flight agent obtained in Step 3.1, an N×6 action combination can be obtained, representing that each flight has 6 selectable actions. Based on this action combination matrix, evaluate the conflict risk and obtain the final action combination for all N flights today, including the following 2 steps: ① Calculate the influence weight of flight i on flight j: .

[0096] Among them, represents the influence weight of the th flight on the th flight; and respectively represent the feature vectors corresponding to the optimal actions of the th flight and the th flight; represents concatenating the feature vectors corresponding to the optimal actions of the th flight and the th flight; represents the activation function, represents the normalization function. In this way, calculate the influence weight of the th flight on the th flight. The greater the weight, the stronger the correlation or potential conflict trend between certain features (such as position, heading, etc.) of the th flight and the th flight, that is, it means that the th flight has a greater influence on the th flight.

[0097] ② Output the final action: Select the action combination that minimizes the global conflict count . Among them, represents the number of global conflicts; represents the second indicator function, ; represents the distance between the th flight and the th flight,

[0098] Select the action combination that minimizes the number of global conflicts through a genetic algorithm to ensure the flight safety of flights throughout the airspace from a global perspective. Specifically, first, consider the action combination of each flight as a chromosome of length N, and randomly generate an initial population of action combinations. Each chromosome represents a choice of action combination. For each action combination in the population, calculate its number of global conflicts as the fitness. Next, perform a selection operation to select the action combinations with fewer conflicts to enter the next generation. Then, through a crossover operation, randomly select two action combinations and exchange some actions to generate new combinations; at the same time, mutate some action combinations with a certain probability to randomly change individual actions. Continuously repeat these steps to gradually evolve the population and finally approach the optimal action combination, that is, the combination with the minimum number of conflicts. Executing the actions corresponding to this combination can achieve the maximum degree of conflict resolution.

[0099] During the process of flight day air traffic controllers' allocation of aircraft, the module designed in this step can read in the real-time status information of each flight, and then analyze and output the optimal action combination as a deployment suggestion for air traffic controllers to avoid potential conflicts in the actual deployment process.

[0100] In summary, this application has the following technical effects: By step 1, designing a dynamic sub-airspace network (DSAN), it can dynamically adjust the division of airspace units according to real-time flight positions, aircraft type parameters, and meteorological data, thereby achieving the optimization of airspace resource utilization. Specifically, DSAN divides the airspace into fine-grained cubic grids and calculates the modularity value based on the flight density between adjacent grids, enabling high-traffic areas to be merged into independent sub-airspaces to reduce interference, and low-traffic areas to be merged into shared airspaces to avoid resource waste. This method not only improves flight efficiency but also reduces the risk of congestion and delays.

[0101] Through Step 2, a multi-type aircraft classification scheduling method (MTAS) is designed. It can classify flights according to the different performances and task urgencies of aircraft and generate spatio-temporal buffer zones, so as to reduce scheduling conflicts and improve flight safety. MTAS first classifies aircraft into three categories according to speed and maneuverability, then uses queuing theory and kinematic constraints to generate the safety zones of each flight, and inserts time windows at path intersections to ensure sufficient intervals and prevent potential conflicts from occurring. Therefore, flights with different priorities can be reasonably scheduled according to their characteristics, ensuring the orderly progress of each flight in air transportation.

[0102] Through Step 3, a conflict prediction and resolution module based on multi-agent reinforcement learning (Multi-Agent Reinforcement Learning for Conflict Detection and Resolution, MARL-CDR) is constructed. It enables each flight to act as an agent and make autonomous decisions based on its own and surrounding environmental states, thus achieving more efficient conflict prediction and resolution strategies. This module adopts a "local-global" double-loop reinforcement learning architecture, where local agents use the DQN algorithm to optimize individual actions, while the global coordinator evaluates the conflict risks of all local action proposals through a graph attention network and selects the best action combination to minimize the number of global conflicts. Such a design not only enhances the real-time response ability of the system but also ensures flight safety and efficiency throughout the airspace.

[0103] This application also provides an application scenario that applies the above-mentioned aircraft allocation method based on a dynamic airspace network and reinforcement learning. Specifically: The aircraft allocation method based on a dynamic airspace network and reinforcement learning provided in this embodiment can be applied to the airspace management scenario around an airport. The airspace management scenario around an airport includes a takeoff phase, a cruise phase, and a landing phase; flights enter the cruise phase from the takeoff phase, obtain the optimal flight path through the allocation of the dynamic airspace network, and enter the landing phase. The aircraft allocation method based on a dynamic airspace network and reinforcement learning provided in this embodiment belongs to the flight path optimization phase in the cruise phase. Specifically, this method conducts real-time monitoring and path planning for flights through dynamic sub-airspace network division and spatio-temporal buffer zone models to ensure flight safety and efficiency during the cruise phase of flights, while reducing flight delays and conflict risks and improving the automation and intelligence level of overall air traffic control.

[0104] In an exemplary embodiment, a computer device is provided. This computer device can be a server or a terminal, and its internal structure diagram can be as Figure 4As shown in the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store flight information processing data. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements an aircraft deployment method based on a dynamic airspace network and reinforcement learning.

[0105] Those skilled in the art can understand that Figure 4 the structure shown in the figure is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0106] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0107] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0108] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0109] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0110] The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logics, data processing logics based on quantum computing, etc., without limitation.

[0111] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0112] In this text, specific examples are used to elaborate on the principles and implementation manners of this application. The descriptions of the above embodiments are only used to help understand the method and its core idea of this application; at the same time, for those of ordinary skill in the art, according to the idea of this application, there will be changes in the specific implementation manners and application scopes. To sum up, the content of this specification should not be construed as a limitation to this application.

Claims

1. An aircraft deployment method based on a dynamic airspace network and reinforcement learning, characterized in that The aircraft deployment method based on the dynamic airspace network and reinforcement learning includes: Obtain the flight information and task priority weight information of each flight in the historical time period and the current time period; According to the flight information of each flight in the historical time period, divide the airspace units through the dynamic sub-airspace network to obtain the airspace division result; the airspace division result includes independent sub-airspaces and shared airspaces; Input the flight information and task priority weight information of each flight in the current time period into the spatio-temporal buffer model to obtain the current flight path set; the spatio-temporal buffer model is used to calculate the safety area area of each flight according to the flight information, and determine the starting point, passing points, ending point and flight time arrangement of each flight according to the safety area area and task priority weight information of each flight to obtain the current flight path set; According to the flight information, airspace division result and current flight path set of each flight in the current time period, obtain the current release action instruction set; the release action instruction set includes altitude adjustment instructions, heading correction instructions and speed change instructions; Deploy each flight in the current time period according to the current release action instruction set.

2. The aircraft deployment method based on a dynamic airspace network and reinforcement learning according to claim 1, wherein According to the flight information of each flight in the historical time period, divide the airspace units through the dynamic sub-airspace network to obtain the airspace division result, specifically including: Divide the airspace into multiple cubic grids; According to the flight information of each flight in the historical time period, determine the flight density between adjacent cubic grids, and calculate the airspace division modularity based on the flight density through the modularity function; Through the greedy algorithm, with the goal of maximizing the increase in the change of the airspace division modularity, perform a merging operation on adjacent cubic grids to obtain the initial airspace unit division result; Calculate the flight density between adjacent airspace units in the initial airspace unit division result, and return to the step of "through the greedy algorithm, with the goal of maximizing the increase in the change of the airspace division modularity, perform a merging operation on adjacent cubic grids to obtain the initial airspace unit division result", until the airspace is divided into a state that only contains two airspace units, to obtain the airspace division result; among them, a high-traffic airspace unit with a high flight density is used as an independent sub-airspace; the other low-traffic airspace unit with a low flight density is used as a shared airspace.

3. The aircraft deployment method based on a dynamic airspace network and reinforcement learning according to claim 2, wherein The modularity function is: ； Among them, represents the modularity of airspace division at the t-th moment; represents the flight density from the i-th cube grid to the j-th cube grid at the t-th moment; represents the total number of flights flowing in and out of the i-th cube grid at the t-th moment; represents the total number of flights flowing in and out of the j-th cube grid at the t-th moment; represents the total number of flights flowing in and out of the airspace at the t-th moment; represents the airspace unit to which the i-th cube grid belongs at the t-th moment; represents the airspace unit to which the j-th cube grid belongs at the t-th moment; represents the first indicator function.

4. The aircraft deployment method based on dynamic airspace network and reinforcement learning according to claim 1, characterized in that: Input the flight information and task priority weight information of each flight in the current time period into the spatio-temporal buffer model to obtain the current flight path set, specifically including: The spatio-temporal buffer model includes a safety area calculation module and a time window insertion module connected in sequence; The safety area calculation module uses the following formula to calculate the safety area area of each flight in the current time period: ； Among them, represents the safety area area of the th flight at the t-th moment; The task priority weight of the th flight; represents the minimum turning radius of the aircraft of the th flight; is the control response time; represents the maximum speed of the aircraft of the th flight; The time window insertion module is used for: Traverse the current flight path set. If the time interval between any two flights arriving at the current path intersection point is less than the preset time interval, then according to the task priority weight information, insert a preset time window to delay the flight with a low priority weight until all flights at the current path intersection point meet the preset stop condition, and then judge the next path intersection point.

5. The aircraft deployment method based on a dynamic airspace network and reinforcement learning according to claim 4, wherein Mission priority weight information specifically includes: commercial mission priority weight, cargo mission priority weight, and military mission priority weight; among which, cargo mission priority weight < commercial mission priority weight < military mission priority weight.

6. The aircraft deployment method based on a dynamic airspace network and reinforcement learning according to claim 4, wherein The calculation formula for the preset time interval is: ； Among them, represents the preset time interval between the arrival of any th flight and the th flight at the current path intersection point at the t-th moment; represents the safety area area of the th flight at the t-th moment; represents the safety area area of the th flight at the t-th moment; represents the speed of the th flight at the t-th moment; represents the speed of the th flight at the t-th moment; represents the preset minimum time interval.

7. The aircraft deployment method based on a dynamic airspace network and reinforcement learning according to claim 1, characterized in that According to the flight information, airspace division results and current flight path set of each flight in the current time period, the current release action instruction set is obtained, including: The flight information and airspace division results of each flight in the current time period are used as the input status information of each flight; Based on the input state information of each flight, the DQN algorithm is used to adjust the action of each flight to obtain the current local release action set; According to the current local release action set, the current release action instruction set is obtained through genetic algorithm.

8. The aircraft deployment method based on a dynamic airspace network and reinforcement learning according to claim 7, wherein Based on the input state information of each flight, the DQN algorithm is used to adjust the action of each flight to obtain the current local release action set, which includes: The input state information of each flight is fed into the trained deep Q network model, and the Q value of all actions is calculated using the following formula: ； in, Indicates the The Q value of the flight taking the ath action; Indicates the The mission priority weight of each flight; 、 and represent the first weight coefficient, the second weight coefficient and the third weight coefficient respectively; Indicates the preset minimum separation distance; Indicates the The change in energy consumption after a flight takes the ath action; By comparing the Q values of all actions in each flight, and selecting the action corresponding to the largest Q value as the optimal action for each flight, the current local release action set is obtained.

9. The aircraft allocation method based on a dynamic airspace network and reinforcement learning according to claim 7, characterized in that According to the current local release action set, the current release action instruction set is obtained through the genetic algorithm, which specifically includes: Based on the current local release action set, the first Flights to Flight impact weight: ； Among them, represents the influence weight of the th flight on the th flight; and respectively represent the eigenvectors corresponding to the optimal actions of the th flight and the th flight; means concatenating the eigenvectors corresponding to the optimal actions of the th flight and the th flight; represents the activation function, represents the normalization function; Based on the th flight's influence weight on the th flight, the action combination that minimizes the global conflict count is selected through the genetic algorithm to obtain the current relief action instruction set.

10. The aircraft deployment method based on a dynamic airspace network and reinforcement learning according to claim 9, wherein The calculation formula for the number of global conflicts is: ； ； Among them, represents the number of global conflicts; represents the second indication function, represents the th flight and the th flight, represents the preset safety distance.

Images