Aircraft collision avoidance method and device fusing static and dynamic cost tables

Abstract

The invention relates to an aircraft collision avoidance method and device fusing static and dynamic cost tables. The method comprises the following steps: firstly, performing state discretization on a conflict encountering scene of a controlled aircraft to obtain a finite state-action space, constructing a state transition probability and a comprehensive event cost function, defining a possible action suggestion set and determining a legal action transition matrix; then, generating a Markov decision process state transition table, performing value iteration based on the Markov decision process state transition table in combination with the current flight state of the invading aircraft, and determining an optimal action strategy and a static offline cost table; and finally, predicting the future trajectory change of the invading aircraft, determining the optimal action and value as an initial strategy, detecting the height and speed change of the invading aircraft in real time, and when the invading aircraft deviates from the static hypothesis, adjusting the initial strategy in real time, and determining the optimal collision avoidance path of the controlled aircraft. Collision risks and maneuvering cost are continuously balanced in a complex, multi-aircraft-type and uncertain airspace.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of air traffic control, in particular to an aircraft collision avoidance method and device fusing static and dynamic cost tables. BACKGROUND

[0002] In the field of air traffic control, flight conflict resolution is the core link to ensure the safety of air traffic operation. With the parallel operation of commercial flights, high-density unmanned aerial vehicles and electric vertical take-off and landing aircraft (eVTOL), the existing flight conflict resolution technology faces many challenges, as follows: The existing conflict resolution core equipment ACAS Xa generates a decision lookup table through offline partially observable Markov decision process (POMDP) reinforcement learning to replace the rule logic of TCAS II. Although this equipment can significantly reduce false alarms under experimental conditions, it has many defects in actual operation: first, the static lookup table not only occupies a lot of on-board resources, but also cannot be adaptively adjusted with environmental bias; second, the robustness to uncertainties such as height error, slant range error and pilot delay is insufficient, resulting in a higher risk of unresolved conflicts or induced near miss (NMAC) than TCAS II; third, in mixed operation and complex maneuver scenarios, it is easy to add about 10% of conflicts, and it is difficult for the crew to follow the resolution advisory (RA); fourth, the offline artificial intelligence (AI) optimization presents a black box feature, lacks real-time value visualization, and brings difficulties to airworthiness certification and crew trust.

[0003] The existing method based on dynamic programming (DP) can provide a theoretically optimal maneuvering scheme, but there are two typical problems: one type relies on one-time offline dynamic programming, which can generate a globally optimal cost table, but is affected by the state-action space explosion, making it difficult to meet the demand for on-board storage and offline computing power, and losing applicability when the environment deviates from the modeling assumption; the other type uses pure online global recalculation, which can follow real-time disturbances, but requires large-scale solutions at the millisecond level to obtain millisecond-level decisions, resulting in high computational delay, large value function jitter, and difficulty in airworthiness certification.

[0004] Therefore, in the related art, there is an urgent need for a way to balance the safety margin, real-time performance, computing power consumption and explainability of aircraft conflict resolution. SUMMARY

[0005] Therefore, in the related art, there is an urgent need for a way to balance the safety margin, real-time performance, computing power consumption and explainability of aircraft conflict resolution.

[0006] In a first aspect, the present application provides an aircraft collision avoidance method fusing static and dynamic cost tables. The method comprises: The conflict encounter scene of the controlled aircraft is discretized to obtain a finite state-action space, a state transition probability and a comprehensive event cost function are constructed, a possible action recommendation set is defined, and a legal action transition matrix is determined; A Markov decision process state transition table is generated based on the finite state-action space, the state transition probability, the comprehensive event cost function and the legal action transition matrix, combined with the current flight state of the invading aircraft, a value iteration is performed based on the Markov decision process state transition table to determine an optimal action strategy and a static offline cost table; Based on the optimal action strategy and the static offline cost table, the future trajectory change of the invading aircraft is predicted based on a dynamic local reconstruction method, the optimal action and value are determined as an initial strategy, the height and speed changes of the invading aircraft are detected in real time, and when the invading aircraft deviates from the static assumption, the initial strategy is adjusted in real time to determine the optimal collision avoidance path of the controlled aircraft.

[0007] Optionally, in an embodiment of the present application, the state discretization of the conflict encounter scene of the controlled aircraft to obtain a finite state-action space comprises: The relative height and vertical speed are spaced at a preset interval and range; The relative approach speed is calculated based on the horizontal speed of the controlled aircraft and the invading aircraft, and the closest point time is calculated based on the relative distance; The predicted speed of the controlled aircraft is calculated based on the current vertical speed of the controlled aircraft, the command acceleration and the discrete time interval, and the speed and height are updated based on the predicted speed of the controlled aircraft, the maximum vertical speed and the current vertical speed and height of the controlled aircraft.

[0008] Optionally, in an embodiment of the present application, the construction of the state transition probability and the comprehensive event cost function comprises: The state transition probability is determined by using a multi-linear interpolation method, and the formula is as follows:

[0009] wherein, is a weight function, =1, is the possible state of the aircraft at the next time, is an interpolation point obtained by multi-linear interpolation; The expression of the comprehensive event cost function is:

[0010] wherein, represents the cost of performing action , and is the state transition probability, indicating that the state is changed by action probability of reaching the next state is a discount factor is a state is an expected cost at the nth step.

[0011] Optionally, in an embodiment of the present application, the set of possible action suggestions includes maintaining the current state, climbing / descending, reversing, and enhancing.

[0012] Optionally, in an embodiment of the present application, the generating a Markov decision process state transition table based on the finite state-action space, state transition probability, comprehensive event cost function, and legal action transition matrix includes: in the finite state-action space, traversing all possible action pairs of the controlled aircraft and the intruder aircraft based on the legal action transition matrix; for each possible action pair, determining the next state after action execution, determining the transition probability and termination flag based on the state transition probability, and determining the state transition reward based on the comprehensive event cost function; sequentially entering the next state after action execution, state transition probability, state transition reward, and termination flag into the Markov decision process state transition table.

[0013] Optionally, in an embodiment of the present application, after the generating a Markov decision process state transition table based on the finite state-action space, state transition probability, comprehensive event cost function, and legal action transition matrix, the method further includes: pruning using a maneuver action cumulative number threshold strategy based on the set of possible action suggestions to remove unreasonable maneuver sequences.

[0014] Optionally, in an embodiment of the present application, the determining an optimal action strategy and static offline cost table based on the Markov decision process state transition table in combination with the current flight state of the intruder aircraft includes: determining the optimal action strategy based on a value tie-height difference priority selection rule.

[0015] Optionally, in an embodiment of the present application, the real-time detecting the height and speed changes of the intruder aircraft, real-time adjusting the initial strategy when the intruder aircraft deviates from the static assumption, and determining the optimal collision avoidance path of the controlled aircraft include: locating the disturbed nodes to form a local sub-tree; performing a top-down traversal based on the local sub-tree to update a local dynamic index; performing value iteration based on the updated local sub-tree to solve an optimal value function and determine a local optimal strategy. ​​

[0016] Optionally, in an embodiment of the present application, the method further comprises: drawing a heat map based on the optimal collision avoidance path.

[0017] In a second aspect, the present application also provides an aircraft collision avoidance device fusing static and dynamic cost tables. The device comprises: a data modeling module for discretizing a conflict encounter scenario of a controlled aircraft into states, obtaining a finite state-action space, constructing a state transition probability and a comprehensive event cost function, defining a possible action recommendation set, and determining a legal action transition matrix; a static planning module for generating a Markov decision process state transition table based on the finite state-action space, the state transition probability, the comprehensive event cost function, and the legal action transition matrix, combining a current flight state of an intruding aircraft, performing value iteration based on the Markov decision process state transition table, determining an optimal action policy and a static offline cost table; an online optimization and execution module for predicting future trajectory changes of the intruding aircraft based on the optimal action policy and the static offline cost table in combination with a dynamic local reconstruction method, determining an optimal action and value as an initial strategy, detecting height and speed changes of the intruding aircraft in real time, adjusting the initial strategy in real time when the intruding aircraft deviates from the static assumption, and determining an optimal collision avoidance path for the controlled aircraft.

[0018] The aircraft collision avoidance method and device fusing static and dynamic cost tables, first, discretizes a conflict encounter scene of a controlled aircraft, obtains a finite state-action space, constructs a state transition probability and a comprehensive event cost function, defines a possible action recommendation set, and determines a legal action transition matrix; then, generates a Markov decision process state transition table based on the finite state-action space, the state transition probability, the comprehensive event cost function, and the legal action transition matrix, combines a current flight state of an invading aircraft, performs value iteration based on the Markov decision process state transition table, determines an optimal action strategy and a static offline cost table; finally, predicts future trajectory changes of the invading aircraft based on the optimal action strategy and the static offline cost table in combination with a dynamic local reconstruction method, determines an optimal action and value as an initial strategy, detects height and speed changes of the invading aircraft in real time, adjusts the initial strategy in real time when the invading aircraft deviates from a static assumption, and determines an optimal collision avoidance path of the controlled aircraft. That is, in a single DP framework, the offline stage and the online stage are closely coupled: the offline end first completes global value iteration based on a countdown hierarchical state diagram and a sparse action set, and generates a static optimal cost table covering the complete scene; the online end takes the value function provided by the static table as the initial value, only performs in-place quick recovery, i.e., cost refreshing and local value iteration, on a local subtree composed of a disturbed node and its successors, and realizes quick strategy updating. In a complex, multi-type, uncertain airspace, collision risk and maneuver cost are continuously balanced, an efficient, interpretable, and easy-to-authenticate conflict resolution solution is provided for the next generation of air traffic control systems. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 A flowchart of an aircraft collision avoidance method fusing static and dynamic cost tables in one embodiment; Figure 2 A local discrete state space diagram in the resolution process in one embodiment; Figure 3 A legal action transition diagram in one embodiment; Figure 4 A value iteration solution process diagram in one embodiment; Figure 5 A static cost table and dynamic reconstruction diagram in one embodiment; Figure 6 A heat map diagram in one embodiment Figure 7 A structural block diagram of an aircraft collision avoidance device fusing static and dynamic cost tables in one embodiment. DETAILED DESCRIPTION

[0020] ​In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not used to limit the present application.

[0021] In one embodiment, as shown in Figure 1 , an aircraft collision avoidance method fusing static and dynamic cost table is provided, comprising the following steps: S101: discretizing the state of the controlled aircraft conflict encounter scene to obtain a finite state-action space, constructing a state transition probability and a comprehensive event cost function, defining a possible action recommendation set, and determining a legal action transition matrix.

[0022] In the embodiments of the present application, first, the controlled aircraft conflict encounter scene meeting the requirements of DO-385A is discretized with five state variables of relative height, vertical speed, acceleration, maneuvering action and time to closest point to form a finite state set and a vertical maneuvering action set . As shown in Figure 2 , for each state of the aircraft, different actions can be selected to reach different states. After discretization, the position reached by the aircraft is different grid, and when reaching the closest point, due to flight uncertainty, the intruder aircraft can appear in different grids in the state space, and the specific position of the intruder aircraft is obtained by state transition probability.

[0023] When the aircraft takes an action from the current state, the next state is not necessarily located on the discrete state point, therefore, a multi-linear interpolation is used to determine the state transition probability, and the state after transition is fitted to ensure that the state at each time is on the discrete state point, and a comprehensive event cost function of collision, warning and maneuvering switching is established. Define a possible action recommendation set, determine a legal action transition matrix, and according to the matrix, the range of actions that can be selected in the current state can be determined. As shown in Figure 3 , it is a legal action state transition diagram, each node represents a legal action, and different branches are legal actions that the current action can be transferred to.

[0024] Specifically, in one embodiment of the present application, the discretization of the state of the controlled aircraft conflict encounter scene to obtain a finite state-action space comprises: S201: interval the relative height and vertical speed with a preset interval and range.

[0025] S203: calculate the relative approach speed based on the horizontal speed of the controlled aircraft and the intruder aircraft, and calculate the time to closest point in combination with the relative distance.

[0026] S205: Calculate the predicted speed of the controlled aircraft based on the current vertical speed of the controlled aircraft, the commanded acceleration, and the discrete time interval, and update the speed and altitude based on the predicted speed of the controlled aircraft, the maximum vertical speed, and the current vertical speed and current altitude of the controlled aircraft.

[0027] In one embodiment of the present application, the aircraft state is discretized, the relative altitude is discretized in intervals ranging from to ; the vertical speed is discretized in intervals ranging from to ; and the simulation step is seconds.

[0028] Aircraft flight simulation modeling: Let the vertical acceleration of the aircraft be , and the maximum vertical speed be . Calculate the time to closest point for the two aircraft, and let the horizontal speeds of the two aircraft be , and the relative approach speed be :

[0029] Let the distance between the two aircraft be , and the relative approach speed be . Let represent the time required for the two aircraft to reach the closest point from the current state, then the time approach parameter is:

[0030] The following are the altitude-velocity simulation calculation formulas for various flight conditions of the aircraft, including level flight, climb / descent, reversal, and augmentation: The predicted speed without speed limit is:

[0031] where is the current vertical speed of the aircraft, is the commanded acceleration, and different actions have different accelerations, is the discrete time interval.

[0032] The speed update calculation formula is as follows:

[0033] where is the maximum speed of the aircraft, is the time to reach the maximum speed. ​​

[0034] The formula for high-fidelity simulation calculation is as follows:

[0035] in, The aircraft's current altitude. The updated altitude of the aircraft.

[0036] Specifically, in one embodiment of this application, the construction of the state transition probability and the comprehensive event cost function includes: The state transition probability is determined using a multilinear interpolation method, as shown in the following formula:

[0037] in, It is a weighting function. =1, This is the possible state of the aircraft at the next moment. These are the interpolation points obtained through multilinear interpolation; The expression for the comprehensive event cost function is:

[0038] in, Indicates the execution of an action The cost, Let be the state transition probability, indicating the state. Through action Reaching the next state The probability, As a discount factor, For state In the The expected cost of each step.

[0039] In one embodiment of this application, a multilinear interpolation method is used to determine the state transition probability, as shown in the following formula:

[0040] in, It is a weighting function. =1, This is the possible state of the aircraft at the next moment. These are the interpolation points obtained through multilinear interpolation.

[0041] In practical applications, conflict resolution is performed in two dimensions, with the horizontal axis... The vertical coordinate represents the aircraft's horizontal coordinates. This represents the aircraft's coordinates in the vertical direction. If the predicted state point of the aircraft at the next moment is not at a discrete state point, the four vertices of the discrete grid containing the predicted state point at the next moment are selected. To interpolate, the probability value is assigned according to the distance between the grid vertex and the discrete state point, and the probability value is calculated as follows:

[0042]

[0043]

[0044]

[0045] Finally, the corresponding discrete point is selected as the state point of the next time of the aircraft according to the probability.

[0046] The near miss collision (NMAC) threshold is a grid height , and the cost is large; the impact of the reverse suggestion on the original track is greater than that of the enhancement, so the cost of the reverse is greater than that of the enhancement; the cost of COC (no suggestion) is negative, indicating the reward for not issuing an alarm or resolving the conflict. The event cost is defined as follows: Near miss collision: the cost is ; near miss collision, seconds, and the relative height is less than , which occurs when the relative height is . Alarm: the cost is ; conflict warning, active intervention measures, high cost. Reverse: the cost is ; reverse flight trajectory, active collision avoidance. Enhancement: the cost is ; enhance conflict resolution suggestions, divided into enhanced climb and enhanced descent, operated at an acceleration of . No suggestion: the cost is ; indicating no suggestion or no threat at present.

[0047] The stage cost function is defined as , which represents the minimum cumulative cost of the state in the th iteration, initialized as , and the stage cost function is defined as follows:

[0048] wherein represents the cost of executing the action ; is the state transition probability, indicating the probability of state reaching the next state by action ; is the discount factor, usually The value is used to balance current and future costs, and the process continues iterating until a set threshold is reached. For state In the The expected cost of each step.

[0049] In this embodiment, by discretizing the five-dimensional state variables such as relative height, vertical velocity, and time to the closest point, and constructing a fine state transition probability using multilinear interpolation, the conflict evolution can be accurately characterized within the dynamic programming framework.

[0050] In one embodiment of this application, the set of possible action suggestions includes maintaining the current state, climbing / descending, reversing, and enhancing.

[0051] In one embodiment of this application, the altitude of the intruding aircraft relative to the controlled aircraft is... Vertical speed of the controlled aircraft Vertical speed of intruding aircraft Remaining time to reach the closest point and current status recommendations There are five discrete variables representing the state. Four main types of actions are defined: maintaining the current state (COC), climbing the CL / descending the DES, reversing, and enhancing. For COC, it means that no suggestion has been issued at present, the current state is maintained, and the action instruction can be executed if there are all action states. For ascent (CL) / descent (DES) suggestions, the action suggestions include CL1000 / DES1000: where 1000 is the vertical velocity of the calibrated target, and there is a delay after receiving such suggestions. Execute the operation in seconds. Vertical upward / downward acceleration allows the aircraft to achieve vertical upward / downward direction. The speed at which the action instruction can be executed can only be suggested by COC action; For reversal suggestions, the action suggestions included when combined with climb / descent are SCL1000 / SDES1000: a delay occurs upon receiving such suggestions. Execute the operation in seconds. The acceleration maneuver allows the aircraft to reach the opposite direction of the original suggested maneuver. Vertical speed; SCL1000 can be executed by action suggestions from DES1000, SDES1000, SDES2000, and SCL2000; SDES1000 can be executed by action suggestions from CL1000, SCL1000, SCL2000, and SDES2000. For the enhancement suggestion, the action suggestion based on the climbing / descending contains SCL2000 / SDES2000: wherein 2000 is the target vertical speed, after receiving the suggestion, the operation is executed with a delay of seconds in the original suggested direction to reach the vertical speed of ; SCL2000 can execute the action instruction by CL1000, SCL1000 action suggestion; SDES2000 can execute the action instruction by DES1000, SDES1000 action state.

[0052] It should be noted that the above speed and acceleration parameters need to meet:

[0053] The (action suggestion set) is composed of 23 delay execution sub-states, in addition to COC, the actions of the three suggestions of climbing / descending, reversing, and enhancing are all represented by , representing the currently selected action suggestion name, representing how many seconds are left before the current suggestion starts to be executed, such as CL1000-1 representing 1 second left before starting to climb.

[0054] S103: Based on the finite state-action space, state transition probability, comprehensive event cost function, and legal action transition matrix, a Markov decision process state transition table is generated, combined with the current flight state of the intruding aircraft, based on the Markov decision process state transition table, value iteration is performed to determine the optimal action strategy and static offline cost table.

[0055] In the embodiments of the application, the breadth-first search is used to traverse the finite state-action space, in each step of the transition process, the next state is obtained according to the aircraft dynamics formula and the speed height restriction condition, and the value of the cost is calculated. Combined with the current flight state of the intruding aircraft, the global optimal value function is calculated using value iteration and the optimal strategy is extracted . As shown in Figure 4 , value iteration is a process of continuously optimizing the value function . For the current state, the expected value of selecting each action is calculated, and the maximum value (i.e. the action with the maximum expected value) is used to optimize the value function . This process is repeated until the change of is less than a predetermined threshold, and the static optimal strategy is obtained, i.e. the optimal action for each state can be obtained according to .

[0056] The process is only based on the static enemy machine assumption for strategy generation, in order to improve the efficiency of online state aircraft decision, a global solution is first carried out on the static conflict scene, and a state-action-value table is generated, which contains the optimal action and optimal cost of each state, so as to facilitate fast query online. In the problem of collision avoidance, the dynamic programming method of reverse iteration is used to construct the state-action decision table; the core steps are: dividing the continuous state space into grid according to the relative height, horizontal distance and other dimensions; calculating the optimal cost function from the terminal state to the initial state in reverse; through action priority sorting, the collision avoidance operation with the smallest disturbance to the flight trajectory is screened.

[0057] The first step state action cost is calculated as follows:

[0058] Among them, represents the cost of executing action ; is the state transition probability, which indicates the probability that state reaches the next state through action ; is the discount factor, usually taking the value of , which is used to balance the current and future cost, and the iteration is continued until the set threshold is reached, is the expected cost of state in the first step.

[0059] The formula for obtaining the optimal strategy in the first step is as follows:

[0060] The optimal strategy in the first step is the action corresponding to the minimum value of .

[0061] The value iteration algorithm of dynamic programming is used to solve the optimal MDP, and the core idea of value iteration is to use the Bellman optimal equation to iterate repeatedly, and the specific steps are as follows: Initialization: for all non-terminal states , an initial value is assigned, where is the set of all states.

[0062] Iterative update, repeat the following process until the maximum change , is a given threshold: set to 0, and for each state , record the old value Then the new value is calculated:

[0063]

[0064] where, is the current state The expected return of the action is selected; is the next state of the state The iteration is completed The value function of the round; represents, the current state Performing the action , moving to the state And get the immediate reward ; represents the state Iteration The value function of the round; write back the value function after taking the maximum value of all The expected return of the action , so that all states The optimal value of the round obtained.

[0065] Update the current maximum change:

[0066] After extracting the optimal strategy and iterating to convergence, the optimal strategy is extracted according to the following formula:

[0067] In combination with the current flight state of the invading aircraft, the generated MDP state transition table is used to solve the optimal strategy, i.e. the optimal action of each state, using the value iteration algorithm.

[0068] Specifically, in an embodiment of the present application, the generating a Markov decision process state transition table based on the finite state-action space, state transition probability, comprehensive event cost function and legal action transition matrix comprises: S301: In the finite state-action space, all possible action pairs of the controlled aircraft and the invading aircraft are traversed based on the legal action transition matrix.

[0069] S303: For each possible action pair, determine the next state after action execution, determine the transition probability and termination flag based on the state transition probability, and determine the state transition reward based on the comprehensive event cost function.

[0070] S305: Enter the next state after the action is executed, the state transition probability, the state transition reward, and the termination flag into the Markov decision process state transition table in sequence.

[0071] In one embodiment of this application, using As a loop counter, a breadth-first traversal of the queue is used, iterating through all possible action pairs between the controlled and intruding aircraft every second. For each state group, the next state after the action is calculated based on the aircraft's vertical kinematics formula and maximum vertical speed limit. ,in, The altitude of the intruding aircraft relative to the controlled aircraft. The vertical velocity of the controlled aircraft. The vertical velocity of the intruding aircraft. The time it takes for the two aircraft to reach their closest point in the current state. The action chosen to transition from the previous state to the next state, i.e., the conflict resolution suggestion. The action chosen based on the current state is determined using a probabilistic model. Execute action Transferred to probability And whether to terminate the flag. Calculate the maneuver cost, collision cost, and alarm cost for each transfer based on the cost of each action. , The reward for performing this step, Record each state transition. Enter the MDP state transition table sequentially. By access ,available until Decrease to 0 to complete the traversal and recording of all states.

[0072] In one embodiment of this application, after generating the Markov decision process state transition table based on the finite state-action space, state transition probabilities, comprehensive event cost function, and legal action transition matrix, the method further includes: Based on the set of possible action suggestions, a pruning strategy using a threshold for the cumulative number of maneuvers is used to remove unreasonable maneuver sequences.

[0073] In one embodiment of this application, the cumulative number of ascent and descent commands (including reversal and enhancement) executed by the controlled aircraft is counted in real time. When the cumulative number exceeds a preset threshold... At that time, large maneuvers are automatically removed from the candidate maneuver set, retaining only track-keeping or small maneuvers; the formula for calculating the number of maneuvers for each path is as follows:

[0074] wherein, is the cumulative number of climb / descent, is the time to closest point of approach of the two aircrafts, is the action selected at time is an indicator function, which is 1 if the condition in the bracket is satisfied, and 0 otherwise.

[0075] For each path, if the current planned path has already satisfied , the path is deleted, and the pruning is completed; the threshold constraint reduces the sharp maneuver, guarantees the pilot recovery time, and improves the human-machine acceptability of the strategy; the threshold constraint reduces the size of the MDP state transition table while pruning the action, thereby reducing the computational complexity of the subsequent dynamic programming solution.

[0076] In an embodiment of the present application, the value iteration based on the Markov decision process state transition table, the current flight state of the intruder aircraft, the determination of the optimal action strategy and the static offline cost table include: determining the optimal action strategy based on the value tie-height difference priority selection rule.

[0077] In an embodiment of the present application, the value tie-height difference priority selection rule is introduced in the process of solving the optimal strategy, that is, when the expected values of multiple candidate actions are the same or fall within a set threshold range, the action that can make the absolute value of the vertical separation between the controlled aircraft and the intruder aircraft increase the most is preferentially selected; if there are still ties, the final action is determined by progressively comparing the action disturbance amplitude, action cost, and other preset orders.

[0078] In this embodiment, the collision, alarm, maneuver disturbance, and other event costs are integrated into the same value function, and the value tie-height difference priority selection rule is used, so that the strategy can reduce unnecessary maneuvers and alarms under the premise of ensuring sufficient safety margin, which is superior to the traditional scheme that relies on fixed cost tables or single risk indicators.

[0079] S105: Based on the optimal action strategy and the static offline cost table, the future trajectory change of the intruder aircraft is predicted by combining the dynamic local reconstruction method, the optimal action and the value are determined as the initial strategy, the height and speed changes of the intruder aircraft are detected in real time, when the intruder aircraft deviates from the static assumption, the initial strategy is adjusted in real time, and the optimal collision avoidance path of the controlled aircraft is determined.

[0080] In the embodiments of the present application, the future trajectory change of the invading aircraft is dynamically predicted, and the calculation amount is reduced according to the static offline cost table combined with the dynamic local reconstruction method; the offline cost table is read, and the optimal action and value under the default condition are quickly obtained based on the current state as the initial strategy. The entire reachable state space is organized into a state tree, each node corresponds to a discrete state, and each layer of the state tree is all the states under the current state, which is diffused from the root downward until or collision; the time at which the two aircrafts reach the closest point, is the maximum value of ; the subtree is the entire successor state set developed downward from a certain node in the state tree as the root.

[0081] The height and speed change of the invading aircraft are detected in real time, when the invading aircraft deviates from the static assumption, the dynamic prediction mechanism is triggered, the future trajectory estimation of the enemy aircraft is adjusted in real time, and the affected area is located in the state tree; for the subtree corresponding to the area, the cost is locally updated, and the value iteration calculation is re-performed for the states in the subtree range in the original decision structure, and a new optimal action sequence is quickly generated, thereby significantly reducing the online calculation burden.

[0082] As shown in Figure 5 , the aircraft action decision is divided into offline and online two stages, the offline cost table and the dynamic subtree local replanning are combined, the global optimal cost and the optimal strategy are obtained based on the current state of the aircraft using the value iteration method, then the state of the invading aircraft is monitored, if there is no change, the optimal action sequence obtained before is executed, if there is a change, the affected subtree is located, and local replanning is performed to obtain a new optimal strategy, which is grafted back into the global tree.

[0083] Specifically, the online real-time strategy optimization link is the bridge connecting the offline and online integrated decision, and the controlled aircraft polls the latest vertical speed and height data of the invading aircraft with a decision cycle of every second, and compares the real-time trajectory of the invading aircraft with the static predicted trajectory according to the following formula:

[0084]

[0085] wherein, represents the current state of the invading aircraft, including its height and vertical speed , is the time at which the two aircrafts reach the closest point, ​Let the state vector of the intruding aircraft be the state vector of the previous state. The set disturbance threshold.

[0086] If the state changes, i.e., the altitude or speed of the intruding aircraft changes, and the remaining decision time... When the value is still greater than zero, a dynamic reconstruction process of the local subtree (the currently affected state and all its successors) is triggered. The key to local subtree reconstruction lies in the alignment of spatial constraints and time: the algorithm no longer recalculates the entire global state graph, but instead locks the disturbed node and its successor set, constructing a "local state transition subgraph" much smaller than the global one. Subsequently, collision, alarm, and maneuver costs are reassessed only within this constrained subgraph, and the local optimal value function and corresponding strategy are quickly solved using value iteration in dynamic programming, reducing time complexity. The local value iteration formula is as follows:

[0087] in, For state A collection of legal action recommendations It is a set of local states.

[0088] The updated local policy immediately replaces the corresponding branch of the original decision tree and obtains the next maneuver instruction; the controlled aircraft executes the action accordingly, simultaneously refreshing its own status and the global timeline, and enters the next decision cycle; this cycle of "monitoring-judgment-reconstruction-solving-execution" continues until the remaining decision time is exhausted or the system confirms that the conflict risk has been completely eliminated.

[0089] Specifically, in one embodiment of this application, the real-time detection of changes in the altitude and speed of the intruding aircraft, and the real-time adjustment of the initial strategy to determine the optimal collision avoidance path for the controlled aircraft when the intruding aircraft deviates from the static assumption, includes: S401: Locate the disturbed node and form a local subtree.

[0090] S403: Perform a top-down traversal based on the local subtree and update the local dynamic indicators.

[0091] S405: Based on the updated local subtree, perform value iteration, solve for the optimal value function, and determine the local optimal strategy.

[0092] In one embodiment of this application, in each decision cycle, the real-time altitude and vertical velocity of the intruding aircraft are first compared with the existing predicted trajectory. When a significant deviation occurs in the relative altitude or velocity, the algorithm locates the corresponding disturbed node in the global decision tree and marks it as a new temporary root node. Subsequently, only this node and all its successors are extracted to form a local subtree of limited size. This process can be expressed as the following formula:

[0093] wherein, is the time when the two aircraft reach the closest point, is the disturbed node at time is the successor state set rooted at The entire extraction process is completed based on pointer operations without copying the entire tree structure, and the time complexity is .

[0094] Then, on the extracted subtree, a top-down traversal is performed once to immediately update dynamic indicators such as collision cost, alert cost, and action recommendation cost. The specific formula is as follows:

[0095] wherein, is the state new cost, is the new collision cost, is the new alert cost, is the maneuver cost; the node internal field is overwritten in place, and a new mapping table is not generated, so that the refresh cost is linearly related to the number of subtree nodes.

[0096] Then, the updated subtree is directly regarded as a local state-action transition graph; the nodes and edges thereof only cover the disturbed range, so that the state dimension is greatly reduced compared with the global MDP; within the local graph, value iteration is performed to quickly converge to obtain a new optimal value function and obtain a local optimal strategy; since the calculation domain is accurately cropped, the number of iteration steps and the amount of computation are significantly reduced; the local value iteration stop condition is as follows:

[0097] The obtained local optimal strategy replaces the corresponding branch of the original decision tree in a pointer backwriting manner, without the need to reconstruct other undisturbed parts; in this way, the strategy can be updated in real time while maintaining the continuity of the global decision tree. The specific implementation is as follows:

[0098] wherein, is the optimal strategy, is the optimal strategy obtained through local strategy optimization, is the best known strategy of the current undisturbed region.

[0099] In an embodiment of the present application, the method further comprises: drawing a heat map based on the optimal collision avoidance path.

[0100] In one embodiment of this application, a simulation dataset is constructed that includes multiple types of conflict encounters, such as chasing in the same direction, merging in opposite directions, and reversing suggestions, to cover typical scenarios. Then, collision avoidance planning is performed on each encounter using the method of this application, and the process is iteratively run until the remaining reaction time. Reset to zero, output the optimal path, and obtain the optimal action to be selected at each decision time; to present the conflict situation in real time, generate the following in real time during the test: Figure 6 shown Heatmap: The time it takes for two aircraft to reach their closest point. The horizontal axis represents the altitude difference between the controlled aircraft and the intruding aircraft. A regular grid with the vertical axis represents the optimal value. The risk is mapped to a color gradient, with different colors representing the corresponding levels of risk. During the simulation, the real-time coordinates of both sides are refreshed every second, and the optimal action label is superimposed on the corresponding grid to achieve synchronous visualization of risk distribution and maneuver suggestions, which is used to intuitively verify the rationality of the algorithm's decision.

[0101] In the aforementioned aircraft collision avoidance method integrating static and dynamic cost tables, firstly, the conflict encounter scenario of the controlled aircraft is discretized to obtain a finite state-action space. State transition probabilities and a comprehensive event cost function are constructed, a set of possible action suggestions is defined, and a legal action transition matrix is ​​determined. Then, a Markov decision process state transition table is generated based on the finite state-action space, state transition probabilities, comprehensive event cost function, and legal action transition matrix. Combining the current flight state of the intruding aircraft, value iteration is performed based on the Markov decision process state transition table to determine the optimal action strategy and a static offline cost table. Finally, based on the optimal action strategy and the static offline cost table, a dynamic local reconstruction method is used to predict the future trajectory changes of the intruding aircraft, determining the optimal action and value as the initial strategy. The altitude and speed changes of the intruding aircraft are detected in real time. When the intruding aircraft deviates from the static assumptions, the initial strategy is adjusted in real time to determine the optimal collision avoidance path for the controlled aircraft. In other words, within a single DP framework, the offline and online phases are tightly coupled: the offline end first completes global value iteration based on the countdown hierarchical state diagram and sparse action set, generating a static optimal cost table covering the entire scenario; the online end, when monitoring data updates, uses the value function provided by the static table as the initial value, and only performs in-situ rapid recovery, instant cost refresh, and local value iteration on the local subtrees formed by the disturbed nodes and their successors, achieving rapid policy updates. This continuously balances collision risk and maneuver costs in complex, multi-aircraft, and uncertain airspace, providing a highly efficient, interpretable, and easily verifiable conflict resolution solution for next-generation air traffic control systems.

[0102] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0103] Based on the same inventive concept, this application also provides an aircraft collision avoidance device that integrates static and dynamic cost tables for implementing the aircraft collision avoidance method that integrates static and dynamic cost tables as described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of an aircraft collision avoidance device integrating static and dynamic cost tables provided below can be found in the limitations of the aircraft collision avoidance method integrating static and dynamic cost tables described above, and will not be repeated here.

[0104] In one embodiment, such as Figure 7 As shown, an aircraft collision avoidance device 700 integrating static and dynamic cost tables is provided, including: a data modeling module 701, a static planning module 703, and an online optimization and execution module 705, wherein: The data modeling module 701 is used to discretize the state of the controlled aircraft conflict encounter scenario, obtain a finite state-action space, construct the state transition probability and the comprehensive event cost function, define the set of possible action suggestions, and determine the legal action transition matrix.

[0105] The static planning module 703 is used to generate a Markov decision process state transition table based on the finite state-action space, state transition probability, comprehensive event cost function and legal action transition matrix. Combined with the current flight state of the intruding aircraft, it performs value iteration based on the Markov decision process state transition table to determine the optimal action strategy and static offline cost table.

[0106] The online optimization and execution module 705 is used to predict the future trajectory changes of the intruding aircraft based on the optimal action strategy and the static offline cost table combined with the dynamic local reconstruction method, determine the optimal action and value as the initial strategy, detect the altitude and speed changes of the intruding aircraft in real time, adjust the initial strategy in real time when the intruding aircraft deviates from the static assumption, and determine the optimal collision avoidance path for the controlled aircraft.

[0107] In an embodiment of the present application, the data modeling module is further configured to: interval the relative height, the vertical speed in preset intervals and ranges; calculate the relative approaching speed based on the horizontal speed of the controlled aircraft and the intruder aircraft, and calculate the closest point time in combination with the relative distance; calculate the predicted speed of the controlled aircraft based on the current vertical speed of the controlled aircraft, the instructed acceleration and the discrete time interval, and update the speed and the height based on the predicted speed of the controlled aircraft, the maximum vertical speed and the current vertical speed and the current height of the controlled aircraft.

[0108] In an embodiment of the present application, the data modeling module is further configured to: determine the state transition probability by using a multi-linear interpolation method, and the formula is as follows:

[0109] wherein, is a weight function, = 1, is the possible state of the aircraft at the next time, is an interpolation point obtained by multi-linear interpolation; the expression of the comprehensive event cost function is:

[0110] wherein, represents the cost of performing an action , is a state transition probability, indicating the probability that the state reaches the next state through the action , is a discount factor, is the expected cost of the state at the first step. In an embodiment of the present application, the possible action suggestion set includes maintaining the current state, climbing / descending, reversing, and enhancing. In an embodiment of the present application, the static planning module is further configured to:

[0111] in a limited state-action space, based on the legal action transition matrix, traverse all possible action pairs of the controlled aircraft and the intruder aircraft; for each possible action pair, determine the next state after the action is performed, determine the transition probability and the termination flag based on the state transition probability, and determine the state transition reward based on the comprehensive event cost function;

[0112] In an embodiment of the present application, the static planning module is further configured to: in a limited state-action space, based on the legal action transition matrix, traverse all possible action pairs of the controlled aircraft and the intruder aircraft; ​The next state after the action is performed, the state transition probability, the state transition reward and the termination flag are sequentially entered into a Markov decision process state transition table.

[0113] In an embodiment of the present application, the static planning module is further configured to: Pruning is performed based on the possible action suggestion set using a maneuver action cumulative number threshold strategy to remove unreasonable maneuver sequences.

[0114] In an embodiment of the present application, the static planning module is further configured to: An optimal action strategy is determined based on a value tie-height difference priority selection rule.

[0115] In an embodiment of the present application, the online optimization and execution module is further configured to: A disturbed node is located, and a local sub-tree is formed; A top-down traversal is performed based on the local sub-tree, and a local dynamic index is updated; A value iteration is performed based on the updated local sub-tree, an optimal value function is solved, and a local optimal strategy is determined.

[0116] In an embodiment of the present application, the online optimization and execution module is further configured to: A hot map is drawn based on a collision avoidance optimal path.

[0117] The above-described various modules in the aircraft collision avoidance device fusing static and dynamic cost tables can be realized wholly or partially by software, hardware and combinations thereof. The above-described various modules can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in a computer device in software form, so as to be called and executed by a processor to perform operations corresponding to the above-described various modules.

[0118] In an embodiment, a computer device is provided, including a memory and a processor, the memory stores a computer program, and the processor implements the steps in the above-described various method embodiments when executing the computer program.

[0119] In an embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program implements the steps in the above-described various method embodiments when executed by a processor.

[0120] In an embodiment, a computer program product is provided, which includes a computer program, and the computer program implements the steps in the above-described various method embodiments when executed by a processor.

[0121] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties.

[0122] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiments. Any reference to memory, database or other medium used in the embodiments provided by the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided by the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided by the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0123] The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, but as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present application.

[0124] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. An aircraft collision avoidance method integrating static and dynamic cost tables, characterized in that, The method includes: The controlled aircraft conflict encounter scenario is discretized to obtain a finite state-action space. The state transition probability and comprehensive event cost function are constructed, the possible action suggestion set is defined, and the legal action transition matrix is ​​determined. Based on the finite state-action space, state transition probability, comprehensive event cost function and legal action transition matrix, a Markov decision process state transition table is generated. Combined with the current flight state of the intruding aircraft, value iteration is performed based on the Markov decision process state transition table to determine the optimal action strategy and static offline cost table. Based on the optimal action strategy and the static offline cost table combined with the dynamic local reconstruction method, the future trajectory changes of the intruding aircraft are predicted, the optimal action and value are determined as the initial strategy, the altitude and speed changes of the intruding aircraft are detected in real time, and the initial strategy is adjusted in real time when the intruding aircraft deviates from the static assumption to determine the optimal collision avoidance path for the controlled aircraft.

2. The aircraft collision avoidance method integrating static and dynamic cost tables according to claim 1, characterized in that, The state discretization of the controlled aircraft conflict encounter scenario yields a finite state-action space including: The relative height and vertical speed are spaced out in preset intervals and ranges; The relative approach speed is calculated based on the horizontal speeds of the controlled aircraft and the intruding aircraft, and the time to the closest point is calculated by combining the relative distance. The predicted speed of the controlled aircraft is calculated based on the current vertical speed, commanded acceleration, and discrete time interval. The speed and altitude are updated based on the predicted speed, maximum vertical speed, current vertical speed, and current altitude of the controlled aircraft.

3. The aircraft collision avoidance method integrating static and dynamic cost tables according to claim 1, characterized in that, The constructed state transition probability and comprehensive event cost function include: The state transition probability is determined using a multilinear interpolation method, as shown in the following formula: in, It is a weighting function. =1, This is the possible state of the aircraft at the next moment. These are the interpolation points obtained through multilinear interpolation; The expression for the comprehensive event cost function is: in, Indicates the execution of an action The cost, Let be the state transition probability, indicating the state. Through action Reaching the next state The probability, As a discount factor, For state In the The expected cost of each step.

4. The aircraft collision avoidance method integrating static and dynamic cost tables according to claim 1, characterized in that, The set of possible action suggestions includes maintaining the current state, climbing / descending, reversing, and enhancing.

5. The aircraft collision avoidance method integrating static and dynamic cost tables according to claim 1, characterized in that, The process of generating a Markov decision process state transition table based on the finite state-action space, state transition probabilities, comprehensive event cost function, and legal action transition matrix includes: In a finite state-action space, all possible action pairs between the controlled aircraft and the intruding aircraft are traversed based on the legal action transition matrix. For each possible action pair, determine the next state after the action is executed, determine the transition probability and termination flag based on the state transition probability, and determine the state transition reward based on the comprehensive event cost function. The next state after the action is executed, the state transition probability, the state transition reward, and the termination flag are sequentially entered into the Markov decision process state transition table.

6. The aircraft collision avoidance method integrating static and dynamic cost tables according to claim 1, characterized in that, After generating the Markov decision process state transition table based on the finite state-action space, state transition probability, comprehensive event cost function, and legal action transition matrix, the process further includes: Based on the set of possible action suggestions, a pruning strategy using a threshold for the cumulative number of maneuvers is used to remove unreasonable maneuver sequences.

7. The aircraft collision avoidance method integrating static and dynamic cost tables according to claim 1, characterized in that, The process of combining the current flight state of the intruding aircraft with the value iteration based on the Markov decision process state transition table to determine the optimal action strategy and static offline cost table includes: The optimal action strategy is determined based on the value juxtaposition-height difference priority selection rule.

8. The aircraft collision avoidance method integrating static and dynamic cost tables according to claim 1, characterized in that, The real-time detection of changes in the altitude and speed of the intruding aircraft, and the real-time adjustment of the initial strategy to determine the optimal collision avoidance path for the controlled aircraft when the intruding aircraft deviates from the static assumption, include: Locate the disturbed node and form a local subtree; Based on the local subtree, a top-down traversal is performed to update the local dynamic indicators; Based on the updated local subtree, value iteration is performed to solve for the optimal value function and determine the local optimal strategy.

9. The aircraft collision avoidance method integrating static and dynamic cost tables according to claim 1, characterized in that, The method further includes: Heatmaps are generated based on the optimal collision avoidance path.

10. An aircraft collision avoidance device integrating static and dynamic cost tables, characterized in that, The device includes: The data modeling module is used to discretize the state of the controlled aircraft conflict encounter scenario, obtain a finite state-action space, construct the state transition probability and comprehensive event cost function, define the set of possible action suggestions, and determine the legal action transition matrix. The static planning module is used to generate a Markov decision process state transition table based on the finite state-action space, state transition probability, comprehensive event cost function and legal action transition matrix. Combined with the current flight state of the intruding aircraft, the module performs value iteration based on the Markov decision process state transition table to determine the optimal action strategy and static offline cost table. The online optimization and execution module is used to predict the future trajectory changes of the intruding aircraft based on the optimal action strategy and the static offline cost table combined with the dynamic local reconstruction method, determine the optimal action and value as the initial strategy, detect the altitude and speed changes of the intruding aircraft in real time, and adjust the initial strategy in real time when the intruding aircraft deviates from the static assumption to determine the optimal collision avoidance path for the controlled aircraft.

Images