A traffic decision-making collaboration method based on Markov decision

Abstract

The present invention discloses a traffic decision-making coordination method based on Markov decision, belonging to the field of airborne collision avoidance systems, and comprising the steps of: S1, generating by using a decision-making consultation coordination strategy based on Markov: taking the individual and global consultations generated by a decision-making consultation component as inputs, referring to the state factors of the own aircraft and the intruding aircraft, and constructing a coordination information operation process; calculating the optimal coordination information according to the vertical speed, consultation status and intruding aircraft information, selecting the optimal coordination strategy, and performing point-to-point output in the transponder of the own aircraft according to the optimal coordination strategy; the decision-making consultation component includes an airborne collision avoidance decision-making consultation component; S2, processing a response strategy based on active coordination; S3, generating a decision-making consultation based on the decision-making consultation coordination information. The present invention makes the coordination information more precise and secure, and at the same time is compatible with the existing TCAS II airborne traffic alert and collision avoidance system, greatly reducing the false alarm rate and saving the maintenance cost of the airborne collision avoidance algorithm.

Description

Technical Field

[0001] The present invention relates to the field of airborne collision avoidance systems, and more specifically, to a traffic decision-making coordination method based on Markov decision-making. Background Art

[0002] As a product of the deep integration of safety control and aerospace, the airborne collision avoidance system uses secondary radar and target tracking technologies to plan collision avoidance and provide real-time maneuver suggestions for air threats, meeting the safety requirements of civil aviation for airspace control. As one of the core technologies of the system, the real-time performance and safety of the decision-making consultation and coordination processing technology directly determine the quality of the system.

[0003] The real-time performance and safety of the decision-making consultation and coordination processing strategy directly determine the quality of the airborne collision avoidance system. However, for the currently available TCAS II airborne traffic alert and collision avoidance system in the market, due to its long release time, it is difficult to meet the current situation of airspace control with more accurate GPS satellite data sources and higher-density airspace. At the same time, the hard-coded method of the TCAS II airborne traffic alert and collision avoidance system leads to higher maintenance costs. Summary of the Invention

[0004] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a traffic decision-making coordination method based on Markov decision-making, making the coordination information more accurate and secure, while being compatible with the existing TCAS II airborne traffic alert and collision avoidance system, significantly reducing the false alarm rate, and saving the maintenance cost of the airborne collision avoidance algorithm, etc.

[0005] The purpose of the present invention is achieved through the following solutions:

[0006] A traffic decision-making coordination method based on Markov decision-making includes the following steps:

[0007] S1, generating a decision-making consultation and coordination strategy based on Markov: taking the individual and global consultations generated by the decision-making consultation component as inputs, referring to the state factors of the host aircraft and the intruding aircraft, constructing a coordination information operation process; calculating the optimal coordination information according to the vertical speed, consultation status, and intruding aircraft information, selecting the optimal coordination strategy, and outputting point-to-point to the target aircraft in the transponder of the host aircraft; the decision-making consultation component includes an airborne collision avoidance decision-making consultation component;

[0008] S2, processing the response strategy based on active coordination;

[0009] S3, generating decision-making consultations based on the decision-making consultation and coordination information.

[0010] Further, in step S1, taking the individual and global consultations generated by the decision-making consultation component as inputs includes the steps of: obtaining the current and previous cycle decision-making consultation status of the local machine. The ACAS X system uses Markov decision-making with a probability dynamic model and a multi-objective utility model as inputs, discretizes the models, and uses a cost function and dynamic programming method to back-calculate the optimal decision for the entire encounter process and generate the individual and global consultations.

[0011] Further, in step S1, constructing a coordinated information operation process with reference to the status factors of the local machine and the intruder includes the steps of: the source of the coordinated information generation is the decision-making consultation status of the local machine, specifically including the global decision-making consultation in the t cycle and the individual decision-making consultation V t * , the global decision-making consultation in the t - 1 cycle and the individual decision-making consultation where the current global decision-making consultation includes the global policy state S, the global minimum vertical speed V min , the global maximum vertical speed V max , the global vertical acceleration V m ' ax ;

[0012] Then establish an effective vertical speed selection model to obtain the optimal vertical speed suggestions under different states; the effective vertical speed selection model includes the following situations:

[0013] When is for COC threat elimination and V t * = 0, use the parameters corresponding to COC; when is for MAINTAIN hold decision and V t * is also MAINTAIN, use the parameters corresponding to MAINTAIN, otherwise use the parameters corresponding to the V t * state;

[0014] When is for MAINTAIN hold decision and V t * is also MAINTAIN, use the parameters corresponding to MAINTAIN, otherwise use the parameters corresponding to the V t * state;

[0015] When is a decision consultation other than COC and MAINTAIN, use the parameters corresponding to the V t * state.

[0016] Further, in step S1, calculating the optimal coordination information according to the vertical rate, consultation status, and intruder information includes the steps of:

[0017] For the optimal vertical rate parameters selected for individual intruders, respectively perform multiple basic output element selections for the following coordination information settings:

[0018] Vertical rate azimuth selection: This azimuth is determined by the magnitudes of the maximum vertical rate and the absolute value of the minimum vertical rate. If |V min |-|V max |>0, the azimuth is upward; otherwise, it is downward. If |V min |-|V max | = 0, the azimuth determination is level flight;

[0019] Vertical Resolution Advisory Supplement VRC selection: First, determine whether the intruder is equipped with a TCAS system, an ACAS X system, a CAS system with reply ability, or a DAA system with reply ability, and then calculate the VRC according to the vertical rate azimuth;

[0020] Cancellation of Vertical Resolution Advisory Supplement CVC selection: Use the periodic vertical rate azimuth and the current vertical rate azimuth as inputs to construct a CVC decision-making consultation coding algorithm to generate the optimal cancellation consultation supplement strategy;

[0021] Vertical Direction Bit VSB algorithm encryption: Use CVC * 4 + VRC + 1 as the subscript to look up the data result in the check table S[0,14,7,9,11,5,12,2,13,3,10,4,6,8,1,15].

[0022] Further, in step S1, selecting the optimal coordination strategy and performing point-to-point output in the local transponder includes the steps of: Packetize the current cycle's VRC, CVC, and VSB and send them to the local transponder, and use the transponder UF16 - UDS30 to query the intruder point-to-point to inform the other party of the local coordination information so that the intruder can maneuver or respond according to the current situation.

[0023] Further, in step S2, it includes sub-steps: When receiving the UF16 - UDS30 roll call coordination sent by the intruder, use the intruder's CVC, VRC, and VSB as inputs, and combine with the S-mode address and timestamp to perform a compliance and rationality judgment operation.

[0024] Further, the compliance and rationality judgment operation specifically includes the steps of:

[0025] S21. Coordinate message correctness and compliance check: First, determine whether the intruding aircraft in the received UF16 interrogation exists in the track. Calculate the verification value VSB based on the received CVC and VRC to determine the integrity and correctness of the current message. Then, synchronously determine the rationality of the CVC and VRC. When all three conditions are met simultaneously, record the timestamp of the current coordination.

[0026] S22. Update the VRC of the intruding aircraft: First, poll the target database and all intruders to ensure that the target has requested an update in the TRM and can be found in the database of the STM surveillance and tracking module. If the ID matches and the intruder is equipped with a TCAS system with decision consultation capabilities, the target in the STM database will be used to update the intruder's VRC to ensure that the most recently received coordination data is used in the online cost estimation. The coordination interrogation sent by the aircraft's own air traffic alert and collision avoidance system is received by the S-mode transponder of the threatened aircraft, which sends a coordination reply message as a technical notification of receipt and transmits the relevant message to the air traffic alert and collision avoidance system unit of the threatened aircraft for processing.

[0027] S23. Coordination timeout check: Using the target retrieved from memory and the generation report time containing coordination data as input, if the difference between the report time and the last known coordination update time is greater than the set value, reset the coordination data of the target and delete the vertical intention.

[0028] Furthermore, in step S3, the decision consultation generation based on the decision consultation coordination information includes the steps of: ACAS X decision consultation based on the coordination information, compatible with the collaborative interaction of the TCAS II airborne traffic alert and collision avoidance system.

[0029] Furthermore, the ACAS X decision consultation based on the coordination information, compatible with the collaborative interaction of the TCAS II airborne traffic alert and collision avoidance system, includes sub-steps: Taking the own aircraft as the subject matter, the own aircraft has a higher S-mode address than the target aircraft. Step S3 includes the following two-phase cycles:

[0030] First RA decision generation cycle: The own aircraft detects that the target aircraft is an intruder and confirms whether the target aircraft has the collaborative function. When the own aircraft makes the first RA decision, if the target aircraft has sent a collaborative message before, the own aircraft selects the opposite maneuver direction; if the target aircraft has not sent a collaborative message, the own aircraft selects the maneuver recommendation generated by the TRM module and sends its own collaborative information to the target aircraft, and the target aircraft responds and stores the collaborative message.

[0031] Non-first RA decision generation cycle: The local machine checks the collaborative message from the target machine to confirm whether the target machine has initiated an incompatible direction; if incompatible, the local machine needs to reverse to ensure compatibility with the target machine's direction; the target machine does not need to check whether it is compatible with the local machine's collaborative message; at the end of this processing cycle, each machine displays the selected RA.

[0032] Further, in the first RA decision generation cycle, after a period of time, when the target machine makes an RA decision, it has detected that the local machine is a threatening aircraft, and it is assumed that the local machine has pre-sent an altitude crossing intention message within the set time when the target machine detected that the local machine is a threatening aircraft. The target machine operates according to its own decision; if the selected direction is also altitude crossing, then the direction is compatible; if the selected direction is non-altitude crossing, the local machine will be forced to reverse, from crossing to non-crossing; if the above conditions are not met, the target machine selects a direction compatible with the local machine's intention, then sends its collaborative information to the local transponder. The local machine responds and stores the collaborative message. At the end of this processing cycle, each aircraft displays the selected RA.

[0033] The beneficial effects of the present invention include:

[0034] The technical solution of the present invention adopts Markov-based traffic decision-making collaboration to integrate multi-source air surveillance data, and uses the decision-making consultation under the MDP model as the target to generate collaborative strategies, making the collaborative information more accurate and secure. At the same time, it is compatible with the existing TCAS II airborne traffic alert and collision avoidance system, greatly reducing the false alarm rate and saving the maintenance cost of the airborne collision avoidance algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0036] Figure 1 It is the flowchart of Markov-based traffic decision-making collaboration according to the embodiment of the present invention;

[0037] Figure 2 It is the schematic diagram of the generation of coordination strategies based on decision-making consultation according to the embodiment of the present invention;

[0038] Figure 3 It is the flowchart of asynchronous collaborative processing according to the embodiment of the present invention;

[0039] Figure 4 It is the flowchart of synchronous collaborative processing according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0040] The present invention will be further described below in conjunction with the accompanying drawings and embodiments. All features disclosed in all embodiments in this specification, or all steps in any methods or processes implicitly disclosed, can be combined and / or extended, replaced in any manner, except for mutually exclusive features and / or steps.

[0041] Considering the disadvantages of the prior art in the background, the technical solution of the present invention provides a traffic decision-making collaboration method based on Markov decision-making. By adopting traffic decision-making collaboration based on Markov, multi-source air surveillance data is integrated, and a collaboration strategy is generated using the decision-making consultation under the MDP model as the target, making the collaboration information more accurate and secure. At the same time, it is compatible with the existing TCAS II airborne traffic alert and collision avoidance system, significantly reducing the false alarm rate and saving the maintenance cost of the airborne collision avoidance algorithm.

[0042] In a further concept, based on the air model based on probability theory, a traffic decision-making collaboration strategy is generated according to the threat processing module in ACAS X, and collaborative logic processing is performed with the airborne threat target through the RF data link. Finally, the current RA consultation strategy and excitation suggestions are determined according to the collaborative data, and at the same time, compatibility with the collaborative strategy of the existing airborne collision avoidance equipment in the air is achieved.

[0043] A traffic decision-making collaboration method based on Markov decision-making according to the technical solution of the present invention, as Figure 1 shown, includes the following steps:

[0044] S1: Generate a decision-making consultation coordination strategy based on Markov;

[0045] S2: Process the response strategy based on active coordination;

[0046] S3: Generate a decision-making consultation based on the decision-making consultation coordination information.

[0047] In a further specific implementation process, the step S1 is specifically: taking the individual and global consultations generated by the airborne collision avoidance decision-making consultation component as the input, referring to the state factors of the local aircraft and the intruding aircraft, constructing a coordination information operation module, selecting the best coordination strategy, and finally using it for the point-to-point output of UF16-UDS30 in the local transponder. As Figure 1 shown, this process includes the following 4 steps:

[0048] (1) Obtain the current and previous cycle decision-making consultation status of the local aircraft

[0049] (2) Select the effective vertical rate

[0050] (3) Calculate the best coordination information according to the vertical rate, consultation status and intruding aircraft information

[0051] (4) Use the transponder to perform point-to-point coordination with the intruding aircraft

[0052] ACAS X uses Markov decision-making to take a probabilistic dynamic model and a multi-objective utility model as inputs, discretizes the models, and finally uses a cost function and dynamic programming methods to inversely deduce the optimal decision for the entire encounter process. Six state variables are used in the ACAS X prototype: h: the altitude of the intruder relative to the ownship; The ownship's vertical rate; The intruder's vertical rate; τ: the time of possible collision; s adv : the current alert advisory (i.e., the RA issued in the previous cycle); s res : whether the pilot is responding to the current alert advisory.

[0053] This model assumes that the intruder's random acceleration is independent and is sampled once per second. The acceleration of the intruder is sampled from a Gaussian distribution with a mean of zero and a variance of 3 ft / s 2 . Given a, s' res The state update of the model is as shown in Equation 1:

[0054]

[0055] The source of the coordination information generation is the ownship's resolution advisory state, which mainly includes the global resolution advisory at cycle t and the individual resolution advisory V t * , the global resolution advisory at cycle t-1 and the individual resolution advisory Among them, the current global resolution advisory consists of 4 elements: the global policy state S, the global minimum vertical rate V min , the global maximum vertical rate V max , the global vertical acceleration V m ' ax A valid vertical speed selection model is established to obtain the optimal vertical speed suggestions in different states, as shown in Table 1, which mainly includes 4 cases:

[0056] a) When is COC threat elimination and V t * = 0, the parameters corresponding to COC are used;

[0057] b) When is MAINTAIN hold decision and V t * is also MAINTAIN, the parameters corresponding to MAINTAIN are used, otherwise the parameters corresponding to the V t * state are used;

[0058] c) When is the MAINTAIN decision, and V t * is also MAINTAIN, use the parameters corresponding to MAINTAIN, otherwise use the parameters corresponding to the V t * status;

[0059] When is a decision consultation other than COC and MAINTAIN, use the parameters corresponding to the V t * status.

[0060] Table 1 Decision Consultation Corresponding Parameters

[0061]

[0062]

[0063] For the optimal vertical rate parameters selected for individual intrusion aircraft, four basic output element selections for coordinated information setting are carried out respectively:

[0064] a) Vertical rate azimuth selection. This azimuth is determined by the magnitudes of the maximum vertical rate and the absolute value of the minimum vertical rate. If |V min |-|V max |>0, the azimuth is up, otherwise it is down. If |V min |-|V max | = 0, the azimuth is determined to be level flight;

[0065] b) VRC (Vertical Resolution Consultation Supplement) selection. First, determine whether the intrusion aircraft is equipped with a TCAS system, an ACAS X system, a CAS system with reply capability, or a DAA system with reply capability, and then calculate the VRC according to the vertical rate azimuth;

[0066] c) CVC (Cancel Vertical Resolution Consultation Supplement) selection. Use the vertical rate azimuth of the above cycle and the current vertical rate azimuth as inputs to construct a CVC resolution consultation coding algorithm to generate the best cancellation consultation supplement strategy;

[0067] d) VSB (Vertical Direction Bit) algorithm encryption. To increase the accuracy of data reception, use CVC * 4 + VRC + 1 as the subscript to look up the data result of the check table S[0,14,7,9,11,5,12,2,13,3,10,4,6,8,1,15].

[0068] Finally, packetize the VRC, CVC, and VSB of the current cycle and send them to the local transponder. Use the transponder to conduct a point-to-point interrogation of the intruding aircraft via UF16-UDS30 to inform the other party of the local coordination information so that the intruding aircraft can maneuver or respond according to the current situation.

[0069] For the further specific implementation process, step S2 is specifically as follows: Similarly, when receiving the UF16-UDS30 roll call coordination sent by the intruding aircraft, use the CVC, VRC, and VSB of the intruding aircraft as inputs, and combine the S-mode address and timestamp to perform a compliance and rationality judgment operation. As Figure 2 shown, this process includes the following three steps:

[0070] (1) Check the correctness and compliance of the coordination message. First, determine whether the intruding aircraft being queried by the received UF16 exists in the track. Calculate the check value VSB based on the received CVC and VRC to determine the integrity and correctness of the current message. Then, synchronously determine the rationality of the CVC and VRC. When all three conditions are met simultaneously, record the timestamp of the current coordination. If the CVC is valid and the CVC and VRC are positively correlated, then delete the coordination intention of the corresponding intruding aircraft; otherwise, update the VRC again.

[0071] (2) Update the VRC of the intruding aircraft. Receive the list of intruding aircraft in all valid TRM threat processing modules and update it using the latest VRC. First, poll the target database and all intruders to ensure that the target has been requested for update in the TRM and can be found in the database of the STM surveillance and tracking module. If the IDs match and the intruder is equipped with a TCAS system with decision-making consultation capabilities, then update the intruder's VRC using the target in the STM database to ensure that the most recently received coordination data is used in the online cost estimation.

[0072] The coordination inquiry sent by the local air traffic alert and collision avoidance system is received by the S-mode transponder of the threatened aircraft. The transponder sends a coordination response message as a technical notification of receipt and transmits the relevant message to the air traffic alert and collision avoidance system unit of the threatened aircraft for processing.

[0073] Table 2 Corresponding parameters for decision-making consultation

[0074]

[0075] (3) Check for coordination timeout. Use the target retrieved from memory and the generation report time containing the coordination data as inputs. If the difference between the report time and the last known coordination update time is greater than 6 s, then reset the coordination data of the target and delete the vertical intention.

[0076] For the further specific implementation process, step S3 is specifically as follows: Generation of decision-making consultation based on coordination information

[0077] ACAS X Decision Advisory based on coordination information, compatible with the collaborative interaction of TCAS II Airborne Traffic Alert and Collision Avoidance System. As Figure 3 shown, taking the own aircraft as the subject, this process includes the following two-phase cycles:

[0078] (1) First RA Decision Generation Cycle: As Figure 3 , Figure 4 shown, the vertical columns represent the own aircraft (left) and the target aircraft (right); the arrows between the aircraft are coordination messages; from top to bottom is the time axis, and the boxes represent the time when the aircraft completes the coordination process. Figure 3 represents standard (non-simultaneous) coordination, Figure 4 represents simultaneous coordination.

[0079] The own aircraft (left) (with a higher S-mode address) detects that the target aircraft (right) is an intruder and confirms whether the target aircraft has collaborative capabilities. When the own aircraft (left) makes the first RA decision, if the target aircraft (right) has sent a collaborative message before, the own aircraft (left) needs to select the opposite maneuvering direction; if the target aircraft (right) has not sent a collaborative message, the own aircraft (left) selects the maneuvering recommendation generated by the TRM module and sends its own collaborative information to the target aircraft (right), and the target aircraft (right) responds and stores the collaborative message.

[0080] After a period of time, when the target aircraft (right) makes the RA decision, it has detected that the own aircraft (left) is a threatening aircraft. Since the target aircraft has a smaller S-mode address, it can force the aircraft with a higher S-mode address to reverse if necessary. However, if the own aircraft (left) has sent an intention message of altitude crossing within 3 seconds before the target aircraft (right) detects that the own aircraft is a threatening aircraft, the target aircraft (right) operates according to its own decision. If the selected direction is also altitude crossing, the directions are compatible; if the selected direction is non-altitude crossing, the own aircraft (left) will be forced to reverse, from crossing to non-crossing; if the above conditions are not met, the target aircraft (right) selects a direction compatible with the intention of the own aircraft, then sends its collaborative information to the own aircraft, and the own aircraft responds and stores the collaborative message. At the end of the processing cycle, each aircraft displays the selected RA.

[0081] (2) Non-first RA Decision Generation Cycle: Since the own aircraft (left) has a higher S-mode address, it checks the collaborative message from the target aircraft (right) to confirm whether the target aircraft (right) has initiated an incompatible direction. If it is incompatible, the own aircraft needs to reverse to ensure compatibility with the direction of the target aircraft (right). Since the target aircraft (right) has a lower S-mode address, it does not need to check whether the collaborative message with the own aircraft is compatible. At the end of the processing cycle, each aircraft displays the selected RA.

[0082] Example 1

[0083] A traffic decision-making collaboration method based on Markov decision-making, comprising the following steps:

[0084] S1. Generate using a decision-making consultation coordination strategy based on Markov: Taking the individual and global consultations generated by the decision-making consultation component as inputs, referring to the state factors of the own aircraft and the intruding aircraft, construct an operation process for coordination information; Calculate the optimal coordination information according to the vertical speed, consultation status and intruding aircraft information, select the optimal coordination strategy, and perform point-to-point output in the transponder of the own aircraft to the target aircraft; The decision-making consultation component includes an on-board collision avoidance decision-making consultation component;

[0085] S2. Process the response strategy based on active coordination;

[0086] S3. Generate decision-making consultations based on the coordination information of decision-making consultations.

[0087] Embodiment 2

[0088] On the basis of Embodiment 1, in step S1, taking the individual and global consultations generated by the decision-making consultation component as inputs includes the steps: Obtain the current and previous cycle decision-making consultation status of the own aircraft, and the ACAS X system uses Markov decision-making to take the probability dynamic model and the multi-objective utility model as inputs, and perform discretization processing on the models, and use the cost function and dynamic programming methods to reverse-infer the optimal decision for the entire encounter process, and generate the individual and global consultations for the corresponding target aircraft.

[0089] Embodiment 3

[0090] On the basis of Embodiment 1, in step S1, referring to the state factors of the own aircraft and the intruding aircraft and constructing an operation process for coordination information includes the steps: The source of the coordination information generation is the decision-making consultation status of the own aircraft, specifically including the global decision-making consultation in the t cycle and the individual decision-making consultation V t * , the global decision-making consultation in the t-1 cycle and the individual decision-making consultation where the current global decision-making consultation includes the global policy state S, the global minimum vertical speed V min , the global maximum vertical speed V max , the global vertical acceleration V m ' ax ;

[0091] Then establish an effective vertical speed selection model to obtain the optimal vertical speed suggestions under different states; The effective vertical speed selection model includes the following situations:

[0092] When is the elimination of the COC threat and V t * = 0, use the parameters corresponding to the COC; When For the MAINTAIN hold decision, and for V t * When it is also MAINTAIN, use the parameters corresponding to MAINTAIN, otherwise use the parameters corresponding to V t * status;

[0093] When For the MAINTAIN hold decision, and for V t * When it is also MAINTAIN, use the parameters corresponding to MAINTAIN, otherwise use the parameters corresponding to V t * status;

[0094] When For decision consultations other than COC and MAINTAIN, use the parameters corresponding to V t * status.

[0095] Embodiment 4

[0096] On the basis of Embodiment 1, in step S1, the calculating of the optimal coordination information according to the vertical rate, consultation status and intruder information includes the steps of:

[0097] For the optimal vertical rate parameters selected for each intruder individual, respectively perform multiple basic output element selections for the following coordination information settings:

[0098] Vertical rate azimuth selection: This azimuth is determined by the magnitudes of the maximum vertical rate and the absolute value of the minimum vertical rate. If |V min |-|V max |>0, the azimuth is up, otherwise it is down. If |V min |-|V max | = 0, the azimuth is determined to be level flight;

[0099] Vertical decision consultation supplement VRC selection: First, determine whether the intruder is equipped with a TCAS system, an ACAS X system, a CAS system with reply ability or a DAA system with reply ability, and then calculate the VRC according to the vertical rate azimuth;

[0100] Cancellation of vertical decision consultation supplement CVC selection: Use the periodic vertical rate azimuth and the current vertical rate azimuth as inputs to construct a CVC decision consultation coding algorithm to generate the optimal cancellation consultation supplement strategy;

[0101] Vertical direction bit VSB algorithm encryption: Using CVC*4+VRC+1 as the subscript, look up the data result in the check table S[0,14,7,9,11,5,12,2,13,3,10,4,6,8,1,15].

[0102] Embodiment 5

[0103] Based on Embodiment 4, in step S1, when selecting the best coordination strategy, point-to-point output is performed in the local transponder according to the best coordination strategy, including the steps of: sending the current cycle's VRC, CVC, and VSB packet to the local transponder, and through the transponder UF16-UDS30, point-to-point query the intruder aircraft, and inform the other party of the local coordination information so that the intruder aircraft can maneuver or respond according to the current situation.

[0104] Embodiment 6

[0105] Based on Embodiment 1, in step S2, it includes sub-steps: when receiving the UF16-UDS30 roll call coordination sent by the intruder aircraft, using the intruder aircraft's CVC, VRC, and VSB as inputs, combined with the S-mode address and timestamp, perform a compliance and rationality judgment operation.

[0106] Embodiment 7

[0107] Based on Embodiment 6, the compliance and rationality judgment operation specifically includes the steps of:

[0108] S21, Check the correctness and compliance of the coordination message: First, determine whether the intruder aircraft queried by the received UF16 exists in the track. Calculate the check value VSB according to the received CVC and VRC to determine the integrity and correctness of the current message. Then, synchronously determine the rationality of the CVC and VRC. When all three conditions are met simultaneously, record the timestamp of the current coordination;

[0109] S22, Update the VRC of the intruder aircraft: Receive the intruder aircraft list in all effective TRM threat processing modules and update it with the latest VRC. First, poll the target database and all intruders to ensure that the target has been requested to be updated in the TRM and can be found in the database of the STM surveillance and tracking module. If the ID matches and the intruder is equipped with a TCAS system with decision-making consultation capabilities, then update the intruder's VRC with the target in the STM database to ensure that the most recently received coordination data is used in the online cost estimation; The coordination query sent by the local air traffic alert and collision avoidance system is received by the S-mode transponder of the threatened aircraft, and the transponder sends a coordination response message as a technical notice of receipt and transmits the relevant message to the air traffic alert and collision avoidance system unit of the threatened aircraft for processing;

[0110] S23, Coordination timeout check: Use the target retrieved from memory and the generated report time containing coordination data as inputs. If the difference between the report time and the last known coordination update time is greater than the set value, reset the coordination data of the target and delete the vertical intent.

[0111] Embodiment 8

[0112] Based on Embodiment 1, in step S3, the decision consultation generation based on the decision consultation coordination information includes the steps of: ACAS X decision consultation based on the coordination information, and compatible with the collaborative interaction of the TCAS II airborne traffic alert and collision avoidance system.

[0113] Embodiment 9

[0114] Based on Embodiment 8, the ACAS X decision consultation based on the coordination information, which is compatible with the collaborative interaction of the TCAS II airborne traffic alert and collision avoidance system, includes sub-steps: Taking the local aircraft as the subject matter, the local aircraft has a higher S-mode address than the target aircraft. Step S3 includes the following two-phase cycles:

[0115] First RA decision generation cycle: The local aircraft detects that the target aircraft is an intruder and confirms whether the target aircraft has the collaborative function; when the local aircraft makes the first RA decision, if the target aircraft has sent a collaborative message before, the local aircraft selects the opposite maneuver direction; if the target aircraft has not sent a collaborative message, the local aircraft selects the maneuver recommendation generated by the TRM module and sends the local aircraft's collaborative information to the target aircraft, and the target aircraft responds and stores the collaborative message.

[0116] Non-first RA decision generation cycle: The local aircraft checks the collaborative message from the target aircraft to confirm whether the target aircraft has initiated an incompatible direction; if it is incompatible, the local aircraft needs to reverse to ensure compatibility with the target aircraft's direction; the target aircraft does not need to check whether it is compatible with the local aircraft's collaborative message; at the end of this processing cycle, each aircraft displays the selected RA.

[0117] Embodiment 10

[0118] Based on Embodiment 9, in the first RA decision generation cycle, after a period of time, when the target aircraft makes an RA decision, it has detected that the own aircraft is a threatening aircraft. Since the S-mode address of the target aircraft is relatively small, if necessary, an aircraft with a higher S-mode address can be forced to reverse. However, if the own aircraft has pre-sent an intention message for altitude crossing within the set time when the target aircraft detected that the own aircraft is a threatening aircraft, the target aircraft operates according to its own decision. Assume that the own aircraft has pre-sent an intention message for altitude crossing within the set time when the target aircraft detected that the own aircraft is a threatening aircraft, and the target aircraft operates according to its own decision. If the selected direction is also altitude crossing, then the directions are compatible. If the selected direction is non-altitude crossing, the own aircraft will be forced to reverse, changing from crossing to non-crossing. If the above conditions are not all met, the target aircraft selects a direction compatible with the intention of the own aircraft, then sends its coordination information to the own aircraft's transponder. The own aircraft responds and stores the coordination message. At the end of this processing cycle, each aircraft displays the selected RA.

[0119] The units involved in the embodiments of the present invention can be implemented in software or in hardware. The described units can also be provided in a processor. Among them, the names of these units do not constitute a limitation to the unit itself in some cases.

[0120] According to one aspect of the present application, there is provided a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above various optional implementation manners.

[0121] As another aspect, the present application also provides a computer-readable medium. The computer-readable medium can be included in the electronic device described in the above embodiments; or it can exist alone without being assembled into the electronic device. The above computer-readable medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device implements the methods described in the above embodiments.

[0122] Parts not involved in the present invention are the same as or can be implemented by the prior art.

[0123] The above technical solution is only one implementation manner of the present invention. For those skilled in the art, based on the application methods and principles disclosed in the present invention, it is very easy to make various types of improvements or deformations, not limited to the methods described in the above specific implementation manners of the present invention. Therefore, the above-described manner is only preferred and does not have a restrictive meaning.

[0124] In addition to the above examples, those skilled in the art can obtain inspiration based on the above disclosure or make modifications by using the knowledge or technology in related fields to obtain other embodiments. The features of each embodiment can be interchanged or replaced. As long as the modifications and changes made by those skilled in the art do not depart from the spirit and scope of the present invention, they should all fall within the protection scope of the appended claims of the present invention.

Claims

1. A traffic decision-making collaboration method based on Markov decision, characterized in that, it includes the following steps: S1. Generate using a decision-making consultation coordination strategy based on Markov: taking the individual and global consultations generated by the decision-making consultation component as inputs, referring to the state factors of the own aircraft and the intruding aircraft, and constructing a coordination information operation process; calculating the optimal coordination information according to the vertical rate, consultation status and intruding aircraft information, selecting the optimal coordination strategy, and outputting point-to-point to the target aircraft in the own aircraft transponder according to the optimal coordination strategy; the decision-making consultation component includes an on-board collision avoidance decision-making consultation component; S2. Process the response strategy based on active coordination; S3. Generate decision-making consultation based on the decision-making consultation coordination information; In step S1, a coordination information operation process is constructed by referring to the state factors of the local machine and the intrusion machine, including the steps: The source of the generation of coordination information is the decision consultation state of the local machine, specifically including the global decision consultation in the t cycle and the individual decision consultation the global decision consultation in the t-1 cycle and the individual decision consultation where the current global decision consultation includes the global policy state S, the global minimum vertical rate V min , the global maximum vertical rate V max , the global vertical acceleration V m ' ax ; Then establish an effective vertical speed selection model to obtain optimal vertical speed suggestions under different states; the effective vertical speed selection model includes the following situations: When is for COC threat elimination, use the parameters corresponding to COC; when is for MAINTAIN hold decision and is also MAINTAIN, use the parameters corresponding to MAINTAIN, otherwise use the parameters corresponding to the status; When is the MAINTAIN retention decision and is also MAINTAIN, use the parameters corresponding to MAINTAIN; otherwise use the parameters corresponding to the status; When U t * For decision-making consultations other than COC and MAINTAIN, use the parameters corresponding to the V t * status; In step S1, the calculating the optimal coordination information according to the vertical rate, consultation status and intruding aircraft information includes the steps: For the optimal vertical rate parameters selected for the individual intruding aircraft, respectively make the following multiple basic output element selections for coordination information setting: Vertical rate azimuth selection: This azimuth is determined by the magnitude of the global maximum vertical rate and the absolute value of the global minimum vertical rate. If |V min | - |V max | > 0, the azimuth is upward; otherwise, it is downward. If |V min | - |V max | = 0, the azimuth is determined to be level flight; Vertical decision-making consultation supplement VRC selection: First determine whether the intruding aircraft is equipped with a TCAS system, an ACAS X system, a CAS system with reply ability or a DAA system with reply ability, and then calculate the VRC according to the vertical rate azimuth; Cancel vertical decision-making consultation supplement CVC selection: Take the periodic vertical rate azimuth and the current vertical rate azimuth as inputs, construct a CVC decision-making consultation coding algorithm, and generate the best cancellation consultation supplement strategy; Vertical direction bit VSB algorithm encryption: Use CVC*4+VRC+1 as the subscript to look up the data result of the check table S[0,14,7,9,11,5,12,2,13,3,10,4,6,8,1,15]; In step S1, the selecting the optimal coordination strategy and outputting point-to-point in the own aircraft transponder according to the optimal coordination strategy includes the steps: Pack the current period's VRC, CVC and VSB and send them to the own aircraft transponder, and ask the intruding aircraft point-to-point through the transponder UF16-UDS30 to inform the other party of the own coordination information so that the intruding aircraft can maneuver or respond according to the current situation; In step S2, it includes sub-steps: when receiving the UF16-UDS30 roll call coordination sent by the intruding aircraft, take the CVC, VRC and VSB of the intruding aircraft as inputs, and combine the S-mode address and timestamp to perform a compliance and rationality judgment operation; The performing the compliance and rationality judgment operation specifically includes the steps: S21. Check the correctness and compliance of the coordination message: First determine whether the intruding aircraft queried by the received UF16 exists in the track, calculate the check value VSB according to the received CVC and VRC, determine the integrity and correctness of the current message, and then synchronously determine the rationality of the CVC and VRC. When all three conditions are met, record the timestamp of the current coordination; S22. Update the VRC of the intruder aircraft: First, poll the target database and all intruders to ensure that the target has requested an update in the TRM and the corresponding target can be found in the database of the STM surveillance and tracking module. If the ID matches and the intruder is equipped with a TCAS system with decision consultation capabilities, the target in the STM database will be used to update the VRC of the intruder to ensure that the most recently received coordination data is used in the online cost estimation. The coordination interrogation sent by the own aircraft's Air Traffic Alert and Collision Avoidance System is received by the S-mode transponder of the threatened aircraft, which sends a coordination reply message as a technical notice of receipt and transmits the relevant message to the Air Traffic Alert and Collision Avoidance System unit of the threatened aircraft for processing. S23. Coordination timeout check: Using the target retrieved from memory and the generation report time containing coordination data as input, if the difference between the report time and the last known coordination update time is greater than the set value, reset the coordination data of the target and delete the vertical intention. In step S3, the decision consultation generation based on the resolution advisory coordination information includes the steps of: ACAS X decision consultation based on the coordination information, and compatible collaborative interaction with the TCAS II Airborne Traffic Alert and Collision Avoidance System. The ACAS X decision consultation based on the coordination information, compatible with the collaborative interaction of the TCAS II Airborne Traffic Alert and Collision Avoidance System, includes sub-steps: Taking the own aircraft as the subject matter, the own aircraft has a higher S-mode address than the target aircraft. Step S3 includes the following two-phase cycles: First RA decision generation cycle: The own aircraft detects that the target aircraft is an intruder and confirms whether the target aircraft has collaborative capabilities. When the own aircraft makes an RA decision first, if the target aircraft has sent a collaborative message before, the own aircraft selects a maneuver direction opposite to it; if the target aircraft has not sent a collaborative message, the own aircraft selects the maneuver recommendation generated by the TRM module and sends its own collaborative information to the target aircraft, and the target aircraft responds and stores the collaborative message. Non-first RA decision generation cycle: The own aircraft checks the collaborative message from the target aircraft to confirm whether the target aircraft has initiated an incompatible direction; if it is incompatible, the own aircraft needs to reverse to ensure compatibility with the target aircraft's direction; the target aircraft does not need to check whether its collaborative message is compatible with the own aircraft's; at the end of this processing cycle, each aircraft displays the selected RA.

2. The traffic decision-making collaboration method based on Markov decision according to claim 1, characterized in that in step S1, taking the individual and global consultations generated by the decision consultation component as input, includes the steps of: obtaining the current and previous cycle decision consultation states of the own aircraft, the ACAS X system uses Markov decision to take the probability dynamic model and multi-objective utility model as input, and discretizes the model, and uses the cost function and dynamic programming method to inversely deduce the optimal decision of the entire encounter process, and generates the individual and global consultations corresponding to the target aircraft.

3. The traffic decision-making collaboration method based on Markov decision according to claim 1, characterized in that During the first RA decision-making generation cycle, after a period of time, when the target aircraft makes an RA decision, it has detected that the local aircraft is a threatening aircraft, and it is assumed that the local aircraft has pre-sent an intention message with altitude crossing within the set time when the target aircraft detected that the local aircraft is a threatening aircraft. The target aircraft operates according to its own decision; if the selected direction is also altitude crossing, the directions are compatible; if the selected direction is non-altitude crossing, the local aircraft will be forced to reverse, changing from crossing to non-crossing; if none of the conditions are met, the target aircraft will select a direction compatible with the intention of the local aircraft, then send its cooperative information to the local transponder. After the local aircraft responds, it stores the cooperative message, and at the end of this processing cycle, each aircraft displays the selected RA.

Images