Aircraft airport taxiing scheduling model training method and system

Abstract

The present invention discloses a method and system for training an aircraft airport taxiing scheduling model, comprising obtaining aircraft airport data to be trained and vehicle topological structure data to be trained, and determining a target digital twin model according to the aircraft airport data to be trained and the vehicle topological structure data to be trained; performing simulation operation on the target digital twin model to generate target simulation operation data; constructing an airport aircraft taxiing model using the target simulation operation data, the aircraft airport data to be trained and the vehicle topological structure data to be trained; performing simulation operation on the airport aircraft taxiing model to output an aircraft taxiing motion data set; performing distributed training on an initial aircraft airport taxiing scheduling model according to the aircraft taxiing motion data set, the target simulation operation data and preset actual operation data to determine a target aircraft airport taxiing scheduling model; and solving the technical problem that the existing aircraft airport taxiing scheduling technology leads to low safety of the overall operation of the airport.

Description

Technical Field

[0001] The invention relates to the technical field of airport resource scheduling, and in particular to a training method and system for an aircraft airport taxiing scheduling model. Background Art

[0002] Civil airports usually have at least dozens of aircraft participating in flights every flight day. Some busy international airports may have more flights taking off and landing during peak hours, with the number of flights per day possibly ranging from 100 to 500. At the same time, there may be more than 10 aircraft taking off or landing on the airport taxiway. This requires the airport ground command center to efficiently conduct aircraft taxiing command and dispatch to ensure that there is no path conflict during the taxiing process and that the aircraft can quickly reach the designated location to complete the take-off or landing mission.

[0003] However, with the increase in the number of aircraft, it is becoming increasingly difficult to quickly schedule aircraft taxiing at airports given the limited airport area and scheduling time. In an airport environment, aircraft will take off, land, and taxi, and their positions and states will change. The movement of aircraft will lead to adjustments to ground traffic and recalculation of path planning, making airport scheduling very complicated.

[0004] Most of the existing airport taxiing scheduling technologies use the method of changing the taxiing path to reduce time costs or conduct research on a single intersection. However, in actual operation, the above methods make it difficult for aircraft to turn, and when the aircraft is subsequently taxiing, the presence of vehicles will affect the route selection of the aircraft during taxiing and takeoff, causing potential conflicts and delays, resulting in lower safety of the overall airport operation. Summary of the invention

[0005] The present invention provides an aircraft airport taxiing scheduling model training method and system, which are used to solve the technical problem that the existing aircraft airport taxiing scheduling technology leads to low safety of the overall operation of the airport.

[0006] A first aspect of the present invention provides an aircraft airport taxiing scheduling model training method, comprising:

[0007] Acquire aircraft airport data to be trained and vehicle topology data to be trained, and determine a target digital twin model according to the aircraft airport data to be trained and the vehicle topology data to be trained;

[0008] Performing simulation operation on the target digital twin model to generate target simulation operation data;

[0009] Using the target simulation operation data, the airport data of the aircraft to be trained and the topological structure data of the vehicle to be trained, constructing an airport aircraft taxiing model;

[0010] Performing simulation calculation on the airport aircraft taxiing model and outputting an aircraft taxiing motion data set;

[0011] The initial aircraft airport taxiing scheduling model is distributedly trained according to the aircraft taxiing motion data set, the target simulation operation data and the preset actual operation data to determine the target aircraft airport taxiing scheduling model.

[0012] Optionally, the step of determining the target digital twin model according to the aircraft airport data to be trained and the vehicle topology data to be trained includes:

[0013] Using the aircraft airport data to be trained and the vehicle topology data to be trained to construct an initial digital twin model;

[0014] Performing simulation operation on the initial digital twin model to generate initial simulation operation data;

[0015] The preset actual operation data and the initial simulation operation data are used to train the initial digital twin model to determine the target digital twin model.

[0016] Optionally, the step of performing distributed training on the initial aircraft airport taxiing scheduling model according to the aircraft taxiing motion data set, the target simulation operation data and preset actual operation data to determine the target aircraft airport taxiing scheduling model includes:

[0017] Based on the preset danger zone screening conditions, determining multiple target position relationships according to the target simulation operation data and the preset actual operation data;

[0018] Using a preset multi-agent reinforcement learning algorithm, according to the aircraft taxiing motion data set, multiple simulation environment information in the target simulation operation data, multiple actual environment information in the preset actual operation data, and each target position relationship, calculate the simulation importance degree value corresponding to each simulation environment information and the actual importance degree value corresponding to each actual environment information;

[0019] Comparing each of the simulated importance degree values, each of the actual importance degree values ​​and a preset degree threshold;

[0020] Taking any simulation environment information corresponding to the simulation importance degree value greater than the preset degree threshold and the actual environment information corresponding to the actual importance degree value as the target environment information;

[0021] Sampling the aircraft taxiing motion data set to determine a plurality of sampled aircraft taxiing motion data;

[0022] Using the target environment information and the plurality of sampled aircraft taxiing motion data as inputs of an initial strategy network in the initial aircraft airport taxiing scheduling model, and outputting a plurality of aircraft taxiing distributed strategies to be trained;

[0023] The initial aircraft airport taxiing scheduling model is trained by using the taxiing distributed strategies of the aircraft to be trained, multiple simulation environment information in the target simulation operation data, and multiple actual environment information in the preset actual operation data to determine the target aircraft airport taxiing scheduling model.

[0024] Optionally, the step of determining the position relationship of multiple targets based on the preset danger zone screening condition and according to the target simulation operation data and preset actual operation data includes:

[0025] Determining multiple simulated aircraft position relationships and multiple actual aircraft position relationships according to multiple simulated environment information in the target simulation operation data and multiple actual environment information in the preset actual operation data;

[0026] Based on the preset danger zone screening conditions, the simulated aircraft position relationships and the actual aircraft position relationships are screened to determine a plurality of target position relationships.

[0027] Optionally, the initial aircraft airport taxi scheduling model further includes an initial value function network and an initial replica network; the target aircraft airport taxi scheduling model includes a target strategy network, a target value function network and a target replica network; the step of training the initial aircraft airport taxi scheduling model using the taxi distributed strategies of the aircraft to be trained, the multiple simulation environment information in the target simulation operation data and the multiple actual environment information in the preset actual operation data to determine the target aircraft airport taxi scheduling model includes:

[0028] Interacting the distributed taxiing strategies of the aircraft to be trained, the multiple simulation environment information in the target simulation operation data, and the multiple actual environment information in the preset actual operation data to generate interaction results, multiple new simulation environment information, and multiple new actual environment information;

[0029] Calculate a reward function value according to the interaction result;

[0030] Scoring each of the taxiing distributed strategies of the aircraft to be trained and the probability distribution of each of the taxiing distributed strategies of the aircraft to be trained respectively through the initial value function network, and outputting a first scoring result and a second scoring result;

[0031] Using the target replica network to score each of the aircraft taxiing distributed strategies to be trained, and outputting a third scoring result;

[0032] Substituting the reward function value, the first scoring result, the second scoring result and the third scoring result into a preset loss function and taking derivatives to determine a model gradient;

[0033] Based on the preset optimizer, the model parameters of the initial policy network, the model parameters of the initial value function network and the model parameters of the initial copy network are updated using the model gradient, an intermediate policy network, an intermediate value function network and an intermediate copy network are determined, and the number of simulation iterations is counted in real time;

[0034] Determine whether the number of simulation iterations reaches a preset iteration threshold;

[0035] If the number of simulation iterations reaches the preset iteration threshold, the intermediate policy network is used as the target policy network, the intermediate value function network is used as the target value function network, and the intermediate replica network is used as the target replica network.

[0036] Optionally, it also includes:

[0037] Acquire the airport data of the aircraft to be scheduled and the topological structure data of the vehicles to be scheduled, and determine the digital twin model of the target to be scheduled according to the airport data of the aircraft to be scheduled and the topological structure data of the vehicles to be scheduled;

[0038] Performing simulation calculation on the digital twin model of the target to be scheduled to generate simulation operation data of the target to be scheduled;

[0039] Using the simulation operation data of the target to be scheduled, the data of the airport to be scheduled and the topological structure data of the vehicle to be scheduled, a taxiing model of the aircraft at the airport to be scheduled is constructed;

[0040] Performing simulation calculation on the taxiing model of the aircraft at the airport to be scheduled, and outputting a taxiing motion data set of the aircraft to be scheduled;

[0041] Based on the preset danger zone screening conditions and the preset multi-agent reinforcement learning algorithm, the environmental information of the target to be scheduled is determined according to the simulation operation data of the target to be scheduled, the preset actual operation data and the taxiing motion data set of the aircraft to be scheduled;

[0042] The target environment information to be scheduled and the taxiing motion data set of the aircraft to be scheduled are used as inputs of the target strategy network in the airport taxiing scheduling model of the target aircraft, and multiple target aircraft taxiing distributed strategies are output.

[0043] A second aspect of the present invention provides an aircraft airport taxiing scheduling model training system, comprising:

[0044] An acquisition module, used to acquire aircraft airport data to be trained and vehicle topology data to be trained, and determine a target digital twin model according to the aircraft airport data to be trained and the vehicle topology data to be trained;

[0045] A first computing module is used to perform simulation computing on the target digital twin model to generate target simulation operation data;

[0046] A construction module, used to construct an airport aircraft taxiing model by using the target simulation operation data, the aircraft airport data to be trained and the vehicle topology structure data to be trained;

[0047] A second computing module is used to perform simulation computing on the airport aircraft taxiing model and output an aircraft taxiing motion data set;

[0048] The training module is used to perform model training on the initial aircraft airport taxiing scheduling model according to the aircraft taxiing motion data set, the target simulation operation data and the preset actual operation data, and determine the target aircraft airport taxiing scheduling model.

[0049] A third aspect of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the aircraft airport taxiing scheduling model training method as described in any one of the above items.

[0050] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed, the steps of the aircraft airport taxiing scheduling model training method as described in any one of the above items are implemented.

[0051] A fifth aspect of the present invention provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions, wherein when the program instructions are executed by a computer, the computer executes the steps of the aircraft airport taxi scheduling model training method as described in any one of the above items.

[0052] It can be seen from the above technical solutions that the present invention has the following advantages:

[0053] The above technical scheme of the present invention provides an aircraft airport taxiing scheduling model training method, firstly, the aircraft airport data to be trained and the vehicle topological structure data to be trained are obtained, and the target digital twin model is determined according to the aircraft airport data to be trained and the vehicle topological structure data to be trained; then, the target digital twin model is simulated to generate target simulation operation data; the airport aircraft taxiing model is constructed by using the target simulation operation data, the aircraft airport data to be trained and the vehicle topological structure data to be trained; the airport aircraft taxiing model is simulated to output the aircraft taxiing motion data set; the initial aircraft airport taxiing scheduling model is distributedly trained according to the aircraft taxiing motion data set, the target simulation operation data and the preset actual operation data to determine the target aircraft airport taxiing scheduling model; based on the above scheme, the aircraft airport data to be trained and the vehicle topological structure data to be trained are obtained to determine the target digital twin model, and then the initial aircraft airport taxiing scheduling model is distributedly learned by using the actual operation data of the physical space, the simulation operation data generated by the target digital twin model and the generated aircraft taxiing motion data set, and the determined target aircraft airport taxiing scheduling model can effectively deal with possible risk areas and further improve the safety of the overall operation of the airport. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0055] Figure 1 A flowchart of the steps of a method for training an aircraft airport taxiing scheduling model provided by an embodiment of the present invention;

[0056] Figure 2 A flowchart of another method for training an aircraft airport taxiing scheduling model provided by an embodiment of the present invention;

[0057] Figure 3 A schematic diagram of the overall framework of model training provided by an embodiment of the present invention;

[0058] Figure 4 A schematic diagram of the structure of a binary tree provided by an embodiment of the present invention;

[0059] Figure 5 A structural block diagram of an aircraft airport taxiing scheduling model training system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The embodiment of the present invention provides an aircraft airport taxiing scheduling model training method and system, which are used to solve the technical problem that the existing aircraft airport taxiing scheduling technology leads to low safety of the overall operation of the airport.

[0061] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] See also Figure 1 , Figure 1 A flowchart of the steps of a method for training an aircraft airport taxiing scheduling model provided by an embodiment of the present invention.

[0063] The present invention provides an aircraft airport taxiing scheduling model training method, comprising:

[0064] Step 101: Acquire the aircraft airport data to be trained and the vehicle topology data to be trained, and determine the target digital twin model according to the aircraft airport data to be trained and the vehicle topology data to be trained.

[0065] The aircraft airport data to be trained includes aircraft topology data and airport basic environment data for model training; the vehicle topology data to be trained is vehicle topology data for model training; wherein the airport basic environment data includes road condition data, climate data, personnel data and cargo data; the aircraft topology data, vehicle topology data and airport basic environment data are obtained through the information acquisition component; the information acquisition component includes GPS data sensor, acceleration sensor, gyroscope sensor, attitude sensor, laser ranging sensor, environmental sensor and camera. The GPS data sensor is mainly used to collect positioning information of running equipment to determine the orientation and speed of the vehicle or aircraft. The acceleration sensor is used to obtain the running acceleration of the vehicle and the aircraft. The gyroscope sensor is used to sense the change of the acceleration of the vehicle or aircraft and the attitude of the transportation equipment to further obtain the overall state of the vehicle or aircraft. The attitude sensor can obtain data such as the three-dimensional attitude and orientation of the vehicle or aircraft, and accurately obtain the state of the vehicle or aircraft in three-dimensional space. The laser ranging sensor can be used to measure the target distance. The environmental sensor is used to collect the temperature, humidity and wind direction of the environment where the vehicle or aircraft is located. Cameras can be used to obtain image data of vehicles or aircraft during operation.

[0066] Specifically, an initial digital twin model is constructed using the airport data of the aircraft to be trained and the topological structure data of the vehicle to be trained; the initial digital twin model is simulated to generate initial simulation operation data; the initial digital twin model is trained using the preset actual operation data and the initial simulation operation data to determine the target digital twin model; wherein, the obtained initial digital twin model is simulated, and the initial digital twin model is trained according to the obtained initial simulation operation data and the preset actual operation data to obtain the target digital twin model; the preset actual operation data is the operation data and vehicle operation data generated by the aircraft during the historical taxiing process.

[0067] In this embodiment, the aircraft airport data to be trained and the vehicle topology structure data to be trained are obtained, and the target digital twin model is determined based on the aircraft airport data to be trained and the vehicle topology structure data to be trained.

[0068] Step 102: Perform simulation operations on the target digital twin model to generate target simulation operation data.

[0069] It should be noted that the virtual data (target simulation operation data) generated by the target digital twin model is used for the subsequent centralized training of the distributed execution framework's multi-agent deep reinforcement learning algorithm to perform distributed strategy learning on the initial aircraft airport taxi scheduling model to be trained, and obtain the distributed strategy corresponding to the probability distribution of different aircraft taxiing actions, thereby determining the trained target aircraft airport taxi scheduling model.

[0070] In this embodiment, a simulation operation is performed on the target digital twin model to generate target simulation operation data.

[0071] Step 103: construct an airport aircraft taxiing model using the target simulation operation data, the aircraft airport data to be trained, and the vehicle topology data to be trained.

[0072] It should be noted that the constructed airport aircraft taxiing model can simulate the taxiing action of an actual aircraft, and all simulated taxiing actions of actual aircraft will be used in subsequent model training.

[0073] In this embodiment, the airport aircraft taxiing model is constructed using target simulation operation data, airport data of aircraft to be trained, and topological structure data of vehicles to be trained.

[0074] Step 104: Perform simulation calculation on the airport aircraft taxiing model and output the aircraft taxiing motion data set.

[0075] In this embodiment, a simulation operation is performed on the airport aircraft taxiing model to output an aircraft taxiing motion data set.

[0076] Step 105: Distributed training is performed on the initial aircraft airport taxiing scheduling model according to the aircraft taxiing motion data set, the target simulation operation data and the preset actual operation data to determine the target aircraft airport taxiing scheduling model.

[0077] The initial aircraft airport taxiing scheduling model includes the initial value function network (Q network, i.e. Critic network, deep learning model), the initial policy network (Actor network) and the initial replica network (i.e. the replica of the Q network and the policy network). Since the current state quantity in the reinforcement learning algorithm is only related to the state at the previous moment, it is impossible to obtain valuable learning experience from more distant state information. Therefore, the Long Short Term Memory (LSTM) network is introduced into the model to enhance the model's long-term memory and learning capabilities.

[0078] Specifically, based on the preset danger zone screening conditions, multiple target position relationships are determined according to the target simulation operation data and the preset actual operation data; wherein, the target simulation operation data includes multiple simulation environment information, and the preset actual operation data includes multiple actual environment information, and the environment information, i.e., the state space, includes the position of the aircraft itself, the position of obstacles within the local observation range, the position of the aircraft target point, and the position of other aircraft within the local observation range. In addition, it also includes the speed and heading of the aircraft itself and the speed and heading of other aircraft within the local field of view; therefore, according to each simulation environment information in the target simulation operation data and each actual environment information of the preset actual operation data, the simulated aircraft position relationship corresponding to each simulation environment information and the actual aircraft position relationship corresponding to each actual environment information can be determined, and the aircraft position relationship is the distance value and direction between the aircraft and the remaining obstacles and other aircraft, and then the preset danger zone screening conditions are used to determine multiple target position relationships according to the above aircraft position relationship.

[0079] Furthermore, the preset danger zone screening conditions are divided into three parts: "no collision danger zone", "possible collision danger zone" and "serious collision danger zone"; "no collision danger zone" refers to the area where the distance between the aircraft and other detected aircraft or static obstacles is far and there is no collision risk, that is, the area outside the expanded aircraft field, specifically outside the distance range of 2 times the aircraft wing span, that is, if the aircraft position relationship corresponding to the simulated environment information or the aircraft position relationship corresponding to the actual environment information is outside the distance range of 2 times the aircraft wing span, the aircraft position relationship corresponding to the simulated environment information or the aircraft position relationship corresponding to the actual environment information will be eliminated; "possible collision danger zone" refers to the area where there is a collision risk between the aircraft and other detected aircraft or static obstacles, that is, between the aircraft fields before and after the expansion. area, specifically within a distance range of 1 to 2 times the aircraft wing span, that is, the aircraft position relationship corresponding to the simulation environment information or the aircraft position relationship corresponding to the actual environment information is within a distance range of 1 to 2 times the aircraft wing span, then the aircraft position relationship corresponding to the simulation environment information or the aircraft position relationship corresponding to the actual environment information is used as the target position relationship; "serious collision danger zone" refers to an area where the distance between the aircraft and other detected aircraft or static obstacles is short, which may result in the aircraft being unable to avoid collision in time and there is a great risk of collision, that is, within the aircraft wing span, that is, the aircraft position relationship corresponding to the simulation environment information or the aircraft position relationship corresponding to the actual environment information is within the aircraft wing span, then the aircraft position relationship corresponding to the simulation environment information or the aircraft position relationship corresponding to the actual environment information is used as the target position relationship.

[0080] Furthermore, a preset multi-agent reinforcement learning algorithm is used to calculate the simulation importance degree value corresponding to each simulation environment information and the actual importance degree value corresponding to each actual environment information according to the aircraft taxiing motion data set, multiple simulation environment information in the target simulation operation data, multiple actual environment information in the preset actual operation data and the position relationship of each target; compare each simulation importance degree value, each actual importance degree value and the preset degree threshold; and take any simulation environment information corresponding to a simulation importance degree value greater than the preset degree threshold and the actual environment information corresponding to the actual importance degree value as the target environment information.

[0081] Furthermore, the aircraft taxiing motion data set is sampled to determine a plurality of sampled aircraft taxiing motion data; the target environment information and the plurality of sampled aircraft taxiing motion data are used as inputs of the initial strategy network in the initial aircraft airport taxiing scheduling model, and a plurality of aircraft taxiing distributed strategies to be trained are output; the initial aircraft airport taxiing scheduling model is trained using the aircraft taxiing distributed strategies to be trained, a plurality of simulation environment information in the target simulation operation data, and a plurality of actual environment information in the preset actual operation data, and the target aircraft airport taxiing scheduling model is determined; wherein, the distributed strategies output by the model of the present invention include speed adjustment and heading adjustment, etc., and the speed adjustment range of the aircraft during actual flight is 20% of the normal speed. The present invention intends to select adjusting the heading and speed of the aircraft as a conflict resolution strategy (distributed strategy); the aircraft taxiing motion data set is a set of flight intentions for resolving conflicts, and there are 9 actions in the discrete action space. Each aircraft taxiing motion data in the aircraft taxiing motion data set includes two dimensions: speed and heading. The action space can be represented by a binary group. The speed includes low speed, medium speed and high speed. 1 represents low speed, 2 represents medium speed, and 3 represents high speed. For the speed transformation, a fixed speed is added or subtracted from the medium speed to obtain high speed or low speed. The heading includes left yaw, straight ahead and right yaw. 1 represents left yaw, 2 represents straight ahead, and 3 represents right yaw. Each aircraft taxiing motion data is a combination of heading and speed.

[0082] In this embodiment, the initial aircraft airport taxiing scheduling model is distributedly trained according to the aircraft taxiing motion data set, the target simulation operation data and the preset actual operation data to determine the target aircraft airport taxiing scheduling model.

[0083] In an embodiment of the present invention, the present invention provides an aircraft airport taxiing scheduling model training method, firstly, obtaining aircraft airport data to be trained and vehicle topology data to be trained, and determining a target digital twin model according to the aircraft airport data to be trained and the vehicle topology data to be trained; then, performing simulation operation on the target digital twin model to generate target simulation operation data; constructing an airport aircraft taxiing model by using the target simulation operation data, the aircraft airport data to be trained and the vehicle topology data to be trained; performing simulation operation on the airport aircraft taxiing model to output an aircraft taxiing motion data set; performing distributed training on an initial aircraft airport taxiing scheduling model according to the aircraft taxiing motion data set, the target simulation operation data and the preset actual operation data to determine a target aircraft airport taxiing scheduling model; based on the above scheme, obtaining aircraft airport data to be trained and vehicle topology data to be trained, determining a target digital twin model, and then using the actual operation data of the physical space, the simulation operation data generated by the target digital twin model and the generated aircraft taxiing motion data set to perform distributed strategy learning on the initial aircraft airport taxiing scheduling model, the determined target aircraft airport taxiing scheduling model can effectively deal with possible risk areas, and further improve the safety of the overall operation of the airport.

[0084] See also Figure 2 , Figure 2 A flowchart of the steps of another aircraft airport taxiing scheduling model training method provided by an embodiment of the present invention.

[0085] The present invention provides an aircraft airport taxiing scheduling model training method, comprising:

[0086] Step 201: Acquire the aircraft airport data to be trained and the vehicle topology data to be trained, and determine the target digital twin model according to the aircraft airport data to be trained and the vehicle topology data to be trained.

[0087] Furthermore, step 201 may include the following sub-steps:

[0088] S11, using the aircraft airport data to be trained and the vehicle topology data to be trained to build an initial digital twin model;

[0089] S12, performing simulation operation on the initial digital twin model to generate initial simulation operation data;

[0090] S13. Use preset actual operation data and initial simulation operation data to train the initial digital twin model and determine the target digital twin model.

[0091] It should be noted that the content of simulation calculation includes comprehensive consideration of the load calculation of components, heat generation calculation, structural stability calculation and other calculations that can fully conform to the physical space. The simulation calculation method can be finite element analysis, aerodynamic calculation, etc. The simulation operation data is mainly used for future intelligent prediction and judgment; through simulation calculation, the degree of conformity between the digital twin model and the physical space can be judged. Generally, some general simulation calculations can be done at the beginning, and the calculation results can be compared with the results of the physical space. According to the comparison results, the digital twin model is adjusted to obtain results consistent with the physical space. Usually, a calibration module can be established to calibrate the digital twin model so that the properties and behaviors in the digital twin model are closer to the real objects in the physical space.

[0092] In this embodiment, the aircraft airport data to be trained and the vehicle topology structure data to be trained are obtained, and the target digital twin model is determined based on the aircraft airport data to be trained and the vehicle topology structure data to be trained.

[0093] Step 202: Perform simulation operations on the target digital twin model to generate target simulation operation data.

[0094] It should be noted that obtaining simulated virtual operation data by simulating the target digital twin model can reduce the time cost of data acquisition compared to the method of obtaining data based on actual physical space experiments.

[0095] In this embodiment, a simulation operation is performed on the target digital twin model to generate target simulation operation data.

[0096] Step 203: construct an airport aircraft taxiing model using the target simulation operation data, the aircraft airport data to be trained, and the vehicle topology data to be trained.

[0097] It should be noted that a real airport is used as a simulation scenario for modeling to verify the practical feasibility of the proposed solution. When building the simulation environment, letters are used to represent the taxiway position and numbers are used to represent the parking positions. In order to simplify the operation mode of the taxiway, the aircraft always maintains the same direction when using the same taxiway. For the scenario of intersection conflict, the intersection of the taxiways is set to conflict, and the conflict scenario is simulated by setting aircraft with different operating trajectories. The performance of the algorithm is evaluated by the average release success rate and the average taxiing time. The calculation formula for the current conflict release success rate of the agent is the number of current successful release rounds / the number of rounds currently trained by the agent. The average taxiing time is the time from the start of the maneuver to the resolution of the conflict.

[0098] In this embodiment, the airport aircraft taxiing model is constructed using target simulation operation data, airport data of aircraft to be trained, and topological structure data of vehicles to be trained.

[0099] Step 204: Perform simulation calculation on the airport aircraft taxiing model and output the aircraft taxiing motion data set.

[0100] In this embodiment, a simulation operation is performed on the airport aircraft taxiing model to output an aircraft taxiing motion data set.

[0101] Step 205: Distributed training is performed on the initial aircraft airport taxiing scheduling model according to the aircraft taxiing motion data set, the target simulation operation data and the preset actual operation data to determine the target aircraft airport taxiing scheduling model.

[0102] It should be noted that, based on the deep reinforcement learning algorithm combined with the airport aircraft taxiing model, the airport aircraft taxiing model is placed in the airport simulation environment for simulation calculation to generate an aircraft taxiing action data set. The Actor network uses a neural network based on a multi-head attention mechanism to generate a strategy in a multi-agent environment, that is, a distributed strategy for aircraft taxiing to be trained. The roulette strategy is used to guide the search process of subsequent solutions. At the same time, in order to seek the diversity of optimal solutions, the deep reinforcement learning algorithm takes into account both the guidance and random search of the neural network. Among them, the target simulation operation data and the environmental information of the preset actual operation data, and the aircraft taxiing motion data set are used as the input information data of the initial aircraft airport taxiing scheduling model to realize intelligent conflict detection. The initial aircraft airport taxiing scheduling model introduces a long short-term memory network for processing, and the obtained airport environment and aircraft status information of unified dimensions are transmitted to the input end of the neural network in the deep reinforcement learning algorithm.

[0103] Further, step 205 may include the following sub-steps:

[0104] S51, based on the preset danger zone screening conditions, according to the target simulation operation data and the preset actual operation data, determining multiple target position relationships;

[0105] Further, S51 may include the following sub-steps:

[0106] S511, determining multiple simulated aircraft position relationships and multiple actual aircraft position relationships according to multiple simulation environment information in the target simulation operation data and multiple actual environment information in the preset actual operation data;

[0107] S512: Based on the preset danger zone screening conditions, the position relationships of the simulated aircraft and the position relationships of the actual aircraft are screened to determine a plurality of target position relationships.

[0108] The multiple target position relationships include multiple target simulated aircraft position relationships and multiple target actual aircraft position relationships.

[0109] It should be noted that on the ground, the wingspan determines the space occupied by the aircraft on the airport taxiway and apron. Therefore, the safety interval between aircraft is set based on the wingspan of the aircraft. A larger span means that the aircraft needs more space to avoid collision and ensure safe operation. Here, the wingspan of the aircraft is uniformly set to 40 meters. In order to simplify the design, a circular range is used to set the airport aircraft safety range, which is generally 1 to 2 times the wingspan of the aircraft. After obtaining the positional relationship between the aircraft, the area around the aircraft is divided into three areas according to the collision risk level from low to high in the encounter situation between the aircraft: "no collision risk area", "possible collision risk area" and "serious collision risk area".

[0110] Furthermore, the local observation field of view of the aircraft is to calculate the positional relationship between other aircraft and the aircraft in the simulation environment, without considering other aircraft in the "no collision danger zone" of the aircraft, and only retain all aircraft in the "possible collision danger zone" and "serious collision danger zone", that is, the simulated aircraft position relationship and the actual aircraft position relationship in the "possible collision danger zone" and "serious collision danger zone" are retained, and used as the corresponding target simulated aircraft position relationship and target actual aircraft position relationship for subsequent model training.

[0111] S52, using a preset multi-agent reinforcement learning algorithm to calculate the simulation importance value corresponding to each simulation environment information and the actual importance value corresponding to each actual environment information according to the aircraft taxiing motion data set, multiple simulation environment information in the target simulation operation data, multiple actual environment information in the preset actual operation data and the position relationship of each target;

[0112] S53, comparing each simulation importance level value, each actual importance level value and a preset level threshold;

[0113] S54, taking any simulation environment information corresponding to the simulation importance value greater than the preset degree threshold and the actual environment information corresponding to the actual importance value as the target environment information;

[0114] The target environment information includes target simulation environment information and target actual environment information.

[0115] The preset multi-agent reinforcement learning algorithm is the MAAC algorithm (Multi-Agent Actor-Critic).

[0116] It should be noted that the importance of multiple aircraft is calculated through the multi-head attention mechanism in the MAAC algorithm according to the aircraft taxiing motion data set, multiple simulation environment information in the target simulation operation data, multiple actual environment information in the preset actual operation data and the position relationship of each target, that is, the simulation importance value corresponding to each simulation environment information and the actual importance value corresponding to each actual environment information, and the simulation importance value, the actual importance value and the preset degree threshold are compared to select the state information of aircraft with higher importance as the input of the strategy network, that is, any simulation environment information corresponding to a simulation importance value greater than the preset degree threshold is used as the target simulation environment information, and any actual environment information corresponding to an actual importance value greater than the preset degree threshold is used as the target actual environment information; each aircraft obtains a distributed strategy according to the current environment and its own state.

[0117] S55, sampling the aircraft taxiing motion data set to determine a plurality of sampled aircraft taxiing motion data;

[0118] S56, using the target environment information and the multiple sampled aircraft taxiing motion data as inputs of the initial strategy network in the initial aircraft airport taxiing scheduling model, and outputting multiple aircraft taxiing distributed strategies to be trained;

[0119] S57, using the taxiing distributed strategies of each aircraft to be trained, multiple simulation environment information in the target simulation operation data, and multiple actual environment information in the preset actual operation data to train the initial aircraft airport taxiing scheduling model, and determine the target aircraft airport taxiing scheduling model.

[0120] It should be noted that all aircraft transfer to the next state according to the distributed strategy and obtain immediate rewards, that is, the reward function value. At this time, a transfer sample is formed, which includes the current state, the action taken, the immediate reward and the new state, and the action is continuously selected according to the current environment and its own state.

[0121] Further, S57 may include the following sub-steps:

[0122] S571, interacting with each taxiing distributed strategy of the aircraft to be trained, multiple simulation environment information in the target simulation operation data, and multiple actual environment information in the preset actual operation data, to generate an interaction result, multiple new simulation environment information, and multiple new actual environment information;

[0123] S572, calculating the reward function value according to the interaction result;

[0124] The interaction results include the number s of aircraft that cannot pass through completely, the distances d between multiple aircraft and other aircraft, that is, the distance d between each aircraft and other aircraft, the maximum number p of aircraft passing through, and the number q of invalidly released aircraft.

[0125] It should be noted that the reward function is constructed using the deep reinforcement learning algorithm, and the reward function is used to calculate the reward function value according to the interaction result. The obtained aircraft taxiing distributed strategy can formulate corresponding conflict resolution rules for different conflict types, guide the aircraft's taxiing speed and heading, realize intelligent detection and resolution of taxiing conflicts, and obtain the model gradient corresponding to the data with the best taxiing action quality. The model gradient is used to update the network parameters in the initial aircraft airport taxiing scheduling model, and the updated initial aircraft airport taxiing scheduling model is used for the next iterative operation until the number of simulation iterations reaches the preset iteration threshold. The distributed strategy learning of aircraft taxiing is completed, and the target intelligent conflict detection and resolution model, that is, the target aircraft airport taxiing scheduling model, is obtained.

[0126] For further information, see Figure 3 In the reinforcement learning process, it is divided into four parts: problem definition, data preprocessing, network construction, and experimental verification. The data preprocessing process is to generate the target simulation operation data based on the acquired aircraft airport data and vehicle topology data to be trained. The experimental verification process is to use the target simulation operation data, aircraft airport data to be trained, and vehicle topology data to be trained to build an airport aircraft taxiing model, and train the model based on the data generated by the airport aircraft taxiing model. In the process of interaction between the agent and the environment, the agent aircraft is guided to learn the optimal taxiing action and path through a pre-set reward function, so as to drive the agent to select the behavior strategy corresponding to the maximum learning reward value in continuous trial and error. In view of the problem that sparse rewards will make it difficult for the agent to quickly learn any useful action control strategy, or even fail to converge, a guided aircraft taxiing reward function is designed.

[0127] Furthermore, the process of calculating the reward function value according to the interaction result is specifically as follows: a reward function is constructed using a deep reinforcement learning algorithm, which includes the safety, efficiency and fault tolerance of the aircraft during taxiing; wherein safety is the most important consideration for aircraft operation, and the collision probability and distance between aircraft are used as safety assessment factors; the safety reward function is expressed as follows: when the distance between aircraft is less than the safety distance threshold of 50m, the safety reward value is -1; when the distance between aircraft is less than the above safety distance threshold of 50m, it is considered that there is a conflict between aircraft, and this round of experiment is considered a failure; otherwise, based on the number of aircraft that cannot all pass through and the distances between multiple aircraft and other aircraft, multiple safety reward values ​​corresponding to each aircraft are calculated, and then all safety reward values ​​corresponding to each aircraft are summed to obtain the target safety reward value corresponding to each aircraft, wherein the calculation process of the safety reward value can be expressed as follows , where η is the collision probability, which can be measured through random experiments, d is the distance between the aircraft and other aircraft, s is the number of aircraft that cannot pass through, and r a is the safety bonus value of this aircraft, is the safety weight factor.

[0128] For example, suppose we know the distance between plane A and the other three planes, and the distance between plane B and the other three planes. We substitute the distance between plane A and the other three planes and the number of aircraft that cannot pass through into In the calculation formula, the three safety bonus values ​​corresponding to aircraft A are obtained, and then the three safety bonus values ​​corresponding to aircraft A are summed to obtain the target safety bonus value corresponding to aircraft A; similarly, the target safety bonus value corresponding to aircraft B can be obtained.

[0129] Furthermore, the efficiency is evaluated by the number of aircraft passing through. When aircraft that are about to arrive at the intersection do not need to wait and can leave immediately upon arrival, the operation efficiency is the highest. However, when multiple aircraft appear on different taxiways, some aircraft must wait before the intersection. The number of aircraft passing through is used as the evaluation factor of efficiency. Specifically, the operation efficiency bonus value is calculated based on the maximum number of aircraft passing through, p. The calculation process of the operation efficiency bonus value can be expressed as , p is the maximum number of aircraft passing, ω is the efficiency discount coefficient, Φ j is the weight coefficient of efficiency, r y As a bonus value for operating efficiency.

[0130] Furthermore, fault tolerance refers to when there is no aircraft entering a taxiway, but the instruction issued by the gate is to go, that is, the issued instruction does not match the actual situation, which is an invalid release, so the agent is punished; the reward value will be multiplied by the discount coefficient μ on the original basis. Specifically, the fault tolerance reward value is calculated based on the number of invalidly released aircraft q. The calculation process of the fault tolerance reward value is: , q is the number of invalidly released aircraft, is the weight coefficient of fault tolerance, r r is the fault tolerance reward value.

[0131] Furthermore, based on the above three factors, the target safety reward value, operating efficiency reward value and fault tolerance reward value corresponding to each aircraft are added to obtain the reward sum value corresponding to each aircraft, and then the reward sum values ​​are summed to obtain the reward function value; for example, assuming that the target safety reward value corresponding to aircraft A and the target safety reward value corresponding to aircraft B are obtained, the target safety reward value corresponding to aircraft A is added to the operating efficiency reward value and the fault tolerance reward value to obtain the reward sum value corresponding to aircraft A, the target safety reward value corresponding to aircraft B is added to the operating efficiency reward value and the fault tolerance reward value to obtain the corresponding reward sum value corresponding to aircraft B, and then the reward sum value corresponding to aircraft A and the corresponding reward sum value corresponding to aircraft B are summed to obtain the reward function value.

[0132] It is worth mentioning that the aircraft taxiing distributed strategy output by the aircraft airport taxiing scheduling model is a conflict resolution rule, which includes different measures for different conflict situations; for intersection conflicts, when conflicts occur at intersections, two methods are usually adopted to resolve them: one is to change the taxiing route; the other is to stop and wait in front of the intersection; Changing the taxiing route: When the controller finds that there are multiple aircraft passing near an intersection and there is a possibility of a collision at the intersection, he or she will command one or more aircraft to change the taxiing route in advance to avoid a collision; For stopping and waiting: When multiple aircraft are taxiing near an intersection, one or more aircraft with lower priority stop taxiing and wait at the intersection, and aircraft with higher priority pass first. Priority is related to many factors, such as aircraft type, mission importance, etc. For head-on conflicts, when two aircraft pass the same taxiway from different directions, they need to predict their respective taxiing times within the same time interval. If there is a possibility of conflict, the aircraft with lower priority should change its taxiing direction or stop to avoid aircraft collision and conflict. For overtaking conflicts, the taxiing speed of the rear aircraft is reduced to keep a safe distance between the front and rear aircraft. This method can reduce the workload of the controller and avoid changing the taxiing path as much as possible to avoid conflicts. Among them, the designed conflict resolution algorithm is added to the strategy selection to judge the conflict situation between the two aircraft, and the next action of the two aircraft is output according to the corresponding situation, so that the aircraft can take corresponding measures to coordinate and avoid collision.

[0133] Furthermore, if the two aircraft do not fall into any of the three conflict situations, the aircraft will take the next action output by the Actor network (aircraft airport taxiing scheduling model), and the Critic network will give corresponding evaluations based on the aircraft status and actions, and iterate continuously until the target point is reached, that is, the corresponding reward value is calculated based on the output interaction results.

[0134] S573, scoring each taxiing distributed strategy of the to-be-trained aircraft and the probability distribution of each taxiing distributed strategy of the to-be-trained aircraft respectively through the initial value function network, and outputting a first scoring result and a second scoring result;

[0135] S573, using the target replica network to score the taxiing distributed strategies of each aircraft to be trained, and outputting a third scoring result;

[0136] S575, substituting the reward function value, the first scoring result, the second scoring result and the third scoring result into the preset loss function and deriving them to determine the model gradient;

[0137] The preset loss function includes a first loss function and a second loss function.

[0138] The model gradient includes a first model gradient and a second model gradient.

[0139] The target aircraft airport taxiing scheduling model includes a target strategy network, a target value function network and a target replica network.

[0140] It should be noted that the first loss function is the Critic's loss function, which is constructed based on the mean square error (MSE), which can be expressed as:

[0141] ;

[0142] in, is the first loss value; For all possible target environment information (Current status) and the distributed taxiing strategy of the aircraft to be trained Calculation of the expectation of the joint distribution (of actions); is the first scoring result, which indicates the target environment information of the Critic network at time t The distributed taxiing strategy of the aircraft to be trained output under (current state) Q-value estimation of (action); is the target Q value (target estimate); is the target environment information at time t; is the taxiing distributed strategy of the aircraft to be trained at time t; is the reward function value; is the discount factor, the value is 0.9; The probability distribution of the taxiing distributed strategy of the training aircraft output by the initial strategy network according to the target environment information; is the third scoring result, which indicates the target environment information of the initial replica network at time t The distributed taxiing strategy of the aircraft to be trained output under (current state) Q-value estimate for (action).

[0143] Furthermore, the second loss function is the loss function of the Actor network. The main goal of the Actor is to optimize the strategy so that it can maximize the Q value estimated by the Critic; it can be expressed as:

[0144] ;

[0145] in, is the second loss value; For all possible target environment information Expected calculation of (current state) (expected value); is the second scoring result, which indicates the target environment information of the Critic network at time t The distributed taxiing strategy of the aircraft to be trained output under (current state) Q-value estimate of the probability distribution of (actions); Distributed taxiing strategy for aircraft to be trained The probability distribution of (actions).

[0146] It is worth mentioning that the initial value function network is used to output the first scoring result of the taxiing distributed strategy of each aircraft to be trained, and the probability distribution of the taxiing distributed strategy of each aircraft to be trained is scored through the initial value function network, and the second scoring result is output; the probability distribution of the taxiing distributed strategy of each aircraft to be trained is scored through the target replica network, and the third scoring result is output; the reward function value, the first scoring result, the second scoring result, and the third scoring result are substituted into the corresponding loss function for differentiation to calculate the gradient of the loss function relative to the model parameters (model gradient), that is, the first scoring result, the third scoring result and the reward function value are substituted into the first loss function and differentiated to determine the first model gradient, and the second scoring result is substituted into the second loss function and differentiated to determine the second model gradient.

[0147] Furthermore, in the MAAC algorithm, the Critic's loss function is used to optimize the Q-value estimation by minimizing the mean square error of the Q-value. The Actor's loss function is based on the policy gradient, which optimizes the policy by maximizing the Critic's Q-value. The target network has a small direct effect in the test phase, and it is mainly used for stability and reliability during the training process.

[0148] S576. Based on the preset optimizer, the model parameters of the initial policy network, the model parameters of the initial value function network, and the model parameters of the initial replica network are updated using the model gradient, the intermediate policy network, the intermediate value function network, and the intermediate replica network are determined, and the number of simulation iterations is counted in real time;

[0149] It should be noted that the model parameters of the initial replica network and the model parameters of the initial value function network are updated through the first model gradient, and the model parameters of the initial policy network are updated through the second model gradient to determine the intermediate policy network, the intermediate value function network and the intermediate replica network.

[0150] S577, determining whether the number of simulation iterations reaches a preset iteration threshold;

[0151] S578. If the number of simulation iterations reaches a preset iteration threshold, the intermediate policy network is used as the target policy network, the intermediate value function network is used as the target value function network, and the intermediate replica network is used as the target replica network.

[0152] It should be noted that, when judging whether the number of steps reaches the minimum batch number of steps (the minimum number of steps is set to 100), that is, judging whether the number of simulation iterations reaches the preset iteration threshold, if the number of simulation iterations does not reach the preset iteration threshold, the intermediate aircraft airport taxi scheduling model is used as the new initial aircraft airport taxi scheduling model, and the execution of S51 is jumped until the number of simulation iterations reaches the preset iteration threshold, and the intermediate policy network determined when the number of simulation iterations reaches the preset iteration threshold is used as the target policy network, the intermediate value function network determined when the number of simulation iterations reaches the preset iteration threshold is used as the target value function network, and the intermediate copy network determined when the number of simulation iterations reaches the preset iteration threshold is used as the target copy network.

[0153] It is worth mentioning that the empirical sample priority extraction method is used to extract samples as data for training the actor network and the critic network; the gradient of the real model is calculated, and the loss function of the real Q network in MAAC is reset; based on the adaptive moment estimation (Adam) optimizer, the gradient is used to update the parameters of the entire network, and the critic network uses the adaptive moment estimation (Adam) optimizer method to update the parameters of the real network; the gradient of the real policy network is calculated, and the loss function of the real policy network in MAAC is redesigned; the actor network uses the adaptive moment estimation optimizer method to update the real policy and the target network. The real-time state data is input into the target operation state prediction model for predictive calculation to obtain the prediction results of the target intelligent detection conflict and release model, that is, the aircraft taxiing conflict release strategy is integrated to obtain the aircraft path. After training, a trained Actor network and Critic network are finally obtained, and then the aircraft departs from the starting point and inputs the current state into the Actor network to output the next action; among them, please refer to Figure 4, updating the network parameters in the deep reinforcement learning algorithm includes: introducing a priority experience replay mechanism based on a binary tree structure model in the deep reinforcement learning algorithm. SumTree adopts a binary tree structure. The data stored in the nodes of each layer is the sample priority. The value is positively correlated with the node priority. The utilization efficiency of high-value samples can be improved through the priority experience replay mechanism based on SumTree. The priority of each node in the SumTree structure is determined by the temporal difference error—TD-error. The difference is the difference between the target network value function and the training network function when the deep reinforcement learning algorithm is updated. The larger the value, the greater the prediction accuracy of the training network. There is still a lot of room for improvement, and the greater the value of being trained, so it is set with a higher priority. The value below the leaf node represents the numerical interval of the sample. The larger the numerical interval, the higher the probability of being selected during the sampling process.

[0154] For example, in order to verify the feasibility of the proposed scheme, real-time airport data, including flight schedules, aircraft positions, taxi routes, taxi conflict information, etc., are collected, and a real airport is used as a simulation scene for modeling; the algorithm gives an instant reward to the aircraft that takes a certain action through a reward function; if the aircraft collides with other aircraft or static obstacles after taking an action, a negative reward is obtained; the sum of the rewards of all aircraft actions in the simulation environment is the current overall reward sum; the algorithm learns through the continuous rewards and summarizes the impact of actions on the state, thereby guiding the aircraft to take corresponding actions respectively, successfully avoid static obstacles and other aircraft, and reach the final target point; the real airport is used as a simulation scene for modeling, and the real scene of a multi-runway airport such as Guangzhou Baiyun International Airport is selected, where the types of airport aircraft are relatively complex and the field operation restrictions are many, which can more comprehensively reflect the occurrence of field operation problems; when building the simulation environment, letters are used to represent the taxiway position and numbers are used to represent the parking position. In order to simplify the operation mode of the taxiway, the aircraft always maintains the same direction when using the same taxiway, and the aircraft on the main taxiway run from east to west.

[0155] Furthermore, for scenarios such as intersection conflicts, the intersection of the three taxiways A, C1, and C2 can be set to conflict, and the conflict scenario simulation can be performed by setting aircraft with different running trajectories; aircraft on taxiway A taxi from east to west, collectively referred to as Class A aircraft; aircraft on taxiway C1 taxi to point Z, then taxi due west, which are Class B aircraft; aircraft on taxiway C2 are Class C aircraft, and after taxiing to point Z, their trajectory is the same as that of Class B aircraft; Class A aircraft are straight-moving aircraft, and their speed is controlled according to the stop-and-wait decision. When they need to stop and wait, the aircraft decelerates to 0, and after waiting for other aircraft at the intersection to pass, the Class A aircraft accelerates to the maximum speed and quickly passes through the intersection to avoid occupying the taxiway for a long time; Class B aircraft are turning aircraft, and their speed is controlled according to the stop-and-wait decision. When it is necessary to stop and wait, the aircraft decelerates to 0, and then after other aircraft at the intersection pass, the aircraft at exit B accelerates to the turning speed and passes through the intersection at a constant speed. After exiting the intersection, the speed is increased to the maximum speed; Class C aircraft and Class B aircraft are turning aircraft, and the turning direction is opposite; the performance of the algorithm proposed in the present invention is evaluated by the average release success rate and the average taxiing time, including: the calculation formula of the current conflict release success rate of the intelligent agent is the number of current successful release rounds / the number of rounds currently trained by the intelligent agent, and the average taxiing time is the time from the start of the maneuver to the conflict being released.

[0156] In this embodiment, the initial aircraft airport taxiing scheduling model is distributedly trained according to the aircraft taxiing motion data set, the target simulation operation data and the preset actual operation data to determine the target aircraft airport taxiing scheduling model.

[0157] Step 206: Acquire the airport data of the aircraft to be scheduled and the topological structure data of the vehicles to be scheduled, and determine the digital twin model of the target to be scheduled based on the airport data of the aircraft to be scheduled and the topological structure data of the vehicles to be scheduled.

[0158] Airport data of aircraft to be dispatched and topological structure data of vehicles to be dispatched.

[0159] The airport data of the aircraft to be scheduled are the topological structure data of the aircraft to be scheduled and the basic environment data of the airport; the topological structure data of the vehicle to be scheduled are the topological structure data of the vehicle to be scheduled.

[0160] In this embodiment, the airport data of the aircraft to be scheduled and the topological structure data of the vehicles to be scheduled are obtained, and the digital twin model of the target to be scheduled is determined based on the airport data of the aircraft to be scheduled and the topological structure data of the vehicles to be scheduled.

[0161] Step 207: Perform simulation calculations on the digital twin model of the target to be scheduled to generate simulation operation data of the target to be scheduled.

[0162] In this embodiment, a simulation operation is performed on the digital twin model of the target to be scheduled to generate simulation operation data of the target to be scheduled.

[0163] Step 208: construct a taxiing model of the aircraft at the airport to be scheduled by using the simulation operation data of the target to be scheduled, the data of the aircraft airport to be scheduled and the topological structure data of the vehicle to be scheduled.

[0164] In this embodiment, the taxiing model of the aircraft at the airport to be scheduled is constructed by using the simulation operation data of the target to be scheduled, the airport data of the aircraft to be scheduled and the topological structure data of the vehicle to be scheduled.

[0165] Step 209: Simulate the taxiing model of the aircraft at the airport to be scheduled, and output a taxiing motion data set of the aircraft to be scheduled.

[0166] In this embodiment, a simulation operation is performed on the taxiing model of the aircraft at the airport to be scheduled, and a taxiing motion data set of the aircraft to be scheduled is output.

[0167] Step 210: Based on the preset danger zone screening conditions and the preset multi-agent reinforcement learning algorithm, the environmental information of the target to be scheduled is determined according to the simulation operation data of the target to be scheduled, the preset actual operation data and the taxiing motion data set of the aircraft to be scheduled.

[0168] In this embodiment, based on preset danger zone screening conditions and preset multi-agent reinforcement learning algorithms, the environmental information of the target to be scheduled is determined according to the simulation operation data of the target to be scheduled, the preset actual operation data and the taxiing motion data set of the aircraft to be scheduled.

[0169] Step 211: Use the target environment information and the taxiing motion data set of the aircraft to be scheduled as inputs to the target strategy network in the airport taxiing scheduling model of the target aircraft, and output multiple distributed taxiing strategies of the target aircraft.

[0170] It is worth mentioning that in the actual application process, the taxiing motion data set of the aircraft to be scheduled does not need to be sampled. The data set is directly used as the input of the target strategy network in the taxiing scheduling model of the target aircraft airport, and the target aircraft taxiing distributed strategy is output, so that the aircraft can execute the target aircraft taxiing distributed strategy to taxi.

[0171] In this embodiment, the target environment information and the taxiing motion data set of the aircraft to be scheduled are used as inputs of the target strategy network in the airport taxiing scheduling model of the target aircraft, and multiple target aircraft taxiing distributed strategies are output.

[0172] As a comparison of technical effects, it can be combined with existing technologies for reference. Most previous studies focus on the surface conflicts of large airports, ignoring the lack of surface resources and management difficulties of small and medium-sized airports. Therefore, the surface management of small and medium-sized airports should be studied in depth. In order to avoid conflicts, the method of changing the taxiing path is usually adopted to reduce the time cost, but this method does not take into account the difficulty of frequent turns of aircraft. For large passenger aircraft, aircraft turning operations are difficult. According to the actual operation situation, aircraft turning should be avoided as much as possible, and speed control is more practical. Most literatures only study a single intersection when studying conflict resolution, and after a conflict resolution, it is considered that the conflict resolution is successful. However, in the actual operation process, the aircraft may still have conflicts during subsequent taxiing, so it is necessary to consider global conflict resolution. At the same time, in the airport environment, aircraft will take off, land, taxi and other activities, and their positions and states will change. The movement of aircraft may lead to adjustments to ground traffic and recalculation of path planning. At the same time, the airport is not just the territory of aircraft. There are many other types of vehicles that may move and dispatch in the airport area, such as commuter vehicles, transport vehicles, tractors, etc. The presence of these vehicles not only affects the route selection of aircraft during taxiing and takeoff, but may also cause potential conflicts and delays. However, existing research rarely involves this aspect, which means that there may be potential blind spots in airport management.

[0173] In view of the above problems, the present invention provides an aircraft airport taxiing scheduling model training method, which builds a digital twin model based on the topological structure data of aircraft and vehicles and the basic environment data of the airport, and conducts distributed strategy learning on the actual data of the physical space and the virtual data generated by the target digital twin model. The reward function is constructed using a deep reinforcement learning algorithm, and corresponding conflict resolution rules are formulated for different conflict types to guide the taxiing speed and heading of the aircraft. The network parameters in the deep reinforcement learning algorithm are updated using the data with the best taxiing action quality, and the updated deep reinforcement learning algorithm is used for the next iterative operation to form a target intelligent detection conflict and release model, which effectively reduces the potential risks in the taxiing process, minimizes the taxiing time of airport aircraft, reduces the average waiting time of aircraft, and improves the airport operation efficiency; by analyzing and integrating aircraft taxiing data and vehicle driving data to intelligently detect and resolve possible conflicts in the taxiing process, the problem that the existing technology is difficult to achieve real-time optimization of paths and avoid conflicts according to environmental changes is solved; specifically, an environmental map is constructed and parameters are set. The environmental map is constructed according to the actual airport, and the parameters of the aircraft are set, that is, the airport aircraft taxiing model is constructed using the target simulation operation data, the airport data of the aircraft to be trained, and the topological structure data of the vehicle to be trained; the taxiing conflict intelligent detection and release model is trained for all aircraft, the environmental parameters are initialized, and the initial state of all aircraft is obtained; the position relationship between other aircraft and this aircraft is calculated, and the aircraft outside the "possible collision danger zone" and "serious collision danger zone" are filtered out. After other aircraft, the communication information processing module calculates the information interaction between aircraft; each aircraft generates a distributed strategy according to the strategy network, and obtains the new state and reward of each aircraft after interacting with the environment, that is, the taxiing distributed strategy of each aircraft to be trained, multiple simulation environment information in the target simulation operation data and multiple actual environment information in the preset actual operation data are interacted to generate interaction results, multiple new simulation environment information and multiple new actual environment information; the reward function value is calculated according to the interaction result; after a certain number of rounds, the strategy loss of the current aircraft in the past 100 steps is updated; according to the experience sample priority extraction method, a small batch of sample experience is extracted; the loss of the critic network is calculated, and the parameters of the critic real network are updated; the loss of the actor network is calculated, and the parameters of the actor real network are updated; the target network parameters are updated, and the final path and speed of each aircraft at each step are output; if the current position is not the target point state of the aircraft, the current position and heading angle of the current aircraft are output; the encounter situation between adjacent aircraft is judged, and when it belongs to the three conflict situations, actions are taken according to the algorithm conflict resolution strategy, and transferred to the next state.

[0174] In summary, the present invention can perform real-time status updates on the digital twin model to ensure that it can accurately reflect the current state of the physical space. This model continuously monitors and analyzes the state of the physical space and can make accurate state predictions. The state prediction data generated by the digital twin space is imported into the trained target intelligent detection conflict and release model (target aircraft airport taxiing scheduling model), so as to achieve effective strategy planning for aircraft taxiing and vehicle operation. The complexity and danger of the strategy adopted are evaluated by indicators to indicate the safety level of the current physical space state and issue early warnings in a timely manner. On this basis, necessary adjustments are made to the strategy planning for aircraft taxiing and vehicle operation generated by the current physical space to ensure the safety and efficiency of the entire operation process. This comprehensive and dynamic strategy adjustment mechanism helps to cope with complex and changing airport environments and minimize potential safety risks.

[0175] Furthermore, the present invention imports the state data generated by the digital twin model into the trained target intelligent detection conflict and release model to effectively plan the strategy for aircraft taxiing and vehicle operation. Then the state in the digital twin model is disturbed, and simulation calculation is performed therein to obtain virtual state data after disturbance. This process can not only evaluate the anti-disturbance degree of the current physical space strategy, but also identify the area where the disturbance causes the change degree or danger degree to exceed the threshold. By issuing prompts in time, possible risk areas can be effectively dealt with, thereby ensuring the safety and stability of the overall operation. The digital twin model continuously monitors and analyzes the physical space state and can make accurate state predictions. The state prediction data generated by the digital twin space is imported into the trained target intelligent detection conflict and release model, thereby realizing effective strategy planning for aircraft taxiing and vehicle operation. The complexity and danger degree of the adopted strategy are evaluated by indicators, indicating the safety degree of the current physical space state, and issuing warnings in time. On this basis, necessary adjustments are made to the strategic planning of aircraft taxiing and vehicle operation generated in the current physical space to ensure the safety and efficiency of the entire operation process; the digital twin model is updated in real time through the taxiing conflict intelligent detection and release algorithm, and its state data is imported into the trained target intelligent detection conflict and release model to effectively plan the strategy for aircraft taxiing and vehicle operation. Then the state in the digital twin model is disturbed, and simulation calculations are performed in it to obtain virtual state data after disturbance. This process can not only evaluate the anti-disturbance degree of the current physical space strategy, but also identify areas where the degree of change or danger caused by disturbance exceeds the threshold. By issuing prompts in a timely manner, possible risk areas can be effectively dealt with, thereby ensuring the safety and stability of the overall operation.

[0176] Furthermore, an environmental map is constructed according to the actual airport, and the parameters of the aircraft are set. The path planning model is trained for all aircraft, and the environmental parameters are initialized to obtain the initial state of all aircraft. According to the actual airport, the environmental map is constructed, and the real-time data of the airport is first collected, including flight plans, aircraft positions, taxi routes, taxi conflict information, etc., and the real airport is used as the simulation scene for modeling. You can select the real scene of an airport with multiple runways, such as Guangzhou Baiyun International Airport, where the types of airport aircraft are more complex and there are many restrictions on the field operation, which can more comprehensively reflect the occurrence of field operation problems. When building the simulation environment, letters are used to represent the taxiway position and numbers are used to represent the parking position. In order to simplify the operation mode of the taxiway, the aircraft always maintains the same direction when using the same taxiway, and the aircraft on the main taxiway always run from east to west.

[0177] It can be concluded that the present invention constructs a digital twin model based on the topological structure data of aircraft and vehicles and the basic environment data of the airport, comprehensively considers the aircraft taxiing data and vehicle driving data, imports the operation data and vehicle operation data generated by the aircraft in the historical taxiing process into the digital twin model, and trains the initialized digital twin model according to the obtained simulation operation data and historical operation data; the actual data of the physical space and the virtual data generated by the obtained target digital twin model are used for distributed strategy learning using a multi-agent deep reinforcement learning algorithm based on a centralized training distributed execution framework; the state space is constructed using a deep reinforcement learning algorithm, and the collision risk in the aircraft encounter situation is ranked from low to high by the position between the aircraft, and the area around the aircraft is divided into three areas, and only all aircraft in the "possible collision risk area" and the "serious collision risk area" are retained and input into the communication information processing module, and after detecting the aircraft taxiing conflict, corresponding conflict resolution rules are formulated for different conflict types to guide the aircraft's taxiing speed and heading, and realize intelligent detection and resolution of taxiing conflicts; and then the target taxiing conflict intelligent detection and resolution model (target aircraft airport taxiing scheduling model) is obtained through training,

[0178] Furthermore, the present invention enhances the learning efficiency of the algorithm by adding a priority experience playback mechanism to the deep reinforcement learning algorithm mechanism to enhance the utilization of effective experience; the network parameters in the deep reinforcement learning algorithm are updated using the data with the best taxiing action quality, and the updated deep reinforcement learning algorithm is used for the next iterative operation until the strategy evaluation function of the taxiing action converges to the optimal, completing the strategy learning of the aircraft taxiing and forming a target intelligent detection conflict and release model. The present invention improves the exploratory nature and learning efficiency of the algorithm, effectively reduces the potential risks in the taxiing process, minimizes the taxiing time of airport aircraft, reduces the average waiting time of aircraft, and improves the airport operation efficiency.

[0179] In an embodiment of the present invention, the present invention provides an aircraft airport taxiing scheduling model training method, firstly, obtaining aircraft airport data to be trained and vehicle topology data to be trained, and determining a target digital twin model according to the aircraft airport data to be trained and the vehicle topology data to be trained; then, performing simulation operation on the target digital twin model to generate target simulation operation data; constructing an airport aircraft taxiing model by using the target simulation operation data, the aircraft airport data to be trained and the vehicle topology data to be trained; performing simulation operation on the airport aircraft taxiing model to output an aircraft taxiing motion data set; performing distributed training on an initial aircraft airport taxiing scheduling model according to the aircraft taxiing motion data set, the target simulation operation data and the preset actual operation data to determine a target aircraft airport taxiing scheduling model; based on the above scheme, obtaining aircraft airport data to be trained and vehicle topology data to be trained, determining a target digital twin model, and then using the actual operation data of the physical space, the simulation operation data generated by the target digital twin model and the generated aircraft taxiing motion data set to perform distributed strategy learning on the initial aircraft airport taxiing scheduling model, the determined target aircraft airport taxiing scheduling model can effectively deal with possible risk areas, and further improve the safety of the overall operation of the airport.

[0180] See also Figure 5 , Figure 5 A structural block diagram of an aircraft airport taxiing scheduling model training system provided by an embodiment of the present invention

[0181] The present invention provides an aircraft airport taxiing scheduling model training system, comprising:

[0182] An acquisition module 501 is used to acquire the aircraft airport data to be trained and the vehicle topology structure data to be trained, and determine the target digital twin model according to the aircraft airport data to be trained and the vehicle topology structure data to be trained;

[0183] The first operation module 502 is used to perform simulation operation on the target digital twin model to generate target simulation operation data;

[0184] A construction module 503 is used to construct an airport aircraft taxiing model using target simulation operation data, aircraft airport data to be trained, and vehicle topology data to be trained;

[0185] The second operation module 504 is used to perform simulation operation on the airport aircraft taxiing model and output the aircraft taxiing motion data set;

[0186] The training module 505 is used to perform model training on the initial aircraft airport taxiing scheduling model according to the aircraft taxiing motion data set, the target simulation operation data and the preset actual operation data, and determine the target aircraft airport taxiing scheduling model.

[0187] Furthermore, the acquisition module 501 is specifically used for:

[0188] The initial digital twin model is constructed using the aircraft airport data to be trained and the vehicle topology data to be trained;

[0189] Perform simulation operations on the initial digital twin model to generate initial simulation operation data;

[0190] The initial digital twin model is trained using preset actual operation data and initial simulation operation data to determine the target digital twin model.

[0191] Furthermore, the training module 505 includes:

[0192] The first submodule is used to determine the position relationship of multiple targets based on the preset danger zone screening conditions, according to the target simulation operation data and the preset actual operation data;

[0193] The second submodule is used to calculate the simulation importance value corresponding to each simulation environment information and the actual importance value corresponding to each actual environment information according to the aircraft taxiing motion data set, multiple simulation environment information in the target simulation operation data, multiple actual environment information in the preset actual operation data and the position relationship of each target by using a preset multi-agent reinforcement learning algorithm;

[0194] The third submodule is used to compare each simulated importance degree value, each actual importance degree value and a preset degree threshold;

[0195] A fourth submodule is used to use any simulation environment information corresponding to a simulation importance value greater than a preset degree threshold and actual environment information corresponding to an actual importance value as target environment information;

[0196] A fifth submodule is used to sample the aircraft taxiing motion data set to determine a plurality of sampled aircraft taxiing motion data;

[0197] The sixth submodule is used to use the target environment information and a plurality of sampled aircraft taxiing motion data as inputs of an initial strategy network in an initial aircraft airport taxiing scheduling model, and output a plurality of aircraft taxiing distributed strategies to be trained;

[0198] The seventh submodule is used to train the initial aircraft airport taxiing scheduling model by using the taxiing distributed strategies of each aircraft to be trained, multiple simulation environment information in the target simulation operation data and multiple actual environment information in the preset actual operation data, and determine the target aircraft airport taxiing scheduling model.

[0199] Furthermore, the first submodule is specifically used for:

[0200] Determining a plurality of simulated aircraft position relationships and a plurality of actual aircraft position relationships according to a plurality of simulation environment information in the target simulation operation data and a plurality of actual environment information in the preset actual operation data;

[0201] Based on the preset danger zone screening conditions, the position relationships of each simulated aircraft and each actual aircraft are screened to determine multiple target position relationships.

[0202] Furthermore, the initial aircraft airport taxiing scheduling model also includes an initial value function network and an initial replica network; the target aircraft airport taxiing scheduling model includes a target strategy network, a target value function network and a target replica network; the seventh submodule is specifically used for:

[0203] Interacting the taxiing distributed strategies of each aircraft to be trained, multiple simulation environment information in the target simulation operation data, and multiple actual environment information in the preset actual operation data, to generate interaction results, multiple new simulation environment information, and multiple new actual environment information;

[0204] Calculate the reward function value based on the interaction results;

[0205] Scoring the taxiing distributed strategies of each aircraft to be trained and the probability distribution of each taxiing distributed strategy of each aircraft to be trained respectively through the initial value function network, and outputting a first scoring result and a second scoring result;

[0206] The target replica network is used to score the taxiing distributed strategies of each aircraft to be trained, and the third scoring result is output;

[0207] Substitute the reward function value, the first scoring result, the second scoring result and the third scoring result into the preset loss function and derive it to determine the model gradient;

[0208] Based on the preset optimizer, the model parameters of the initial policy network, the model parameters of the initial value function network and the model parameters of the initial replica network are updated using the model gradient, the intermediate policy network, the intermediate value function network and the intermediate replica network are determined, and the number of simulation iterations is counted in real time;

[0209] Determine whether the number of simulation iterations reaches a preset iteration threshold;

[0210] If the number of simulation iterations reaches a preset iteration threshold, the intermediate policy network is used as the target policy network, the intermediate value function network is used as the target value function network, and the intermediate replica network is used as the target replica network.

[0211] In an optional embodiment, the system further includes:

[0212] Obtain the airport data of the aircraft to be scheduled and the topological structure data of the vehicles to be scheduled, and determine the digital twin model of the target to be scheduled based on the airport data of the aircraft to be scheduled and the topological structure data of the vehicles to be scheduled;

[0213] Perform simulation calculations on the digital twin model of the target to be scheduled to generate simulation operation data of the target to be scheduled;

[0214] The aircraft taxiing model at the airport to be scheduled is constructed by using the simulation operation data of the target to be scheduled, the airport data of the aircraft to be scheduled and the topological structure data of the vehicle to be scheduled;

[0215] Perform simulation calculation on the taxiing model of the aircraft at the airport to be scheduled, and output the taxiing motion data set of the aircraft to be scheduled;

[0216] Based on the preset danger zone screening conditions and the preset multi-agent reinforcement learning algorithm, the environmental information of the target to be scheduled is determined according to the simulated operation data of the target to be scheduled, the preset actual operation data and the taxiing motion data set of the aircraft to be scheduled;

[0217] The target environment information and taxiing motion data set of the aircraft to be scheduled are used as the input of the target strategy network in the airport taxiing scheduling model of the target aircraft, and multiple taxiing distributed strategies of target aircraft are output.

[0218] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0219] An embodiment of the present invention further provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory; when the computer program is executed by the processor, the processor executes the steps of the aircraft airport taxiing scheduling model training method as described in any of the above embodiments.

[0220] An embodiment of the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the steps of the aircraft airport taxiing scheduling model training method as described in any of the above embodiments are implemented.

[0221] An embodiment of the present invention further provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the aircraft airport taxiing scheduling model training method as described in any of the above embodiments.

[0222] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0223] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0224] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for training an aircraft airport taxiing scheduling model, characterized in that: include: Acquire aircraft airport data to be trained and vehicle topology data to be trained, and determine a target digital twin model according to the aircraft airport data to be trained and the vehicle topology data to be trained; Performing simulation operation on the target digital twin model to generate target simulation operation data; Using the target simulation operation data, the airport data of the aircraft to be trained and the topological structure data of the vehicle to be trained, constructing an airport aircraft taxiing model; Performing simulation calculation on the airport aircraft taxiing model and outputting an aircraft taxiing motion data set; Performing distributed strategy training on the initial aircraft airport taxiing scheduling model according to the aircraft taxiing motion data set, the target simulation operation data and the preset actual operation data to determine the target aircraft airport taxiing scheduling model; The initial aircraft airport taxi scheduling model includes an initial value function network, an initial strategy network and an initial replica network; The step of performing distributed strategy training on the initial aircraft airport taxiing scheduling model according to the aircraft taxiing motion data set, the target simulation operation data and the preset actual operation data to determine the target aircraft airport taxiing scheduling model includes: Using target environment information and multiple sampled aircraft taxiing motion data as inputs of an initial strategy network in the initial aircraft airport taxiing scheduling model, and outputting multiple aircraft taxiing distributed strategies to be trained; The initial aircraft airport taxiing scheduling model is trained by using the taxiing distributed strategies of the aircraft to be trained, multiple simulation environment information in the target simulation operation data, and multiple actual environment information in the preset actual operation data to determine the target aircraft airport taxiing scheduling model.

2. The aircraft airport taxiing scheduling model training method according to claim 1 is characterized in that: The step of determining the target digital twin model according to the aircraft airport data to be trained and the vehicle topology data to be trained comprises: Using the aircraft airport data to be trained and the vehicle topology data to be trained to construct an initial digital twin model; Performing simulation operation on the initial digital twin model to generate initial simulation operation data; The preset actual operation data and the initial simulation operation data are used to train the initial digital twin model to determine the target digital twin model.

3. The aircraft airport taxiing scheduling model training method according to claim 1, characterized in that: The step of performing distributed strategy training on the initial aircraft airport taxiing scheduling model according to the aircraft taxiing motion data set, the target simulation operation data and the preset actual operation data to determine the target aircraft airport taxiing scheduling model includes: Based on the preset danger zone screening conditions, determining multiple target position relationships according to the target simulation operation data and the preset actual operation data; Using a preset multi-agent reinforcement learning algorithm, according to the aircraft taxiing motion data set, multiple simulation environment information in the target simulation operation data, multiple actual environment information in the preset actual operation data, and each target position relationship, calculate the simulation importance degree value corresponding to each simulation environment information and the actual importance degree value corresponding to each actual environment information; Comparing each of the simulated importance degree values, each of the actual importance degree values ​​and a preset degree threshold; Taking any simulation environment information corresponding to the simulation importance degree value greater than the preset degree threshold and the actual environment information corresponding to the actual importance degree value as the target environment information; The aircraft taxiing motion data set is sampled to determine a plurality of sampled aircraft taxiing motion data.

4. The aircraft airport taxiing scheduling model training method according to claim 3 is characterized in that: The step of determining the position relationship of multiple targets based on the preset danger zone screening conditions and according to the target simulation operation data and the preset actual operation data includes: Determining multiple simulated aircraft position relationships and multiple actual aircraft position relationships according to multiple simulated environment information in the target simulation operation data and multiple actual environment information in the preset actual operation data; Based on the preset danger zone screening conditions, the simulated aircraft position relationships and the actual aircraft position relationships are screened to determine a plurality of target position relationships.

5. The aircraft airport taxiing scheduling model training method according to claim 3 is characterized in that: The initial airport taxiing scheduling model also includes an initial value function network and an initial replica network; the target airport taxiing scheduling model includes a target strategy network, a target value function network and a target replica network; the step of training the initial airport taxiing scheduling model using the taxiing distributed strategies of the aircraft to be trained, the multiple simulation environment information in the target simulation operation data and the multiple actual environment information in the preset actual operation data to determine the target airport taxiing scheduling model includes: Interacting the distributed taxiing strategies of the aircraft to be trained, the multiple simulation environment information in the target simulation operation data, and the multiple actual environment information in the preset actual operation data to generate interaction results, multiple new simulation environment information, and multiple new actual environment information; Calculate a reward function value according to the interaction result; Scoring each of the taxiing distributed strategies of the aircraft to be trained and the probability distribution of each of the taxiing distributed strategies of the aircraft to be trained respectively through the initial value function network, and outputting a first scoring result and a second scoring result; Using the target replica network to score each of the aircraft taxiing distributed strategies to be trained, and outputting a third scoring result; Substituting the reward function value, the first scoring result, the second scoring result and the third scoring result into a preset loss function and taking derivatives to determine a model gradient; Based on the preset optimizer, the model parameters of the initial policy network, the model parameters of the initial value function network and the model parameters of the initial copy network are updated using the model gradient, an intermediate policy network, an intermediate value function network and an intermediate copy network are determined, and the number of simulation iterations is counted in real time; Determine whether the number of simulation iterations reaches a preset iteration threshold; If the number of simulation iterations reaches the preset iteration threshold, the intermediate policy network is used as the target policy network, the intermediate value function network is used as the target value function network, and the intermediate replica network is used as the target replica network.

6. The aircraft airport taxiing scheduling model training method according to claim 1, characterized in that: Also includes: Acquire the airport data of the aircraft to be scheduled and the topological structure data of the vehicles to be scheduled, and determine the digital twin model of the target to be scheduled according to the airport data of the aircraft to be scheduled and the topological structure data of the vehicles to be scheduled; Performing simulation calculation on the digital twin model of the target to be scheduled to generate simulation operation data of the target to be scheduled; Using the simulation operation data of the target to be scheduled, the data of the airport to be scheduled and the topological structure data of the vehicle to be scheduled, a taxiing model of the aircraft at the airport to be scheduled is constructed; Performing simulation calculation on the taxiing model of the aircraft at the airport to be scheduled, and outputting a taxiing motion data set of the aircraft to be scheduled; Based on the preset danger zone screening conditions and the preset multi-agent reinforcement learning algorithm, the environmental information of the target to be scheduled is determined according to the simulation operation data of the target to be scheduled, the preset actual operation data and the taxiing motion data set of the aircraft to be scheduled; The target environment information to be scheduled and the taxiing motion data set of the aircraft to be scheduled are used as inputs of the target strategy network in the airport taxiing scheduling model of the target aircraft, and multiple target aircraft taxiing distributed strategies are output.

7. An aircraft airport taxiing scheduling model training system, characterized in that: include: An acquisition module, used to acquire aircraft airport data to be trained and vehicle topology data to be trained, and determine a target digital twin model according to the aircraft airport data to be trained and the vehicle topology data to be trained; A first computing module is used to perform simulation computing on the target digital twin model to generate target simulation operation data; A construction module, used to construct an airport aircraft taxiing model by using the target simulation operation data, the aircraft airport data to be trained and the vehicle topology structure data to be trained; A second computing module is used to perform simulation computing on the airport aircraft taxiing model and output an aircraft taxiing motion data set; A training module, used for performing distributed strategy training on the initial aircraft airport taxiing scheduling model according to the aircraft taxiing motion data set, the target simulation operation data and preset actual operation data, and determining the target aircraft airport taxiing scheduling model; The initial aircraft airport taxi scheduling model includes an initial value function network, an initial strategy network and an initial replica network; The training module is specifically used for: Using target environment information and multiple sampled aircraft taxiing motion data as inputs of an initial strategy network in the initial aircraft airport taxiing scheduling model, and outputting multiple aircraft taxiing distributed strategies to be trained; The initial aircraft airport taxiing scheduling model is trained by using the taxiing distributed strategies of the aircraft to be trained, multiple simulation environment information in the target simulation operation data, and multiple actual environment information in the preset actual operation data to determine the target aircraft airport taxiing scheduling model.

8. A computer device, characterized in that: It comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the aircraft airport taxiing scheduling model training method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the aircraft airport taxiing scheduling model training method as described in any one of claims 1-6 is implemented.

10. A computer program product, characterized in that The computer program product includes a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program includes program instructions, wherein when the program instructions are executed by a computer, the computer is caused to execute the aircraft airport taxi scheduling model training method as described in any one of claims 1-6.

Images