Airport special vehicle scheduling method and system based on deep reinforcement learning

Abstract

The invention relates to the technical field of airport special vehicle scheduling, in particular to an airport special vehicle scheduling method and system based on deep reinforcement learning, and the method comprises the following steps: taking the shortest flight service time as a target, modeling an airport special vehicle scheduling problem as a vehicle path planning problem with a time window, and solving a vehicle path planning problem with a time window; decomposing a vehicle path planning problem according to preset service priorities of various types of special vehicles; performing Markov decision process modeling on each sub-problem, building a neural network model based on an attention mechanism, generating a path solution of vehicle planning by using the neural network model, calculating a total return according to the path solution of each sub-problem, training the neural network model by using a deep reinforcement learning algorithm based on the reward of the total return, and performing vehicle planning. And obtaining a trained neural network model, and solving the airport special vehicle scheduling problem by using the trained neural network model. According to the method, scheduling solving of various types of special vehicles and various types of airport special vehicles can be realized, and the calculation efficiency is higher.

Description

Technical Field

[0001] The present invention relates to the technical field of airport special vehicle dispatching, and in particular to a method and system for dispatching airport special vehicles based on deep reinforcement learning. Background Art

[0002] With the rapid development of the civil aviation industry, the scale of airports continues to expand and the throughput is increasing dramatically. Airports are facing major problems such as low operating efficiency and insufficient collaborative decision-making capabilities. This has put forward higher requirements for airport ground operations and ground service guarantees. Therefore, how to quickly respond to basic business during airport operations and provide decision-making support to relevant departments has become a key issue in achieving efficient airport operations and effectively reducing flight delay rates.

[0003] At present, most airports mainly rely on manual dispatch for special vehicles, that is, if a flight arrives, a vehicle with free time is dispatched temporarily to carry out the aircraft support task. After completing this task, it returns to the designated parking place and waits for the next assigned order to arrive. This single-vehicle service single-flight dispatch method is extremely inefficient. On the one hand, it will lead to unbalanced vehicle task allocation and unreasonable route planning. Special vehicle tasks are prone to errors. The increase in flight ground support service time may also cause flight delays, especially for large hub airports or small and medium-sized airports in peak transportation periods. Flights can take off and land at a high density in a short period of time, and the inefficient dispatch method obviously cannot meet the needs of this scenario; on the other hand, due to the generally high cost of special vehicles, airports only provide limited vehicles to complete the service, and the single-vehicle service single-flight method will also cause a waste of resources.

[0004] At present, the research on special vehicle scheduling is still mainly focused on the scheduling of special vehicles of a single service type. For example, the airport refueling truck scheduling problem is studied, considering the optimization of the refueling truck route to minimize the total number of miles traveled, evenly distribute tasks, and shorten the time for flights to receive services, so as to achieve the purpose of reducing flight delays. However, there are various types of special vehicles at airports, including catering trucks, cleaning trucks, luggage transport trucks, etc. in addition to refueling trucks; in addition, the models of aircraft are not exactly the same. Taking airport refueling trucks as an example, different types of aircraft require different amounts of fuel and service times when receiving refueling services. Therefore, special vehicle scheduling also needs to consider factors such as model matching and restrictions.

[0005] To sum up, how to realize the dispatching solution of various types of special vehicles and various models of airport special vehicles is a technical problem that needs to be solved urgently. Summary of the invention

[0006] The purpose of the present invention is to provide an airport special vehicle scheduling method and system based on a deep reinforcement learning algorithm. The airport special vehicle scheduling problem is decomposed into multiple sub-problems according to the service priority of the airport special vehicle, and the entire scheduling problem is optimized by optimizing each sub-problem. Different aircraft types are distinguished according to the service time of each flight. Different types of airport special vehicles and different aircraft types are taken into account, so that the scheduling of airport special vehicles of multiple types and multiple aircraft models can be solved. In addition, an encoder-decoder strategy network based on an attention mechanism is used to generate a scheduling strategy and deep reinforcement learning is introduced to update it. Compared with traditional scheduling algorithms, the calculation efficiency is higher and the problem scale that can be handled is larger, which can be used to solve the airport special vehicle scheduling problem of multiple types of special vehicles and multiple aircraft models.

[0007] The present invention is implemented by the following technical scheme: A method for dispatching special vehicles at an airport based on a deep reinforcement learning algorithm, comprising the following steps:

[0008] With the goal of minimizing the flight service time, the airport special vehicle scheduling problem is modeled as a vehicle path planning problem with a time window, and the vehicle path planning problem is decomposed into several sub-problems according to the preset service priorities of various types of special vehicles;

[0009] Markov decision process modeling is performed for each sub-problem, and the interaction process between special vehicles and flights is predicted based on the Markov decision process;

[0010] Building a neural network model based on an attention mechanism, and using the neural network model to generate a path solution for vehicle planning, wherein the neural network model adopts an encoder-decoder structure;

[0011] Calculate the total reward according to the path solution of each sub-problem, and train the neural network model using a deep reinforcement learning algorithm based on the reward of the total reward to obtain a trained neural network model;

[0012] The trained neural network model is used to solve the airport special vehicle scheduling problem.

[0013] According to a preferred embodiment, the airport special vehicle scheduling problem is modeled as a vehicle path planning problem with a time window, specifically comprising: constructing the airport special vehicle scheduling problem as an undirected complete graph G(N, E), where N = {0, 1, 2, ..., n, n &} is used to represent the set of all nodes in G, where nodes 0, n & Used to distinguish leaving the garage and returning to the garage, E = {(i, j) | i, j∈N; i≠j} is used to represent the set of all edges in G, and i and j are any nodes in the set N.

[0014] According to a preferred embodiment, in the vehicle path planning problem with a time window, the time constraint of each special vehicle for flight service is within the time window. The time window is updated as follows:

[0015]

[0016] In the above formula, represents the estimated arrival time of the flight at flight node i, represents the estimated departure time of the flight from flight node i, represents the completion time of the current special vehicle service flight node i, Indicates the service time of the current special vehicle at flight node i.

[0017] According to a preferred embodiment, the Markov decision process model is composed of a quaternion M = {S, A, P a , R} definition, where S is a state set, and the state set S is represented by the state It contains graph embedding vector, feature context embedding vector and served flight information. A is an action set, and the action set A is represented by action Indicates the selection of the next flight to be served, P a is the transfer function, Indicates that from the current state Transition to next state The probability, P a =1, R is the reward function, which is defined as the distance traveled by a certain type of special vehicle after serving all flights. f ), R = -tour(a f ).

[0018] According to a preferred embodiment, the processing process of the encoder is as follows:

[0019] Use the embedding layer to embed the node position information g i (i=1, 2, ...n) is mapped to a high-dimensional feature vector The flight demand and time window are linearly mapped to the feature vector Add together to get the initial node embedding vector The expression is as follows:

[0020]

[0021] In the above formula, v0 represents the corresponding node position information g i The eigenvector of (i=0), is the initial node embedding vector, is the demand of the current flight node i, W and b are trainable parameters;

[0022] The node embedding vector obtained in the previous step is updated through the multi-head attention layer and the fully connected feed-forward layer. The expression is as follows:

[0023]

[0024] In the above formula, BN represents the batch normalization sublayer, represents the node embedding vector of node i in the l-1th layer, MHA represents the multi-head attention sublayer, represents the node embedding vector of node i in layer l, and FF represents the fully connected sublayer;

[0025] Repeat the above update steps to get the node embedding of the last layer and graph embedding vector

[0026] According to a preferred embodiment, the processing process of the decoder is as follows:

[0027] Define the vehicle feature context embedding vector at step t Where F represents the fleet information embedding, which is used to distinguish different types of special vehicles, and C t Represents the remaining capacity of the current vehicle, T t Indicates the time when the current vehicle completed the service at the previous node. represents the embedding of the previous node;

[0028] The vehicle feature context embedding vector is updated using a multi-head attention layer,

[0029] Calculate the query vector Q corresponding to the vehicle feature context embedding vector t The key vector K corresponding to the node embedding vector t ,in W Q , W K is a trainable parameter;

[0030] Calculate the query vector Q t and the key vector K t The compatibility score u between t , and the compatibility score u is calculated by the softmax function t Normalize it and get the probability distribution π θ , the expression is as follows:

[0031]

[0032] In the above formula, C is a constant, tanh() represents the bitangent sine function, and d h Q t The dimension of Indicates that flight node i can be selected when decoding in step t, Indicates that flight node i cannot be selected during decoding in step t. Indicates the flight nodes that have been served from step 1 to step t-1;

[0033] Repeat the above steps until the special vehicle has served all flight nodes and returned to the corresponding garage node.

[0034] According to a preferred embodiment, when a flight node meets one of the following conditions, The conditions are as follows:

[0035] Condition 1: The current flight node i has not been visited, expressed as

[0036] Condition 2: The demand of the current flight node i does not exceed the remaining capacity of the current special vehicle, expressed as

[0037] Condition 3: The current special vehicle can complete the service before the end of the time window of flight node i, expressed as in, Indicates the time taken from the previous flight node i-1 to the current flight node i;

[0038] When the garage node meets the following conditions at the same time, The conditions are as follows:

[0039] Condition 1: The special vehicle has visited the garage node in the previous step;

[0040] Condition 2: There are still unserved flight nodes.

[0041] According to a preferred embodiment, the expression of training the neural network model using a deep reinforcement learning algorithm based on the total reward is as follows:

[0042]

[0043] In the above formula, x f Represents the sub-problems obtained by decomposing the vehicle path planning problem X.

[0044] According to a preferred embodiment, for each sub-problem x f , the loss function is defined as:

[0045] L clipped (θ|x f )=E[min(tour(a f )A t ,(tour(a f ), 1-ε, 1+ε)At )]

[0046] In the above formula, A t is the advantage function at time t, ε is the cutting coefficient, θ is the strategy parameter, and E represents the expectation.

[0047] The present invention also provides an airport special vehicle dispatching system based on deep reinforcement learning, comprising:

[0048] A problem modeling module is used to model the airport special vehicle scheduling problem as a vehicle path planning problem with a time window with the goal of minimizing the flight service time, and decompose the vehicle path planning problem into several sub-problems according to the preset service priorities of various types of special vehicles;

[0049] The Markov decision process modeling module is used to perform Markov decision process modeling on each sub-problem and predict the interaction process between special vehicles and flights based on the Markov decision process;

[0050] A neural network model building module, used to build a neural network model based on an attention mechanism, and use the neural network model to generate a path solution for vehicle planning, wherein the neural network model adopts an encoder-decoder structure;

[0051] A model training module, used to calculate the total reward according to the path solution of each sub-problem, and train the neural network model using a deep reinforcement learning algorithm based on the reward of the total reward to obtain a trained neural network model;

[0052] The path planning module is used to solve the airport special vehicle scheduling problem using the trained neural network model.

[0053] The technical solution of the airport special vehicle scheduling method and system based on deep reinforcement learning provided by the present invention has at least the following advantages and beneficial effects: (1) The present invention decomposes the airport special vehicle scheduling problem into multiple sub-problems according to the service priority of the airport special vehicle, optimizes the entire scheduling problem by optimizing each sub-problem, and distinguishes different aircraft types according to the service time of each flight. It takes into account both different types of airport special vehicles and different types of aircraft, thereby realizing the scheduling solution of airport special vehicles of multiple types and multiple models; (2) The present invention adopts an encoder-decoder strategy network based on the attention mechanism to generate the scheduling strategy and introduces deep reinforcement learning to update it. Compared with the traditional scheduling algorithm, the present invention has higher computational efficiency and can handle a larger problem scale, and can be used to solve the airport special vehicle scheduling problem of multiple types of special vehicles and multiple models. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1A schematic diagram of the overall process of the airport special vehicle dispatching method based on the deep reinforcement learning algorithm provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0056] Example 1

[0057] Figure 1 A schematic diagram of the overall process of an airport special vehicle dispatching method based on a deep reinforcement learning algorithm provided in an embodiment of the present invention.

[0058] See also Figure 1 As shown, this embodiment divides the airport special vehicle scheduling method into seven steps, wherein steps one to three are modeling steps, step four is an encoding step, step five is a decoding step, step six is ​​a parameter updating step, and step seven is a solving step.

[0059] The following describes steps one to seven respectively:

[0060] Step 1: With the goal of minimizing the flight service time, the airport special vehicle scheduling problem is modeled as a vehicle path planning problem with a time window.

[0061] Specifically in this embodiment, the airport special vehicle scheduling problem is modeled as a vehicle path planning problem with a time window, which specifically includes:

[0062] The airport special vehicle scheduling problem is constructed as an undirected complete graph G(N,E), where N = {0, 1, 2, ..., n, n &} is used to represent the set of all nodes in G. Considering that the aircraft is parked at the parking position waiting for service, N includes the flight nodes that need service and the parking garage nodes for special vehicles, where nodes 0 and n are & Used to distinguish special vehicles leaving the garage and returning to the garage; E = {(i, j) | i, j∈N; i≠j} is used to represent the set of all edges in G, any two nodes are connected by an edge, and i and j are any nodes in the set N.

[0063] Step 2: Consider various types of special vehicles and decompose the vehicle path planning problem x into several sub-problems x according to the preset service priorities of each type of special vehicles. f , by optimizing each subproblem x f To solve the entire scheduling problem.

[0064] Specifically in this embodiment, in the vehicle path planning problem with time windows, the time constraints of each special vehicle for flight service are within the time window. The time window is updated as follows:

[0065]

[0066] In the above formula, represents the estimated arrival time of the flight at flight node i, represents the estimated departure time of the flight from flight node i, represents the completion time of the current special vehicle service flight node i, Indicates the service time of the current special vehicle at flight node i.

[0067] It should be noted that the present invention decomposes the airport special vehicle scheduling problem into multiple sub-problems according to the service priority of the airport special vehicles, optimizes the entire scheduling problem by optimizing each sub-problem, and distinguishes different aircraft types according to the service time of each flight. It takes into account both different types of airport special vehicles and different aircraft types, thereby realizing the scheduling solution of airport special vehicles of multiple types and multiple models.

[0068] Step 3: Model each sub-problem using the Markov decision process, and predict the interaction between special vehicles and flights based on the Markov decision process.

[0069] Specifically in this embodiment, the Markov decision process model is composed of a quaternion M = {S, A, P a , R} definition.

[0070] Where S is the state set, and the state set S is represented by the state Contains graph embedding vectors, feature context embedding vectors, and flight information that has been served; A is an action set, and action set A is represented by action Indicates selecting the next flight to be served; P a is the transfer function, Indicates that from the current state Transition to next state The probability of P is that the next state is certain in the airport special vehicle scheduling problem. a =1; R is the reward function, which is defined as the distance traveled by a certain type of special vehicle after serving all flights. f ), R = -tour(a f ).

[0071] Step 4: Build a neural network model based on the attention mechanism. The neural network model adopts an encoder-decoder structure.

[0072] Specifically in this embodiment, for each sub-problem x f First, use the encoder to respectively encode the node position information g in the completely undirected graph G(N, E) i (i=1, 2, ...n) as well as flight demand and time window are encoded to obtain the corresponding feature representation, namely the node embedding vector and the graph embedding vector.

[0073] The encoder process is as follows:

[0074] Use the embedding layer to embed the node position information g i (i=1, 2, ...n) is mapped to a high-dimensional feature vector The flight demand and time window are linearly mapped to the feature vector Add together to get the initial node embedding vector The expression is as follows:

[0075]

[0076] In the above formula, v0 represents the corresponding node position information g i The eigenvector of (i=0), is the initial node embedding vector, is the demand of the current flight node i, W and b are trainable parameters.

[0077] Furthermore, the node embedding vector obtained in the previous step is updated through the multi-head attention layer and the fully connected feed-forward layer. The expression is as follows:

[0078]

[0079] In the above formula, BN represents the batch normalization sublayer, represents the node embedding vector of node i in the l-1th layer, MHA represents the multi-head attention sublayer, represents the node embedding vector of node i in layer l, and FF represents the fully connected sublayer.

[0080] Repeat the above update steps to get the node embedding of the last layer and graph embedding vector

[0081] Step 5: For each sub-problem x f , use the decoder to decode, and in each decoding step, the feature representation obtained in the previous step And the current special vehicle status V t (including fleet information embedded in F, the remaining capacity of the current vehicle C tAnd the time T when the current vehicle completed the service at the previous node t ) and the history path (embedding of the previous node ) is composed of a context embedding vector Update the context embedding vector using a multi-head attention layer At the same time, a mask mechanism is used to block out nodes that have been visited, and a single-head attention layer is used to output the probability distribution to obtain the path solution for vehicle planning.

[0082] Specifically in this embodiment, the processing process of the decoder is as follows:

[0083] Define the vehicle feature context embedding vector at step t Represents the embedding of the previous node.

[0084] Furthermore, a multi-head attention layer is used to update the vehicle feature context embedding vector. During the updating process, in order to ensure that the generated path solution is feasible, a mask mechanism is introduced, as follows:

[0085] For flight nodes, Indicates that flight node i can be selected when decoding in step t, Indicates that flight node i cannot be selected during the t-step decoding.

[0086] When a flight node meets one of the following conditions, The conditions are as follows:

[0087] Condition 1: The current flight node i has not been visited, expressed as

[0088] Condition 2: The demand of the current flight node i does not exceed the remaining capacity of the current special vehicle, expressed as

[0089] Condition 3: The current special vehicle can complete the service before the end of the time window of flight node i, expressed as in, Indicates the time taken from the previous flight node i-1 to the current flight node i.

[0090] When the garage node meets the following conditions at the same time, The conditions are as follows:

[0091] Condition 1: The special vehicle has visited the garage node in the previous step;

[0092] Condition 2: There are still unserved flight nodes.

[0093] Furthermore, the query vector Q corresponding to the vehicle feature context embedding vector is calculated tThe key vector K corresponding to the node embedding vector t ,in W Q , W K is a trainable parameter.

[0094] Calculate the query vector Q t and the key vector K t The compatibility score u between t , the expression is as follows:

[0095]

[0096] In the above formula, C is a constant, tanh() represents the bitangent sine function, and d h Q t Dimension.

[0097] The compatibility score u is calculated by the softmax function. t Normalize it and get the probability distribution π θ , the expression is as follows:

[0098]

[0099] In the above formula, Represents the flight nodes that have been served from step 1 to step t-1.

[0100] Repeat the above steps until the special vehicle has served all flight nodes and returned to the corresponding garage node. The node sequence selected in this process is the path solution to the problem.

[0101] Step 6: Calculate the total reward based on the path solutions of each sub-problem, and use the deep reinforcement learning algorithm to train the neural network model based on the reward of the total reward to obtain the trained neural network model.

[0102] Specifically in this embodiment, the method of training the neural network model is as follows: randomly generate a simulation example set, and divide all problem instances into a training set, a validation set, and a test set, use the training set to train the encoder and decoder multiple times, and perform a solution evaluation on the validation set after each batch of training is completed.

[0103] In one implementation of this embodiment, the deep reinforcement learning algorithm adopted is the PPO algorithm. In addition, it can also be DQN, DDPG, TRPO, SAC, etc., which are not specifically limited here.

[0104] Preferably, this embodiment uses a deep reinforcement learning algorithm to update the parameters of the encoder and decoder according to the total reward, and the expression is as follows:

[0105]

[0106] In the above formula, x f Represents the sub-problems obtained by decomposing the vehicle path planning problem X.

[0107] In this embodiment, for each sub-problem x f , the loss function is defined as:

[0108] L clipped (θ|x f )=E[min(tour(a f )A t ,(tour(a f ), 1-ε, 1+ε)A t )]

[0109] In the above formula, A t is the advantage function at time t, ε is the clipping coefficient, usually taken as 0.2, θ is the strategy parameter, and E represents the expectation.

[0110] Step 7: Use the trained neural network model to solve the airport special vehicle scheduling problem.

[0111] It should be noted that the present invention adopts an encoder-decoder strategy network based on the attention mechanism to generate a scheduling strategy and introduces deep reinforcement learning to update it. Compared with traditional scheduling algorithms, it has higher computational efficiency and can handle larger problems. It can be used to solve the scheduling problems of various types of special vehicles and various types of aircraft at airports.

[0112] Example 2

[0113] This embodiment, based on the technical solution provided in Embodiment 1, provides an airport special vehicle dispatching system based on deep reinforcement learning, including:

[0114] The problem modeling module is used to model the airport special vehicle scheduling problem as a vehicle path planning problem with a time window with the goal of minimizing the flight service time, and decompose the vehicle path planning problem into several sub-problems based on the preset service priorities of various types of special vehicles;

[0115] The Markov decision process modeling module is used to perform Markov decision process modeling on each sub-problem and predict the interaction process between special vehicles and flights based on the Markov decision process;

[0116] The neural network model building module is used to build a neural network model based on the attention mechanism and use the neural network model to generate a path solution for vehicle planning. The neural network model adopts an encoder-decoder structure.

[0117] The model training module is used to calculate the total reward based on the path solutions of each sub-problem, and train the neural network model using a deep reinforcement learning algorithm based on the reward of the total reward to obtain the trained neural network model;

[0118] The path planning module is used to solve the airport special vehicle scheduling problem using the trained neural network model.

[0119] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for dispatching special vehicles at an airport based on a deep reinforcement learning algorithm, characterized in that: The following steps are involved: With the goal of minimizing the flight service time, the airport special vehicle scheduling problem is modeled as a vehicle path planning problem with a time window, and the vehicle path planning problem is decomposed into several sub-problems according to the preset service priorities of various types of special vehicles; Markov decision process modeling is performed for each sub-problem, and the interaction process between special vehicles and flights is predicted based on the Markov decision process; Building a neural network model based on an attention mechanism, and using the neural network model to generate a path solution for vehicle planning, wherein the neural network model adopts an encoder-decoder structure; Calculate the total reward according to the path solution of each sub-problem, and train the neural network model using a deep reinforcement learning algorithm based on the reward of the total reward to obtain a trained neural network model; The trained neural network model is used to solve the airport special vehicle scheduling problem.

2. The airport special vehicle dispatching method based on deep reinforcement learning algorithm according to claim 1 is characterized in that: The airport special vehicle scheduling problem is modeled as a vehicle path planning problem with a time window, specifically comprising: constructing the airport special vehicle scheduling problem as an undirected complete graph G(N,E), where N = {0,1,2,…,n,n & } is used to represent the set of all nodes in G, where nodes 0, n & Used to distinguish leaving the garage and returning to the garage, E = {(i, j) | i, j∈N; i≠j} is used to represent the set of all edges in G, and i and j are any nodes in the set N.

3. The airport special vehicle dispatching method based on deep reinforcement learning algorithm as claimed in claim 2 is characterized in that: In the vehicle routing problem with time windows, the time constraints of each special vehicle for flight service are within the time window. The time window is updated as follows: In the above formula, represents the estimated arrival time of the flight at flight node i, represents the estimated departure time of the flight from flight node i, represents the completion time of the current special vehicle service flight node i, Indicates the service time of the current special vehicle at flight node i.

4. The airport special vehicle dispatching method based on deep reinforcement learning algorithm according to any one of claims 1 to 3, characterized in that: The Markov decision process model is composed of a quaternion M = {S, A, P a , R} definition, where S is a state set, and the state set S is represented by the state It contains graph embedding vector, feature context embedding vector and served flight information. A is an action set, and the action set A is represented by action Indicates the selection of the next flight to be served, P a is the transfer function, Indicates that from the current state Transition to next state The probability, P a =1, R is the reward function, which is defined as the distance traveled by a certain type of special vehicle after serving all flights. f ), R = -tour(a f ).

5. The airport special vehicle dispatching method based on deep reinforcement learning algorithm as claimed in claim 4 is characterized in that: The processing of the encoder is as follows: Use the embedding layer to embed the node position information g i (i=1,2,…n) is mapped to a high-dimensional feature vector The flight demand and time window are linearly mapped to the feature vector Add together to get the initial node embedding vector The expression is as follows: In the above formula, v0 represents the corresponding node position information g i The eigenvector of (i=0), is the initial node embedding vector, is the demand of the current flight node i, W and b are trainable parameters; The node embedding vector obtained in the previous step is updated through the multi-head attention layer and the fully connected feed-forward layer. The expression is as follows: In the above formula, BN represents the batch normalization sublayer, represents the node embedding vector of node i in the l-1th layer, MHA represents the multi-head attention sublayer, represents the node embedding vector of node i in layer l, and FF represents the fully connected sublayer; Repeat the above update steps to get the node embedding of the last layer and graph embedding vector 6. The airport special vehicle dispatching method based on deep reinforcement learning algorithm as claimed in claim 5, characterized in that: The processing of the decoder is as follows: Define the vehicle feature context embedding vector at step t Where F represents the fleet information embedding, which is used to distinguish different types of special vehicles, and C t Represents the remaining capacity of the current vehicle, T t Indicates the time when the current vehicle completed the service at the previous node. represents the embedding of the previous node; The vehicle feature context embedding vector is updated using a multi-head attention layer, Calculate the query vector Q corresponding to the vehicle feature context embedding vector t The key vector K corresponding to the node embedding vector t ,in W Q , W K is a trainable parameter; Calculate the query vector Q t and the key vector K t The compatibility score u between t , and the compatibility score u is calculated by the softmax function t Normalize it and get the probability distribution π θ , the expression is as follows: In the above formula, C is a constant, tanh( ) represents the bitangent sine function, and d h Q t The dimension of Indicates that flight node i can be selected when decoding in step t, Indicates that flight node i cannot be selected during decoding in step t. Indicates the flight nodes that have been served from step 1 to step t-1; Repeat the above steps until the special vehicle has served all flight nodes and returned to the corresponding garage node.

7. The airport special vehicle dispatching method based on deep reinforcement learning algorithm as claimed in claim 6, characterized in that: When a flight node meets one of the following conditions, The conditions are as follows: Condition 1: The current flight node i has not been visited, expressed as Condition 2: The demand of the current flight node i does not exceed the remaining capacity of the current special vehicle, expressed as Condition 3: The current special vehicle can complete the service before the end of the time window of flight node i, expressed as in, Indicates the time taken from the previous flight node i-1 to the current flight node i; When the garage node meets the following conditions at the same time, The conditions are as follows: Condition 1: The special vehicle has visited the garage node in the previous step; Condition 2: There are still unserved flight nodes.

8. The airport special vehicle dispatching method based on deep reinforcement learning algorithm as claimed in claim 6, characterized in that: The expression for training the neural network model using the deep reinforcement learning algorithm based on the total reward is as follows: In the above formula, x f Represents the sub-problems obtained by decomposing the vehicle path planning problem X.

9. The airport special vehicle dispatching method based on deep reinforcement learning algorithm as claimed in claim 8, characterized in that: For each subproblem x f , the loss function is defined as: L clipped (θ|x f )=E[min(turn(a f )A t ,(tour(a f ),1-e,1+e)A t )] In the above formula, A t is the advantage function at time t, ε is the cutting coefficient, θ is the strategy parameter, and E represents the expectation.

10. An airport special vehicle dispatching system based on deep reinforcement learning, characterized in that: include: A problem modeling module is used to model the airport special vehicle scheduling problem as a vehicle path planning problem with a time window with the goal of minimizing the flight service time, and decompose the vehicle path planning problem into several sub-problems according to the preset service priorities of various types of special vehicles; The Markov decision process modeling module is used to perform Markov decision process modeling on each sub-problem and predict the interaction process between special vehicles and flights based on the Markov decision process; A neural network model building module, used to build a neural network model based on an attention mechanism, and use the neural network model to generate a path solution for vehicle planning, wherein the neural network model adopts an encoder-decoder structure; A model training module, used to calculate the total reward according to the path solution of each sub-problem, and train the neural network model using a deep reinforcement learning algorithm based on the reward of the total reward to obtain a trained neural network model; The path planning module is used to solve the airport special vehicle scheduling problem using the trained neural network model.

Images