Airport flight guarantee vehicle scheduling method and device based on deep reinforcement learning, equipment and medium

Abstract

The invention provides an airport flight guarantee vehicle scheduling method, device and equipment based on deep reinforcement learning, and a medium, and the method comprises the steps: inputting an initial flight guarantee plan into an encoder network layer of a vehicle scheduling strategy network model, and carrying out the extraction of initial features, performing multi-head attention mechanism processing on the initial features based on a next encoder network layer until flight demand features of each flight guarantee task are output; and inputting the mean value feature of the flight demand feature and the encoder feature of the decision action of the previous step into a decoder network layer, performing Markov decision process solution based on the action mask and the multi-head attention mechanism feature with the mask, and determining a vehicle scheduling scheme. The large-scale flight guarantee vehicle scheduling problem can be solved with high quality within the specified response time, dynamic decision can be made according to the flight task change, and the vehicle scheduling scheme corresponding to the flight guarantee task can be quickly determined.

Description

Technical Field

[0001] The present application relates to the technical field of airport support vehicle scheduling, and in particular to a method, device, equipment and medium for scheduling airport flight support vehicles based on deep reinforcement learning. Background Art

[0002] Airport flight support vehicles include tractors, baggage trailers, refueling trucks, de-icing trucks and other vehicles. In actual application scenarios, the demand for flight support vehicles is highly time-varying (such as changes in flight arrival times and flight schedules caused by various factors such as weather), which makes it difficult for manual scheduling to respond to these changes quickly and accurately. Traditional vehicle scheduling algorithms are generally divided into exact algorithms and heuristic algorithms. Exact algorithms generally use mixed integer programming models for solution, but their time complexity is too high, and they are inefficient when processing large-scale data, making it difficult to meet real-time requirements. Heuristic algorithms such as genetic algorithms, ant colony algorithms, and particle swarm algorithms can obtain feasible solutions in a short time, but the accuracy of their solutions is low, and the time cost of further improving the accuracy is high. Therefore, how to improve the efficiency of generating airport flight support vehicle scheduling solutions has become a technical issue that cannot be underestimated. Summary of the invention

[0003] In view of this, the purpose of this application is to provide an airport flight support vehicle scheduling method, device, equipment and medium based on deep reinforcement learning, which can achieve high-quality solution to large-scale flight support vehicle scheduling problems within the specified response time, and quickly determine the vehicle scheduling plan corresponding to the flight support task.

[0004] The embodiment of the present application provides an airport flight support vehicle scheduling method based on deep reinforcement learning, and the airport flight support vehicle scheduling method based on deep reinforcement learning includes: Input the initial flight support plan into the first encoder network layer of the pre-trained vehicle scheduling strategy network model, extract the initial features of each flight support task in the initial flight support plan, and process the initial features with a multi-head attention mechanism based on the next encoder network layer until the last encoder network layer outputs the flight demand features of each flight support task; wherein the vehicle scheduling strategy network model is obtained by iteratively training the neural network model through a deep reinforcement learning algorithm and a fine-tuning method of temporal partial random reconstruction; The mean features of the flight demand features of each of the flight support tasks and the encoder features of the support vehicle executing the previous decision action are input into the decoder network layer of the vehicle scheduling strategy network model, and the mean features of each of the flight demand features and the encoder features are solved by the Markov decision process based on the action mask and the masked multi-head attention mechanism to determine the original vehicle scheduling plans corresponding to the multiple flight support tasks; wherein the original vehicle scheduling plan includes the flight execution sequence of each of the support vehicles, and the decision action is the flight executed by the support vehicle.

[0005] In a possible implementation manner, after determining the original vehicle scheduling schemes corresponding to the plurality of flight support tasks, the airport flight support vehicle scheduling method based on deep reinforcement learning further includes: When the initial flight support plan changes, a new flight support set is determined, a decision action that has been executed in the original vehicle scheduling plan is determined based on a timestamp, and the executed decision action is used as the initial visited flight set; Input the new flight support plan into the encoder network layer to determine the second flight demand characteristics of each new flight support task; Determine the executed decision actions of the flight execution sequence of the support vehicle, and process the second flight demand feature based on the decoder network layer and the greedy strategy to determine the probability of new action selection after the executed decision action; The flight corresponding to the maximum action selection probability is used as a new execution decision action until the flight execution sequence iterations of all support vehicles are completed and a new vehicle scheduling scheme for the new flight support plan is determined.

[0006] In a possible implementation manner, the action mask and masked multi-head attention mechanism performs a Markov decision process to solve the mean feature of each flight demand feature and the encoder feature, and determines the original vehicle scheduling schemes corresponding to the plurality of flight support tasks, including: Determine the current state feature of each of the flight support tasks based on the mean feature of the flight demand feature and the encoder feature; Processing each of the current state features based on the action mask and the masked multi-head attention mechanism to determine the attention features of each of the flight support tasks; Solve the attention feature and the flight demand feature through a Markov decision process to determine the action selection probability of the flight support task being selected at the current step, and select the maximum action selection probability according to the greedy strategy; The flight corresponding to the maximum action selection probability is used as the decision action of the support vehicle at the current step, until the decision action is determined for each step of the support vehicle, and then the flight execution sequence of the support vehicle is generated.

[0007] In a possible implementation manner, the vehicle scheduling strategy network model is determined by the following steps: Obtain a reference flight plan, and construct the flight support vehicle scheduling problem corresponding to the reference flight plan into a Markov decision process; wherein the Markov decision process includes the execution status of the flight support task, the executed flight support task, the state transition probability, and the reward; Input the reference flight plan into the neural network model, perform a Markov decision process to solve the mean features of the reference features of the reference flight plan and the reference encoder features based on the action mask of the reference flight support task and the masked multi-head attention mechanism, and determine the reference action selection probability of the reference flight support task being selected at the current step; Based on the greedy strategy and the probability sampling strategy, the reference action selection probabilities are processed to generate a reference flight execution sequence for each support vehicle, and based on the reference flight execution sequences, the network parameters of the neural network model are iteratively updated to generate an initial vehicle scheduling strategy network model; The network parameters of the initial vehicle scheduling strategy network model are adjusted based on a fine-tuning method of random reconstruction of the time portion to determine the vehicle scheduling strategy network model.

[0008] In a possible implementation, the action mask of the reference flight support task is determined by the following steps, including: Based on the completion time of the previous reference flight support task, the remaining capacity of the current support vehicle, the time when the support vehicle moves from the previous reference flight task to the current reference flight task, the start service time of the current reference flight task, the visited reference flight task set, the support vehicle capacity required for the current reference flight, and the service time window of the current reference flight task, the action mask of the reference flight support task is determined.

[0009] In a possible implementation manner, the greedy strategy and the probability sampling strategy are used to process the plurality of reference action selection probabilities to generate a reference flight execution sequence for each support vehicle, and the network parameters of the neural network model are iteratively updated based on the plurality of reference flight execution sequences to generate an initial vehicle scheduling strategy network model, including: Processing the plurality of reference action selection probabilities based on a greedy strategy to generate a first execution sequence for the support vehicle to perform the reference flight support task; Processing the plurality of reference action selection probabilities based on a probability sampling strategy to generate a second execution sequence for the support vehicle to perform the reference flight support task; Based on multiple first execution sequences and multiple second execution sequences, the gradient value of the loss function of the neural network model is determined, and the network parameters of the neural network model are updated based on the optimizer and the gradient ascent algorithm until the initial vehicle scheduling strategy network model is determined when the number of iterations is reached.

[0010] In a possible implementation manner, the fine-tuning method based on random reconstruction of the time portion adjusts the network parameters of the initial vehicle scheduling strategy network model to determine the vehicle scheduling strategy network model, including: Setting a required change time of the reference flight plan, determining multiple completed decision actions before the change time, and adding the multiple completed decision actions to the visited reference flight set; The reconstruction decision action after the change time is determined, and the reconstruction decision action is used as a supervisory signal to iteratively adjust the network parameters of the initial vehicle scheduling strategy network model to generate the vehicle scheduling strategy network model.

[0011] The embodiment of the present application also provides an airport flight support vehicle dispatching device based on deep reinforcement learning, and the airport flight support vehicle dispatching device based on deep reinforcement learning includes: A feature encoding module is used to input the initial flight support plan into the first encoder network layer of the pre-trained vehicle scheduling strategy network model, extract the initial features of each flight support task in the initial flight support plan, and process the initial features with a multi-head attention mechanism based on the next encoder network layer until the last encoder network layer outputs the flight demand features of each flight support task; wherein the vehicle scheduling strategy network model is obtained by iteratively training the neural network model through a deep reinforcement learning algorithm and a fine-tuning method of temporal partial random reconstruction; A scheduling plan generation module is used to input the mean features of the flight demand features of each of the flight support tasks and the encoder features of the support vehicle executing the previous decision action into the decoder network layer of the vehicle scheduling strategy network model, and solve the mean features of each of the flight demand features and the encoder features based on the action mask and the masked multi-head attention mechanism to perform a Markov decision process to determine the original vehicle scheduling plans corresponding to multiple flight support tasks; wherein the original vehicle scheduling plan includes the flight execution sequence of each of the support vehicles, and the decision action is the flight executed by the support vehicle.

[0012] An embodiment of the present application also provides an electronic device, including: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned airport flight support vehicle scheduling method based on deep reinforcement learning are performed.

[0013] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned airport flight support vehicle scheduling method based on deep reinforcement learning are executed.

[0014] The embodiment of the present application provides a method, device, equipment and medium for scheduling airport flight support vehicles based on deep reinforcement learning. The method for scheduling airport flight support vehicles based on deep reinforcement learning includes: inputting an initial flight support plan into a first encoder network layer of a pre-trained vehicle scheduling strategy network model, extracting initial features of each flight support task in the initial flight support plan, and performing multi-head attention mechanism processing on the initial features based on the next encoder network layer until the last encoder network layer outputs the flight demand features of each flight support task; wherein the vehicle scheduling strategy network model is a vehicle scheduling strategy network model trained through a deep reinforcement learning algorithm and a time component. The method is obtained by iteratively training the neural network model using a fine-tuning method with random reconstruction; the mean feature of the flight demand feature of each flight support task and the encoder feature of the support vehicle executing the previous decision action are input into the decoder network layer of the vehicle scheduling strategy network model, and the mean feature of each flight demand feature and the encoder feature are solved by a Markov decision process based on an action mask and a masked multi-head attention mechanism to determine the original vehicle scheduling plan corresponding to multiple flight support tasks; wherein the original vehicle scheduling plan includes the flight execution sequence of each support vehicle, and the decision action is the flight executed by the support vehicle. The beneficial effects of the present application are: it realizes high-quality solution to large-scale flight support vehicle scheduling problems within the specified response time, and can make dynamic decisions according to changes in flight tasks, and quickly determine the vehicle scheduling plan corresponding to the flight support task.

[0015] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A flowchart of a method for dispatching airport flight support vehicles based on deep reinforcement learning provided in an embodiment of the present application; Figure 2 One of the structural schematic diagrams of a method and device for dispatching airport flight support vehicles based on deep reinforcement learning provided in an embodiment of the present application; Figure 3 The second structural diagram of a method and device for dispatching airport flight support vehicles based on deep reinforcement learning provided in an embodiment of the present application; Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work belongs to the scope of protection of the present application.

[0019] First, the application scenarios to which the present application is applicable are introduced. The present application can be applied to the technical field of airport support vehicle dispatching.

[0020] Based on this, an embodiment of the present application provides an airport flight support vehicle scheduling method based on deep reinforcement learning, which achieves high-quality solution to large-scale flight support vehicle scheduling problems within the specified response time, and can make dynamic decisions based on changes in flight missions, and quickly determine the vehicle scheduling plan corresponding to the flight support mission.

[0021] See also Figure 1 , Figure 1 This is a flow chart of a method for dispatching airport flight support vehicles based on deep reinforcement learning provided in an embodiment of the present application. Figure 1 As shown in , the airport flight support vehicle scheduling method based on deep reinforcement learning provided by the embodiment of the present application includes: S101: Input the initial flight support plan into the first encoder network layer of the pre-trained vehicle scheduling strategy network model, extract the initial features of each flight support task in the initial flight support plan, and perform a multi-head attention mechanism on the initial features based on the next encoder network layer, until the last encoder network layer outputs the flight demand features of each flight support task.

[0022] In this step, the initial flight support plan is input into the first encoder network layer of the vehicle scheduling strategy network model, the initial features of each flight support task in the initial flight support plan are extracted, and the initial features are processed by a multi-head attention mechanism according to the next encoder network layer until the last encoder network layer outputs the flight demand features of each flight support task. Among them, the vehicle scheduling strategy network model is obtained by iteratively training the neural network model through a deep reinforcement learning algorithm and a fine-tuning method of random reconstruction of the time part.

[0023] Here, the initial features of the flight support task are obtained in the following way: (Formula 1) in, Flight support mission j The initial features of Representative j The parking spaces corresponding to the flight support tasks The location coordinates of . , , are all learnable parameters. Flight support mission j The execution time period, The capabilities required for flight support mission j (such as fuel volume, cargo capacity, etc.), The guarantee service time of flight guarantee task j, an example of the airport flight guarantee vehicle scheduling problem is a problem with n flights and Sorted ordered set .

[0024] Here, the flight support task j is Characteristics of the layer encoder network layers for: (Formula 2) (Formula 3) Among them, BN stands for batch normalization, MHA stands for multi-head attention mechanism, and FFN stands for feedforward network layer. Flight support mission j In the Characteristics of the encoder network layers.

[0025] Here, the airport geometry can be represented by Abstract representation, vertex set , where 0 represents the parking garage, 1 to m represent the parking spaces, and the vertex is The corresponding coordinates are , edge set , For edge The weight function, the weight value is from the position Move to Required distance.

[0026] S102: Input the mean features of the flight demand features of each of the flight support tasks and the encoder features of the support vehicle executing the previous decision action into the decoder network layer of the vehicle scheduling strategy network model, and perform a Markov decision process on the mean features of each of the flight demand features and the encoder features based on the action mask and the masked multi-head attention mechanism to determine the original vehicle scheduling plans corresponding to the multiple flight support tasks.

[0027] In this step, the mean features of the flight demand characteristics of each flight support task and the encoder features of the support vehicle executing the previous decision action are input into the decoder network layer. The Markov decision process is solved for the mean features and encoder features of each flight demand feature based on the action mask and the masked multi-head attention mechanism to determine the original vehicle scheduling plan corresponding to multiple flight support tasks.

[0028] The original vehicle scheduling plan includes the flight execution sequence of each support vehicle, and the decision action is the flight executed by the support vehicle. For example, if the flight execution sequence of the support vehicle is {1,4,2,5,0}, it means that the support vehicle first executes the first flight support task, then executes the fourth flight support task, and finally 0 represents returning to the garage.

[0029] Among them, the Markov decision process can be composed of 4 tuples Indicates that the agent uses the vehicle scheduling strategy network model Gradually select flights for vehicle service. (1) Execution status of flight support tasks It is represented by the airport's geometry, the airport's current flight support vehicle demand, the previous scheduling decision and its completion time and remaining capacity. Represented by the vehicle scheduling strategy network model Flight missions to be performed, if Indicates the decision process of the current vehicle returning to the garage and starting another vehicle. This method designs an action mask to make the action meet the constraints of the problem. When all flights are After the first decision is satisfied, the final vehicle scheduling solution can be obtained. , where each subsequence ending with 0 represents a vehicle's dispatch plan. State transition probability Indicates a given state With action , transfer to the next state The probability that .award By function Determine that this application minimizes the total path length by maximizing the reward, and we can get: (Formula 4) in, , Representative flight mission The index of the corresponding position, W is the index from the position Move to Required distance.

[0030] In this application, a neural network model is constructed to solve the Markov decision process of airport flight support vehicle scheduling. A neural network with an encoder-decoder architecture using a multi-head attention mechanism is used to adapt to changes in the demand for flight support vehicles. In order to better fit the geometric features of a specified airport, the encoder uses learnable features to represent the location coordinates of the airport's parking spaces and garages, and then integrates the demand, location, and time window features of each flight, and maps the flight demand information to latent space features through a multi-head attention mechanism. The decoder obtains the features of the current optional flight based on the action mask, and outputs the action selection probability based on the previous decision information and the current state information. The entire support vehicle scheduling solution is obtained by iteratively outputting the action sequence step by step through the strategy network.

[0031] In a possible implementation manner, the action mask and masked multi-head attention mechanism performs a Markov decision process to solve the mean feature of each flight demand feature and the encoder feature, and determines the original vehicle scheduling schemes corresponding to the plurality of flight support tasks, including: A: Based on the mean characteristics of the flight demand characteristics and the encoder characteristics, the current state characteristics of each of the flight support tasks are determined.

[0032] Here, the current state characteristics of each flight support task are determined based on the mean characteristics of the flight demand characteristics and the encoder characteristics.

[0033] Among them, the encoder feature is the current state feature corresponding to the flight support task when the support vehicle performs the previous decision action.

[0034] Here, the current state characteristics of the flight support task are constructed by the following formula: (Formula 5) in, is the current state feature, is the mean characteristic of flight demand characteristics, are learnable parameters, for t-1 The encoder characteristics during step decision action, To ensure the remaining capacity of the vehicle in the current step, The completion time of the previous flight.

[0035] B: Process each of the current state features based on the action mask and the masked multi-head attention mechanism to determine the attention features of each of the flight support tasks.

[0036] Here, each current state feature is processed according to the action mask and the masked multi-head attention mechanism to determine the attention feature of each flight support task.

[0037] Among them, the attention features are calculated by the action mask and the masked multi-head attention mechanism : (Formula 6) (Formula 7) (Formula 8) in, is a learnable parameter, M is the number of longs, is the action mask, is a multi-head attention mechanism, is the characteristic of the nth flight support mission, is the action mask of the k-th flight support task at the t-th step, For the k The characteristics of each flight support mission are: For the m The attention of a person.

[0038] C: Solve the attention feature and the flight demand feature through a Markov decision process to determine the action selection probability of the flight support task being selected at the current step, and select the maximum action selection probability according to the greedy strategy.

[0039] Among them, the action selection probability is determined by the following formula: (Formula 9) (Formula 10) in, is the vehicle scheduling strategy network model, is the action selection probability corresponding to the flight determined at step t, For the j Flight support missions, For flights j The current state characteristics, For flights j Flight demand characteristics, is a learnable parameter, is the dimension of the feature, C is a constant parameter and tanh is the hyperbolic tangent function.

[0040] D: The flight corresponding to the maximum action selection probability is used as the decision action of the support vehicle at the current step, until the decision action is determined for each step of the support vehicle, and then the flight execution sequence of the support vehicle is generated.

[0041] Here, the flight corresponding to the maximum action selection probability is used as the decision action of the support vehicle at the current step, until the decision action is determined for each step of the support vehicle, and then the flight execution sequence of the support vehicle is generated.

[0042] In a possible implementation manner, after determining the vehicle scheduling schemes corresponding to the plurality of flight support tasks, the airport flight support vehicle scheduling method based on deep reinforcement learning further includes: a: When the initial flight support plan changes, a new flight support set is determined, and the decision action that has been executed in the original vehicle scheduling plan is determined based on the timestamp, and the executed decision action is used as the initial visited flight set.

[0043] Here, when the initial flight support plan changes, a new flight support set is determined, and the decision action that has been executed in the original vehicle scheduling plan is determined according to the timestamp, and the executed decision action is used as the initial visited flight set.

[0044] b: Input the new flight support plan into the encoder network layer to determine the second flight demand characteristics of each new flight support task.

[0045] Here, the new flight support plan is input into the encoder network layer to determine the second demand characteristics of each new flight support task.

[0046] c: Determine the executed decision actions of the flight execution sequence of the guarantee vehicle, and process the second flight demand characteristics based on the decoder network layer and the greedy strategy to determine the probability of new action selection after the executed decision actions.

[0047] Here, the already executed decision actions of the flight execution sequence of the guarantee vehicle are determined, and the second flight demand characteristics are processed according to the decoder network layer and the greedy strategy to determine the probability of new action selection after the already executed decision actions.

[0048] d: The flight corresponding to the maximum action selection probability is taken as the new execution decision action until the flight execution sequence of all support vehicles is iterated and the new vehicle scheduling plan for the new flight support plan is determined.

[0049] Here, the flight corresponding to the largest action selection probability is used as the new execution decision action until the flight execution sequence of all support vehicles is iterated and the new vehicle scheduling plan for the new flight support plan is determined. When the demand for flight support vehicles changes, the scheduling plan can be adjusted in real time according to the current operating status of the support vehicles, so as to achieve the reasonable allocation of support vehicles under the new demand.

[0050] In a specific embodiment, according to the initial flight plan Get the current flight demand characteristics and Input the vehicle scheduling strategy network model and obtain the current scheduling plan according to formula (9) and greedy strategy iterative reasoning. If the demand for flight support vehicles changes, the new flight support task set is , change the time to NT. Generally, In relative to Changed flight arrival time Should be greater than NT. Recalculate using encoder Corresponding features .according to And the corresponding scheduling plan Get the dispatch plan for each support vehicle. Specifically, get the global dispatch plan Split into Flight execution sequence ending with 0 , is the subsequence index, For sequence The starting index, is the sequence length, and the completion time of the last service flight corresponding to the sequence is determined according to formula (12) , where each sequence represents a train scheduling plan. Determine the executed decision actions and their sequence numbers: For each sequence i, calculate , and regard the task being executed as an executed action, and get the executed action sequence number = , taking the executed actions as the initial set of visited flights For each sequence , retain the executed actions, and obtain a new action sequence through the decoder and greedy strategy iteration Specifically, if ,but ;like , fixed serial number and previous decisions, for t> according to Use greedy strategy to iteratively update the sequence number Subsequent decisions until =0, we can get ,in = 0. , Merge into a new decision sequence If the flight is not fully serviced, continue to use Make decisions until all flights are served, and a new decision plan can be obtained ,in The number of newly added subsequences. Update , , and obtain a new scheduling plan.

[0051] In a possible implementation manner, the vehicle scheduling strategy network model is determined by the following steps: (1): Obtain a reference flight plan, and construct the flight support vehicle scheduling problem corresponding to the reference flight plan into a Markov decision process; wherein the Markov decision process includes the execution status of the flight support task, the executed flight support task, the state transition probability, and the reward.

[0052] Here, a reference flight plan is obtained, and the flight support vehicle scheduling problem corresponding to the reference flight plan is constructed into a Markov decision process.

[0053] (2): Input the reference flight plan into the neural network model, and solve the mean features of the reference features of the reference flight plan and the reference encoder features based on the action mask of the reference flight support task and the masked multi-head attention mechanism to determine the reference action selection probability of the reference flight support task being selected in the current step.

[0054] Here, the reference flight plan is input into the neural network model, and the Markov decision process is solved for the mean features of the reference features of the reference flight plan and the reference encoder features according to the action mask of the reference flight support task and the masked multi-head attention mechanism to determine the reference action selection probability of the reference flight support task being selected in the current step.

[0055] Here, the determination process of the reference action selection probability is consistent with the determination process of the above-mentioned action selection probability, and will not be described in detail in this part.

[0056] In a possible implementation, the action mask of the reference flight support task is determined by the following steps, including: Based on the completion time of the previous reference flight support task, the remaining capacity of the current support vehicle, the time when the support vehicle moves from the previous reference flight task to the current reference flight task, the start service time of the current reference flight task, the visited reference flight task set, the support vehicle capacity required for the current reference flight, and the service time window of the current reference flight task, the action mask of the reference flight support task is determined.

[0057] Here, the action mask of the flight support task is determined based on the completion time of the previous flight support task, the remaining capacity of the current support vehicle, the time when the support vehicle moves from the previous flight task to the current flight task, the start service time of the current flight task, the set of visited flight tasks, the support vehicle capacity required for the current flight, and the service time window of the current flight task.

[0058] The design process of the action mask is as follows: Step 1: Assume the remaining capacity of the current support vehicle is , the completion time of the previous flight is , visited flights collection ,initialization , , , For the t-th action decision , updated by the following formula Step value: (Formula 11) (Formula 12) (Formula 13) in, is the capacity required for flight j at step t (such as fuel volume, cargo capacity, etc.), for Flight guarantee service time, Represents the vehicle's decision from the previous step Flights to flights time, represent Flight start time Step 2: Flights No. t Step mask Determined by the following formula: (Formula 14) (Formula 15) in, The mask corresponding to the vehicle returning to the garage in step t is: is the set of flights visited at step t-1, Represents the vehicle's decision from the previous step Flights to flights time, represent The flight's service start time, the remaining capacity of the current support vehicle is .

[0059] In this application, in order to adapt to the dynamic changes in flight support needs, the airport flight support vehicle scheduling process is defined as a Markov decision process. The action is assumed to be the next destination of the current vehicle, and the entire action sequence constitutes the scheduling plan for all vehicles. The reward is assumed to be a negative total vehicle driving distance.

[0060] (3): Based on the greedy strategy and the probability sampling strategy, the multiple reference action selection probabilities are processed to generate a reference flight execution sequence for each support vehicle. Based on the multiple reference flight execution sequences, the network parameters of the neural network model are iteratively updated to generate an initial vehicle scheduling strategy network model.

[0061] Here, multiple reference action selection probabilities are processed according to the greedy strategy and the probability sampling strategy to generate a reference flight execution sequence for each support vehicle. The network parameters of the neural network model are iteratively updated according to the multiple reference flight execution sequences to generate the initial vehicle scheduling strategy network model.

[0062] In a possible implementation manner, the greedy strategy and the probability sampling strategy are used to process the plurality of reference action selection probabilities to generate a reference flight execution sequence for each support vehicle, and the network parameters of the neural network model are iteratively updated based on the plurality of reference flight execution sequences to generate an initial vehicle scheduling strategy network model, including: I: Based on the greedy strategy, the plurality of reference action selection probabilities are processed to generate a first execution sequence for the support vehicle to perform the reference flight support task.

[0063] Here, multiple reference action selection probabilities trained by the current model are selected and processed according to the greedy strategy to generate the first execution sequence reference of the flight support task for the support vehicle flight execution reference.

[0064] II: Processing the plurality of reference action selection probabilities based on a probability sampling strategy to generate a second sequence reference for the execution of the vehicle flight support mission.

[0065] Here, a plurality of reference action selection probabilities are selected and processed according to a probability sampling strategy to generate a second execution sequence for the support vehicle to perform the reference flight support task.

[0066] Here, the multiple reference action selection probabilities of the baseline model are processed according to the probability sampling strategy to generate a second execution sequence for the support vehicle to perform the reference flight support task.

[0067] III: Based on multiple first execution sequences and multiple second execution sequences, the gradient value of the loss function of the neural network model is determined, and the network parameters of the neural network model are updated based on the optimizer and the gradient ascent algorithm until the number of iterations is reached to determine the initial vehicle scheduling strategy network model.

[0068] Here, the gradient value of the loss function is determined by the following formula: (Formula 16) in, is the gradient value of the loss function of the neural network model, B is the reference quantity, Execute sequence for the second reference flight The reward value, Execute sequence for first reference flight The reward value, are the parameters of the neural network model, For the neural network model. Initialize the network parameters of the neural network model , baseline network parameters , randomly generate a validation set of 1000 instances based on and Generate decision sequences using a greedy strategy and calculate their rewards and , and a one-sided paired t test was performed. If the significance level is less than , then use renew .

[0069] (4): Based on the fine-tuning method of temporal partial random reconstruction, the network parameters of the initial vehicle scheduling strategy network model are adjusted to determine the vehicle scheduling strategy network model.

[0070] Here, the network parameters of the initial vehicle scheduling strategy network model are adjusted according to the fine-tuning method of temporal partial random reconstruction to determine the vehicle scheduling strategy network model.

[0071] In a possible implementation manner, the fine-tuning method based on random reconstruction of the time portion adjusts the network parameters of the initial vehicle scheduling strategy network model to determine the vehicle scheduling strategy network model, including: i: Setting the required change time of the reference flight plan, and determining a plurality of decision actions that have been completed before the change time, and adding the plurality of decision actions that have been completed to the set of visited reference flights.

[0072] Here, the demand change time of the reference flight plan is set, and multiple decision actions that have been completed before the change time are determined, and the multiple decision actions that have been completed are added to the visited reference flight set.

[0073] ii: Determine a reconstruction decision action after the change time, and use the reconstruction decision action as a supervisory signal to iteratively adjust the network parameters of the initial vehicle scheduling strategy network model to generate the vehicle scheduling strategy network model.

[0074] Here, the reconstruction decision action after the change time is determined, and the reconstruction decision action is used as a supervision signal to iteratively adjust the network parameters of the initial vehicle scheduling strategy network model to generate a vehicle scheduling strategy network model.

[0075] Among them, the reconstructed reference flight execution sequence composed of the reconstructed decision actions is recorded as , where the action index set that undergoes reconstruction is recorded as .like , then take this reference as a feasible reconstruction solution, otherwise reconstruct it again, where is the defined reward function. Execute the sequence for the reference flight, generating A feasible reconstruction solution and its corresponding action index set that undergoes reconstruction , .Will The reconstructed part is used as a one-hot supervision signal, and the network is fine-tuned using the cross-entropy loss function to generate a vehicle scheduling strategy network model.

[0076] In this application, in order to enable the vehicle scheduling strategy network model to better cope with the dynamic changes in flight support vehicle demand in real-time scenarios, a self-improvement training method based on partial random reconstruction of time is designed. Specifically, the trained initial vehicle scheduling strategy network model is used for iterative reasoning to obtain the solution of the training reference, and a time point is randomly selected as the demand change time, the decision actions before this time are fixed, and they are added to the visited flights. Then, the decision actions after this time are re-iteratively sampled by the vehicle scheduling strategy network model according to probability. If the reward function value of the reconstructed decision is larger, the reconstructed action after this time point is used as a supervision signal, and the network is fine-tuned using the cross entropy loss function, so as to improve the ability of the vehicle scheduling strategy network model to solve problems containing partially fixed actions when demand changes dynamically.

[0077] An embodiment of the present application provides an airport flight support vehicle scheduling method based on deep reinforcement learning, and the airport flight support vehicle scheduling method based on deep reinforcement learning includes: inputting an initial flight support plan into a first encoder network layer of a pre-trained vehicle scheduling strategy network model, extracting initial features of each flight support task in the initial flight support plan, and performing multi-head attention mechanism processing on the initial features based on the next encoder network layer until the last encoder network layer outputs the flight demand features of each flight support task; wherein the vehicle scheduling strategy network model is obtained through a deep reinforcement learning algorithm and a temporal partial random repetition. The neural network model is iteratively trained by the fine-tuning method of the structure; the mean feature of the flight demand feature of each flight support task and the encoder feature of the support vehicle executing the previous decision action are input into the decoder network layer of the vehicle scheduling strategy network model, and the mean feature of each flight demand feature and the encoder feature are solved by the Markov decision process based on the action mask and the masked multi-head attention mechanism to determine the original vehicle scheduling plan corresponding to multiple flight support tasks; wherein, the original vehicle scheduling plan includes the flight execution sequence of each support vehicle, and the decision action is the flight executed by the support vehicle. It realizes high-quality solution to large-scale flight support vehicle scheduling problems within the specified response time, and can make dynamic decisions according to changes in flight tasks, and quickly determine the vehicle scheduling plan corresponding to the flight support task.

[0078] See also Figure 2 , Figure 3 , Figure 2 One of the structural schematic diagrams of a method and device for dispatching airport flight support vehicles based on deep reinforcement learning provided in an embodiment of the present application; Figure 3 This is a second structural diagram of a method and device for dispatching airport flight support vehicles based on deep reinforcement learning provided in an embodiment of the present application. Figure 2 As shown in , the airport flight support vehicle dispatching device 200 based on deep reinforcement learning includes: The feature encoding module 210 is used to input the initial flight support plan into the first encoder network layer of the pre-trained vehicle scheduling strategy network model, extract the initial features of each flight support task in the initial flight support plan, and perform multi-head attention mechanism processing on the initial features based on the next encoder network layer until the last encoder network layer outputs the flight demand features of each flight support task; wherein the vehicle scheduling strategy network model is obtained by iteratively training the neural network model through a deep reinforcement learning algorithm and a fine-tuning method of temporal partial random reconstruction; The scheduling plan generation module 220 is used to input the mean features of the flight demand features of each of the flight support tasks and the encoder features of the support vehicle executing the previous decision action into the decoder network layer of the vehicle scheduling strategy network model, and perform a Markov decision process to solve the mean features of each of the flight demand features and the encoder features based on the action mask and the masked multi-head attention mechanism to determine the original vehicle scheduling plans corresponding to multiple flight support tasks; wherein the original vehicle scheduling plan includes the flight execution sequence of each of the support vehicles, and the decision action is the flight executed by the support vehicle.

[0079] Further, such as Figure 3 As shown, the airport flight support vehicle scheduling device 200 based on deep reinforcement learning also includes a solution updating module 230, which is used to: When the initial flight support plan changes, a new flight support set is determined, a decision action that has been executed in the original vehicle scheduling plan is determined based on a timestamp, and the executed decision action is used as the initial visited flight set; Input the new flight support plan into the encoder network layer to determine the second demand feature of each new flight support task; Determine the executed decision actions of the flight execution sequence of the support vehicle, and process the second flight demand feature based on the decoder network layer and the greedy strategy to determine the probability of new action selection after the executed decision action; The flight corresponding to the maximum action selection probability is used as a new execution decision action until the flight execution sequence iterations of all support vehicles are completed and a new vehicle scheduling scheme for the new flight support plan is determined.

[0080] Further, when the scheduling scheme generation module 220 is used to solve the mean feature of each flight demand feature and the encoder feature based on the action mask and the masked multi-head attention mechanism to determine the original vehicle scheduling scheme corresponding to the plurality of flight support tasks, the scheduling scheme generation module 220 is specifically used to: Determine the current state feature of each of the flight support tasks based on the mean feature of the flight demand feature and the encoder feature; Processing each of the current state features based on the action mask and the masked multi-head attention mechanism to determine the attention features of each of the flight support tasks; Solve the attention feature and the flight demand feature through a Markov decision process to determine the action selection probability of the flight support task being selected at the current step, and select the maximum action selection probability according to the greedy strategy; The flight corresponding to the maximum action selection probability is used as the decision action of the support vehicle at the current step, until the decision action is determined for each step of the support vehicle, and then the flight execution sequence of the support vehicle is generated.

[0081] Furthermore, the airport flight support vehicle dispatching device 200 based on deep reinforcement learning also includes a model training module 240, which is used to: Obtain a reference flight plan, and construct the flight support vehicle scheduling problem corresponding to the reference flight plan into a Markov decision process; wherein the Markov decision process includes the execution status of the flight support task, the executed flight support task, the state transition probability, and the reward; Input the reference flight plan into the neural network model, perform a Markov decision process to solve the mean features of the reference features of the reference flight plan and the reference encoder features based on the action mask of the reference flight support task and the masked multi-head attention mechanism, and determine the reference action selection probability of the reference flight support task being selected at the current step; Based on the greedy strategy and the probability sampling strategy, the reference action selection probabilities are processed to generate a reference flight execution sequence for each support vehicle, and based on the reference flight execution sequences, the network parameters of the neural network model are iteratively updated to generate an initial vehicle scheduling strategy network model; The network parameters of the initial vehicle scheduling strategy network model are adjusted based on a fine-tuning method of random reconstruction of the time portion to determine the vehicle scheduling strategy network model. Furthermore, the model training module 240 determines the action mask of the reference flight support task through the following steps: Based on the completion time of the previous reference flight support task, the remaining capacity of the current support vehicle, the time when the support vehicle moves from the previous reference flight task to the current reference flight task, the start service time of the current reference flight task, the visited reference flight task set, the support vehicle capacity required for the current reference flight, and the service time window of the current reference flight task, the action mask of the reference flight support task is determined.

[0082] Further, when the model training module 240 is used to process the plurality of reference action selection probabilities based on the greedy strategy and the probability sampling strategy, generate a reference flight execution sequence for each support vehicle, and iteratively update the network parameters of the neural network model based on the plurality of reference flight execution sequences to generate an initial vehicle scheduling strategy network model, the model training module 240 is specifically used to: Processing the plurality of reference action selection probabilities based on a greedy strategy to generate a first execution sequence for the support vehicle to perform the reference flight support task; Processing the plurality of reference action selection probabilities based on a probability sampling strategy to generate a second execution sequence for the support vehicle to perform the reference flight support task; Based on multiple first execution sequences and multiple second execution sequences, the gradient value of the loss function of the neural network model is determined, and the network parameters of the neural network model are updated based on the optimizer and the gradient ascent algorithm until the initial vehicle scheduling strategy network model is determined when the number of iterations is reached.

[0083] Furthermore, when the model training module 240 uses the fine-tuning method based on the time partial random reconstruction to adjust the network parameters of the initial vehicle scheduling strategy network model and determines the vehicle scheduling strategy network model, the model training module 240 is specifically used to: Setting a required change time of the reference flight plan, determining multiple completed decision actions before the change time, and adding the multiple completed decision actions to the visited reference flight set; The reconstruction decision action after the change time is determined, and the reconstruction decision action is used as a supervisory signal to iteratively adjust the network parameters of the initial vehicle scheduling strategy network model to generate the vehicle scheduling strategy network model.

[0084] An embodiment of the present application provides an airport flight support vehicle scheduling device based on deep reinforcement learning, and the airport flight support vehicle scheduling device based on deep reinforcement learning includes: a feature encoding module, which is used to input the initial flight support plan into the first encoder network layer of a pre-trained vehicle scheduling strategy network model, extract the initial features of each flight support task in the initial flight support plan, and perform a multi-head attention mechanism on the initial features based on the next encoder network layer until the last encoder network layer outputs the flight demand features of each flight support task; wherein the vehicle scheduling strategy network model is a random reconstruction of the time portion through a deep reinforcement learning algorithm. The fine-tuning method is used to iteratively train the neural network model; the scheduling plan generation module is used to input the mean features of the flight demand features of each flight support task and the encoder features of the support vehicle executing the previous decision action into the decoder network layer of the vehicle scheduling strategy network model, and the Markov decision process is solved for the mean features of each flight demand feature and the encoder features based on the action mask and the masked multi-head attention mechanism to determine the original vehicle scheduling plan corresponding to multiple flight support tasks; wherein, the original vehicle scheduling plan includes the flight execution sequence of each support vehicle, and the decision action is the flight executed by the support vehicle. It realizes the solution of large-scale flight support vehicle scheduling problems within the specified response time, and quickly determines the vehicle scheduling plan corresponding to the flight support task.

[0085] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown in , the electronic device 400 includes a processor 410 , a memory 420 and a bus 430 .

[0086] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, the above-mentioned Figure 1 The steps of the airport flight support vehicle scheduling method based on deep reinforcement learning in the method embodiment shown in the method embodiment, the specific implementation method can be found in the method embodiment, and will not be repeated here.

[0087] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 The steps of the airport flight support vehicle scheduling method based on deep reinforcement learning in the method embodiment shown in the method embodiment, the specific implementation method can be found in the method embodiment, and will not be repeated here.

[0088] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0089] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0090] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0091] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0092] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0093] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for dispatching airport flight support vehicles based on deep reinforcement learning, characterized in that: The airport flight support vehicle scheduling method based on deep reinforcement learning includes: Input the initial flight support plan into the first encoder network layer of the pre-trained vehicle scheduling strategy network model, extract the initial features of each flight support task in the initial flight support plan, and process the initial features with a multi-head attention mechanism based on the next encoder network layer until the last encoder network layer outputs the flight demand features of each flight support task; wherein the vehicle scheduling strategy network model is obtained by iteratively training the neural network model through a deep reinforcement learning algorithm and a fine-tuning method based on temporal partial random reconstruction; The mean features of the flight demand features of each of the flight support tasks and the encoder features of the support vehicle executing the previous decision action are input into the decoder network layer of the vehicle scheduling strategy network model, and the mean features of each of the flight demand features and the encoder features are solved by the Markov decision process based on the action mask and the masked multi-head attention mechanism to determine the original vehicle scheduling plans corresponding to the multiple flight support tasks; wherein the original vehicle scheduling plan includes the flight execution sequence of each of the support vehicles, and the decision action is the flight executed by the support vehicle.

2. The airport flight support vehicle scheduling method based on deep reinforcement learning according to claim 1 is characterized in that: After determining the original vehicle scheduling schemes corresponding to the plurality of flight support tasks, the airport flight support vehicle scheduling method based on deep reinforcement learning further includes: When the initial flight support plan changes, a new flight support set is determined, a decision action that has been executed in the original vehicle scheduling plan is determined based on a timestamp, and the executed decision action is used as the initial visited flight set; Input the new flight support plan into the encoder network layer to determine the second flight demand characteristics of each new flight support task; Determine the executed decision actions of the flight execution sequence of the support vehicle, and process the second flight demand feature based on the decoder network layer and the greedy strategy to determine the probability of new action selection after the executed decision action; The flight corresponding to the maximum action selection probability is used as a new execution decision action until the flight execution sequence iterations of all support vehicles are completed and a new vehicle scheduling scheme for the new flight support plan is determined.

3. The airport flight support vehicle scheduling method based on deep reinforcement learning according to claim 1 is characterized in that: The action mask-based and masked multi-head attention mechanism performs a Markov decision process to solve the mean feature of each flight demand feature and the encoder feature, and determines the original vehicle scheduling schemes corresponding to the multiple flight support tasks, including: Determine the current state feature of each of the flight support tasks based on the mean feature of the flight demand feature and the encoder feature; Processing each of the current state features based on the action mask and the masked multi-head attention mechanism to determine the attention features of each of the flight support tasks; Solve the attention feature and the flight demand feature through a Markov decision process to determine the action selection probability of the flight support task being selected at the current step, and select the maximum action selection probability according to the greedy strategy; The flight corresponding to the maximum action selection probability is used as the decision action of the support vehicle at the current step, until the decision action is determined for each step of the support vehicle, and then the flight execution sequence of the support vehicle is generated.

4. The airport flight support vehicle scheduling method based on deep reinforcement learning according to claim 1 is characterized in that: The vehicle dispatching strategy network model is determined by the following steps: Obtain a reference flight plan, and construct the flight support vehicle scheduling problem corresponding to the reference flight plan into a Markov decision process; wherein the Markov decision process includes the execution status of the flight support task, the executed flight support task, the state transition probability, and the reward; Input the reference flight plan into the neural network model, perform a Markov decision process to solve the mean features of the reference features of the reference flight plan and the reference encoder features based on the action mask of the reference flight support task and the masked multi-head attention mechanism, and determine the reference action selection probability of the reference flight support task being selected at the current step; Based on the greedy strategy and the probability sampling strategy, the reference action selection probabilities are processed to generate a reference flight execution sequence for each support vehicle, and based on the reference flight execution sequences, the network parameters of the neural network model are iteratively updated to generate an initial vehicle scheduling strategy network model; The network parameters of the initial vehicle scheduling strategy network model are adjusted based on a fine-tuning method of random reconstruction of the time portion to determine the vehicle scheduling strategy network model.

5. The airport flight support vehicle scheduling method based on deep reinforcement learning according to claim 4 is characterized in that: The action mask of the reference flight support task is determined through the following steps, including: Based on the completion time of the previous reference flight support task, the remaining capacity of the current support vehicle, the time when the support vehicle moves from the previous reference flight task to the current reference flight task, the start service time of the current reference flight task, the visited reference flight task set, the support vehicle capacity required for the current reference flight, and the service time window of the current reference flight task, the action mask of the reference flight support task is determined.

6. The airport flight support vehicle scheduling method based on deep reinforcement learning according to claim 4 is characterized in that: The method processes the plurality of reference action selection probabilities based on the greedy strategy and the probability sampling strategy to generate a reference flight execution sequence for each support vehicle, and iteratively updates the network parameters of the neural network model based on the plurality of reference flight execution sequences to generate an initial vehicle scheduling strategy network model, including: Processing the plurality of reference action selection probabilities based on a greedy strategy to generate a first execution sequence for the support vehicle to perform the reference flight support task; Processing the plurality of reference action selection probabilities based on a probability sampling strategy to generate a second execution sequence for the support vehicle to perform the reference flight support task; Based on multiple first execution sequences and multiple second execution sequences, the gradient value of the loss function of the neural network model is determined, and the network parameters of the neural network model are updated based on the optimizer and the gradient ascent algorithm until the initial vehicle scheduling strategy network model is determined when the number of iterations is reached.

7. The airport flight support vehicle scheduling method based on deep reinforcement learning according to claim 4 is characterized in that: The fine-tuning method based on random reconstruction of the time part adjusts the network parameters of the initial vehicle scheduling strategy network model to determine the vehicle scheduling strategy network model, including: Setting a required change time of the reference flight plan, determining multiple completed decision actions before the change time, and adding the multiple completed decision actions to the visited reference flight set; The reconstruction decision action after the change time is determined, and the reconstruction decision action is used as a supervisory signal to iteratively adjust the network parameters of the initial vehicle scheduling strategy network model to generate the vehicle scheduling strategy network model.

8. An airport flight support vehicle dispatching device based on deep reinforcement learning, characterized in that: The airport flight support vehicle dispatching device based on deep reinforcement learning includes: A feature encoding module is used to input the initial flight support plan into the first encoder network layer of the pre-trained vehicle scheduling strategy network model, extract the initial features of each flight support task in the initial flight support plan, and process the initial features with a multi-head attention mechanism based on the next encoder network layer until the last encoder network layer outputs the flight demand features of each flight support task; wherein the vehicle scheduling strategy network model is obtained by iteratively training the neural network model through a deep reinforcement learning algorithm and a fine-tuning method of temporal partial random reconstruction; A scheduling plan generation module is used to input the mean features of the flight demand features of each of the flight support tasks and the encoder features of the support vehicle executing the previous decision action into the decoder network layer of the vehicle scheduling strategy network model, and solve the mean features of each of the flight demand features and the encoder features based on the action mask and the masked multi-head attention mechanism to perform a Markov decision process to determine the original vehicle scheduling plans corresponding to multiple flight support tasks; wherein the original vehicle scheduling plan includes the flight execution sequence of each of the support vehicles, and the decision action is the flight executed by the support vehicle.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are run by the processor, the steps of the airport flight support vehicle scheduling method based on deep reinforcement learning as described in any one of claims 1 to 7 are executed.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the airport flight support vehicle scheduling method based on deep reinforcement learning as described in any one of claims 1 to 7 are executed.

Images