Methods for training dynamic scheduling models and generating dynamic scheduling schemes for tightly coupled jobs

Abstract

This invention provides a method for training a dynamic scheduling model and generating dynamic scheduling schemes for strongly coupled operations. The method includes: inputting a first state representation into an initial airborne logistics support decision model to obtain a first decision output; updating a simulation environment based on the first decision output, obtaining a second state representation representing the status of airborne material transportation operations from the updated simulation environment, inputting the second state representation into the initial airborne material transportation decision model to obtain a second decision output; using the simulation environment after executing the second decision output as the simulation environment for obtaining the first state representation, obtaining the target reward for executing the first and second decision outputs in the simulation environment, and updating the model parameters based on the target reward. This invention simulates the serial interaction and state influence between two decision-making stages through a shared simulation environment, and performs collaborative training of the two models based on the target reward, enabling joint scheduling optimization of airborne logistics support and airborne material transportation operations.

Description

Technical Field

[0001] The present invention relates to the technical field of support scheduling for seaplanes, and in particular to a dynamic scheduling model training and dynamic scheduling scheme generation method for strongly coupled operations. Background Art

[0002] In seaplane operations, sortie and recovery capability is the core factor determining the overall operational efficiency. Among them, deck maintenance service support is the key link that takes the longest time and involves the most complicated procedures in the process. Constrained by the narrow deck space and urgent sortie time, this link faces strict space-time resource constraints, and it is urgent to achieve efficient parallelization and collaboration of multiple processes through optimal scheduling of limited support resources and personnel. As a highly complex sub-process in maintenance support, airborne material transportation covers a complete logistics chain from material transfer to mounting, and has many constraint conditions. If unified global modeling is performed on this process and maintenance service support, solution space explosion will occur in large-scale task scenarios, making it difficult to obtain an optimal solution within a limited time.

[0003] At present, methods for solving such support operation scheduling problems mainly focus on heuristic optimization algorithms. For example, in the existing technology, heuristic strategies such as differential evolution algorithm are used to perform off-line planning and optimization for seaplane support tasks. In addition, with the development of artificial intelligence technology, deep reinforcement learning algorithms based on Markov decision processes have also been tried to be applied in the scheduling field. This algorithm establishes the mapping from environmental states to scheduling actions through a neural network, and optimizes the strategy through trial-and-error training, so as to achieve real-time decision-making in dynamic environments.

[0004] However, the existing technical solutions still have obvious deficiencies. First, the heuristic optimization methods represented by differential evolution algorithm belong to the off-line planning mode, which has large computational overhead, lags in response to sudden disturbances, and can hardly meet the real-time requirements of dynamic scheduling. Second, when unified global modeling is performed for support operation scheduling and strongly coupled sub-processes (for example, airborne material transportation tasks strongly coupled with support tasks), the solution space will expand sharply, making traditional heuristic algorithms face the "curse of dimensionality", resulting in low solution efficiency and even difficulty in convergence. Finally, although deep reinforcement learning theoretically supports dynamic decision-making, how to construct a composite task state that can effectively handle such strong coupling, multiple constraints, and strong timing, and learn a stable and efficient collaborative scheduling strategy from it, is still an unsolved problem. Summary of the Invention

[0005] The present invention provides a dynamic scheduling model training and dynamic scheduling scheme generation method for strongly coupled operations, which is used to solve the defect that the prior art fails to effectively implement real-time, efficient and convergent collaborative scheduling planning in a dynamic, strongly coupled, multi-constraint complex operation environment.

[0006] This invention provides a method for training a dynamic scheduling model for tightly coupled jobs. The dynamic scheduling model is obtained by iteratively executing the following steps until a preset iteration condition is met: Obtain the initial dynamic scheduling model; the initial dynamic scheduling model includes the initial maintenance support decision model and the initial airborne material transportation decision model; A first state representation is obtained from the simulation environment to characterize the operational status of aircraft maintenance support. The first state representation is then input into the initial aircraft maintenance support decision model to obtain the first decision output of the initial aircraft maintenance support decision model. The simulation environment is updated based on the first decision output, and a second state representation for characterizing the airborne material transportation operation status is obtained from the updated simulation environment. The second state representation is then input into the initial airborne material transportation decision model to obtain the second decision output of the initial airborne material transportation decision model. The simulation environment after executing the second decision output is used as the simulation environment for obtaining the first state representation, so as to obtain the target reward generated by executing the first decision output and the second decision output in the simulation environment, and update the model parameters of the initial maintenance service support decision model and the initial airborne material transportation decision model based on the target reward.

[0007] According to the present invention, a dynamic scheduling model training method for strongly coupled tasks includes the following steps in configuring the simulation environment: Configure the simulation environment based on the scenario file; The scenario file includes aircraft fleet data, transport vehicle attributes, support operation procedure attributes, support resource attributes, and support personnel set; The aircraft fleet data includes individual data tuples for each aircraft. Each individual data tuple includes the aircraft number, aircraft mission type, aircraft support position number, list of all loadout types on all loadout positions in the initial aircraft status, list of loadout types to be loaded on the aircraft, and the start time of the aircraft support mission.

[0008] According to the present invention, a dynamic scheduling model training method for strongly coupled operations includes the following constraints on the initial maintenance support decision model: The constraint condition for the coverage of the resource station with the available backup positions is used to constrain the number of backup positions that a single resource station can serve at any given time. The resource guarantee constraint is used to constrain the total demand of various types of resources at any given time. The resource guarantee constraint is determined based on the total demand of each type of resource at any given time and a preset upper limit for the quantity of each type of resource. The personnel constraint condition is used to constrain the total demand for various types of personnel at any given time. The personnel constraint condition is determined based on the total demand for each type of personnel at any given time and a preset upper limit for the number of each type of personnel. The execution constraints of the guarantee process are used to constrain the execution timing relationship between the subsequent guarantee process and the preceding guarantee process in the directed graph relationship between guarantee processes. The execution constraints of the guarantee process are determined based on the preset execution time of the preceding guarantee process, the execution time of the subsequent guarantee process, and the execution time of the preceding guarantee process. The completion time constraint for the guarantee process is used to constrain guarantee processes of different task types to have different execution times. The completion time constraint for the guarantee process is determined based on the task type to which the guarantee process belongs and the inherent attributes of the guarantee process. The start time constraint condition is used to ensure that the start time of any of the support procedures is not earlier than the start time of the aircraft support mission.

[0009] According to the present invention, a dynamic scheduling model training method for strongly coupled operations includes the following constraints on the initial airborne material transportation decision model: The loading point operation sequence constraint is used to constrain the execution order of material unloading and loading operations at the same loading point. The loading point operation sequence constraint is determined based on the end time of the unloading event and the start time of the loading event at the loading point. The load-bearing type constraint condition of the transport vehicle is used to restrict a single transport vehicle to carry only one type of material in one trip; The transport vehicle scheduling priority constraint is used to determine the priority order of the execution tasks of the transport vehicles to be scheduled when there are multiple transport vehicles to be scheduled. The transport vehicle scheduling priority constraint is determined based on the current loading matching status and the historical cumulative number of dispatches of each transport vehicle to be scheduled. The task stage constraints of the transport vehicle are used to constrain the sequence of sub-tasks that the transport vehicle must follow when performing different types of material transportation tasks. The task stage constraints of the transport vehicle are determined based on a preset directed graph, which includes multiple sub-task stage nodes.

[0010] According to the training method for a dynamic scheduling model for strongly coupled operations provided by the present invention, the step of determining the priority constraint of the transport vehicle scheduling includes: Based on the current loading matching situation, a first priority weight is determined; wherein, the first priority weight of the first dispatchable transport vehicle is higher than the first priority weight of the second dispatchable transport vehicle, the first dispatchable transport vehicle is a vehicle whose current loading situation matches the material transportation operation to be performed, and the second dispatchable transport vehicle is a vehicle whose current loading situation does not match the material transportation operation to be performed. Based on the historical cumulative dispatch count, a second priority weight is determined; wherein, the second priority weight of the third dispatchable transport vehicle is higher than that of the fourth dispatchable transport vehicle, and the historical cumulative dispatch count of the third dispatchable transport vehicle is less than that of the fourth dispatchable transport vehicle. Based on the first priority weight and the second priority weight, the scheduling priority constraints of the transport vehicle are determined.

[0011] According to the present invention, a dynamic scheduling model training method for strongly coupled operations is provided, wherein the initial maintenance service support decision model includes a first state space converter, a first neural network model, and a first action space converter; the first state space converter is used to encode the first state representation into a first situation feature vector, the first neural network model is used to output a first action value vector based on the first situation feature vector, and the first action space converter is used to convert the first action value vector into the first decision output. The initial airborne material transportation decision model includes a second state space converter, a second neural network model, and a second action space converter. The second state space converter is used to encode the second state representation into a second situation feature vector. The second neural network model is used to output a second action value vector based on the second situation feature vector. The second action space converter is used to convert the second action value vector into the second decision output.

[0012] According to the present invention, a dynamic scheduling model training method for strongly coupled operations is provided, wherein the second state represents the task time step of airborne material transportation, the aircraft group loading status and the transport vehicle status corresponding to the task time step; The status of the transport vehicle includes the transport vehicle type, the target mounting point of the transport vehicle, the current stage of the current operation task of the transport vehicle, the type of material being carried, the quantity of material being carried, and the number of times it has been dispatched in the past. The target reward is determined based on a preset optimization target; the optimization target is determined based on the completion time of the aircraft fleet, the load balancing variance of the transport vehicle, and the cumulative transfer time of the transport vehicle.

[0013] The present invention also provides a method for generating a dynamic scheduling scheme, comprising: Obtain the initial situation of the maritime aircraft support operations to be scheduled; The initial situation is deduced based on the dynamic scheduling model to obtain a dynamic scheduling scheme; The dynamic scheduling model is obtained by performing the training method described above.

[0014] The present invention also provides a training device for a dynamic scheduling model for strongly coupled jobs, comprising the following units: The acquisition unit is used to acquire the initial dynamic scheduling model; the initial dynamic scheduling model includes the initial maintenance support decision model and the initial airborne material transportation decision model. The first input unit is used to obtain a first state representation from the simulation environment to characterize the operational status of aircraft maintenance support, and input the first state representation into the initial aircraft maintenance support decision model to obtain the first decision output of the initial aircraft maintenance support decision model. The second input unit is used to update the simulation environment based on the first decision output, and obtain a second state representation from the updated simulation environment to characterize the airborne material transportation operation status, and input the second state representation into the initial airborne material transportation decision model to obtain the second decision output of the initial airborne material transportation decision model. The training unit is used to take the inference environment after executing the second decision output as the inference environment to obtain the first state representation, so as to obtain the target reward generated by executing the first decision output and the second decision output in the inference environment, and update the model parameters of the initial maintenance service support decision model and the initial airborne material transportation decision model based on the target reward.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the dynamic scheduling model training method for strongly coupled jobs as described above, or implements the dynamic scheduling scheme generation method.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the dynamic scheduling model training method for strongly coupled jobs as described above, or implements the dynamic scheduling scheme generation method.

[0017] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the dynamic scheduling model training method for strongly coupled jobs as described above, or implements the dynamic scheduling scheme generation method.

[0018] This invention provides a method for training a dynamic scheduling model and generating a dynamic scheduling scheme for strongly coupled operations. The method involves obtaining an initial dynamic scheduling model, which includes an initial maintenance support decision model and an initial airborne material transportation decision model. A first state representation characterizing the operational status of maintenance support is obtained from a simulation environment. This first state representation is input into the initial maintenance support decision model to obtain a first decision output. The simulation environment is updated based on the first decision output, and a second state representation characterizing the operational status of airborne material transportation is obtained from the updated simulation environment. This second state representation is input into the initial airborne material transportation decision model to obtain a second decision output. The simulation environment after executing the second decision output is used as the simulation environment for obtaining the first state representation, to obtain the target reward generated by executing the first and second decision outputs in the simulation environment. The model parameters of the initial maintenance support decision model and the initial airborne material transportation decision model are updated based on the target reward. This invention simulates the serial interaction and state influence between two decision-making stages through a shared inference environment. Then, based on a unified objective reward reflecting global performance, it collaboratively trains the initial aircraft maintenance support decision model and the initial airborne material transportation decision model. This allows the two decision models to learn how to cooperate to achieve global optimization during training, fully considering the influence of each other's decisions. This effectively handles the strong coupling between operational tasks, enabling joint scheduling optimization of aircraft maintenance support and airborne material transportation operations, thus improving the overall efficiency of maritime aircraft support operations. Simultaneously, because the trained dynamic scheduling model establishes an end-to-end mapping from real-time operational status to scheduling actions, it solves the problems of lag in response to dynamic uncertainty and low solution efficiency due to the curse of dimensionality in large-scale complex scenarios in offline planning methods. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the training method for a dynamic scheduling model for strongly coupled jobs provided by the present invention.

[0021] Figure 2 This is a schematic diagram of the directed graph of the guarantee process provided by the present invention.

[0022] Figure 3 This is a flowchart illustrating the directed graph of the transport vehicle task execution provided by the present invention.

[0023] Figure 4 This is a flowchart illustrating the inference process followed by the inference engine provided by this invention.

[0024] Figure 5 This is a schematic diagram of the training framework of the dynamic scheduling model provided by the present invention.

[0025] Figure 6 This is a flowchart illustrating the dynamic scheduling scheme generation method provided by the present invention.

[0026] Figure 7 This is a schematic diagram of the structure of the dynamic scheduling model training device for strongly coupled tasks provided by the present invention.

[0027] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0029] The terms "first," "second," etc., used in this invention are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and that the objects distinguished by "first," "second," etc., are generally of the same class.

[0030] This invention provides a dynamic scheduling model training method for strongly coupled operations. This method constructs a serial-parallel inference model comprising two coupled decision agents: aircraft maintenance support and airborne material transportation. It then trains this model using deep reinforcement learning techniques. The aim is to solve the scheduling challenges arising from the strong coupling and dynamic uncertainty between tasks in maritime aircraft support operations. This method can generate dynamic scheduling schemes with strong adaptability and stability, achieving joint scheduling optimization of aircraft maintenance support and airborne material transportation operations, thereby improving the overall efficiency of support operations. Figure 1This is a flowchart illustrating the training method for a dynamic scheduling model for strongly coupled jobs provided by the present invention, as shown below. Figure 1 As shown, the dynamic scheduling model is obtained by iteratively executing the following steps until a preset iteration condition is met: Step 110: Obtain the initial dynamic scheduling model; the initial dynamic scheduling model includes the initial maintenance support decision model and the initial airborne material transportation decision model.

[0031] Specifically, the core idea of ​​the training method proposed in this invention is to decompose a complex, strongly coupled dynamic scheduling problem into two relatively independent but mutually influential sub-problems: aircraft maintenance support and airborne material transportation. A decision model is then constructed for each sub-problem. In a shared, interactive inference environment, the two decision models alternately make decisions, observe the consequences, and obtain target rewards. The target reward signals are then used to simultaneously optimize both models, enabling them to learn to work collaboratively and ultimately achieve optimal global scheduling. This training process is iterative until preset iteration conditions are met, such as reaching a preset number of training rounds, the model's performance converging on the validation set or reaching a preset performance threshold, or the total training time reaching a preset value.

[0032] First, an initial dynamic scheduling model is obtained, which includes an initial maintenance support decision model and an initial airborne material transportation decision model. The parameters of the initial dynamic scheduling model can be preset or randomly generated; this embodiment of the invention does not impose specific limitations on this.

[0033] There are several ways to obtain the initial dynamic scheduling model. In one embodiment, obtaining the initial dynamic scheduling model may refer to instantiating the initial aircraft maintenance support decision model and the initial airborne material allocation decision model based on a preset neural network structure, such as a policy-based deep reinforcement learning network structure like the Actor-Critic structure, and initializing the parameters internal to the initial dynamic scheduling model using random values ​​or preset values, such as Xavier initialization, He initialization, etc. In another embodiment, obtaining the model may refer to loading an existing model that has already undergone preliminary training or pre-training on similar tasks, and then performing further specialized training based on this model. This approach can accelerate the convergence process of training.

[0034] The initial maintenance support decision model is an intelligent agent model used to make decisions regarding maintenance support tasks in maritime aircraft support operations. Maintenance support operations typically refer to a series of support procedures performed after an aircraft is recovered to quickly restore it to a ready-to-fly state, such as external aircraft inspection, cockpit inspection, refueling, oxygen and nitrogen replenishment, and wing folding and unfolding. The initial maintenance support decision model's responsibility is to determine, given the operational situation, which maintenance support procedures should be performed next.

[0035] The initial airborne supplies relocation decision model is an intelligent agent model used to make decisions regarding airborne supplies relocation tasks involved in aircraft maintenance and support operations. Airborne supplies relocation operations typically refer to the unloading or loading of supplies, such as pods, onto aircraft. This involves the transportation and loading / unloading of supplies from warehouses or assembly points to aircraft stands, and between different stands. The initial airborne supplies relocation decision model's responsibility is to decide how to allocate transport vehicles, plan transport routes and sequences, and complete all supplies relocation tasks with maximum efficiency, given the operational situation.

[0036] Step 120: Obtain a first state representation from the simulation environment to characterize the operational status of aircraft maintenance support, input the first state representation into the initial aircraft maintenance support decision model, and obtain the first decision output of the initial aircraft maintenance support decision model.

[0037] Specifically, a first state representation is obtained from the simulation environment to characterize the operational status of aircraft maintenance support. This first state representation is then input into the initial aircraft maintenance support decision model to obtain the first decision output of the initial aircraft maintenance support decision model.

[0038] The simulation environment is a digital simulation platform used to model real-world maritime aircraft support operations. Within the simulation environment, all key physical entities and logical rules of the operational scenario are modeled digitally. These include, for example, the attributes of the aircraft fleet, the layout and status of support positions, the quantity and constraints of various support resources (such as refueling trucks and power supply vehicles) and personnel, the attributes of transport vehicles, and the complex serial and parallel execution constraints between all support procedures. This simulation environment can receive external decision-making instructions and update its internal state accordingly, simulating the progression of time to fully reproduce the entire operational process.

[0039] Obtaining the first state representation refers to extracting all information related to aircraft maintenance support decisions from the simulation environment and organizing all such information into a preset data format, namely, the first state representation. The first state representation comprehensively reflects the overall status of the aircraft maintenance support operation at a given moment. In a specific embodiment, the first state representation can be a multi-dimensional vector or tensor. It may include the current simulation time, the status of each aircraft in the aircraft group, the status of all support resource stations (e.g., availability), and the available number of various support personnel. The status of each aircraft in the aircraft group may include its assigned position, the currently executing procedure, the remaining time for each procedure, a list of procedures to be executed, and a list of completed procedures. This embodiment of the invention does not specifically limit this.

[0040] The first state representation is provided as input to the initial maintenance support decision model. The neural network inside the initial maintenance support decision model processes and calculates this input information, and finally outputs a decision result, namely the first decision output. In one embodiment, the first decision output can be a specific action instruction, such as "arrange a refueling procedure for aircraft 101 at gate 3", or a probability distribution of actions, representing the degree of superiority or inferiority of various possible actions under the current situation, as determined by the initial maintenance support decision model. Then, sampling is performed based on this probability distribution to determine the final action to be executed.

[0041] Step 130: Update the simulation environment based on the first decision output, and obtain a second state representation from the updated simulation environment to characterize the airborne material transportation operation status. Input the second state representation into the initial airborne material transportation decision model to obtain the second decision output of the initial airborne material transportation decision model.

[0042] Specifically, the simulation environment is updated based on the first decision output, and a second state representation characterizing the airborne material handling operation status is obtained from the updated simulation environment. This second state representation is then input into the initial airborne material handling decision model to obtain the second decision output from the initial airborne material handling decision model. This step clearly demonstrates the strongly coupled, serial interaction mode between aircraft maintenance support decisions and airborne material handling decisions. First, the simulation environment receives and executes the first decision output generated by the initial aircraft maintenance support decision model. For example, if the first decision output is to start a refueling procedure for an aircraft, the state of the simulation environment will change: the aircraft's state will be updated to "refueling," and the states of the refueling truck and support personnel required to perform the procedure will also be updated to "occupied," while the simulation time will advance accordingly.

[0043] After the simulation environment is updated due to the execution of maintenance decisions, the system will obtain a second state representation from this new simulation environment to characterize the airborne material handling operation status. This second state representation reflects all information related to the airborne material handling decisions. It is particularly important to emphasize that the airborne material handling operation status is based on the latest environment after the execution of maintenance decisions, reflecting a coupling relationship. For example, if the previous maintenance decision was precisely the "material loading" procedure, then new and specific material handling requirements will be generated in the simulation environment, and these requirements will be reflected in the second state representation. Specifically, the second state representation may include the current simulation time, the target and current loaded material lists for each aircraft in the aircraft fleet, the aircraft's position information, and the real-time status of each transport vehicle. The real-time status of each transport vehicle may include its current location, carrying status, and whether it is idle; this embodiment of the invention does not specifically limit this.

[0044] Similar to the previous step, the second state representation is input into the initial airborne supplies allocation decision model. The initial airborne supplies allocation decision model will then perform calculations based on this and output the second decision output. The second decision output is the action decision made by the supplies allocation model in response to the current supplies allocation situation. For example, the second decision output could be "assign the No. 2 manual transport vehicle to the supplies warehouse to retrieve and transport two Type A supplies for aircraft No. 105".

[0045] Step 140: Use the simulation environment after executing the second decision output as the simulation environment for obtaining the first state representation, so as to obtain the target reward generated by executing the first decision output and the second decision output in the simulation environment, and update the model parameters of the initial maintenance service support decision model and the initial airborne material transportation decision model based on the target reward.

[0046] Specifically, once the second decision output is executed by the simulation environment, a complete serial simulation step of "aircraft maintenance decision - material decision - environmental evolution" is completed. The environmental state at this point is the joint result of the collaborative decision-making of the two models within this time step. This final environmental state will serve as the starting point for the next iteration, used to generate a new first state representation, thus forming a cyclical closed-loop training process.

[0047] At the end of each such closed-loop decision-making cycle, the merits of the decisions made jointly by the initial maintenance support decision-making model and the initial airborne material transportation decision-making model within this cycle must be quantitatively evaluated. The evaluation index is the target reward. The target reward is one or a set of values ​​used to reflect the extent to which the decision sequence executed by the model has achieved the preset global optimization goal.

[0048] Obtaining the target reward refers to calculating it based on one or more predefined reward functions, according to the state changes of the simulated environment before and after the decision is executed. For example, a core objective of this invention is to complete all support tasks in the shortest possible time. Therefore, the reward function can be designed to be inversely proportional to the total task completion time; that is, the faster the task is completed, the higher the target reward value. Another objective might be to use various resources in a balanced manner to avoid bottlenecks. Therefore, the target reward function can also include terms related to the load balance of various transport vehicles; the more balanced the load, the higher the reward.

[0049] In one specific embodiment, the target reward can be composed of a weighted sum of multiple sub-rewards. For example, the reward for aircraft maintenance support tasks can be a negative value for the calculated time, while the reward for airborne material handling tasks can be determined based on the completion time of the aircraft fleet, the load balancing variance of the transport vehicles, and the cumulative transfer time of the transport vehicles. By designing such a target reward, the initial aircraft maintenance support decision model and the initial airborne material handling decision model can be guided to learn how to cooperate to achieve global optimization.

[0050] Finally, based on the obtained target rewards, the system needs to update the model parameters of the initial maintenance support decision model and the initial airborne material allocation decision model. The system employs an advanced reinforcement learning algorithm, such as Proximal Policy Optimization (PPO), to calculate the loss function and updates the model parameters through backpropagation based on the gradient of the loss function with respect to the two model parameters. The basic principle of the update is: if a series of decisions leads to a high target reward, the model parameters will be adjusted in a direction that favors making such decisions in similar situations; conversely, if decisions result in a low target reward, the parameter adjustments will reduce the probability of making similar decisions in the future. Through tens of thousands or even millions of trial-and-error learning iterations, the two decision models will gradually evolve from initial random decisions into an "expert" decision-making system capable of efficient collaboration.

[0051] In one specific embodiment, in order to improve training efficiency, the above training process can be carried out in a distributed manner.

[0052] The sample generation process of distributed deep reinforcement learning is as follows: Multiple inference engines can be launched in parallel. Each engine independently interacts with the maintenance support agent and the onboard material transportation agent, thereby generating a large amount of "experience" data, i.e., initial samples, in parallel. After all the parallel inference engines have completed one round of interaction, a sample manager is responsible for collecting the initial samples generated by all engines.

[0053] The interaction process is as follows: The inference engine outputs the current environmental state data at each time step. Upon receiving this state data, the agent (whether it's a maintenance agent or a material handling agent) first processes it through a state space transformer and extracts situational features suitable for model input. These features are then input into a decision neural network model (which consists of a policy network and a value network). The agent receives the output of the decision neural network model (e.g., the probability distribution of actions and the evaluation value of the state), processes it through an action space transformer, and generates a specific environmental action. The inference engine receives this environmental action, executes it, and generates the corresponding reward and the state data for the next time step, completing one interaction loop.

[0054] The initial sample is in the form of a data tuple: ,in, Indicates the type of intelligent agent. Represents an intelligent agent for aircraft maintenance and support. Represents an intelligent entity for airborne material transportation. Indicates the current moment. Indicates the total task duration; Represents intelligent agents A The situational characteristics at time t, i.e., the feature vector output by the state-space converter; Represents intelligent agents A At time t, the action value is obtained by sampling the action distribution output by its policy network. The policy network is a deep neural network used to directly decide the output action. Represents intelligent agents A In action The immediate environmental reward value returned by the environment after execution; Represents intelligent agents A At time t, its value network reflects the characteristics of the current situation. The output evaluation value; the value network is a deep neural network used to evaluate the quality of the current state. Represents intelligent agents A At time t, its policy network exhibits the following situational characteristics. Select action The logarithmic probability.

[0055] After collecting an initial set of samples, a reward calculation method is needed to obtain the final training sample set in order to more accurately evaluate the long-term value of each action. In this embodiment, the Generalized Advantage Estimator (GAE) algorithm is used to calculate the advantage function value at each time step. The specific formula is as follows: ; in, This represents the number of time steps taken looking backward from the current time t; It is a discount factor, an adjustable coefficient with a value greater than or equal to 0 and less than or equal to 1, used to balance the importance of current rewards and future rewards; It is the smoothing factor of the GAE algorithm, which is an adjustable coefficient with a value greater than or equal to 0 and less than or equal to 1; Represents intelligent agents A At any moment The evaluation value of its value network outputting the characteristics of the current situation; Represents intelligent agents A At any moment +1 represents the evaluation value of its value network outputting the current situation characteristics.

[0056] The final training sample set is represented as follows: It has one more advantage function value than the initial sample set. .

[0057] In a specific embodiment, the training method for a distributed deep reinforcement learning model is as follows: The training sample data generated in the preceding steps is input into the training neural network model in batches. A composite loss function is used, and gradient descent (such as the Adam optimizer) is employed to calculate the gradient of the loss function with respect to the model parameters. The parameters of the training neural network model are then updated accordingly. After each round or every few rounds of updates, the updated training neural network parameters are assigned to the decision neural network model for use in the next round of sample generation.

[0058] Wherein, the loss function The calculation formula is: ; Among them, strategy loss The pruning objective function using the PPO algorithm is formulated as follows: ; in, Indicates at time Below, based on the current neural network parameters The calculated ratio of the old and new strategies, , Representation strategy exist log probability of the sample at time step This indicates that historical strategies are in The log probability of a sample at time step 1. It is the value of the dominance function calculated earlier. The function will ratio Limited to [ , Within the interval, This is a hyperparameter.

[0059] Here, strategy exist The value loss of the sample at time step. The formula used to update the value network is: ; in, Representation Strategy exist The estimated value at time, express The target value at any given time.

[0060] Through the detailed training steps described above, this invention can efficiently train a high-performance dynamic scheduling model.

[0061] Understandably, this method solves the problems of poor dynamic adaptability and difficulty in solving strongly coupled scheduling tasks. The generated scheme has strong adaptability and stability, and can realize the joint scheduling optimization of aircraft maintenance support and airborne material transportation operations, thereby improving the operational efficiency of maritime aircraft support operations.

[0062] The method provided in this embodiment of the invention obtains an initial dynamic scheduling model; the initial dynamic scheduling model includes an initial maintenance support decision model and an initial airborne material transportation decision model; a first state representation characterizing the operational status of maintenance support is obtained from the simulation environment, and the first state representation is input into the initial maintenance support decision model to obtain a first decision output; the simulation environment is updated based on the first decision output, and a second state representation characterizing the operational status of airborne material transportation is obtained from the updated simulation environment, and the second state representation is input into the initial airborne material transportation decision model to obtain a second decision output; the simulation environment after executing the second decision output is used as the simulation environment for obtaining the first state representation, so as to obtain the target reward generated by executing the first and second decision outputs in the simulation environment, and the model parameters of the initial maintenance support decision model and the initial airborne material transportation decision model are updated based on the target reward. This invention simulates the serial interaction and state influence between two decision-making stages through a shared inference environment. Then, based on a unified objective reward reflecting global performance, it collaboratively trains the initial aircraft maintenance support decision model and the initial airborne material transportation decision model. This allows the two decision models to learn how to cooperate to achieve global optimization during training, fully considering the influence of each other's decisions. This effectively handles the strong coupling between operational tasks, enabling joint scheduling optimization of aircraft maintenance support and airborne material transportation operations, thus improving the overall efficiency of maritime aircraft support operations. Simultaneously, because the trained dynamic scheduling model establishes an end-to-end mapping from real-time operational status to scheduling actions, it solves the problems of lag in response to dynamic uncertainty and low solution efficiency due to the curse of dimensionality in large-scale complex scenarios in offline planning methods.

[0063] Based on the above embodiments, the configuration steps of the simulation environment include: Configure the simulation environment based on the scenario file; The scenario file includes aircraft fleet data, transport vehicle attributes, support operation procedure attributes, support resource attributes, and support personnel set; The aircraft fleet data includes individual data tuples for each aircraft. Each individual data tuple includes the aircraft number, aircraft mission type, aircraft support position number, list of all loadout types on all loadout positions in the initial aircraft status, list of loadout types to be loaded on the aircraft, and the start time of the aircraft support mission.

[0064] Specifically, the inference environment can be configured based on a scenario file. A scenario file can be understood as a structured data file, such as XML, JSON, or a text file with a custom format. The purpose of the scenario file is to provide all the initial scenario parameters and static entity attributes for a specific training or inference task, essentially creating a detailed initial blueprint for the entire assurance operation inference task. By loading different scenario files, various different operational scenarios can be easily constructed for comprehensive training and testing of the model.

[0065] Here, the scenario file may include aircraft fleet data, transport vehicle attributes, support operation procedure attributes, support resource attributes, and support personnel sets.

[0066] Among them, aircraft fleet data This is used to define the set of all aircraft involved in this support mission and their respective initial states, where This indicates the total number of aircraft in the fleet. Indicates the first Individual data tuples of an aircraft .

[0067] Transport vehicle attributes This is used to define the type, quantity, and characteristics of all vehicles that can be used to perform airborne material handling tasks, such as distinguishing between manually operated and electrically operated pallet trucks. This is a set of vehicle numbers for manual handling vehicles. This is a set of vehicle numbers for electric pallet trucks.

[0068] Ensure the attributes of work procedures This defines a set of all possible aircraft maintenance support procedures, along with the types of resources and personnel required to perform each procedure. Different support procedures require different types of resources and personnel, represented as follows: , For process The resource and personnel requirements function.

[0069] Support resource attributes are used to define the set of various non-human resources required for support operations, such as defining the type of support resource station and the types of resources it can provide. Support Station Set Safeguard resource collection , To ensure the number of resource types; to ensure any resource station. Responsible for executing a specific type of resource The task of ensuring safety.

[0070] Support personnel assembly , To ensure the number of personnel types, the support personnel set is used to define all personnel teams that can participate in support operations, including the types and numbers of personnel with different specialties and skill levels.

[0071] To more accurately describe the status of each aircraft, aircraft fleet data is further defined as individual data tuples for each aircraft. An individual data tuple is a data structure used to describe all relevant initial information about a single aircraft. Here, an individual data tuple includes the aircraft number, aircraft mission type, aircraft's support bay number, a list of all loadout types on all loadout bays in the aircraft's initial status, a list of loadout types to be loaded onto the aircraft, and the start time of the aircraft's support mission.

[0072] For each aircraft Its attributes can be defined as .in, For the aircraft number, the range of values ​​is: ; For aircraft mission types, the value range is: These correspond to the five different sortie missions performed by the aircraft. This refers to the support stand number where the aircraft is located, and its value ranges from the set of support stands. , To ensure the number of available workstations; This is a list of all cargo types mounted on all hardpoints of the aircraft in its initial configuration. , Indicates airplane The The types of supplies mounted on each mounting slot, among which .in Represents a set of material types; This is a list of the types of supplies to be loaded onto the aircraft, indicating that... ; This marks the start time of the support mission for the aircraft.

[0073] The aircraft number is a unique identifier used to distinguish individual aircraft within the fleet, such as "Aircraft 101". The mission type indicates the category of mission the aircraft is scheduled to perform. Different mission types typically correspond to different support requirements and priorities. The support bay number indicates the specific physical location where the aircraft is parked at the start of the support mission, such as "Bay 3". The list of all hardpoints with the aircraft in its initial configuration records the types and quantities of supplies already mounted on each weapon or equipment hardpoint at the initial moment of the aircraft's return to the deck.

[0074] The aircraft's pending cargo type list defines the types and quantities of target cargo that the aircraft needs to carry for its next mission. The aircraft support mission initiation time is a preset point in time before which no support work can be performed on the aircraft. This simulates the time required for preparation work such as landing, taxiing, and securing the aircraft.

[0075] It should be noted that the ultimate goal of aircraft maintenance support operations is to complete all maintenance support procedures for the aircraft group within the shortest possible time, represented as follows: ,in, Indicates the process, Indicates the first i Aircraft completion process End time, i Indicates the aircraft's number or index.

[0076] The ultimate goal of airborne cargo handling operations is to complete the unloading and loading of cargo from the aircraft in the shortest possible time, while ensuring a balanced load on the cargo handling vehicles. This is represented as... ,in, This is a collection of unloading and mounting tasks for materials to be executed. Indicates task i Completion time; The standard deviation of the cumulative operating time of all material handling vehicles, of which For the number The cumulative operating time of the material handling vehicles. Indicates the number is j The cumulative operating time of the material handling vehicles. This indicates the total number of all material handling vehicles.

[0077] The method provided in this invention configures the simulation environment using a structured scenario file, thereby ensuring that the simulation environment can accurately and completely reproduce the initial state of a specific and complex real-world maritime aircraft pre-deployment support scenario, including the specific details of each aircraft and each type of resource. This allows the entire training process of the dynamic scheduling model to occur in a highly realistic digital twin environment, ensuring that the scheduling strategies learned by the dynamic scheduling model are targeted at specific and realistic initial conditions and constraints, rather than based on vague or overly simplified assumptions. This greatly improves the practicality and relevance of the trained dynamic scheduling model, ensuring that the generated scheduling scheme can better adapt to complex real-world situations.

[0078] Based on the above embodiments, the constraints of the initial maintenance support decision model include: The constraint condition for the coverage of the resource station with the available backup positions is used to constrain the number of backup positions that a single resource station can serve at any given time. The resource guarantee constraint is used to constrain the total demand of various types of resources at any given time. The resource guarantee constraint is determined based on the total demand of each type of resource at any given time and a preset upper limit for the quantity of each type of resource. The personnel constraint condition is used to constrain the total demand for various types of personnel at any given time. The personnel constraint condition is determined based on the total demand for each type of personnel at any given time and a preset upper limit for the number of each type of personnel. The execution constraints of the guarantee process are used to constrain the execution timing relationship between the subsequent guarantee process and the preceding guarantee process in the directed graph relationship between guarantee processes. The execution constraints of the guarantee process are determined based on the preset execution time of the preceding guarantee process, the execution time of the subsequent guarantee process, and the execution time of the preceding guarantee process. The completion time constraint for the guarantee process is used to constrain guarantee processes of different task types to have different execution times. The completion time constraint for the guarantee process is determined based on the task type to which the guarantee process belongs and the inherent attributes of the guarantee process. The start time constraint condition is used to ensure that the start time of any of the support procedures is not earlier than the start time of the aircraft support mission.

[0079] Specifically, the constraints of the initial maintenance support decision model are a series of rules that must be satisfied throughout the simulation process. These rules reflect the limitations of physical space, resource availability, personnel capabilities, and workflows in the real world.

[0080] The constraints of the initial maintenance service support decision model may include constraints on support resource station coverage of support positions, support resources, support personnel, support process execution, support process completion time, and support start time.

[0081] Here, the constraint on the coverage of support resource stations to support aircraft positions is a spatial constraint. It restricts the fact that at any given time, a single fixed support resource station (e.g., a deck-mounted well power station, oxygen station, etc.) can only provide services to a limited number of specific support aircraft positions within a finite area due to its physical location and pipeline length limitations. Simultaneously, the constraint on the coverage of support resource stations to support aircraft positions also implies exclusivity, meaning that one support resource station... Only one backup server can be served at a time. An aircraft on board. That is, the constraint condition for ensuring resource station coverage and ensuring aircraft position is used to constrain the situation at any given time. The number of backup servers that a single backup resource station can serve is expressed as .

[0082] Among them, the guaranteed resource constraint is a type of resource quantity constraint. The guaranteed resource constraint is used to constrain the quantity of resources at any given time. The simulation involves all ongoing support procedures in the environment for a specific type of support resource. The total demand for (e.g., refueling trucks, power supply vehicles, tractor units, etc.) shall not exceed the maximum quantity limit for such support resources preset in the scenario plan. , represented as .

[0083] Among them, the personnel constraint is a type of personnel quantity constraint, similar to the resource constraint. The personnel constraint is used to constrain the number of personnel at any given time. All operations require a certain type of professional support personnel The total demand for (e.g., mechanics, electronics engineers, weapons engineers, etc.) shall not exceed the preset total number of available personnel for this type. , represented as .

[0084] Here, the constraint on the execution of the support procedure is a temporal logical constraint of the work process. In maritime aircraft support operations, many procedures have a strict sequential order, which can be represented by a directed graph. This constraint is used to ensure that the start time of any subsequent support procedure in this directed graph must be later than or equal to the completion time of all its preceding support procedures. The support procedure execution constraint ensures the scientific and safe operation of the workflow; for example, aircraft inspection must be completed before refueling. That is, for any pair of process nodes with a preceding relationship in the graph... It needs to meet the following requirements. ,in, Indicates the preceding process. Indicates the subsequent process. Indicates the start time of the preceding process. Indicates the start time of the subsequent process. This indicates the time taken for the preceding process in the task.

[0085] Figure 2 This is a schematic diagram of the directed graph of the guarantee process provided by the present invention, such as... Figure 2As shown, the entire support process begins at support mission initiation node ①. Afterward, the process branches into several initial procedures that can be executed in parallel, including: visual inspection I② requiring Type I support personnel; visual inspection II④ requiring Type II support personnel; oxygen replenishment ⑥ requiring Type I support personnel and Type III support equipment; visual inspection III⑧ requiring Type III support personnel and Type II support equipment; nitrogen replenishment ⑪ requiring Type IV support personnel and Type IV support equipment; visual inspection IV⑫ requiring Type IV support personnel; and engine inspection ⑬ requiring Type IV support personnel. Some procedures have strict sequential dependencies. For example, the completion of visual inspection I② is a prerequisite for starting cabin inspection I③ (requiring Type I support personnel and Type II support equipment), visual inspection II④ is a prerequisite for cabin inspection II⑤ (requiring Type II support personnel and Type III support equipment), and visual inspection III⑧ is a prerequisite for cabin inspection III⑨ (requiring Type III support personnel and Type II and V support equipment). A key convergence node in the process is the cockpit inspection IV⑩, which requires Category IV support personnel. Its initiation must wait for the completion of all its preceding processes: cockpit inspection I③, cockpit inspection II⑤, refueling ⑦ (requiring Category IV support personnel and Category I and III support equipment), and cockpit inspection III⑨. This reflects the synchronization constraints of multiple operational lines. After cockpit inspection IV⑩ is completed, the process progresses to wing deployment ⑭ (requiring Category IV support personnel and Category I and II support equipment), which then allows for cargo loading ⑮, requiring Category III support personnel. After cargo loading is completed, wing folding ⑯ is performed by Category IV support personnel and Category I and II support equipment. Finally, all parallel main processes converge at the inertial navigation alignment ⑰ node (requiring Category II support personnel and Category II support equipment). The execution of this step depends not only on the completion of wing folding ⑯ but also on the completion of nitrogen refueling ⑪, external inspection IV⑫, and engine inspection ⑬ in the initial parallel branches. After the inertial navigation system is aligned (17), the entire process finally reaches the support mission completion (18) node, marking the end of all maintenance support operations for an aircraft.

[0086] The completion time constraint for a support procedure is a type of operation time constraint. It is used to constrain different support procedures, or the same support procedure applied to aircraft of different mission types, to have different execution times. For example, performing an armament check on an aircraft that has performed a combat mission may take longer than on an aircraft performing a reconnaissance mission. This constraint makes the simulation more realistic because the procedure time is not constant. In other words, the completion time constraint for a support procedure is used to constrain support procedures of different mission types to have different execution times. The completion time constraint for a support procedure is determined based on the mission type to which the support procedure belongs and the inherent attributes of the support procedure itself, and can be expressed as follows: .

[0087] The assurance start time constraint is a task start time constraint. It ensures that the start time of any assurance procedure must not be earlier than the start time of the overall aircraft assurance task to which the assurance procedure belongs, i.e., the aircraft assurance task start time defined in the aforementioned individual data tuples, which can be expressed as: ,in, Indicates the first i The first maintenance procedure for an aircraft The actual start time of execution, Indicates the first i The planned support mission for the aircraft was launched at that moment.

[0088] The method provided in this invention comprehensively integrates various real-world physical and logical constraints, such as the spatial coverage of resources, the upper limit of their quantity, the upper limit of the number of personnel, the strict execution sequence of support procedures, differentiated completion times, and task start times, into the simulation environment. This ensures that the decision-making model must operate within the framework of these rigid rules at every step of its training, fundamentally preventing the generation of any unrealistic scheduling schemes that do not conform to physical laws or operational procedures. Consequently, the scheduling strategies learned by the dynamic scheduling model inherently meet the high complexity and constraints of real-world operations, ensuring that the final output scheduling scheme is feasible and executable. This greatly improves the practicality and reliability of the schemes generated by the trained dynamic scheduling model, ensuring that the scheduling results can be directly applied to real work processes and avoiding the problem of schemes being unexecutable in reality due to the neglect of key constraints.

[0089] Based on the above embodiments, the constraints of the initial airborne material transportation decision model include: The loading point operation sequence constraint is used to constrain the execution order of material unloading and loading operations at the same loading point. The loading point operation sequence constraint is determined based on the end time of the unloading event and the start time of the loading event at the loading point. The load-bearing type constraint condition of the transport vehicle is used to restrict a single transport vehicle to carry only one type of material in one trip; The transport vehicle scheduling priority constraint is used to determine the priority order of the execution tasks of the transport vehicles to be scheduled when there are multiple transport vehicles to be scheduled. The transport vehicle scheduling priority constraint is determined based on the current loading matching status and the historical cumulative number of dispatches of each transport vehicle to be scheduled. The task stage constraints of the transport vehicle are used to constrain the sequence of sub-tasks that the transport vehicle must follow when performing different types of material transportation tasks. The task stage constraints of the transport vehicle are determined based on a preset directed graph, which includes multiple sub-task stage nodes.

[0090] Specifically, the constraints of the initial airborne material transportation decision model include the loading point operation sequence constraints, the transport vehicle carrying type constraints, the transport vehicle scheduling priority constraints, and the transport vehicle task stage constraints.

[0091] The loading point operation sequence constraint is a timing logic constraint for a single aircraft loading point. This constraint ensures that unloading and loading operations at the same loading point cannot occur simultaneously, and must adhere to the principle of "unloading first, loading later." Specifically, this constraint requires that the start time of a loading event at a loading point must be greater than or equal to the end time of an unloading event at that loading point, expressed by the formula: ,in Indicates airplane i The j One mount point, Indicates the end time of the unmount event at the mount point. Indicates the start time of the unmount event at the mount point.

[0092] The constraint on the type of materials carried by the transport vehicle is a single-type constraint. This constraint ensures that any transport vehicle, during a single transport mission, can only carry one type of material. For example, a transport vehicle cannot simultaneously carry a type A missile and a type B pod. Specifically, for any two materials i and j in the set of materials carried by any transport vehicle v, their material type indication parameters must satisfy… ,in This indicates that the transport vehicle (v) is carrying supplies. i type This indicates that the transport vehicle (v) is carrying supplies. j The type.

[0093] Here, the vehicle scheduling priority constraint is a heuristic rule constraint for resource scheduling, designed to improve scheduling efficiency. The vehicle scheduling priority constraint is used to determine which vehicle should be assigned first when multiple vehicles are available to perform a new task. This priority determination considers two factors: the vehicle's current loading status and its historical cumulative dispatch count.

[0094] The task stage constraint for the transport vehicle is a sub-process constraint for a complex task. It constrains the transport vehicle to follow a pre-defined, fixed sequence of multiple sub-task stages when performing certain complex material handling tasks (e.g., unloading / loading at two different loading points). This fixed sequence is defined by a pre-defined directed graph, which includes multiple sub-task stage nodes and the temporal relationships between them.

[0095] Figure 3 This is a flowchart illustrating the directed graph of the transport vehicle task execution provided by the present invention, as shown below. Figure 3 As shown, for example, when a transport vehicle receives a task to load supplies to two loading points, it must sequentially execute the following sub-task sequence: ① Standby at base -> ② Load supplies at base -> ③ Move from base to loading point -> ④ Install supplies at loading point -> ⑥ Move from one loading point to another -> ④ Install supplies at loading point -> ⑦ Move from loading point to base -> ① Standby at base. This ensures the standardization and correctness of complex task flows.

[0096] In one specific embodiment, the task phase constraints of the transport vehicle are defined by defining a directed graph of the transport vehicle. This indicates that the task of the transport vehicle is... For any pair of task nodes in the graph that have a predecessor relationship It needs to meet the following requirements. When a transport vehicle receives a task to load onto two attachment points, it will execute the tasks in stages ①②③④⑥④⑦① of the directed graph in sequence; when a transport vehicle receives a task to unload from two attachment points, it will execute the tasks in stages ①③⑤⑥⑤⑦⑧① of the directed graph in sequence; when a transport vehicle receives a task to load onto one attachment point, it will execute the tasks in stages ①②③④⑦① of the directed graph in sequence; when a transport vehicle receives a task to unload from one attachment point, it will execute the tasks in stages ①③⑤⑦⑧① of the directed graph in sequence.

[0097] In addition, the constraints of the initial airborne material handling decision model also include constraints on the handling vehicles and the operation time during the airborne material handling operation. The constraint on the handling vehicles during the airborne material handling operation is that any type of handling vehicle has a maximum carrying capacity for different types of materials; the handling vehicles are defined as manually operated handling vehicles. and electric pallet truck There are two types, and the constraints are represented as follows: , ,in, For manual transport vehicles Maximum carrying capacity Electric pallet truck Maximum carrying capacity.

[0098] The time constraints during airborne material handling operations are as follows: different handling vehicles require different handling times for different types of materials, which is represented as follows: .

[0099] Based on the above embodiments, the step of determining the priority constraint for transport vehicle scheduling includes: Step 210: Based on the current loading matching situation, determine the first priority weight; wherein, the first priority weight of the first dispatchable transport vehicle is higher than the first priority weight of the second dispatchable transport vehicle, the first dispatchable transport vehicle is a vehicle whose current loading situation matches the material transportation operation to be performed, and the second dispatchable transport vehicle is a vehicle whose current loading situation does not match the material transportation operation to be performed. Step 220: Determine the second priority weight based on the historical cumulative dispatch count; wherein, the second priority weight of the third dispatchable transport vehicle is higher than the second priority weight of the fourth dispatchable transport vehicle, and the historical cumulative dispatch count of the third dispatchable transport vehicle is less than the historical cumulative dispatch count of the fourth dispatchable transport vehicle. Step 230: Determine the scheduling priority constraints of the transport vehicle based on the first priority weight and the second priority weight.

[0100] Specifically, firstly, a first priority weight is determined based on the current loading matching status. The current loading matching status reflects whether the type of materials currently loaded on a transport vehicle matches the type of materials required by the new material transfer task to be executed.

[0101] The rule for determining the first priority weight is that if a transport vehicle awaiting dispatch (hereinafter referred to as the first transport vehicle awaiting dispatch) is currently loaded with materials that are exactly needed for the new task, then its first priority weight is significantly higher than that of another transport vehicle that is currently empty or loaded with unrelated materials (hereinafter referred to as the second transport vehicle awaiting dispatch). This avoids unnecessary back-and-forth loading and unloading processes at the base, greatly improving efficiency.

[0102] Secondly, the weight of the second priority is determined based on the historical cumulative dispatch count. The historical cumulative dispatch count records the total number of times each transport vehicle has been dispatched since the start of the task.

[0103] The rule for determining the second priority weight is that, given the same first priority, a transport vehicle with fewer historical deployments (hereinafter referred to as the third transport vehicle to be dispatched) will have a higher second priority weight than a transport vehicle with more historical deployments (hereinafter referred to as the fourth transport vehicle to be dispatched). This is done to balance the workload of each vehicle, avoid overuse of some vehicles while others are idle, thereby extending the lifespan of the entire fleet and improving the robustness of the system.

[0104] Finally, based on the first priority weight and the second priority weight, the final transport vehicle is determined. Scheduling priority constraints. Specifically, priorities can be... Represented as a formula ,in The current loading matching status (1 for a match, 0 for no match) corresponds to the first priority weight; The cumulative number of deployments in history corresponds to a negative impact on the second priority weight; It is a constant that is much larger than the maximum possible number of sorties. , used to ensure The matching situation is the absolute priority determining factor, that is... It is the highest priority.

[0105] The method provided in this invention is based on modeling more refined constraints such as the timing of loading point operations, the type of transport vehicle, and a dynamic scheduling priority that balances efficiency and load balancing in airborne material handling operations. This allows the decision space faced by the airborne material handling decision model during training to more closely resemble the complex constraints and optimization techniques of real operations. Consequently, the dynamic scheduling model can not only learn basic strategies for completing tasks but also learn advanced scheduling intelligence such as how to balance equipment usage. This significantly improves the intelligence level of airborne material handling decision-making and the precision of the scheduling scheme, making the final material handling process more efficient, reasonable, and balanced.

[0106] In a specific embodiment, to transform the dynamic scheduling model training method for strongly coupled operations into an executable computer simulation process, this invention further discloses a specific method for constructing a Markov decision process for maritime aircraft support operations. The core of this method is to construct a simulation engine that combines temporal and task-based advancement. This simulation engine is the cornerstone of the entire reinforcement learning training framework, responsible for simulating the dynamic evolution of support operations. The construction of the simulation engine is simultaneously geared towards aircraft maintenance support tasks and airborne material handling and scheduling tasks, serving as key elements for constructing their respective Markov decision processes. Within the engine, the passage of time is decomposed into two independent logical clocks: the temporal advancement of the aircraft maintenance support task is achieved through simulation time steps. Within the time range The internal increase is achieved; while the timing of airborne material transport missions is advanced through time-step simulation. Within the time range The addition within is implemented.

[0107] Figure 4 This is a flowchart illustrating the inference process followed by the inference engine provided by this invention, as shown below. Figure 4As shown, the process begins at the maintenance support start node, first completing engine initialization based on externally input task initialization data. After initialization, the engine enters a main loop, the core of which is the judgment of "Is there an executable aircraft?", that is, determining whether there is currently an aircraft available for support operations based on various constraints. If the judgment is "no", it means that there are currently no executable maintenance procedures, and the engine will perform a timing-advanced execution operation, that is, only advance the simulation time, and then return to continue judging; if the judgment is "yes", it means that there is a schedulable opportunity, at which point the process enters the maintenance support decision stage, that is, selecting the maintenance support procedure to be executed. This step corresponds to the output decision of the maintenance support decision model. Subsequently, the process uses the key judgment node of "Is this procedure an onboard material support procedure?" to reflect the strong coupling relationship between maintenance support and material transportation. If the process is not an airborne supplies support process (judged as "No"), it indicates that this is an independent maintenance process. The process will directly execute the process advancement for this process, and after completion, return to the beginning of the main loop to re-determine if there are any executable aircraft. If the process is an airborne supplies support process (judged as "Yes"), it indicates that the maintenance decision triggered a supplies allocation requirement. The process will then add the new process task to the list of tasks to be executed. Next, it enters the supplies allocation sub-process. First, it determines if there are any executable supplies allocation actions. If there are no available actions (judged as "No"), it executes the timing advancement of the supplies allocation logic clock and waits until an executable action appears. If there are available actions (judged as "Yes"), it enters the supplies allocation decision stage, that is, selects the airborne supplies allocation process to be executed. This step corresponds to the output decision of the airborne supplies allocation decision model, and performs process advancement for it. After the supplies allocation process advances, the process will check the completion status of the supplies task by judging whether there are any completed supplies allocation processes. At any given moment, regardless of whether the material transportation process is complete, the process will then check whether the entire support mission is finished by determining whether all aircraft maintenance support procedures have been completed. If not all are completed (judged as "No"), the entire process returns to the starting point of the main loop, i.e., "Are there any available aircraft?", and continues to the next round of scheduling decisions; if all tasks have been completed (judged as "Yes"), the process ends, reaching the aircraft maintenance support completion node, marking the completion of a complete dynamic scheduling scheme simulation.

[0108] Specifically, the inference engine follows the following inference process: S101 Task Initialization: The system first obtains task initialization data, including the time step. For example, by loading the scenario file and performing internal state initialization settings for the scheduling engine, all time steps, such as and Set to 0.

[0109] S102 Aircraft Availability Assessment: Based on the constraints of maintenance support operations, the engine determines whether there is at least one aircraft available to perform a specific support task. If not, proceed to S103; if so, proceed to S104.

[0110] S103 Aircraft Maintenance Support Sequence Advancement: When there are no executable aircraft maintenance procedures, the system advances the logical clock of the aircraft maintenance support task, i.e., updates the time step. Then return to S102 to continue the judgment until a usable aircraft appears.

[0111] S104 Aircraft Maintenance Support Procedure Decision: This step corresponds to the aircraft maintenance decision-making process. The engine provides the current state (represented by the first state) to the aircraft maintenance support decision model. The model selects a current aircraft maintenance support procedure to be executed, and the engine executes the process, such as updating the procedure status and resource usage.

[0112] S105 Process Attribute Recognition: The engine determines whether the process executed in S104 belongs to the "Airborne Material Transportation Support Process" (e.g., material loading or unloading). If yes, it means that the maintenance decision triggered a new material transportation requirement, and proceeds to S106; if no, it means that the process does not involve material transportation, and the material transportation clock is directly advanced, proceeding to S108.

[0113] S106 Task Enlistment: The engine adds the material transportation requirements corresponding to the new process identified in S105 as a new task to the list of pending tasks for material transportation, and then proceeds to S107.

[0114] S107 Material Transfer Judgment: Based on the various constraints of the airborne material transfer operation, the engine determines whether there is an executable material transfer operation. If yes, proceed to S108 to make a material transfer decision; otherwise, directly advance to S109.

[0115] S108 Material Allocation Decision: This step corresponds to the material allocation decision-making process. The engine provides the current state (represented by the second state) to the material allocation decision-making model. The model selects an onboard material allocation procedure to be executed, and the engine then proceeds with the execution of that procedure, updating the time step. Then we entered S109.

[0116] S109 Airborne Supplies Transfer Completion Status Judgment: The engine determines whether any aircraft's supplies transfer procedures have been completed. If yes, it means the main maintenance support process can continue, and the system returns to S104 to make subsequent maintenance support procedure decisions; if no, it indicates that supplies transfer is still in progress, and the system proceeds to S110 to determine whether the entire mission is complete.

[0117] S110 Mission Completion Judgment: Based on the final goal of the aircraft maintenance support operation (e.g., all aircraft support procedures have been completed), the engine determines whether the entire support process is complete. If not, it means there are still aircraft that have not completed their tasks, and the process returns to S102 to continue looping to determine the status of available aircraft; if yes, the aircraft maintenance support process is determined to be complete, and a complete simulation is finished.

[0118] Furthermore, the method for constructing the key elements of a Markov Decision Process (MDP) for aircraft maintenance support tasks is elaborated in detail. A Markov Decision Process (MDP) is the mathematical foundation of reinforcement learning, and its key elements include a state space, an action space, and a reward function. The specific method in this embodiment is as follows: based on the constraints in the aircraft maintenance support task process, the state space of the aircraft maintenance support task is constructed. Action space and reward function .

[0119] Here, the status space for aircraft maintenance and support tasks. Defined as It contains all the information needed to make maintenance decisions, including the action space. The first state space specifically includes: Aircraft maintenance support mission time steps ; Real-time aircraft fleet status This is a collection where each element The type of aircraft i is described in detail. Affiliated camera position Current process Remaining time for the process Resource allocation guarantee station List of procedures to be executed List of completed processes .

[0120] Time Resource Station Status This is a set , indicating each resource site Is it currently available?

[0121] Always ensure the condition of personnel This is a set , representing the current available number of people of each type p.

[0122] Executable process set ,in This represents a list of executable processes selected from the candidate processes, where the resources and personnel meet the execution conditions.

[0123] Among them, the operational space of aircraft maintenance support tasks Represented as: for an aircraft j with a task to be performed, the set of executable procedure numbers. .

[0124] Aircraft maintenance support task reward function Designed to be time-consuming for deduction, i.e. (Usually negative values ​​are used because the goal of reinforcement learning is to maximize the reward, which corresponds to minimizing the time spent).

[0125] Based on the above embodiments, the initial maintenance service support decision model includes a first state space converter, a first neural network model, and a first action space converter; the first state space converter is used to encode the first state representation into a first situation feature vector, the first neural network model is used to output a first action value vector based on the first situation feature vector, and the first action space converter is used to convert the first action value vector into the first decision output. The initial airborne material transportation decision model includes a second state space converter, a second neural network model, and a second action space converter. The second state space converter is used to encode the second state representation into a second situation feature vector. The second neural network model is used to output a second action value vector based on the second situation feature vector. The second action space converter is used to convert the second action value vector into the second decision output.

[0126] Specifically, the initial maintenance service support decision model is concretized into three core modules: the first state space converter, the first neural network model, and the first action space converter.

[0127] Here, the function of the first state-space converter is to encode and process the raw and complex first-state representation obtained from the simulation environment, converting it into a uniformly formatted, information-dense numerical vector, namely the first situation feature vector. This conversion process may include normalization, embedding, and other operations on different types of data (such as discrete aircraft states and continuous time values) to facilitate processing by the neural network model.

[0128] The first neural network model is the core of the decision-making process. It receives a first situation feature vector as input, performs forward propagation calculations through its internal multi-layer network structure, and finally outputs a first action value vector. The first action value vector can represent the value assessment or selection probability of all possible actions.

[0129] The function of the first action space converter is to parse the first action value vector and convert it into specific instructions that the inference environment can understand and execute, i.e., the first decision output. This process may include sampling an action from a probability distribution, or selecting the optimal action based on value assessment, and performing necessary format conversions.

[0130] Similarly, the initial airborne material transportation decision model also includes three corresponding modules: the second state space converter, the second neural network model, and the second action space converter. Their functions are similar to the three modules of the above-mentioned aircraft maintenance decision model, but they process the second state representation related to airborne material transportation and output the second decision output.

[0131] Here, both the first and second neural network models include an input layer, a hidden layer, and an output layer. The input layer consists of a fully connected layer, and its input is a situation feature vector. The hidden layer consists of a long short-term memory structure. The output layer is divided into two categories: discrete action value vector output and evaluation value output, each consisting of two fully connected layers. The discrete action value vector is the probability distribution weight of all candidate actions in the action space. The evaluation value is used to evaluate the discrete action value vector decided by the neural network.

[0132] Here, the first state-space converter and the second state-space converter are used to transform the discrete action value vector output by the neural network model into executable actions. or The filtering process ultimately yields the actions based on the current physical and logical constraints.

[0133] Based on the above embodiments, the second state represents the mission time step of airborne material transportation, the aircraft group loading status and the transport vehicle status corresponding to the mission time step; The status of the transport vehicle includes the transport vehicle type, the target mounting point of the transport vehicle, the current stage of the current operation task of the transport vehicle, the type of material being carried, the quantity of material being carried, and the number of times it has been dispatched in the past. The target reward is determined based on a preset optimization target; the optimization target is determined based on the completion time of the aircraft fleet, the load balancing variance of the transport vehicle, and the cumulative transfer time of the transport vehicle.

[0134] Specifically, the state space of airborne material transport mission (second state space). The motion space is The second state space specifically includes: Airborne supplies transport mission timeline .

[0135] Real-time aircraft fleet status This includes the current load list, the assigned hangar position, and the target load list for each aircraft i.

[0136] Real-time transport vehicle status ,in, This indicates the type of transport vehicle, v. Indicates the target mounting point of the transport vehicle. Indicates the starting position. Indicates the end position. This indicates the current stage of the current task being performed by the transport vehicle. Indicates the remaining time in the current stage. Indicates the type of goods being carried. Indicates the quantity of goods carried. Indicates the number of historical deployments. Indicates the duration of movement.

[0137] A set of executable material loading procedures This represents the list of executable actions generated under the current state after determining the aircraft's needs, the status of the transport vehicle, and constraints. ,in This represents a list of executable actions generated in the action space by judging the loading and unloading requirements of the aircraft mounting points and the current status of the transport vehicles. The number of vehicles required for each mounting scheme is calculated based on the load constraints of the transport vehicles and the task constraints. Actions that do not meet the current vehicle quantity requirements are considered as non-executable actions.

[0138] Among them, the operational space of airborne material transport missions This is constructed as a list of loading schemes. Based on the types and quantities of materials that different types of transport vehicles can carry, the motion space is constructed as a set of loading schemes oriented towards the loading and unloading task requirements of the loading points. The mounting scheme Given a list of length 8, the first 4 elements represent the unloading requirements of the mounting points, and the last 4 elements represent the loading requirements of the mounting points. The transport vehicles are defined as follows: there are 6 manually operated transport vehicles M and 3 electrically powered transport vehicles E. Each mounting point is assigned one type of transport vehicle for unloading or loading. This problem is transformed into a permutation and combination problem. There are 8 mounting positions, and each position is assigned either a transport vehicle M or E. The total number of mounting schemes is: .

[0139] Here, the reward function for the airborne supplies transport mission is designed as a multi-objective optimization function, namely... The weighting coefficient The goal is to minimize three objectives simultaneously: It refers to the completion time of the aircraft fleet. It is the variance of the load balancing of the transport vehicle. This refers to the cumulative transfer time using the transport vehicle. Among these... This indicates the number of times the transport vehicle v performs the transport task. This represents the average number of times all transport vehicles perform transport tasks. This indicates the completion time of aircraft j. This indicates the transfer time of the transport vehicle v.

[0140] The method provided in this invention constructs a structured and extremely detailed state space containing information such as time, entity status, and available actions, as well as an action space corresponding to actual decisions and a reward function aimed at achieving multi-objective optimization, based on two sub-problems: aircraft maintenance and material transportation. This successfully formalizes and mathematizes a complex, unstructured scheduling problem into a standard Markov decision process, making it possible to solve the problem using advanced deep reinforcement learning algorithms, which is a core prerequisite for achieving intelligent scheduling.

[0141] Based on any of the above embodiments Figure 5 This is a schematic diagram of the training framework of the dynamic scheduling model provided by the present invention, as shown below. Figure 5As shown, the core of this framework lies in decoupling the entire training process into two collaborative phases: the sample generation phase and the model training phase. The entire process begins with the scenario setup at the bottom, where the user or system can configure the simulation environment and, after training starts, inject the initial / terminal situation of the scenario into the training control module. In the sample generation phase, a core training control module coordinates two parallel decision-making units: support and materials. The support unit contains an agent that processes state information (via feature / output transformation) obtained from a shared inference engine through its maintenance support decision neural network model and generates action feedback to the engine, which then performs simulation after receiving the parameter configuration. Crucially, the state / action output of this inference engine is passed as input to the materials unit, reflecting the driving force and influence of maintenance decisions on materials transportation tasks. The materials unit also contains an agent and its onboard materials transportation decision neural network model, which makes decisions through its own feature / output loop after receiving upstream state. The empirical data generated by these two units during their interaction is packaged into initial samples and sent to the sample storage section of the sample management module on the right. Subsequently, the reward calculation module post-processes these stored initial samples (e.g., calculates the advantage function) to generate the final training samples. These training samples are then fed into the model training phase, where the maintenance support training neural network model and the airborne material handling training neural network model use these samples for learning and parameter optimization. After training is complete, the updated model parameters are sent back to the sample generation phase to update the decision neural network models in the two decision units, thus forming a complete closed-loop learning process. By continuously generating samples and training models, the agent's decision-making ability is continuously and iteratively enhanced.

[0142] After training to obtain the dynamic scheduling model, the dynamic scheduling model can be used to generate dynamic scheduling schemes.

[0143] Based on any of the above embodiments, the present invention provides a method for generating dynamic scheduling schemes. Figure 6 This is a flowchart illustrating the dynamic scheduling scheme generation method provided by the present invention, as shown below. Figure 6 As shown, the method includes: Step 610: Obtain the initial status of the maritime aircraft support operations to be scheduled; Step 620: Based on the dynamic scheduling model, the initial situation is deduced to obtain a dynamic scheduling scheme; The dynamic scheduling model is obtained by performing the training method described above.

[0144] Specifically, firstly, the initial situation of the maritime aircraft support operation to be scheduled can be obtained. This initial situation describes the overall state of the maritime aircraft support operation scenario at the start of the scheduling plan. The initial situation can include the actual status of all entities related to scheduling at that moment.

[0145] In a specific application scenario, the methods for obtaining the initial situation can be diverse. For example, on-site command personnel can manually input information such as the status of each aircraft on the deck, available resources, and personnel status into a dispatch system via a user interface. In more advanced systems, this status information can be automatically and in real-time acquired by interfacing with the ship's integrated monitoring system and automated deck management system. The data structure and content of the initial situation are completely consistent with the state representations used in the aforementioned training methods, such as the first and second state representations, to ensure that the dynamic dispatch model can correctly understand the input information. For example, the initial situation also needs to include aircraft group data, resource status, and personnel status, and its data format must be compatible with the format defined in the scenario file.

[0146] After obtaining the initial situation, a dynamic scheduling scheme can be derived by extrapolating the initial situation based on the dynamic scheduling model. The dynamic scheduling model is obtained by executing the training method described above.

[0147] The method provided in this invention obtains the initial situation of the maritime aircraft support operation to be scheduled; the initial situation is deduced based on the dynamic scheduling model to obtain a dynamic scheduling scheme; the dynamic scheduling model is obtained by executing the training method of the above-mentioned dynamic scheduling model, thereby greatly improving the efficiency of command and dispatch and the overall effectiveness of the operation process.

[0148] The following describes the training apparatus for a dynamic scheduling model for strongly coupled jobs provided by the present invention. The training apparatus for a dynamic scheduling model for strongly coupled jobs described below can be referred to in correspondence with the training method for a dynamic scheduling model for strongly coupled jobs described above.

[0149] Based on any of the above embodiments, the present invention provides a dynamic scheduling model training device for strongly coupled tasks. Figure 7 This is a schematic diagram of the structure of the dynamic scheduling model training device for strongly coupled tasks provided by the present invention, as shown below. Figure 7 As shown, the device includes: The acquisition unit 710 is used to acquire the initial dynamic scheduling model; the initial dynamic scheduling model includes the initial maintenance support decision model and the initial airborne material transportation decision model. The first input unit 720 is used to obtain a first state representation from the simulation environment to characterize the operational status of aircraft maintenance support, and input the first state representation into the initial aircraft maintenance support decision model to obtain the first decision output of the initial aircraft maintenance support decision model. The second input unit 730 is used to update the simulation environment based on the first decision output, and obtain a second state representation from the updated simulation environment to characterize the airborne material transportation operation status, and input the second state representation into the initial airborne material transportation decision model to obtain the second decision output of the initial airborne material transportation decision model. The training unit 740 is used to take the inference environment after executing the second decision output as the inference environment to obtain the first state representation, so as to obtain the target reward generated by executing the first decision output and the second decision output in the inference environment, and update the model parameters of the initial maintenance service support decision model and the initial airborne material transportation decision model based on the target reward.

[0150] The apparatus provided in this embodiment of the invention acquires an initial dynamic scheduling model; the initial dynamic scheduling model includes an initial maintenance service support decision model and an initial airborne material transportation decision model; it acquires a first state representation characterizing the operational status of maintenance service support from a simulation environment, inputs the first state representation into the initial maintenance service support decision model to obtain a first decision output; it updates the simulation environment based on the first decision output, and acquires a second state representation characterizing the operational status of airborne material transportation from the updated simulation environment, inputs the second state representation into the initial airborne material transportation decision model to obtain a second decision output; it uses the simulation environment after executing the second decision output as the simulation environment for acquiring the first state representation, in order to obtain the target reward generated by executing the first and second decision outputs in the simulation environment, and updates the model parameters of the initial maintenance service support decision model and the initial airborne material transportation decision model based on the target reward. This invention simulates the serial interaction and state influence between two decision-making stages through a shared inference environment. Then, based on a unified objective reward reflecting global performance, it collaboratively trains the initial aircraft maintenance support decision model and the initial airborne material transportation decision model. This allows the two decision models to learn how to cooperate to achieve global optimization during training, fully considering the influence of each other's decisions. This effectively handles the strong coupling between operational tasks, enabling joint scheduling optimization of aircraft maintenance support and airborne material transportation operations, thus improving the overall efficiency of maritime aircraft support operations. Simultaneously, because the trained dynamic scheduling model establishes an end-to-end mapping from real-time operational status to scheduling actions, it solves the problems of lag in response to dynamic uncertainty and low solution efficiency due to the curse of dimensionality in large-scale complex scenarios in offline planning methods.

[0151] Based on any of the above embodiments, a configuration unit is further included, wherein the configuration unit is specifically used for: Configure the simulation environment based on the scenario file; The scenario file includes aircraft fleet data, transport vehicle attributes, support operation procedure attributes, support resource attributes, and support personnel set; The aircraft fleet data includes individual data tuples for each aircraft. Each individual data tuple includes the aircraft number, aircraft mission type, aircraft support position number, list of all loadout types on all loadout positions in the initial aircraft status, list of loadout types to be loaded on the aircraft, and the start time of the aircraft support mission.

[0152] Based on any of the above embodiments, the constraints of the initial maintenance support decision model include: The constraint condition for the coverage of the resource station with the available backup positions is used to constrain the number of backup positions that a single resource station can serve at any given time. The resource guarantee constraint is used to constrain the total demand of various types of resources at any given time. The resource guarantee constraint is determined based on the total demand of each type of resource at any given time and a preset upper limit for the quantity of each type of resource. The personnel constraint condition is used to constrain the total demand for various types of personnel at any given time. The personnel constraint condition is determined based on the total demand for each type of personnel at any given time and a preset upper limit for the number of each type of personnel. The execution constraints of the guarantee process are used to constrain the execution timing relationship between the subsequent guarantee process and the preceding guarantee process in the directed graph relationship between guarantee processes. The execution constraints of the guarantee process are determined based on the preset execution time of the preceding guarantee process, the execution time of the subsequent guarantee process, and the execution time of the preceding guarantee process. The completion time constraint for the guarantee process is used to constrain guarantee processes of different task types to have different execution times. The completion time constraint for the guarantee process is determined based on the task type to which the guarantee process belongs and the inherent attributes of the guarantee process. The start time constraint condition is used to ensure that the start time of any of the support procedures is not earlier than the start time of the aircraft support mission.

[0153] Based on any of the above embodiments, the constraints of the initial airborne material transportation decision model include: The loading point operation sequence constraint is used to constrain the execution order of material unloading and loading operations at the same loading point. The loading point operation sequence constraint is determined based on the end time of the unloading event and the start time of the loading event at the loading point. The load-bearing type constraint condition of the transport vehicle is used to restrict a single transport vehicle to carry only one type of material in one trip; The transport vehicle scheduling priority constraint is used to determine the priority order of the execution tasks of the transport vehicles to be scheduled when there are multiple transport vehicles to be scheduled. The transport vehicle scheduling priority constraint is determined based on the current loading matching status and the historical cumulative number of dispatches of each transport vehicle to be scheduled. The task stage constraints of the transport vehicle are used to constrain the sequence of sub-tasks that the transport vehicle must follow when performing different types of material transportation tasks. The task stage constraints of the transport vehicle are determined based on a preset directed graph, which includes multiple sub-task stage nodes.

[0154] Based on any of the above embodiments, a constraint determination unit is further included, wherein the constraint determination unit is specifically used for: Based on the current loading matching situation, a first priority weight is determined; wherein, the first priority weight of the first dispatchable transport vehicle is higher than the first priority weight of the second dispatchable transport vehicle, the first dispatchable transport vehicle is a vehicle whose current loading situation matches the material transportation operation to be performed, and the second dispatchable transport vehicle is a vehicle whose current loading situation does not match the material transportation operation to be performed. Based on the historical cumulative dispatch count, a second priority weight is determined; wherein, the second priority weight of the third dispatchable transport vehicle is higher than that of the fourth dispatchable transport vehicle, and the historical cumulative dispatch count of the third dispatchable transport vehicle is less than that of the fourth dispatchable transport vehicle. Based on the first priority weight and the second priority weight, the scheduling priority constraints of the transport vehicle are determined.

[0155] Based on any of the above embodiments, the initial maintenance service support decision model includes a first state space converter, a first neural network model, and a first action space converter; the first state space converter is used to encode the first state representation into a first situation feature vector, the first neural network model is used to output a first action value vector based on the first situation feature vector, and the first action space converter is used to convert the first action value vector into the first decision output. The initial airborne material transportation decision model includes a second state space converter, a second neural network model, and a second action space converter. The second state space converter is used to encode the second state representation into a second situation feature vector. The second neural network model is used to output a second action value vector based on the second situation feature vector. The second action space converter is used to convert the second action value vector into the second decision output.

[0156] Based on any of the above embodiments, the second state represents the task time step of airborne material transportation, the aircraft group loading status and the transport vehicle status corresponding to the task time step; The status of the transport vehicle includes the transport vehicle type, the target mounting point of the transport vehicle, the current stage of the current operation task of the transport vehicle, the type of material being carried, the quantity of material being carried, and the number of times it has been dispatched in the past. The target reward is determined based on a preset optimization target; the optimization target is determined based on the completion time of the aircraft fleet, the load balancing variance of the transport vehicle, and the cumulative transfer time of the transport vehicle.

[0157] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8 As shown, the electronic device may include: a processor 810, a communications interface 820, a memory 830, and a communications bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other through the communications bus 840. The processor 810 can call logical instructions in the memory 830 to execute a dynamic scheduling model training method for strongly coupled operations. This method includes: acquiring an initial dynamic scheduling model; the initial dynamic scheduling model includes an initial maintenance support decision model and an initial airborne material transportation decision model; acquiring a first state representation from a simulation environment to characterize the status quo of maintenance support operations, inputting the first state representation into the initial maintenance support decision model to obtain a first decision output from the initial maintenance support decision model; updating the simulation environment based on the first decision output, and acquiring a second state representation from the updated simulation environment to characterize the status quo of airborne material transportation operations, inputting the second state representation into the initial airborne material transportation decision model to obtain a second decision output from the initial airborne material transportation decision model; using the simulation environment after executing the second decision output as the simulation environment for acquiring the first state representation, to obtain a target reward generated by executing the first and second decision outputs in the simulation environment, and updating the model parameters of the initial maintenance support decision model and the initial airborne material transportation decision model based on the target reward.

[0158] The processor 810 can also call logical instructions in the memory 830 to execute a dynamic scheduling scheme generation method, which includes: acquiring the initial situation of the maritime aircraft support operation to be scheduled; deducing the initial situation based on the dynamic scheduling model to obtain a dynamic scheduling scheme; wherein the dynamic scheduling model is obtained by executing the training method of the above-mentioned dynamic scheduling model.

[0159] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0160] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the dynamic scheduling model training method for strongly coupled operations provided by the above methods. The method includes: obtaining an initial dynamic scheduling model; the initial dynamic scheduling model includes an initial maintenance service support decision model and an initial airborne material transportation decision model; obtaining a first state representation from the simulation environment to characterize the status quo of maintenance service support operations; and inputting the first state representation into the initial maintenance service support decision model to obtain the initial maintenance service support decision model. The initial airborne material transportation decision model is used to generate a first decision output. Based on the first decision output, the simulation environment is updated, and a second state representation for characterizing the airborne material transportation operation status is obtained from the updated simulation environment. The second state representation is input into the initial airborne material transportation decision model to obtain the second decision output of the initial airborne material transportation decision model. The simulation environment after executing the second decision output is used as the simulation environment for obtaining the first state representation, so as to obtain the target reward generated by executing the first decision output and the second decision output in the simulation environment. Based on the target reward, the model parameters of the initial maintenance service support decision model and the initial airborne material transportation decision model are updated.

[0161] When the computer program is executed by the processor, the computer can execute the dynamic scheduling scheme generation method provided by the above methods. The method includes: obtaining the initial situation of the maritime aircraft support operation to be scheduled; deducing the initial situation based on the dynamic scheduling model to obtain a dynamic scheduling scheme; the dynamic scheduling model is obtained by executing the training method of the above dynamic scheduling model.

[0162] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a method for training a dynamic scheduling model for strongly coupled operations provided by the methods described above. This method includes: obtaining an initial dynamic scheduling model; the initial dynamic scheduling model includes an initial maintenance support decision model and an initial airborne material transportation decision model; obtaining a first state representation from the simulation environment to characterize the operational status of maintenance support; inputting the first state representation into the initial maintenance support decision model to obtain a first decision output from the initial maintenance support decision model; and based on the... The first decision output updates the simulation environment, and a second state representation characterizing the airborne material transportation operation status is obtained from the updated simulation environment. The second state representation is input into the initial airborne material transportation decision model to obtain the second decision output of the initial airborne material transportation decision model. The simulation environment after executing the second decision output is used as the simulation environment for obtaining the first state representation, so as to obtain the target reward generated by executing the first decision output and the second decision output in the simulation environment. The model parameters of the initial maintenance service support decision model and the initial airborne material transportation decision model are updated based on the target reward.

[0163] When the computer program is executed by the processor, it implements a method for generating dynamic scheduling schemes provided by the above methods. The method includes: acquiring the initial situation of the maritime aircraft support operation to be scheduled; deducing the initial situation based on the dynamic scheduling model to obtain a dynamic scheduling scheme; wherein the dynamic scheduling model is obtained by executing the training method of the above dynamic scheduling model.

[0164] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0165] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0166] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A training method for a dynamic scheduling model for strongly coupled jobs, characterized in that, The dynamic scheduling model is obtained by iteratively executing the following steps until a preset iteration condition is met: Obtain the initial dynamic scheduling model; the initial dynamic scheduling model includes the initial maintenance support decision model and the initial airborne material transportation decision model; A first state representation is obtained from the simulation environment to characterize the operational status of aircraft maintenance support. The first state representation is then input into the initial aircraft maintenance support decision model to obtain the first decision output of the initial aircraft maintenance support decision model. The simulation environment is updated based on the first decision output, and a second state representation for characterizing the airborne material transportation operation status is obtained from the updated simulation environment. The second state representation is then input into the initial airborne material transportation decision model to obtain the second decision output of the initial airborne material transportation decision model. The simulation environment after executing the second decision output is used as the simulation environment for obtaining the first state representation, so as to obtain the target reward generated by executing the first decision output and the second decision output in the simulation environment, and update the model parameters of the initial maintenance service support decision model and the initial airborne material transportation decision model based on the target reward; The configuration steps for the simulation environment include: Configure the simulation environment based on the scenario file; The scenario file includes aircraft fleet data, transport vehicle attributes, support operation procedure attributes, support resource attributes, and support personnel set; The aircraft fleet data includes individual data tuples for each aircraft. Each individual data tuple includes the aircraft number, aircraft mission type, support position number where the aircraft is located, a list of types of materials loaded on all hardpoints in the initial state of the aircraft, a list of types of materials to be loaded on the aircraft, and the start time of the aircraft support mission. The initial maintenance service support decision model includes a first state space converter, a first neural network model, and a first action space converter; the first state space converter is used to encode the first state representation into a first situation feature vector, the first neural network model is used to output a first action value vector based on the first situation feature vector, and the first action space converter is used to convert the first action value vector into the first decision output. The initial airborne material transportation decision model includes a second state space converter, a second neural network model, and a second action space converter; the second state space converter is used to encode the second state representation into a second situation feature vector, the second neural network model is used to output a second action value vector based on the second situation feature vector, and the second action space converter is used to convert the second action value vector into the second decision output. The second state represents the mission time step for airborne material transportation, the aircraft group loading status corresponding to the mission time step, and the transport vehicle status. The status of the transport vehicle includes the transport vehicle type, the target mounting point of the transport vehicle, the current stage of the current operation task of the transport vehicle, the type of material being carried, the quantity of material being carried, and the number of times it has been dispatched in the past. The target reward is determined based on a preset optimization target; the optimization target is determined based on the completion time of the aircraft fleet, the load balancing variance of the transport vehicle, and the cumulative transfer time of the transport vehicle. The first state includes the current simulation time, the state of each aircraft in the aircraft group, the state of all support resource stations, and the available number of various support personnel; the state of each aircraft in the aircraft group includes its assigned position, the currently executing procedure, the remaining time for each procedure, a list of procedures to be executed, and a list of completed procedures.

2. The training method for a dynamic scheduling model for strongly coupled jobs according to claim 1, characterized in that, The constraints of the initial maintenance support decision model include: The constraint condition for the coverage of the resource station with the available backup positions is used to constrain the number of backup positions that a single resource station can serve at any given time. The resource guarantee constraint is used to constrain the total demand of various types of resources at any given time. The resource guarantee constraint is determined based on the total demand of each type of resource at any given time and a preset upper limit for the quantity of each type of resource. The personnel constraint condition is used to constrain the total demand for various types of personnel at any given time. The personnel constraint condition is determined based on the total demand for each type of personnel at any given time and a preset upper limit for the number of each type of personnel. The execution constraints of the guarantee process are used to constrain the execution timing relationship between the subsequent guarantee process and the preceding guarantee process in the directed graph relationship between guarantee processes. The execution constraints of the guarantee process are determined based on the preset execution time of the preceding guarantee process, the execution time of the subsequent guarantee process, and the execution time of the preceding guarantee process. The completion time constraint for the guarantee process is used to constrain guarantee processes of different task types to have different execution times. The completion time constraint for the guarantee process is determined based on the task type to which the guarantee process belongs and the inherent attributes of the guarantee process. The start time constraint condition is used to ensure that the start time of any of the support procedures is not earlier than the start time of the aircraft support mission.

3. The training method for a dynamic scheduling model for strongly coupled jobs according to any one of claims 1 to 2, characterized in that, The constraints of the initial airborne material transport decision model include: The loading point operation sequence constraint is used to constrain the execution order of material unloading and loading operations at the same loading point. The loading point operation sequence constraint is determined based on the end time of the unloading event and the start time of the loading event at the loading point. The load-bearing type constraint condition of the transport vehicle is used to restrict a single transport vehicle to carry only one type of material in one trip; The transport vehicle scheduling priority constraint is used to determine the priority order of the execution tasks of the transport vehicles to be scheduled when there are multiple transport vehicles to be scheduled. The transport vehicle scheduling priority constraint is determined based on the current loading matching status and the historical cumulative number of dispatches of each transport vehicle to be scheduled. The task stage constraints of the transport vehicle are used to constrain the sequence of sub-tasks that the transport vehicle must follow when performing different types of material transportation tasks. The task stage constraints of the transport vehicle are determined based on a preset directed graph, which includes multiple sub-task stage nodes.

4. The training method for a dynamic scheduling model for strongly coupled jobs according to claim 3, characterized in that, The steps for determining the priority constraints for the transport vehicle scheduling include: Based on the current loading matching situation, a first priority weight is determined; wherein, the first priority weight of the first dispatchable transport vehicle is higher than the first priority weight of the second dispatchable transport vehicle, the first dispatchable transport vehicle is a vehicle whose current loading situation matches the material transportation operation to be performed, and the second dispatchable transport vehicle is a vehicle whose current loading situation does not match the material transportation operation to be performed. Based on the historical cumulative dispatch count, a second priority weight is determined; wherein, the second priority weight of the third dispatchable transport vehicle is higher than that of the fourth dispatchable transport vehicle, and the historical cumulative dispatch count of the third dispatchable transport vehicle is less than that of the fourth dispatchable transport vehicle. Based on the first priority weight and the second priority weight, the scheduling priority constraints of the transport vehicle are determined.

5. A method for generating a dynamic scheduling scheme, characterized in that, include: Obtain the initial situation of the maritime aircraft support operations to be scheduled; The initial situation is deduced based on the dynamic scheduling model to obtain a dynamic scheduling scheme; The dynamic scheduling model is obtained by performing the training method of the dynamic scheduling model as described in any one of claims 1 to 4.

6. A training device for a dynamic scheduling model for strongly coupled tasks, characterized in that, include: The acquisition unit is used to acquire the initial dynamic scheduling model; the initial dynamic scheduling model includes the initial maintenance support decision model and the initial airborne material transportation decision model. The first input unit is used to obtain a first state representation from the simulation environment to characterize the operational status of aircraft maintenance support, and input the first state representation into the initial aircraft maintenance support decision model to obtain the first decision output of the initial aircraft maintenance support decision model. The second input unit is used to update the simulation environment based on the first decision output, and obtain a second state representation from the updated simulation environment to characterize the airborne material transportation operation status, and input the second state representation into the initial airborne material transportation decision model to obtain the second decision output of the initial airborne material transportation decision model. The training unit is used to take the inference environment after executing the second decision output as the inference environment to obtain the first state representation, so as to obtain the target reward generated by executing the first decision output and the second decision output in the inference environment, and update the model parameters of the initial maintenance service support decision model and the initial airborne material transportation decision model based on the target reward; It also includes a configuration environment unit, which is specifically used for: Configure the simulation environment based on the scenario file; The scenario file includes aircraft fleet data, transport vehicle attributes, support operation procedure attributes, support resource attributes, and support personnel set; The aircraft fleet data includes individual data tuples for each aircraft. Each individual data tuple includes the aircraft number, aircraft mission type, support position number where the aircraft is located, a list of types of materials loaded on all hardpoints in the initial state of the aircraft, a list of types of materials to be loaded on the aircraft, and the start time of the aircraft support mission. The initial maintenance service support decision model includes a first state space converter, a first neural network model, and a first action space converter; the first state space converter is used to encode the first state representation into a first situation feature vector, the first neural network model is used to output a first action value vector based on the first situation feature vector, and the first action space converter is used to convert the first action value vector into the first decision output. The initial airborne material transportation decision model includes a second state space converter, a second neural network model, and a second action space converter; the second state space converter is used to encode the second state representation into a second situation feature vector, the second neural network model is used to output a second action value vector based on the second situation feature vector, and the second action space converter is used to convert the second action value vector into the second decision output. The second state represents the mission time step for airborne material transportation, the aircraft group loading status corresponding to the mission time step, and the transport vehicle status. The status of the transport vehicle includes the transport vehicle type, the target mounting point of the transport vehicle, the current stage of the current operation task of the transport vehicle, the type of material being carried, the quantity of material being carried, and the number of times it has been dispatched in the past. The target reward is determined based on a preset optimization target; the optimization target is determined based on the completion time of the aircraft fleet, the load balancing variance of the transport vehicle, and the cumulative transfer time of the transport vehicle. The first state includes the current simulation time, the state of each aircraft in the aircraft group, the state of all support resource stations, and the available number of various support personnel; the state of each aircraft in the aircraft group includes its assigned position, the currently executing procedure, the remaining time for each procedure, a list of procedures to be executed, and a list of completed procedures.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the dynamic scheduling model training method for strongly coupled jobs as described in any one of claims 1 to 4, or the dynamic scheduling scheme generation method as described in claim 5.

Images