Application method, device, storage medium and electronic equipment of artificial experience self-learning technology in dynamic adjustment of technical operation station transportation plan

Abstract

The application provides an application method, device, storage medium and electronic equipment of artificial experience self-learning technology in dynamic adjustment of transportation plan of technical operation station, relates to train transportation operation scheduling technical field, and the method comprises the following steps: obtaining a to-be-adjusted scheduling operation plan; inputting the to-be-adjusted scheduling operation plan and train delay information into an operation plan optimization model, obtaining an adjusted target scheduling operation plan based on the train delay information; the train delay information comprises the time of the train actually arriving at the station included in the to-be-adjusted scheduling operation, and the time of the train actually leaving the station included in the to-be-adjusted scheduling operation.The application provides an application method, device, storage medium and electronic equipment of artificial experience self-learning technology in dynamic adjustment of transportation plan of technical operation station, which is used for updating the scheduling operation plan automatically based on the train arrival time and departure time at the current time when the train arrival time or departure time does not conform to the operation scheduling plan.

Description

Technical Field

[0001] This application relates to the field of train transportation scheduling technology, and in particular to a method, apparatus, storage medium and electronic device for applying artificial experience self-learning technology in the dynamic adjustment of transportation plans at technical work stations. Background Technology

[0002] Technical stations include marshalling yards and section stations. Their main functions are to supply locomotives to adjacent railway sections, handle the dismantling and marshalling of freight trains, and perform prescribed technical operations for arriving dismantled trains, self-assembled originating trains, non-marshalling transfer trains, and partially marshalling transfer trains. Technical stations are the basic production units of railway transportation. Their main task is to efficiently complete the daily work plan issued by the railway bureau's dispatching and command center by making full use of the station's traffic flow and equipment resources while ensuring smooth network operation. According to relevant statistics, the time vehicles spend at technical stations accounts for more than 50% of the total freight turnover time; therefore, the operational organization level of technical stations directly determines the efficiency of railway transportation production.

[0003] In related technologies, train operation scheduling within technical work stations mainly relies on manual adjustment of train operation plans based on human experience. This scheduling method not only suffers from differences in adjustment results due to human factors, but may also lead to some unnecessary errors.

[0004] Therefore, there is an urgent need for a dynamic scheduling optimization method for transportation plans that can automatically update the scheduling plan when the train arrival time or departure time does not conform to the operation scheduling plan. Summary of the Invention

[0005] The purpose of this application is to provide a method, device, storage medium, and electronic device for applying artificial experience self-learning technology in the dynamic adjustment of transportation plans at technical work stations, which is used to automatically update the scheduling work plan based on the current train arrival and departure times when the train arrival or departure times do not conform to the work scheduling plan.

[0006] This application provides a method for applying artificial experience self-learning technology in the dynamic adjustment of transportation plans at technical work stations, including: Obtain the scheduling operation plan to be adjusted; the scheduling operation plan to be adjusted includes: the planned arrival time of the train at the station, and the planned departure time of the train at the station; input the scheduling operation plan to be adjusted and the train delay information into the operation plan optimization model, and obtain the adjusted target scheduling operation plan based on the train delay information; wherein, the train delay information includes: the delay time of the trains included in the scheduling operation to be adjusted; the target scheduling operation plan is obtained by the operation plan optimization model after adjusting the scheduling operation plan to be adjusted based on the train delay information; the operation plan optimization model includes: a Markov decision process agent adjusting the arrival time and departure time of the trains affected by delays in the scheduling operation plan to be adjusted based on the train delay information; the multiple candidate actions that the agent can select are: human experience expressions based on historical scheduling data; the historical scheduling data includes: human measures taken when handling train delay events and the final adjustment results.

[0007] Optionally, the step of inputting the scheduling operation plan to be adjusted and the train delay information into the operation plan optimization model, and obtaining the adjusted target scheduling operation plan based on the train delay information, includes: initializing the state based on the train delay information to obtain an initial state; and repeatedly executing the plan update operation based on the initial state until the target scheduling operation plan that satisfies the target constraints and whose scheduling result is better than the historical scheduling result indicated by the historical scheduling data is obtained; wherein, the plan update operation includes: a first update operation, a second update operation, and a third update operation; the first update operation includes: the agent selecting actions from the plurality of candidate actions that are related to the current state based on the current state corresponding to the current round. The first matching action, and based on the first action, the adjustment of the scheduling operation plan to be adjusted using a mathematical optimization model to obtain the target scheduling operation plan; the second update operation includes: updating the current state based on the first action to obtain the state corresponding to the next round; the third update operation includes: determining whether the scheduling operation plan to be evaluated after adjustment in the current round meets the target constraint condition, and whether the scheduling result corresponding to the scheduling operation plan to be evaluated after adjustment in the current round is better than the historical scheduling result indicated by the historical scheduling data; the scheduling operation plan to be evaluated after adjustment in the current round is obtained based on the updated state; the target constraint condition is the constraint that the train is subject to during actual operation.

[0008] Optionally, the step of selecting a first action that matches the current state from the plurality of candidate actions based on the current state corresponding to the current round includes: obtaining a first action that matches the current state based on a target algorithm network; wherein the target algorithm network is an improved deep Q network; the target algorithm network includes: the agent selecting the best action corresponding to the current state from the plurality of candidate actions based on an action selection strategy.

[0009] Optionally, the objective function of the mathematical optimization model is:

[0010] in, This indicates the total train delay time. Indicates the number of trains. Indicates the number of stations. Indicates train At the station The departure time Indicates train At the station The scheduled departure times are as follows. Indicates train At the station Arrival time Indicates train At the station Given the fixed arrival times at all stations, the objective function above represents minimizing the total delay time of the train across all stations.

[0011] Optionally, the job plan optimization model is trained based on the following steps: initializing the current state based on the current time; selecting a second action that matches the current state from the multiple candidate actions based on the current state corresponding to the current round, and based on the second action; when selecting an action, the first probability value of randomly selecting an action decreases as the number of iterations increases; the sum of the first probability value and the second probability of the action with the maximum value of the selected action value function is 1; updating the current state based on the second action, and adjusting the scheduling job plan of the previous time step based on the updated state and the mathematical optimization model to obtain the adjusted scheduling job plan; the updated state... The job plan optimization model is trained using the state of the next time step. If the adjusted scheduling job plan satisfies the target constraint and the scheduling result corresponding to the adjusted scheduling job plan is better than the historical scheduling result indicated by the historical scheduling data, then the current round of training ends. Otherwise, based on the reward function value at the current time step, the state at the current time step, and the optimal prediction of future value by the target network of the deep Q network, the loss function value of the target loss function is calculated, and the parameters of the job plan optimization model are adjusted based on the loss function value. The target loss function is used to minimize the error between the predicted value of the action value function of the deep Q network and the target value of the action value function.

[0012] Optionally, the step of calculating the loss function value of the target loss function based on the reward function value at the current time and the state at the current time includes: The loss function value of the target loss function is calculated based on the following formula. L :

[0013] in, express The action value function at time t. Indicates the learning rate. express Momentary reward value, This represents the reward decay factor. This represents the target network's optimal prediction of future value in a deep Q-network.

[0014] Optionally, after updating the current state based on the second action, the method further includes: performing an experience storage operation if the adjusted scheduling job plan does not meet the target constraint, and / or the scheduling result corresponding to the adjusted scheduling job plan is not better than the historical scheduling result indicated by the historical scheduling data; updating the action value function of the next time step based on the action value function of the current time step, the reward function value of the current time step, and the optimal prediction of future value by the target network of the deep Q network; wherein, the experience storage operation is used to store the experience of the current round into the experience pool; the experience of the current round includes: the current state, the state of the next time step, the reward function value, and the action of the current time step; the reward function value is calculated based on the second action.

[0015] The application also provides an application device for the dynamic adjustment of transportation plans at technical work stations using artificial experience self-learning technology, comprising: An acquisition module is used to acquire the scheduling operation plan to be adjusted; the scheduling operation plan to be adjusted includes: the planned arrival time of the train at the station and the planned departure time of the train at the station; an adjustment module is used to input the scheduling operation plan to be adjusted and the train delay information into the operation plan optimization model, and obtain the adjusted target scheduling operation plan based on the train delay information; wherein, the train delay information includes: the delay time of the trains included in the scheduling operation to be adjusted; the target scheduling operation plan is obtained by the operation plan optimization model after adjusting the scheduling operation plan to be adjusted based on the train delay information; the operation plan optimization model includes: a Markov decision process agent adjusting the arrival time and departure time of the trains affected by delays in the scheduling operation plan to be adjusted based on the train delay information; the multiple candidate actions that the agent can select are: human experience expressions based on historical scheduling data; the historical scheduling data includes: human measures taken when handling train delay events and the final adjustment results.

[0016] Optionally, the adjustment module is specifically used to initialize the state based on the train delay information to obtain an initial state; the adjustment module is further used to repeatedly execute plan update operations based on the initial state until the target scheduling operation plan that satisfies the target constraints and whose scheduling result is better than the historical scheduling result indicated by the historical scheduling data is obtained; wherein, the plan update operation includes: a first update operation, a second update operation, and a third update operation; the first update operation includes: the agent selecting a first action that matches the current state from the plurality of candidate actions based on the current state corresponding to the current round, and based on the first action, using The mathematical optimization model is used to adjust the scheduling operation plan to be adjusted to obtain the target scheduling operation plan; the second update operation includes: updating the current state based on the first action to obtain the state corresponding to the next round; the third update operation includes: determining whether the scheduling operation plan to be evaluated after adjustment in the current round meets the target constraint condition, and whether the scheduling result corresponding to the scheduling operation plan to be evaluated after adjustment in the current round is better than the historical scheduling result indicated by the historical scheduling data; the scheduling operation plan to be evaluated after adjustment in the current round is obtained based on the updated state; the target constraint condition is the constraint that the train is subject to during actual operation.

[0017] Optionally, the adjustment module is specifically used to obtain a first action that matches the current state based on the target algorithm network; wherein the target algorithm network is an improved deep Q network; the target algorithm network includes: the agent selecting the best action corresponding to the current state from the multiple candidate actions based on an action selection strategy.

[0018] Optionally, the objective function of the mathematical optimization model is:

[0019] in, This indicates the total train delay time. Indicates the number of trains. Indicates the number of stations. Indicates train At the station The departure time Indicates train At the station The scheduled departure times are as follows. Indicates train At the station Arrival time Indicates train At the station Given the fixed arrival times at all stations, the objective function above represents minimizing the total delay time of the train across all stations.

[0020] Optionally, the device further includes: a model training module; the model training module is used to initialize the current state based on the current time; the model training module is also used to filter out a second action matching the current state from the plurality of candidate actions based on the current state corresponding to the current round, and based on the second action; when selecting an action, the first probability value of randomly selecting an action decreases with the increase of the iteration round; the sum of the first probability value and the second probability of the action with the maximum action value of the action selection value function is 1; the model training module is also used to update the current state based on the second action, and adjust the scheduling job plan of the previous time step based on the updated state and the mathematical optimization model to obtain the adjusted scheduling job plan; The updated state is used as the state for the next time step to train the job plan optimization model. The model training module is further configured to terminate the current training if the adjusted scheduling job plan satisfies the target constraint and the scheduling result corresponding to the adjusted scheduling job plan is better than the historical scheduling result indicated by the historical scheduling data; otherwise, based on the reward function value at the current time step, the current state, and the optimal prediction of future value by the target network of the deep Q network, the loss function value of the target loss function is calculated, and the parameters of the job plan optimization model are adjusted based on the loss function value. The target loss function is used to minimize the error between the predicted value of the action value function of the deep Q network and the target value of the action value function.

[0021] Optionally, the model training module is specifically used to calculate the loss function value of the target loss function based on the following formula. L :

[0022] in, express The action value function at time t. Indicates the learning rate. express Momentary reward value, This represents the reward decay factor. This represents the target network's optimal prediction of future value in a deep Q-network.

[0023] Optionally, the device further includes: an operation module; the operation module is configured to perform an experience storage operation when the adjusted scheduling job plan does not meet the target constraint, and / or the scheduling result corresponding to the adjusted scheduling job plan is not better than the historical scheduling result indicated by the historical scheduling data; the model training module is specifically configured to update the action value function of the next time step based on the action value function of the current time step, the reward function value of the current time step, and the optimal prediction of future value by the target network of the deep Q-network; wherein, the experience storage operation is configured to store the experience of the current round into an experience pool; the experience of the current round includes: the state of the current time step, the state of the next time step, the reward function value, and the action of the current time step; the reward function value is calculated based on the second action.

[0024] This application also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the method for applying the artificial experience self-learning technology as described above in the dynamic adjustment of transportation plans at technical workstations.

[0025] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method for applying the artificial experience self-learning technology described above in the dynamic adjustment of transportation plans at technical workstations.

[0026] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for applying the artificial experience self-learning technology as described above in the dynamic adjustment of transportation plans at technical workstations.

[0027] This application provides a method, apparatus, storage medium, and electronic device for applying human experience self-learning technology in the dynamic adjustment of transportation plans at technical work stations. First, it acquires a scheduling work plan to be adjusted, including the planned arrival and departure times of trains at the station. Then, it inputs the scheduling work plan and train delay information into a work plan optimization model, obtaining an adjusted target scheduling work plan based on the train delay information. The train delay information includes the delay times of the trains included in the scheduling work to be adjusted. The target scheduling work plan is obtained by adjusting the scheduling work plan based on the train delay information using the work plan optimization model. The work plan optimization model includes a Markov decision process agent that adjusts the arrival and departure times of trains affected by delays in the scheduling work plan to be adjusted based on the train delay information. The agent can select multiple candidate actions based on human experience expressed using historical scheduling data, including human intervention measures taken when handling train delay events and the final adjustment results. In this way, when the train arrival time or departure time does not conform to the operation scheduling plan, the scheduling operation plan can be automatically updated based on the current train arrival time and departure time. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a flowchart illustrating the action function definition method based on human experience provided in this application; Figure 2 This is a schematic diagram of the operational framework for the application of the artificial experience self-learning technology provided in this application in the dynamic adjustment of transportation plans at technical work stations; Figure 3 This is one of the flowcharts illustrating the application of the human experience self-learning technology provided in this application in the dynamic adjustment of transportation plans at technical work stations; Figure 4 This is the second schematic diagram of the application method of the artificial experience self-learning technology provided in this application in the dynamic adjustment of transportation plans at technical work stations; Figure 5 This is the third flowchart illustrating the application of the human experience self-learning technology provided in this application in the dynamic adjustment of transportation plans at technical work stations; Figure 6This is the fourth flowchart illustrating the application of the human experience self-learning technology provided in this application in the dynamic adjustment of transportation plans at technical work stations; Figure 7 This is a schematic diagram of the device for applying the artificial experience self-learning technology provided in this application to the dynamic adjustment of transportation plans at technical work stations; Figure 8 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0032] The following is a detailed description of the technical terms used in the embodiments of this application: Reinforcement learning (RL), also known as reward learning, evaluation learning, or reinforcement learning, is a paradigm and methodology in machine learning used to describe and solve problems where an agent learns strategies to maximize rewards or achieve specific goals during interactions with its environment. A common model in reinforcement learning is the standard Markov Decision Process (MDP). A Markov Decision Process is a mathematical model of sequential decision-making, used to simulate stochastic policies and rewards achievable by an agent in an environment where the system state possesses Markov properties.

[0033] Deep Q Network (DQN) is a deep learning-based Q-learning algorithm used to solve reinforcement learning problems. Q-learning is a model-free algorithm used to teach agents how to take actions in an environment to maximize cumulative rewards. In Q-learning, Q-values, or Q(s,a), represent the value of a state-action pair. Here, s is the agent's current state, and a is the possible action the agent can take. The basic principle of DQN is that it combines deep learning and Q-learning, using a neural network to approximate the Q-function, thus handling complex problems with a large number of states and actions. In DQN, a neural network is used to estimate the Q-value. The network's input is the state s, and the output is the Q-value corresponding to each possible action. DQN improves learning efficiency and stability through experience replay and target network training. Key technology: Experience Replay: The agent's experience is stored in a dataset, and random samples are drawn for learning, breaking the correlation between data points. Target Network: Uses two neural networks, one for selecting actions and the other for calculating the Temporal-Difference Target (TD).

[0034] To address the aforementioned technical problems in related technologies, this application provides a method for applying human experience self-learning technology in the dynamic adjustment of transportation plans at technical work stations. This method mainly includes the following stages: P1, simulation model establishment, i.e., establishing a simulation model based on the actual operation scenario of freight trains at technical work stations, serving as the environment for interaction with deep reinforcement learning algorithms; P2, problem modeling based on deep reinforcement learning. The core elements of a Markov decision process—state, action, and reward function—are defined in conjunction with the actual problem. In this part, human scheduling experience is integrated into the action definition to update the state; P3, algorithm description and algorithm framework construction based on the train scheduling problem. The DQN algorithm, suitable for discrete action spaces, is selected to solve the train scheduling problem. During algorithm iteration, historical operation data is input into the algorithm and compared with the real-time output obtained in each cycle to optimize the algorithm's adjustment strategy, making the final scheduling result better than the manual scheduling result.

[0035] The embodiments of this application provide a method for applying human experience self-learning technology in the dynamic adjustment of transportation plans at technical work stations. This method enables the on-site experience of station dispatchers to be applied to intelligent algorithms, improving the feasibility and practicality of intelligent algorithms. Furthermore, it inputs historical dispatch data into intelligent algorithms to form a negative feedback loop, thereby making the algorithm training effect better and more accurate.

[0036] For example, stage P1 mentioned above specifically includes: P101. The problem of adjusting the operation of freight trains within the dispatching area of ​​the technical operation station is described as a constrained mathematical optimization problem. The optimization objective is the total delay time. The objective constraint is that the train is subject to environmental constraints during actual operation. The decision variables are the actual arrival and departure times of the train at the station, i.e., the arrival time and departure time.

[0037] P102. The mathematical optimization model constructed based on the mathematical optimization problem has the parameters and variables defined as shown in Table 1 below:

[0038] Table 1

[0039] For example, the objective function of the above mathematical optimization model is shown in the following formula: (Formula 1)

[0040] in, This indicates the total train delay time. Indicates the number of trains. Indicates the number of stations. Indicates train At the station The departure time Indicates train At the station The scheduled departure times are as follows. Indicates train At the station Arrival time Indicates train At the station Given the fixed arrival times at all stations, the objective function above represents minimizing the total delay time of the train across all stations.

[0041] P103. The objective constraints of the above mathematical optimization model are shown in Formula 2 below: (Formula 2)

[0042] Formula 2 above is used to indicate that the train departure time must not be earlier than the scheduled departure time.

[0043] (Formula 3)

[0044] Formula 3 above indicates that the arrival and departure times of the next station must be greater than the arrival and departure times of the previous station.

[0045] (Formula 4)

[0046] Formula 4 above represents the minimum travel time constraint for trains in each section, meaning that the travel time of the train in each section must not be less than a minimum value. It indicates the train In the interval Minimum runtime.

[0047] (Formula 5)

[0048] Formula 5 above represents the minimum train stopping time constraint, meaning that the train's stopping time at each station must not be less than the minimum operating time. It indicates the train At the station The minimum stopping time.

[0049] (Formula 6)

[0050] Formula 6 above represents the maximum stopping time constraint for trains, meaning that the stopping time of a train at each station cannot exceed a maximum value. It indicates the train At the station Maximum stop time.

[0051] (Formula 7)

[0052] Formula 7 above represents the station capacity constraint, meaning that the number of trains a station can accommodate cannot exceed the number of tracks. It indicates the station The number of lanes, Indicates station The number of trains.

[0053] (Formula 8)

[0054] Formula 8 above represents the departure interval constraint, which states that the departure time interval between two adjacent trains at the same station must not be less than a minimum value. It indicates the station The minimum departure interval is set to ensure the safe operation of trains.

[0055] (Formula 9)

[0056] Formula 9 above represents the arrival time interval constraint, meaning that the arrival time interval between two adjacent trains at the same station must not be less than a minimum value. It indicates the station Minimum arrival intervals are required to ensure safe train operation.

[0057] (Formula 10)

[0058] Formula 10 above states that the difference between the arrival time of the next train and the departure time of the previous train at the same station must not be less than the minimum arrival-departure time interval. To ensure the safe operation of trains.

[0059] P104. The decision variables of the above mathematical optimization model include: : The actual departure time of train i at station j; :train At the station The actual arrival time; For example, in stage P2 above, problem modeling based on deep reinforcement learning includes: P201. The state of a Markov decision process is defined as the actual arrival and departure times of a train from the station, which can be specifically expressed by the following formula eleven: (Formula Eleven)

[0060] Formula 11 above represents the state definition in the reinforcement learning framework, where... Indicates in t Time Train i At the station j The collection of arrival times, Indicates in t Time Train i At the station j The collection of departure times.

[0061] P202, Markov Decision Process: Action is defined as the action of an agent at the current moment. t After observing the situation, adjustments are made to the arrival and departure times of trains affected by delays. This section incorporates the manual experience of station dispatchers into the action descriptions, primarily including common methods used by station dispatchers in handling emergencies, such as reducing train dwell time at stations, increasing speed to shorten travel time between sections while ensuring safety, and changing train sequence. Figure 1 As shown, the action function definition method analyzes historical train operation data, proposes delay data caused by factors such as section speed limits and severe weather, and performs data analysis to finally obtain a process based on human experience expression of historical scheduling data.

[0062] P203. The reward function of a Markov decision process is defined as the negative of the total train delay time, which is the negative of the objective function mentioned above. Specifically, it can be expressed by the following formula: (Formula 12)

[0063] Formula 12 above represents the definition of the reward function in the reinforcement learning framework, where represents the reward value at time t. Since the learning goal of the agent is to maximize the reward value, and the goal of train scheduling is to minimize the total delay time, the total delay time needs to be adjusted accordingly to adapt to the reinforcement learning algorithm.

[0064] For example, after completing the construction of the aforementioned mathematical optimization model and the definition of the Markov decision process, the algorithm description and framework for the technical work station train scheduling problem can be developed. Figure 2 As shown, the framework of the entire method consists of two parts: the environment part and the algorithm part. The environment part takes into account the train schedule, constant parameters in the target constraints, and line parameters, and defines the target constraints as a separate module as a criterion for judging whether the timetable generated by the algorithm is feasible.

[0065] The following description, in conjunction with the accompanying drawings, details the application method of the artificial experience self-learning technology provided in this application in the dynamic adjustment of transportation plans at technical workstations through specific embodiments and application scenarios.

[0066] like Figure 3 As shown in the embodiment of this application, a method for applying artificial experience self-learning technology in the dynamic adjustment of transportation plans at technical work stations can be included in the following steps 301 and 302: Step 301: Obtain the scheduling plan to be adjusted.

[0067] The scheduling plan to be adjusted includes: the planned arrival time of the train at the station and the planned departure time of the train from the station.

[0068] For example, after a scheduling plan is developed, it needs to be adjusted based on the actual arrival and departure times of the trains to obtain a new scheduling plan. The scheduling plan indicates the arrival time and departure time of each train at the station.

[0069] Step 302: Input the scheduling operation plan to be adjusted and the train delay information into the operation plan optimization model, and obtain the adjusted target scheduling operation plan based on the train delay information.

[0070] The train delay information includes the delay time of the trains included in the adjusted scheduling operation, which includes at least one of the following: delayed arrival time and delayed departure time. The aforementioned station can be the aforementioned technical station or any other station. The target scheduling operation plan is obtained by adjusting the scheduling operation plan to be adjusted based on the train delay information using the operation plan optimization model. The operation plan optimization model includes a Markov decision process agent that adjusts the arrival and departure times of the trains affected by delays in the scheduling operation plan to be adjusted based on the train delay information. The multiple candidate actions that the agent can select are: human experience expressions based on historical scheduling data. The historical scheduling data includes: human measures taken when handling train delay events and the final adjustment results.

[0071] For example, in this embodiment of the application, when a train is delayed at a station, the scheduling operation plan can be adjusted by the operation plan optimization model to generate a new scheduling operation plan. The new scheduling operation plan includes other trains affected by the train delay and the arrival and departure times of the train at the station and subsequent stations, i.e. the aforementioned target scheduling operation plan.

[0072] Specifically, step 302 above may also include the following steps 302a and 302b: Step 302a: Initialize the state based on the train delay information to obtain the initial state.

[0073] For example, the initial state mentioned above is used to indicate the arrival and departure times of each train corresponding to the current time. Based on the initial state, the above-mentioned scheduling operation plan to be adjusted can be adjusted to obtain the adjusted target scheduling operation plan.

[0074] Step 302b: Based on the initial state, perform the plan update operation repeatedly until the target scheduling job plan is obtained that satisfies the target constraints and the scheduling result is better than the historical scheduling result indicated by the historical scheduling data.

[0075] The plan update operation includes: a first update operation, a second update operation, and a third update operation; the first update operation includes: the agent selecting a first action that matches the current state from the plurality of candidate actions based on the current state corresponding to the current round, and adjusting the scheduling operation plan to be adjusted using a mathematical optimization model based on the first action to obtain the target scheduling operation plan; the second update operation includes: updating the current state based on the first action to obtain the state corresponding to the next round; the third update operation includes: determining whether the scheduling operation plan to be evaluated after adjustment in the current round meets the target constraint condition, and whether the scheduling result corresponding to the scheduling operation plan to be evaluated after adjustment in the current round is better than the historical scheduling result indicated by the historical scheduling data; the scheduling operation plan to be evaluated after adjustment in the current round is obtained based on the updated state; the target constraint condition is the constraint that the train is subject to during actual operation.

[0076] Specifically, the first update operation in step 302b above may also include the following step 302b1: Step 302b1: Obtain a first action that matches the current state based on the target algorithm network.

[0077] The target algorithm network is an improved deep Q-network; the target algorithm network includes: the agent selecting the best action corresponding to the current state from the multiple candidate actions based on the action selection strategy.

[0078] For example, the steps of the job planning optimization model are as follows: Figure 4 As shown, after the agent selects an action based on the current state s (i.e., the initial state mentioned above), it obtains action at (i.e., the first action mentioned above). Then, this action a is input into the train operation environment for calculation to obtain the real-time operation diagram (i.e., the target scheduling operation plan mentioned above). Finally, the next state is selected through experience storage and experience replay.

[0079] It should be noted that the application method of the artificial experience self-learning technology provided in this application embodiment in the dynamic adjustment of transportation plans at technical work stations achieves the following: 1. Improved decision-making quality: Station dispatcher experience is usually based on long-term practical experience and professional knowledge, which can help the algorithm avoid some common errors and poor decisions. By incorporating experience, the algorithm can make more reasonable and accurate decisions, improving scheduling efficiency and accuracy. 2. Enhanced robustness: Station dispatcher experience is accumulated in the actual environment, taking into account various complex and abnormal situations. Incorporating this experience into the algorithm can enhance its robustness, making it more adaptable to different working environments and able to cope with emergencies. 3. Increased user acceptance: Algorithms incorporating station dispatcher experience are more in line with actual work needs and operating habits, making them easier for users to accept and trust. This helps improve the acceptability and usability of the algorithm in practical applications. 4. Integration with professional knowledge: Station dispatcher experience integrates industry professional knowledge and best practices, which can help the algorithm better understand and solve practical problems. By integrating professional knowledge, the algorithm can more accurately grasp the core needs and objectives of scheduling tasks.

[0080] Optionally, in this embodiment of the application, the above-mentioned job planning optimization model is trained based on the following steps 401 to 404: Step 401: Initialize the current state based on the current moment.

[0081] Step 402: Based on the current state corresponding to the current round, select a second action that matches the current state from the multiple candidate actions, and then apply the second action.

[0082] Among them, the first probability value of randomly selecting an action during action selection decreases as the number of iterations increases; the sum of the first probability value and the second probability of selecting the action with the maximum action value in the action value function is 1.

[0083] Step 403: Update the current state based on the second action, and adjust the scheduling job plan of the previous moment based on the updated state and the mathematical optimization model to obtain the adjusted scheduling job plan.

[0084] The updated state is used as the state for the next time step to train the job plan optimization model.

[0085] Step 404: If the adjusted scheduling job plan satisfies the target constraint and the scheduling result corresponding to the adjusted scheduling job plan is better than the historical scheduling result indicated by the historical scheduling data, then the current round of training ends; otherwise, based on the reward function value at the current moment, the current state, and the optimal prediction of future value by the target network of the deep Q network, the loss function value of the target loss function is calculated, and the parameters of the job plan optimization model are adjusted based on the loss function value.

[0086] The target loss function is used to minimize the error between the predicted value of the action value function of the deep Q network and the target value of the action value function.

[0087] For example, such as Figure 5 The diagram shows the overall flowchart of the job scheduling optimization model. First, at the beginning of the algorithm, the algorithm parameters are initialized and defined, and the total number of training episodes is set. Then, at the beginning of each episode, the state is initialized. In the loop of each episode, the algorithm outputs an adjusted timetable that satisfies all target constraints. This is achieved through the interaction between the algorithm and the environment. Specifically, the algorithm interacts with the environment for many steps until all target constraints are met and the scheduling is better than historical scheduling data before ending the loop of this episode.

[0088] For example, such as Figure 6 As shown, in all iterations of each round, a real-time operation diagram is generated through the above-mentioned work plan optimization model. Then, the real-time operation diagram is compared with the historical data obtained by the station dispatcher based on his experience. If the total delay time of the historical data is smaller, it means that the adjustment result obtained by this method has not achieved the expected effect. Therefore, the algorithm iteration continues to repeat until the result obtained is better than the historical data before exiting the loop of this round.

[0089] Specifically, such as Figure 6 As shown, after obtaining the current state Then, based on the action selection strategy ε-greedy, the best action for the current state is selected. The ε-greedy strategy sets a parameter when selecting actions. , The value of is between 0 and 1, and the probability of the agent randomly choosing an action (i.e., the first probability mentioned above) is: The probability of selecting the action value that maximizes the action value function (i.e., the second probability mentioned above) is... And the ε-greedy strategy The value decreases as the number of iterations increases, meaning that early exploration is prioritized to make the agent's search space more comprehensive, while later reuse aims to allow the agent to fully utilize previously learned experience. Then, the state is updated based on the current action. With reward function value As previously defined, the action and reward function incorporate human experience to guide the algorithm's decision-making process when updating the state. This involves filtering historical data for delay information to obtain manually scheduled data, then analyzing its patterns to determine the spatiotemporal distribution of delays, identifying factors influencing delays to derive computer-understandable heuristic rules for handling delays, and finally representing these rules as actions to update the state.

[0090] For example, after obtaining the new state and reward value, it is necessary to make a constraint judgment on the new state, because the state represents the adjusted timetable. Therefore, it is necessary to make a constraint judgment on the adjusted timetable. If the adjusted timetable meets the target constraint conditions, it is compared with the historical scheduling data. If the adjusted timetable is better than the historical scheduling results, the loop of this game ends here. If either of the above two conditions is not met, training continues until the conditions are met and the game exits.

[0091] Specifically, such as Figure 5 As shown, control is achieved through the total number of rounds M and the current round number e. At the start of each round, two states, done1=False and done2=False, need to be initialized. The values ​​of the two states, done=done1 AND done2, are calculated using a AND operation. The values ​​of done1 and done2 ​​are then adjusted based on whether the target constraints are met and the comparison results with historical data.

[0092] For example, after updating the current state based on the second action in step 403 above, the application method of the artificial experience self-learning technology in the dynamic adjustment of transportation plans at technical workstations provided in this application embodiment may further include the following steps 405 and 406: Step 405: If the adjusted scheduling job plan does not meet the target constraints, and / or the scheduling result corresponding to the adjusted scheduling job plan is not better than the historical scheduling result indicated by the historical scheduling data, perform an experience storage operation.

[0093] Step 406: Based on the action value function at the current moment, the reward function value at the current moment, and the target network of the deep Q network's optimal prediction of future value, update the action value function at the next moment.

[0094] The experience storage operation is used to store the experience of the current round into the experience pool; the experience of the current round includes: the current state, the next state, the reward function value, and the action at the current moment; the reward function value is calculated based on the second action.

[0095] For example, the next part is the experience storage and experience replay section. Experience replay is a method to enhance learning efficiency and robustness by replaying previous experiences. This method uses Prioritized Experience Replay, which improves the way experience is replayed, enabling the agent to learn more effectively and improve performance. Each experience is assigned a priority, usually based on its impact or importance on learning. During experience replay, experiences are selected according to their priority, so that higher-priority experiences are more likely to be selected.

[0096] For example, in this embodiment of the application, the action value function can be updated using the following formula thirteen. That is, the Q-function of a deep Q-network: (Formula Thirteen)

[0097] in, The function representing the action value at the next moment. express The action value function at time t. Indicates the learning rate. express Momentary reward value, This represents the reward decay factor. This represents the target network's optimal prediction of future value in a deep Q-network.

[0098] For example, in this embodiment of the application, the loss function value of the target loss function can be calculated using the following formula fourteen: (Formula Fourteen)

[0099] The target loss function is the mean squared error (MSE) function. Finally, it is replaced with [the function name], and the next iteration is performed.

[0100] Understandably, in deep Q-networks, the Q-neural network continuously learns and updates, while the target network's parameters are copied from the Q-neural network's parameters periodically. The target network's role is to calculate the Temporal-Difference Target (TD) in the Q-value update formula, which is the maximum Q-value for the next state s'. By using the target network, we can reduce the overestimation of Q-values, thus enabling faster training and achieving more stable learning results.

[0101] The method for applying human experience self-learning technology in the dynamic adjustment of transportation plans at technical work stations, as provided in this application embodiment, firstly, obtains the scheduling work plan to be adjusted; then, the scheduling work plan to be adjusted and the train delay information are input into the work plan optimization model, and an adjusted target scheduling work plan is obtained based on the train delay information; wherein, the train delay information includes: the delay time of the trains included in the scheduling work to be adjusted; the target scheduling work plan is obtained by the work plan optimization model after adjusting the scheduling work plan to be adjusted based on the train delay information; the work plan optimization model includes: a Markov decision process agent adjusting the arrival and departure times of trains affected by delays in the scheduling work plan to be adjusted based on the train delay information; the multiple candidate actions that the agent can select are: human experience expressions based on historical scheduling data; the historical scheduling data includes: human measures taken when handling train delay events and the final adjustment results. Thus, when the train arrival or departure time does not conform to the work scheduling plan, the scheduling work plan can be automatically updated based on the current train arrival and departure times.

[0102] It should be noted that the application method of artificial experience self-learning technology in the dynamic adjustment of transportation plans at technical work stations provided in this application embodiment can be executed by an application device for the application of artificial experience self-learning technology in the dynamic adjustment of transportation plans at technical work stations, or by a control module within that application device for executing the application method of artificial experience self-learning technology in the dynamic adjustment of transportation plans at technical work stations. This application embodiment uses the execution of the application method of artificial experience self-learning technology in the dynamic adjustment of transportation plans at technical work stations by the application device as an example to illustrate the application device of artificial experience self-learning technology in the dynamic adjustment of transportation plans at technical work stations provided in this application embodiment.

[0103] It should be noted that, in the embodiments of this application, the methods illustrated in the accompanying drawings of the various methods above, and the application methods of artificial experience self-learning technology in the dynamic adjustment of transportation plans at technical work stations, are all illustrated by way of example in conjunction with one of the accompanying drawings in the embodiments of this application. In specific implementation, the application methods of artificial experience self-learning technology in the dynamic adjustment of transportation plans at technical work stations, as illustrated in the accompanying drawings of the various methods above, can also be implemented in conjunction with any other accompanying drawings that can be combined with those illustrated in the above embodiments, which will not be elaborated here.

[0104] The following describes the application device of the artificial experience self-learning technology provided in this application in the dynamic adjustment of transportation plans at technical work stations. The application method of the artificial experience self-learning technology in the dynamic adjustment of transportation plans at technical work stations described below can be referred to in correspondence with the above description.

[0105] Figure 7 A schematic diagram of the structure of the device for applying the human experience self-learning technology in the dynamic adjustment of transportation plans at technical work stations, as provided in the embodiments of this application, is shown below. Figure 7 As shown, it specifically includes: The acquisition module 701 is used to acquire the scheduling operation plan to be adjusted; the adjustment module 702 is used to input the scheduling operation plan to be adjusted and the train delay information into the operation plan optimization model, and obtain the adjusted target scheduling operation plan based on the train delay information; wherein, the train delay information includes: the delay time of the trains included in the scheduling operation to be adjusted; the target scheduling operation plan is obtained by the operation plan optimization model after adjusting the scheduling operation plan to be adjusted based on the train delay information; the operation plan optimization model includes: a Markov decision process agent adjusting the arrival time and departure time of the trains affected by delays in the scheduling operation plan to be adjusted based on the train delay information; the multiple candidate actions that the agent can select are: human experience expressions based on historical scheduling data; the historical scheduling data includes: human measures taken when handling train delay events and the final adjustment results.

[0106] Optionally, the adjustment module 702 is specifically used to initialize the state based on the train delay information to obtain an initial state; the adjustment module 702 is further used to cyclically execute the plan update operation based on the initial state until the target scheduling operation plan that satisfies the target constraints and whose scheduling result is better than the historical scheduling result indicated by the historical scheduling data is obtained; wherein, the plan update operation includes: a first update operation, a second update operation, and a third update operation; the first update operation includes: the agent selecting a first action that matches the current state from the plurality of candidate actions based on the current state corresponding to the current round, and based on the first action The first action involves adjusting the scheduled operation plan to be adjusted using a mathematical optimization model to obtain the target scheduled operation plan. The second update operation includes updating the current state based on the first action to obtain the state corresponding to the next round. The third update operation includes determining whether the scheduled operation plan to be evaluated after adjustment in the current round meets the target constraint condition, and whether the scheduling result corresponding to the scheduled operation plan to be evaluated after adjustment in the current round is better than the historical scheduling result indicated by the historical scheduling data. The scheduled operation plan to be evaluated after adjustment in the current round is obtained based on the updated state. The target constraint condition is the constraint that the train is subject to during actual operation.

[0107] Optionally, the adjustment module 702 is specifically used to obtain a first action that matches the current state based on the target algorithm network; wherein the target algorithm network is an improved deep Q network; the target algorithm network includes: the agent selecting the best action corresponding to the current state from the multiple candidate actions based on the action selection strategy.

[0108] Optionally, the objective function of the mathematical optimization model is:

[0109] in, This indicates the total train delay time. Indicates the number of trains. Indicates the number of stations. Indicates train At the station The departure time Indicates train At the station The scheduled departure times are as follows. Indicates train At the station Arrival time Indicates train At the station Given the fixed arrival times at all stations, the objective function above represents minimizing the total delay time of the train across all stations.

[0110] Optionally, the device further includes: a model training module; the model training module is used to initialize the current state based on the current time; the model training module is also used to filter out a second action matching the current state from the plurality of candidate actions based on the current state corresponding to the current round, and based on the second action; when selecting an action, the first probability value of randomly selecting an action decreases with the increase of the iteration round; the sum of the first probability value and the second probability of the action with the maximum action value of the action selection value function is 1; the model training module is also used to update the current state based on the second action, and adjust the scheduling job plan of the previous time step based on the updated state and the mathematical optimization model to obtain the adjusted scheduling job plan; The updated state is used as the state for the next time step to train the job plan optimization model. The model training module is further configured to terminate the current training if the adjusted scheduling job plan satisfies the target constraint and the scheduling result corresponding to the adjusted scheduling job plan is better than the historical scheduling result indicated by the historical scheduling data; otherwise, based on the reward function value at the current time step, the current state, and the optimal prediction of future value by the target network of the deep Q network, the loss function value of the target loss function is calculated, and the parameters of the job plan optimization model are adjusted based on the loss function value. The target loss function is used to minimize the error between the predicted value of the action value function of the deep Q network and the target value of the action value function.

[0111] Optionally, the model training module is specifically used to calculate the loss function value of the target loss function based on the following formula. L :

[0112] in, express The action value function at time t. Indicates the learning rate. express Momentary reward value, This represents the reward decay factor. This represents the target network's optimal prediction of future value in a deep Q-network.

[0113] Optionally, the device further includes: an operation module; the operation module is configured to perform an experience storage operation when the adjusted scheduling job plan does not meet the target constraint, and / or the scheduling result corresponding to the adjusted scheduling job plan is not better than the historical scheduling result indicated by the historical scheduling data; the model training module is specifically configured to update the action value function of the next time step based on the action value function of the current time step, the reward function value of the current time step, and the optimal prediction of future value by the target network of the deep Q-network; wherein, the experience storage operation is configured to store the experience of the current round into an experience pool; the experience of the current round includes: the state of the current time step, the state of the next time step, the reward function value, and the action of the current time step; the reward function value is calculated based on the second action.

[0114] The device for applying human experience self-learning technology in the dynamic adjustment of transportation plans at technical work stations, as provided in this application, firstly acquires the scheduling work plan to be adjusted; then, it inputs the scheduling work plan to be adjusted and the train delay information into a work plan optimization model, and obtains the adjusted target scheduling work plan based on the train delay information. The train delay information includes the delay time of the trains included in the scheduling work to be adjusted; the target scheduling work plan is obtained by the work plan optimization model after adjusting the scheduling work plan to be adjusted based on the train delay information; the work plan optimization model includes a Markov decision process agent that adjusts the arrival and departure times of trains affected by delays in the scheduling work plan to be adjusted based on the train delay information; the multiple candidate actions that the agent can select are: expressions of human experience based on historical scheduling data; the historical scheduling data includes: manual measures taken when handling train delay events and the final adjustment results. Thus, when the train arrival or departure time does not conform to the work scheduling plan, the scheduling work plan can be automatically updated based on the current train arrival and departure times.

[0115] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8As shown, the electronic device may include: a processor 810, a communications interface 820, a memory 830, and a communications bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other through the communications bus 840. The processor 810 can call logic instructions in the memory 830 to execute a method for applying human experience self-learning technology in the dynamic adjustment of transportation plans at technical work stations. This method includes: acquiring a scheduling operation plan to be adjusted; inputting the scheduling operation plan to be adjusted and the train delay information into an operation plan optimization model, and obtaining an adjusted target scheduling operation plan based on the train delay information; wherein the train delay information includes: the delay time of the trains included in the scheduling operation to be adjusted; the target scheduling operation plan is obtained by the operation plan optimization model after adjusting the scheduling operation plan to be adjusted based on the train delay information; the operation plan optimization model includes: a Markov decision process agent adjusting the arrival and departure times of trains affected by delays in the scheduling operation plan to be adjusted based on the train delay information; the multiple candidate actions that the agent can select are: expressions of human experience based on historical scheduling data; the historical scheduling data includes: human measures taken when handling train delay events and the final adjustment results.

[0116] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0117] On the other hand, this application also provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer can execute the application method of the artificial experience self-learning technology provided by the above methods in the dynamic adjustment of transportation plans at technical work stations. This method includes: obtaining a scheduling operation plan to be adjusted; inputting the scheduling operation plan to be adjusted and the train delay information into an operation plan optimization model, and obtaining an adjusted target scheduling operation plan based on the train delay information; wherein, the train delay information includes The delay times of the trains included in the scheduling operation to be adjusted are defined as follows: the target scheduling operation plan is obtained by adjusting the scheduling operation plan to be adjusted based on the train delay information by the operation plan optimization model; the operation plan optimization model includes: a Markov decision process agent adjusting the arrival and departure times of trains affected by delays in the scheduling operation plan to be adjusted based on the train delay information; the multiple candidate actions that the agent can select are: human experience expressions based on historical scheduling data; the historical scheduling data includes: human measures taken when handling train delay events and the final adjustment results.

[0118] On another front, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements a method for applying the aforementioned self-learning techniques of human experience in the dynamic adjustment of transportation plans at technical work stations. This method includes: acquiring a scheduling operation plan to be adjusted; inputting the scheduling operation plan to be adjusted and the train delay information into an operation plan optimization model; and obtaining an adjusted target scheduling operation plan based on the train delay information. The train delay information includes the delay time of the trains included in the scheduling operation to be adjusted; the target scheduling operation plan is obtained by the operation plan optimization model after adjusting the scheduling operation plan to be adjusted based on the train delay information; the operation plan optimization model includes a Markov decision process agent adjusting the arrival and departure times of trains affected by delays in the scheduling operation plan to be adjusted based on the train delay information; the agent can select multiple candidate actions based on human experience expressions from historical scheduling data; the historical scheduling data includes human measures taken when handling train delay events and the final adjustment results.

[0119] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0120] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for applying artificial experience self-learning technology to the dynamic adjustment of transportation plans at technical work stations, characterized in that, include: Obtain the scheduled job plan to be adjusted; The scheduling operation plan to be adjusted includes: the planned arrival time of the train at the station and the planned departure time of the train from the station; The scheduling operation plan to be adjusted is input into the operation plan optimization model, and the adjusted target scheduling operation plan is obtained based on the train delay information; The train delay information includes: the delay time of the trains included in the scheduling operation to be adjusted; the target scheduling operation plan is obtained by the operation plan optimization model after adjusting the scheduling operation plan to be adjusted based on the train delay information; the operation plan optimization model includes: a Markov decision process agent adjusting the arrival and departure times of the trains affected by delays in the scheduling operation plan to be adjusted based on the train delay information; the multiple candidate actions that the agent can select are: human experience expressions based on historical scheduling data; the historical scheduling data includes: human measures taken when handling train delay events and the final adjustment results.

2. The method according to claim 1, characterized in that, The step of inputting the scheduling operation plan to be adjusted and the train delay information into the operation plan optimization model, and obtaining the adjusted target scheduling operation plan based on the train delay information, includes: The state is initialized based on the train delay information to obtain the initial state; Based on the initial state, the plan update operation is executed cyclically until the target scheduling job plan is obtained that satisfies the target constraints and whose scheduling result is better than the historical scheduling result indicated by the historical scheduling data. The plan update operation includes: a first update operation, a second update operation, and a third update operation; the first update operation includes: the agent selecting a first action that matches the current state from the plurality of candidate actions based on the current state corresponding to the current round, and adjusting the scheduling operation plan to be adjusted using a mathematical optimization model based on the first action to obtain the target scheduling operation plan; the second update operation includes: updating the current state based on the first action to obtain the state corresponding to the next round; the third update operation includes: determining whether the scheduling operation plan to be evaluated after adjustment in the current round meets the target constraint condition, and whether the scheduling result corresponding to the scheduling operation plan to be evaluated after adjustment in the current round is better than the historical scheduling result indicated by the historical scheduling data; the scheduling operation plan to be evaluated after adjustment in the current round is obtained based on the updated state; the target constraint condition is the constraint that the train is subject to during actual operation.

3. The method according to claim 2, characterized in that, The objective function of the mathematical optimization model is: in, This indicates the total train delay time. Indicates the number of trains. Indicates the number of stations. Indicates train At the station The departure time Indicates train At the station The scheduled departure times are as follows. Indicates train At the station Arrival time Indicates train At the station Given the fixed arrival times at all stations, the objective function above represents minimizing the total delay time of the train across all stations.

4. The method according to claim 2, characterized in that, The step of selecting a first action that matches the current state from the plurality of candidate actions based on the current state corresponding to the current round includes: Based on the target algorithm network, a first action matching the current state is obtained; The target algorithm network is an improved deep Q-network; the target algorithm network includes: the agent selecting the best action corresponding to the current state from the multiple candidate actions based on the action selection strategy.

5. The method according to claim 4, characterized in that, The job scheduling optimization model was trained based on the following steps: Initialize the current state based on the current moment; Based on the current state corresponding to the current round, a second action matching the current state is selected from the multiple candidate actions, and based on the second action; when selecting an action, the first probability value of randomly selecting an action decreases as the iteration round increases; the sum of the first probability value and the second probability of selecting the action with the maximum action value in the action value function is 1; The current state is updated based on the second action, and the scheduling job plan of the previous time step is adjusted based on the updated state and the mathematical optimization model to obtain the adjusted scheduling job plan. The updated state is used as the state for the next time step to train the job plan optimization model; If the adjusted scheduling job plan satisfies the target constraint and the scheduling result corresponding to the adjusted scheduling job plan is better than the historical scheduling result indicated by the historical scheduling data, then the current training round ends; otherwise, based on the reward function value at the current moment, the current state, and the target network's optimal prediction of future value, the loss function value of the target loss function is calculated, and the parameters of the job plan optimization model are adjusted based on the loss function value. The target loss function is used to minimize the error between the predicted value of the action value function of the deep Q network and the target value of the action value function.

6. The method according to claim 5, characterized in that, The calculation of the target loss function value based on the reward function value and the state at the current moment includes: The loss function value of the target loss function is calculated based on the following formula. L : in, express The action value function at time t. Indicates the learning rate. express Momentary reward value, This represents the reward decay factor. This represents the target network's optimal prediction of future value in a deep Q-network.

7. The method according to claim 5, characterized in that, After updating the current state based on the second action, the method further includes: If the adjusted scheduling job plan does not meet the target constraints, and / or the scheduling result corresponding to the adjusted scheduling job plan is not better than the historical scheduling result indicated by the historical scheduling data, an experience storage operation is performed. Based on the action value function at the current moment, the reward function value at the current moment, and the target network of the deep Q network's optimal prediction of future value, update the action value function at the next moment; The experience storage operation is used to store the experience of the current round into the experience pool; the experience of the current round includes: the current state, the next state, the reward function value, and the action at the current moment; the reward function value is calculated based on the second action.

8. A device for applying artificial experience self-learning technology in the dynamic adjustment of transportation plans at technical work stations, characterized in that, The device includes: The acquisition module is used to acquire the scheduling operation plan to be adjusted; the scheduling operation plan to be adjusted includes: the planned arrival time of the train at the station and the planned departure time of the train from the station. The adjustment module is used to input the scheduling operation plan to be adjusted and the train delay information into the operation plan optimization model, and obtain the adjusted target scheduling operation plan based on the train delay information; The train delay information includes: the delay time of the trains included in the scheduling operation to be adjusted; the target scheduling operation plan is obtained by the operation plan optimization model after adjusting the scheduling operation plan to be adjusted based on the train delay information; the operation plan optimization model includes: a Markov decision process agent adjusting the arrival and departure times of the trains affected by delays in the scheduling operation plan to be adjusted based on the train delay information; the multiple candidate actions that the agent can select are: human experience expressions based on historical scheduling data; the historical scheduling data includes: human measures taken when handling train delay events and the final adjustment results.

9. An electronic device, characterized in that, The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the method for applying the artificial experience self-learning technology as described in any one of claims 1 to 7 to the dynamic adjustment of transportation plans at technical workstations.

10. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by a processor, implements the steps of the method for applying the artificial experience self-learning technology as described in any one of claims 1 to 7 in the dynamic adjustment of transportation plans at technical work stations.

Images