Train rescheduling method based on neural network and related device

Abstract

The invention relates to a train rescheduling method based on a neural network and a related device. The method for train rescheduling comprises the steps that time sequence data indicating the states at all moments in the train running process are obtained, and the states comprise the train running state, the resource allocation state and the external environment state; the obtained time series data are provided for a trained neural network model to generate a scheduling strategy for train rescheduling, the scheduling strategy comprises actions executed by a train scheduling system, and when an emergency occurs in the train running process, the scheduling strategy is not executed by the train scheduling system compared with the action under the condition that no emergency occurs in the train running process. The time series data have different noises, and the neural network model comprises a diffusion model and a reinforcement learning model, the diffusion model is configured to remove the noises in the time series data based on an inverse diffusion process to obtain de-noised time series data, and the reinforcement learning model is configured to generate a scheduling strategy based on the de-noised time series data.

Description

Technical Field

[0001] The present disclosure relates to the technical field of train scheduling, and more specifically, to a method for train rescheduling, a method for training a neural network model for train rescheduling, and related computing devices, non-transitory storage media, and computer program products. Background Art

[0002] With the advancement of artificial intelligence, big data, and multimodal information fusion technology, train dispatching systems are gradually developing from the traditional reliance on fixed train schedules and manual decision-making to intelligent, automated, and unmanned operations. Summary of the invention

[0003] A brief overview of the disclosure is given below in order to provide a basic understanding of some aspects of the disclosure. However, it should be understood that this overview is not an exhaustive overview of the disclosure. It is not intended to identify the key or important parts of the disclosure, nor is it intended to limit the scope of the disclosure. Its purpose is simply to give some concepts of the disclosure in a simplified form as a prelude to a more detailed description given later.

[0004] According to a first aspect of the present disclosure, a method for train rescheduling is provided, comprising: acquiring time series data indicating the state at each moment in a train operation process, the state comprising a train operation state, a resource allocation state and an external environment state; providing the acquired time series data to a trained neural network model to generate a scheduling strategy for train rescheduling, the scheduling strategy comprising actions performed by a train scheduling system, wherein the time series data has different noise in the case of an emergency occurring during train operation than in the case of no emergency occurring during train operation, and wherein the neural network model comprises a diffusion model and a reinforcement learning model, the diffusion model being configured to remove noise in the time series data based on an inverse diffusion process to obtain denoised time series data, and the reinforcement learning model being configured to generate a scheduling strategy based on the denoised time series data.

[0005] According to a second aspect of the present disclosure, a method for training a neural network model for train rescheduling is provided, the neural network model including a diffusion model and a reinforcement learning model, the method including: obtaining historical time series data indicating the state at each moment in the historical operation process of the train and the train historical scheduling strategy corresponding to the historical operation process of the train, the state including the train operation state, resource allocation state and external environment state, the train historical scheduling strategy including actions performed by the train scheduling system; adding noise to the historical time series data based on a positive diffusion process via the diffusion model to obtain noisy historical time series data; using the noisy historical time series data as sample data and the train historical scheduling strategy as label data to train the reinforcement learning model.

[0006] According to a third aspect of the present disclosure, a computing device is provided, comprising a processor and a memory storing computer executable instructions, wherein when the computer executable instructions are executed by the processor, the processor executes the method for train rescheduling according to the first aspect of the present disclosure or the method for training a neural network model for train rescheduling according to the second aspect of the present disclosure.

[0007] According to a fourth aspect of the present disclosure, a non-volatile storage medium having computer executable instructions stored thereon is provided. When the computer executable instructions are executed by a processor, the processor executes the method for train rescheduling according to the first aspect of the present disclosure or the method for training a neural network model for train rescheduling according to the second aspect of the present disclosure.

[0008] According to a fifth aspect of the present disclosure, a computer program product is provided, which includes instructions, which, when executed by a processor, implement the method for train rescheduling according to the first aspect of the present disclosure or the method for training a neural network model for train rescheduling according to the second aspect of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The foregoing and other features and advantages of the present disclosure will become apparent from the following description of the embodiments of the present disclosure in conjunction with the accompanying drawings. The accompanying drawings are incorporated herein and form a part of the specification, and are further used to explain the principles of the present disclosure and enable those skilled in the art to make and use the present disclosure. Among them:

[0010] Figure 1 A flow chart of a method for train rescheduling according to some embodiments of the present disclosure is shown;

[0011] Figure 2 A flowchart of a method for training a neural network model for train rescheduling according to some embodiments of the present disclosure is shown;

[0012] Figure 3 A schematic block diagram of a computing device according to some embodiments of the present disclosure is shown;

[0013] Figure 4 A schematic block diagram of a computer system is shown on which embodiments of the present disclosure may be implemented.

[0014] Note that in the embodiments described below, sometimes the same reference numerals are used in common between different drawings to represent the same parts or parts with the same functions, and their repeated descriptions are omitted. In some cases, similar numbers and letters are used to represent similar items, so once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0015] For ease of understanding, the positions, sizes, and ranges of various structures shown in the drawings and the like may not represent actual positions, sizes, and ranges, etc. Therefore, the present disclosure is not limited to the positions, sizes, and ranges disclosed in the drawings and the like. DETAILED DESCRIPTION

[0016] Various exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0017] The following description of at least one exemplary embodiment is in fact merely illustrative and is in no way intended to limit the present disclosure and its application or use. That is, the structures and methods herein are shown in an exemplary manner to illustrate different embodiments of the structures and methods in the present disclosure. However, those skilled in the art will appreciate that they merely illustrate exemplary ways of the present disclosure that can be implemented, rather than exhaustive ways. In addition, the drawings need not be drawn to scale, and some features may be enlarged to illustrate the details of specific components.

[0018] In addition, technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0019] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0020] At present, the train dispatching system still mainly relies on fixed train operation diagrams and manual dispatching decisions, with a low level of intelligence, lack of autonomous learning and decision-making capabilities, and difficulty in quickly and automatically responding to dynamically changing operating environments. Especially in the face of emergencies (such as bad weather, equipment failures, etc.), the flexibility and real-time adjustment capabilities of the train dispatching system are obviously insufficient, making it impossible to efficiently adjust the train operation plan, which easily leads to train delays and service interruptions.

[0021] In addition, the current train scheduling method has an unbalanced resource allocation problem, resulting in the failure to optimize the use of resources such as tracks, trains and manpower. For example, during the peak period of train operation, the shortage of resources exacerbates this problem, while during the non-peak period of train operation, the idleness of resources causes waste.

[0022] To this end, the present disclosure provides a method for train rescheduling, which realizes autonomous learning and decision-making capabilities for complex situations and emergency response capabilities for dealing with emergencies by combining a diffusion model and a reinforcement learning model to obtain a neural network model for train rescheduling. The method according to the present disclosure can dynamically adjust the train operation plan in real time, optimize resource allocation, flexibly respond to emergencies, and improve the overall operating efficiency and reliability of the train system.

[0023] Below, the method for train rescheduling according to various embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. It is understood that the actual method for train rescheduling may also include other steps, but in order to avoid blurring the main points of the present disclosure, these other steps are not discussed herein and are not shown in the accompanying drawings.

[0024] Figure 1 FIG. 1 is a flow chart of a method 100 for train rescheduling according to some embodiments of the present disclosure. Figure 1 As shown, the method 100 includes steps S102 to S104.

[0025] At step S102, time series data indicating the status at each time during the operation of the train is obtained, and the status includes the train operation status, resource allocation status and external environment status. In some embodiments, the train operation status may include the current position, speed, and planned arrival time of the train, the resource allocation status may include the station status and track availability, and the external environment status may include weather conditions, equipment conditions, passenger conditions, etc.

[0026] In this article, "noise" can be used to describe the complexity and / or uncertainty of the train scheduling situation or the train operation environment. Specifically, the time series data may have different noise when an emergency occurs during the train operation than when no emergency occurs during the train operation.

[0027] In some embodiments, emergencies may include equipment failure (e.g., track signal failure, train failure, power outage, etc.), severe weather (e.g., heavy rain, heavy snow, strong wind, etc.), and passenger emergencies (e.g., medical emergencies, overloading, congestion, etc.).

[0028] It can be understood that the time series data of the train operation process in which no emergency occurs can have a relatively stable noise distribution characteristic. Once an emergency occurs during the train operation, the emergency will disturb the state at the corresponding moment (and possibly some subsequent moments), thereby causing the noise distribution characteristics of the time series data to change, such as having increased noise at the corresponding moment (and possibly some subsequent moments).

[0029] In some embodiments, for example, a first emergency and a second emergency of different types or priorities, the time series data may have different noises when the first emergency occurs during the train operation compared to when the second emergency occurs during the train operation. Specifically, different emergencies may be characterized by noises with different characteristics, such as noise characteristics that may include but are not limited to the distribution that the noise obeys (e.g., standard normal distribution or Gaussian distribution, etc.), the strategy of noise variation (i.e., a function of noise variation over time, such as a linear noise strategy, a cosine noise strategy, an exponential noise strategy, etc.), the weight of the noise, and the like.

[0030] As an illustration, it can be assumed that the first emergency event is a track signal failure and the second emergency event is an emergency medical event. The adjustment of the train dispatching strategy corresponding to the track signal failure is to solve the equipment failure as soon as possible to reduce the impact on the train operation while ensuring the normal operation of the train, and the adjustment of the train dispatching strategy corresponding to the emergency medical event is to determine the appropriate stop site as soon as possible to provide passengers with sufficient medical resources. Since the occurrence of different types of emergencies causes different degrees of disturbance to the operation of the train system, the noise with different characteristics in the time series data can characterize the corresponding different types of emergencies.

[0031] In some examples, different types of emergencies have different priorities. For example, track signal failures and severe weather events can be considered higher priorities, while passenger congestion and individual equipment failures can be considered lower priorities. In addition, in some examples, even the same type of emergencies can have different priorities depending on their impact. For example, small-scale severe weather events (such as severe weather in a single station area) can have a lower priority than large-scale severe weather events (such as severe weather in the entire route area). Therefore, emergencies of different priorities can be characterized by noises with different characteristics.

[0032] For example, it can be assumed that when the starting station and the terminal station of the train are Shanghai Station and Fuzhou Station respectively, the first emergency event is heavy rainfall in East China, and the second emergency event is heavy rainfall in a local area of ​​Shanghai. Since the first emergency event is a large-scale severe weather event and covers the entire running route of the train, the priority of the first emergency event is higher than the priority of the second emergency event. Accordingly, the weight of the noise used to characterize the first emergency event can be greater than the weight of the noise used to characterize the second emergency event, so that the scheduling strategy generated by method 100 can give priority to the first emergency event.

[0033] At step S104, the acquired time series data is provided to the trained neural network model to generate a scheduling strategy for train rescheduling, the scheduling strategy including actions performed by the train scheduling system. In some embodiments, the actions that can be performed by the train scheduling system include determining train departure and arrival times, determining train operation priorities, selecting tracks, and selecting stations.

[0034] The neural network model includes a diffusion model (DM) and a reinforcement learning model (RLM). Specifically, the diffusion model can be regarded as a type of variational inference, which uses a denoising network to converge step by step towards the real sample across a sequence of estimation steps. The reinforcement learning model is a branch of the machine learning model, which involves the agent learning through trial and error and feedback in the environment (Environment), aiming to maximize the cumulative reward. This article does not specifically limit the specific model structure and algorithm of the diffusion model and the reinforcement learning model, and any diffusion model and reinforcement learning model known now or developed later can be applied to various embodiments of the present disclosure.

[0035] In the neural network model according to the present disclosure, the diffusion model is configured to remove noise in the time series data based on the inverse diffusion process to obtain denoised time series data, and the reinforcement learning model is configured to generate a scheduling strategy based on the denoised time series data.

[0036] In some embodiments, removing noise from time series data based on an inverse diffusion process includes: in,

[0037] The time series data includes (T+1) moments, where the time series data indicates the state x0 at moment 0 and the state x at moment T. T ,

[0038] The inverse diffusion process is defined as following a Markov chain consisting of T time steps from state x T The diffusion process to state x0, t is the time step in the Markov chain, where t∈[1,T],

[0039] Status x t-1 Represents state x t After one step of denoising,

[0040] ε θ (x t ,t) represents (specifically, through the denoising network in the diffusion model) for the state x t and the noise predicted at time step t, θ represents the scheduling policy parameters,

[0041] α t=1-β t ,

[0042] σ t represents the noise coefficient at time step t, σ t It can also be called noise scheduling.

[0043] z represents Gaussian noise, and z~N(0,I) means that z is sampled from a standard normal distribution with mean 0 and variance I, which can be used to further adjust the denoising effect.

[0044] In some embodiments, σ t Determined based on the sudden events that occurred at time step t.

[0045] In some further embodiments, σ t It can be determined based on one of the following noise strategies: Based on the linear noise strategy, σ t Determined as Based on the cosine noise strategy, σ t Determined as Based on the exponential noise strategy, σ t Determined as Among them, σ min represents the minimum noise coefficient for T time steps, σ max represents the maximum noise factor for T time steps. As a non-limiting example, we can set σ min =0.1,σ max =1.

[0046] In the above embodiment, for different noise strategies, σ t The time-varying patterns are different, which results in the noise term σ t The pattern of z changing over time is different. Depending on whether an unexpected event occurs and what kind of unexpected event occurs, the corresponding noise strategy is selected. In some examples, σ is t The rate of change over time can increase. In some examples, the higher the priority of the emergency event, the higher the t The rate of change over time can be larger. For example, the noise strategy can be selected based on the following operations: in response to no sudden event occurring at time step t, a linear noise strategy is selected; in response to a sudden event with a first priority occurring at time step t, a cosine noise strategy is selected; in response to a sudden event with a second priority occurring at time step t, an exponential noise strategy is selected, wherein the second priority is higher than the first priority.

[0047] For example, when no emergency occurs during train operation, the linear noise strategy is selected and σ t Determined as As the train runs, when an emergency event "heavy rainfall in some areas of Shanghai" occurs, it is considered that this emergency event has the first priority, so the cosine noise strategy is selected, and σ t Determined as The noise at this time may be greater than the noise that would have been present if the emergency had not occurred. As time goes by, the emergency is upgraded to "heavy rainfall in East China", and the emergency is considered to become the second priority. Therefore, the exponential noise strategy is selected, and based on the exponential noise strategy, σ t Determined as At this time, the rate of change of noise over time will become larger and larger. By representing the occurrence and development of emergencies during train operation from the perspective of noise, the neural network model can adjust the train scheduling strategy in time, thereby realizing the train rescheduling for the emergency.

[0048] Combined with the above description of the priority of emergencies, it can be understood that according to the priority of emergencies, the corresponding noise strategy is selected to configure σ t The convergence speed of the neural network model can be adaptively controlled to quickly adjust the train scheduling strategy, so that the train can respond to emergencies of different priorities in a timely manner according to the adjusted scheduling strategy.

[0049] Thus, the diffusion model removes the noise in the time series data through the inverse diffusion process to preprocess the input data for the reinforcement learning model, so that the reinforcement learning model can generate an optimized and accurate scheduling strategy from a complex and uncertain environment. Since the neural network model according to the present disclosure makes decisions based on the denoised state, it can ensure that the train scheduling process is more robust.

[0050] In some embodiments, the above-mentioned neural network model is trained through the following process: obtaining historical time series data indicating the state at each moment in the historical operation process of the train and the historical train scheduling strategy corresponding to the historical operation process of the train, the state includes the train operation state, resource allocation state and external environment state, and the historical train scheduling strategy includes actions performed by the train scheduling system; adding noise to the historical time series data based on the positive diffusion process via the diffusion model to obtain noisy historical time series data; using the noisy historical time series data as sample data and the historical train scheduling strategy as label data to train the reinforcement learning model.

[0051] Further, in some embodiments, adding noise to the historical time series data based on the forward diffusion process includes: in,

[0052] The historical time series data includes (T+1) moments. The historical time series data indicates the state x0 at moment 0 and the state x at moment T. T ,

[0053] The positive diffusion process is defined as following a Markov chain consisting of T time steps from state x0 to state x T diffusion process, t is the time step t in the Markov chain, where t∈[1,T],

[0054] Status x t represents the state of state x0 after t steps of noise addition,

[0055] α t represents the weight coefficient used to control the noise increase at time step t,

[0056] ε represents Gaussian noise, and ε~N(0,I) means that σ is sampled from a standard normal distribution with mean 0 and variance I, which can be used to inject into the state.

[0057] In some embodiments, α t It decreases with the increase of t.

[0058] In other embodiments, σ t represents the noise coefficient at time step t, σ t Determined based on the contingency to be simulated at time step t.

[0059] In some further embodiments, σ t It can be determined based on one of the following noise strategies: Based on the linear noise strategy, σ t Determined as Based on the cosine noise strategy, σ t Determined as Based on the exponential noise strategy, σ t Determined as As a non-limiting example, one can set σ min =0.1,σ max =1.

[0060] In the above embodiment, the noise strategy can be selected based on the following operations: in response to no emergency event occurring in the simulation at time step t, a linear noise strategy is selected; in response to an emergency event with a first priority occurring in the simulation at time step t, a cosine noise strategy is selected; in response to an emergency event with a second priority occurring in the simulation at time step t, an exponential noise strategy is selected, wherein the second priority is higher than the first priority.

[0061] The diffusion model generates sample data by gradually adding noise to historical time series data through a positive diffusion process, which enables the reinforcement learning model to better learn how to make robust decisions in complex and uncertain environments. The reinforcement learning model trained in this way can handle environments with high noise, thereby ensuring that robust and efficient scheduling strategies can still be generated under conditions of uncertainty and high noise.

[0062] In some embodiments, the loss function of the neural network model is determined as: L(θ)=L d (θ)+L q (θ), where

[0063] θ represents the scheduling policy parameters, which can be optimized through training,

[0064] L(θ) represents the loss function of the neural network model.

[0065] L d (θ) represents the behavior cloning loss, L d (θ) can be used to describe the difference between the action a generated according to θ and the action in the train historical scheduling strategy, and

[0066]

[0067] L q (θ) represents the Q-Learning loss, L q (θ) can be used to describe the (train) scheduling strategy π θ Differences from historical train scheduling strategies, and

[0068]

[0069] in,

[0070] E is the expected symbol, which represents the average value sampled from the empirical data set D. The empirical data set D includes historical time series data and historical train scheduling strategies.

[0071] i represents the time step in the diffusion process, i~U means that i is sampled from the uniform distribution U. In the scheduling strategy, i can be understood as different stages of the scheduling process.

[0072] ε represents noise, ε~N(0,I),

[0073] (s,a)~D means that the state s and the corresponding action a are sampled from the experience data set D.

[0074] represents (specifically, via a denoising network) the noise predicted for state s, action a, and time step i, and is used to predict the noise level of the train system during the diffusion process,

[0075] is the cumulative weight coefficient for controlling the increase of noise from time step 1 to time step i, ||·|| 2 represents the square of the second norm, which is used to calculate the difference between the predicted noise and the actual noise.

[0076] α represents the learning rate coefficient, and 0≤α≤1,

[0077] s~D means that the state s is sampled from the empirical data set D.

[0078] a0~π θ (·|s) indicates that action a0 follows the scheduling strategy π θ (·|s) generates the scheduling strategy π θ (·|s) represents the probability distribution of selecting the corresponding action in state s,

[0079] is the Q-value function, which represents the expected cumulative reward for selecting action a0 in state s. is the parameter of the Q-value function.

[0080] In some embodiments, the Q-value function includes:

[0081]

[0082] in,

[0083] (s i ,a i ,s i+1 )~D represents sampling state-action pairs from the empirical data set D, where state s i+1 Indicates that in state s i Execute the selected action a i The next state after

[0084] Represents the scheduling strategy π θ′ Generate a new dispatch action in Generated by the scheduling policy parameter θ′,

[0085] r(s i ,a i ) means in state s i Next, perform action a i Instant rewards received,

[0086] γ represents the influence of future rewards on the current scheduling strategy, 0≤γ≤1.

[0087] Here, the min function can ensure that the reinforcement learning model selects relatively conservative actions to reduce risks.

[0088] r(s i ,a i ) can be used to measure the impact of actions performed in the scheduling strategy on various scheduling objectives (e.g., delay, resource utilization, etc.). i ,a i ) balance multiple scheduling objectives. For example, i The action that can achieve the desired goal a i In some embodiments, r(s i ,a i ) is configured to promote the minimization of delays and the maximization of resource utilization. i ,a i ) is configured to promote the minimization of delay time and the maximization of resource utilization, so that the scheduling strategy generated by the neural network model tends to minimize the delay time of the train and maximize the utilization of various resources, thereby improving the operation efficiency and resource utilization of the train. Of course, it can be understood that according to the actual scheduling needs of the train, r(s i ,a i ) can also be configured to balance other multiple scheduling objectives (for example, minimizing the number of stops of the train, minimizing the running time of a certain journey of the train, etc.) to generate different scheduling strategies.

[0089] Additionally or alternatively, in some embodiments, r(s i ,a i ) imposes train scheduling constraints, such as train interval time, station stop requirements, etc., to better meet the actual needs of train scheduling.

[0090] In addition, γ can be used to weigh long-term and short-term benefits. A larger γ indicates that future rewards have a greater impact on the current scheduling strategy. A larger γ value will make the model pay more attention to long-term returns, while a smaller γ value emphasizes immediate benefits.

[0091] Therefore, through the forward diffusion process in model training and the reverse diffusion process in model reasoning, the method for train rescheduling according to various embodiments of the present disclosure can maintain stable scheduling performance under complex and changeable circumstances, and has excellent scheduling efficiency and robustness.

[0092] The present disclosure also provides a method for training a neural network model for train rescheduling. As mentioned above, the neural network model according to the present disclosure includes a diffusion model and a reinforcement learning model. Figure 2 , which shows a flowchart of a method 200 for training a neural network model for train rescheduling according to some embodiments of the present disclosure.

[0093] like Figure 2 As shown, method 200 includes: at step S202, obtaining historical time series data indicating the state at each moment in the historical operation process of the train and the historical train scheduling strategy corresponding to the historical operation process of the train, the state includes the train operation state, resource allocation state and external environment state, and the historical train scheduling strategy includes actions performed by the train scheduling system; at step S204, adding noise to the historical time series data based on the positive diffusion process via a diffusion model to obtain noisy historical time series data; at step S206, using the noisy historical time series data as sample data and the historical train scheduling strategy as label data to train the reinforcement learning model.

[0094] Various embodiments of method 200 may be similarly referred to the embodiments related to training a neural network model in the aforementioned method 100, and will not be described in detail here.

[0095] The present disclosure also provides a computing device, which may include a processor and a memory storing computer executable instructions, wherein when the computer executable instructions are executed by the processor, the processor executes the method for train rescheduling or the method for training a neural network model for train rescheduling according to any of the aforementioned embodiments of the present disclosure. Figure 3 , which shows a schematic block diagram of a computing device 300 according to some embodiments of the present disclosure. Figure 3As shown, the computing device 300 includes a processor 302 and a memory 304 storing computer executable instructions, which, when executed by the processor 302, causes the processor 302 to perform the method for train rescheduling or the method for training a neural network model for train rescheduling according to any of the foregoing embodiments of the present disclosure. The processor 302 may be, for example, a central processing unit (CPU) of the computing device 300. The processor 302 may be any type of general-purpose processor, or may be a processor specifically designed for train rescheduling or training a neural network model for train rescheduling, such as an application-specific integrated circuit (“ASIC”). The memory 304 may be coupled to the processor 302 and may include various computer-readable media accessible by the processor 302. In various embodiments, the memory 304 described herein may include volatile and non-volatile media, removable and non-removable media. For example, the memory 304 may include any combination of the following: random access memory (“RAM”), dynamic RAM (“DRAM”), static RAM (“SRAM”), read-only memory (“ROM”), flash memory, cache memory, and / or any other type of non-transitory computer-readable media. The memory 304 may store instructions that, when executed by the processor 302, cause the processor 302 to execute the method for train rescheduling or the method for training a neural network model for train rescheduling according to any of the aforementioned embodiments of the present disclosure.

[0096] The present disclosure also provides a non-volatile storage medium having computer executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method for train rescheduling described in any of the foregoing embodiments of the present disclosure or the method for training a neural network model for train rescheduling described in any of the foregoing embodiments.

[0097] The present disclosure also provides a computer program product, which may include instructions, and when the instructions are executed by a processor, the method for train rescheduling described in any of the foregoing embodiments of the present disclosure or the method for training a neural network model for train rescheduling described in any of the foregoing embodiments may be implemented. The instructions may be any set of instructions to be executed directly by the processor, such as machine code, or any set of instructions to be executed indirectly, such as a script. The instructions may be stored in an object code format for direct processing by the processor, or in any other computer language, including a script or collection of independent source code modules that are interpreted on demand or compiled in advance.

[0098] Figure 4A schematic block diagram of a computer system 400 on which embodiments of the present disclosure may be implemented is shown. The computer system 400 includes a bus 402 or other communication mechanism for transmitting information, and a processing device 404 coupled to the bus 402 for processing information. The computer system 400 also includes a memory 406 coupled to the bus 402 for storing instructions to be executed by the processing device 404, which may be a random access memory (RAM) or other dynamic storage device. The memory 406 may also be used to store temporary variables or other intermediate information during the execution of instructions to be executed by the processing device 404. The computer system 400 also includes a read-only memory (ROM) 408 or other static storage device coupled to the bus 402 for storing static information and instructions for the processing device 404. A storage device 410 such as a magnetic disk or optical disk is provided and coupled to the bus 402 for storing information and instructions. The computer system 400 may be coupled to an output device 412 via the bus 402 for providing output to a user, such as, but not limited to, a display (such as a cathode ray tube (CRT) or a liquid crystal display (LCD)), a speaker, etc. An input device 414 such as a keyboard, a mouse, a microphone, etc. is coupled to the bus 402 for transmitting information and command selections to the processing device 404. The computer system 400 may perform embodiments of the present disclosure. Consistent with certain implementations of the present disclosure, the results are provided by the computer system 400 in response to the processing device 404 executing one or more sequences of one or more instructions contained in the memory 406. Such instructions may be read into the memory 406 from another computer-readable medium such as the storage device 410. Execution of the sequence of instructions contained in the memory 406 causes the processing device 404 to perform the methods described herein. Alternatively, hard-wired circuitry may be used instead of or in combination with software instructions to implement the present teachings. Therefore, implementations of the present disclosure are not limited to any particular combination of hardware circuitry and software. In various embodiments, computer system 400 can be connected to one or more other computer systems such as computer system 400 across a network via network interface 416 to form a networked system. The network can include a private network or a public network such as the Internet. In a networked system, one or more computer systems can store data and supply data to other computer systems. The term "computer-readable medium" as used herein refers to any medium that participates in providing instructions to processing device 404 for execution. Such media can take many forms, including but not limited to non-volatile media, volatile media, and transmission media. Non-volatile media include, for example, optical disks or disks such as storage device 410. Volatile media include dynamic memories such as memory 406. Transmission media include coaxial cables, copper wires, and optical fibers, including wiring that includes bus 402.Common forms of computer-readable media or computer program products include, for example, floppy disks, flexible disks, hard disks, tapes, or any other magnetic media, CD-ROMs, digital video disks (DVDs), Blu-ray disks, any other optical media, thumb drives, memory cards, RAMs, PROMs and EPROMs, flash EPROMs, any other memory chips or boxes, or any other tangible media from which a computer can read. Various forms of computer-readable media may be involved when carrying one or more sequences of one or more instructions to processing device 404 for execution. For example, the instructions may initially be carried on a disk of a remote computer. The remote computer may load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to computer system 400 may receive data over a telephone line and convert the data into an infrared signal using an infrared transmitter. An infrared detector coupled to bus 402 may receive the data carried in the infrared signal and place the data on bus 402. Bus 402 carries the data to memory 406, from which processing device 404 retrieves the instructions and executes them. Optionally, the instructions received by memory 406 may be stored on storage device 410 either before or after execution by processing device 404 .

[0099] According to various embodiments, instructions configured to be executed by a processing device to perform a method are stored on a computer readable medium. A computer readable medium may be a device that stores digital information. For example, a computer readable medium includes a compact disk read-only memory (CD-ROM) as known in the art for storing software. The computer readable medium is accessed by a processor adapted to execute the instructions configured to be executed.

[0100] One or more exemplary embodiments of the present disclosure are described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0101] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a server system. Of course, the present disclosure does not exclude that with the development of computer technology in the future, the computer that implements the functions of the above embodiments may be, for example, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0102] Although one or more embodiments of the present disclosure provide method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-creative means. The order of steps listed in the embodiments is only one way of executing the order of many steps and does not represent the only execution order. When the device or terminal product in practice is executed, it can be executed in sequence or in parallel according to the method shown in the embodiments or the drawings (for example, in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment).

[0103] The terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, product or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, product or apparatus. In the absence of further restrictions, it is not excluded that there are other identical or equivalent elements in the process, method, product or apparatus comprising the elements. For example, if the words "first", "second" and the like are used to indicate names, they do not indicate any particular order.

[0104] For the convenience of description, the above devices are described in various modules according to their functions. Of course, when implementing one or more embodiments of the present disclosure, the functions of each module can be implemented in the same or more software and / or hardware, or the module implementing the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0105] The present disclosure is described with reference to the flowchart and / or block diagram of the method, device (system), and computer program product according to the embodiment of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the process and / or box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the function specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.

[0106] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a particular manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device that implements the functions specified in one or more flows of a flowchart and / or one or more blocks of a block diagram. These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operating steps are performed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more flows of a flowchart and / or one or more blocks of a block diagram.

[0107] Those skilled in the art will appreciate that one or more embodiments of the present disclosure may be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, one or more embodiments of the present disclosure may be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0108] One or more embodiments of the present disclosure may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present disclosure may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0109] The same or similar parts between the various embodiments of the present disclosure can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts refer to the partial description of the method embodiment. In the description of the present disclosure, the description of the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In the present disclosure, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in the present disclosure and the features of the different embodiments or examples without contradiction.

[0110] In addition, when used in the present disclosure, the words "herein," "above," "below," "hereunder," "above," and words of similar meaning shall refer to the present disclosure as a whole rather than to any particular portion of the present disclosure. Furthermore, unless expressly stated otherwise or understood otherwise in the context of use, conditional language used herein, such as "may," "might," "for example," "such as," and the like, is generally intended to express that certain embodiments include, while other embodiments do not include, certain features, elements, and / or states. Thus, such conditional language is generally not intended to imply that one or more embodiments require features, elements, and / or states in any way, or whether such features, elements, and / or states are included or performed in any particular embodiment.

[0111] The above description is only an example of one or more embodiments of the present disclosure, and is not intended to limit one or more embodiments of the present disclosure. For those skilled in the art, one or more embodiments of the present disclosure may have various changes and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the scope of the claims.

Claims

1. A method for train rescheduling, comprising: Acquire time series data indicating the status at each time point during the operation of the train, wherein the status includes the train operation status, resource allocation status, and external environment status; providing the acquired time series data to a trained neural network model to generate a scheduling strategy for train rescheduling, the scheduling strategy including actions performed by a train scheduling system, Wherein, when an emergency occurs during train operation, the time series data has different noise than when no emergency occurs during train operation, and Among them, the neural network model includes a diffusion model and a reinforcement learning model, the diffusion model is configured to remove noise in the time series data based on an inverse diffusion process to obtain denoised time series data, and the reinforcement learning model is configured to generate the scheduling strategy based on the denoised time series data.

2. The method according to claim 1, wherein: When a first emergency event occurs during train operation, the time series data has different noises than when a second emergency event occurs during train operation. The first emergency event has a different type from that of the second emergency event, or the first emergency event has a different priority from that of the second emergency event.

3. The method according to claim 1, wherein: Removing noise from the time series data based on the inverse diffusion process includes: in, The time series data includes (T+1) moments, the time series data indicates the state x0 at moment 0 and indicates the state x at moment T. T , The inverse diffusion process is defined as following a Markov chain consisting of T time steps from state x T The diffusion process to state x0, t is the time step in the Markov chain, where t∈[1,T], Status x t-1 Represents state x t After one step of denoising, ε θ (x t ,t) represents the state x t and the noise predicted at time step t, θ represents the scheduling policy parameters, α t =1-β t , σ t represents the noise coefficient at time step t, z represents Gaussian noise.

4. The method according to claim 3, wherein: σ t Determined based on the sudden events that occurred at time step t.

5. The method according to claim 4, wherein: σ t Determined based on one of the following noise strategies: Based on the linear noise strategy, σ t Determined as or Based on the cosine noise strategy, σ t Determined as or Based on the exponential noise strategy, σ t Determined as in, σ min represents the minimum noise coefficient for T time steps, σ max represents the maximum noise factor over T time steps.

6. The method according to claim 5, comprising: In response to no sudden event occurring at time step t, selecting the linear noise strategy; In response to an emergency event with a first priority occurring at time step t, selecting the cosine noise strategy; The exponential noise strategy is selected in response to an emergency event having a second priority occurring at time step t, the second priority being higher than the first priority.

7. The method according to claim 5, wherein: s min =0.1,σ max =1.

8. The method according to claim 1, wherein: The neural network model is trained through the following process: Acquire historical time series data indicating the status at each time point in the historical operation process of the train and a historical train dispatching strategy corresponding to the historical operation process of the train, wherein the status includes the train operation status, resource allocation status and external environment status, and the historical train dispatching strategy includes actions executed by the train dispatching system; adding noise to the historical time series data based on a positive diffusion process via the diffusion model to obtain noisy historical time series data; The reinforcement learning model is trained by using the noisy historical time series data as sample data and the historical train scheduling strategy as label data.

9. The method according to claim 8, wherein: Adding noise to the historical time series data based on a positive diffusion process includes: in, The historical time series data includes (T+1) moments, and the historical time series data indicates the state x0 at moment 0 and the state x at moment T. T , The positive diffusion process is defined as following a Markov chain consisting of T time steps from state x0 to state x T diffusion process, t is the time step t in the Markov chain, where t∈[1,T], Status x t represents the state x0 after t steps of noise addition, α t represents the weight coefficient used to control the noise increase at time step t, ε represents Gaussian noise.

10. The method according to claim 9, wherein: α t It decreases with the increase of t.

11. The method according to claim 9, wherein: σ t represents the noise coefficient at time step t, σ t Determined based on the contingency to be simulated at time step t.

12. The method according to claim 11, wherein: σ t Determined based on one of the following noise strategies: Based on the linear noise strategy, σ t Determined as or Based on the cosine noise strategy, σ t Determined as or Based on the exponential noise strategy, σ t Determined as in, σ min represents the minimum noise coefficient for T time steps, σ max represents the maximum noise factor over T time steps.

13. The method according to claim 12, comprising: In response to no unexpected event occurring during simulation at time step t, selecting the linear noise strategy; In response to a simulated occurrence of an emergency event with a first priority at time step t, selecting the cosine noise strategy; The exponential noise strategy is selected in response to a simulated occurrence of an emergency event having a second priority at time step t, the second priority being higher than the first priority.

14. The method according to claim 12, wherein: s min =0.1,σ max =1.

15. The method according to claim 8, wherein: The loss function of the neural network model is determined as: L(θ)=L d (θ)+L q (θ), where θ represents the scheduling strategy parameter, L(θ) represents the loss function of the neural network model, L d (θ) represents the behavior cloning loss, and L q (θ) represents the Q-Learning loss, and in, E is an expected symbol, which represents the average value sampled from the empirical data set D, where the empirical data set D includes the historical time series data and the historical train scheduling strategy. i represents the time step in the diffusion process, i~U means i is sampled from the uniform distribution U, ε represents noise, ε~N(0,I) means that ε is sampled from a standard normal distribution with a mean of 0 and a variance of I. (s,a)~D means that the state s and the corresponding action a are sampled from the experience data set D. represents the noise predicted for state s, action a and time step i, represents the cumulative weight coefficient used to control the noise increase from time step 1 to time step i, α t represents the weight coefficient used to control the noise increase at time step t, α represents the learning rate coefficient, 0≤α≤1, s~D means that the state s is sampled from the empirical data set D. a0~π θ (·|s) indicates that action a0 follows the scheduling strategy π θ (·|s) generates the scheduling strategy π θ (·|s) represents the probability distribution of selecting the corresponding action in state s, is the Q-value function, which represents the expected cumulative reward for selecting action a0 in state s. is the parameter of the Q-value function.

16. The method according to claim 15, wherein: The Q value functions include: in, (s i ,a i ,s i+1 )~D represents sampling state-action pairs from the empirical data set D, where state s i+1 Indicates that in state s i Execute the selected action a i The next state after Represents the scheduling strategy π θ′ Generate a new dispatch action in Generated by the scheduling policy parameter θ′, r(s i ,a i ) means in state s i Next, perform action a i Instant rewards received, γ represents the influence of future rewards on the current scheduling strategy, 0≤γ≤1.

17. The method of claim 16, wherein: r(s i ,a i ) are configured to facilitate minimization of delays and maximization of resource utilization; and / or The larger the γ, the greater the impact of future rewards on the current scheduling strategy.

18. The method according to claim 1, wherein: The emergencies include: equipment failure, bad weather, and passenger emergencies.

19. The method according to claim 1, wherein: The train operation status includes the current position, speed, and planned arrival time of the train; the resource allocation status includes station status and track availability; the external environment status includes weather conditions, equipment conditions, and passenger conditions; and the actions that can be performed by the train dispatching system include determining train departure and arrival times, determining train operation priorities, selecting tracks, and selecting stations.

20. A method for training a neural network model for train rescheduling, the neural network model comprising a diffusion model and a reinforcement learning model, the method comprising: Acquire historical time series data indicating the status at each time point in the historical operation process of the train and a historical train dispatching strategy corresponding to the historical operation process of the train, wherein the status includes the train operation status, resource allocation status and external environment status, and the historical train dispatching strategy includes actions executed by the train dispatching system; adding noise to the historical time series data based on a positive diffusion process via the diffusion model to obtain noisy historical time series data; The reinforcement learning model is trained by using the noisy historical time series data as sample data and the historical train scheduling strategy as label data.

21. The method according to claim 20, wherein: Adding noise to the historical time series data based on a positive diffusion process includes: in, The historical time series data includes (T+1) moments, and the historical time series data indicates the state x0 at moment 0 and the state x at moment T. T , The positive diffusion process is defined as following a Markov chain consisting of T time steps from state x0 to state x T diffusion process, t is the time step t in the Markov chain, where t∈[1,T], Status x t represents the state x0 after t steps of noise addition, α t represents the weight coefficient used to control the noise increase at time step t, ε represents Gaussian noise.

22. The method according to claim 21, wherein: α t It decreases with the increase of t.

23. The method according to claim 21, wherein: σ t represents the noise coefficient at time step t, σ t Determined based on the contingency to be simulated at time step t.

24. The method according to claim 23, wherein: σ t Determined based on one of the following noise strategies: Based on the linear noise strategy, σ t Determined as or Based on the cosine noise strategy, σ t Determined as or Based on the exponential noise strategy, σ t Determined as in, σ min represents the minimum noise coefficient for T time steps, σ max represents the maximum noise factor over T time steps.

25. The method according to claim 24, comprising: In response to no unexpected event occurring during simulation at time step t, selecting the linear noise strategy; In response to a simulated occurrence of an emergency event with a first priority at time step t, selecting the cosine noise strategy; The exponential noise strategy is selected in response to a simulated occurrence of an emergency event having a second priority at time step t, the second priority being higher than the first priority.

26. The method according to claim 24, wherein: s min =θ.1,σ max =1.

27. A computing device comprising: processor; as well as A memory storing computer executable instructions which, when executed by the processor, cause the processor to perform a method according to any one of claims 1 to 26.

28. A non-transitory storage medium having stored thereon computer executable instructions which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 26.

29. A computer program product comprising instructions which, when executed by a processor, implement the method according to any one of claims 1 to 26.

Images