Method and related apparatus for train rescheduling based on neural network

Abstract

The present disclosure relates to a method for train rescheduling based on neural network and related apparatus. A method for train rescheduling comprises: obtaining time series data indicating states at each time instant in a train operation process, the states comprising train operation states, resource allocation states and external environment states; providing the obtained time series data to a trained neural network model to generate a scheduling policy for train rescheduling, the scheduling policy comprising actions performed by a train scheduling system, wherein the time series data has different noise in a case where a sudden event occurs in the train operation process compared to a case where no sudden event occurs in the train operation process, and wherein the neural network model comprises a diffusion model and a reinforcement learning model, the diffusion model is configured to remove noise in the time series data based on an inverse diffusion process to obtain denoised time series data, and the reinforcement learning model is configured to generate the scheduling policy based on the denoised time series data.

Description

Technical Field

[0001] This disclosure relates to the field of train scheduling technology, and more specifically, to a method for train rescheduling, a method for training a neural network model for train rescheduling, and related computing devices, non-transient storage media, and computer program products. Background Technology

[0002] With the advancement of artificial intelligence, big data, and multimodal information fusion technologies, train dispatching systems are gradually evolving from traditional methods that rely on fixed train schedules and manual decision-making towards intelligent, automated, and unmanned operation. Summary of the Invention

[0003] A brief overview of this disclosure is given below to provide a basic understanding of some aspects of it. However, it should be understood that this overview is not an exhaustive summary of this disclosure. It is not intended to identify key or essential parts of this disclosure, nor is it intended to limit the scope of this disclosure. Its purpose is merely to present certain concepts of this disclosure in a simplified form as a prelude to the more detailed description that follows.

[0004] According to a first aspect of this disclosure, a method for train rescheduling is provided, comprising: acquiring time-series data indicating the state at various times during train operation, the state including train operation state, resource allocation state, and external environment state; providing the acquired time-series data to a trained neural network model to generate a scheduling strategy for train rescheduling, the scheduling strategy including actions performed by a train scheduling system, wherein the time-series data has different noise levels when an emergency occurs during train operation compared to when no emergency occurs during train operation, and wherein the neural network model includes a diffusion model and a reinforcement learning model, the diffusion model being configured to remove noise from the time-series data based on a reverse diffusion process to obtain denoised time-series data, and the reinforcement learning model being configured to generate the scheduling strategy based on the denoised time-series data.

[0005] According to a second aspect of this disclosure, a method is provided for training a neural network model for train rescheduling. The neural network model includes a diffusion model and a reinforcement learning model. The method includes: acquiring historical time-series data indicating the state at each moment in the historical operation of a train and historical train scheduling strategies corresponding to the historical operation of the train. The states include train operation state, resource allocation state, and external environment state. The historical train scheduling strategies include actions performed by a train scheduling system. Noise is added to the historical time-series data via a diffusion model based on a positive diffusion process to obtain noisy historical time-series data. The noisy historical time-series data is used as sample data, and the historical train scheduling strategies are used as label data to train the reinforcement learning model.

[0006] According to a third aspect of this disclosure, a computing device is provided, including a processor and a memory storing computer-executable instructions, which, when executed by the processor, cause the processor to perform a method for train rescheduling according to a first aspect of this disclosure or a method for training a neural network model for train rescheduling according to a second aspect of this disclosure.

[0007] According to a fourth aspect of this disclosure, a non-transient storage medium having computer-executable instructions stored thereon is provided, which, when executed by a processor, cause the processor to perform the method for train rescheduling according to a first aspect of this disclosure or the method for training a neural network model for train rescheduling according to a second aspect of this disclosure.

[0008] According to a fifth aspect of this disclosure, a computer program product is provided, the computer program product including instructions that, when executed by a processor, implement the method for train rescheduling according to a first aspect of this disclosure or the method for training a neural network model for train rescheduling according to a second aspect of this disclosure. Attached Figure Description

[0009] The foregoing and other features and advantages of this disclosure will become clear from the following description of embodiments illustrated in conjunction with the accompanying drawings. The drawings, incorporated herein and forming a part of the specification, are further used to explain the principles of this disclosure and to enable those skilled in the art to make and use it. Wherein:

[0010] Figure 1 A flowchart of a method for train rescheduling according to some embodiments of the present disclosure is shown;

[0011] Figure 2 A flowchart illustrating a method for training a neural network model for train rescheduling according to some embodiments of the present disclosure is shown;

[0012] Figure 3 A schematic block diagram of a computing device according to some embodiments of the present disclosure is shown;

[0013] Figure 4 A schematic block diagram of a computer system on which embodiments of the present disclosure may be implemented is shown.

[0014] Note that in the embodiments described below, the same reference numerals are sometimes used across different figures to denote the same parts or parts with the same function, and repeated descriptions are omitted. In some cases, similar reference numerals and letters are used to denote similar items, so once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0015] For ease of understanding, the positions, dimensions, and extents of the structures shown in the accompanying drawings and other materials may not represent actual positions, dimensions, and extents. Therefore, this disclosure is not limited to the positions, dimensions, and extents disclosed in the accompanying drawings and other materials. Detailed Implementation

[0016] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0017] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the scope of this disclosure or its application or use. That is, the structures and methods herein are shown in an exemplary manner to illustrate different embodiments of the structures and methods in this disclosure. However, those skilled in the art will understand that they merely illustrate exemplary ways that can be used to implement this disclosure, and not exhaustive ways. Furthermore, the drawings are not necessarily drawn to scale, and some features may be enlarged to show details of specific components.

[0018] In addition, techniques, methods and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods and equipment should be considered part of the specification.

[0019] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0020] Currently, train dispatching systems still primarily rely on fixed train timetables and manual dispatching decisions, exhibiting a low level of intelligence and lacking autonomous learning and decision-making capabilities. This makes it difficult to quickly and automatically respond to dynamically changing operating environments. Especially when facing emergencies (such as severe weather or equipment failures), the flexibility and real-time adjustment capabilities of train dispatching systems are clearly insufficient, hindering efficient adjustments to train operation plans and easily leading to train delays and service interruptions.

[0021] Furthermore, current train scheduling methods suffer from uneven resource allocation, resulting in suboptimal utilization of resources such as tracks, trains, and manpower. For example, resource shortages exacerbate this problem during peak train operating hours, while idle resources lead to waste during off-peak hours.

[0022] To address this, this disclosure provides a method for train rescheduling. By combining a diffusion model and a reinforcement learning model to obtain a neural network model for train rescheduling, it achieves autonomous learning and decision-making capabilities in complex situations, as well as emergency response capabilities to unforeseen events. According to this method, train operation plans can be dynamically adjusted in real time, resource allocation optimized, and unforeseen events flexibly responded to, thereby improving the overall operational efficiency and reliability of the train system.

[0023] The methods for train rescheduling according to various embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It will be understood that actual methods for train rescheduling may include other steps, but to avoid obscuring the essential points of the disclosure, these other steps will not be discussed herein and are not shown in the accompanying drawings.

[0024] Figure 1 A flowchart of a method 100 for train rescheduling according to some embodiments of the present disclosure is shown. Figure 1 As shown, method 100 includes steps S102 to S104.

[0025] In step S102, time-series data indicating the status at various points in time during train operation are acquired. The status includes train operation status, resource allocation status, and external environment status. In some embodiments, the train operation status may include the train's current location, speed, and scheduled arrival time, the resource allocation status may include station conditions and track availability, and the external environment status may include weather conditions, equipment conditions, and passenger conditions.

[0026] In this paper, "noise" can be used to describe the complexity and / or uncertainty of train scheduling scenarios or train operating environments. Specifically, time series data can have different levels of noise when an emergency occurs during train operation compared to when no emergency occurs.

[0027] In some embodiments, emergencies may include equipment failure (e.g., track signal failure, train failure, power outage, etc.), severe weather (e.g., heavy rain, heavy snow, strong winds, etc.), and passenger emergencies (e.g., emergency medical events, overloading, congestion, etc.).

[0028] It is understandable that the time-series data of train operation without any emergencies can have relatively stable noise distribution characteristics. Once an emergency occurs during train operation, the emergency will disturb the state at the corresponding moment (and possibly some subsequent moments), thereby changing the noise distribution characteristics of the time-series data, such as increased noise at that corresponding moment (and possibly some subsequent moments).

[0029] In some embodiments, for example, first and second bursts of different types or priorities, the time-series data may have different noise levels when the first burst occurs during train operation compared to when the second burst occurs during train operation. Specifically, different bursts can be characterized by noise with different characteristics, such as, but not limited to, the distribution to which the noise follows (e.g., a standard normal distribution or a Gaussian distribution), the noise variation strategy (i.e., a function of noise variation over time, such as a linear noise strategy, a cosine noise strategy, an exponential noise strategy, etc.), the noise weights, etc.

[0030] As an explanation, we can assume the first emergency is a track signal failure and the second emergency is a medical emergency. The train dispatching strategy for a track signal failure would be to resolve the equipment failure as quickly as possible to minimize its impact on train operations while ensuring normal train operation. Conversely, the train dispatching strategy for a medical emergency would be to quickly determine appropriate stops to provide sufficient medical resources for passengers. Since different types of emergencies cause varying degrees of disturbance to the train system, noise with different characteristics in the time-series data can characterize the corresponding different types of emergencies.

[0031] In some examples, different types of emergencies have different priorities. For example, track signal failures and severe weather events may be considered higher priorities, while passenger congestion and individual equipment failures may be considered lower priorities. Additionally, in some examples, even emergencies of the same type may have different priorities depending on their impact. For example, small-scale severe weather events (such as severe weather in a single station area) may have a lower priority than large-scale severe weather events (such as severe weather across an entire operating route). Therefore, emergencies of different priorities can be characterized by noise with different characteristics.

[0032] For example, assuming the train's origin and destination stations are Shanghai Station and Fuzhou Station respectively, the first emergency event is heavy rainfall in East China, and the second emergency event is heavy rainfall in a localized area of ​​Shanghai. Since the first emergency event is a large-scale severe weather event covering the entire train route, its priority is higher than the second emergency event. Accordingly, the weight of the noise used to characterize the first emergency event can be greater than the weight of the noise used to characterize the second emergency event, thus allowing the scheduling strategy generated by method 100 to prioritize the first emergency event.

[0033] In step S104, the acquired time-series data is provided to a trained neural network model to generate a scheduling strategy for train rescheduling. The scheduling strategy includes actions performed by the train scheduling system. In some embodiments, actions that can be performed by the train scheduling system include determining train departure and arrival times, determining train operation priorities, selecting tracks, and selecting stations.

[0034] This neural network model includes a diffusion model (DM) and a reinforcement learning model (RLM). Specifically, the diffusion model can be viewed as a type of variational inference that employs a denoising network to progressively converge toward the true samples across a sequence of estimation steps. The reinforcement learning model is a branch of machine learning models that involves an agent learning in an environment through trial and error and feedback, aiming to maximize cumulative reward. This paper does not impose any particular limitations on the specific model structures and algorithms of the diffusion model and the reinforcement learning model, and any diffusion model and reinforcement learning model known now or developed hereafter can be applied to various embodiments of this disclosure.

[0035] In the neural network model according to this disclosure, the diffusion model is configured to remove noise from the time series data based on the inverse diffusion process to obtain denoised time series data, and the reinforcement learning model is configured to generate a scheduling strategy based on the denoised time series data.

[0036] In some embodiments, removing noise from time-series data based on a reverse diffusion process includes: in,

[0037] The time series data includes (T+1) time points, where the time series data indicates state x0 at time 0 and state x at time T. T ,

[0038] The reverse diffusion process is defined as a process that proceeds from state x along a Markov chain consisting of T time steps. T The diffusion process to state x0, where t is the time step in the Markov chain, and t∈[1,T],

[0039] State x t-1 Represents state x t The state after one step of noise reduction.

[0040] ε θ (x t ,t) represents (specifically, through the denoising network in the diffusion model) for state x t The noise predicted at time step t, where θ represents the scheduling policy parameters.

[0041] α t=1-β t ,

[0042] σ t σ represents the noise figure at time step t. t It can also be called noise scheduling.

[0043] z represents Gaussian noise, and z ~ N(0,I) means that z is sampled from a standard normal distribution with a mean of 0 and a variance of I, which can be used to further adjust the denoising effect.

[0044] In some embodiments, σ t Determined based on the sudden event that occurs at time step t.

[0045] In some further embodiments, σ t It can be determined based on one of the following noise strategies: based on a linear noise strategy, σ t Determined as Based on the cosine noise strategy, σ t Determined as Based on the exponential noise strategy, σ t Determined as Where, σ min σ represents the minimum noise figure over T time steps. max This represents the maximum noise figure over T time steps. As a non-limiting example, σ can be set... min =0.1, σ max =1.

[0046] In the above embodiments, for different noise strategies, σ t Different patterns of change over time result in different noise terms σ. t The pattern of z changing over time varies. The appropriate noise strategy is selected based on whether and what kind of sudden event occurs. In some examples, σ changes more when a sudden event occurs compared to when no sudden event occurs. t The rate of change over time can increase. In some examples, the higher the priority of the sudden event, the higher the σ. t The rate of change over time can be greater. For example, a noise strategy can be selected based on the following operations: a linear noise strategy is selected in response to no burst event occurring at time step t; a cosine noise strategy is selected in response to a burst event with first priority occurring at time step t; and an exponential noise strategy is selected in response to a burst event with second priority occurring at time step t, where the second priority is higher than the first priority.

[0047] For example, during train operation, when no sudden event occurs, a linear noise strategy is selected, and σ is... t Determined as As the train operates, when a sudden event such as "heavy rainfall in parts of Shanghai" occurs, this event is considered to have the highest priority. Therefore, a cosine noise strategy is selected, and σ... t Determined as The noise at this point can be higher than the noise level assuming the event had not occurred. As time progresses and the event escalates to "heavy rainfall in East China," it is considered a second-priority event. Therefore, an exponential noise strategy is chosen, and σ is calculated based on this strategy. t Determined as At this point, the rate of change of noise over time will increase. By representing the occurrence and development of sudden events during train operation from the perspective of noise, the neural network model can adjust the train scheduling strategy in a timely manner, thereby realizing the rescheduling of trains in response to the sudden event.

[0048] Based on the aforementioned description of the priority of emergencies, it is clear that the appropriate noise strategy should be selected to configure σ according to the priority of the emergencies. t It can adaptively control the convergence speed of the neural network model to quickly adjust the train scheduling strategy, so that the train can respond to emergencies of different priorities in a timely manner according to the adjusted scheduling strategy.

[0049] Therefore, the diffusion model removes noise from time-series data through a reverse diffusion process, preprocessing the input data for the reinforcement learning model. This enables the reinforcement learning model to generate optimized and accurate scheduling strategies from complex and uncertain environments. Since the neural network model according to this disclosure makes decisions based on the denoised state, it ensures a more robust train scheduling process.

[0050] In some embodiments, the neural network model described above is trained through the following process: acquiring historical time-series data indicating the state at each moment in the historical operation of the train and the historical train scheduling strategy corresponding to the historical train operation, wherein the state includes the train operation state, resource allocation state, and external environment state, and the historical train scheduling strategy includes actions performed by the train scheduling system; adding noise to the historical time-series data based on a positive diffusion process using a diffusion model to obtain noisy historical time-series data; and using the noisy historical time-series data as sample data and the historical train scheduling strategy as label data to train a reinforcement learning model.

[0051] Furthermore, in some embodiments, adding noise to historical time-series data based on a forward diffusion process includes: in,

[0052] The historical time series data includes (T+1) time points. The historical time series data indicates state x0 at time 0 and state x at time T. T ,

[0053] The forward diffusion process is defined as moving along a Markov chain with T time steps from state x0 to state x. T The diffusion process, where t is the time step t in the Markov chain, and t∈[1,T].

[0054] State x t This represents the state of state x0 after t steps of noise addition.

[0055] α t This represents the weighting coefficient used to control the increase in noise at time step t.

[0056] ε represents Gaussian noise, and ε~N(0,I) means that σ is sampled from a standard normal distribution with a mean of 0 and a variance of I, which can be used to inject into the state.

[0057] In some embodiments, α t It decreases as t increases.

[0058] In other embodiments, σ t σ represents the noise figure at time step t. t The determination is based on the sudden events that need to be simulated at time step t.

[0059] In some further embodiments, σ t It can be determined based on one of the following noise strategies: based on a linear noise strategy, σ t Determined as Based on the cosine noise strategy, σ t Determined as Based on the exponential noise strategy, σ t Determined as As a non-restrictive example, σ can be set. min =0.1, σ max =1.

[0060] In the above embodiments, a noise strategy can be selected based on the following operations: a linear noise strategy is selected in response to the simulation of no burst event occurring at time step t; a cosine noise strategy is selected in response to the simulation of a burst event with a first priority occurring at time step t; and an exponential noise strategy is selected in response to the simulation of a burst event with a second priority occurring at time step t, wherein the second priority is higher than the first priority.

[0061] The diffusion model generates sample data by progressively adding noise to historical time-series data through a positive diffusion process. This enables reinforcement learning models to better learn how to make robust decisions in complex and uncertain environments. Reinforcement learning models trained in this way can handle environments with high noise levels, thus ensuring the generation of robust and efficient scheduling strategies even under conditions of high uncertainty and noise.

[0062] In some embodiments, the loss function of the neural network model is determined as: L(θ) = L d (θ)+L q (θ), where

[0063] θ represents the scheduling policy parameters, which can be optimized through training.

[0064] L(θ) represents the loss function of the neural network model.

[0065] L d (θ) represents the behavioral cloning loss, L d (θ) can be used to describe the difference between the action a generated based on θ and the action in the train historical scheduling strategy, and

[0066]

[0067] L q (θ) represents the Q-Learning loss, L q (θ) can be used to describe the (train) scheduling strategy π θ The differences from historical train scheduling strategies, and

[0068]

[0069] in,

[0070] E is the expectation symbol, representing the average value sampled from the empirical dataset D, which includes historical time-series data and historical train scheduling strategies.

[0071] i represents the time step in the diffusion process, and i ~ U indicates that i samples from the uniform distribution U. In the scheduling strategy, i can be understood as different stages of the scheduling process.

[0072] ε represents noise, and ε ~ N(0,I).

[0073] (s,a)~D represents the state s and the corresponding action a sampled from the empirical dataset D.

[0074] This represents (specifically, through a denoising network) the noise predicted for state s, action a, and time step i, used to predict the noise level of the train system during the diffusion process.

[0075] This represents the cumulative weighting coefficient used to control the increase in noise from time step 1 to time step i. ||·|| 2 This represents the square of the L2 norm, used to calculate the difference between predicted noise and actual noise.

[0076] α represents the learning rate coefficient, and 0 ≤ α ≤ 1.

[0077] s~D indicates that state s is sampled from empirical dataset D.

[0078] a0~π θ (·|s) represents action a0 from scheduling policy π θ Generated in (·|s), scheduling policy π θ (·|s) represents the probability distribution of choosing the corresponding action in state s.

[0079] Let Q be a function representing the expected cumulative reward for choosing action a0 in state s. These are the parameters of the Q-value function.

[0080] In some embodiments, the Q-value function includes:

[0081]

[0082] in,

[0083] (s i ,a i ,s i+1 ) ~D represents sampling state-action pairs from the empirical dataset D, where state s i+1 Indicates that in state s i Execute the selected action a i The next state after that,

[0084] Indicates the scheduling policy π θ′ Generate new scheduling actions Generated by scheduling policy parameter θ′

[0085] r(s i ,a i ) indicates that in state s i Next, execute action a i Instant rewards received

[0086] γ represents the degree of influence of future rewards on the current scheduling strategy, 0≤γ≤1.

[0087] Here, the min function ensures that the reinforcement learning model chooses relatively conservative actions to reduce risk.

[0088] r(s i ,a i R(s) can be used to measure the impact of actions performed in a scheduling strategy on various scheduling objectives (e.g., delays, resource utilization, etc.). It can be expressed as r(s) i ,a i Balancing multiple scheduling objectives. For example, in state s... i The following action a can achieve the desired goal i The immediate reward is set relatively large. In some embodiments, r(s) i ,a i ) is configured to promote minimizing delay time and maximizing resource utilization. By using r(s) i ,a i Configured to minimize delay time and maximize resource utilization, the scheduling strategy generated by the neural network model tends to minimize train delay time and maximize the utilization of various resources, thereby improving train operation efficiency and resource utilization. Of course, it is understandable that, based on the actual scheduling needs of the trains, r(s) i ,a i It can also be configured to balance other multiple scheduling objectives (e.g., minimizing the number of stops a train stops at, minimizing the travel time of a train on a particular journey, etc.) to generate different scheduling strategies.

[0089] Additionally or alternatively, in some embodiments, it can be achieved through r(s) i ,a i Imposing train scheduling constraints, such as train intervals and station stopping requirements, can better adapt to the actual needs of train scheduling.

[0090] Additionally, γ can be used to weigh short-term and long-term gains. A larger γ indicates a greater influence of future rewards on the current scheduling strategy. A larger γ value makes the model prioritize long-term returns, while a smaller γ value emphasizes immediate gains.

[0091] Therefore, through the forward diffusion process in model training and the reverse diffusion process in model inference, the train rescheduling method according to various embodiments of this disclosure can maintain stable scheduling performance under complex and ever-changing conditions, and has excellent scheduling efficiency and robustness.

[0092] This disclosure also provides a method for training a neural network model for train rescheduling. As previously described, the neural network model according to this disclosure includes a diffusion model and a reinforcement learning model. (Reference) Figure 2 The diagram illustrates a flowchart of a method 200 for training a neural network model for train rescheduling according to some embodiments of the present disclosure.

[0093] like Figure 2 As shown, method 200 includes: in step S202, acquiring historical time-series data indicating the state at each moment in the historical operation process of the train and the historical train scheduling strategy corresponding to the historical train operation process, the state including the train operation state, resource allocation state and external environment state, and the historical train scheduling strategy including actions executed by the train scheduling system; in step S204, adding noise to the historical time-series data based on the positive diffusion process through a diffusion model to obtain noisy historical time-series data; in step S206, using the noisy historical time-series data as sample data and the historical train scheduling strategy as label data to train a reinforcement learning model.

[0094] Various embodiments of method 200 can be similarly referred to the embodiments related to training neural network models in the aforementioned method 100, and will not be repeated here.

[0095] This disclosure also provides a computing device that may include a processor and a memory storing computer-executable instructions, which, when executed by the processor, cause the processor to perform a method for train rescheduling or a method for training a neural network model for train rescheduling according to any of the foregoing embodiments of this disclosure. Reference Figure 3 This illustrates a schematic block diagram of a computing device 300 according to some embodiments of the present disclosure. Figure 3As shown, computing device 300 includes a processor 302 and a memory 304 storing computer-executable instructions that, when executed by the processor 302, cause the processor 302 to perform a method for train rescheduling or a method for training a neural network model for train rescheduling according to any of the foregoing embodiments of this disclosure. The processor 302 may be, for example, a central processing unit (CPU) of computing device 300. The processor 302 may be any type of general-purpose processor, or it may be a processor specifically designed for train rescheduling or training a neural network model for train rescheduling, such as an application-specific integrated circuit (“ASIC”). The memory 304 may be coupled to the processor 302 and may include various computer-readable media accessible by the processor 302. In various embodiments, the memory 304 described herein may include volatile and non-volatile media, removable and non-removable media. For example, the memory 304 may include any combination of random access memory (“RAM”), dynamic RAM (“DRAM”), static RAM (“SRAM”), read-only memory (“ROM”), flash memory, cache memory, and / or any other type of non-transient computer-readable media. The memory 304 may store instructions that, when executed by the processor 302, cause the processor 302 to execute the method for train rescheduling or the method for training a neural network model for train rescheduling according to any of the foregoing embodiments of the present disclosure.

[0096] This disclosure also provides a non-transient storage medium having computer-executable instructions stored thereon, which, when executed by a processor, cause the processor to execute the method for train rescheduling according to any of the foregoing embodiments of this disclosure or the method for training a neural network model for train rescheduling according to any of the foregoing embodiments.

[0097] This disclosure also provides a computer program product that may include instructions that, when executed by a processor, can implement the method for train rescheduling according to any of the foregoing embodiments of this disclosure or the method for training a neural network model for train rescheduling according to any of the foregoing embodiments. The instructions may be any set of instructions that will be executed directly by a processor, such as machine code, or any set of instructions that will be executed indirectly, such as a script. The instructions may be stored in an object code format for direct processing by a processor, or stored in any other computer language, including scripts or sets of independent source code modules that are interpreted on demand or compiled in advance.

[0098] Figure 4A schematic block diagram of a computer system 400 on which embodiments of the present disclosure may be implemented is shown. The computer system 400 includes a bus 402 or other communication mechanism for transmitting information, and a processing means 404 coupled to the bus 402 for processing information. The computer system 400 also includes a memory 406 coupled to the bus 402 for storing instructions to be executed by the processing means 404; the memory 406 may be random access memory (RAM) or other dynamic storage device. The memory 406 may also be used to store temporary variables or other intermediate information during the execution of instructions to be executed by the processing means 404. The computer system 400 also includes a read-only memory (ROM) 408 or other static storage device coupled to the bus 402 for storing static information and instructions for the processing means 404. A storage device 410, such as a magnetic disk or optical disk, is provided and coupled to the bus 402 for storing information and instructions. Computer system 400 may be coupled via bus 402 to output device 412 for providing output to a user, such as, but not limited to, a display (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD)), speakers, etc. Input device 414, such as a keyboard, mouse, microphone, etc., is coupled to bus 402 for transmitting information and command selections to processing device 404. Computer system 400 may execute embodiments of this disclosure. Consistent with certain implementations of this disclosure, results are provided by computer system 400 in response to processing device 404 executing one or more sequences of one or more instructions contained in memory 406. Such instructions may be read into memory 406 from another computer-readable medium, such as storage device 410. Execution of the sequence of instructions contained in memory 406 causes processing device 404 to perform the methods described herein. Alternatively, the teachings may be implemented using hardwired circuitry in place of or in combination with software instructions. Therefore, implementations of this disclosure are not limited to any particular combination of hardware circuitry and software. In various embodiments, computer system 400 may be connected across a network to one or more other computer systems, such as computer system 400, via network interface 416 to form a networked system. This network may include a private network or a public network such as the Internet. In a networked system, one or more computer systems may store data and supply data to other computer systems. As used herein, the term "computer-readable medium" refers to any medium that participates in providing instructions to processing device 404 for execution. Such media may take many forms, including but not limited to non-volatile media, volatile media, and transmission media. Non-volatile media include, for example, optical discs or magnetic disks such as storage device 410. Volatile media include dynamic memory such as memory 406. Transmission media include coaxial cables, copper wires, and optical fibers, including wiring that includes bus 402.Common forms of computer-readable media or computer program products include, for example, floppy disks, flexible disks, hard disks, magnetic tapes, or any other magnetic media, CD-ROMs, digital video discs (DVDs), Blu-ray discs, any other optical media, thumb drives, memory cards, RAM, PROMs and EPROMs, fast EPROMs, any other memory chips or cartridges, or any other tangible media from which a computer can read. Various forms of computer-readable media may be involved when carrying one or more sequences of one or more instructions to processing device 404 for execution. For example, instructions may initially be carried on a disk of a remote computer. The remote computer may load the instructions into its dynamic memory and transmit the instructions over a telephone line using a modem. A modem local to computer system 400 may receive data over a telephone line and convert the data into an infrared signal using an infrared transmitter. An infrared detector coupled to bus 402 may receive the data carried in the infrared signal and place the data on bus 402. Bus 402 carries the data to memory 406, from which processing device 404 retrieves and executes the instructions. Optionally, the instructions received by the memory 406 may be stored on the storage device 410 before or after execution by the processing device 404.

[0099] According to various embodiments, instructions configured to be executed by a processing device to perform a method are stored on a computer-readable medium. The computer-readable medium may be a device for storing digital information. For example, a computer-readable medium includes a compact disc read-only memory (CD-ROM) as known in the art for storing software. The computer-readable medium is accessed by a processor adapted to execute the instructions configured to be executed.

[0100] The foregoing has described one or more exemplary embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0101] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. A typical implementation device is a server system. Of course, this disclosure does not exclude the possibility that, with the future development of computer technology, the computer implementing the functions of the above embodiments can be, for example, a personal computer, a laptop computer, an in-vehicle human-machine interaction device, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0102] While one or more embodiments of this disclosure provide the method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only execution order. In actual device or terminal product execution, the methods shown in the embodiments or drawings can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment).

[0103] The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, product, or apparatus. Without further limitation, the presence of other identical or equivalent elements in the process, method, product, or apparatus that includes said elements is not excluded. For example, the use of terms such as "first" or "second" to denote names does not indicate any particular order.

[0104] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, when implementing one or more embodiments of this disclosure, the functions of each module can be implemented in one or more software and / or hardware, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0105] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0106] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0107] Those skilled in the art will understand that one or more embodiments of this disclosure may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0108] One or more embodiments of this disclosure can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. One or more embodiments of this disclosure can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can reside in local and remote computer storage media, including storage devices.

[0109] The same or similar parts between the various embodiments of this disclosure can be referred to mutually, and each embodiment focuses on describing the differences from other embodiments. In particular, for the apparatus embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and relevant parts can be referred to the description of the method embodiments. In the description of this disclosure, the descriptions of terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., mean that the specific feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of this disclosure. In this disclosure, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this disclosure and the features of the different embodiments or examples.

[0110] Additionally, when used in this disclosure, the terms “here,” “above,” “below,” “below,” “in the following,” “overall,” and similar terms should refer to the entirety of this disclosure and not any particular part thereof. Furthermore, unless expressly stated otherwise or otherwise understood in the context in which they are used, conditional language used herein, such as “may,” “possibly,” “for example,” “like,” etc., is generally intended to express that certain embodiments include, while other embodiments do not, certain features, elements, and / or states. Therefore, such conditional language is not generally intended to imply that one or more embodiments require features, elements, and / or states in any way, or whether such features, elements, and / or states are included or performed in any particular embodiment.

[0111] The above description is merely an embodiment of one or more embodiments of this disclosure and is not intended to limit the scope of the one or more embodiments of this disclosure. Various modifications and variations can be made to the one or more embodiments of this disclosure by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of the claims.

Claims

1. A method for train rescheduling, comprising: Acquire time-series data indicating the status at various points in time during train operation, including train operation status, resource allocation status, and external environment status; The acquired time-series data is fed into a trained neural network model to generate a scheduling strategy for train rescheduling, the scheduling strategy including actions to be performed by the train scheduling system. Specifically, the time-series data exhibits different noise levels when a sudden event occurs during train operation compared to when no sudden event occurs. The noise distribution characteristics of the time-series data during train operation with a sudden event change compared to the noise distribution characteristics of the time-series data during train operation without a sudden event. This change occurs at the corresponding moment when the sudden event occurs. The neural network model includes a diffusion model and a reinforcement learning model. The diffusion model is configured to remove noise from the time series data based on a reverse diffusion process to obtain denoised time series data. The reinforcement learning model is configured to generate the scheduling strategy based on the denoised time series data.

2. The method according to claim 1, wherein, The time-series data exhibits different noise levels when the first unexpected event occurs during train operation compared to when the second unexpected event occurs. Wherein, the first emergency has a different type than the second emergency, or the first emergency has a different priority than the second emergency.

3. The method according to claim 1, wherein, Removing noise from the time-series data based on the reverse diffusion process includes: ， in, The time series data includes (T+1) time points, and the time series data indicates the state at time 0. and the state indicated at time T , The reverse diffusion process is defined as a process along a Markov chain consisting of T time steps from state to state. to state The diffusion process, where t is the time step in the Markov chain, and t∈[1,T]. state Representing state The state after one step of noise reduction. Indicating targeting and time step Predicted noise, Indicates scheduling policy parameters, , , This represents the noise figure at time step t. This represents Gaussian noise.

4. The method according to claim 3, wherein, Determined based on the sudden event that occurs at time step t.

5. The method according to claim 4, wherein, Determined based on one of the following noise strategies: Based on the linear noise strategy, Determined as ; or Based on the cosine noise strategy, Determined as ; or Based on the exponential noise strategy, Determined as , in, This represents the minimum noise figure over T time steps. This represents the maximum noise figure over T time steps.

6. The method according to claim 5, comprising: In response to the absence of a sudden event at time step t, the linear noise strategy is selected; In response to a burst event of first priority occurring at time step t, the cosine noise strategy is selected; In response to a burst event with a second priority occurring at time step t, the exponential noise strategy is selected, where the second priority is higher than the first priority.

7. The method according to claim 5, wherein, ， 。 8. The method according to claim 1, wherein, The neural network model is trained through the following process: The system acquires historical time-series data indicating the status at each moment during the historical operation of a train, as well as historical train scheduling strategies corresponding to the historical operation of the train. The status includes train operation status, resource allocation status, and external environment status. The historical train scheduling strategy includes actions executed by the train scheduling system. Noise is added to the historical time series data based on the positive diffusion process using the diffusion model to obtain noisy historical time series data. The reinforcement learning model is trained using the noisy historical time-series data as sample data and the historical train scheduling strategy as label data.

9. The method according to claim 8, wherein, Adding noise to the historical time-series data based on the forward diffusion process includes: ,in, The historical time series data includes (T+1) time points, and the historical time series data indicates the state at time 0. and the state indicated at time T , The forward diffusion process is defined as a process that proceeds from state T along a Markov chain consisting of T time steps. arrive The diffusion process, where t is the time step t in the Markov chain, and t∈[1,T]. state Representing state go through The state after adding noise. This represents the weighting coefficient used to control the increase in noise at time step t. This represents Gaussian noise.

10. The method according to claim 9, wherein, along with It decreases as it increases.

11. The method according to claim 9, wherein, , This represents the noise figure at time step t. The determination is based on the sudden events that need to be simulated at time step t.

12. The method according to claim 11, wherein, Determined based on one of the following noise strategies: Based on the linear noise strategy, Determined as ; or Based on the cosine noise strategy, Determined as ; or Based on the exponential noise strategy, Determined as , in, This represents the minimum noise figure over T time steps. This represents the maximum noise figure over T time steps.

13. The method of claim 12, comprising: In response to the fact that no sudden event occurs during the simulation at time step t, the linear noise strategy is selected; In response to a burst event of first priority occurring at time step t, the cosine noise strategy is selected; In response to a burst event with a second priority occurring at time step t, the exponential noise strategy is selected, where the second priority is higher than the first priority.

14. The method according to claim 12, wherein, ， 。 15. The method according to claim 8, wherein, The loss function of the neural network model is determined as follows: ,in, Indicates scheduling policy parameters, This represents the loss function of the neural network model. This represents the behavioral cloning loss, and ， This represents Q-Learning loss, and ， in, The expected symbol represents the value derived from the empirical dataset. The average value of the sampled data, empirical dataset This includes the historical time-series data and the historical train scheduling strategy. This represents the time step in the diffusion process. express From uniform distribution Mid-sampling, Indicates noise. express Sampling is performed from a standard normal distribution with a mean of 0 and a variance of I. Representing state and corresponding actions From empirical datasets Mid-sampling, Indicates the state ,action and time step Predicted noise, Indicates from time step 1 to time step 2. The cumulative weighting coefficient used to control noise increase. , This represents the weighting coefficient used to control the increase in noise at time step t. This represents the learning rate coefficient. , Representing state From empirical datasets Mid-sampling, Indicates action From scheduling strategy Generation and scheduling strategies Representing state The probability distribution of choosing the corresponding action. It is a Q-value function, representing the state. Select action The expected cumulative reward, These are the parameters of the Q-value function.

16. The method according to claim 15, wherein, Q-value functions include: ， in, Indicates from empirical dataset Mid-sampled state-action pairs, state Indicates the state Execute the selected action The next state after that, Indicates from scheduling policy Generate new scheduling actions , Based on scheduling policy parameters generate, Indicates the state Next action Instant rewards received This indicates the degree to which future rewards influence the scheduling strategy determined in the present moment. .

17. The method of claim 16, wherein: Configured to promote minimizing delay time and maximizing resource utilization; and / or The larger This indicates that the greater the impact of future rewards on the scheduling strategy determined at the moment, the better.

18. The method according to claim 1, wherein, The emergencies mentioned include: equipment failure, severe weather, and passenger emergencies.

19. The method according to claim 1, wherein, The train operation status includes the train's current location, speed, and planned arrival time. The resource allocation status includes station status and track availability. The external environment status includes weather conditions, equipment status, and passenger status. Actions that can be performed by the train dispatching system include determining train departure and arrival times, determining train operation priorities, selecting tracks, and selecting stations.

20. A method for training a neural network model for train rescheduling, said neural network model comprising a diffusion model and a reinforcement learning model, said method comprising: The system acquires historical time-series data indicating the status at each moment during the historical operation of a train, as well as historical train scheduling strategies corresponding to the historical operation of the train. The status includes train operation status, resource allocation status, and external environment status. The historical train scheduling strategy includes actions executed by the train scheduling system. Noise-added historical time-series data is obtained by adding noise to the historical time-series data based on the positive diffusion process via the diffusion model. The noise distribution characteristics of the historical time-series data of the train historical operation process in which no sudden event occurred are different from those of the historical time-series data of the train historical operation process in which a sudden event occurred. The change occurs at the corresponding moment when the sudden event occurs. The reinforcement learning model is trained using the noisy historical time-series data as sample data and the historical train scheduling strategy as label data.

21. The method according to claim 20, wherein, Adding noise to the historical time-series data based on the forward diffusion process includes: ,in, The historical time series data includes (T+1) time points, and the historical time series data indicates the state at time 0. and the state indicated at time T , The forward diffusion process is defined as a process that proceeds from state T along a Markov chain consisting of T time steps. arrive The diffusion process, where t is the time step t in the Markov chain, and t∈[1,T]. state Representing state go through The state after adding noise. This represents the weighting coefficient used to control the increase in noise at time step t. This represents Gaussian noise.

22. The method according to claim 21, wherein, along with It decreases as it increases.

23. The method according to claim 21, wherein, , This represents the noise figure at time step t. The determination is based on the sudden events that need to be simulated at time step t.

24. The method according to claim 23, wherein, Determined based on one of the following noise strategies: Based on the linear noise strategy, Determined as ; or Based on the cosine noise strategy, Determined as ; or Based on the exponential noise strategy, Determined as , in, This represents the minimum noise figure over T time steps. This represents the maximum noise figure over T time steps.

25. The method of claim 24, comprising: In response to the fact that no sudden event occurs during the simulation at time step t, the linear noise strategy is selected; In response to a burst event of first priority occurring at time step t, the cosine noise strategy is selected; In response to a burst event with a second priority occurring at time step t, the exponential noise strategy is selected, where the second priority is higher than the first priority.

26. The method according to claim 24, wherein, ， 。 27. A computing device, comprising: processor; as well as A memory storing computer-executable instructions, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 26.

28. A non-transient storage medium having stored thereon computer-executable instructions, which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 26.

29. A computer program product comprising instructions that, when executed by a processor, implement the method according to any one of claims 1 to 26.

Images