Novel intelligent optimization method for running speed curve of heavy-load train
By optimizing the speed curve of heavy-haul trains using a deep reinforcement learning method based on historical data, the problems of high model dependence and poor adaptability of traditional methods are solved, achieving smooth, punctual, and energy-saving train operation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-25
- Publication Date
- 2026-03-10
AI Technical Summary
Existing methods for optimizing the speed curves of heavy-haul trains rely on accurate train dynamics models, which make it difficult to respond quickly under complex operating conditions while balancing safety and efficiency. Furthermore, existing reinforcement learning methods lack generalization ability in practical applications.
A deep reinforcement learning method based on historical data is adopted to construct a reinforcement learning environment that integrates static and dynamic data of the railway line. The neural network is trained using real operation data to optimize the speed curve of heavy-load trains and achieve multi-objective collaborative optimization through a reward mechanism.
It improves the smoothness, punctuality, and energy efficiency of heavy-haul trains, reduces the risk of operational errors, and enhances transportation safety and efficiency.
Smart Images
Figure CN121638005A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent railway transportation control technology, specifically to a novel intelligent optimization method for the operating speed curve of heavy-haul trains. Background Technology
[0002] The railway transportation system is characterized by complex line environments, diverse external interference factors, and frequent long-distance cross-line operations. Due to their large load capacity, significant nonlinearity of air braking, and drastic changes in line gradient, heavy-haul trains exhibit strong nonlinearity and uncertainty in their operation. Currently, heavy-haul trains in my country are still mainly driven manually. Drivers need to continuously adjust their operating strategies under high-intensity working conditions. However, with the expansion of train formation size and the increase in operating distance, the risk of operating errors increases significantly, which can easily lead to safety hazards such as decoupling and broken coupling, directly affecting transportation efficiency and train operation safety.
[0003] Optimizing train speed curves is a core technology for improving the automation and intelligence of trains. Its generation requires comprehensive consideration of multiple factors, including train dynamics characteristics (such as heavy load and axle load), nonlinear constraints (such as the accuracy of aerodynamic braking modeling), track conditions (such as gradient and speed limit), and environmental interference (such as tunnels and strong winds). Traditional speed curve optimization algorithms rely on accurate train dynamics models, but in actual operation, model parameters are difficult to calibrate in real time, and it is difficult to balance the requirements of rapid response and smooth safety under complex operating conditions. For example, problems such as the dynamic matching of aerodynamic braking charging time and minimum release speed, and the collaborative optimization of multiple objectives (timeliness, energy consumption, and comfort) all pose challenges to the adaptability and robustness of the algorithms.
[0004] In recent years, with the improvement of computing power and the development of artificial intelligence technology, model-free optimization methods based on reinforcement learning have shown significant advantages in multi-stage decision-making problems. Through self-learning and interaction with environmental data, they can dynamically adapt to uncertain scenarios and approximate the optimal solution without relying on an accurate model. However, existing methods mostly generate training data based on simulation environments, which deviates from actual line conditions, resulting in insufficient generalization ability of optimization results in practical applications. Therefore, how to combine real historical data with reinforcement learning technology to achieve real-time and accurate optimization of the speed curve of heavy-haul trains has become a key issue in improving the safety, energy efficiency and punctuality of train operation.
[0005] To address the aforementioned technical bottlenecks, this invention proposes an intelligent optimization method for speed curves based on deep reinforcement learning using historical data. By constructing a reinforcement learning environment that integrates static line data and dynamic train data, and training neural network parameters using real operational data, this method achieves collaborative optimization of multiple objectives (stability, punctuality, and energy consumption). This solves the problems of high model dependence and poor adaptability in traditional methods, providing technical support for automated driving of heavy-haul trains. Summary of the Invention
[0006] To address the aforementioned issues, this invention provides a novel intelligent optimization method for the speed curve of heavy-haul trains to improve the train's stability, punctuality, and energy efficiency. This novel intelligent optimization method for the speed curve of heavy-haul trains employs reinforcement learning and historical operating data of heavy-haul trains to train neural network parameters. This ensures that the optimization of the speed curve is independent of specific train models, avoiding the adverse effects of complex and variable train operating environments on the solution. Simultaneously, it learns from real historical data, calculates reward functions, improves the learning model, and enhances the speed and quality of finding the approximate optimal solution for the speed curve.
[0007] The technical solution of the present invention is as follows:
[0008] A novel intelligent optimization method for the speed curve of heavy-haul trains includes the following steps:
[0009] Step S1: Construct a train operation reinforcement learning environment based on the dynamic model of heavy-haul trains that takes into account train length, static data of the line, and dynamic data such as train operation and braking status;
[0010] Step S2: Establish a reward mechanism based on target optimization, with the goals of smooth, punctual, and energy-efficient train operation;
[0011] Step S3: Collect historical train operation data, build a historical information database, and process the data based on a reward mechanism;
[0012] Step S4: Through interaction between the agent and the reinforcement learning environment, train the neural network model using the historical data processed in Step 3 to optimize the working conditions and actions.
[0013] Furthermore, in step S1, the static data of the line includes the line mileage, gradient, and section speed limit.
[0014] Furthermore, in step S1, the train operation dynamic data includes the train's current operating position, speed, operating time, brake pipe pressure, and locomotive brake cylinder pressure.
[0015] Furthermore, in step S1, the reinforcement learning environment is constructed based on the train dynamics model, including:
[0016]
[0017] In the formula: The train's position is in meters (m).
[0018] The train's speed is expressed in m / s.
[0019] This refers to the train's slewing mass coefficient.
[0020] The unit traction force of the train, N / kN;
[0021] The unit electric braking force of the train is N / KN;
[0022] The unit air braking force of the train, N / KN;
[0023] The basic operating resistance per unit of train, N / KN;
[0024] Add gradient resistance to the train unit, N / KN;
[0025] t represents time, in seconds;
[0026]
[0027] In the formula: Parameters describing the operation of train air brakes;
[0028] t represents time, in seconds;
[0029] Taking into account the train length, additional track resistance is also calculated. The calculation formula is:
[0030]
[0031] In the formula: The length of the train is in meters (m).
[0032] , , The first covered by the train The unit additional resistance of ramps, curves, and tunnels, in N / kN;
[0033] , , The first covered by the train The length of each ramp, curve, and tunnel, in meters.
[0034] Furthermore, in step S2, the target optimization includes improving the smoothness of heavy-haul trains, the on-time performance of train operations, and reducing train operation energy consumption.
[0035] Furthermore, in step S2, the reward function in the reward mechanism is represented as the sum of time reward, energy consumption reward, and comfort reward, expressed as:
[0036]
[0037] In the formula: For the reward function;
[0038] For the reward function related to timeliness;
[0039] For the reward function related to energy consumption;
[0040] For a stable reward function.
[0041] Furthermore, the reward function in the reward mechanism is represented as a weighted sum of time reward, energy consumption reward, and comfort reward, expressed as:
[0042]
[0043] In the formula: , , These are the corresponding weights for the reward functions related to timeliness, energy consumption, and stability;
[0044] .
[0045] Furthermore, in step S3, the historical train operation data includes train operation status information, and the operation status of each train is set as follows:
[0046]
[0047] In the formula: , , , , , , The train is currently The position, speed, acceleration value, gradient, speed limit value, train brake pipe pressure, and locomotive brake cylinder pressure under the specified conditions.
[0048] Furthermore, in step S4, the interactive learning between the agent and the reinforcement learning environment includes the following steps:
[0049] The intelligent agent acts as an on-board controller, forming a closed-loop interactive system with the reinforcement learning environment;
[0050] After receiving the working condition actions output by the intelligent agent, the environment generates new states, reward values, and state value functions in real time and feeds them back to the intelligent agent.
[0051] The agent evaluates and improves policies through value functions, and selects the optimal action based on the maximum action value function.
[0052] The selected action is fed back to the environment, and the operating condition values are updated iteratively to ultimately generate an optimized train speed curve.
[0053] Furthermore, the interaction learning between the agent and the reinforcement learning environment is implemented using a deep Q-learning network algorithm.
[0054] The beneficial effects of this invention are as follows:
[0055] 1. This invention discloses a novel intelligent optimization method for the speed curve of heavy-haul trains. This novel intelligent optimization method for the speed curve of heavy-haul trains adopts reinforcement learning and uses historical operating data of heavy-haul trains to train neural network parameters. On the one hand, it makes the optimization of the speed curve independent of the specific train model, avoiding the adverse effects of the complex and ever-changing train operating environment on the solution. On the other hand, it learns from real historical data, calculates reward functions, improves the learning model, and improves the speed and quality of solving the approximate optimal solution of the speed curve.
[0056] 2. The present invention discloses a novel intelligent optimization method for the operating speed curve of heavy-haul trains. This novel intelligent optimization method for the operating speed curve of heavy-haul trains performs multi-objective optimization on the operating speed curve of heavy-haul trains. The optimization objectives include train smoothness, punctuality and energy consumption, ensuring that the train achieves a certain degree of energy saving while being as punctual as possible, and improving passenger comfort.
[0057] 3. The present invention discloses a novel intelligent optimization method for the operating speed curve of heavy-haul trains. This novel intelligent optimization method for the operating speed curve of heavy-haul trains utilizes real data, including train operation, track basic data and train operation condition data, which have a wide range and large amount of data.
[0058] 4. The present invention discloses a novel intelligent optimization method for the operating speed curve of heavy-haul trains. This novel intelligent optimization method for the operating speed curve of heavy-haul trains adopts an intelligent algorithm based on reinforcement learning, which can quickly generate high-quality train operating speed curves. Attached Figure Description
[0059] Figure 1 This is a structural diagram of the reinforcement learning method according to an embodiment of the present invention;
[0060] Figure 2 This is a structural diagram of the train speed curve optimization method based on reinforcement learning, according to an embodiment of the present invention. Detailed Implementation
[0061] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings. Similar elements in different embodiments are referred to by associated similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of this application. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, certain operations related to this application are not shown or described in the specification. This is to avoid obscuring the core parts of this application with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.
[0062] Furthermore, the features, operations, or characteristics described in the specification can be combined in any suitable manner to form various embodiments. At the same time, the steps or actions in the method description can be rearranged or adjusted in a manner obvious to those skilled in the art. Therefore, the various orders in the specification and drawings are only for the clear description of a particular embodiment and do not imply a necessary order, unless otherwise stated that a particular order must be followed.
[0063] The new intelligent optimization method for the speed curve of heavy-haul trains includes the following steps:
[0064] S1: Based on the dynamics model of heavy-haul trains that takes into account train length, a reinforcement learning environment for train operation is built according to static track data and dynamic train operation and braking data. The static track data includes track mileage, gradient, and speed limit. The dynamic train operation data includes the current train position, speed, train operation time, train brake line pressure, and locomotive brake cylinder pressure.
[0065] The research object of this invention is heavy-haul trains. Compared with high-speed trains, the trains are much longer. Therefore, a train dynamics model that takes into account the train length is established, while the train length of high-speed trains does not need to be considered. Since the braking mechanisms of heavy-haul trains and high-speed trains are different (high-speed trains use electric braking, while heavy-haul trains use electro-pneumatic cycle braking, and the delay of air braking is very large), it is necessary to consider the braking state of heavy-haul trains.
[0066] S2: With the optimization goals of smooth, punctual, and energy-saving train operation, a reward mechanism is established, in which the on-board controller determines the reward value corresponding to different working actions in each state.
[0067] The optimization goals of heavy-haul trains and high-speed trains are different. Heavy-haul trains aim for smooth operation and reduced pressure on the couplers, while high-speed trains aim for passenger comfort. These are fundamentally different.
[0068] S3: Collect train operation data from real railway scenarios, including train location, speed, line mileage, gradient, etc., and build a historical information database; process the data in the historical database according to the reward mechanism in step S2 for training neural network parameters;
[0069] S4: The train agent and the reinforcement learning environment continuously interact and learn, improving the operating conditions output by the onboard controller. The value function is updated using a neural network trained with historical data obtained in step S3. The reinforcement learning process adapts to heavy-haul trains with different loads and lengths based on the established heavy-haul train dynamics model; high-speed trains do not need to consider different loads and lengths.
[0070] In the above offline training process, steps S1-S4 are repeatedly performed in the reinforcement learning-based train speed curve optimization method until neural network parameters with good training effect are obtained, of which S4 is the core part of the present invention.
[0071] In step S1, a train operation reinforcement learning environment is constructed. The data in the reinforcement learning environment includes static line data and dynamic train operation data. The static line data includes line mileage, gradient, and section speed limit. The dynamic train operation data includes the current train position, speed, and train operation time.
[0072] Using this data, constraints are given to the solution of train speed, reducing the solution space:
[0073]
[0074] In the formula: The train's position is in meters (m).
[0075] The train's speed is expressed in m / s.
[0076] This refers to the train's slewing mass coefficient.
[0077] The unit traction force of the train, N / kN;
[0078] The unit electric braking force of the train is N / KN;
[0079] The unit air braking force of the train, N / KN;
[0080] The basic operating resistance per unit of train, N / KN;
[0081] Add gradient resistance to the train unit, N / KN;
[0082] t represents time, in seconds;
[0083]
[0084] In the formula: Parameters describing the operation of train air brakes;
[0085] t represents time, in seconds;
[0086] Taking into account the train length, additional track resistance is also calculated. The calculation formula is:
[0087]
[0088] In the formula: The length of the train is in meters (m).
[0089] , , The first covered by the train The unit additional resistance of ramps, curves, and tunnels, in N / kN;
[0090] , , The first covered by the train The length of each ramp, curve, and tunnel, in meters.
[0091] The shortest running time curve of the train under maximum traction, maximum braking and section speed limit constraints is given. This curve gives the maximum speed that the train can reach. It serves as a constraint condition for data sampling using similarity in step S3, so as to obtain a solution space that conforms to the actual train operation.
[0092] In step S2, the objective of this invention is to improve the smoothness and punctuality of heavy-haul trains, and reduce train operation energy consumption. The reward function is the optimization objective; therefore, it is set as a correlation function of train running time, energy consumption, and acceleration. This can be expressed as follows:
[0093]
[0094] The design of the stable reward function is as follows:
[0095]
[0096] The reward function for timeliness is designed as follows:
[0097]
[0098] To ensure timeliness, at the end of each trial, an additional term will be added to the reward function related to timeliness, namely:
[0099]
[0100] The reward function for energy consumption is designed as follows:
[0101]
[0102] in, , These represent the actual time and the maximum time taken for the train to travel one step length, respectively. , These are the actual operating time and the planned operating time between stations, respectively. , These are the train acceleration and the train position step length, respectively. This represents the total number of steps. This represents the maximum impact rate during train operation.
[0103] If different performance indicators have different requirements, a weight can be set for each performance indicator, as shown below:
[0104]
[0105]
[0106] , , These are the weights for the time, energy consumption, and comfort functions, respectively.
[0107] After establishing the above reward mechanism, the reward value for the train to take different actions in each state can be obtained. For example... Figure 1 As shown, the information such as time and acceleration in the reward function is obtained from step S1, that is, the reward value is provided by the train operation reinforcement learning environment.
[0108] In step S3, the collected train operation data includes the train's location, speed, acceleration / deceleration values, and time. The operating status of each train is set as follows:
[0109]
[0110] In the formula: , , , , , , The train is currently The position, speed, acceleration value, gradient, speed limit value, train brake pipe pressure, and locomotive brake cylinder pressure under the specified conditions.
[0111] This invention requires generating data for reinforcement learning training from these historical data. Specifically, for the current train operating state, it identifies the set of states with the highest similarity from this data set, and the similarity between data points can be measured using Manhattan distance.
[0112]
[0113] in, , They are respectively and The first in Each element.
[0114] For each state Find using the above method The closest state Accordingly, this can be obtained Each state corresponds to a state-behavior pair. It is worth noting that It is not fixed; it is determined based on the constraints provided in step S1.
[0115] The train's trajectory can then be described as a set of states and actions, as shown below:
[0116]
[0117] Multiple train routes constitute a database of historical train operation information, namely:
[0118]
[0119] The historical information database in this step contains a large number of train operation trajectories. The reward mechanism provided in step S2 is used to calculate the reward for each state-action pair in each trajectory, resulting in the following data set:
[0120]
[0121] That is, the train historical information database is updated to The updated database information includes reward values for state-action pairs, which are then used for training the neural network in the next step, S4.
[0122] Step S4 is the core part of the present invention. In this part, the agent and the reinforcement learning environment continuously interact and learn, and through the evaluation and improvement of the policy value function, the optimal working condition action is selected and fed back to the train operation reinforcement learning environment.
[0123] like Figure 2 As shown, the intelligent agent is equivalent to an onboard controller. During train operation, the intelligent agent interacts with the environment. The environment generates new states, reward values, and state value functions, which are fed back to the intelligent agent. The intelligent agent continuously evaluates and improves the strategy through the value function, selects the maximum action value function, and feeds back the action corresponding to the maximum action value to the train operation reinforcement learning environment. Through such a closed-loop structure, the operating condition values are continuously updated, and finally the optimal operating condition action is selected to generate the optimal train speed curve, achieving the goals of energy saving, punctuality, and comfort.
[0124] The agent's action value function is updated using a deep Q-learning network, employing gradient descent as the update method.
[0125]
[0126] In the formula: , These are the network parameters for the target network and the network for value function approximation, respectively.
[0127] The basic process of optimizing the velocity curve using DQN is as follows:
[0128] Input: Status Train movement Value function Establish mapping
[0129] Initial capacity is Experience replay data area
[0130] Using random weights Initialize state-action value function
[0131] make Initialize the target neural network
[0132] start:
[0133] For the first training segment
[0134] Based on the dynamics model of a heavy-haul train considering train length, the initial state is obtained. (Initial state is zero vector)
[0135] for
[0136] Select from historical information database strip Data storage to middle
[0137] from Medium sampling training samples
[0138] calculate
[0139]
[0140] Solve using gradient descent algorithm
[0141] go through Update the target network weights after each step
[0142] End each event loop
[0143] End the event loop
[0144] After the above steps, the parameters of the trained neural network used to approximate the value function are finally obtained. Using these parameters, an optimized speed curve can be obtained by inputting the train's operating status and relevant track conditions during train operation.
[0145] As can be seen from the above, unlike previous DQN algorithms, the experience playback data area in the DQN algorithm of this invention stores data obtained by processing real historical train operation data, rather than experience data generated by the reinforcement learning environment. This data can be obtained from the train's onboard computer. This processing method, on the one hand, allows the invention to be independent of specific train dynamics models, avoiding the adverse effects of complex train operating environments on modeling and solving. On the other hand, learning from real historical data, calculating reward functions, and improving the learning model enhances the speed and quality of solving for the approximate optimal solution of the speed curve. The neural network trained using historical data can output the optimal action based on the current state, that is, output the optimal train operating condition based on the current train operating state. Through offline training and online optimization, the goals of energy saving, punctuality, and comfort are achieved.
[0146] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the embodiments of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0147] The above examples illustrate the present invention only to aid in understanding it and are not intended to limit the scope of the invention. Those skilled in the art can make various simple deductions, modifications, or substitutions based on the principles of this invention.
Claims
1. A novel heavy-haul train running speed curve intelligent optimization method, characterized in that, The method comprises the following steps: Step S1: constructing a train operation reinforcement learning environment based on line static data and train operation dynamic data; constructing a train operation reinforcement learning environment based on a heavy haul train dynamics model considering train length, line static data, and train operation and braking state dynamic data; Step S2: establishing a reward mechanism based on target optimization, with the goal of train operation smoothness, punctuality, and energy saving; Step S3: collecting historical train operation data, constructing a historical information database, and processing the data based on the reward mechanism; Step S4: learning through interaction between the agent and the reinforcement learning environment, training a neural network model using the historical data processed in step three, and optimizing the working condition action.
2. The novel heavy-haul train operation speed curve intelligent optimization method of claim 1, wherein, In step S1, the line static data includes line mileage, slope, and section speed limit value.
3. The novel heavy-haul train operation speed curve intelligent optimization method of claim 1, wherein, In step S1, the train operation dynamic data includes train current operation position, speed, operation time, brake pipe pressure, and locomotive brake cylinder pressure.
4. The novel heavy-haul train operation speed curve intelligent optimization method of claim 3, wherein, In step S1, the reinforcement learning environment is constructed based on a train dynamics model, which includes: In the formula: m is the train running position; V is the train running speed, m / s; Kt is the train rotation mass coefficient; For train unit tractive effort, N / KN; N / KN for the unit electric braking force of the train; For train unit air braking force, N / KN; Train unit basic running resistance, N / KN; Slope resistance for train unit, N / KN; t is time, s; In the formula: to describe the parameters related to the train air brake application process; t is time, s; Considering the train length, add the additional resistance of the line The calculation formula is: In the formula: L is the train length, m; , , unit additional resistance of the first hill, curve and tunnel covered by the train, N / KN; , , The first covered by the train The length of each ramp, curve, and tunnel, in meters.
5. The novel heavy-haul train operation speed curve intelligent optimization method of claim 1, wherein, In step S2, the target optimization includes improving heavy haul train smoothness, train operation punctuality rate, and reducing train operation energy consumption.
6. The novel heavy haul train operation speed curve intelligent optimization method of claim 5, wherein, In step S2, the reward function in the reward mechanism is represented as the sum of time reward, energy consumption reward, and comfort reward, and is represented as: In the formula: is a reward function; a reward function for punctuality; reward function for energy consumption; The reward function is stationary.
7. The novel heavy haul train operation speed curve intelligent optimization method of claim 6, wherein, The reward function in the reward mechanism is represented as the weighted sum of time reward, energy consumption reward, and comfort reward, and is represented as: wherein: , , are the respective weights of the reward functions related to punctuality, energy consumption and smoothness, respectively. 。 8. The novel heavy haul train operation speed curve intelligent optimization method of claim 1, wherein, In step S3, the historical train operation data includes train operation state information, and each train operation state is set as follows: In the formula: , , , , , , are respectively the position, speed, acceleration value, slope, speed limit value, train brake pipe pressure and locomotive brake cylinder pressure of the train in the current state.
9. The novel heavy haul train operation speed curve intelligent optimization method of claim 1, wherein, In step S4, the interaction learning between the agent and the reinforcement learning environment comprises the following steps: The agent acts as a vehicle-mounted controller and constitutes a closed-loop interactive system with the reinforcement learning environment; After receiving the working condition action output by the agent, the environment generates a new state, reward value, and state value function in real time and feeds them back to the agent; The agent evaluates and improves the strategy through the value function, selects the optimal working condition action based on the maximum action value function, and feeds the selected action back to the environment; The selected action is fed back to the environment, and the working condition value is updated through iteration to finally generate an optimized train speed curve.
10. The novel heavy haul train operation speed curve intelligent optimization method of claim 9, wherein, The interaction learning between the agent and the reinforcement learning environment is realized by using a deep Q learning network algorithm.