A train full-line running time allocation and speed curve hierarchical collaborative optimization method and system
By optimizing the train's speed curve and running time through a hierarchical collaborative architecture and a multi-objective pelican algorithm, the problem of high energy consumption across the entire train line was solved, achieving minimum energy consumption and punctual operation, thus improving the overall energy efficiency of the train.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TONGJI UNIV
- Filing Date
- 2026-05-22
- Publication Date
- 2026-07-24
AI Technical Summary
In existing technologies, the speed curve and running time allocation of trains lack coordinated optimization, resulting in high energy consumption across the entire line and making it difficult to achieve global optimization while ensuring punctuality and comfort.
A hierarchical collaborative architecture is adopted, and the improved Q-Learning algorithm and RMSprop algorithm are used to allocate the running time between stations. The multi-objective pelican algorithm (MOPOA) is used to solve the optimal speed trajectory, and the upper and lower layer iterative optimization models are constructed to realize the optimal running strategy of train time and speed between stations along the entire line.
It significantly reduced the total energy consumption of the entire line, improved the punctuality and comfort of train operation, and optimized the overall energy efficiency of the train.
Smart Images
Figure CN122232701B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of automatic driving technology for rail transit, and more specifically to a method and system for hierarchical collaborative optimization of train operation time allocation and speed curve. Background Technology
[0002] In the process of high-speed train automatic driving, optimizing the system's traction energy consumption is crucial. Different speed curves and operation control strategies affect the overall energy consumption of the train. For example, unreasonable speed planning and operation methods can lead to a significant increase in train traction energy consumption, raising operating costs. Therefore, optimizing the speed curve is an important way to improve train traction energy efficiency. However, existing optimization methods mostly optimize independently of a single section, lacking a coordinated design of the running time of multiple sections along the entire line and the underlying speed trajectory, making it difficult to achieve global energy consumption optimization.
[0003] Furthermore, since traction energy consumption is typically negatively correlated with interval travel time, and traction energy consumption is closely related to operating speed and operating condition switching strategies, current time allocation and speed curve optimization are independent and lack coordination. Existing research often employs the maximum principle or dynamic programming to seek the optimal driving strategy under ideal conditions. However, these methods, when dealing with related optimization problems, mostly treat energy consumption as the sole optimization objective, failing to comprehensively consider the coupling relationship between multiple objectives such as punctuality and comfort. Moreover, the optimization process is often based on preset fixed travel times, lacking linkage with inter-station time allocation, making it difficult to achieve global optimization across the entire line.
[0004] Therefore, in view of the shortcomings of existing technologies, how to provide a hierarchical collaborative optimization method and system for train operation time allocation and speed curve, solve the problem of independent time allocation and speed planning between stations, and significantly reduce the total energy consumption of the entire line while ensuring punctuality and comfort, which is applicable to the field of automatic driving of rail transit. Summary of the Invention
[0005] In view of this, the present invention provides a hierarchical collaborative optimization method and system for train running time allocation and speed curve of the whole line. It adopts a hierarchical collaborative architecture to deal with the problem of difficulty in coordinating the running time allocation between stations of the whole line and the speed curve planning of a single section, so as to minimize the total energy consumption.
[0006] To achieve the above objectives, the present invention adopts the following technical solution: a hierarchical collaborative optimization method for train full-line running time allocation and speed curve, comprising: Acquire train operation data, track constraints, and operational target parameters; A hierarchical collaborative architecture with upper and lower closed-loop iteration is constructed. Train operation data, line constraints and operation parameters are input into the hierarchical collaborative architecture. After upper and lower layer iterative optimization, the optimal time and speed operation strategy for trains between stations along the entire line is output. In the hierarchical collaborative architecture, the upper layer is used for inter-station running time allocation, and the lower layer is used for interval speed curve optimization.
[0007] Preferably, the lower layer performs force analysis on the train in operation, builds a dynamic model, and constructs a multi-objective optimization model based on the dynamic model; A multi-condition alternating mixed operation strategy is adopted to limit the train's condition switching rules; Based on the multi-objective optimization model and the train's operating condition switching rules, the multi-objective pelican algorithm is used to solve for the optimal speed trajectory at the lower level. Based on the optimal speed trajectory, the upper layer uses an improved Q-Learning algorithm to dynamically decide on the allocation scheme of running time between stations; The upper layer transmits the updated inter-station running time to the lower layer. The lower layer re-solves the optimal speed trajectory based on the time constraint and feeds back the energy consumption. The upper and lower layers iterate repeatedly until the total running time constraint is met and the energy consumption converges to the optimal level. After the iteration terminates, the upper layer outputs the optimal running time allocation results between all stations along the line, and combines them with the corresponding optimal speed trajectory at the lower layer to form the final optimal running strategy for the entire line in terms of time and speed.
[0008] Preferably, the multi-objective pelican algorithm is used to solve for the optimal velocity trajectory in the lower layer, including: Simulate a pelican flying towards its prey location and update the train's operational condition transition point sequence; The optimal velocity trajectory is obtained by simulating the behavior of a pelican spreading its wings to hunt on the water surface and performing a local neighborhood search.
[0009] Preferably, simulating a pelican flying towards its prey location and updating the train's operational condition transition point sequence includes: Each individual pelican in the population represents a complete velocity curve. Let's assume there are m pelicans in N-dimensional space. Indicates the first The position of a single pelican in N-dimensional space. In the multi-objective pelican algorithm, the population matrix of all pelicans. The mathematical expression is as follows: ; in, Indicates the first The Pelicans in The position in the dimension; During the initialization phase, the pelican population is generated randomly, and the position update formula is as follows: ; in, , , and These represent the lower and upper limits of the search interval, respectively. express Random numbers between; Based on the pelican's global exploration mechanism of locking onto prey, the individual's position is updated during the exploration phase by locking onto the prey's location. The update formula is as follows: ; in, This indicates the number of iterations at the current moment; Indicates the first phase The Pelicans in The position in the dimension; No. The Pelicans' new position; Indicates the prey is Position in 3D space; parameters It is a constant; The objective function value; For the first The fitness function value of a single pelican; Furthermore, if the objective function value improves at the Pelicans' new position, then accept the Pelicans' current position; otherwise, retain the original position, as shown in the following expression: ; in, Indicates the first The Pelicans' new state Indicates the first The Pelicans' last form, It is based on the objective function value of stage one.
[0010] Preferably, the optimal velocity trajectory is obtained by simulating the behavior of a pelican spreading its wings to hunt on the water surface and performing a local neighborhood search, including: Pelicans' hunting behavior is described as follows: ; in, This indicates the total number of iterations set. It is the number of iterations at the current moment. It is a constant, neighborhood radius This indicates the scope of a local search around an individual. The position update formula for the Pelican algorithm is as follows: ; in, Indicates the first The Pelicans' new state Its objective function value is based on stage two. After traversing all individuals and completing all iterations, the optimal velocity trajectory is output.
[0011] Preferably, based on the optimal speed trajectory, an improved Q-Learning algorithm is used to dynamically determine the time allocation scheme between stations, including: Input the current train status information and complete the algorithm initialization parameter settings; The exploration and development process of the ε-greedy and Boltzmann fusion strategy balancing algorithm is described. The learning rate is adaptively and dynamically adjusted using the RMSprop algorithm. The reward function is designed with the goal of minimizing traction energy consumption and optimizing on-time operation, and the running time between stations is dynamically and optimally allocated.
[0012] Preferably, the reward function is designed as follows: ; in, After the agent executes the current time allocation action between stations, the cumulative running time between stations has been completed; The optimal inter-station traction energy consumption is obtained by the lower-level MOPOA solution under a given time allocation. and These are the weighting coefficients for energy consumption cost and time reward, respectively; This represents the timeout penalty coefficient.
[0013] Preferably, the learning rate is adaptively and dynamically adjusted using the RMSprop algorithm, including: ; ; in, It is the moving average of the gradient. This is the current gradient. It is the initial learning rate, and It is a constant; The formula for updating the Q value is then expressed as: ; in, It is a time difference error. It is the moving average of the squared gradient.
[0014] Preferably, a hierarchical collaborative optimization system for train operation time allocation and speed curves across the entire line includes: The information acquisition module is used to acquire train operation data, line constraints, and operational target parameters; The hierarchical optimization module is used to construct a hierarchical collaborative architecture with upper and lower closed-loop iteration. Train operation data, line constraints, and operation parameters are input into the hierarchical collaborative architecture. After iterative optimization by the upper and lower layers, the optimal time and speed operation strategy for trains between stations along the entire line is output. In the hierarchical collaborative architecture, the upper layer is used for the allocation of inter-station running time, and the lower layer is used for the optimization of speed curves between sections.
[0015] As can be seen from the above technical solution, compared with the prior art, this invention discloses a hierarchical collaborative optimization method and system for train running time allocation and speed curve, adopting a hierarchical collaborative architecture: the upper layer dynamically decides the running time between stations through an improved Q-Learning algorithm, and introduces the RMSprop algorithm to adaptively adjust the learning rate to improve convergence stability; the lower layer, under given time and line constraints, constructs a multi-objective optimization model including energy consumption, comfort, and punctuality, and uses the Multi-Objective Pelican Algorithm (MOPOA) to solve for the optimal speed trajectory. This invention solves the problem of independent time allocation between stations and speed planning, and significantly reduces the total energy consumption of the entire line while ensuring punctuality and comfort. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0017] Figure 1 This is a schematic diagram of a hierarchical collaborative architecture provided in an embodiment of the present invention.
[0018] Figure 2 This is a schematic diagram of a train hybrid control strategy provided in an embodiment of the present invention.
[0019] Figure 3 The MOPOA visualization diagram provided for embodiments of the present invention.
[0020] Figure 4 The flowchart illustrates the action selection strategy of the Q-Learning algorithm provided in this embodiment of the invention.
[0021] Figure 5(a) shows the simulation results of the vehicle speed curve under the running time constraint provided by the embodiment of the present invention.
[0022] Figure 5(b) shows the simulation results of the vehicle energy consumption curve under the running time constraint provided by the embodiment of the present invention.
[0023] Figure 5(c) shows the simulation results of the vehicle speed and acceleration under the running time constraint provided by the embodiment of the present invention in MOPOA.
[0024] Figure 6(a) shows the simulation results of the optimization of the inter-station running time allocation provided by the embodiment of the present invention.
[0025] Figure 6(b) shows the simulation results of the overall energy consumption distribution optimization provided by the embodiment of the present invention. Detailed Implementation
[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0027] This invention discloses a hierarchical collaborative optimization method for train running time allocation and speed curves across the entire line, such as... Figure 1 As shown, it includes: Acquire train operation data, track constraints, and operational target parameters; A hierarchical collaborative architecture with upper and lower closed-loop iteration is constructed. Train operation data, line constraints and operation parameters are input into the hierarchical collaborative architecture. After upper and lower layer iterative optimization, the optimal time and speed operation strategy for trains between stations along the entire line is output. In the hierarchical collaborative architecture, the upper layer is used for inter-station running time allocation, and the lower layer is used for interval speed curve optimization. There is a bidirectional iterative and feedback optimization process between the two layers.
[0028] This invention employs a hierarchical collaborative architecture to address the challenge of coordinating the allocation of operating time between stations along the entire line with the planning of speed curves for single sections, thereby minimizing total energy consumption.
[0029] Specifically, the lower level performs force analysis on the train in operation, builds a dynamic model, and constructs a multi-objective optimization model based on the dynamic model; Based on the basic condition of the train, a multi-condition alternating mixed operation strategy is adopted to limit the train's condition switching rules; Based on the multi-objective optimization model and the train's operating condition switching rules, the multi-objective pelican algorithm is used to solve for the optimal speed trajectory at the lower level. Based on the optimal speed trajectory, the upper layer uses an improved Q-Learning algorithm to dynamically decide on the allocation scheme of running time between stations; The upper layer transmits the updated inter-station running time to the lower layer. The lower layer re-solves the optimal speed trajectory based on the time constraint and feeds back the energy consumption. The upper and lower layers iterate repeatedly until the total running time constraint is met and the energy consumption converges to the optimal level. After the iteration terminates, the upper layer outputs the optimal running time allocation results between all stations along the line, and combines them with the corresponding optimal speed trajectory at the lower layer to form the final optimal running strategy for the entire line in terms of time and speed.
[0030] This invention employs a multi-objective pelican algorithm to optimize the speed curve of automatic train operation, aiming to reduce traction energy consumption and improve passenger comfort, thereby obtaining the optimal speed curve for the train.
[0031] Specifically, based on the basic condition of the train, a mixed operation strategy will be adopted: For the three basic operating states of high-speed trains—traction, coasting, and braking—a multi-condition alternating mixed operation strategy is adopted while adhering to strict operating condition switching constraints. For example... Figure 2 As shown, by rationally planning the timing of switching between traction and coasting modes, while meeting the time constraints of inter-station operation, the train's kinetic energy is maximized for coasting to reduce traction energy consumption and improve operational stability.
[0032] Specifically, the multi-objective pelican algorithm is used to solve for the optimal velocity trajectory in the lower layer, including: Simulate a pelican flying towards its prey location and update the train's operational condition transition point sequence; The optimal velocity trajectory is obtained by simulating the behavior of a pelican spreading its wings to hunt on the water surface and performing a local neighborhood search.
[0033] Specifically, simulating a pelican flying towards its prey location and updating the train's operational condition transition point sequence includes: Each individual pelican in the population represents a complete velocity curve. Let's assume there are m pelicans in N-dimensional space. Indicates the first The position of a single pelican in N-dimensional space. In the multi-objective pelican algorithm, the population matrix of all pelicans. The mathematical expression is as follows: ; in, Indicates the first Only the Pelicans in The position in the dimension; During the initialization phase, the pelican population is generated randomly, and the position update formula is as follows: ; in, , , and These represent the lower and upper limits of the search interval, respectively. express Random numbers between; In solving the multi-objective speed curve of a train, purely random exploration is prone to slow convergence in the nonlinear solution space. MOPOA, based on the global exploration mechanism of a pelican locking onto its prey, updates the individual position by locking onto the prey's location during the exploration phase. The update formula is as follows: ; in, This indicates the number of iterations at the current moment; Indicates the first phase Only the Pelicans in The position in the dimension; No. The Pelicans' new position; Indicates the prey is Position in 3D space; parameters It is a constant that can be randomly equal to 1 or 2, and is randomly selected in each iteration and in each individual. When the parameter value is 2, the individual can be given more displacements to accurately scan the search space. The objective function value; For the first The fitness function value of a single pelican; Furthermore, if the objective function value improves at the Pelicans' new position, then accept the Pelicans' current position; otherwise, retain the original position, as shown in the following expression: ; in, Indicates the first The Pelicans' new state Indicates the first The Pelicans' last form, It is based on the objective function value of stage one.
[0034] This update method is called efficient update, and its purpose is to avoid the algorithm exploring non-optimal solution regions, thereby ensuring that the search space is accurately scanned.
[0035] Specifically, by simulating the behavior of pelicans spreading their wings to hunt on the water's surface, a local neighborhood search is performed to obtain the optimal velocity trajectory, including: Step 3.2 Simulate a pelican launching an attack on the water's surface, and fine-tune its speed and trajectory: MOPOA simulates the behavior of pelicans swooping down to hunt on the water's surface for local neighborhood search. To ensure stable convergence in the later stages of iteration, the fine-tuning step size of the control strategy is set to dynamically decrease with each iteration. The behavior of pelicans during hunting is described as follows: ; in, This indicates the total number of iterations set. It is the number of iterations at the current moment. It is a constant of 0.2, and the neighborhood radius is... This coefficient represents the range of the local search around an individual; it aims to guide the algorithm towards a high-quality solution. In the initial iterations, this coefficient is relatively large, covering a wide area near each individual. As the number of iterations increases, this coefficient gradually decreases, thus shortening the neighborhood radius of each individual. This mechanism ensures that the algorithm can explore the nearest neighbor of each individual at a more precise pace, ultimately increasing the likelihood that the MOPOA algorithm will approach the global optimum in the solution space.
[0036] At this point, the position update formula of the Pelican algorithm can be shown as follows: ; in, Indicates the first The Pelicans' new state Its objective function value is based on stage two. After traversing all individuals and completing all iterations, the optimal velocity trajectory is output.
[0037] Specifically, based on the optimal speed trajectory, an improved Q-Learning algorithm is used to dynamically determine the time allocation scheme between stations, including: Input the current train status information and complete the algorithm initialization parameter settings; The exploration and development process of the ε-greedy and Boltzmann fusion strategy balancing algorithm is described. The learning rate is adaptively and dynamically adjusted using the RMSprop algorithm. The reward function is designed with the goal of minimizing traction energy consumption and optimizing on-time operation, and the running time between stations is dynamically and optimally allocated.
[0038] Specifically, the reward function is designed as follows: ; in, After the agent executes the current time allocation action between stations, the cumulative running time between stations has been completed; The optimal inter-station traction energy consumption is obtained by the lower-level MOPOA solution under a given time allocation. and These are the weighting coefficients for energy consumption cost and time reward, respectively; This represents the timeout penalty coefficient.
[0039] Specifically, the RMSprop algorithm is used to adaptively and dynamically adjust the learning rate, including: ; ; in, It is the moving average of the gradient. This is the current gradient. It is the initial learning rate, and To prevent constants with a denominator of 0; To apply RMSprop in Q-Learning, the formula for updating the Q value is expressed as: ; in, It is a time difference error. It is the moving average of the squared gradient.
[0040] In one specific embodiment of the present invention, a hierarchical collaborative optimization system for train full-line running time allocation and speed curve includes: The information acquisition module is used to acquire train operation data, line constraints, and operational target parameters; The hierarchical optimization module is used to construct a hierarchical collaborative architecture with upper and lower closed-loop iteration. Train operation data, line constraints, and operation parameters are input into the hierarchical collaborative architecture. After iterative optimization by the upper and lower layers, the optimal time and speed operation strategy for trains between stations along the entire line is output. In the hierarchical collaborative architecture, the upper layer is used for the allocation of inter-station running time, and the lower layer is used for the optimization of speed curves between sections.
[0041] In a specific embodiment of the present invention, this embodiment considers the problem of optimizing the energy-saving operation trajectory of trains under complex track conditions. The control variable is the traction force of the train's underlying structure; the constraints include multi-level speed limits and road gradients; the control objective is to reduce the overall traction energy consumption of the train while also ensuring passenger comfort. The control problem involves the independent allocation of time between stations and speed planning, aiming to significantly reduce the total energy consumption of the entire line while ensuring punctuality and comfort. The following will use the above embodiment as an example to describe the specific implementation of the method of the present invention: Step 1: Build a system dynamics model and construct a multi-objective optimization model for the system. To simplify the study, the train is approximated as a rigid point in a moving train force analysis, and its dynamic model is constructed as follows: (1) (2) (3) in, and These represent traction force and braking force, respectively. As the basic resistance, and , , This is the drag coefficient, which is related to the train model; To add resistance, , , These are the train's operating time, location, and speed.
[0042] To achieve optimal overall performance of high-speed trains during operation, this embodiment constructs a multi-objective optimization model that comprehensively considers precise stopping, punctual operation, passenger comfort, and traction energy consumption, including the train stopping point position error. Total running time Comfort of riding and traction energy consumption The specific mathematical definitions of the individual performance indicators are as follows: (4) (5) (6) (7) in, , These are the actual and target parking locations, respectively. Position in operation This represents the efficiency of regenerative braking energy.
[0043] To eliminate dimensional differences, the rate of change was used to normalize each indicator: (8) (9) (10) (11) Therefore, based on the train's kinematic equations, track constraints, and operational optimization objectives, a multi-objective optimization model can be obtained as shown in equation (12).
[0044] (12)
[0045] in, It indicates the traction or braking rate of the train, enabling the train to be in one of four states: maximum traction, cruising, coasting, and braking. and These represent the characteristic curves of the train's maximum traction force and maximum braking force, respectively. Speed limits were imposed on the line. and These represent the initial velocity and the terminal velocity, respectively. and These represent the initial position and the final position, respectively.
[0046] Step 2: Based on the basic condition of the train, adopt a mixed operation strategy: The operating state of a high-speed train is determined by the resultant forces acting on it, primarily encompassing three basic states: traction, coasting, and braking. To ensure train safety and passenger comfort, and to avoid severe longitudinal impacts, the switching of train operating conditions must adhere to strict state transition constraints. Specifically, direct switching between traction and braking conditions is strictly prohibited; a coasting condition with gentle acceleration must serve as the transition. To balance on-time performance and optimal energy efficiency, this embodiment of the invention employs a hybrid operation strategy involving alternating multiple operating conditions, such as... Figure 2 As shown in the figure. This strategy maximizes the use of train kinetic energy for coasting by rationally planning the switching timing between traction and coasting modes, while meeting the constraints of inter-station running time, thereby effectively reducing overall traction energy consumption and improving operational stability.
[0047] Step 3: Use the Multi-Objective Pelican Algorithm (MOPOA) to solve for the optimal velocity trajectory in the lower layer: When solving for the multi-objective speed curve of high-speed trains, complex nonlinear resistance and multiple speed limit constraints can easily cause traditional algorithms to get stuck in local optima. The Pelican Optimization Algorithm (POA) simulates the hunting behavior of pelicans in nature, which involves moving towards prey (global exploration) and swooping down on the water (local exploitation). Extending this to the Multi-Objective Pelican Optimization Algorithm (MOPOA), the algorithm combines a broad solution space search breadth with accurate local convergence depth, effectively handling the multi-objective coupling relationship between energy consumption, timeliness, and comfort, and providing high-quality Pareto optimal fronts for trains.
[0048] Step 3.1 Simulate the pelican flying towards its prey location and update the train's operational condition transition point sequence: When applying MOPOA to solve for the optimal speed trajectory of the train in the lower level, each individual pelican in the population represents a complete speed curve.
[0049] Suppose there are m pelicans in N-dimensional space. Indicates the first The position of a single pelican in N-dimensional space. In the pelican optimization algorithm, the population matrix of all pelicans... The mathematical expression is as follows: (13) in, , Indicates the first Only the Pelicans in The position within the dimension. During the initialization phase, the pelican population is generated randomly, and the position is updated as shown in equation (14). (14) in , , and These represent the lower and upper limits of the search interval, respectively. express A random number between [a certain number of points].
[0050] When solving for the multi-objective velocity curve of a train, purely random exploration struggles to converge quickly in the vast nonlinear solution space. MOPOA borrows a global exploration mechanism similar to a pelican locking onto its prey. During the pelican's exploration phase, it locks onto the prey's location in the air and then swoops down towards it. The updated position is as follows: (15) in, This indicates the number of iterations at the current moment; Indicates the first phase Only the Pelicans in The position in the dimension; No. The Pelicans' new position; Indicates the prey is Position in 3D space; parameters It is a constant that can be randomly equal to 1 or 2, and is randomly selected in each iteration and in each individual. When the parameter value is 2, the individual can be given more displacements to accurately scan the search space. The objective function value; For the first The fitness function value of a single pelican.
[0051] If the objective function value of the new position of the pelican is improved, the new position is accepted; otherwise, the original position is retained. This "efficient update" method aims to avoid the algorithm exploring non-optimal regions and ensure accurate scanning of the search space. A detailed description follows: (16) in, Indicates the first The Pelicans' new state Indicates the first The Pelicans' last form, It is based on the objective function value of stage one.
[0052] Step 3.2 Simulate a pelican launching an attack on the water's surface, and fine-tune its speed and trajectory: MOPOA simulates pelican predation behavior by performing a local neighborhood search. To ensure stable convergence of the algorithm in the later stages of iteration, the control strategy dynamically decays the fine-tuning step size with the number of iterations. Its mathematical description is as follows: (17) in, This indicates the total number of iterations set. It is the number of iterations at the current moment. It is a constant of 0.2, and the neighborhood radius is... The range of local search around an individual is defined to guide the algorithm toward a high-quality solution. In the initial iteration phase of the algorithm, this coefficient is relatively large, covering a wide area near each individual. As the number of iterations increases, this coefficient gradually decreases, thus shortening the neighborhood radius of each individual. This mechanism ensures that the algorithm can explore the neighborhood of each individual at a more precise pace, ultimately increasing the likelihood that the MOPOA algorithm will approach the global optimum in the solution space. At this point, equation (18) can be used to illustrate the position update problem of the Pelican algorithm.
[0053] (18)
[0054] in, Indicates the first The Pelicans' new state It is based on the objective function value of stage two.
[0055] In summary, the solution process for the lower-level velocity curve, combining the physical constraints of high-speed trains and the MOPOA optimization mechanism, is as follows: Figure 3 As shown.
[0056] Step 4: Perform dynamic time allocation between upper-level stations using the Q-Learning algorithm: The core of the improved Q-Learning algorithm is reflected in three aspects: First, a heuristic Q-table initialization method based on the remaining train time is proposed to improve the initial training efficiency; second, a hybrid action selection strategy of ε-greedy and Boltzmann is adopted to solve the exploration-exploitation dilemma of the algorithm, enhance the exploration ability in the initial stage, and ensure the convergence accuracy in the later stage; finally, the RMSprop algorithm is introduced to dynamically adjust the learning rate, thereby improving the convergence speed and operational stability of the algorithm.
[0057] The algorithm determines the optimal action that the intelligent agent should take in different states by querying the maximum Q value of each row in the Q table. Indicates that the agent is in a specific state. Take action The estimated reward value obtained at that time. The Q-value update formula is as follows: (19) in, This represents the state of the intelligent agent at time t, determined by the current station. Discretization interval with consumed time Together constitute; Indicates that the agent is in state The following actions were taken; This represents the current Q value; Indicates that the agent is in state Take action The reward received at that time; Indicates the discount factor; Indicates a new state and action Below value; Indicates that the agent is taking action And reach the state The maximum potential future reward that can be obtained at that time; This represents the learning rate, and its value ranges from 1 to 2. .
[0058] Step 4.1 Input the current status information and initialization parameters: To accelerate algorithm convergence and guide the agent to quickly discover feasible time allocation schemes, this application proposes a heuristic Q-table initialization method based on the remaining train time. In train running time allocation decisions, the remaining available time of the train is the core prior knowledge for evaluating the merits of the current state.
[0059] Let the current station of the train be . The time interval in which the time consumed is discretized is The total required running time for the entire line is The theoretical remaining time in this state is defined as... For any valid time allocation action in the current state. The corresponding Q table initial value Set as an heuristic expectation that follows exponential decay: (20) in, This represents the remaining time decay coefficient. When the train faces a small time margin in a certain state, it is assigned a higher exploration weight in its initial Q value to guide the agent to rationally plan the consumption of time resources in the early exploration.
[0060] Step 4.2 Introduction - Exploration and development of a fusion strategy combining greedy and Boltzmann strategies: In reinforcement learning, a single exploration strategy struggles to balance the breadth of the initial global search with the accuracy of local convergence in the later stages. Therefore, this invention introduces a fusion approach. - Greedy and Boltzmann mechanism action selection strategy; in the early stages of algorithm training, the agent lacks sufficient awareness of the time allocation environment, and random action selection can easily lead to state repetition. A detailed description of this strategy can be found in [link to relevant documentation]. Figure 4 As shown.
[0061] To achieve the synergistic optimization of minimizing train energy consumption and punctuality across the entire line, the reward function is designed as shown in equation (24): (twenty one) in, After the agent executes the current time allocation action between stations, the cumulative running time between stations has been completed; The optimal inter-station traction energy consumption is obtained by the lower-level MOPOA solution under a given time allocation. and These are the weighting coefficients for energy consumption cost and time reward, respectively; This represents the timeout penalty coefficient.
[0062] Step 4.3: Dynamically adjust the learning rate using the RMSprop algorithm: Traditional Q-learning algorithms typically use a fixed learning rate, which is difficult to adapt to the dynamic environment of high-speed train time allocation. Therefore, this embodiment of the invention introduces RMSprop to achieve adaptive adjustment of the learning rate, and its mathematical update rule is as follows: (twenty two) (twenty three) in, It is the moving average of the gradient. This is the current gradient. It is the initial learning rate, and To prevent constants with a denominator of 0, the update formula for Q-values is expressed as follows: (This is to be repeated in the original text.) (twenty four) in, It is a time difference error. It is the moving average of the squared gradient.
[0063] The simulation results are as follows: The above embodiments use the CRH3-350 high-speed train as the research object, setting the line length to 87km. The simulation environment uses Matlab to solve the constructed multi-objective train model. Figures 5(a)-5(c) show the simulation results of the train speed curves obtained by the MOPOA algorithm under specific running time constraints in this embodiment of the invention. Figures 6(a)-6(b) show the simulation results of the optimization of the running time allocation between stations along the entire line. Through the joint simulation analysis of Figures 5(a)-5(c) and 6(a)-6(b), it can be seen that the hierarchical collaborative optimization framework described in this embodiment of the invention can effectively overcome the limitations of independent optimization between sections, minimizing global traction energy consumption while ensuring punctual operation throughout the line. Compared with traditional methods that consider the time allocation between stations along the entire line, this framework saves 4.26% of traction energy consumption, verifying the effectiveness of the proposed framework.
[0064] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.
[0065] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined in the embodiments of the invention may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown in the present invention, but is to be accorded the widest scope consistent with the principles and novel features disclosed in the embodiments of the invention.
Claims
1. A hierarchical collaborative optimization method for train operation time allocation and speed curve, characterized in that, include: Acquire train operation data, track constraints, and operational target parameters; A hierarchical collaborative architecture with upper and lower closed-loop iteration is constructed. Train operation data, line constraints and operation parameters are input into the hierarchical collaborative architecture. After upper and lower layer iterative optimization, the optimal time and speed operation strategy for trains between stations along the entire line is output. In the hierarchical collaborative architecture, the upper layer is used for inter-station running time allocation, and the lower layer is used for interval speed curve optimization. The lower level performs force analysis on the train in operation, builds a dynamic model, and constructs a multi-objective optimization model based on the dynamic model; A multi-condition alternating mixed operation strategy is adopted to limit the train's condition switching rules; Based on the multi-objective optimization model and the train's operating condition switching rules, the multi-objective pelican algorithm is used to solve for the optimal speed trajectory at the lower level. Based on the optimal speed trajectory, the upper layer uses an improved Q-Learning algorithm to dynamically decide on the allocation scheme of running time between stations; The upper layer transmits the updated inter-station running time to the lower layer. The lower layer re-solves the optimal speed trajectory based on the time constraint and feeds back the energy consumption. The upper and lower layers iterate repeatedly until the total running time constraint is met and the energy consumption converges to the optimal level. After the iteration terminates, the upper layer outputs the optimal running time allocation results between all stations along the line, and combines them with the corresponding optimal speed trajectory at the lower layer to form the final optimal running strategy for the entire line in terms of time and speed.
2. The method for hierarchical collaborative optimization of train running time allocation and speed curve according to claim 1, characterized in that, The multi-objective pelican algorithm is used to solve for the optimal velocity trajectory in the lower layer, including: Simulate a pelican flying towards its prey location and update the train's operational condition transition point sequence; The optimal velocity trajectory is obtained by simulating the behavior of a pelican spreading its wings to hunt on the water surface and performing a local neighborhood search.
3. The method for hierarchical collaborative optimization of train running time allocation and speed curve according to claim 2, characterized in that, Simulate a pelican flying towards its prey location and update the train's operational condition transition point sequence, including: Each individual pelican in the population represents a complete velocity curve. Let's assume there are m pelicans in N-dimensional space. Indicates the first The position of a single pelican in N-dimensional space; in the multi-objective pelican algorithm, the population matrix of all pelicans. The mathematical expression is as follows: ； in, Indicates the first Only the Pelicans in The position in the dimension; During the initialization phase, the pelican population is generated randomly, and the position update formula is as follows: ； in, , , and These represent the lower and upper limits of the search interval, respectively. express Random numbers between; Based on the pelican's global exploration mechanism of locking onto prey, the individual's position is updated during the exploration phase by locking onto the prey's location. The update formula is as follows: ； in, This indicates the number of iterations at the current moment; Indicates the first phase Only the Pelicans in The position in the dimension; No. The Pelicans' new position; Indicates the prey is Position in 3D space; parameters It is a constant; The objective function value; For the first The fitness function value of a single pelican; Furthermore, if the objective function value improves at the Pelicans' new position, then accept the Pelicans' current position; otherwise, retain the original position, as shown in the following expression: ； in, Indicates the first The Pelicans' new state Indicates the first The Pelicans' last form, It is based on the objective function value of stage one.
4. The method for hierarchical collaborative optimization of train running time allocation and speed curve according to claim 3, characterized in that, The optimal velocity trajectory was obtained by simulating the behavior of a pelican catching prey on the water's surface through a local neighborhood search, including: Pelicans' hunting behavior is described as follows: ； in, This indicates the total number of iterations set. It is the number of iterations at the current moment. It is a constant, neighborhood radius This indicates the scope of a local search around an individual. The position update formula for the Pelican algorithm is as follows: ； in, Indicates the first The Pelicans' new state Its objective function value is based on stage two. After traversing all individuals and completing all iterations, the optimal velocity trajectory is output.
5. The method for hierarchical collaborative optimization of train running time allocation and speed curve according to claim 1, characterized in that, Based on the optimal speed trajectory, an improved Q-Learning algorithm is used to dynamically determine the time allocation scheme between stations, including: Input the current train status information and complete the algorithm initialization parameter settings; let the current station of the train be... The time interval in which the time consumed is discretized is The total required running time for the entire line is ; The theoretical remaining time in this state is defined as follows: ; For any valid time allocation action in the current state The corresponding Q-table initial value Set as an heuristic expectation that follows exponential decay: ； in, The remaining time decay coefficient; The exploration and development process of the ε-greedy and Boltzmann fusion strategy balancing algorithm is described. The learning rate is adaptively and dynamically adjusted using the RMSprop algorithm. The reward function is designed with the goal of minimizing traction energy consumption and optimizing on-time operation, and the running time between stations is dynamically and optimally allocated.
6. The method for hierarchical collaborative optimization of train running time allocation and speed curve according to claim 5, characterized in that, The reward function is designed as follows: ； in, After the agent executes the current time allocation action between stations, the cumulative running time between stations has been completed; The optimal inter-station traction energy consumption is obtained by the lower-level MOPOA solution under a given time allocation. and These are the weighting coefficients for energy consumption cost and time reward, respectively; This represents the timeout penalty coefficient.
7. The method for hierarchical collaborative optimization of train running time allocation and speed curve according to claim 5, characterized in that, The RMSprop algorithm is used to adaptively and dynamically adjust the learning rate, including: ； ； in, It is the moving average of the gradient. This is the current gradient. It is the initial learning rate, and It is a constant; To apply RMSprop in Q-Learning, the formula for updating the Q value is expressed as: ； in, It is a time difference error. It is the moving average of the squared gradient.
8. A hierarchical collaborative optimization system for train operation time allocation and speed curves across the entire line, employing the hierarchical collaborative optimization method for train operation time allocation and speed curves across the entire line as described in any one of claims 1-7, characterized in that, include: The information acquisition module is used to acquire train operation data, line constraints, and operational target parameters; The hierarchical optimization module is used to construct a hierarchical collaborative architecture with upper and lower closed-loop iteration. Train operation data, line constraints, and operation parameters are input into the hierarchical collaborative architecture. After iterative optimization by the upper and lower layers, the optimal time and speed operation strategy for trains between stations along the entire line is output. In the hierarchical collaborative architecture, the upper layer is used for the allocation of inter-station running time, and the lower layer is used for the optimization of speed curves between sections.