A train operation trajectory optimization method

Abstract

This invention belongs to the field of energy-saving control technology for urban rail transit, specifically involving a train trajectory optimization method. Based on the force relationship of the train, a dynamic equation is constructed, and the line is discretized into unequal-length spaces according to gradient change points and speed limit inflection points, ensuring constant gradient and speed limit in each section. Train control is modeled as a Markov decision process, defining a state vector, action space, and multi-objective reward function to obtain future rewards. Rewards, states, and actions are combined into independent decision blocks. Within each block, fully connected attention is used to strengthen causal relationships, and between blocks, a multi-k-value summation filtering strategy is used to retain key temporal information. A gradient and speed limit bias matrix is ​​introduced, and attention weights are dynamically adjusted to explicitly inject domain constraints. A DK-BSDT model integrating the above mechanisms is trained to generate a safe, energy-saving, and punctual optimal operating trajectory with the goal of minimizing trajectory generation error and reward deviation.

Description

Technical Field

[0001] This invention belongs to the field of energy-saving control technology for urban rail transit, and specifically relates to a method for optimizing train operation trajectory. Background Technology

[0002] With the rapid development of urban rail transit, the energy consumption of train operation has become an increasingly prominent issue. According to statistics from the China Rail Transit Association, by the end of 2024, the total operating mileage of urban rail transit in China had exceeded 10,000 kilometers, with an annual passenger volume of nearly 30 billion trips. However, traction energy consumption accounts for more than 50% of the total system energy consumption, becoming a key factor affecting energy conservation and emission reduction in subway systems. Taking Guangzhou Metro as an example, its traction energy consumption in 2022 exceeded 1×10⁻⁶ kilometers. 9 The potential for energy conservation in train operation is enormous. Therefore, researching efficient and intelligent energy-saving control methods for trains is of great significance for achieving the goals of "carbon peaking" and "carbon neutrality".

[0003] The core objective of energy-saving train operation control is to minimize energy consumption throughout the entire journey by rationally allocating speed changes during traction, coasting, cruising, and braking phases, while meeting operational safety and punctuality requirements. This problem is known as Train Trajectory Optimization (TTO), which is essentially a dynamic optimization problem with complex constraints, requiring comprehensive consideration of factors such as track gradient, speed limits, resistance characteristics, and timetables. Early research often employed analytical methods, such as Pontryagin's maximum principle (PMP), modeling train control as an optimal control problem and deriving analytical optimal solutions on straight tracks using differential equations. Results showed that the optimal control sequence under ideal conditions typically includes four phases: maximum traction, cruising, coasting, and maximum braking. However, the PMP method is difficult to directly apply to real-world track environments with complex gradients and multiple speed limits because solving for boundary conditions and state transition points requires extensive manual derivation and assumptions, resulting in computational complexity and limited scalability.

[0004] With advancements in computing technology, researchers have transformed train operation control into a numerical optimization problem, employing methods such as nonlinear programming (NLP), pseudospectral methods, sequential quadratic programming (SQP), and intelligent optimization algorithms (e.g., particle swarm optimization, genetic algorithms). While these methods can handle complex constraints, they suffer from long solution times and poor real-time performance in high-dimensional state spaces, and are prone to getting trapped in local optima. Furthermore, the models are sensitive to initial values, requiring re-solution when track conditions or operating times change, making it difficult to meet the online control requirements of subway systems.

[0005] In recent years, reinforcement learning (RL) has been introduced into the field of train trajectory optimization. RL learns the optimal policy through an agent-environment interaction process, maximizing long-term rewards and possessing self-learning and adaptive characteristics. Value function-based DQN algorithms, policy gradient-based PPO algorithms, and algorithms combining the advantages of both (A3C and DDPG) have all been applied in train control. For example, Zhou et al. used DDPG to achieve optimal energy consumption control for high-speed trains; Cheng et al. proposed the S-PPO algorithm based on safety reinforcement learning to avoid out-of-bounds problems caused by improper penalty function settings. However, traditional RL algorithms still have significant shortcomings in train trajectory optimization: First, train operation requires a long time to rely on the decision-making process, and the reward signal for each step is sparse, with effective feedback only obtained when the destination is reached, resulting in low sampling efficiency and unstable learning; Second, RL relies on discount factors to decay future rewards, which is prone to "short-sightedness" problems, prioritizing immediate gains while ignoring the long-term energy consumption optimization goal; In addition, to meet safety constraints, traditional RL often adopts "soft constraints" by adding a penalty term to the loss function, but the constraints on the sampling process are weak, which may generate a large number of unsafe samples and reduce learning efficiency.

[0006] The Decision Transformer (DT) approach, which has emerged in recent years, offers a novel approach to reinforcement learning. Based on the Transformer architecture, this method treats states, actions, and future rewards as temporal sequences input to the model, directly modeling the global dependencies of the trajectory through a self-attention mechanism. Compared to the iterative interactive learning of traditional reinforcement learning, DT employs a "sequence modeling + trajectory playback" approach, generating policies without online exploration. This effectively alleviates the problems of sparse rewards and shortsightedness, improving sample utilization and policy stability. However, DT has two limitations in trajectory optimization tasks: First, the density of the attention mechanism. While the standard Transformer calculates attention weights pairwise for all temporal elements, possessing global modeling capabilities, the number of states and actions in trajectory data far exceeds the number of key reward signals, leading to diluted reward attention and difficulty in accurately capturing decision factors affecting global energy consumption. Second, the lack of domain knowledge. Train operation is strictly constrained by track gradient and speed limits, while DT relies entirely on data-driven implicit learning, failing to explicitly introduce these physical constraints, potentially generating trajectories that violate speed limits and safety boundaries.

[0007] To address the aforementioned issues, the academic community has proposed improvements such as Sparse Attention Transformer and Dynamic Sparse Attention, which reduce redundant interference and improve computational efficiency and feature focusing capabilities by selecting important attention terms. However, existing sparse mechanisms are based on a uniform ranking threshold and do not distinguish between feature types such as rewards, states, and actions. This can easily lead to the misclassification of key reward signals to zero during sparse selection, weakening the model's long-term reward orientation. Meanwhile, while domain knowledge fusion methods can strengthen physical constraints, they lack synergistic optimization with sparse attention mechanisms, making it difficult to achieve a balance between safety and energy consumption under complex track conditions. In other words, existing methods in train energy-saving operation control suffer from weakened key reward information due to an imbalance in the attention mechanism and insufficient adaptation to domain constraints. Summary of the Invention

[0008] To address the problems existing in the prior art, the purpose of this invention is to provide a train trajectory optimization method that achieves synergistic improvement in train trajectory in terms of safety, energy saving, and punctuality, providing an efficient and reliable solution for the intelligent control of urban rail transit.

[0009] The technical solution of this invention is: A method for optimizing train trajectory includes the following steps: Obtain the train operation parameter set, input the train operation parameter set into the DK-BSDT model, and obtain the train's energy-optimal operating trajectory; The construction of the DK-BSDT model includes the following steps: Based on the set of train operating parameters, construct dynamic equations; Based on the quantification of the dynamic relationship of various parameters in the train operation parameter set by the dynamic equation, the line for continuous train operation is discretized into several sections based on the gradient change point and the speed limit inflection point, and the speed limit value and gradient value of each section are obtained. Based on the discretization of unequal-length space, train control is modeled as a Markov decision process, thereby defining the train's state vector and action space. A reward function is established based on the train running time deviation and energy consumption level within each section, and the future reward for each time step is obtained based on the reward function. The future reward, state vector, and action space of each time step within each segment are combined into an independent decision block. Within a single decision block, a fully connected attention calculation is performed between the future reward, state vector, and action space. Between different decision blocks, the discretized gradient value and speed limit value are transformed into an attention bias matrix, which is used as an explicit injection of the train operation physical constraints to determine the attention weights. Based on the fully connected attention calculation and determined attention weights, and combined with the future reward, state vector, action space, independent decision blocks, and attention bias matrix, the DK-BSDT model is trained. When the loss function of the DK-BSDT model reaches the preset requirements, the parameters of the DK-BSDT model are fixed.

[0010] Preferably, the dynamic equation is determined according to the following formula: , in, , In the formula: t Runtime; x For the running location; v For running speed; For traction force, ; For braking force, ; For the total operating resistance, ; This serves as the basic resistance for operation; Add resistance to the line; M For the quality of the train.

[0011] Preferably, the state vector defined by the Markov decision process includes train position, speed, time, and energy consumption characteristics, and the action space includes the train's maximum traction, cruising, coasting, and maximum braking; the reward function is obtained based on the train's running time deviation and energy consumption level within each segment.

[0012] Preferably, after the line is spatially discretized into unequal lengths, the running time and energy consumption level of each segment are determined according to the following formula: , In the formula, This refers to the train's travel time. For the train's energy consumption; i= 1, 2, ... N , N The total number of sections is defined; x The train's position; For traction force, ; For the train from the i Starting point of each section To the finish line .

[0013] Preferably, the reward function is determined according to the following formula: , in, , , In the formula, Rewards for train running time deviations; Incentives for train operation energy consumption; For the planned running time, E For train energy consumption; L This represents the line length.

[0014] Preferably, the decision block performs fully connected attention calculations among the reward, state, and action, determined according to the following formula: , In the formula, , , They are respectively The projection matrix; , , They are respectively The mapped query vector, key vector, and value vector; , r For future returns, S For state, a For action.

[0015] Preferably, the attention weights employ multiple... k The summation and filtering strategy is determined according to the following formula: , In the formula, Inter-block attention weights; For inter-block attention scoring; for The selection operator, The number of most important key-value pairs retained for each query vector, used to dynamically control sparsity; ,in k Hyperparameters for controlling sparsity.

[0016] Preferably, the attention bias matrix includes a gradient bias and a speed limit bias, wherein the gradient bias is dynamically adjusted based on the absolute value of the gradient to strengthen the attention weight of steep gradient sections, and the speed limit bias is dynamically adjusted based on the difference between the current train speed and the speed limit.

[0017] Preferably, the slope offset is determined according to the following formula: , in, , In the formula, For slope offset; The slope attenuation is sparse; and This refers to the index of the decision block; This represents the index of the target block of the query vector that is currently being computed. Index representing all source blocks ,|t-t'| It is the relative distance between blocks.

[0018] Preferably, the speed limiting bias is determined according to the following formula: , in, , , In the formula, For speed limiting offset; This refers to the relative relationship between train speed and speed limit. For train speed, Speed ​​limits for trains.

[0019] Compared with the prior art, the train trajectory optimization method of the present invention has the following beneficial effects: 1. This invention introduces a "sparse attention mechanism with block rules" to combine the future reward, state vector, and action space of each time step into an indivisible decision block. This design forces a strong correlation among the three within the decision block, ensuring that long-term reward signals can be effectively perceived and utilized in each decision step. This overcomes the dilemma of the model getting stuck in local optima due to excessive focus on the immediate state. This allows the model to quickly lock in effective policies in the early stages of training. Experiments show that its convergence speed can be improved by more than 70% compared with traditional methods, and the training process is more stable.

[0020] 2. This invention achieves explicit modeling and precise adherence to track physical constraints, enhancing the safety and physical rationality of the trajectory. It innovatively incorporates a domain knowledge bias matrix into the Transformer attention calculation, directly encoding the two key physical constraints—track gradient and speed limit—into the model's learning mechanism. This mechanism dynamically guides the model's attention to focus on key sections such as gradient inflection points and speed limit change zones, enabling the generated trajectory to proactively and accurately adapt to track conditions. This not only effectively prevents safety hazards such as speeding but also, because the model deeply understands the impact of gradient on train inertia and energy consumption, generates a smoother trajectory that better conforms to physical laws, avoiding frequent or unreasonable switching of operating conditions.

[0021] 3. Significantly reduced energy consumption is achieved while strictly ensuring on-time performance. Through the synergistic optimization of the aforementioned attention mechanism, the model can more intelligently balance the two objectives of "on-time" and "energy saving." It can learn more economical strategies, such as using early coasting instead of the "cruise-braking" combination before speed limits, and timely traction to compensate for speed loss after traversing steep inclines, among other optimal control sequences. Multi-scenario experiments demonstrate that the trajectory generated by the proposed method, while keeping the running time deviation within a very small range (far less than the allowable deviation of 2 seconds), reduces energy consumption by an average of more than 10% compared to traditional DT methods, exhibiting excellent overall performance.

[0022] 4. Improved model interpretability and generalization adaptability under different track conditions. The method of this invention makes the model's decision-making process traceable and understandable. By observing the distribution of attention weights, it is clear how the model focuses its attention on key constraint sections of the track, which is highly consistent with the understanding of domain experts, proving the physical consistency of the model. At the same time, because it explicitly incorporates universal track physics knowledge (gradient, speed limit), rather than relying solely on data-driven approaches specific to a particular track, the model has stronger generalization ability and robustness when facing new scenarios with different gradient combinations and speed limit settings, enabling it to quickly generate safe, energy-efficient, high-quality trajectories. Attached Figure Description

[0023] Figure 1 This is a flowchart of the train trajectory optimization method in an embodiment of the present invention; Figure 2 This is a framework diagram of the train trajectory optimization algorithm in an embodiment of the present invention; Figure 3 This is a schematic diagram of train line discretization in an embodiment of the present invention; Figure 4 This is a schematic diagram of the sparse attention mechanism with block rules in an embodiment of the present invention; Figure 5 This is a diagram illustrating the attention bias mechanism for fusing domain knowledge in an embodiment of the present invention. Figure 6 These are heatmaps of attention scores for different algorithms in embodiments of the present invention. Figure 7 This is a comparison diagram of train trajectories using different algorithms in an embodiment of the present invention; Figure 8 This is a trajectory change diagram under multiple speed limit conditions in an embodiment of the present invention; Figure 9 This is a comparison chart of convergence curves of different algorithms in the embodiments of the present invention; Figure 10 This is a box diagram showing the deviation between energy consumption and operating time in an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0025] Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this invention.

[0026] Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0027] See Figures 1 to 4 As shown, in order to achieve synergistic improvement in train operation trajectory in terms of safety, energy saving, and punctuality, and to provide an efficient and reliable solution for intelligent control of urban rail transit, this embodiment provides a train operation trajectory optimization method, including the following steps: Step 1: Establish a train operation system model 1.1 Train Dynamics Model: Based on the longitudinal force relationships during train operation, train dynamics equations are established, comprehensively considering factors such as traction force, braking force, basic resistance, and additional resistance. By introducing acceleration, velocity, and train mass parameters, an energy consumption calculation model is constructed, providing fundamental physical constraints for subsequent trajectory optimization. This model can realistically reflect the dynamic characteristics and energy consumption distribution features of the train under different gradients and speed limits.

[0028] 1.2 Track Discretization: To meet the modeling requirements of Markov decision processes, the track is discretized spatially. Based on the points of gradient change and speed limit inflection points, the continuous track is divided into several segments, ensuring that the speed limit and gradient remain constant in each segment. This unequal-length spatial partitioning significantly improves the accuracy and computational efficiency of state modeling while ensuring the model's adaptability under different track conditions.

[0029] 1.3 Establishing a Markov Decision Process Model for Train Operation Control: The train control problem is modeled as a Markov Decision Process (MDP). The state vector is defined to include features such as train position, speed, time, and energy consumption; the action space consists of four typical operating conditions: maximum traction, cruise, coasting, and maximum braking; the reward function comprehensively considers both running time deviation and energy consumption level to balance train punctuality and energy efficiency. This modeling approach provides a theoretical basis for reinforcement learning strategy optimization.

[0030] Step 2: Integrating Domain Knowledge with Block-Based Rules in a Sparse Self-Attention Mechanism 2.1 Sparse Attention Mechanism with Block Rules: To address the long-term reward information loss problem in traditional Decision Transformers in sparse reward environments, this invention proposes a sparse attention mechanism with a block structure. This mechanism combines the "reward value-state-action" of each time step into an independent decision block, establishing fully connected attention within the block to strengthen the dependencies between the three; inter-block attention employs a multi-layered filtering strategy to retain highly relevant key temporal information and suppress redundant interference. This structure ensures that each decision unit can effectively retain long-term reward signals, maintain the continuity and interpretability of the trajectory, thereby improving the stability and training efficiency of the model.

[0031] 2.2 Attention Bias Mechanism Integrating Domain Knowledge: To enable the model to explicitly perceive the physical characteristics of orbits, this invention introduces a domain knowledge bias matrix into the attention calculation of the Transformer. This matrix consists of two parts:

[0032] (1) Gradient bias: reflects the impact of changes in track gradient on train inertia and energy consumption, guides the model to allocate more attention weights to key gradient sections, thereby optimizing energy use; (2) Speed ​​limit bias: The attention weight is dynamically adjusted based on the difference between the current speed of the train and the speed limit to strengthen the model’s compliance with the speed limit constraint and prevent speeding during the trajectory generation process.

[0033] By weighted and fused gradient bias and speed limit bias to form a joint bias matrix, the model is guided to focus on the key constraint sections of the line during the training phase, thereby achieving explicit modeling of operational safety and energy consumption constraints.

[0034] Step 3: DK-BSDT Model Training and Optimization In the Transformer architecture with domain knowledge bias, the reward value, state, and action in the trajectory sequence are used as model input. Features are aggregated through a block-based rule-based sparse attention layer, and then guided by domain knowledge through a bias matrix layer. The model employs a supervised optimization method, aiming to minimize trajectory generation error and reward bias, and continuously updates network parameters through backpropagation. Specifically, the loss function objective is to minimize trajectory generation error and reward bias, where trajectory generation error is calculated by the difference between predicted and actual actions, and reward bias is calculated by the difference between predicted and actual rewards. When the loss function value is below a preset threshold, the parameters of the DK-BSDT model are fixed.

[0035] During training, the model can gradually learn the optimal control strategy for the train under different gradients, speed limits, and energy consumption conditions, achieving energy minimization under safety constraints. After training, the model can directly generate energy-efficient optimal trajectories that meet operational safety conditions, which can be used for real-time decision-making in train operation control.

[0036] Step 4: Experimental Verification and Effect Analysis A typical urban rail transit line was selected for verification experiments, covering different gradients and speed limits. Experimental results show that the proposed DK-BSDT method significantly outperforms the traditional Decision Transformer and the improved sparse DT model in terms of convergence speed, energy consumption, and trajectory smoothness.

[0037] Specifically, the model quickly acquires a stable strategy in the early stages of training, significantly improving convergence speed; the trajectory generation results are continuous and smooth, without frequent action switching, and the train running time deviation remains within a very small range; energy consumption is reduced by an average of more than 10% compared to traditional methods. Meanwhile, attention distribution results show that this method can automatically focus on speed limit change points and gradient transition sections, demonstrating good model interpretability and physical consistency.

[0038] To illustrate the real-time feasibility of the above method more specifically, each step is explained in detail below: Establish a train operation system model 1.1 Train Operation Control Equations Train travel time is ( (Total running time), running location is ( (Total line mileage), operating speed is The train operation control model, i.e., the dynamic equations of train operation, can be expressed as: in, For traction force; For braking force; The total resistance of the train is determined by... (Basic resistance of train operation) and (Additional resistance to the line) can be expressed as: During train operation, speed constraints must be met: ( For position (Speed ​​limit at the location); Boundary constraints: , , Logical constraints: Traction and braking conditions cannot be directly switched; a transition through cruise or coasting is required.M For the quality of the train.

[0039] The fundamental problem of train energy-saving control is finding a train speed distribution that satisfies time constraints. Numerous scholars have studied the relationship between speed and energy consumption distribution and proven that the optimal energy-saving control strategy comprises four stages: maximum traction, cruise, coasting, and maximum braking. Therefore, this paper employs maximum traction force under traction conditions and maximum service braking force under braking conditions in train control.

[0040] Traction mode (MT): The train outputs maximum traction force to increase speed, therefore , , .

[0041] Cruise (CR) mode: The train maintains a constant speed, therefore, Coasting (CO): The train has no active power output and relies solely on inertia to coast. , , .

[0042] Braking mode (MB): Actively reduce speed or stop, therefore , , .

[0043] Therefore, the action space is defined as .

[0044] 1.2 Line Discretization To facilitate problem modeling, we discretize the train operation process into N segments based on the line data. Then, the train in segment i ( Train travel time and operating energy consumption for: in, This refers to the train's travel time. For the train's energy consumption; i= 1, 2, ... N , N The total number of sections is defined; x The train's position; For traction force, ; For the train from the i Starting point of each section To the finish line .

[0045] The deviation in running time refers to the actual total running time of the train. With planned running time deviation, that is: in, , These represent the minimum and maximum travel times of the train within the section, respectively.

[0046] In representing train traction energy consumption: Then the objective function It can be defined as: 1.3 Markov Model for Train Operation Control Train operation control can be viewed as a multi-stage Markov decision process (MDP), which can be described as... ,in Representing the state space; Represents the action space; Represents the state transfer function; Represents the reward function. We use... , and Let represent the state, action, and reward at step i (at timestepi), respectively.

[0047] Train trajectory is affected by train position, speed, and line speed limits. Line gradient Operating energy consumption and running time Due to factors such as these, the train's state is defined as: In classic train operation control, the train trajectory is divided into four stages: traction, cruising, coasting, and braking. Therefore, its action space can be defined as... , .

[0048] The reward function should balance the objectives of minimizing runtime deviation and minimizing operating energy consumption, respectively using... and Indicates runtime deviation reward and operating energy consumption reward .

[0049] The reward function is: in, For the planned running time, E For train energy consumption; L Line length Combining formulas (7), (8), and (9), we construct the loss function for model training. L total : , Action prediction is achieved using a mean squared error (MSE) calculation model. With real action a t Deviation: , in, T For sequence length, The model is based on historical sequences (such as...) s <t , a <t , r<t Predicted actions.

[0050] Rewards include runtime deviation bonuses (Formula 7) and operating energy consumption bonus (Formula 8), the total reward function is Formula 9.

[0051] The error in return prediction needs to be measured in terms of the model's predicted returns. With real returns R Deviation: , here, R t It is the instant reward defined by Formula 9. It is the model's prediction of future returns (such as through a Return-to-Go mechanism).

[0052] 1.4 Serialization Modeling and Sparse Attention Mechanism for Train Operation Control In train operation control problems, combinatorial methods are possible. , and The train operation process can be serialized as follows: The Transformer encodes the raw time-series data into feature vectors containing location information. These feature vectors are: Query vector, Key vector, and Value vector. Self-attention uses independent Query(Q), Key(K), and Value(V) matrices, and calculates the association weights of different feature dimensions at each time step through the dot product of Q and K, achieving dynamic weighted aggregation of the input sequence information.

[0053] in, This is represented as the dimension of the Key vector.

[0054] SDT uses attention weighting Sort and retain Higher-ranking elements will mask those assigned lower ranks. Non-essential elements. Smaller than random discarding. The dropout strategy for scoring differs, based on a matrix. Implement adaptive selection of top- k The contribution scoring mechanism aims to retain the most critical components and eliminate useless elements, followed by normalization.

[0055] Here, k is an adjustable parameter that dynamically controls the sparsity. It is top- k The learnable top- k selection operator); for The k-th largest value in the j-th row of QK after sorting from largest to smallest. k "" is a key hyperparameter used to control the sparsification process in the attention mechanism. Specifically, it represents the number of the most important key-value pairs retained for each query vector during sparse attention computation. k The value is an integer (e.g.) k 1, k 2, ..., kM ), by dynamically selecting the highest attention score k By selecting the top-k elements and ignoring the remaining low-scoring items, redundant information is reduced, improving computational efficiency and model focusing ability. Simply put, the top-k mechanism is similar to "focusing only on the top-ranked elements." k "Find the most relevant information" to avoid distraction.

[0056] II. Sparse Self-Attention Mechanism Integrating Domain Knowledge and Block Rules 2.1 Sparse Attention Mechanism with Block Rules Because the DT algorithm concatenates the RTGs, states, and actions into a single sequence, which is then used as the input to the transformer, i.e. As a tuple, these elements collectively reflect the agent's interaction trajectory and goal-oriented information within the environment. However, in train trajectory optimization, the sparse reward evaluating the train's trajectory is only obtained when the train reaches its destination. Therefore, the correlation between the RTGs sequence and the trajectory is low, which manifests in the self-attention stage as an attention score related to the reward r. The score is low, and in sparse attention systems, it is often set to 0 when sorting and filtering, weakening the guiding role of reward in trajectory learning. As for the score related to state s... Essentially, all information is retained, causing the model to focus excessively on the state level (such as current position, speed, and other immediate state information). It can only learn the superficial rules of "what action to take in the current state to maintain trajectory continuity," but cannot establish the deep correlation of how the current action affects long-term rewards. This leads the model into a predicament where trajectory learning deviates from the goal orientation. Ultimately, the model cannot learn a truly globally optimal trajectory that meets the end-point reward requirements, and may get stuck in local optima, severely limiting the actual effectiveness of train trajectory optimization.

[0057] Therefore, we propose a sparse attention mechanism with block rules, addressing the inherent structure of the DT algorithm. (See...) Figure 9 The RTG, state, and action corresponding to any trajectory step t are combined into an independent decision block, i.e. Within each independent decision block, an indivisible decision unit is defined. The attention mechanism first performs fully connected attention modeling on the three elements within the block, and the attention weights within the block do not participate in the "zeroing" of the global sparse sorting. When the train is in a certain running step, its corresponding RTG will form a mandatory association with the current state and action within the block. The RTG is no longer an isolated sequence element, but serves as the "target benchmark for decision-making in this step," deeply bound to the observation information of the state and the execution choice of the action. Even if the initial attention score of the RTG within the block is relatively low, it can be retained through the mandatory association within the block, thereby conveying the key signal of "whether the current action meets the long-term reward goal," helping the model establish an immediate causal cognition of "action-state-reward."

[0058] in, , They represent The projection matrix; They represent The mapped Query, Key, and Value vectors.

[0059] Intra-block attention locking ensures that the association between RTG and state and action is not severed through "intra-block fully connected computation".

[0060] in, , .

[0061] right Normalization is performed to obtain the intra-block attention weights. : Then, weighted summation is used to obtain the intra-block attention. : We take respectively The mean of the decision block The Query, Key, and Value vector, i.e.: The inter-block attention score can be calculated as follows: To avoid top- k In the sparse rules (see Formulas 12 and 13), a single k value leads to the loss of key correlation information. We adopt a multi-k value approach. The summation filtering method combines the filtering results of different granularities to retain key related information while controlling redundant information. Finally, normalization is performed to obtain the attention weights between blocks: In the formula, Inter-block attention weights; For inter-block attention scoring; for The selection operator is used to dynamically control sparsity; ,in k The hyperparameter controlling sparsity represents the number of most important key-value pairs retained for each query vector during sparsification. It determines the "sparseness" of the attention weights.

[0062] Used by inter-block weights Local features of a block After performing a weighted summation, the attention output is obtained as follows: Through this sparse attention mechanism with block rules, the DT algorithm can re-establish the strong correlation logic between RTG, state, and action while maintaining computational efficiency: each decision block becomes the smallest decision unit of "goal-observation-execution", the forced correlation within the block ensures that RTG is not ignored, and the orderly interaction between blocks ensures the unity of trajectory timing and reward orientation. Ultimately, this helps the model break through the attention imbalance bottleneck in sparse reward scenarios and truly realize "guiding each step of trajectory optimization with long-term reward goals", providing more accurate decision support for reducing train energy consumption and improving efficiency.

[0063] 2.2 Attention Bias Mechanism for Integrating Domain Knowledge Train trajectories are closely related to track gradients and speed limits. When a train passes through a steep gradient, a speed-limited turning point, or approaches a speed-limited section, its operating conditions need to be adjusted promptly to meet environmental constraints. However, traditional Transformers rely entirely on data-driven implicit learning, making it difficult to explicitly highlight these key features. They often incorporate irrelevant factors into attention interactions, weakening the model's ability to model in complex scenarios. Introducing domain knowledge can help the model focus more on the key factors of the track scenario, capture the patterns of operating condition changes, and improve its adaptability to complex constraints.

[0064] Its advantages are mainly reflected in: ① enhancing the physical correlation between features, making attention more focused on the key parts of the trajectory sequence and improving decision accuracy; ② providing constraining features, reducing invalid exploration, and accelerating model convergence; ③ improving the ability to finely characterize complex orbital scenarios by integrating static attributes and dynamic changes.

[0065] To address this, this paper proposes a BSDT that integrates domain knowledge, combining the static physical properties of the track (gradient, speed limit) with the dynamic characteristics of train operation (train speed) to achieve rapid response to track scenarios.

[0066] Gradient characteristics: Different gradients directly affect the resultant force on the train and also influence the train's operation control strategy. This effect is particularly pronounced on steeper gradients. When a train travels on a steep gradient, for a train in coasting mode, the rapid change in gradient resistance causes significant fluctuations in train speed, impacting subsequent train operation control strategies.

[0067] We introduce an explicit slope bias into the attention mechanism, which strengthens the attention weight of the steep slope section on the subsequent section, and guides the model to actively capture the causal relationship between the control strategies of the steep slope section and the subsequent section.

[0068] in, For slope offset; To ensure sparse slope attenuation, in this paper, we take... =0.2. t and This refers to the index of the decision block.

[0069] t : Represents the index of the target block that is currently being computed in its query vector.

[0070] : Represents the target block t When calculating attention, it is used as an index of all source blocks in the context. |t-t'| It calculates the relative distance between blocks. This reflects the "block" concept. Slope information for blocks t The impact of a decision decreases as the distance between two blocks increases.

[0071] Speed ​​limit characteristics: Speed ​​limit constraints are key boundary constraints in trajectory optimization, directly affecting train operation control strategies. The relative relationship between speed and speed limit. This reflects the need for dynamic speed adjustment:

[0072] when When the speed is far below the speed limit, it indicates that the average operating speed of the train needs to be increased through methods that can accelerate it; when When the speed is close to the speed limit, it indicates that you need to use cruise control or coasting to smoothly adjust your speed and avoid the safety hazard of speeding; when When the train is speeding, it indicates that the train is speeding.

[0073] Therefore, by introducing a speed limit bias, we strengthen the attentional connection between the key speed limit section and the relevant decision section, guide the model to more accurately adapt to speed constraints, and improve the safety and economy of the control strategy.

[0074] In real-world railway scenarios, speed limits and gradient factors simultaneously influence train control decisions. Gradient bias ( This describes the inertial impact of steep gradient sections on subsequent sections, and the speed limit comprehensive offset ( It also specifically strengthens the control guidance for related sections under different speed states (far below the speed limit / close to the speed limit / speeding). Therefore, we integrate the two biases into a unified domain knowledge bias matrix to achieve collaborative expression of multiple knowledge.

[0075] III. Results and Analysis 3.1 Experimental Design To verify the effectiveness of the proposed DecisionTransformer algorithm, which integrates domain knowledge and dynamic sparse self-attention mechanism, in train trajectory optimization, we designed a multi-scenario comparative experiment to systematically evaluate the performance of the algorithm. The sample data is based on the physical lines of Beijing Metro Yizhuang Line and Guangzhou Metro Line 21, and some virtual lines are also added. The PPO algorithm

[44] is used to obtain the train's running trajectory between stations on each line, totaling 756 trajectory sample data. The experimental hardware environment uses an Intel Core i7-14700K processor, an NVIDIA RTX 4070 graphics card (12GB video memory) and 32GB DDR4 memory. The software is based on the PyTorch 1.13 framework for model training and fine-tuning.

[0076] Experiments were conducted to compare the original DT algorithm, SDT, BSDT, and DK-BSDT algorithm (i.e., the DK-BSDT model), analyzing the synergistic gain effect of the sparse self-attention mechanism with block rules and domain knowledge modules on the DT algorithm. The algorithm's scenario adaptability was verified under different line scenarios. Finally, statistical significance tests were used to ensure the reliability of the experimental conclusions, providing data support for the engineering application of this algorithm.

[0077] To compare the guiding effects of domain knowledge fusion and dynamic sparse self-attention mechanism on the model's focus and to intuitively display the experimental results, this paper sets the sequence length to 20 and selects the attention score of one head for plotting. Figure 10 (a) is a heatmap of the attention score of the DT algorithm. Its attention is relatively scattered and does not focus on a specific area. The attention to each segment of the overall sequence is relatively balanced, making it difficult for the model to highlight key information and effectively focus on areas that are important to the task. Figure 10 (b) is a heatmap of the attention score for the SDT algorithm. It can be seen that all elements related to states are reinforced, which is detrimental to building a connection between the current state and the reward. Compared to SDT... Figure 10(b) While consistently emphasizing state-related elements and weakening elements related to actions and RTGs, the attention score fails to reflect the importance of different trajectory steps to the sequence. In contrast, the sparse rule with block rules, filtering based on inter-block attention scores, significantly weakens the first four trajectory steps while effectively strengthening subsequent steps. This helps ensure consistency between trajectory timing and reward orientation. DK-DSDT, by incorporating domain knowledge and strengthening its focus on key track scenarios, effectively distinguishes between "ordinary trajectory steps" and "core constraint steps," ensuring that the attention score matches the actual needs of train trajectory optimization.

[0078] 3.2 Ablation Experiment We will use the Beijing Subway Yizhuang Line, specifically the section between stations (planned travel time 135 seconds), as an example to test the effectiveness of the algorithm. Figure 9 The convergence comparison curves clearly demonstrate the differences between the DT algorithm and the SDT algorithm (the traditional top-down algorithm). k The differences in performance of the Sparse Transformer (SVT), ISDT, and IDK-ISDT algorithms during online fine-tuning were analyzed. The DT algorithm had the slowest convergence speed, requiring 355 iterations to reach a stable convergence state, reflecting its low efficiency in feature mining and optimization and the need for more iterations to achieve stable performance. The SDT algorithm, by amplifying the expression of advantageous features and reducing redundant information interference to some extent, converged significantly faster than the DT algorithm, achieving a similar convergence level to the DT algorithm after 355 iterations in just 265 iterations. The BSDT algorithm, by using block-based sorting and filtering rules to better highlight contextual relationships, further improved convergence efficiency, achieving convergence in approximately 213 iterations. The DK-BSDT algorithm, by introducing rate limiting and slope domain knowledge on top of the BSDT algorithm, helped the algorithm more accurately focus on key features and reduce invalid exploration, effectively improving convergence efficiency and stability, achieving convergence in only 94 iterations, and demonstrating the best overall performance.

[0079] Figure 9 For DT algorithm, SDT algorithm (traditional top-down algorithm) kTable 2 shows the train trajectories obtained by the sparse transformer, BSDT, and DK-BSDT algorithms in the case section, along with the train's operating energy consumption, operating time, and rewards. The train trajectories obtained by the DT and SDT algorithms are similar, with deviations from the planned operating time within 1 second. The SDT algorithm better controls the relationship between cruise and coasting, achieving relatively low energy consumption of 19.23 kW•h. The BSDT algorithm is better able to acquire contextual features, using coasting instead of a combination of cruise and braking before speed limits to reduce energy consumption. Its energy consumption is 17.83 kW•h, and its operating time is 135.31 seconds, with a deviation from the planned operating time within 1 second. The DK-BSDT algorithm, by incorporating domain knowledge, can effectively identify speed limit features and uses a traction strategy to increase the train's average speed after passing through the speed limit section. Ultimately, it achieves the minimum operating energy consumption of only 17.44 kW•h while ensuring a small deviation in operating time. As can be seen from Table 2, among the four algorithms, the DK-BSDT algorithm has the lowest energy consumption and the smallest running time deviation, thus obtaining the largest reward of 24.48.

[0080] The above experiments fully verify the effectiveness of block-based sparse mechanisms and the integration of domain knowledge in improving the performance of the transformer algorithm. Among them, the block-based sparse attention mechanism (BSDT algorithm) based on sorting and filtering rules outperforms the traditional top-order algorithm. k The Sparse Strategy (SDT algorithm) better strengthens the contextual relationships, which can effectively improve the convergence efficiency and model accuracy. At the same time, knowledge in areas such as speed limits and gradients can be matched with block-based sorting and filtering rules, enabling the algorithm to capture key features more accurately, reduce invalid exploration, and significantly improve the performance of the transformer in train operation optimization scenarios.

[0081] Table 1. Train operating energy consumption, operating time, and rewards 3.3 Performance Comparison of Algorithms in Multiple Scenarios like Figure 10As shown, when a train is delayed, in order not to affect the operation plan of subsequent trains, it is necessary to reduce the train's running time in the section. Furthermore, due to factors such as construction, temporary speed limits may need to be set in specific areas of the section, which also affects the train's handling strategy. Therefore, to test the adaptability of the algorithm under different speed limits and train delay scenarios, this paper uses a section of track containing extreme speed limits (including large downhill and uphill sections) as a basis to test the train's running trajectory in six different scenarios (Scene ①④ speed limit zones are located in the latter part of the track; Scene ②⑤ speed limit zones are located in the latter part of the track; Scene ③⑥ speed limit zones are set at the beginning and end of the track respectively; the planned running time for Scene ①②③ is Pt=150s, and the planned running time for Scene ④⑤⑥ is 155s). The running results obtained by different algorithms are shown in Table 2.

[0082] In three different scenarios, the DT train accelerates initially and uses extensive cruising to maintain its speed. To meet the next speed limit requirement, it directly decelerates by braking, consuming train energy and resulting in relatively high energy consumption. In scenarios ①②③④⑥, the energy consumption is 24.91, 26.60, 32.88, 21.98, and 27.83, respectively. However, the deviation between its running time control and the planned time is less than 0.4 seconds, which meets the requirements.

[0083] SDT's control mode is similar to DT's in scenarios ①②③⑥, and its control strategy is similar. However, in scenarios ①②⑥, it accelerates to a relatively low speed initially and then transitions to cruising. Therefore, braking near the speed limit consumes less energy than DT, with total energy consumption of 24.20, 26.19, and 27.90 respectively. The opposite is true in scenario ③. In scenarios ④ and ⑤, it reduces energy loss due to cruising through a longer inertia period, with energy consumption of 21.81 and 23.60 respectively, which is better than the DT algorithm.

[0084] The BSDT algorithm can optimize train trajectory based on reward guidance, and it has obvious advantages in scenarios ①②④⑤⑥. In different speed-limited sections, it can reduce the braking energy loss caused by speed limit through sufficient inertia. Its total energy consumption is 22.42, 24.61, 20.09, 22.11 and 26.81 respectively, and the running time deviation is less than 0.5s.

[0085] The DK-BSDT algorithm fully considers information such as track speed limits. In scenarios ①②④⑤, after acceleration, sufficient coasting control minimizes braking losses caused by intermediate speed limit constraints, with total energy consumption of 21.83, 24.31, 19.51, and 21.85 respectively, and running time deviations of less than 0.3s. In scenarios ③ and ⑥, to meet the constraints of the two speed limits on train speed, under the guidance of long-term rewards, the trajectory can still reduce energy consumption through coasting control, with total energy consumption of 26.81 and 24.11 respectively.

[0086] In summary, BSDT, by strengthening the reward orientation, enables the algorithm to better control the strategy and obtain train running trajectories with good energy consumption performance while meeting the running time target. The DK-BSDT algorithm, on the other hand, better adapts to complex scenario constraints by accurately adapting to track speed limit constraints, and obtains better train running trajectories through timely and effective traction and coasting cooperative control, thus exhibiting optimal performance.

[0087] Table 2. Operating energy consumption, operating time, and rewards for the following vehicles in different scenarios. Using the PPO algorithm as a benchmark, the deviation between the energy consumption of several algorithms in obtaining the optimal trajectory in different scenarios and the energy consumption obtained by the PPO algorithm was tested, and a box plot of energy consumption deviation was plotted. Figure 10 As shown in (a), the energy consumption deviation of the DT algorithm is relatively stable, but its average deviation is the highest at 2.98. The energy consumption deviation stability of the SDT algorithm is not as good as that of the DT algorithm, but its average energy consumption deviation is 2.79, which is worse than that of the DT algorithm. The average energy consumption deviation of the BSDT algorithm is 1.11, but its stability is poor. The DK-BSDT algorithm can achieve the best average energy consumption deviation of 0.37, and the energy consumption deviation of all its trajectories and the PPO algorithm is within 1, showing strong stability.

[0088] Figure 10 (b) is a box plot showing the deviation between the running time and the planned running time of the optimal trajectory for the DT, SDT, BSDT and DK-BSDT algorithms in various scenarios. The average running time deviations are 0.12, 0.10, 0.18 and 0.21, respectively. It can be seen that the average running time deviation of the four algorithms is within 0.25s, which is much smaller than the allowable deviation of 2s.

[0089] Therefore, DK-BSDT exhibits optimal performance across various scenarios.

[0090] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for optimizing train trajectory, characterized in that, Includes the following steps: Obtain the train operation parameter set, input the train operation parameter set into the DK-BSDT model, and obtain the train's energy-optimal operating trajectory; The construction of the DK-BSDT model includes the following steps: Based on the set of train operating parameters, construct dynamic equations; Based on the quantification of the dynamic relationship of various parameters in the train operation parameter set by the dynamic equation, the line for continuous train operation is discretized into several sections based on the gradient change point and the speed limit inflection point, and the speed limit value and gradient value of each section are obtained. Based on the discretization of unequal-length space, train control is modeled as a Markov decision process, thereby defining the train's state vector and action space. A reward function is established based on the train running time deviation and energy consumption level within each section, and the future reward for each time step is obtained based on the reward function. The future reward, state vector, and action space of each time step within each segment are combined into an independent decision block. Within a single decision block, a fully connected attention calculation is performed between the future reward, state vector, and action space. Between different decision blocks, the discretized gradient value and speed limit value are transformed into an attention bias matrix, which is used as an explicit injection of the train operation physical constraints to determine the attention weights. Based on the fully connected attention calculation and determined attention weights, and combined with the future reward, state vector, action space, independent decision blocks, and attention bias matrix, the DK-BSDT model is trained. When the loss function of the DK-BSDT model reaches the preset requirements, the parameters of the DK-BSDT model are fixed.

2. The train trajectory optimization method according to claim 1, characterized in that, The dynamic equation is determined according to the following formula: ， in, , In the formula: t Runtime; x For the running location; v For running speed; For traction force, ; For braking force, ; For the total operating resistance, ; This serves as the basic resistance for operation; Add resistance to the line; M For the quality of the train.

3. The train trajectory optimization method according to claim 1, characterized in that, The state vector defined by the Markov decision process includes train position, speed, time, and energy consumption characteristics, and the action space includes the train's maximum traction, cruising, coasting, and maximum braking. The reward function is obtained based on the train's running time deviation and energy consumption level within each segment.

4. The train trajectory optimization method according to claim 1, characterized in that, After the line is discretized into unequal-length sections, the running time and energy consumption level of each section are determined according to the following formula: ， In the formula, This refers to the train's travel time. For the train's energy consumption; i= 1, 2, ... N , N The total number of sections is defined; x The train's position; For traction force, ; For the train from the i Starting point of each section To the finish line .

5. The train trajectory optimization method according to claim 3, characterized in that, The reward function is determined according to the following formula: ， in, , ， In the formula, Rewards for train running time deviations; Incentives for train operation energy consumption; For the planned running time, E For train energy consumption; L This represents the line length.

6. The train trajectory optimization method according to claim 1, characterized in that, Within the decision block, a fully connected attention calculation is performed among the reward, state, and action, determined according to the following formula: ， In the formula, , , They are respectively The projection matrix; , , They are respectively The mapped query vector, key vector, and value vector; , r For future returns, S For state, a For action.

7. The train trajectory optimization method according to claim 1, characterized in that, The attention weight uses multiple k The summation and filtering strategy is determined according to the following formula: ， In the formula, Inter-block attention weights; For inter-block attention scoring; for The selection operator, The number of most important key-value pairs retained for each query vector, used to dynamically control sparsity; ,in k Hyperparameters for controlling sparsity.

8. The train trajectory optimization method according to claim 1, characterized in that, The attention bias matrix includes gradient bias and speed limit bias. The gradient bias is dynamically adjusted based on the absolute value of the gradient to strengthen the attention weight of steep gradient sections, and the speed limit bias is dynamically adjusted based on the difference between the train's current speed and the speed limit.

9. The train trajectory optimization method according to claim 8, characterized in that, The slope offset is determined according to the following formula: ， in, , In the formula, For slope offset; The slope attenuation is sparse; and This refers to the index of the decision block; This represents the index of the target block of the query vector that is currently being computed. Index representing all source blocks ,|t-t'| It is the relative distance between blocks.

10. A train trajectory optimization method according to claim 8, characterized in that, The speed limiting offset is determined according to the following formula: ， in, , ， In the formula, For speed limiting offset; This refers to the relative relationship between train speed and speed limit. For train speed, Speed ​​limits for trains.

Images