Virtual formation train control method considering uncertainty

Abstract

The invention provides a virtual formation train control method considering uncertainty, and relates to the technical field of railway train control, and the method comprises the following steps: obtaining static data of stations, train operation and line operation in a marshalling line, and building a POMDP model; collecting real-time state information of train operation, judging whether front and back trains are in a safe action space, if so, ending the decision process, and if not, continuing to carry out the following steps; carrying out belief updating by using an IMM method; a collision envelope and a risk envelope of the worst case are calculated, opportunity constraints are established based on the collision envelope and the risk envelope, and a CC-POMDP model is constructed; the optimal action of the CC-POMDP model is solved; and the rear vehicle executes a decision according to the optimal action. According to the method, the formation trains can make proper decisions under the uncertain high-risk condition, a recommended speed curve does not need to be formulated in advance, and the travel safety of passengers is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of railway train control technology, and in particular to a virtual train formation control method that takes into account uncertainties. Background Technology

[0002] Traditional fixed-formation trains face the problem of mismatch between passenger demand and transport capacity. To overcome these issues, flexible formations have been developed to achieve optimal coordination between passenger demand and transport capacity. Currently, flexible train formation methods include mixed operation of various formations, "Y"-shaped formations, online coupling formations, and virtual formations, among which virtual formations offer the highest adaptability. For trains in a platoon, trains are connected via car-to-car communication instead of physical couplers. Through the comprehensive application of advanced automation technology, wireless communication technology, intelligent algorithms, and safety systems, they can be freely combined and operate collaboratively.

[0003] Research on virtual train platooning is currently in its early stages. Due to the shorter spacing between trains in a platoon, deploying this autonomous system in the real world presents risks due to unknown and unpredictable situations. Trains in a platoon need to make appropriate decisions under uncertain and high-risk conditions. Unlike decisions made under deterministic conditions, decisions made under uncertainty involve unknown probabilities and inherent risks. Currently, urban rail train operation control using ATO and ATP systems has achieved GOA4 level automatic driving, but it lacks intelligent autonomous decision-making capabilities such as adaptive changes to the environment.

[0004] One of the challenges facing driving decision-making systems is the need to handle various uncertainties, such as uncertainties in environmental perception, sensor errors, inaccuracies in behavioral models, and uncertain movements of traffic participants. These uncertainties can lead to biased or even erroneous decisions, which can cause significant safety concerns for traffic.

[0005] Therefore, there is an urgent need in this field for a control method for virtual queuing vehicles that can take into account various instabilities. Summary of the Invention

[0006] The purpose of this invention is to provide a virtual queuing vehicle control method that takes into account uncertainty, which can solve the technical problem in the prior art that uncertainty factors can lead to biased or even erroneous decisions, and cause great safety problems for traffic safety.

[0007] This invention provides a virtual queuing vehicle control method that takes into account uncertainties, comprising the following steps:

[0008] Step S1: Obtain station, train operation and line operation data information within the train formation line, and establish a POMDP model for train decision-making based on the data information;

[0009] Step S2: Collect real-time status information of train operation, and determine whether the trains in front and behind are in a safe operating space based on the real-time status information. If they are in a safe operating space, the decision-making process ends; otherwise, continue to the following steps.

[0010] Step S3: Update beliefs using the IMM method;

[0011] Step S4: Based on the train safety braking model and the actual braking process, calculate the worst-case collision envelope and risk envelope, establish opportunity constraints based on the collision envelope and risk envelope, and construct the CC-POMDP model;

[0012] Step S5: Use the POMCP-DS-PW algorithm to solve for the optimal action of the CC-POMDP model;

[0013] Step S6: The following vehicle executes the decision according to the optimal action and jumps to step S2.

[0014] Furthermore, the establishment of the POMDP model for train decision-making in step S1 includes the following steps: defining the state space, action space, transition function, observation space, observation function, and reward function of the POMDP model.

[0015] Furthermore, the reward function step only considers speed limit factors, efficiency factors, comfort factors, and energy-saving factors.

[0016] Furthermore, the belief update using the IMM method in step S3 includes the following steps: state interaction, model conditional filtering, model probability update and state estimation fusion.

[0017] Furthermore, the optimal action for solving the CC-POMDP model using the POMCP-DS-PW algorithm in step S5 is implemented based on the POMCP module, the double shield module, and the progressive widening module.

[0018] The POMCP module is used to represent history using the MCTS algorithm and to select the optimal action based on the UCB algorithm.

[0019] The dual shield module is used to shield unsafe actions by employing pre-pruning and free backtracking extended POMCP algorithms.

[0020] The progressive widening module is used to add the sampled observation space to the tree to control the rate at which the adopted observations are added to the tree.

[0021] Furthermore, the MCTS algorithm includes the following repeated steps: selection, expansion, simulation, and backtracking.

[0022] The selection step starts from the root node of the tree and moves to the leaf node according to the tree strategy;

[0023] The extension step is used to add leaf nodes to the tree;

[0024] The simulation step begins by executing a pre-simulation strategy from a leaf node to determine the quality of that leaf node.

[0025] The backtracking step returns from the leaf node to the root node, and the UCB algorithm is used to guide the root node to select the optimal action.

[0026] Furthermore, the pre-pruning step in the dual-shield module includes: iteratively updating each action within each time step and calculating the total probability of violating the risk envelope; finding all actions that satisfy the risk envelope constraints after the MCTS algorithm; and pruning the corresponding branches at the root node.

[0027] Furthermore, the free backtracking step in the dual shield module includes: during the MCTS algorithm process of the POMCP module, when an action is selected through the pre-play strategy, checking whether the generated next state satisfies the risk envelope constraint; if the risk envelope constraint is violated, pruning the tree branch starting from that node and shielding the action in that node.

[0028] Further, step S4: Calculating the worst-case collision envelope and risk envelope based on the train safety braking model and the actual braking process, establishing opportunity constraints based on the collision envelope and risk envelope, and constructing the CC-POMDP model includes:

[0029] Based on the train safety braking model and the actual braking process, and based on the emergency braking curve and the braking curve of the following vehicle, the optimal tracking distance is calculated.

[0030] Calculate the worst-case collision envelope and risk envelope; the worst-case scenario is the moment when the two trains are at their minimum distance from the current moment until they come to a complete stop.

[0031] Establish collision envelope constraints and risk envelope constraints, and construct the CC-POMDP model.

[0032] Furthermore, when the safety action space is a non-empty set, the action that maximizes the value function is selected; when the safety action space is an empty set, it means there is no safe action, and the emergency braking action is executed directly.

[0033] The virtual queuing vehicle control method of the present invention, which considers uncertainty, has the following advantages over the prior art:

[0034] Beneficial effects:

[0035] 1. This invention takes into account the uncertainties caused by sensor noise and incomplete measurements, as well as the uncertainties caused by the unknown future behavior of the preceding train, and provides a safety guarantee for train operation decisions. This enables train queuing to make appropriate decisions under uncertain high-risk conditions without the need to pre-determine recommended speed curves, thereby improving operational efficiency and ensuring passenger travel safety.

[0036] 2. This invention proposes a continuous, online solution method for CC-POMDP, which consists of three modules: POMCP, double shield, and progressive widening, referred to as POMCP-DS-PW. This algorithm can solve for the optimal action and reduce the cost of computing the optimal action. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart illustrating the train operation control method for virtual formation according to the present invention;

[0039] Figure 2 A schematic diagram of information transmission for a queuing vehicle decision-making system;

[0040] Figure 3 Here is a block diagram of the IMM algorithm;

[0041] Figure 4 A schematic diagram of a platooning vehicle safety braking model;

[0042] Figure 5 A schematic diagram defining the safety envelope;

[0043] Figure 6 Define a diagram for the worst-case scenario;

[0044] Figure 7 Pseudocode for the search steps of POMCP-DS-PW;

[0045] Figure 8 Pseudocode for the simulation steps of POMCP-DS-PW;

[0046] Figure 9 Pseudocode for the pre-running steps of POMCP-DS-PW;

[0047] Figure 10 This is a schematic diagram of the optimal action obtained as described in Embodiment 1 of the present invention;

[0048] Figure 11 This is a schematic diagram of the speed-distance curve of the rear vehicle as described in Embodiment 1 of the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and the accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0050] like Figure 1 As shown, the present invention provides a virtual queuing vehicle control method that considers uncertainties, comprising the following steps:

[0051] This invention provides a virtual queuing vehicle control method that takes into account uncertainties, comprising the following steps:

[0052] Step S1: Obtain station, train operation and line operation data information within the train formation line, and establish a POMDP model for train decision-making based on the data information;

[0053] Step S2: Collect real-time status information of train operation, and determine whether the trains in front and behind are in a safe operating space based on the real-time status information. If they are in a safe operating space, the decision-making process ends; otherwise, continue to the following steps.

[0054] Step S3: Update beliefs using the IMM method;

[0055] Step S4: Based on the train safety braking model and the actual braking process, calculate the worst-case collision envelope and risk envelope, establish opportunity constraints based on the collision envelope and risk envelope, and construct the CC-POMDP model;

[0056] Step S5: Use the POMCP-DS-PW algorithm to solve for the optimal action of the CC-POMDP model;

[0057] Step S6: The following vehicle executes the decision according to the optimal action and jumps to step S2.

[0058] In further embodiments of the present invention, such as Figure 2As shown, the braking process in a train platoon is mainly a decision-making process for the following train based on the uncertainty of the preceding train's movement, until it stops running or the trains in front and behind reach a safe operating space, i.e., until a collision does not occur. The safe operating space state is defined as: the following train reaches the end of the section, both trains transition from running to stopped, or they maintain a relatively long distance from the preceding train (i.e., disjoint). When the termination state is reached, the decision-making process also stops. Therefore, this invention considers a train platoon consisting of two trains, namely the preceding and following trains, with the following train considered as its own, assuming that the following train can perfectly perceive its own state. Since the following train has uncertainties when receiving observations, the true state of the preceding train is partially observable, and the future movement of the preceding train is unknown. Furthermore, it is assumed that the states of the preceding and following trains are independent. Due to the partial observability of the preceding train's state, the decision-making process of the following train is modeled as a POMDP model.

[0059] In a further embodiment of the present invention, the establishment of the POMDP model for train decision-making in step S1 includes the following steps: defining the state space, action space, transition function, observation space, observation function, and reward function of the POMDP model.

[0060] The following is a detailed explanation of the steps involved in establishing the POMDP model for train decision-making.

[0061] First, the train status is defined as:

[0062]

[0063] The states include the states of the following vehicles (i.e., the vehicles themselves). The condition of the car in front and the interior state of the car in front m t (See Figure 2 The state of the following vehicle at time t is defined as follows:

[0064]

[0065] In the formula, x represents the position, that is, the vertical distance from the starting point. This represents speed. Because the train always travels on the tracks, its path can be considered one-dimensional, requiring only consideration of the longitudinal distance traveled and speed. The states of the preceding and following trains are expressed similarly, defined as:

[0066]

[0067] Where, and These represent the position, speed, and acceleration of the preceding vehicle, respectively. Because the train is quite long, x... p and x fThese refer to the positions of the rear of the vehicle in front and the front of the vehicle behind. Additionally, the interior state of the vehicle in front (m)... t This represents the kinematic model followed by the vehicle at time t. Possible kinematic model assumptions include constant velocity CV and constant acceleration CA.

[0068] The train's motion space includes three operating conditions: maximum traction (MA), coasting (CO), and maximum braking (MB), which correspond to maximum acceleration, no traction / braking force output, and maximum deceleration, respectively. Therefore, the motion space is defined as follows:

[0069]

[0070] Traction and braking are divided into two levels, corresponding to maximum acceleration and half of maximum acceleration. Considering that emergency braking is the highest level of braking and would have severe consequences, it is not used unless a complete stop is absolutely necessary. Therefore, an additional action space is defined, consisting solely of emergency braking EB:

[0071]

[0072] To realize the "fault-safe" concept, a safe action space A is defined. safe Select from the original action space A the current belief b(s). t Actions that, upon execution, simultaneously satisfy both of the envelope constraints defined later constitute A. safe Set: A safe ={a∈A|ρ(b(s)} t ),a,E col )∧ρ(b(s t ),a,E risk )}

[0073] In the formula, a is the acceleration selected from the action space, and E col and E risk For the two types of safety envelopes defined later, ρ(·) indicates whether the safety envelope constraint is satisfied. And A\A safe This is an unsafe action. If A safe (b) If it is an empty set, then the "fault-safe" action is executed, and only one option can be selected. The action in question is immediate emergency braking. Therefore, the agent selects the optimal action from a variable space of safe actions:

[0074]

[0075] In the formula, π safe V(b) refers to the strategy for outputting safe actions, and V(b) refers to the value function.

[0076] The transition function represents the probability of transitioning from one state to another, and is used to update the state at time t to the state at the next time t+1. This is based on the selected action (i.e., acceleration / deceleration) a. t The transfer function of the following vehicle is defined as:

[0077]

[0078] In the formula, f(·) represents the train kinematic model, which uses the mathematical relationship between position, velocity, and acceleration to describe the train's motion:

[0079]

[0080] In the formula, Δt represents the time step between each decision step.

[0081] The transfer function of the preceding vehicle is independent of the selected action and is defined as:

[0082]

[0083] In the formula, w represents the system noise, assumed to be zero-mean Gaussian white noise, and the process noise matrix is ​​Q. The Gaussian distribution has the advantages of wide applicability and ease of computation. Similar to the representation of the following vehicle, f(·) also represents the train kinematic model, but according to m... t There are some differences. Because in reality, the vehicle in front cannot maintain the same kinematic model, we used two different kinematic models to simulate the motion of the vehicle in front: constant velocity (CV) and constant acceleration (CA). These two kinematic models can describe various behaviors, including traction, braking, and maintaining speed.

[0084] When the current vehicle follows the CV model, i.e., the acceleration is 0, denoted as m t =1, its kinematic model Expressed as:

[0085]

[0086] When the current vehicle follows the CA model, it is denoted as m. t =2, and its kinematic model is the same as that of the following vehicle, represented as:

[0087]

[0088] Assuming the following vehicle has perfect awareness of its own state, but only partial awareness of the state (position and speed) of the preceding vehicle, the observation at time t is defined as follows:

[0089]

[0090] Where, and The observations of the following vehicle and the preceding vehicle are respectively defined as follows:

[0091]

[0092] In the formula, y and These are the measured longitudinal position and velocity of the vehicle in front. Additionally, the acceleration / deceleration of the vehicle in front... It is unobservable and therefore not included in the observations.

[0093] Due to measurement noise, the measured values ​​are inaccurate. Therefore, a Gaussian distribution is used to model the sensor measurements, represented by two parameters: mean and covariance. The observation function is defined as:

[0094]

[0095] In the formula, v represents the observation noise, assumed to be a Gaussian distribution with zero mean and zero correlation to the process noise w, and g(·) represents the function that associates the state with the observation. Due to measurement independence, the covariance of v is represented by a diagonal matrix R, and is given by the standard deviation of the position measurements σ. p and speed measurement standard deviation σ v describe.

[0096] The reward function is defined as follows: Since the following vehicles in a virtual formation must balance multiple conflicting objectives, the reward function only considers speed limits, efficiency, comfort, and energy saving factors, while ensuring safety through collision envelopes and risk envelopes. Specifically, since the reward function only considers speed limits, efficiency, comfort, and energy saving factors, it is defined as follows:

[0097]

[0098] In the formula, w1, w2, w3, and w4 are the weights corresponding to speed limit, efficiency, comfort, and energy saving factors, respectively. The rewards related to speed limit, efficiency, comfort, and energy saving factors at time t are defined as follows:

[0099]

[0100]

[0101] In the formula, This represents the speed limit at time t, where ι < 0 is a large constant used to penalize speeding. j represents the maximum spacing between the front and rear train formations. max χ is the maximum value of the impact rate (jerk), and χ < 0 is a constant used to punish excessively high impact rates.

[0102] In further embodiments of the present invention, such as Figure 3 As shown, the IMM algorithm in step S3 includes the following sub-steps:

[0103] For the following vehicle (i.e., the vehicle itself), since its own state is perfectly known, there is no need to calculate its belief. However, the state of the preceding vehicle cannot be directly observed; the preceding vehicle's belief state represents its state s at time t. t The probability distribution of the given information is assumed to follow a Gaussian distribution:

[0104]

[0105] Where, and and represent the mean and covariance of the state estimate of the preceding vehicle at time t, respectively. Since the state of the following vehicle is known, step S3 only discusses the belief state of the preceding vehicle. For simplicity, s will be referred to as s here. p The subscript is omitted and written as s.

[0106] In this invention, step S3 uses the IMM method to update beliefs. Because the IMM algorithm ignores the mutual influence between multiple agents, it has limitations in predicting traffic participants (such as other cars, bicycles, pedestrians, etc.) around the vehicle. However, since each train in a platoon only needs to consider a single traffic participant in its vicinity, namely the specific train in front of it (i.e., the train in front affects the train behind, but the train behind does not affect the train in front), the IMM algorithm is suitable for platooning.

[0107] After receiving the observation state of the preceding vehicle at the next moment, the belief state of the preceding vehicle is updated using the IMM algorithm described in step S3. In this invention, the belief update using the IMM method in step S3 includes the following steps: state interaction, model conditional filtering, model probability update, and state estimation fusion. The steps of state interaction, model conditional filtering, model probability update, and state estimation fusion are described in detail below.

[0108] Step S31: State Interaction

[0109] In this step, the model state estimate and covariance at time t are combined using the following two equations and then used as input to each filter model:

[0110]

[0111] In the formula, since only two kinematic models are considered, the superscript number of the symbol ∑ is 2; It is the conditional model probability, and its calculation formula is:

[0112]

[0113] In the formula, p ij is an element of the state transition matrix Π, representing the prior probability of switching from model i to model j. This is the predicted probability of model j at time t, and its calculation formula is:

[0114]

[0115] Step S32: Model Conditional Filtering

[0116] In this step, a Kalman filter is used for each kinematic model, and As input, the corresponding updated model state estimates and covariance are obtained. Kalman filtering begins with the prediction step, using the transition function of model i to derive the predicted probability distribution, whose state estimates and covariance are calculated as follows:

[0117]

[0118] Next is the update step. For model i, the updated belief is obtained using the predicted probability distribution and the current observations:

[0119]

[0120] in, Let I represent the Kalman gain of model i at time t+1, and let I represent the identity matrix.

[0121] Step S33: Model Probability Update

[0122] Assume the residuals of model j have a mean of 0 and a covariance of . If the distribution is Gaussian, then the likelihood value is calculated as follows:

[0123]

[0124] In the formula, This is the residual estimate for model j at time t+1. and The results are obtained from the following two formulas:

[0125]

[0126] After both filters have completed their update steps, the model probability of model j is updated as follows:

[0127]

[0128] Step S34: State estimation fusion

[0129] After obtaining the updated beliefs and model probabilities for each model, the fused state estimates and covariance are calculated as follows:

[0130]

[0131] The belief state of the preceding vehicle in the POMDP model is then updated to:

[0132]

[0133] Based on the probabilities obtained earlier that the preceding vehicle follows two different kinematic models, the preceding vehicle's belief about the internal kinematic model is expressed as:

[0134]

[0135] In further embodiments of the present invention, such as Figure 4-6 As shown, step S4, which involves calculating the worst-case collision envelope and risk envelope based on the train safety braking model and the actual braking process, establishing opportunity constraints based on the collision envelope and risk envelope, and constructing the CC-POMDP model, includes the following steps:

[0136] S41: Calculate the optimal tracking distance

[0137] Based on the train relative braking distance model and the actual braking process, the optimal tracking distance is defined as:

[0138] L = max(d brake,f -d brake,p +d safety ,d safety )

[0139] Where, d safety This represents a safety margin, used to ensure the minimum distance under any conditions (including when both vehicles are stationary); d brake Braking distance is a function of speed. The braking distance of the vehicle in front is defined as:

[0140]

[0141] In the formula, a max,p This indicates the maximum deceleration of the train itself under emergency braking conditions. This parameter is determined by the train's own characteristics.

[0142] The rear vehicle braking curve consists of five stages: onboard CBTC response stage, traction system disconnection stage, coasting stage, brake establishment stage, and brake execution stage. This can be simplified to three stages (e.g., Figure 4 (As shown): The traction cut-off phase combines the signal system response phase and the traction cut-off phase; the train coasting phase combines the coasting phase and the brake establishment phase; and the final phase is still the brake execution phase. Therefore, the braking distance of the following train is calculated as follows:

[0143] d brake,f =d A +d B +dC

[0144] Where, d A d B and d C These represent the braking distances during the traction cut-off phase, train coasting phase, and braking execution phase, respectively. The first phase is the traction cut-off phase, which is the period from receiving the braking command to the traction system being cut off. Due to the delay in the train system's response under the most unfavorable conditions, the train does not immediately apply brakes but continues to accelerate until traction is cut off; this can be approximated as a uniform acceleration process. The second phase is the train coasting phase, which refers to the period from traction system cut-off to the completion of brake establishment; similarly, for simplicity, this is approximated as a uniform speed process. The third phase is the braking execution phase, which refers to the period from the completion of brake establishment to the final stop of the train; this is approximated as a uniform deceleration process with the most stringent braking rate. The train travel distances for these three phases are calculated as follows:

[0145]

[0146] Where, v A v B and v C The initial velocities for the three stages are a, a, and a, respectively. A a B and a C For the initial acceleration in the three stages, t resp The response time for train traction cutoff is represented by t. cst a represents the coasting time required from the moment the train's traction is cut off to the moment the brakes are applied. C This is the maximum deceleration during train braking. During the train coasting phase, a... B Depending on the slope and resistance at this moment, the calculation formula is:

[0147]

[0148] In the formula, c0, c1, and c2 are Davis constants in N / kN, and g is the acceleration due to gravity. The slope of the track.

[0149] S42: Calculate the worst-case collision envelope and risk envelope.

[0150] The relative braking distance model for trains can significantly shorten the safe distance; however, the calculated safe distance is still too large. To further reduce the distance between the two trains, such as... Figure 5 As shown, this paper defines two safety envelopes based on emergency braking distance and maximum service braking distance: collision envelope E col and risk envelope E risk , respectively represented as:

[0151]

[0152] Where, and Let t represent the collision envelope and risk envelope at time t, respectively, and d represent the distance from the rear of the vehicle in front. and Let represent the minimum safe distances that satisfy the collision envelope and risk envelope at time t, respectively, and calculate them as follows:

[0153]

[0154]

[0155] Where, d emer,brake,f and d serv,brake,f These refer to the emergency braking distance of the following vehicle and the maximum service braking distance, respectively. The difference between the two lies in the application of the term "a" in the preceding text. C The definition is that when the following vehicle brakes suddenly, a C The value is the value of EB; when the following vehicle is under maximum braking force, a C The value is the maximum braking value in A.

[0156] The limitation of the platooning safety braking model based on relative braking distance is that it only applies when the braking performance of the vehicles in front and behind is the same. If the braking performance of the vehicles in front and behind is different, i.e., d... A d B and d C When the values ​​are not exactly the same and the initial distance is small, to ensure safety throughout the braking process, it is not sufficient to guarantee that a collision will not occur only when both vehicles have come to a complete stop. For example... Figure 6 As shown, when the vehicle in front brakes suddenly, even if the vehicle behind maintains a certain safe distance margin (the value is equal to the optimal tracking distance L analyzed in step S41), a collision may still occur.

[0157] To solve this problem, based on step S41, according to the collision envelope E col and risk envelope E risk The definition of the worst-case safety envelope is to calculate the worst-case scenario. The worst-case scenario refers to the moment when the distance between the two vehicles (distance of the front vehicle minus distance of the rear vehicle) is the shortest during the entire braking process. As long as the worst-case scenario does not result in a collision, it can be guaranteed that the two vehicles will not collide at any point during the braking process.

[0158] Therefore, the worst-case distance is defined as: the minimum distance between the two vehicles during the entire braking process, assuming the current distance between the two vehicles is L, and both vehicles brake immediately from the current moment until they come to a stop. The formula is:

[0159]

[0160] In the formula, worst(·) refers to the function used to calculate the worst-case scenario. and L represents the distance traveled by the preceding and following vehicles at time i, respectively, calculated according to step S41. (·) This represents the minimum safe distance that satisfies the collision envelope or risk envelope. The value of i is taken throughout the entire braking process. stop This represents the time taken for both trains to come to a complete stop from this state by braking immediately. This value is related to the braking performance of the trains themselves. If the calculated worst-case distance is less than d... safety If the braking distance is greater than or equal to d, then the two vehicles cannot maintain a safe margin at all times during braking, and a collision may occur; safety A collision will not occur. Since the belief state of the vehicle in front follows a probability distribution, in order to define a safety envelope, The probability density function is denoted as random variables The κ quantile, whose values ​​satisfy:

[0161]

[0162] This paper defines two safety envelopes—the collision envelope E. col and risk envelope E risk —These both refer to the worst-case envelope, defined as follows:

[0163]

[0164]

[0165] When the worst-case spacing calculation result is less than d safety When the range of both envelopes increases, the worst-case spacing calculation result is compared with d. safety The difference in distance allows for braking processes to avoid collisions. When the minimum safe distance increases... At the same time, it can ensure that the minimum distance between the two vehicles during the entire braking process is not less than d. safety Furthermore, as can be seen from the above two equations, the difference between the collision envelope and the risk envelope is the difference between the maximum normal braking distance and the emergency braking distance of the following vehicle.

[0166] S43: Establish collision envelope constraints and risk envelope constraints, and construct the CC-POMDP model.

[0167] Establish opportunity constraints to limit the probability of violating the collision envelope constraint and the risk envelope constraint, respectively. When a following vehicle is within the collision envelope of the preceding vehicle, it is considered a violation of the collision envelope constraint. Because a collision is absolutely not allowed, the probability of violating the collision envelope is constrained to 0 for the entire time interval [0, T1] from start to finish.

[0168]

[0169] In the formula, T1 is the time when all trains stop.

[0170] This invention defines a sub-action space, called the safe action space, based on the initial action space, and calculates the safe action space through collision envelope constraints and risk envelope constraints. This invention quantitatively defines "risk" as the probability of violating safety constraints. When a following vehicle is within the risk envelope of the preceding vehicle, it is considered to have violated the risk envelope. The strategy π generated by the following vehicle should satisfy the expectation that violating the risk envelope within a certain time period [t, t+T2] is below a certain threshold. Therefore, the risk envelope constraint is defined as:

[0171]

[0172] In the formula, ò refers to the risk probability threshold, and T2 is the number of time steps considering the future. Because this formula limits the probability / expectation of constraint violation, it can also be called the safety of opportunity constraints.

[0173] By using collision envelope constraints and risk envelope constraints, and constructing a CC-POMDP model, the safe action space A is calculated. safe .

[0174] The goal of constructing this CC-POMDP model is to maximize the cumulative reward while ensuring no collisions occur and the risk is below a given threshold.

[0175] max π V π (b)

[0176]

[0177]

[0178] In further embodiments of the present invention, such as Figure 7-9 As shown, the POMCP-DS-PW algorithm for solving the optimal action of CC-POMDP in step S5 includes three modules: a POMCP module, a double-shield module, and a progressive widening module. This invention refers to this algorithm as POMCP with a double shield. The specific implementation modules include:

[0179] S51: POMCP module

[0180] As a popular online planning algorithm, POMCP uses the MCTS algorithm to represent history and then selects the optimal policy based on the UCB algorithm, and has been proven to converge to the optimal policy. This algorithm breaks the "curse of dimensionality" by sampling from the particle ensemble and the "curse of history" by sampling history using a black-box model. When calculating the optimal action of the POMCP model based on POMCP, the agent needs to consider a complete history. The history at time t is defined as a series of actions and observations, expressed as:

[0181] h t ={a 1 ,o 1 ,...,a t ,o t},

[0182] h t a a+1 ={a 1 ,o 1 ,...,a t ,o t ,a a+1},

[0183] h t+1 =h t a a+1 o a+1 ={a 1 ,o 1 ,...,a t ,o t ,a a+1 ,o a+1}

[0184] Belief state is defined as:

[0185] b(s t )=Pr(s t |h t )

[0186] The root node of the search tree is represented as:

[0187] T(h) =<N(h),V(h),B(h)>

[0188] In the formula, N(h) refers to the number of times history h has been visited, and V(h) estimates the total number of visits from h. t The initial simulation expected return, B(h), represents the belief estimate.

[0189] like Figure 7The pseudocode shown illustrates the algorithm's process. Starting with a search, within a predefined planning time (no more than the number of planning iterations n), the agent continuously performs a simulation process, beginning with the initial history h and the corresponding belief B(h), to construct and explore the search tree. Once the planning time is exhausted, the agent finds and executes the optimal action that maximizes the historical value V(ha) of the current belief B(h).

[0190] like Figure 8 The pseudocode shown illustrates a simulation process that begins with a state sampled from belief state B(h) and ends at a leaf node where a pre-simulation is performed. This state uses a large number of unweighted particle approximations and consists of a series of sampled actions and observations. When performing the simulation, the UCB algorithm is used to select actions:

[0191]

[0192] In the formula, the parameter c > 0 is used to balance exploration and exploitation, N(ha i (referring to history (h) and action (a)) i The number of times it is visited. Then, given a state and action, a subsequent state, observation, and reward are sampled from the black-box generative model G(s,a). Within a given threshold ò, when a leaf node is reached, for all actions a∈A, the search tree is expanded using the child nodes:

[0193] T(ha) = <N init (ha),V init (ha),B init (ha)>=<0,0,φ>

[0194] Using the pre-launch strategy π rollout The cumulative reward is estimated, and this process continues until all rehearsal steps are performed to evaluate candidate actions in the history.

[0195] S52: Dual Shield Module

[0196] Because there are unsafe actions in the original action space A, this invention uses a double shield, namely pre-pruning and free backtracking to expand the POMCP algorithm, to shield unsafe actions (i.e., A). safe (Apart from actions), to achieve safe online planning of POMDP, the algorithm is called POMCP-DS.

[0197] The pre-pruning steps include: at each time step, for each action a∈A, iteratively updating and calculating the total probability of violating the risk envelope. After MCTS, all actions that satisfy the risk envelope constraints are found, and the corresponding branches are pruned at the root node T(ha).

[0198] To find branches that satisfy / violate the risk envelope constraint, the root node of the search tree is given an attribute, represented as:

[0199] T(h) =<N(h),V(h),F(h),B(h)>

[0200] In the formula, F(h) is a newly added attribute used to estimate all values ​​from h. t The initial simulation expects to violate the safe envelope probability. Similarly, each child node of each node is updated as follows:

[0201] T(ha) =<N(ha),V(ha),F(ha),B(ha)>

[0202] In the formula, F(ha) is a newly added attribute, representing the violation of the risk envelope E. risk The expected probability. Similar to N(ha), V(ha), and B(ha), F(ha) is updated as the tree expands. It is updated using the following formula:

[0203]

[0204] In the formula, N total N represents the total time step of this iteration. risk This represents the total number of times the risk envelope constraint was violated during this iteration.

[0205] By pre-pruning, the resulting CC-POMDP solution strategy can ensure that risk constraints are not violated.

[0206] Because pre-pruning can only mask unsafe actions at the root node T(h), without considering safety at each step of the simulation, the value of V(ha) may be generated based on unsafe actions. This limitation makes pre-pruning unsuitable for collision envelope constraints, while free backtracking can check every action throughout the simulation and is therefore suitable for collision envelope constraints.

[0207] The steps of free backtracking include: during the simulation of the POMCP-DS algorithm, for history h, when action a is selected through the pre-playing strategy, the next state s′ is generated by the black-box model G(s,a), and it is checked whether the state satisfies the collision envelope constraint. If it is violated, the tree branch starting from node T(ha) is pruned, that is, action a is masked in node T(h).

[0208] Through free backtracking, the resulting CC-POMDP solution strategy can completely avoid actions that violate the collision envelope.

[0209] S53: Progressive Widening Module

[0210] Since the state space and observation space defined above are continuous, and POMCP cannot handle continuous state and observation spaces, this invention uses progressive widening to add sampled observations to the tree, controls the rate at which sampled observations are added to the tree, and extends the POMCP-DS algorithm to obtain the POMCP-DS-PW algorithm.

[0211] The progressive widening module uses two parameters, k and α, to define the criteria for exploring new states:

[0212] |C(ha)|>kN(ha) α

[0213] In the formula, |C(ha)| represents the number of children of node N(ha).

[0214] Although the progressive widening module avoids a discrete observation space, the parameter values ​​must be carefully selected and the rate at which sampled observations are added to the tree must be controlled in order to ensure that the branching factor of the search tree is easy to manage.

[0215] Example 1

[0216] In this embodiment, a train speed-distance curve with noise is given as the lead train in the train formation, and the speed limit information and gradient information of the line are given.

[0217] The following vehicle can perfectly perceive its own state, but can only observe the position and velocity information of the noisy preceding vehicle, and cannot observe the acceleration information of the preceding vehicle. Assume that at the initial time t=0, the preceding vehicle accelerates from rest, and the following vehicle is stationary and 10m away from the preceding vehicle. That is, the initial state and observations are represented as follows:

[0218]

[0219] Let the mean and variance of the Gaussian distribution of the initial belief state of the vehicle in front be:

[0220]

[0221] Let the initial belief distribution of the kinematic model inside the vehicle in front be:

[0222] b(m t＝0 )={Pr(m t＝0 =CV),Pr(m t＝0 =CA)}={0.5,0.5}

[0223] The movement space of the rear vehicle is set as follows:

[0224] A = {0.8 m / s} 2 0.4m / s 2 ,aCO -0.5m / s 2 -1.0m / s 2}

[0225] The optimal speed-distance curve of the following vehicle is solved step by step using the method proposed in this invention. Figure 10 The optimal motion solved for each step size (without considering the effect of drag on acceleration), Figure 11 This includes the known speed-distance curve of the preceding vehicle, the speed limit and gradient of the route, and the calculated speed-distance curve of the following vehicle.

[0226] It should be understood that the specific embodiments described above are for illustrative purposes only and are not intended to limit the scope of the invention. Therefore, any modifications, equivalent substitutions, or improvements made without departing from the spirit and scope of the invention should be included within its protection scope. Furthermore, the appended claims are intended to cover all variations and modifications falling within the scope and boundaries of the appended claims, or their equivalents.

Claims

1. A virtual queuing vehicle control method considering uncertainty, characterized in that, Includes the following steps: Step S1: Obtain station, train operation and line operation data information within the train formation line, and establish a POMDP model for train decision-making based on the data information; Step S2: Collect real-time status information of train operation, and determine whether the trains in front and behind are in a safe operating space based on the real-time status information. If they are in a safe operating space, the decision-making process ends; otherwise, continue to the following steps. Step S3: Update beliefs using the IMM method; Step S4: Based on the train safety braking model and the actual braking process, calculate the worst-case collision envelope and risk envelope, establish opportunity constraints based on the collision envelope and risk envelope, and construct the CC-POMDP model; Step S5: Use the POMCP-DS-PW algorithm to solve for the optimal action of the CC-POMDP model; Step S6: The following vehicle executes the decision according to the optimal action and jumps to step S2.

2. The virtual queuing vehicle control method considering uncertainty according to claim 1, characterized in that, The establishment of the POMDP model for train decision-making in step S1 includes the following steps: defining the state space, action space, transition function, observation space, observation function, and reward function of the POMDP model.

3. The virtual queuing vehicle control method considering uncertainty according to claim 2, characterized in that, The reward function step only considers speed limiting factors, efficiency factors, comfort factors, and energy-saving factors.

4. The virtual queuing vehicle control method considering uncertainty according to claim 1, characterized in that, The belief update using the IMM method in step S3 includes the following steps: state interaction, model conditional filtering, model probability update and state estimation fusion.

5. The virtual queuing vehicle control method considering uncertainty according to claim 1, characterized in that, The optimal action for solving the CC-POMDP model using the POMCP-DS-PW algorithm in step S5 is implemented based on the POMCP module, the double shield module, and the progressive widening module. The POMCP module is used to represent history using the MCTS algorithm and to select the optimal action based on the UCB algorithm. The dual shield module is used to shield unsafe actions by employing pre-pruning and free backtracking extended POMCP algorithm. The progressive widening module is used to add the sampled observation space to the tree to control the rate at which the adopted observations are added to the tree.

6. The virtual queuing vehicle control method considering uncertainty according to claim 5, characterized in that, The MCTS algorithm includes the following repeated steps: selection, expansion, simulation, and backtracking. The selection step starts from the root node of the tree and moves to the leaf node according to the tree strategy; The extension step is used to add leaf nodes to the tree; The simulation step begins by executing a pre-simulation strategy from a leaf node to determine the quality of that leaf node. The backtracking step returns from the leaf node to the root node, and the UCB algorithm is used to guide the root node to select the optimal action.

7. The virtual queuing vehicle control method considering uncertainty according to claim 5, characterized in that, The pre-pruning step in the dual-shield module includes: iteratively updating each action within each time step and calculating the total probability of violating the risk envelope; finding all actions that satisfy the risk envelope constraints after the MCTS algorithm; and pruning the corresponding branches at the root node.

8. The virtual queuing vehicle control method considering uncertainty according to claim 5, characterized in that, The free backtracking step in the dual shield module includes: during the MCTS algorithm process of the POMCP module, when an action is selected through the pre-play strategy, it is checked whether the generated next state satisfies the risk envelope constraint; if the risk envelope constraint is violated, the tree branch starting from that node is pruned, and the action is blocked in that node.

9. The virtual queuing vehicle control method considering uncertainty according to claim 1, characterized in that, Step S4: Based on the train safety braking model and the actual braking process, calculate the worst-case collision envelope and risk envelope, establish opportunity constraints based on the collision envelope and risk envelope, and construct the CC-POMDP model, including: Based on the train safety braking model and the actual braking process, and based on the emergency braking curve and the braking curve of the following vehicle, the optimal tracking distance is calculated. Calculate the worst-case collision envelope and risk envelope; the worst-case scenario is the moment when the two trains are at their minimum distance from the current moment until they come to a complete stop. Establish collision envelope constraints and risk envelope constraints, and construct the CC-POMDP model.

10. The virtual queuing vehicle control method considering uncertainty according to claim 1, characterized in that, When the safety action space is a non-empty set, the action that maximizes the value function is selected; when the safety action space is an empty set, it means there is no safe action, and the emergency braking action is executed directly.

Images