Low-carbon driving control method, device and equipment for smart rail vehicle and medium

Abstract

The present application relates to the technical field of urban rail transit and intelligent networked traffic control, and provides a low-carbon driving control method, device, equipment and medium for intelligent rail vehicles. The method combines a reference speed curve, a double-algorithm fusion mechanism, physical rewards, a safety set and an online updating strategy, so that the vehicle converges in the direction of reducing energy consumption, stabilizing acceleration and reducing safety risks under the conditions of meeting safety, comfort and construction, and the effect is jointly ensured by mathematical structure and physical constraints, and does not depend on empirical adjustment. Therefore, the present application can simultaneously meet the following three requirements: 1. online updating of the strategy without breaking the safety constraint; 2. updating of the reference speed curve parameters suitable for the multi-source disturbance characteristics of intelligent rail; and 3. maintaining the stability of energy saving, punctuality and safety performance in the long-term operation cycle.

Description

Technical Field

[0001] This invention relates to the field of urban rail transit and intelligent connected traffic control technology, and more specifically, to a low-carbon driving control method, device, equipment and medium for intelligent rail vehicles. Background Technology

[0002] Energy-efficient operation of intelligent rail transit vehicles typically relies on speed curve planning techniques. Existing rail transit research primarily employs speed curve optimization methods including: analytical-based trajectory segmentation solutions; numerical optimization methods based on dynamic constraints; and Model Predictive Control (MPC), which enables rolling predictions of future time domain conditions. These techniques are widely used in the railcar environment, and their core assumptions are stable track conditions, fixed speed limits, and predictable signal system timings. Therefore, they can calculate the energy-optimal operating speed curve offline based on a given track model.

[0003] However, in actual operational scenarios, the above methods have structural shortcomings. First, the offline speed curve cannot be updated in real time with changes in line conditions. When there are situations such as interference from vehicles ahead, speed limit adjustments, temporary construction, or changes in traffic signals, the original curve is prone to becoming invalid, and may even lead to excessive traction or unnecessary braking, resulting in a significant increase in energy consumption. Second, although model predictive control can perform online replanning to some extent, its computational load is limited by the control platform and relies on an accurate system model. When the actual environment is inconsistent with the model, its planning results still struggle to maintain stable energy-saving performance.

[0004] With the development of data-driven control methods, some studies have begun to explore the use of reinforcement learning (RL) techniques to learn energy-saving driving strategies for trains. Reinforcement learning, by constructing state, action, and reward structures, can explore energy-saving strategies in a simulation environment. However, existing reinforcement learning schemes mainly suffer from three quantifiable problems: first, online learning requires an extremely large number of samples, and the policy convergence speed is much slower than the rate of change in the real vehicle operating environment, making real-time updates difficult; second, during the exploration phase, unsafe actions such as large accelerations or delayed braking may occur, making them unsuitable for direct application to the vehicle control link; and third, when the actual track conditions change significantly, the strategy may degenerate or become unstable, leading to increased fluctuations in energy consumption and operating time. Therefore, existing reinforcement learning methods are essentially still unable to meet the requirements of "online updates, safety and controllability, and convergence within the real vehicle cycle."

[0005] Furthermore, most existing research is based on the dynamic characteristics and signal system structure of rail-based trains, while intelligent rail transit (IRT) systems have significantly different operating characteristics, making it difficult to directly apply traditional optimization and reinforcement learning methods. IRT vehicles operate on rubber tires, whose tire adhesion coefficient changes rapidly with weather and road conditions, resulting in significant fluctuations in braking distance and achievable acceleration. IRT lines typically share urban road systems, with intersection signal phases adjusting in real time according to traffic flow, and green light windows exhibiting continuous drift characteristics. Frequent and short road gradient changes significantly impact traction and regenerative braking energy consumption. Vehicles use batteries as their sole power source, and their SOC (State of Charge), voltage drop characteristics, and energy recovery efficiency dynamically change during operation. These factors collectively lead to: the feasible region of the speed curve itself changing in real time, making offline planning unable to maintain optimality; and control strategies must dynamically balance energy consumption, safety, and the real-time status of the line.

[0006] For the reasons mentioned above, existing energy-saving driving methods have the following common defects in intelligent rail transit operation scenarios: they cannot use historical energy consumption data and real-time line status to jointly drive strategy updates; they cannot perform online strategy iteration without violating safety constraints; and they cannot handle multi-source disturbances unique to intelligent rail transit, such as rubber-wheeled vehicle dynamics, intersection signal fluctuations, and battery performance changes. Summary of the Invention

[0007] To address the shortcomings of existing energy-saving driving methods in intelligent rail transit (IRT) operation scenarios, this invention provides a low-carbon driving control method, device, equipment, and medium for IRT vehicles. This method enables long-term online learning in real-time operation environments, real-time updates of speed curves, and simultaneous consideration of low carbon emissions, driving safety, and comfort.

[0008] This invention provides a low-carbon driving control method for intelligent rail transit vehicles, characterized by comprising: Constructing the status and actions of the intelligent rail transit vehicle's operating environment; Based on the stated state and action, a reward is constructed; The initial policy network is obtained by training the value function based on the reward; the value function is a state-action pair formed by states and actions. Construct a reference velocity curve and calculate the reference acceleration based on the reference velocity curve; Collect online data including status, actions, rewards and next status during the vehicle control cycle and form an experience sample; Based on the policy network and reference acceleration, a fused acceleration is obtained; wherein, a dual-algorithm fusion mechanism is implemented to update the policy network and update the parameters of the reference velocity curve through reinforcement learning; in the dual-algorithm fusion mechanism, empirical samples are used to update the policy network through reinforcement learning, and empirical samples are used to incrementally fit and update the parameters of the reference velocity curve. Real-time safety constraints are introduced based on the fused acceleration to generate the final acceleration; Based on energy consumption performance and operational strategies, adjust reward parameters and the learning rate in the value function and incremental fitting.

[0009] In a preferred embodiment, the state is defined as a discrete-time state vector composed of vehicle operating state quantities and network environment quantities; the elements of the discrete-time state vector include vehicle speed, acceleration, driving distance, road gradient, battery voltage, battery current, tire adhesion coefficient, distance to the vehicle in front, and signal phase encoding; the action is defined as candidate acceleration.

[0010] In a preferred embodiment, the reward is based on a trade-off between energy consumption, punctuality, safety, and comfort, according to corresponding reward parameters.

[0011] In a preferred embodiment, constructing the reference velocity curve and calculating the reference acceleration based on the reference velocity curve includes: Construct a reference velocity curve based on cubic spline basis functions; The reference acceleration is calculated using discrete difference based on the reference velocity curve.

[0012] In a preferred embodiment, obtaining the fused acceleration based on the policy network and the reference acceleration includes: Output candidate accelerations from the policy network; Calculate the difference between the candidate acceleration and the reference acceleration; Compare the difference with the preset threshold: when the difference does not exceed the preset threshold, the policy network takes precedence; when the difference exceeds the preset threshold, the reference velocity curve takes precedence, thus obtaining the fusion acceleration.

[0013] In a preferred embodiment, when incrementally fitting and updating the parameters of the reference velocity curve using empirical samples, three types of constraint checks are performed for each update, including: velocity continuity, speed limit constraint, and acceleration feasibility; if any constraint is violated, the parameters of the reference velocity curve are rolled back to the parameters before the update.

[0014] In a preferred embodiment, the real-time safety constraints include adhesion constraints, braking distance constraints, and impact constraints; The adhesion constraint is constructed based on the tire adhesion coefficient; The braking distance constraint is constructed based on vehicle speed and safe braking distance; The impact constraint is constructed based on the change in acceleration.

[0015] Secondly, the present invention provides a low-carbon driving control device for intelligent rail transit vehicles, comprising: The first processing unit is used to construct the status and actions of the intelligent rail vehicle's operating environment. The second processing unit is used to construct a reward based on the state and action; The third processing unit is used to train the value function based on the reward to obtain the initial policy network; the value function is a state-action pair formed by states and actions; The fourth processing unit is used to construct a reference velocity curve and calculate the reference acceleration based on the reference velocity curve; The fifth processing unit is used to collect online data, including status, action, reward and next status, within the vehicle control cycle and form an experience sample. The sixth processing unit is used to obtain the fused acceleration based on the policy network and the reference acceleration; wherein, it performs a dual-algorithm fusion mechanism of reinforcement learning policy network update and reference velocity curve parameter update; in the dual-algorithm fusion mechanism, empirical samples are used to update the policy network through reinforcement learning, and empirical samples are used to incrementally fit and update the parameters of the reference velocity curve. The seventh processing unit is used to introduce real-time safety constraints on the fused acceleration to generate the final acceleration; The eighth processing unit is used to adjust the reward parameters and the learning rate in the value function and incremental fitting based on energy consumption performance and operational strategies.

[0016] Thirdly, the present invention provides an electronic device, comprising: At least one processor; and a memory communicatively connected to said at least one processor; The memory stores instructions that can be executed by the at least one processor, and the at least one processor executes the instructions stored in the memory to perform the above-described method.

[0017] Fourthly, the present invention provides a computer-readable storage medium for storing instructions that, when executed, cause the above-described method to be implemented.

[0018] In summary, this invention, through a combination of reference speed curves, a dual-algorithm fusion mechanism, physical rewards, safety sets, and online update strategies, ensures that vehicles, while meeting safety, comfort, and construction conditions, will inevitably converge towards reduced energy consumption, stable acceleration, and reduced safety risks. This effect is guaranteed by both mathematical structure and physical constraints, without relying on empirical adjustments. Therefore, this invention simultaneously satisfies the following three requirements: first, it enables online strategy updates without violating safety constraints; second, it adapts to the multi-source disturbance characteristics of intelligent rail transit by updating reference speed curve parameters; and third, it maintains stability in energy saving, punctuality, and safety performance over long-term operation. This invention can be applied to energy consumption optimization and low-carbon operation scenarios for intelligent rail transit systems, urban rail transit, and similar guided electric vehicles. Attached Figure Description

[0019] Figure 1 A flowchart illustrating a low-carbon driving control method for intelligent rail transit vehicles, provided as an embodiment of the present invention.

[0020] Figure 2 This is a schematic diagram of a low-carbon driving control device for intelligent rail transit vehicles, provided as an embodiment of the present invention.

[0021] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0023] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0024] like Figure 1 As shown, this embodiment of the invention provides a low-carbon driving control method for intelligent rail transit vehicles, including: S101: Constructing the status and actions of the intelligent rail vehicle's operating environment.

[0025] The state is defined as a discrete-time state vector composed of the vehicle operating state variables and the network environment variables, expressed as:

[0026] in, The status of the operating environment for intelligent rail transit vehicles. For vehicle speed, For acceleration, For driving distance, For road slope, For battery voltage, For battery current, For tire adhesion coefficient, To maintain distance from the vehicle in front, This is for signal phase encoding.

[0027] The action is defined as a candidate acceleration (longitudinal), used to drive the vehicle's power execution link, and is expressed as:

[0028] in, Actions of the intelligent rail transit vehicle operating environment It is the field of real numbers.

[0029] Step S101 constructs the state and actions of the intelligent rail vehicle's operating environment, ensuring that both reinforcement learning and the reference speed curve use a unified vehicle physical state as input. Specifically, all intelligent rail characteristic quantities are incorporated into the control strategy, enabling it to adapt to road conditions, weather changes, and intersection signal adjustments. For example, when the adhesion coefficient decreases, the constraint set automatically shrinks, inevitably reducing the executed acceleration. This adaptability is driven by real-time state variables; by updating the state variables and the real-time safety set in real time, all control commands are bound to the actual vehicle state, thus possessing online adaptability that model predictive control cannot achieve.

[0030] S102: Based on the stated state and action, construct a reward. The reward, based on corresponding reward parameters, balances energy consumption, punctuality, safety, and comfort, and is expressed as follows:

[0031] in: Vehicle energy consumption, used to weigh energy consumption in the incentive program, is expressed as:

[0032] For vehicle control cycle; This is used to account for runtime deviation and to balance on-time performance in the reward calculation. To accelerate the process and balance comfort in the reward, it is represented as:

[0033] The cost of security constraints, used to weigh security in rewards, is expressed as:

[0034] For the current moment Vehicles need to reserve an effective braking distance for safe braking; The penalty function is calculated based on the safe braking distance. It can be a continuous or piecewise function. The calculation parameters of this function are determined by the track safety level and the vehicle braking capacity.

[0035] For vehicle energy consumption incentive parameters; This is a parameter for runtime deviation reward. For acceleration bonus parameters; The parameters are awarded as a reward for safety constraints.

[0036] Step S102 drives the policy update direction through physically measurable cost terms, ensuring that the policy will inevitably converge in the direction of reducing energy consumption, stabilizing acceleration, and reducing safety risks. Since energy consumption comes from real-time measured voltage-current data, the energy-saving effect is a "measurable-verifiable-repeatable" physical result, and there is no deviation caused by simulation errors.

[0037] S103: Based on the reward training value function, an initial policy network is obtained; the value function is a state-action pair formed by states and actions. The value function updated during training is expressed as:

[0038] in, For the updated value function, The value function before the update. The learning rate is used to update the value function. As a discount factor, For the next state The greatest value of it This is the next optional action, which is the assumed optimal action when updating the value function.

[0039] Step S103 constructs a simulation environment from historical data such as historical speed trajectories, energy consumption records, and network status. Using this historical data and simulation environment, the value function is trained offline, and the output is a strategy with initial energy-saving capabilities. This provides a feasible initial control baseline for online operation, making online updates more stable.

[0040] S104: Construct a reference velocity curve and calculate the reference acceleration based on the reference velocity curve.

[0041] In this embodiment of the invention, a reference velocity curve is constructed based on cubic spline basis functions, and is represented as follows:

[0042] in, The reference speed at the current moment. Parameters for the reference velocity curve, For the first The parameters of the spline basis functions For the first spline basis functions This represents the total number of spline basis functions.

[0043] The velocity reference curve, constructed from cubic spline basis functions, possesses the property of continuous first and second derivatives. Therefore, in each control cycle, the reference acceleration can be calculated using discrete difference based on the reference velocity curve, expressed as:

[0044] in, As a reference speed for the next moment, This is the reference acceleration at the current moment.

[0045] Optionally, the influence of track gradient on longitudinal dynamics can be incorporated into the calculation of reference acceleration to improve feasibility under gradient conditions. The gradient compensation term, which uses the equivalent acceleration caused by the gradient as the reference acceleration, can be expressed as:

[0046] in, This represents the slope angle at the current location. This is the acceleration due to gravity.

[0047] Therefore, the reference acceleration can be corrected to:

[0048] The above modifications are used to reduce the reference traction acceleration when going uphill and increase the reference braking demand when going downhill, so that the control derived from the reference speed curve is more in line with vehicle dynamics and energy-saving operation principles.

[0049] Step S104 outputs an executable, smooth initial velocity curve that satisfies dynamic constraints, providing a benchmark for online fusion.

[0050] S105: Collect status data during the vehicle control cycle. ,action ,award Next state The online data constitutes an empirical sample; the empirical sample is represented as:

[0051] in, These are experience samples. The experience samples are added to the experience pool within the time window to support online reinforcement learning and speed curve updates of the policy network. This step S105 enables the policy network to continuously adapt to real-time line conditions, which is the basis of the online adaptive feature of this invention.

[0052] S106: The fused acceleration is obtained based on the policy network and the reference acceleration; wherein, a dual-algorithm fusion mechanism is implemented to perform reinforcement learning policy network update and reference velocity curve parameter update.

[0053] First, candidate accelerations are output from the policy network. ; Then calculate the candidate acceleration. Difference from reference acceleration:

[0054] Compare the difference with a preset threshold: when the difference At times, policy networks are the primary mode; when At that time, the reference velocity curve was used as the primary reference to obtain the fusion acceleration:

[0055] in, To accelerate integration, For the preset threshold, For scale parameters, To integrate the weighting coefficients, which are used for the weighted synthesis of two control methods—the policy network and the reference velocity curve—at the same time, .

[0056] In the dual-algorithm fusion mechanism, empirical samples are used for reinforcement learning to update the policy network. Reinforcement learning can be evaluated using cumulative reward-based assessment. Empirical samples are also used for incremental fitting to update the parameters of the reference velocity curve, expressed as:

[0057]

[0058] in, For the updated parameters, The parameters before the update. The learning rate is used to update the parameters of the reference velocity curve during incremental fitting. Gradient operator with parameters; Let be the objective function for fitting the reference velocity curve, representing the expression for the current parameters. The fitting error between the generated reference velocity curve and historical sample velocity data; For the first The measured speed of a historical sample should be a reference speed curve representing the segment with the best energy consumption performance in history. For the first The reference speed corresponding to each historical sample For parameters The prior vector represents the reference parameters that we want to refer to during fitting. These parameters can be derived from offline historical optimal parameters, vehicle calibration speed curves, or parameters that have been verified to be feasible in the previous control cycle. The vector norm is, in this embodiment of the invention, the L2 norm, and thus... It is equivalent to the sum of squares of the deviations of each parameter from the prior, and is used to penalize the magnitude of parameter changes; This is the regularization coefficient, used to adjust the weight ratio of the fitting error term and the prior constraint term, to avoid overfitting or excessive oscillation of parameters.

[0059] Furthermore, the strategy network and reference velocity curve are updated only using empirical samples that meet the requirements for cumulative return and risk measurement. Specifically: By station intervals or sliding mileage windows For the evaluation period, the cumulative return of the empirical sample during that evaluation period is defined as:

[0060] in, For instant rewards, As a discount factor, This is the starting index for the evaluation period. The cumulative return... Used to uniformly measure the overall effect of energy consumption, punctuality, comfort, and safety costs.

[0061] The empirical sample risk metric for this evaluation period is defined as follows:

[0062] Constrained by safety costs.

[0063] Preset cumulative return threshold and risk measurement threshold ,when and If the experience samples from the evaluation period are used to update the policy network and the reference velocity curve, then the experience samples from the evaluation period can still be used to update the policy network, but not to update the parameters of the reference velocity curve, in order to avoid polluting the reference velocity curve with bad exploration trajectories and to ensure that the reference velocity curve maintains an executable and stable global trend in the long term.

[0064] Furthermore, when incrementally fitting and updating the parameters of the reference velocity curve using empirical samples, three types of constraint checks are performed for each update: velocity continuity check, velocity limit constraint check, and acceleration feasibility check. If any constraint is violated, the parameters of the reference velocity curve are rolled back to the parameters before the update. .

[0065] The velocity continuity check refers to the absence of abrupt velocity changes in the updated reference velocity curve, meaning the reference velocity curve is continuous and its first derivative is continuous. In a discrete implementation, this can be equivalent to checking that the velocity increments of adjacent discrete sampling points satisfy the following:

[0066] in, The velocity at the (n+1)th discrete sampling point in the updated reference velocity curve. For the velocity of the nth discrete sampling point in the updated reference velocity curve, The threshold is calculated from the sampling interval and the upper limit of the allowable acceleration, and is used to avoid unreasonable velocity jumps in the curve.

[0067] The speed limit constraint check refers to the updated reference speed curve where each reference speed does not exceed the speed limit and a margin is retained, expressed as:

[0068] in, For the velocity of the nth discrete sampling point in the updated reference velocity curve, To limit the line speed corresponding to the nth discrete sampling point in the updated reference speed curve, This is a speed limit margin used to account for speed measurement errors, execution errors, and boundary robustness.

[0069] The acceleration feasibility check refers to calculating the reference acceleration based on the updated reference velocity curve and verifying that the calculated reference acceleration is within the allowable range of vehicle dynamics and comfort. An example of calculating the reference acceleration is as follows:

[0070] in, The reference acceleration is from the nth discrete sampling point in the updated reference velocity curve to the (n+1)th discrete sampling point in the updated reference velocity curve; The reference acceleration must satisfy the traction and braking capacity boundaries as well as the adhesion boundaries, for example:

[0071] in, For reference, the upper limit of acceleration can be determined by the adhesion coefficient and traction limit. The lower limit of acceleration can be determined by braking capacity and safety margin.

[0072] In addition, if necessary, the magnitude of the reference acceleration change can also be checked to constrain the impact intensity, for example:

[0073] in, The reference acceleration is from the (n-1)th discrete sampling point in the updated reference velocity curve to the nth discrete sampling point in the updated reference velocity curve; This is the preset reference acceleration change threshold.

[0074] Step S106, through the fusion of acceleration and dual-algorithm fusion mechanism, ensures that reinforcement learning cannot generate action sequences that deviate from the reference velocity curve. This prevents common problems in pure reinforcement learning, such as trajectory divergence and sharp increases in impact, from occurring during the policy network training process, thus guaranteeing the stability of long-term online learning on real vehicles.

[0075] S107: Based on the fused acceleration, real-time safety constraints are introduced to generate the final acceleration.

[0076] In this embodiment of the invention, the final acceleration is determined by the projection operator and is expressed as:

[0077] in, For the final acceleration, Create a set of actions that include safety constraints. Set of Actions The acceleration within the vehicle is thus suppressed across the entire operating range, resulting in improved ride comfort—a direct consequence of the spline structure and decision structure.

[0078] In this embodiment of the invention, the security constraints in the set of possible actions include: (1) Adhesion constraint, based on tire adhesion coefficient Construct, represented as:

[0079] in, It is the acceleration due to gravity. The safety margin of the attachment constraint is used to further tighten the upper limit of acceleration under theoretical constraints, so as to avoid safety problems caused by other factors (such as road surface instability and control lag) in actual engineering.

[0080] (2) Braking distance constraint, based on vehicle speed and safe braking distance Construct, represented as:

[0081] in, This is a safety margin for braking distance constraints, used to avoid the risk of failing to brake or exceeding the limit due to calculation errors.

[0082] (3) Impact constraint, also known as comfort constraint, is constructed based on the change in acceleration and is expressed as:

[0083] in, The reference acceleration from the previous moment. The reference acceleration at the current moment. The upper limit of the impact constraint represents the maximum allowable rate of change of longitudinal acceleration, used to ensure ride comfort and suppress and control vibration.

[0084] Step S107 ensures that the generated motion fully complies with vehicle dynamics and safety requirements, meaning that the vehicle will never exhibit unsafe behaviors such as insufficient braking force, excessive adhesion, or sudden acceleration changes at any time. This safety is not due to parameter selection, but is necessarily guaranteed by the "closed, convex, and bounded" properties of the set of movable actions and the characteristics of the projection operator, thereby enabling the solution of this invention to be used in real vehicles.

[0085] S108: Based on energy consumption performance and operational strategies, such as periodically analyzing energy consumption per unit mile, punctuality, and passenger comfort indicators, adjust incentive parameters (including vehicle energy consumption incentive parameters). Running time deviation reward parameters acceleration bonus parameters Safety constraint cost reward parameters Learning rate of the value function and the learning rate of incremental fitting This enables the control strategy to adapt to different seasons, passenger flow changes, and operational strategy requirements, achieving long-term adaptive low-carbon performance evolution.

[0086] Application Example 1: This application example illustrates the execution process of steps S101 to S108 in the above method. For ease of explanation, the vehicle parameters, route conditions, sampling data, and test results in this application example are all hypothetical example values, used only to demonstrate the calculation process, and do not constitute a limitation on the scope of protection of this invention.

[0087] This application example assumes the following scenario: an intelligent rail transit vehicle operates within an urban area of ​​approximately 1.0 km in length, with the goal of accelerating, cruising, and decelerating from the starting station to the final destination. The control cycles involved are uniformly taken as... .

[0088] like Figure 1 As shown, a low-carbon driving control method for intelligent rail transit vehicles includes: S101: State and Actions of Constructing the Operating Environment for Intelligent Rail Transit Vehicles: Assuming in a certain control cycle At that time, the following status data was collected:

[0089]

[0090]

[0091] According to step S101, the state is... and actions Represented as:

[0092]

[0093] S102: Based on the stated state and action, construct the reward: Assume the energy consumption for this control cycle is:

[0094] Assuming the planned speed is Therefore, the running time deviation can be approximated by the speed deviation. For example, the running time deviation is taken as:

[0095] Assume the acceleration in the previous cycle is ; Then the jerk is ; Assuming no safety constraints are triggered in this control cycle, the safety constraint cost term is taken as follows: .

[0096] In the example, the weighting coefficient is taken as... .

[0097] According to step S102, the reward is represented as follows:

[0098] The reward for substituting the example value is approximately:

[0099] S103: Based on the reward training value function, an initial policy network is obtained; the value function is a state-action pair formed by states and actions: Assume the current value function is in the state-action pair The old value on

[0100] Assuming discount factor Learning rate .

[0101] Assuming in the next state The highest value is

[0102] Then, according to the value function update formula in step S103, we have:

[0103] Substitute the example value

[0104]

[0105]

[0106]

[0107] This completes an exemplary offline or online value function update.

[0108] S104: Construct a reference velocity curve and calculate the reference acceleration based on the reference velocity curve: Assuming in the interval superior, use spline basis functions

[0109] Example parameters are

[0110] At the current location At this point, assume the spline basis function takes the following values:

[0111] The reference speed is:

[0112] To calculate the reference velocity at the next moment, assume that... At this point, the corresponding spline basis function takes the following values:

[0113] The reference velocity at the next moment is:

[0114] Therefore, the reference acceleration is obtained:

[0115] S105: Collect online data including status, action, reward, and next status within the vehicle control cycle and construct an experience sample: After the acceleration of the previous control cycle is completed in this control cycle, record the empirical sample:

[0116] The experience samples are stored in the experience pool for incremental updates in subsequent steps S103 and S106.

[0117] S106: A fused acceleration is obtained based on the policy network and the reference acceleration; wherein, a dual-algorithm fusion mechanism is implemented to perform reinforcement learning policy network updates and reference velocity curve parameter updates: Suppose that the reinforcement learning policy network at this point depends on the current state. Output candidate acceleration:

[0118] The reference acceleration calculated in step S104 is:

[0119] The difference between the two is:

[0120] Preset threshold Scale parameters Then, according to step S106, the fusion weight coefficients are:

[0121] Therefore, the fusion acceleration is:

[0122] In this application example, due to the large difference, the reference speed curve completely dominates the action.

[0123] Meanwhile, assuming this application example updates the spline parameters using a simple gradient descent approach, the gradient of the parameters with respect to a sample during a certain running segment is approximated as follows:

[0124] Learning rate According to step S106, we have:

[0125] S107: Introducing real-time safety constraints based on the fused acceleration to generate the final acceleration: Assume the tire adhesion coefficient for this control cycle is:

[0126] Assuming the safety margin of the attachment constraints The maximum permissible deceleration is approximately:

[0127] Assume the impact constraint is:

[0128] And assume the final acceleration of the previous control cycle for .

[0129] At this point, the fusion acceleration is:

[0130] The change in acceleration is:

[0131] and

[0132] Therefore, this action satisfies the safety set. According to step S107, the final acceleration is:

[0133] S108: Adjust reward parameters (including vehicle energy consumption reward parameters) based on energy consumption performance and operational strategies. Running time deviation reward parameters acceleration bonus parameters Safety constraint cost reward parameters Learning rate of the value function and the learning rate of incremental fitting .

[0134] Assuming a distance of 1.0 km, the example value of the unit energy consumption of the method of this invention is statistically obtained as follows:

[0135] The energy consumption of the basic strategy (fixed velocity curve only, without reinforcement learning) is as follows:

[0136] The difference between the two is:

[0137] An example of energy saving rate is:

[0138] If the operator subsequently increases the weight of the "energy-saving target," then the vehicle energy consumption reward parameters in the reward can be adjusted according to S108. Slightly increase, for example, from Adjusted to At the same time, appropriately reduce the learning rate of the value function. This reduces the update magnitude of the policy network, aiming to maintain smooth convergence while prioritizing energy efficiency.

[0139] Application Example 2: This application example assumes an adaptive energy-saving driving scenario under high-gradient road conditions, illustrating the operation mode of the present invention on intelligent rail transit lines with significant gradient changes. For ease of explanation, the vehicle parameters, gradient conditions, and calculation results in this application example are all hypothetical example values, used only to demonstrate the specific application of steps S101 to S108 in the above method under high-gradient conditions.

[0140] According to step S101, it is assumed that the sampled data of the vehicle in a certain control cycle includes: Current location:

[0141] speed:

[0142] Road gradient:

[0143] According to step S104, refer to the velocity curve parameters. With spline basis functions It has already been initialized in Application Example 1. This application example only shows the slope compensation part.

[0144] Assuming the base reference velocity is at position The discrete difference calculation results are as follows:

[0145] According to the slope compensation term in step S104, the equivalent acceleration caused by the slope is:

[0146] The reference acceleration after slope compensation for this control cycle is:

[0147] According to step S106, assume that the reinforcement learning policy network outputs candidate accelerations in this state:

[0148] The difference between the two:

[0149] The preset threshold is:

[0150] Therefore, the difference exceeds the preset threshold, and the reference velocity curve completely dominates the action, resulting in the following fused acceleration:

[0151] According to step S107, assume the tire adhesion coefficient for this control cycle is:

[0152] The maximum permissible acceleration is:

[0153] because Therefore, the final acceleration is:

[0154] Based on step S108, assuming the energy consumption of the high-slope section (1.5km) is statistically analyzed, the following example results are obtained: Traditional strategy:

[0155] Method of the present invention:

[0156] Energy saving rate:

[0157] This application example demonstrates that the present invention, through the slope compensation mechanism in step S104 and the dual-algorithm fusion mechanism in step S106, can maintain stable traction in large slope sections and reduce unnecessary energy consumption, achieving the dual effects of energy saving and stability.

[0158] Application Example 3: This application example assumes a scenario of stable driving control under low adhesion (rain, snow, or slippery conditions) to illustrate the application of the present invention in low adhesion, high slip risk conditions. All adhesion coefficients, acceleration values, and calculation results are hypothetical example values, used only to demonstrate the safety control characteristics of steps S101 to S108 in the above method under low adhesion conditions.

[0159] According to step S101, assuming the adhesion coefficient collected by the vehicle on the slippery road section is:

[0160] Assume the current vehicle speed is:

[0161] According to step S104, the corresponding reference acceleration is:

[0162] According to step S106, assuming the reinforcement learning policy network is exploring energy-saving actions, the output candidate acceleration is:

[0163] To avoid uncontrollable outputs in reinforcement learning under low attachment, the actions are converged as follows: Assume the fusion acceleration is:

[0164] According to step S107, the maximum allowable acceleration for this control cycle is:

[0165] because Therefore, the safety filter receives fusion acceleration, and the final acceleration is:

[0166] The actions of this control cycle are limited to a controllable area on a slippery surface, effectively reducing the risk of skidding.

[0167] According to step S108, assuming the statistical data for a 200km example run in slippery weather is as follows: Traditional emergency braking interventions typically require approximately 33 interventions. The method of this invention reduces the number of steps to 16. The percentage decrease is:

[0168] This application example demonstrates that, through action fusion in step S106 and dynamic attachment constraints in step S107, the present invention can maintain the longitudinal stability of the vehicle under low-attachment conditions and avoid instability caused by reinforcement learning exploration behavior.

[0169] Based on the same technological concept, such as Figure 2 As shown, this embodiment of the invention also provides a low-carbon driving control device for intelligent rail vehicles, comprising: The first processing unit is used to construct the status and actions of the intelligent rail vehicle's operating environment. The second processing unit is used to construct a reward based on the state and action; The third processing unit is used to train the value function based on the reward to obtain the initial policy network; the value function is a state-action pair formed by states and actions; The fourth processing unit is used to construct a reference velocity curve and calculate the reference acceleration based on the reference velocity curve; The fifth processing unit is used to collect online data, including status, action, reward and next status, within the vehicle control cycle and form an experience sample. The sixth processing unit is used to obtain the fused acceleration based on the policy network and the reference acceleration; wherein, it performs a dual-algorithm fusion mechanism of reinforcement learning policy network update and reference velocity curve parameter update; in the dual-algorithm fusion mechanism, empirical samples are used to update the policy network through reinforcement learning, and empirical samples are used to incrementally fit and update the parameters of the reference velocity curve. The seventh processing unit is used to introduce real-time safety constraints on the fused acceleration to generate the final acceleration; The eighth processing unit is used to adjust the reward parameters and the learning rate in the value function and incremental fitting based on energy consumption performance and operational strategies.

[0170] The working principle of each processing unit in the above-mentioned device can be referred to the description in the foregoing method embodiments, and will not be repeated here.

[0171] Based on the same technical concept, embodiments of the present invention also provide an electronic device that can implement the low-carbon driving control method flow for intelligent rail vehicles provided in the above embodiments of the present invention. In one embodiment, the electronic device can be a server, a terminal device, or other electronic devices. Figure 3 As shown, the electronic device may include: At least one processor and a memory connected to the at least one processor. In this embodiment of the invention, the specific connection medium between the processor and the memory is not limited. Figure 3 The example used is the connection between the processor and memory via a bus. The bus... Figure 3The connections between other components are indicated by thick lines and are for illustrative purposes only, not as limiting information. Buses can be divided into address buses, data buses, control buses, etc., but for ease of representation, [the specific bus type is not shown here]. Figure 3 The processor is represented by a single thick line, but this does not imply that there is only one bus or one type of bus. Alternatively, a processor can also be called a controller; there are no restrictions on the name.

[0172] In this embodiment of the invention, the memory stores instructions that can be executed by at least one processor. By executing the instructions stored in the memory, at least one processor can execute a low-carbon driving control method for intelligent rail vehicles as described above.

[0173] The processor is the control center of the device. It can connect to various parts of the control device through various interfaces and lines. By running or executing instructions stored in memory and calling data stored in memory, it can monitor the device's various functions and process data, thereby enabling overall monitoring of the device.

[0174] In an alternative design, the processor may include one or more processing units. The processor may integrate an application processor and a modem processor, wherein the application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may also not be integrated into the processor. In some embodiments, the processor and memory may be implemented on the same chip; in some embodiments, they may also be implemented separately on separate chips.

[0175] The processor can be a general-purpose processor, such as a CPU, digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the low-carbon driving control method for intelligent rail vehicles disclosed in the embodiments of this invention can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0176] Memory, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory can include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. Memory is any other medium capable of carrying or storing desired program code in the form of instructions or data structures, and accessible by a computer, but is not limited thereto. In embodiments of the present invention, memory can also be a circuit or any other device capable of implementing storage functions, used to store program instructions and / or data.

[0177] By designing and programming the processor, the code corresponding to the low-carbon driving control method for intelligent rail vehicles described in the foregoing embodiments can be embedded into the chip, enabling the chip to execute the steps of the method described in the foregoing embodiments during runtime. How to design and program the processor is a technique well-known to those skilled in the art and will not be elaborated upon here.

[0178] Based on the same inventive concept, embodiments of the present invention also provide a storage medium storing computer instructions that, when executed on a computer, cause the computer to perform a low-carbon driving control method for intelligent rail transit vehicles as described above.

[0179] In some alternative embodiments, the present invention also provides a low-carbon driving control method for intelligent rail vehicles, which can also be implemented as a program product including program code. When the program product is run on a device, the program code is used to cause the control device to perform the steps in the low-carbon driving control method for intelligent rail vehicles described above according to various exemplary embodiments of the present invention.

[0180] It should be noted that although several units or sub-units of the apparatus have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the invention, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units. Furthermore, although the operation of the method of the invention is described in a specific order in the drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0181] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0182] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a server, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0183] Program code for performing the operations of this invention can be written using any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0184] In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0185] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0186] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0187] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A low-carbon driving control method for intelligent rail transit vehicles, characterized in that, include: Constructing the status and actions of the intelligent rail transit vehicle's operating environment; Based on the stated state and action, a reward is constructed; The initial policy network is obtained by training the value function based on the reward; the value function is a state-action pair formed by states and actions. Construct a reference velocity curve and calculate the reference acceleration based on the reference velocity curve; Collect online data including status, actions, rewards and next status during the vehicle control cycle and form an experience sample; A fused acceleration is obtained based on the policy network and reference acceleration; wherein a dual-algorithm fusion mechanism is implemented to update the policy network and update the parameters of the reference velocity curve using reinforcement learning; in the dual-algorithm fusion mechanism, empirical samples are used to update the policy network through reinforcement learning, and empirical samples are used to incrementally fit and update the parameters of the reference velocity curve; obtaining the fused acceleration based on the policy network and reference acceleration includes: outputting candidate accelerations from the policy network. ; Calculate candidate acceleration and reference acceleration The difference Comparing the difference with a preset threshold: when the difference does not exceed the preset threshold, the policy network takes precedence; when the difference exceeds the preset threshold, the reference velocity curve takes precedence, thus obtaining the fusion acceleration. ,in, To accelerate integration, For the preset threshold, For scale parameters, To integrate the weighting coefficients, which are used for the weighted synthesis of two control methods—the policy network and the reference velocity curve—at the same time, When using empirical samples to incrementally fit and update the parameters of the reference speed curve, three types of constraint checks are performed for each update, including speed continuity, speed limit constraints, and acceleration feasibility. If any constraint is violated, the parameters of the reference speed curve are rolled back to the parameters before the update. At the same time, the evaluation period is based on the inter-station interval or sliding mileage window. The cumulative return and risk metric of the evaluation period are used to determine whether the empirical samples can be used to update the reference speed curve. Empirical samples that meet the threshold are used to update the strategy network and the reference speed curve. Empirical samples that do not meet the threshold can still enter the experience pool for strategy network updates, but are not used to update the reference speed curve parameters. Based on the fused acceleration, real-time safety constraints are introduced to generate the final acceleration; the real-time safety constraints include adhesion constraints, braking distance constraints, and impact constraints; the adhesion constraints are constructed based on the tire adhesion coefficient; the braking distance constraints are constructed based on vehicle speed and safe braking distance; the impact constraints are constructed based on the acceleration change. Based on energy consumption performance and operational strategies, adjust reward parameters and the learning rate in the value function and incremental fitting.

2. The low-carbon driving control method for intelligent rail transit vehicles according to claim 1, characterized in that, The state is defined as a discrete-time state vector composed of the vehicle operating state quantity and the network environment quantity; the elements of the discrete-time state vector include vehicle speed, acceleration, driving distance, road gradient, battery voltage, battery current, tire adhesion coefficient, distance to the vehicle in front, and signal phase encoding; the action is defined as candidate acceleration.

3. The low-carbon driving control method for intelligent rail transit vehicles according to claim 2, characterized in that, The rewards are based on a balance of energy consumption, punctuality, safety, and comfort, according to the corresponding reward parameters.

4. The low-carbon driving control method for intelligent rail transit vehicles according to claim 1, characterized in that, The construction of the reference velocity curve and the calculation of the reference acceleration based on the reference velocity curve include: Construct a reference velocity curve based on cubic spline basis functions; The reference acceleration is calculated using discrete difference based on the reference velocity curve.

5. A low-carbon driving control device for intelligent rail transit vehicles, characterized in that, include: The first processing unit is used to construct the status and actions of the intelligent rail vehicle's operating environment. The second processing unit is used to construct a reward based on the state and action; The third processing unit is used to train the value function based on the reward to obtain the initial policy network; the value function is a state-action pair formed by states and actions; The fourth processing unit is used to construct a reference velocity curve and calculate the reference acceleration based on the reference velocity curve; The fifth processing unit is used to collect online data, including status, action, reward and next status, within the vehicle control cycle and form an experience sample. The sixth processing unit is used to obtain the fused acceleration based on the policy network and the reference acceleration; wherein, it executes a dual-algorithm fusion mechanism of reinforcement learning policy network update and reference velocity curve parameter update; in the dual-algorithm fusion mechanism, empirical samples are used to update the policy network through reinforcement learning, and empirical samples are used to incrementally fit and update the parameters of the reference velocity curve; obtaining the fused acceleration based on the policy network and the reference acceleration includes: outputting candidate accelerations from the policy network. ; Calculate the difference between the candidate acceleration and the reference acceleration. Comparing the difference with a preset threshold: when the difference does not exceed the preset threshold, the policy network takes precedence; when the difference exceeds the preset threshold, the reference velocity curve takes precedence, thus obtaining the fusion acceleration. ,in, To accelerate integration, For the preset threshold, For scale parameters, To integrate the weighting coefficients, which are used for the weighted synthesis of two control methods—the policy network and the reference velocity curve—at the same time, When using empirical samples to incrementally fit and update the parameters of the reference speed curve, three types of constraint checks are performed for each update, including speed continuity, speed limit constraints, and acceleration feasibility. If any constraint is violated, the parameters of the reference speed curve are rolled back to the parameters before the update. At the same time, the evaluation period is based on the inter-station interval or sliding mileage window. The cumulative return and risk metric of the evaluation period are used to determine whether the empirical samples can be used to update the reference speed curve. Empirical samples that meet the threshold are used to update the strategy network and the reference speed curve. Empirical samples that do not meet the threshold can still enter the experience pool for strategy network updates, but are not used to update the reference speed curve parameters. The seventh processing unit is used to introduce real-time safety constraints based on the fused acceleration to generate the final acceleration; the real-time safety constraints include adhesion constraints, braking distance constraints, and impact constraints; the adhesion constraints are constructed based on the tire adhesion coefficient; the braking distance constraints are constructed based on vehicle speed and safe braking distance; the impact constraints are constructed based on the acceleration change. The eighth processing unit is used to adjust the reward parameters and the learning rate in the value function and incremental fitting based on energy consumption performance and operational strategies.

6. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor. By executing the instructions stored in the memory, the at least one processor causes the at least one processor to perform the low-carbon driving control method for intelligent rail vehicles as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store instructions that, when executed, enable the low-carbon driving control method for intelligent rail vehicles as described in any one of claims 1-4 to be implemented.

Images