A Machine Learning-Based Energy Management Method for Range-Extended Rail Vehicles

Abstract

This invention discloses a machine learning-based energy management method for range-extended electric rail vehicles, belonging to the field of rail vehicle energy management technology. The method includes the following steps: Step 1, collecting data on travel speed, track gradient angle, battery state of charge, range extender output power, and position coordinates; Step 2, generating a power demand prediction sequence, and training a loss function with a physical constraint penalty term to constrain the power change rate; Step 3, generating target power sequences for the range extender and battery, and introducing the encoded feature vector of the predictive model encoder as auxiliary state information during the training of the agent's policy network to achieve end-to-end joint optimization; Step 4, using constraint-satisfying model predictive control, and with the battery state of charge dynamic equation, range extender power change rate constraint, and charge / discharge power limiting constraint as conditions, solving the optimization problem to obtain the corrected power command; Step 5, executing energy management control.

Description

Technical Field

[0001] This invention relates to the field of energy management technology for rail vehicles, and more particularly to an energy management method for range-extended rail vehicles based on machine learning. Background Technology

[0002] A range-extended electric rail vehicle is a type of rail vehicle that uses a power battery as its primary energy source and a range extender (a generator set consisting of an engine and a generator) as an auxiliary energy source. During operation, the range-extended electric rail vehicle needs to make power distribution decisions between the power battery and the range extender based on real-time operating conditions in order to maximize energy utilization efficiency and reduce operating costs.

[0003] Currently, energy management strategies for range-extended electric rail vehicles mainly fall into the following categories. The first category is a deterministic rule-based energy management strategy. This method triggers the start and stop of the range extender based on the threshold of the battery's state of charge (SBC). The range extender starts when the SBC falls below a preset lower threshold and shuts down when the SBC recovers to a preset upper threshold. This method has simple control logic and is easy to implement, but it lacks the ability to predict future operating conditions. The vehicle cannot store energy in advance before entering long uphill sections, leading to a sharp increase in fuel consumption and a decrease in power performance under low battery conditions. The second category is an instantaneous optimization-based energy management strategy, represented by the equivalent fuel consumption minimization strategy. This method solves for the optimal power allocation at the current moment in each control cycle. However, because it only considers the instantaneous optimum at the current moment, it lacks forward-looking information on future power demands and cannot achieve global energy efficiency optimization. The third category is energy management strategies based on global optimization, represented by dynamic programming algorithms. This method can obtain the global optimal solution offline, but it relies on complete prior information about the operating conditions and cannot be used for real-time online control. Furthermore, it requires recalculation offline when the operating route changes, resulting in poor adaptability.

[0004] In recent years, researchers have introduced deep reinforcement learning into the energy management field of hybrid vehicles. For example, some solutions use deep deterministic policy gradient algorithms combined with long short-term memory neural networks for vehicle speed prediction. The prediction results are then input into a reinforcement learning agent along with the current state, which generates control commands for power allocation. However, this approach has the following drawbacks: the deep learning model acts as a "black box" predictor, and its output predictions may violate the inherent physical constraints of the range-extended electric vehicle's power system. This leads to the reinforcement learning agent generating control commands based on infeasible predictions, severely impacting control quality. Another approach attempts to combine deep reinforcement learning agents with model predictive control frameworks (MMC) to improve the real-time performance of the policy using MMC's rolling optimization mechanism. However, in this combination, the reinforcement learning agent and the MMC solver are loosely coupled. Reinforcement learning merely acts as a reference trajectory generator, failing to embed physical constraints into the policy learning process. This results in insufficient generalization ability of the control policy when facing operating conditions outside the training data coverage, leading to significant performance degradation.

[0005] In summary, the core technical challenge facing existing technologies is how to balance real-time performance with the energy management strategy of range-extended electric rail vehicles. This strategy aims to achieve both predictive capabilities for future operating conditions to enable forward-looking decision-making, and ensure that the predicted results and final control commands satisfy the physical constraints of the power system, avoiding infeasible solutions that violate energy conservation. Furthermore, it seeks to demonstrate good generalization performance when facing unknown operating conditions outside the training data coverage. These three aspects constitute a mutually constraining technical contradiction: introducing predictive capabilities increases model complexity and introduces prediction errors; adding physical constraints limits the optimization solution space; and improving generalization performance requires the strategy to have sufficient robustness. Current technologies have not yet provided an effective solution that simultaneously resolves these contradictions. Summary of the Invention

[0006] To achieve the above objectives, this invention provides a machine learning-based energy management method for range-extended electric rail vehicles, comprising the following steps: Step 1: Collect the travel speed, track gradient angle, power battery state of charge, range extender output power, and track vehicle position coordinates of the range-extended railcar; Step 2: Input the current travel speed, the current track slope angle, and the current position coordinates of the railcar into the power demand prediction model. The power demand prediction model is a deep learning model with physical information embedded. The power demand prediction model outputs a power demand prediction sequence in the future prediction time domain. The training loss function of the power demand prediction model consists of the sum of a data fitting loss term and a physical constraint penalty term. The physical constraint penalty term is used to constrain the power change rate at any adjacent time in the power demand prediction sequence to not exceed the maximum power change rate constraint of the range-extended railcar. Step 3: Input the current state of charge of the power battery, the current output power of the range extender, and the power demand prediction sequence into the reinforcement learning energy management agent. The reinforcement learning energy management agent adopts a deep deterministic policy gradient algorithm and outputs the target power sequence of the range extender and the target power sequence of the power battery in the future prediction time domain. During the training process, the policy network of the reinforcement learning energy management agent uses the encoded feature vector output by the encoder of the power demand prediction model as auxiliary state information and inputs it into the policy network for end-to-end joint optimization. Step 4: Taking the target power value of the range extender at the first moment in the target power sequence of the range extender as a reference, under the constraints of the dynamic equation of the state of charge of the power battery of the range-extended rail vehicle, the power change rate constraint of the range extender, and the power charging and discharging power limit constraint, the optimization problem of minimizing the deviation between the actual output power of the range extender and the reference is solved by the constraint-satisfying model predictive control, so as to obtain the corrected output power command of the range extender and the corrected output power command of the power battery in the current control cycle. Step 5: Send the modified range extender output power command to the range extender controller, and send the modified power battery output power command to the battery management system to execute energy management control.

[0007] Preferably, the power demand prediction model in step 2 consists of an encoder and a decoder; the encoder employs a bidirectional long short-term memory network, which performs forward and reverse temporal encoding on the input time-series data to generate the encoded feature vector; the decoder employs a unidirectional long short-term memory network based on an attention mechanism, which uses an attention mechanism to weight the encoded feature vector and generates the power demand prediction sequence through recursive decoding; the calculation formula for the physical constraint penalty term in step 2 is:

[0008] in, For the physical constraint penalty term, Here, (H) is the preset penalty coefficient, and (H) is the number of prediction steps included in the future prediction time domain. This represents the predicted power demand value at the (k)th prediction step in the power demand prediction sequence. This represents the predicted power demand value at the (k+1)th prediction step in the power demand prediction sequence. The time interval between adjacent prediction steps. The maximum power change rate of the range-extended railcar, the The smaller of the maximum power change rate of the traction motor of the range-extended railcar and the maximum power change rate of the range extender is taken; the data fitting loss term is the mean square error between the predicted power demand sequence and the actual power demand sequence.

[0009] Preferably, the training process of the reinforcement learning energy management agent in step 3 adopts the following reward function:

[0010] in, Let (t) be the reward value at time (t). Let be the instantaneous fuel consumption rate at time (t). This instantaneous fuel consumption rate is obtained by querying the fuel consumption rate characteristic curve of the range extender based on the output power of the range extender at time (t). The absolute value of the offset of the state of charge of the power battery at time (t) relative to the preset target state of charge of the power battery. Let be the absolute value of the change in the output power of the range extender at time (t). This is the fuel economy weighting coefficient. Weighting coefficients for maintaining the state of charge. This is a weighting factor for the smoothness of power change. , and All are preset positive real numbers; before being input into the policy network, the encoded feature vector is dimensionally mapped through a feature transformation network, which is a fully connected neural network. The network parameters of the feature transformation network are synchronously updated with the network parameters of the policy network during the training of the reinforcement learning energy management agent; the input state of the policy network is composed of the current state of charge of the power battery, the current output power of the range extender, the power demand prediction sequence, and the encoded feature vector mapped by the feature transformation network; the output action of the policy network is the target power value of the range extender at the first moment in the target power sequence of the range extender.

[0011] Preferably, the objective function of the optimization problem for solving the constraint-satisfying model predictive control in step 4 is:

[0012] in, The corrected range extender output power command is given for the current control cycle. The target power value of the range extender is the value at the first moment in the target power sequence of the range extender; the dynamic equation of the state of charge of the power battery of the range-extended rail vehicle is:

[0013] in, This is a predicted value for the state of charge of the power battery in the next control cycle. This is the measured value of the state of charge of the power battery during the current control cycle. The output power of the power battery during the current control cycle. The charging and discharging efficiency function of the power battery is defined by the parameters of which are obtained by looking up a table based on the state of charge and output power of the power battery. This represents the maximum usable energy of the power battery. To control the duration of the cycle, the power change rate constraint of the range extender is:

[0014] in, This represents the actual output power of the range extender in the previous control cycle. The maximum allowable power variation of the range extender; the power battery charging and discharging power limiting constraint is:

[0015] in, This represents the maximum charging power of the power battery. The maximum discharge power of the power battery, the maximum charging power of the power battery and the maximum discharge power of the power battery are determined by the parameter table of the battery management system according to the state of charge of the power battery; the optimization problem is solved by the interior point method, and the solution time is no more than 50% of the length of the control cycle.

[0016] Preferably, the training data for the power demand prediction model is constructed as follows: continuous time series segments are extracted from the historical operation data of the range-extended electric vehicle (REEV). Each time series segment includes a vehicle speed sequence, a track gradient angle sequence, and a vehicle position coordinate sequence as input features, and a corresponding power demand sequence as a label. Each time series segment is divided into multiple training samples using a sliding window approach. The input window length of each training sample is a first preset number of steps, and the output window length of each training sample is the number of prediction steps (H) contained in the future prediction time domain. The power demand prediction model is trained using the Adam optimizer with an initial learning rate of 0.001, a batch size of 64, and at least 10,000 training iterations. The penalty coefficient... Initially set to 0, the penalty coefficient is gradually increased after the data fitting loss term converges to a first preset threshold. The power demand prediction model is set to a second preset threshold to ensure that it meets physical constraints while maintaining prediction accuracy.

[0017] Preferably, the feature transformation network comprises two fully connected hidden layers, with 256 neurons in the first fully connected hidden layer and 128 neurons in the second fully connected hidden layer, and the activation function is a rectified linear unit (RCU). The policy network comprises three fully connected hidden layers, with 512, 256, and 128 neurons in each layer, respectively, and the activation function is an RCU. The output layer activation function of the policy network is a hyperbolic tangent function. The reinforcement learning energy management agent further comprises an evaluation network, the network structure of which is similar to that of the feature transformation network. The policy network is the same, and the output of the evaluation network is a state-action value estimate. The training of the reinforcement learning energy management agent adopts an experience replay mechanism. The capacity of the experience replay buffer is 100,000 sets of experience samples. In each training session, 128 sets of experience samples are randomly selected from the experience replay buffer to update the policy network and the evaluation network. The soft update coefficient of the target network is 0.005. The exploration noise of the reinforcement learning energy management agent adopts the Ornstein-Uhlenbeck process. The initial noise standard deviation is 0.3, which gradually decreases during the training process.

[0018] Preferably, the predictive control time domain length of the optimization problem is equal to the number of prediction steps (H) contained in the future prediction time domain corresponding to the power demand prediction sequence; in the dynamic equation of the state of charge of the power battery, the maximum available energy of the power battery... The correction is made based on the health status of the power battery, and the correction formula is as follows: ,in This refers to the nominal maximum usable energy of the power battery. This is an estimate of the current state of health of the power battery, provided by the battery management system; when the range-extended rail vehicle is in regenerative braking mode, the charge / discharge efficiency function of the power battery is... The energy recovery efficiency value is obtained by looking up the regenerative braking efficiency table, which is independent of the discharge efficiency table, based on the state of charge of the power battery and the braking power. When solving the optimization problem, if the interior point method fails to converge to the preset accuracy within 50% of the control cycle time, the range extender output power command solved in the previous control cycle is used as the corrected range extender output power command for the current control cycle after being limited.

[0019] Preferably, in step 1, the travel speed of the range-extended railcar is acquired through an inertial measurement unit, the track slope angle is acquired through a slope sensor, the state of charge of the power battery is acquired through a battery management system, the output power of the range extender is acquired through a range extender controller, and the position coordinates of the railcar are acquired through a global navigation satellite positioning system receiver. In step 1, the acquired travel speed, track slope angle, power battery state of charge, range extender output power, and railcar position coordinates are time-stamp aligned to ensure all data corresponds to the same moment. All time-stamp aligned data is filtered using a Kalman filter, with the process noise covariance matrix and observation noise covariance matrix of the Kalman filter set according to the specifications of each sensor. Outlier detection is performed on all filtered data, removing values ​​exceeding the normal range of each physical quantity and filling them with the corresponding values ​​from the previous moment to generate the current travel speed, track slope angle, power battery state of charge, range extender output power, and railcar position coordinates for input in steps 2 and 3.

[0020] Preferably, the duration of the control cycle is... The value ranges from 50 milliseconds to 200 milliseconds, which is the time interval between adjacent prediction steps in the power demand prediction sequence. Equal to the time length of the control cycle The length of the future prediction time domain is 3 to 10 seconds, and the corresponding prediction step number (H) is the length of the future prediction time domain divided by the time interval. The power demand prediction model and the inference computation of the reinforcement learning energy management agent are completed in an in-vehicle embedded computing unit. The in-vehicle embedded computing unit has a graphics processor acceleration function. The single inference time of the power demand prediction model does not exceed 15 milliseconds, and the single inference time of the policy network of the reinforcement learning energy management agent does not exceed 10 milliseconds.

[0021] Preferably, after step 5 is completed, when a control cycle arrives, step 1 is returned to repeat steps 1 to 5 until the end signal of the extended-range railcar operation task is received; when the extended-range railcar detects that the positioning signal is lost during the cyclic execution, resulting in the railcar position coordinates being unavailable, the input of the power demand prediction model in step 2 is replaced by the railcar position coordinates recursively estimated by the inertial navigation system, and at the same time, the power demand prediction sequence output by the power demand prediction model is weighted and fused with a preset conservative power demand sequence. The weighting coefficient is dynamically adjusted according to the duration of the positioning signal loss. The longer the duration of the positioning signal loss, the greater the weight of the conservative power demand sequence; the conservative power demand sequence is the power demand sequence estimated based on the current slope and current driving speed and the vehicle's longitudinal dynamics model.

[0022] The beneficial effects of this invention are: 1. This invention constructs a collaborative architecture of three functional modules: a deep learning power demand prediction model with embedded physical information, a reinforcement learning energy management agent, and a constraint-satisfying model predictive control. This organically integrates forward-looking prediction, policy optimization decision-making, and physical constraint safety verification into a closed-loop control system. The power demand prediction model embeds physical constraint penalty terms into the training loss function, ensuring that the predicted sequence naturally satisfies the power change rate constraint of the power system, thus eliminating infeasible prediction results at the source. Based on this, the encoded feature vector generated by the prediction model encoder is used as auxiliary state information input to the reinforcement learning policy network, enabling the policy network to directly perceive the internal representation of the operating conditions during the training phase. End-to-end joint optimization unifies the objective functions of the prediction and decision-making stages, overcoming the error accumulation problem caused by the independent step-by-step optimization of prediction and decision-making in traditional schemes. Furthermore, the constraint-satisfying model predictive control performs deterministic constraint verification on the power commands output by the reinforcement learning, forming a two-layer safety closed loop of "agent suggestion plus controller verification," ensuring the absolute physical feasibility of the execution layer commands. The three-layer module forms a deep functional coupling by encoding feature vectors and constructing optimization problems. It achieves a synergistic improvement in three dimensions: prediction capability, optimization performance, and constraint satisfaction. This solves the problem of balancing comprehensive energy efficiency and safety, which cannot be achieved by implementing the three types of technical solutions individually.

[0023] 2. This invention introduces the encoded feature vector generated by the encoder of the power demand prediction model into the policy network input of the reinforcement learning energy management agent, and performs end-to-end joint optimization of the encoder, feature transformation network, and policy network during training. This allows the deep structural information of the operating conditions extracted by the encoder to directly serve the learning of the decision-making strategy. The encoded feature vector, as a dense representation of the original sensor data in a low-dimensional latent space, carries structural prior information about the evolution trend of the current operating conditions over a future period. Using this encoded feature vector as input, the policy network can establish a direct mapping relationship from the encoded feature space to the optimal power allocation action during training, without completely relying on the complete prediction sequence output by the power demand prediction model. This reduces the disturbance of the decision-making strategy caused by the deviation of individual prediction steps in the prediction sequence. When the range-extended electric vehicle enters an unknown operating condition outside the coverage of the training data, the structural features of the operating condition extracted by the encoder can still provide effective operating condition identification information for the policy network, enabling the policy to generate reasonable control decisions and overcoming the defect of the sharp degradation of decision-making performance of traditional reinforcement learning strategies under unknown operating conditions.

[0024] 3. In the safety verification and correction stage of the constraint-satisfying model predictive control, this invention uses the range extender target power value at the first moment of the range extender target power sequence output by the reinforcement learning energy management agent as a reference input. Under the joint constraints of the dynamic equation of the battery state of charge, the range extender power change rate constraint, and the battery charge / discharge power limiting constraint, the actual power command is obtained by solving a constrained optimization problem. The range extender power change rate constraint in this stage limits the maximum change in range extender output power between adjacent control cycles, and the battery charge / discharge power limiting constraint dynamically determines the maximum charge / discharge power boundary based on the real-time state of charge. Both of these constraints jointly ensure the transient feasibility and steady-state safety of the output command. The charge / discharge efficiency function in the dynamic equation of the battery state of charge is obtained in real time through a two-dimensional lookup table, accurately reflecting the nonlinear characteristics of efficiency changes with state of charge and power level, ensuring the accuracy of the recursive prediction of the state of charge. This security verification layer, as an essential link between the reinforcement learning agent and the actuator, performs deterministic physical feasibility verification on all control commands, thus avoiding the security risk of control commands that violate the physical constraints of the dynamic system being issued and executed in a pure reinforcement learning scheme. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart of the steps of the method of the present invention; Figure 2 A flowchart illustrating the steps involved in constructing training data for a power demand prediction model using the method of this invention. Detailed Implementation

[0027] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. It should also be noted that, to make the embodiments more comprehensive, the following embodiments are the best and preferred embodiments, and those skilled in the art can use other alternative methods to implement some well-known technologies; moreover, the accompanying drawings are only for more specific description of the embodiments and are not intended to specifically limit the present invention.

[0028] Please see Figures 1-2 This invention provides a machine learning-based energy management method for range-extended electric rail vehicles. Addressing the power distribution problem between the power battery and the range extender during real-time operation, this method constructs a collaborative control architecture comprising three functional layers: model prediction, intelligent decision-making, and constraint verification. By embedding physical constraints into the deep learning model training process, performing end-to-end joint optimization of the intermediate features of the prediction model with a reinforcement learning policy network, and utilizing model predictive control to perform safety verification of the intelligent decision-making results, this method systematically resolves the technical contradiction in existing technologies where it is difficult to simultaneously achieve forward-looking prediction capabilities, physical constraint satisfaction, and generalization performance under unknown operating conditions.

[0029] Deep learning power demand prediction model with embedded physical information The power demand prediction model constructed in this invention is a deep learning model that embeds physical constraints during training. Its network structure consists of an encoder and a decoder. The encoder employs a bidirectional long short-term memory network, which performs temporal encoding on the input time-series data in both forward and reverse directions. Forward temporal encoding processes the data sequentially along the timeline, capturing causal relationships from the past to the future; reverse temporal encoding processes the data in reverse direction, capturing inverse dependencies from the future to the past. The results of the two encoding directions are concatenated to form an encoded feature vector, which contains the feature representation of each time step in the input time series within the global context. The decoder employs a unidirectional long short-term memory network based on an attention mechanism. In each decoding step, the attention mechanism performs a weighted summation of all encoded feature vectors output by the encoder. The weight coefficients reflect the degree of attention paid to features from different historical moments in the current decoding step. The decoder recursively generates the power demand prediction value for the next moment based on the weighted context vector and its own hidden state, and after successive recursion, a complete power demand prediction sequence is obtained.

[0030] The loss function used in training this model consists of the sum of a data fitting loss term and a physical constraint penalty term. This design is one of the key innovations of this invention. The data fitting loss term uses the mean squared error between the predicted power demand sequence and the actual power demand sequence, and its function is to enable the model to learn the changing patterns of power demand from historical operating data. The physical constraint penalty term is a specially constructed constraint violation penalty function, and its mathematical form is:

[0031] The penalty term checks the absolute value of the power change rate between two adjacent prediction steps in the prediction sequence one by one. If this change rate exceeds the maximum power change rate of the range-extended rail vehicle, the penalty term will be applied. When the excess is included in the loss in square form, the model parameters are updated in the direction that satisfies the power change rate constraint during the optimization process. The selection rule is as follows: Obtain the maximum power change rate of the traction motor and the maximum power change rate of the range extender of the range-extended railcar, and take the smaller of the two. This smaller value represents the limit of change capability of the slower-responding component in the entire power system. Using this as the upper bound constraint ensures that the prediction results are physically feasible for all components of the power system. Penalty coefficient. Used to adjust the relative weight of the physical constraint penalty term in the total loss.

[0032] In terms of training strategy, this invention adopts a phased training method: in the initial phase, a penalty coefficient is set. Setting the value to 0 initially allows the model to focus on learning the statistical regularity of power demand; once the data fitting loss term converges to the first preset threshold, it is gradually increased. Until the second preset threshold is reached, the model achieves a balance between maintaining prediction accuracy and meeting physical constraints, ultimately ensuring that the power demand prediction sequence output by the prediction model maintains high prediction accuracy while naturally meeting the physical rate of change constraints of the power system.

[0033] Reinforcement learning energy management agent and its joint optimization with prediction model The reinforcement learning energy management agent employs a deep deterministic policy gradient algorithm, whose core consists of a policy network and an evaluation network. The policy network generates the target power sequence for the range extender and the target power sequence for the battery based on the current state of charge of the battery, the current output power of the range extender, and the power demand prediction sequence output by the power demand prediction model. The evaluation network is used to estimate the long-term cumulative reward of taking a specific action in the current state to guide the parameter updates of the policy network.

[0034] Another key innovation of this invention lies in introducing the encoded feature vector output by the encoder of the power demand prediction model as auxiliary state information into the policy network. This design is not a simple information concatenation, but involves end-to-end joint optimization of two deep neural networks at the encoded feature space level. During training, the encoder of the power demand prediction model extracts features from the input time-series data (driving speed, track slope angle, and track vehicle position coordinates) to generate an encoded feature vector. This vector is a dense representation of the original input data in a low-dimensional latent space, carrying deep structural information about the current operating conditions. Before being input into the policy network, this encoded feature vector undergoes dimensionality mapping through a feature transformation network. The feature transformation network is a fully connected neural network whose function is to transform the dimension of the encoded feature vector from the output dimension of the encoder to the input dimension desired by the policy network.

[0035] The complete input state of the policy network consists of four parts: the current state of charge of the power battery, the current output power of the range extender, the power demand prediction sequence, and the encoded feature vector mapped by the feature transformation network. During training, the total loss of the reinforcement learning energy management agent is simultaneously backpropagated to the encoders of the policy network, the feature transformation network, and the power demand prediction model. This allows the encoder to not only serve the prediction task but also optimize its feature representation for the decision-making task. This joint optimization mechanism enables the policy network to directly perceive the internal representation of the current operating condition by the prediction model during the training phase, without relying on the complete sequence output by the prediction model as the sole information bridge. This effectively reduces the interference of prediction errors on the decision-making strategy and significantly improves the policy network's ability to understand changes in operating conditions and its decision robustness.

[0036] The reward function for the reinforcement learning energy management agent is designed as follows:

[0037] This reward function considers three optimization objectives simultaneously: fuel economy, state of charge maintenance, and power change smoothness. Instantaneous fuel consumption rate. The fuel consumption rate characteristic curve of the range extender is obtained by querying the output power of the range extender at time (t). This characteristic curve is pre-determined through steady-state bench tests of the range extender and stored in the form of a two-dimensional table. It is the absolute value of the difference between the current state of charge and the preset target state of charge, used to guide the strategy to maintain the battery charge at a reasonable level. It is the absolute value of the change in the range extender's output power, used to suppress frequent large power fluctuations. These three factors are weighted by a coefficient. , , The negative value of the weighted sum is used as the reward, which optimizes the strategy in the direction of reducing fuel consumption, maintaining a stable state of charge, and reducing power fluctuations.

[0038] Safety verification and correction of constraint-satisfying model predictive control While the range extender target power sequence output by the reinforcement learning energy management agent is statistically close to optimal, reinforcement learning, as an approximate solution method, cannot provide deterministic constraints that are satisfied in its output. In safety-critical applications such as range-extended electric vehicles, control commands must be verified against strict physical constraints before execution. Therefore, this invention designs a safety verification and correction module for constraint-satisfying model predictive control. This module uses the range extender target power value at the first moment of the range extender target power sequence as a reference input in each control cycle, solves the optimization problem under the physical constraints of the power system, and obtains the corrected range extender output power command and the corrected power battery output power command actually executed in the current control cycle.

[0039] The core constraints of this optimization problem include three levels. The first is the dynamic equation of the battery's state of charge, which describes the change in battery energy during charging and discharging, including the charging / discharging efficiency function. The introduction of this feature enables the model to accurately reflect the nonlinear characteristics of battery efficiency changes with state of charge and power level during actual operation. The efficiency value is obtained through a pre-calibrated two-dimensional lookup table. Secondly, there is a range extender power change rate constraint, which limits the change in range extender output power between adjacent control cycles to no more than the maximum allowable power change. This constraint is crucial for protecting the lifespan of the range extender's mechanical components and ensuring the quality of generated power. Thirdly, there is a power battery charging and discharging power limit constraint, which restricts the power battery output power to between the maximum charging power and the maximum discharging power. Both limits change with the battery state of charge and are determined in real-time by the battery management system's parameter table based on the measured state of charge.

[0040] The objective function of the optimization problem requires the corrected range extender output power command to be as close as possible to the target power reference value given by the reinforcement learning agent, which is equivalent to preserving the agent's optimization decision-making intent to the greatest extent possible under strict physical constraints. The optimization problem is solved using the interior-point method, which is suitable for handling nonlinear programming problems with inequality constraints. To ensure real-time performance, the solution time is limited to no more than 50% of the control cycle length. If convergence to the preset accuracy is not achieved within the specified time, the range extender output power command from the previous control cycle, after amplitude limiting, is used as the output for the current cycle to ensure the continuity and physical feasibility of the control quantity. The control cycle length ranges from 50 milliseconds to 200 milliseconds. This range satisfies the real-time computing capabilities of the embedded system while being sufficient to cover the dynamic response bandwidth of the range-extended rail vehicle's power system.

[0041] Example This embodiment focuses on the energy management and control of a range-extended electric vehicle (REEV) on a 45-kilometer-long urban rail transit line with 22 stations. The main power system parameters of the REEV are as follows: [Parameters not provided in the original text]. The rated power of the range extender is 160 kW, and the maximum allowable power variation of the range extender is 120 kWh. With a control cycle power of 40 kW, the maximum power change rate of the traction motor is 80 kW per control cycle, and the maximum power change rate of the range extender is 50 kW per control cycle. Therefore, the maximum power change rate is... The smaller of the two values ​​is taken, which is 50 kW per control cycle. The target power battery state of charge is preset to 0.6, and the control cycle length is... The time interval is set to 100 milliseconds, the length of the future prediction time domain is set to 5 seconds, and the prediction step time interval is... The control period is equal to 100 milliseconds, and the prediction steps (H) are equal to 5 seconds divided by 100 milliseconds, which is 50 steps. The implementation process of this embodiment is described in detail below.

[0042] Step 1: Collect real-time data from multiple sources During the operation of the range-extended railcar, real-time data is collected through the following devices: An inertial measurement unit (IMU) is installed in the center of the car body underframe to collect the current travel speed at a sampling frequency of 100 Hz. A gradient sensor is installed on the car body underframe near the bogie to collect the current track gradient angle at a sampling frequency of 100 Hz. The battery management system collects the current state of charge of the power battery through an internal sampling chip, with a data update frequency of 10 Hz. The range extender controller transmits the current range extender output power via the CAN bus, with a data update frequency of 50 Hz. A Global Navigation Satellite System (GPS) receiver is installed on the roof to collect the current railcar's position coordinates, including longitude, latitude, and altitude, with an update frequency of 10 Hz.

[0043] All collected data is first aligned by timestamp, and linear interpolation is used to unify data with different update frequencies to a reference sampling time of 100 Hz. Using the second pulse signal from the Global Navigation Satellite System (GNSS) receiver as a unified clock source, data frames from each sensor are correlated to the most recent reference sampling time. The aligned data is then filtered by a Kalman filter. The Kalman filter's state vector is a four-dimensional vector, including the travel speed, track slope angle, and their respective first derivatives. The process noise covariance matrix is ​​set as a diagonal matrix based on the linearization error of the range-extended vehicle's dynamic model. The observation noise covariance matrix is ​​set based on the inertial measurement unit's speed measurement accuracy of 0.1 m / s and the slope sensor's angle measurement accuracy of 0.05 degrees. The filtered data undergoes outlier detection. The normal value ranges for each physical quantity are set as follows: travel speed 0 to 120 km / h, track slope angle -5 degrees to 5 degrees, battery state of charge 0.05 to 1.0, range extender output power 0 to 160 kW, and the longitude and latitude of the track vehicle's position coordinates within the effective range of the track area. Values ​​outside the normal range are discarded and replaced with valid values ​​that passed outlier detection at the previous baseline sampling time. The data processed in the above way are used as the current travel speed, current track gradient angle, current battery state of charge, current range extender output power, and current track vehicle position coordinates for subsequent steps.

[0044] Step 2: Power demand forecasting model generates power demand forecast series The current travel speed, current track slope angle, and current track vehicle position coordinates obtained in step 1 are input into the deep learning power demand prediction model embedded with physical information. The model outputs a power demand prediction sequence for 50 prediction steps within the next 5 seconds.

[0045] The encoder of this power demand prediction model is a three-layer bidirectional long short-term memory network, with 128, 64, and 32 hidden units in each layer, respectively. The input sequences of travel speed, track slope angle, and track vehicle position coordinates are first mapped into 64-dimensional feature vectors through an input layer, with the mapper being a fully connected layer. The encoder performs forward and backward temporal encoding on the mapped feature sequences, concatenating the forward and backward hidden states at each layer and time step to obtain an encoded feature vector sequence, where each encoded feature vector has 64 dimensions. The decoder is a two-layer unidirectional long short-term memory network based on an attention mechanism, with 64 hidden units in each layer. The attention mechanism uses Luong-style dot product attention, calculating the dot product similarity between the current decoding step's hidden state and all encoded feature vectors. After Softmax normalization, the attention weights are obtained. A weighted sum of the encoded feature vector sequences is then performed to obtain a context vector. This context vector is concatenated with the current decoding step's hidden state and passed through an output fully connected layer to generate the power demand prediction value for the current prediction step. The decoder runs recursively for 50 steps to generate a complete power demand prediction sequence.

[0046] This embodiment details the training process of the power demand prediction model. The training data comes from the actual operating data of the line over the past 180 days, from which 30,000 consecutive time series segments are extracted, each segment lasting 600 seconds. A sliding window method is used, with an input window length of 100 sampling points (corresponding to 10 seconds of historical data) and an output window length of 50 sampling points (corresponding to 5 seconds of prediction time domain). The sliding step size is 10 sampling points, ultimately generating approximately 1.5 million training samples and 300,000 validation samples.

[0047] The Adam optimizer was used during training, with an initial learning rate of 0.001 and a batch size of 64. A penalty coefficient was set in the first stage. The initial value is 0. After 10,000 training iterations, the data fitting loss term converges to below the first preset threshold of 0.01. In the second stage, the penalty coefficient is increased, and every 1000 iterations... Increase by 0.01 and continue training for 5000 iterations until... Once the second preset threshold of 0.05 is reached, the physical constraint violation rate on the validation set is less than 1%. After training, the power demand prediction model is permanently deployed in the vehicle-mounted embedded computing unit.

[0048] In actual operation, the power demand prediction model accepts input from 100 historical sampling points in a single forward inference, performs encoding and decoding processes, and outputs a power demand prediction sequence of 50 prediction steps. With the onboard embedded computing unit equipped with a graphics processor for acceleration, the model's single inference time is 8.6 milliseconds.

[0049] Step 3: Reinforcement learning energy management agent generates power allocation sequence The current state of charge of the power battery and the current output power of the range extender obtained in step 1, along with the power demand prediction sequence in the future prediction time domain output in step 2, are jointly input into the reinforcement learning energy management agent based on the deep deterministic policy gradient algorithm. The reinforcement learning energy management agent outputs the target power sequence of the range extender and the target power sequence of the power battery in the future prediction time domain.

[0050] In this embodiment, the policy network of the reinforcement learning energy management agent contains three fully connected hidden layers, with 512, 256, and 128 neurons in each layer, respectively. The activation function of each layer is a rectified linear unit function (RCU), and the activation function of the output layer is a hyperbolic tangent function. The output range is scaled and mapped to the allowable output range of the range extender power from 0 to 160 kW. The input state of the policy network is composed of four parts: the first part is a scalar representing the current state of charge of the power battery; the second part is a scalar representing the current output power of the range extender; the third part is a 50-dimensional vector representing the power demand prediction sequence output in step 2; and the fourth part is a 32-dimensional vector representing the encoded feature vector after mapping by the feature transformation network. The feature transformation network consists of two fully connected hidden layers, with the first layer containing 256 neurons and the second layer containing 128 neurons, both with RCU activation functions. The input to the feature transformation network is the 64-dimensional encoded feature vector of the last time step in the encoded feature vector sequence output by the encoder of the power demand prediction model, and the output is a 32-dimensional mapped vector. The four parts are spliced ​​together to form a complete 564-dimensional state vector input policy network.

[0051] The evaluation network has the same network structure as the policy network. The difference is that the evaluation network additionally concatenates the action values ​​output by the policy network into the input state of the evaluation network. The evaluation network finally outputs a scalar, namely the state-action value estimate.

[0052] The reward function for the reinforcement learning energy management agent is calculated as follows. The instantaneous fuel consumption rate of the range extender at the current output power is obtained through a lookup table: the fuel consumption rate characteristic curve provided by the range extender manufacturer is given in a two-dimensional table. The first column of the table is the range extender output power, divided into 33 discrete points at 5 kW intervals from 0 kW to 160 kW. The second column of the table is the fuel consumption rate corresponding to each discrete power point, in grams per kilowatt-hour. Linear interpolation is used to obtain the fuel consumption rate at non-discrete points during the lookup. State of charge offset. This is the absolute value of the difference between the current state of charge and the target state of charge (0.6). Range extender power change. This is the absolute value of the difference between the current output power of the range extender and the output power of the range extender at the previous moment. Fuel economy weighting coefficient. Set to 1.0, the weighting factor for maintaining the state of charge. Set the weighting factor to 500.0 for power change smoothness. The value is set to 0.2, and all three values ​​are preset values ​​determined through offline parameter tuning.

[0053] The training of the reinforcement learning energy management agent adopts the following configuration: the experience replay buffer has a capacity of 100,000 experience samples; each update randomly selects 128 experience samples from the experience replay buffer to form a mini-batch; both the policy network and the evaluation network use the Adam optimizer for parameter updates, with a learning rate of 0.0001 for the policy network and 0.001 for the evaluation network; the soft update coefficient of the target network is 0.005, that is, the target network parameters are updated to the sum of 0.005 times the current network parameters and 0.995 times the target network parameters in each training step; the exploration policy adopts a high noise level in the early stage of training to fully explore the state-action space, the noise adopts the Ornstein-Uhlenbeck process, the initial noise standard deviation is 0.3, and the decay rate is 0.995 times the current value every 1000 training steps until the standard deviation is lower than 0.01 and then remains unchanged. The training consisted of 5000 rounds, each round simulating a complete round trip of the line starting from a random initial state of charge. When H equals 50, the lengths of the target power sequence for the range extender and the target power sequence for the power battery are both 50.

[0054] During training, the encoder parameters of the power demand prediction model in step 2 participate in gradient backpropagation updates, achieving end-to-end joint optimization of the encoder and policy network at the encoding feature space level. After training, the parameters of the policy network, feature transformation network, and encoder are jointly fixed and deployed to the in-vehicle embedded computing unit. In actual operation, the single inference time of the policy network is 7.2 milliseconds.

[0055] Step 4: Perform safety verification and correction on the constraint-satisfaction model predictive control. The range extender target power value at the first moment in the range extender target power sequence output in step 3 is used as the reference value. Within the current control cycle, the optimization problem is solved with this reference value as the objective and under strict physical constraints.

[0056] The objective function of the optimization problem is:

[0057] The objective function characterizes the modified range extender output power command actually executed in the current control cycle. Compared with reference value The degree of deviation between them is taken in square form to ensure that both positive and negative deviations are penalized equally.

[0058] The constraints of the optimization problem are as follows.

[0059] The first constraint is the dynamic equation of the state of charge of the power battery:

[0060] This equation establishes a quantitative relationship between the current cycle's power battery output power and the predicted state of charge for the next control cycle. In the equation... The measured value of the power battery's state of charge for the current control cycle obtained in step 1. This is a predicted value for the state of charge of the power battery in the next control cycle. The decision variable is the output power of the power battery in the current control cycle; positive values ​​indicate discharging and negative values ​​indicate charging. This represents the maximum usable energy of the power battery. The control cycle length is 100 milliseconds, or 0.1 seconds. Charge / discharge efficiency function. The numerical values ​​are obtained as follows: The battery management system pre-stores a two-dimensional table of discharge efficiency. The row index of the table represents the state of charge of the power battery, ranging from 0.05 to 1.0, divided into 20 discrete values ​​at 0.05 intervals. The column index represents the output power of the power battery, ranging from -80 kW to 240 kW, divided into 33 discrete values ​​at 10 kW intervals, where negative values ​​correspond to charging and positive values ​​correspond to discharging. Each cell in the table records the charge and discharge efficiency of the battery under the corresponding operating condition. When the range-extended railcar is in regenerative braking mode, the power battery is in a charging state driven by the traction motor. Instead of looking up the value from the discharge efficiency table, the value is looked up from the regenerative braking efficiency table stored in the battery management system. The structure of the regenerative braking efficiency table is the same as that of the discharge efficiency table, but the values ​​are independently calibrated to reflect the changes in battery charging efficiency during the braking energy recovery process.

[0061] The second constraint is the range extender power change rate constraint:

[0062] In the formula The actual output power of the range extender in the previous control cycle, collected in step 1. The maximum allowable power variation for the range extender is 40 kilowatts.

[0063] The third constraint is the power battery charging and discharging power limitation constraint:

[0064] In the formula The maximum discharge power of the power battery under the current state of charge (SOC) is obtained by querying the parameter table of the battery management system based on the SOC data collected in step 1. This parameter table uses SOC as an index and records the maximum allowable discharge power and maximum charging power for each SOC, both in kilowatts (kW). For example, when the SOC is 0.8, the maximum discharge power is 240 kW, and the absolute value of the maximum charging power is 60 kW; when the SOC drops to 0.2, the maximum discharge power decreases to 120 kW, while the absolute value of the maximum charging power increases to 80 kW.

[0065] In addition, there is a power balance equation constraint, that is, the sum of the output power of the range extender and the output power of the power battery is equal to the power demand value at the current moment. This power demand value is obtained by longitudinal dynamics calculation based on the driving speed and track slope angle collected in step 1. The dynamic parameters include vehicle mass, rolling resistance coefficient, air resistance coefficient and rotational mass conversion coefficient.

[0066] The optimization problem is solved using the interior-point method, implemented in an onboard embedded computing unit. The convergence accuracy threshold for the interior-point method is set to 0.01 kW, the maximum number of iterations is 20, and the computation time is budgeted as 50% of the control cycle length of 100 milliseconds, i.e., 50 milliseconds. In this embodiment, the interior-point method converges to the preset accuracy within 50 milliseconds in 98.7% of the control cycles, outputting the corrected range extender output power command. and the revised power battery output power command For the remaining 1.3% of control cycles that fail to converge within 50 milliseconds, the range extender output power command from the previous control cycle is used. The value after limiting is used as the corrected range extender output power command for the current control cycle. The upper and lower bounds of the limiting process are respectively... and The corrected power battery output power command is calculated in reverse based on the power balance equation constraint.

[0067] Step 5: Execute energy management control commands The corrected range extender output power command from step 4 is sent to the range extender controller via the CAN bus. Upon receiving the command, the range extender controller adjusts the engine throttle opening and generator excitation current to ensure the actual output power of the range extender tracks the command value, with an adjustment response time within 200 milliseconds. The corrected power battery output power command is then sent to the battery management system via the CAN bus. The battery management system controls the switching duty cycle of the DC-DC converter, causing the power battery to output or absorb power according to the command value. This completes the energy management control for the current control cycle.

[0068] Step 6: Execute repeatedly When a control cycle arrives, return to step 1 and repeat steps 1 through 5. This loop continues until a mission completion signal is received from the train operation management system. During line operation, if the GPS signal is lost due to tunnel or building obstruction, causing the track vehicle's position coordinates to be lost, the track vehicle position coordinates in the power demand prediction model input of step 2 are replaced by the position coordinates recursively estimated by the inertial navigation system. The inertial navigation system uses the three-axis acceleration and three-axis angular velocity information provided by the inertial measurement unit in step 1 for dead reckoning, and the positioning error increases with time and mileage. During the period of lost positioning signal, a weighted fusion mechanism is activated: the power demand prediction sequence output by the power demand prediction model is weighted and fused with a preset conservative power demand sequence. The conservative power demand sequence is estimated based on the current track slope angle and the current travel speed, using a vehicle longitudinal dynamics model. The dynamics model uses a slope resistance coefficient of 1.2 times the actual slope and a rolling resistance coefficient of 1.1 times the normal value. This conservative assumption ensures that the value of the conservative power demand sequence is slightly higher than the normally predicted power demand value under the same conditions. The weighting coefficients of the weighted fusion are dynamically adjusted according to the duration of the positioning signal loss: when the loss duration is 0 to 10 seconds, the weight of the conservative power demand sequence increases linearly from 0 to 0.3; when the loss duration is 10 to 30 seconds, the weight increases linearly from 0.3 to 0.7; and when the loss duration exceeds 30 seconds, the weight is fixed at 1.0, meaning that the conservative power demand sequence is completely used to replace the model prediction sequence. Once the Global Navigation Satellite System signal is restored, the system immediately switches back to normal mode.

[0069] Effect verification To verify the technical effectiveness of this embodiment, three different energy management strategies were compared and tested under the same line operating conditions. Comparative Example 1 is a deterministic rule-based strategy: when the state of charge (SOC) of the power battery is below 0.3%, the range extender is activated and operates at a constant power of 100 kW; when the SOC is above 0.7%, the range extender is deactivated. Comparative Example 2 is a combination of a deep learning prediction model without embedded physical constraints and the same reinforcement learning energy management agent. However, in Comparative Example 2, the prediction model is not trained with physical constraint penalties, and the reinforcement learning agent and the prediction model are trained independently without joint optimization. This embodiment is the method of the present invention. All three strategies are run on the same onboard embedded computing unit, and the entire route from the starting station to the terminal station is 45 kilometers long and includes 22 stations.

[0070] The test results are shown in the table below:

[0071] As shown in the table above, the total fuel consumption of this embodiment is 31.2 liters, a reduction of 26.6% compared to Comparative Example 1 and 12.8% compared to Comparative Example 2. The state of charge (SOC) of the power battery at the destination is 0.58, close to the target value of 0.6, indicating that the energy management strategy effectively maintains the balance of battery charge. The average operating efficiency of the range extender is 38.7%, higher than both comparative examples, indicating that the range extender operates more in the high-efficiency range. More importantly, Comparative Example 2, due to the lack of embedded physical constraints in the prediction model training, experienced 47 instances of predicted sequences violating physical constraints during the entire operation, while this embodiment maintained zero instances of physical constraint violations, demonstrating the effectiveness of embedding physical constraints in the training. The average total computation time for control commands is only 18.5 milliseconds, far below 50% of the 100-millisecond control cycle, i.e., the 50-millisecond time limit requirement, fully meeting the engineering requirements for online real-time control.

[0072] This invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments; however, those skilled in the art will fully understand the invention even without these details. Furthermore, to avoid unnecessary misunderstanding of the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0073] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A machine learning-based energy management method for range-extended electric rail vehicles, characterized in that, Includes the following steps: Step 1: Collect the travel speed, track gradient angle, power battery state of charge, range extender output power, and track vehicle position coordinates of the range-extended railcar; Step 2: Input the current travel speed, the current track slope angle, and the current position coordinates of the railcar into the power demand prediction model. The power demand prediction model is a deep learning model with physical information embedded. The power demand prediction model outputs a power demand prediction sequence in the future prediction time domain. The training loss function of the power demand prediction model consists of the sum of a data fitting loss term and a physical constraint penalty term. The physical constraint penalty term is used to constrain the power change rate at any adjacent time in the power demand prediction sequence to not exceed the maximum power change rate constraint of the range-extended railcar. Step 3: Input the current state of charge of the power battery, the current output power of the range extender, and the power demand prediction sequence into the reinforcement learning energy management agent. The reinforcement learning energy management agent adopts a deep deterministic policy gradient algorithm and outputs the target power sequence of the range extender and the target power sequence of the power battery in the future prediction time domain. During the training process, the policy network of the reinforcement learning energy management agent uses the encoded feature vector output by the encoder of the power demand prediction model as auxiliary state information and inputs it into the policy network for end-to-end joint optimization. Step 4: Taking the target power value of the range extender at the first moment in the target power sequence of the range extender as a reference, under the constraints of the dynamic equation of the state of charge of the power battery of the range-extended rail vehicle, the power change rate constraint of the range extender, and the power charging and discharging power limit constraint, the optimization problem of minimizing the deviation between the actual output power of the range extender and the reference is solved by the constraint-satisfying model predictive control, so as to obtain the corrected output power command of the range extender and the corrected output power command of the power battery in the current control cycle. Step 5: Send the modified range extender output power command to the range extender controller, and send the modified power battery output power command to the battery management system to execute energy management control.

2. The energy management method for range-extended electric rail vehicles based on machine learning according to claim 1, characterized in that, The power demand prediction model in step 2 consists of an encoder and a decoder. The encoder uses a bidirectional long short-term memory (LSTM) network, which performs forward and reverse temporal encoding on the input time-series data to generate the encoded feature vector. The decoder uses a unidirectional LSM network based on an attention mechanism, which weights the encoded feature vector using an attention mechanism and generates the power demand prediction sequence through recursive decoding. The formula for calculating the physical constraint penalty term in step 2 is as follows: ;in, For the physical constraint penalty term, Here, (H) is the preset penalty coefficient, and (H) is the number of prediction steps included in the future prediction time domain. This represents the predicted power demand value at the (k)th prediction step in the power demand prediction sequence. This represents the predicted power demand value at the (k+1)th prediction step in the power demand prediction sequence. The time interval between adjacent prediction steps. The maximum power change rate of the range-extended railcar, the The smaller of the maximum power change rate of the traction motor of the range-extended railcar and the maximum power change rate of the range extender is taken; the data fitting loss term is the mean square error between the predicted power demand sequence and the actual power demand sequence.

3. The energy management method for range-extended electric rail vehicles based on machine learning according to claim 1, characterized in that, The training process of the reinforcement learning energy management agent described in step 3 uses the following reward function: ;in, Let (t) be the reward value at time (t). Let be the instantaneous fuel consumption rate at time (t). This instantaneous fuel consumption rate is obtained by querying the fuel consumption rate characteristic curve of the range extender based on the output power of the range extender at time (t). The absolute value of the offset of the state of charge of the power battery at time (t) relative to the preset target state of charge of the power battery. Let be the absolute value of the change in the output power of the range extender at time (t). This is the fuel economy weighting coefficient. Weighting coefficients for maintaining the state of charge. This is a weighting factor for the smoothness of power change. , and All are preset positive real numbers; before being input into the policy network, the encoded feature vector is dimensionally mapped through a feature transformation network, which is a fully connected neural network. The network parameters of the feature transformation network are synchronously updated with the network parameters of the policy network during the training of the reinforcement learning energy management agent; the input state of the policy network is composed of the current state of charge of the power battery, the current output power of the range extender, the power demand prediction sequence, and the encoded feature vector mapped by the feature transformation network; the output action of the policy network is the target power value of the range extender at the first moment in the target power sequence of the range extender.

4. The energy management method for range-extended electric rail vehicles based on machine learning according to claim 1, characterized in that, The objective function of the optimization problem for solving the constraint-satisfying model predictive control in step 4 is: ;in, The corrected range extender output power command is given for the current control cycle. The target power value of the range extender is the value at the first moment in the target power sequence of the range extender; the dynamic equation of the state of charge of the power battery of the range-extended rail vehicle is: ;in, This is a predicted value for the state of charge of the power battery in the next control cycle. This is the measured value of the state of charge of the power battery during the current control cycle. The output power of the power battery during the current control cycle. The charging and discharging efficiency function of the power battery is defined by the parameters of which are obtained by looking up a table based on the state of charge and output power of the power battery. This represents the maximum usable energy of the power battery. To control the duration of the cycle, the power change rate constraint of the range extender is: ;in, This represents the actual output power of the range extender in the previous control cycle. The maximum allowable power variation of the range extender; the power battery charging and discharging power limiting constraint is: ;in, This represents the maximum charging power of the power battery. The maximum discharge power of the power battery, the maximum charging power of the power battery and the maximum discharge power of the power battery are determined by the parameter table of the battery management system according to the state of charge of the power battery; the optimization problem is solved by the interior point method, and the solution time is no more than 50% of the length of the control cycle.

5. The energy management method for a range-extended electric rail vehicle based on machine learning according to claim 2, characterized in that, The training data for the power demand prediction model is constructed in the following way: Continuous time series segments are extracted from the historical operation data of the range-extended railcar. Each time series segment contains the railcar speed sequence, track gradient angle sequence, and railcar position coordinate sequence as input features, and the corresponding power demand sequence as a label. Each time series segment is divided into multiple training samples using a sliding window approach. The input window length of each training sample is a first preset number of steps, and the output window length of each training sample is the number of prediction steps (H) contained in the future prediction time domain. The power demand prediction model is trained using the Adam optimizer with an initial learning rate of 0.001, a batch size of 64, and at least 10,000 training iterations. The penalty coefficient... Initially set to 0, the penalty coefficient is gradually increased after the data fitting loss term converges to a first preset threshold. The power demand prediction model is set to a second preset threshold to ensure that it meets physical constraints while maintaining prediction accuracy.

6. The energy management method for a range-extended electric railcar based on machine learning according to claim 3, characterized in that, The feature transformation network comprises two fully connected hidden layers, with 256 neurons in the first fully connected hidden layer and 128 neurons in the second fully connected hidden layer, using rectified linear units (RCUs) as the activation function. The policy network comprises three fully connected hidden layers, with 512, 256, and 128 neurons in each layer, respectively, also using RCUs as the activation function. The output layer activation function of the policy network is a hyperbolic tangent function. The reinforcement learning energy management agent also includes an evaluation network with the same network structure as the policy network, and its output is a state-action value estimate. The training of the reinforcement learning energy management agent employs an experience replay mechanism, with a replay buffer capacity of 100,000 experience samples. During each training iteration, 128 experience samples are randomly selected from the replay buffer to update the policy network and the evaluation network, with a soft update coefficient of 0.005 for the target network. The exploration noise of the reinforcement learning energy management agent uses an Ornstein-Uhlenbeck process, with an initial noise standard deviation of 0.3, which gradually decreases during training.

7. The energy management method for a range-extended electric railcar based on machine learning according to claim 4, characterized in that, The predictive control time domain length of the optimization problem is equal to the number of prediction steps (H) contained in the future prediction time domain corresponding to the power demand prediction sequence; in the dynamic equation of the state of charge of the power battery, the maximum available energy of the power battery is... The correction is made based on the health status of the power battery, and the correction formula is as follows: ,in The nominal maximum usable energy of the power battery is denoted as , and the current state of health of the power battery is estimated by the battery management system. When the range-extended rail vehicle is in regenerative braking mode, the charge / discharge efficiency function of the power battery is denoted as . The energy recovery efficiency value is obtained by looking up the regenerative braking efficiency table, which is independent of the discharge efficiency table, based on the state of charge of the power battery and the braking power. When solving the optimization problem, if the interior point method fails to converge to the preset accuracy within 50% of the control cycle time, the range extender output power command solved in the previous control cycle is used as the corrected range extender output power command for the current control cycle after being limited.

8. The energy management method for range-extended electric rail vehicles based on machine learning according to claim 1, characterized in that, In step 1, the travel speed of the range-extended railcar is collected through an inertial measurement unit, the track slope angle is collected through a slope sensor, the state of charge of the power battery is collected through a battery management system, the output power of the range extender is collected through a range extender controller, and the position coordinates of the railcar are collected through a global navigation satellite positioning system receiver. In step 1, the collected travel speed, track slope angle, state of charge of the power battery, output power of the range extender, and position coordinates of the railcar are time-stamped to ensure that all data correspond to the same moment. All data after timestamp alignment is filtered using a Kalman filter. The process noise covariance matrix and observation noise covariance matrix of the Kalman filter are set according to the specifications of each sensor. Outlier detection is performed on all filtered data. Values ​​that exceed the normal range of each physical quantity are removed and filled with the corresponding values ​​from the previous moment to generate the current driving speed, track gradient angle, power battery state of charge, range extender output power, and track vehicle position coordinates for use as input in steps 2 and 3.

9. The energy management method for a range-extended electric railcar based on machine learning according to claim 1, characterized in that, The duration of the control cycle The value ranges from 50 milliseconds to 200 milliseconds, which is the time interval between adjacent prediction steps in the power demand prediction sequence. Equal to the time length of the control cycle The length of the future prediction time domain is 3 to 10 seconds, and the corresponding prediction step number (H) is the length of the future prediction time domain divided by the time interval. The power demand prediction model and the inference computation of the reinforcement learning energy management agent are completed in an in-vehicle embedded computing unit. The in-vehicle embedded computing unit has a graphics processor acceleration function. The single inference time of the power demand prediction model does not exceed 15 milliseconds, and the single inference time of the policy network of the reinforcement learning energy management agent does not exceed 10 milliseconds.

10. The energy management method for a range-extended electric rail vehicle based on machine learning according to claim 1, characterized in that, After step 5 is completed, when a control cycle arrives, the process returns to step 1 and repeats steps 1 to 5 until the end signal of the extended-range railcar operation task is received. When the extended-range railcar detects that the positioning signal is lost during the cyclic execution, resulting in the railcar's position coordinates being unavailable, the input of the power demand prediction model in step 2 is replaced by the railcar's position coordinates recursively estimated by the inertial navigation system. At the same time, the power demand prediction sequence output by the power demand prediction model is weighted and fused with a preset conservative power demand sequence. The weighting coefficient is dynamically adjusted according to the duration of the positioning signal loss. The longer the duration of the positioning signal loss, the greater the weight of the conservative power demand sequence. The conservative power demand sequence is the power demand sequence estimated based on the current gradient and current driving speed using the vehicle's longitudinal dynamics model.

Images