Virtual-real interaction hydrogen-lithium hybrid power generation system multi-energy flow regulation and control method

Abstract

The invention discloses a virtual-real interaction hydrogen-lithium hybrid power generation system multi-energy flow regulation and control method, and relates to the technical field of new energy power systems, and the method comprises the steps: building a stress model and a power model in different driving modes according to the driving characteristics of a hydrogen energy hybrid power generation vehicle; driving mode classification is carried out by using extreme gradient rise; running parameters of the hydrogen energy hybrid power generation vehicle are collected and input into the trained extreme gradient rising network for recognition; establishing a mathematical model, a target function and an operation constraint of the hydrogen energy hybrid power generation vehicle energy system; considering uncertainty in a power generation mode, and respectively establishing reinforcement learning models in different driving modes and power generation modes; parameters are set, the reinforcement learning model is trained by using actual data, and a trained network is stored; and testing the scheduling result by using the test data. According to the method, an interactive training mechanism of a virtual simulation environment and actual driving data is introduced, so that the generalization ability and practicability of a regulation and control strategy to a complex operation environment are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of new energy power system technology, and in particular to a multi-energy flow control method for a hydrogen-lithium hybrid power generation system with virtual-real interaction. Background Technology

[0002] As users increasingly demand stable power supply and high power quality, the requirements for high reliability in power distribution networks are becoming more stringent. The continuous and stable supply of electricity has a more significant impact on the normal operation of society in the future. In the event of a sudden incident or emergency, a power outage could paralyze critical sectors such as transportation, communications, and finance. On the other hand, new power systems also require continuous maintenance and upkeep of distribution network substations. Mobile vehicle-mounted emergency power supply can reduce the impact on users during substation maintenance, thus maintaining the normal operation and stability of society.

[0003] Currently, research on fuel cell power supply modules lacks focus on hydrogen flow regulation strategies; current research mainly concentrates on fuel cell energy management strategies and test system optimization. In energy management, the Pontryagin Minimum Principle (PMP) is used to optimize control strategies, offering lower computational burden and comparable fuel economy compared to dynamic programming (DP) methods. Furthermore, the PMP method combines driving mode recognition and Markov speed prediction to achieve real-time updates of costate variables. The Adaptive Equivalent Fuel Consumption Minimum Strategy (ECMS), combined with vehicle speed prediction, provides a basis for real-time adjustment of the equivalent factor. Simultaneously, considering the fuel cell state of health (SOH), ECMS is optimized based on SOH estimation to achieve adaptive control.

[0004] However, these strategies only consider the fuel cell power supply in driving mode or power generation mode alone, without considering multi-energy flow control strategies that have both modes simultaneously. In addition, existing algorithms lack interaction with reality. Summary of the Invention

[0005] The problem to be solved by this invention is to provide a method for multi-energy flow regulation of a hydrogen-lithium hybrid power generation system with virtual-real interaction, which can realize multi-energy flow regulation of hydrogen hybrid power generation vehicles under different driving and power generation modes, while realizing interaction with reality.

[0006] This invention adopts the following technical solution: a multi-energy flow control method for a virtual-real interactive hydrogen-lithium hybrid power generation system, comprising the following steps:

[0007] S1. Based on the driving characteristics of hydrogen hybrid electric vehicles, establish force and power models under different driving modes;

[0008] S2. Use an extreme gradient ascent network to classify driving modes and collect driving parameters of the hydrogen hybrid electric vehicle and input them into the trained extreme gradient ascent network for identification.

[0009] S3. Establish the mathematical model, objective function, and operational constraints of the hydrogen hybrid power generation vehicle energy system;

[0010] S4. Based on the uncertainty in the power generation mode, reinforcement learning models for different driving modes and power generation modes are established respectively. Parameters are set and reinforcement learning models are trained using actual data.

[0011] S5. Based on the trained reinforcement learning model, energy scheduling of hydrogen-lithium hybrid power generation vehicles is carried out.

[0012] Preferably, in step S1, the driving modes of the hydrogen-lithium hybrid power generation vehicle include: acceleration, constant speed, deceleration, climbing, and descending. Based on the driving characteristics of the hydrogen hybrid power generation vehicle, force models and power models are established for different driving modes.

[0013] Preferably, in step S2, the XGBoost algorithm is used to construct multiple decision trees for driving mode classification based on the extreme gradient ascent machine learning method, including the following sub-steps:

[0014] S2.1 Collect driving mode data, including: hydrogen-lithium hybrid vehicle speed v, acceleration a, uphill tilt angle θ, downhill tilt angle γ, fuel cell operating current I. FC Fuel cell operating voltage U FC Lithium battery current I Li Lithium battery voltage U Li ;

[0015] S2.2 Data Cleaning: Preprocess the collected driving mode data, including handling missing values ​​and outliers to ensure data reliability;

[0016] S2.3 Constructing an extreme gradient ascent classification model:

[0017] Based on the XGBoost network, using the extreme gradient ascent algorithm, multiple decision trees are created through the ensemble learning method Bagging. Each tree is trained based on different random subsamples and feature subsets, and the prediction result is determined by majority voting or average value.

[0018] This includes: generating simulated data, separating features and labels, dividing the training and test sets, training the XGBoost network, predicting on the test set, and evaluating the model.

[0019] S2.4 Training and Testing of Extreme Gradient Ascent Classification Models:

[0020] The XGBoost network is trained using the input training set data. The driving modes are divided into five categories: acceleration, constant speed, deceleration, climbing, and descending. The trained XGBoost network is then tested using the input test set data, and the XGBoost network is adjusted based on the model accuracy.

[0021] Preferably, in step S3, the hydrogen hybrid power generation vehicle energy system includes: a proton exchange membrane fuel cell, a storage battery, and a kinetic energy recovery device, and a mathematical model of the hydrogen hybrid power generation vehicle energy system is established.

[0022] Establish operational constraints for the energy system of hydrogen hybrid power generation vehicles, including: electrical balance constraints, upper and lower limits of proton exchange membrane fuel cell power, upper and lower limits of battery state, and battery initialization state constraints.

[0023] Reinforcement learning models are established for different driving and power generation modes, including defining the state space, action space, and reward function.

[0024] The method for setting parameters and training a reinforcement learning model using real data is as follows:

[0025] S4.1 Before the start of each training round, within the typical task scenario and the feasible domain for safe operation of the equipment, add no more than 20% normal random fluctuation to the mean of the load source data, and continuously generate new typical task training scenarios until the end of training.

[0026] S4.2, Perform iterative training: Before the start of each training round, obtain the initial state S0 of the reinforcement learning model; for each time step, the state s t The policy network is based on P(s) t |θ P Output action a t A reward r is obtained after performing the action. t And obtain the next state s t+1 Training experience (s) t ,a t ,r t ,s t+1 Save to the experience replay pool, update the current status, and complete one round of training;

[0027] S4.3 Reinforcement learning model parameter update: When the experience parameters in the experience replay pool meet the batch size, the network parameters are updated at each time step, the policy network and the evaluation network are updated, and the target network is updated.

[0028] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects:

[0029] 1. Integrating virtual and real interaction to improve strategy adaptability: The control method of this invention improves the generalization ability and practicality of the control strategy to complex operating environments by introducing an interactive training mechanism between the virtual simulation environment and actual driving data.

[0030] 2. Integrated XGBoost driving mode recognition to improve recognition accuracy and response speed: This invention uses the XGBoost algorithm for driving mode classification, which improves the accuracy and real-time performance of driving state recognition, providing reliable input for subsequent energy regulation.

[0031] 3. Introducing the PPO (Proximal Policy Optimization) reinforcement learning algorithm to optimize energy management strategy: This invention utilizes the Proximal Policy Optimization algorithm to achieve dynamic adaptive scheduling of the energy system under uncertain operating conditions, thereby enhancing the robustness and efficiency of the system. Attached Figure Description

[0032] Figure 1 This is a flowchart of a multi-energy flow control method for a hydrogen-lithium hybrid power generation system;

[0033] Figure 2 This is the XGBoost error convergence plot;

[0034] Figure 3 This is the XGBoost confusion matrix diagram;

[0035] Figure 4 This is an XGBoost feature importance map;

[0036] Figure 5 This is a cumulative reward chart for PPO training in driving mode;

[0037] Figure 6 This is a cumulative reward graph for PPO training in power generation mode;

[0038] Figure 7 This is a diagram showing the dispatch results under the power generation mode;

[0039] Figure 8 This is a diagram showing the scheduling results in accelerated mode;

[0040] Figure 9 This is a diagram showing the scheduling results under deceleration mode;

[0041] Figure 10 This is a diagram showing the scheduling results under the ramp mode;

[0042] Figure 11 This is a diagram showing the scheduling results under downhill mode;

[0043] Figure 12 This is a diagram showing the scheduling results under constant speed mode. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the application will be further described in detail below with reference to the accompanying drawings. The described embodiments are only a part of the embodiments involved in this invention. All non-innovative embodiments based on these embodiments by other researchers in the art are within the protection scope of this invention. Furthermore, the step numbers in the embodiments of this invention are only set for ease of explanation and do not limit the order of the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0045] In one embodiment of the present invention, a multi-energy flow control method for a hybrid hydrogen-lithium power generation system with virtual-real interaction is provided, such as... Figure 1 As shown, it includes the following steps:

[0046] S1. Based on the driving characteristics of hydrogen hybrid power generation vehicles, establish force and power models under different driving modes. The driving modes of hydrogen-lithium hybrid power generation vehicles include: acceleration, constant speed, deceleration, climbing, and descending.

[0047] S2. Use an extreme gradient ascent network to classify driving modes and collect driving parameters of the hydrogen hybrid electric vehicle and input them into the trained extreme gradient ascent network for identification.

[0048] S3. Establish the mathematical model, objective function, and operational constraints of the hydrogen hybrid power generation vehicle energy system;

[0049] S4. Based on the uncertainty in the power generation mode, reinforcement learning models for different driving modes and power generation modes are established respectively. Parameters are set and reinforcement learning models are trained using actual data.

[0050] S5. Based on the trained reinforcement learning model, energy scheduling of hydrogen-lithium hybrid power generation vehicles is carried out.

[0051] Specifically, in step S1 of this embodiment, the driving modes of the hydrogen-lithium hybrid power generation vehicle include: acceleration, constant speed, deceleration, climbing, and descending. Based on the driving characteristics of the hydrogen hybrid power generation vehicle, force models and power models are established for different driving modes. Details are as follows:

[0052] (1) When the hydrogen-lithium hybrid electric vehicle accelerates, the force model is as follows:

[0053]

[0054] The power model is as follows:

[0055] P a (t)=T a (t)·v(t);

[0056] Where W is the gravity of the hydrogen-lithium hybrid power generation vehicle, N is the support force provided by the ground, and T a (t) represents the traction force during acceleration, μ is the coefficient of friction, m is the total mass of the hydrogen-lithium hybrid power generation vehicle, and a a (t) represents the acceleration of the hydrogen-lithium hybrid electric vehicle during acceleration, P a (t) represents the real-time power during acceleration, and v(t) represents the real-time speed of the hydrogen-lithium hybrid power generator.

[0057] (2) When the hydrogen-lithium hybrid power generation vehicle decelerates, the force model is as follows:

[0058]

[0059] The power model is as follows:

[0060] P m (t)=T m (t)·v(t);

[0061] Among them, T m (t) represents the traction force during deceleration, a m (t) represents the acceleration of the hydrogen-lithium hybrid power generation vehicle during deceleration, P m (t) represents the real-time power during deceleration.

[0062] (3) When the hydrogen-lithium hybrid power generation vehicle is moving at a constant speed, the force model is as follows:

[0063]

[0064] The power model is as follows:

[0065] P c (t)=T c (t)·v(t);

[0066] Among them, T c (t) represents the traction force when the velocity is constant, P c (t) represents the real-time power at a constant speed.

[0067] (4) When the hydrogen-lithium hybrid power generation vehicle climbs a slope, the force model is as follows:

[0068]

[0069] The power model is as follows:

[0070] P cl (t)=T cl (t)·v(t);

[0071] Among them, T cl (t) represents the traction force during the uphill climb, θ represents the angle of elevation during the climb, and P cl(t) represents the real-time power during the climb.

[0072] (5) When the hydrogen-lithium hybrid power generation vehicle goes downhill, the force model is as follows:

[0073]

[0074] The power model is as follows:

[0075] P do (t)=T do (t)·v(t);

[0076] Among them, T do (t) represents the traction force during downhill driving, γ represents the downhill angle of depression, and P do (t) represents the real-time power during downhill.

[0077] Specifically, in step S2 of this embodiment, based on the force model and power model of different driving modes established in step S1, extreme gradient ascent is used to classify and identify driving modes, as follows:

[0078] S21. Combining the force and power models established in step S1 for different driving modes, generate rule labels based on acceleration and slope angle to represent different operating condition types. Specific rules are as follows:

[0079] (1) Large acceleration and small slope: acceleration

[0080] When the acceleration is greater than a certain positive value (e.g., >0.5) and the absolute value of the slope angle is small (e.g., |slope| <5°), the vehicle is considered to be in an acceleration state.

[0081] (2) Acceleration is small and close to 0: uniform speed

[0082] When the acceleration is close to 0 (e.g., -0.5 < acceleration < 0.5) and the slope angle is small (|slope| < 5°), the vehicle's speed change is not obvious, and the vehicle is considered to be in a constant speed state.

[0083] (3) Negative acceleration and small gradient: Deceleration

[0084] When the acceleration is negative (e.g., <-0.5) and the absolute value of the slope angle is small (|slope| < 5°), the vehicle is in a deceleration state.

[0085] (4) The slope is steep and positive: climbing

[0086] When the slope angle is large and positive (e.g., >5°), the vehicle will be judged to be climbing regardless of the acceleration.

[0087] (5) Slope is steep and negative: downhill

[0088] When the slope angle is large and negative (e.g., <-5°), the vehicle will be judged as going downhill regardless of the acceleration.

[0089] S22. Use extreme gradient ascent to classify and identify driving modes.

[0090] Extreme Gradient Boosting (XGBoost) uses a Bagging ensemble learning method to create multiple decision trees. Each tree is trained on different random subsamples and feature subsets, and the final prediction result is determined by majority voting or average value. The main steps include: generating simulated data, separating features and labels, splitting the training and test sets, XGBoost training, test set prediction, and model evaluation.

[0091] (1) Generate simulation data

[0092] a) Feature generation:

[0093] Speed ​​is represented by generating a uniformly distributed random number between 0 and 120, and the unit is km / h.

[0094] Acceleration is represented by generating uniformly distributed random numbers between -3 and 3, and the unit is m / s². 2 .

[0095] The slope angle is represented by a uniformly distributed random number between -10 and 10, with the unit being degrees (°). Negative values ​​indicate downhill slopes, and positive values ​​indicate uphill slopes.

[0096] Power demand is represented by generating a uniformly distributed random number between 0 and 150, and the unit is kW.

[0097] b) Tag generation:

[0098] Based on the features (acceleration and slope angle) generated above, each sample is labeled with different working conditions (e.g., acceleration, constant speed, deceleration, climbing, and descending) using rules.

[0099] The rules are based on acceleration and slope angle conditions to determine and assign labels:

[0100] If the slope is greater than 5°, it is labeled "Uphill";

[0101] If the slope is less than -5°, it is labeled "downhill";

[0102] If the acceleration is greater than 0.5 m / s 2 If so, the label will be "accelerate";

[0103] If the acceleration is less than -0.5 m / s 2 If so, the label will be "deceleration";

[0104] In other cases, if the acceleration is between -0.5 and 0.5, the label is "uniform speed".

[0105] c) Organize into a DataFrame:

[0106] Organize the generated data into a pandas DataFrame, with columns including: speed, acceleration, slope angle, power requirement, and operating condition type generated according to rules.

[0107] d) Print category distribution:

[0108] Use the value_counts() function to print the sample distribution of each working condition type, that is, the frequency of each working condition type.

[0109] (2) Separation of features and labels

[0110] a) Feature and label separation

[0111] In this embodiment, X = df[['velocity', 'acceleration', 'slope angle', 'power demand']] is set: four feature columns (i.e., velocity, acceleration, slope angle, and power demand) are selected from the df data frame and stored in the X variable. X will serve as the input features of the model.

[0112] Set y = df['Operating Condition Type']: Extract the label column (i.e., the vehicle's operating condition type, such as acceleration, constant speed, etc.) from the df data frame and store it in the y variable. y will be used as the target variable of the model.

[0113] b) Label Encoding

[0114] LabelEncoder is used to encode labels (i.e., operating condition types). The purpose of label encoding is to convert category labels (such as "acceleration" and "constant speed") into integers, making it easier for machine learning models to process them.

[0115] Setting le = LabelEncoder(): Initializes the LabelEncoder object.

[0116] Setting y_encoded = le.fit_transform(y): The fit_transform method converts the category label (e.g., string) in label y into a numeric encoding. The converted y_encoded variable is an array of numbers, where each number represents a working condition type.

[0117] c) Category Mapping

[0118] Using `print("Category code:")` will output a message indicating that the mapping between the label and its code will be printed next.

[0119] The function `for idx,label in enumerate(le.classes_)` generates the index (idx) and label value (label) for each class label. `le.classes_` contains all the class labels, and `enumerate` generates the index (idx) and label value (label) for each class label.

[0120] print(f'{idx}:{label}'): Prints the numerical code and corresponding label for each category.

[0121] (3) Divide the training set and the test set

[0122] The feature data X and the label data y_encoded are divided into training and test sets.

[0123] The specific method is as follows: use the `train_test_split` function to split the data, with the training set accounting for 80% of the total dataset and the test set accounting for 20%. By setting `random_state` to a constant value of 42, we ensure that the split results are consistent each time, avoiding fluctuations in model performance due to different data splits.

[0124] In addition, the stratify parameter is used to ensure that the proportion of label classes in the training and test sets is consistent with that in the original dataset, thereby ensuring class balance during model training and avoiding bias caused by class imbalance.

[0125] (4) XGBoost Training

[0126] Using XGBoost for model training, in Python, edit the relevant code, input training data, and train the model to categorize driving modes into five types: acceleration, constant speed, deceleration, climbing, and descending. Input a test set to test the trained XGBoost network, analyze the model's accuracy, and adjust the XGBoost network accordingly.

[0127] First, a training dataset dtrain of type DMatrix is ​​created using the training set data X_train and the label y_train. At the same time, a test dataset dtest is created using the test set data X_test and the label y_test.

[0128] Configure XGBoost parameters; the main parameters are shown in Table 1.

[0129] Table 1

[0130]

[0131] In this configuration: `objective` is set to `multi:softmax`, indicating a multi-class classification task. `num_class` is set to 5, indicating 5 classes. `max_depth` is set to 6, limiting the maximum depth of the tree to prevent overfitting. `eta` is set to 0.1, representing the learning rate. `subsample` is set to 0.8, meaning 80% of the training data is used to train the model each time, enhancing its generalization ability. `colsample_bytree` is set to 0.8, meaning 80% of the features are used for training each tree. `eval_metric` is set to `merror`, meaning the error rate is used as the evaluation metric during training. `seed` is set to 42 to ensure the repeatability of the randomization process.

[0132] After setting the parameters, define the number of training rounds (num_round) as 500, and set a watchlist, which is a monitoring list for the training and testing datasets. Start model training and execute 500 iterations by calling the xgb.train method. Set early_stopping_rounds = 30, indicating that if the evaluation metric on the validation set does not improve within 30 consecutive rounds, training will stop early to avoid unnecessary computation and prevent overfitting.

[0133] (5) Test set prediction

[0134] The trained XGBoost model bst is used to make predictions on the test set.

[0135] In this embodiment, the model predicts the test set dtest using the bst.predict(dtest) method and converts the prediction result into an integer type, storing it in y_pred. LabelEncoder is used to decode the test set label y_test and the prediction result y_pred to convert them back from numeric labels to the original work condition type labels. The le.inverse_transform method first decodes the actual test set label y_test into the actual work condition type y_test_labels, and then decodes the model's prediction result y_pred into the predicted work condition type labels y_pred_labels. These two decoded results, y_test_labels and y_pred_labels, represent the actual work condition type of the test set and the work condition type predicted by the model, respectively.

[0136] (6) Model Evaluation

[0137] The performance of the trained XGBoost model is evaluated.

[0138] In this embodiment, the prediction accuracy of the model is calculated using test set data, and the result is output as a percentage, such as... Figure 2 As shown, a classification report is used to evaluate the model's performance across all categories, outputting precision, recall, F1 score, and sample size. Figure 3 As shown, the confusion matrix of the test set is calculated and visualized as a heatmap to intuitively show the correspondence between the actual categories and the predicted categories.

[0139] In terms of feature importance analysis, the feature importance plotting function built into XGBoost is used to select and display the features most important based on gain, such as... Figure 4 As shown. Finally, save the trained model.

[0140] Specifically, in step S3 of this embodiment, the mathematical model, objective function, and operational constraints of the hydrogen hybrid power generation vehicle energy system are established, as follows:

[0141] (1) Establishing a mathematical model

[0142] A mathematical model of the energy system of a hydrogen-powered hybrid electric vehicle is established. The hydrogen-lithium hybrid power generation system of the hydrogen-lithium hybrid electric vehicle mainly includes a proton exchange membrane fuel cell, a storage battery, and a kinetic energy recovery device.

[0143] a) The mathematical model of a proton exchange membrane fuel cell includes:

[0144]

[0145] In the formula, H FC I is the hydrogen consumption rate. FC For the operating current, n FC Where F represents the number of batteries;

[0146] The reaction rate of a proton exchange membrane fuel cell is represented by the current density j:

[0147]

[0148] In the formula, A cell The reaction area of ​​the battery cell;

[0149] Thermal power output of proton exchange membrane fuel cells Represented as:

[0150]

[0151] In the formula, ηT For the thermal efficiency of a fuel cell, η E For fuel cell electrical efficiency, This indicates the electrical power output of the fuel cell;

[0152] b) When the battery operates in battery pack mode, its total power is expressed as:

[0153]

[0154] In the formula, This represents the total power output of the battery pack. Where n1 is the power of the battery cell, and n2 is the number of rows and columns of the battery, respectively.

[0155] Battery pack current I b as follows:

[0156]

[0157] In the formula, V oc For the open-loop voltage, R b The internal resistance of the battery pack;

[0158] The dynamic characteristics of a battery pack, SOC, can be expressed as:

[0159]

[0160] Among them, SOC t+1 and SOC t Let Q represent the State of Charge (SOC) of the battery pack at time t+1 and time t, respectively, where t is the scheduling time, T is the scheduling period, and 0 ≤ t ≤ T. B For battery cell capacity, I b,t Let I be the current at time t, and I be the current during charging. b,t When it is negative, I is discharged. b,t It is positive.

[0161] c) The kinetic energy recovery device stores the kinetic energy lost during vehicle braking and reuses it in the system:

[0162] Kinetic energy recovery rate formula:

[0163]

[0164] Where η is the recovery efficiency, E rec The recovered energy will be stored in lithium batteries.

[0165] (2) Establish the objective function

[0166] Establish the objective function for the multi-energy flow regulation strategy of the lithium-hydrogen hybrid power generation system. The objective function includes the equipment operating cost and energy storage state cost of the lithium-hydrogen hybrid power generation system:

[0167]

[0168] Among them, C1 and These represent the equivalent hydrogen consumption and power at time t of the equipment in the hydrogen fuel cell drone energy system, respectively, where i is 1 or 2, k1 and k2 are weighting coefficients, and SOC0 and SOC are... T The SOC (State of Charge) of the battery pack is the dynamic characteristic at the beginning and end of the scheduling cycle, respectively.

[0169] (2) Constraint Settings

[0170] Establish operational constraints for the energy system of hydrogen hybrid power generation vehicles, including: electrical balance constraints, upper and lower limits of proton exchange membrane fuel cell power, upper and lower limits of battery state, and battery initialization state constraints.

[0171]

[0172] SOC min ≤SOC t ≤SOC max

[0173] SOC0 = 0.5·(SOC min +SOC max )

[0174] In the formula, These represent the output power of the lithium battery and the power consumption during driving or power generation in the hydrogen hybrid electric vehicle, respectively. This represents the minimum output power of the fuel cell. State of Charge (SOC) represents the maximum output power of the fuel cell. min and SOC max These are the minimum and maximum values ​​of the battery pack's dynamic characteristic SOC, respectively, with SOC0 representing the battery pack's dynamic characteristic value at the initial moment of scheduling.

[0175] Specifically, in step S4 of this embodiment, based on the uncertainty in the power generation mode, reinforcement learning models are established for different driving modes and power generation modes, parameters are set, and the reinforcement learning models are trained using actual data. The method is as follows:

[0176] (1) Uncertainty

[0177] Considering the uncertainties in the power generation mode, reinforcement learning models are established for different driving modes and power generation modes respectively. Parameters are set and reinforcement learning models are trained using actual data and the trained network is stored.

[0178] (2) Reinforcement learning parameter settings

[0179] In the reinforcement learning model, the state space, action space, and reward function are defined as follows:

[0180] a) State space S:

[0181]

[0182] Where t is the time step, At time t, the electrical load is the motor power in driving mode and the electrical load in generator mode. SOC t The state of charge of the lithium battery at time t;

[0183] b) Action Space A:

[0184]

[0185] in, For the electric power of hydrogen fuel cells;

[0186] c) Reward function R:

[0187]

[0188] in, Defined as the scheduling action based on the policy network at time t, representing the excess (greater than 0) or deficiency (less than 0) when the power balance cannot be met, l1 is the weight tuned based on training experience and results, and α is the energy storage coefficient of the reward function. These represent the equipment operating cost and the equipment energy storage status, respectively.

[0189] In this embodiment, the reinforcement learning model parameters, including training parameters and reward function parameters, are shown in Table 2:

[0190] Table 2 Parameters of Reinforcement Learning Model

[0191]

[0192] Furthermore, different parameters are set according to different driving modes and power generation modes, as shown in Table 3.

[0193] Table 3 Reinforcement Learning Parameter Settings in Examples

[0194]

[0195] (3) Set parameters and train the reinforcement learning model using actual data, as follows:

[0196] a) Before the start of each training round, within the typical task scenario and the feasible domain for safe operation of the equipment, add no more than 20% normal random fluctuation to the mean of the load source data, and continuously generate new typical task training scenarios until the end of training.

[0197] b) Perform cyclical training:

[0198] Before the start of each training round, obtain the initial state S0 of the reinforcement learning model and initialize the policy network, evaluation network, and target network.

[0199] For the state s at each time step t The policy network is based on P(s) t |θ P Output action a t A reward r is obtained after performing the action. t And obtain the next state s t+1 Training experience (s) t ,a t ,r t ,s t+1 Save the data to the experience replay pool, update the current state, and complete one round of training; where θ represents the value regression rate and P represents the policy network.

[0200] In this embodiment, the Policy Network is the core module in the reinforcement learning model used for decision-making. It generates a specific action or action distribution based on the current environmental state, guiding the agent to interact with the environment. Through continuous optimization, the Policy Network enables the agent to adopt the policy that yields the maximum long-term benefit under different states. It outputs fixed actions in deterministic policies and parameterized action distributions in stochastic policies to enhance exploration capabilities.

[0201] The value network is used to evaluate the quality of a policy by calculating the value (i.e., the expected total reward) of a state or state-action pair. In this embodiment, the value network takes the form of a Q-function, takes a state and an action as input, and outputs the expected reward of that action in that state. It serves as the basis for training the policy network, providing the target signal and gradient information needed for policy optimization.

[0202] The target network is a stable copy of the policy or evaluation network. Its main function is to provide a smooth and stable reference for calculating the target value during training, reducing gradient oscillations and training instability. The target network parameters are synchronized with the main network parameters through "soft updates," ensuring that the learning objective does not diverge due to frequent changes in the main network, thereby improving the convergence and performance of the reinforcement learning model.

[0203] c) Reinforcement learning model parameter update: When the experience parameters in the experience replay pool meet the batch size, the network parameters are updated at each time step, the policy network and the evaluation network are updated, and the target network is updated.

[0204] Furthermore, the energy flow scheduling was explored based on different driving and power generation modes. For example... Figure 5 As shown, the cumulative energy dispatch rewards for acceleration, constant speed, deceleration, hill climbing, and downhill driving modes correspond to Run-1, Run-2, Run-3, Run-4, and Run-5, respectively. The cumulative energy dispatch rewards for power generation mode are as follows: Figure 6 As shown in the figure, the cumulative rewards converge. The scheduling results for various driving modes are as follows: Figures 7 to 12 As shown, the energy dispatch results are all within a reasonable range, and the lithium battery plays a good regulatory role. The dispatch results are consistent with the actual situation.

[0205] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for multi-energy flow regulation in a hybrid hydrogen-lithium power generation system with virtual-real interaction, characterized in that, Includes the following steps: S1. Based on the driving characteristics of hydrogen hybrid power generation vehicles, establish force and power models under different driving modes. The driving modes of hydrogen-lithium hybrid power generation vehicles include: acceleration, constant speed, deceleration, climbing, and descending. S2. Use an extreme gradient ascent network to classify driving modes and collect driving parameters of the hydrogen hybrid electric vehicle and input them into the trained extreme gradient ascent network for identification. S3. Establish the mathematical model, objective function, and operational constraints of the hydrogen hybrid power generation vehicle energy system; S4. Based on the uncertainty in the power generation mode, reinforcement learning models for different driving modes and power generation modes are established respectively. Parameters are set and reinforcement learning models are trained using actual data. S5. Based on the trained reinforcement learning model, energy scheduling of hydrogen-lithium hybrid power generation vehicles is carried out.

2. The method for multi-energy flow regulation of a hydrogen-lithium hybrid power generation system with virtual-real interaction according to claim 1, characterized in that: The force model is as follows when the hydrogen-lithium hybrid electric vehicle accelerates: The power model is as follows: P a (t)=T a (t)·v(t); Where W is the gravity of the hydrogen-lithium hybrid power generation vehicle, N is the support force provided by the ground, and T a (t) represents the traction force during acceleration, μ is the coefficient of friction, m is the total mass of the hydrogen-lithium hybrid power generation vehicle, and a a (t) represents the acceleration of the hydrogen-lithium hybrid electric vehicle during acceleration, P a (t) represents the real-time power during acceleration, and v(t) represents the real-time speed of the hydrogen-lithium hybrid power generation vehicle; The force model is as follows when the hydrogen-lithium hybrid power generation vehicle decelerates: The power model is as follows: P m (t)=T m (t)·v(t); Among them, T m (t) represents the traction force during deceleration, a m (t) represents the acceleration of the lithium-hydrogen hybrid power generation vehicle during deceleration, P m (t) represents the real-time power during deceleration.

3. The method for multi-energy flow regulation of a hydrogen-lithium hybrid power generation system with virtual-real interaction according to claim 2, characterized in that: When the hydrogen-lithium hybrid power generation vehicle is moving at a constant speed, the force model is as follows: The power model is as follows: P c (t)=T c (t)·v(t); Among them, T c (t) represents the traction force when the velocity is constant, P c (t) represents the real-time power at a constant speed.

4. The method for multi-energy flow regulation of a hydrogen-lithium hybrid power generation system with virtual-real interaction according to claim 2, characterized in that: When the hydrogen-lithium hybrid power generation vehicle climbs a hill, the force model is as follows: The power model is as follows: P cl (t)=T cl (t)·v(t); Among them, T cl (t) represents the traction force during the uphill climb, θ represents the angle of elevation during the climb, and P cl (t) represents the real-time power during the climb; The force model of the hydrogen-lithium hybrid power generation vehicle when going downhill is as follows: The power model is as follows: P do (t)=T do (t)·v(t); Among them, T do (t) represents the traction force during downhill driving, γ represents the downhill angle of depression, and P do (t) represents the real-time power during downhill.

5. The method for multi-energy flow regulation of a hydrogen-lithium hybrid power generation system with virtual-real interaction according to claim 2, characterized in that: In step S2, the XGBoost algorithm, based on the extreme gradient ascent machine learning method, is used to construct multiple decision trees for driving mode classification, including the following sub-steps: S2.1 Collect driving mode data, including: hydrogen-lithium hybrid vehicle speed v, acceleration a, uphill tilt angle θ, downhill tilt angle γ, fuel cell operating current I. FC Fuel cell operating voltage U FC lithium battery current I Li Lithium battery voltage U Li ; S2.2 Data Cleaning: Preprocess the collected driving mode data, including handling missing values ​​and outliers to ensure data reliability; S2.3 Constructing an extreme gradient ascent classification model: Based on the XGBoost network, using the extreme gradient ascent algorithm, multiple decision trees are created through the ensemble learning method Bagging. Each tree is trained based on different random subsamples and feature subsets, and the prediction result is determined by majority voting or average value. This includes: generating simulated data, separating features and labels, dividing the training and test sets, training the XGBoost network, predicting on the test set, and evaluating the model. S2.4 Training and Testing of Extreme Gradient Ascent Classification Models: The XGBoost network is trained using the input training set data. The driving modes are divided into five categories: acceleration, constant speed, deceleration, climbing, and descending. The trained XGBoost network is then tested using the input test set data, and the XGBoost network is adjusted based on the model accuracy.

6. The method for multi-energy flow regulation of a hydrogen-lithium hybrid power generation system with virtual-real interaction according to claim 5, characterized in that: In step S3, the hydrogen hybrid power generation vehicle energy system includes: a proton exchange membrane fuel cell, a storage battery, and a kinetic energy recovery device. Establishing a mathematical model of the hydrogen hybrid power generation vehicle energy system includes the following sub-steps: S3.1 Construct a mathematical model for a proton exchange membrane fuel cell, expressed as follows: In the formula, H FC I is the hydrogen consumption rate. FC For the operating current, n FC Where F represents the number of batteries; The reaction rate of a proton exchange membrane fuel cell is represented by the current density j: In the formula, A cell The reaction area of ​​the battery cell; Thermal power output of proton exchange membrane fuel cells Represented as: In the formula, η T For the thermal efficiency of a fuel cell, η E For fuel cell electrical efficiency, This indicates the electrical power output of the fuel cell; S3.2 The battery operates in battery pack mode, and the total power is expressed as: In the formula, This represents the total power output of the battery pack. Where n1 is the power of the battery cell, and n2 is the number of rows and columns of the battery, respectively. Battery pack current I b The calculation is as follows: In the formula, V oc R is the open-loop voltage. b The internal resistance of the battery pack; The dynamic characteristics of the battery pack, SOC, are expressed as follows: Among them, SOC t+1 and SOC t Let Q represent the State of Charge (SOC) of the battery pack at time t+1 and time t, respectively, where t is the scheduling time, T is the scheduling period, and 0 ≤ t ≤ T. B For battery cell capacity, I b,t Let I be the current at time t, and I be the current during charging. b,t When it is negative, I is discharged. b,t It is positive; S3.3 The kinetic energy recovery device stores and reuses the kinetic energy lost during vehicle braking. The formula for the kinetic energy recovery rate is: Where η is the recovery efficiency, E rec The recovered energy will be stored in lithium batteries.

7. The method for multi-energy flow control of a hydrogen-lithium hybrid power generation system with virtual-real interaction according to claim 6, characterized in that: In step S3, the objective function of the hydrogen hybrid power generation vehicle energy system is established, which includes the equipment operating cost and energy storage state cost of the hydrogen-lithium hybrid power generation system, as shown in the following formula: Among them, C1 and These represent the equivalent hydrogen consumption and power at time t of the equipment in the hydrogen fuel cell drone energy system, respectively, where i is 1 or 2, k1 and k2 are weighting coefficients, and SOC0 and SOC are... T The SOC (State of Charge) of the battery pack is the dynamic characteristic at the beginning and end of the scheduling cycle, respectively.

8. The method for multi-energy flow regulation of a hydrogen-lithium hybrid power generation system with virtual-real interaction according to claim 7, characterized in that: In step S3, operational constraints are established for the hydrogen hybrid power generation vehicle's energy system, including: electrical balance constraints, upper and lower limits for proton exchange membrane fuel cell power, upper and lower limits for battery state, and battery initialization state constraints, expressed as: SOC min ≤SOC t ≤SOC max SOC0=0.5·(SOC min +SOC max ) In the formula, These represent the output power of the lithium battery and the power consumption during driving or power generation in the hydrogen hybrid electric vehicle, respectively. This represents the minimum output power of the fuel cell. State of Charge (SOC) represents the maximum output power of the fuel cell. min and SOC max These are the minimum and maximum values ​​of the battery pack's dynamic characteristic SOC, respectively, with SOC0 representing the battery pack's dynamic characteristic value at the initial moment of scheduling.

9. The method for multi-energy flow regulation of a hydrogen-lithium hybrid power generation system with virtual-real interaction according to claim 8, characterized in that: In step S4, reinforcement learning models for different driving modes and power generation modes are established respectively, defining the state space, action space, and reward function, as follows: Define the state space S: Where t is the time step, At time t, the electrical load is the motor power in driving mode and the electrical load in generator mode. SOC t The state of charge of the lithium battery at time t; Define action space A: in, For the electric power of hydrogen fuel cells; Reward function R: in, Let be the scheduling action based on the policy network at time t, representing the excess or deficiency when the power balance cannot be met, l1 be the weight tuned based on training experience and results, and α be the energy storage coefficient of the reward function. These represent the equipment operating cost and the equipment energy storage status, respectively.

10. The method for multi-energy flow regulation of a hydrogen-lithium hybrid power generation system with virtual-real interaction according to claim 9, characterized in that: In step S4, parameters are set and a reinforcement learning model is trained using actual data, as follows: S4.1 Before the start of each training round, within the typical task scenario and the feasible domain for safe operation of the equipment, add no more than 20% normal random fluctuation to the mean of the load source data, and continuously generate new typical task training scenarios until the end of training. S4.2, Perform cyclical training: Before the start of each training round, obtain the initial state S0 of the reinforcement learning model and initialize the policy network, evaluation network, and target network. For the state s at each time step t The policy network is based on P(s) t |θ P Output action a t A reward r is obtained after performing the action. t And obtain the next state s t+1 Training experience (s) t ,a t ,r t ,s t+1 Save the data to the experience replay pool, update the current state, and complete one round of training; where θ represents the value regression rate and P represents the policy network. S4.3 Reinforcement learning model parameter update: When the experience parameters in the experience replay pool meet the batch size, the network parameters are updated at each time step, the policy network and the evaluation network are updated, and the target network is updated.

Images