Hydrogen energy fuel cell energy management system and method based on reinforcement learning

Abstract

The invention relates to the technical field of battery energy management, in particular to a hydrogen energy fuel cell energy management system and method based on reinforcement learning, and the system comprises an environment module and a fuel cell, which are used for converting hydrogen into electric energy and heat energy; adjusting power output according to an instruction of the energy management module; the energy storage system is electrically connected with the fuel cell, and the energy management module is in communication connection with the environment module, the fuel cell and the energy storage system and is used for receiving environment information sent by the environment module; based on a reinforcement learning algorithm, power output and start-stop operation of the fuel cell are managed and optimized in real time; the working states of the fuel cell and the energy storage system are monitored in real time, and power output and start-stop instructions are adjusted according to different conditions; the whole vehicle VCU is in communication connection with the fuel cell, the energy storage system and the energy management module and is used for transmitting and collecting data of the whole system; the work of each module is coordinated, the overall operation efficiency of the system is ensured, and the energy utilization efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of battery energy management, in particular to a hydrogen fuel cell energy management system and method based on reinforcement learning. Background Art

[0002] With the improvement of environmental protection awareness and the development of renewable energy technologies, hydrogen fuel cells, as a clean and efficient energy solution, have gradually received extensive attention in fields such as transportation and distributed power generation. However, the energy management of hydrogen fuel cell systems has always been a key challenge in the popularization and application of this technology.

[0003] Traditional hydrogen fuel cell energy management systems usually adopt rule-based control strategies or simple optimization algorithms. These methods often perform poorly when dealing with complex and variable operating environments and are difficult to achieve the optimal operation of the system. For example, although the rule-based control strategy is simple to implement, it lacks adaptability and cannot effectively respond to changes in energy demand under different working conditions. And although simple optimization algorithms can improve system efficiency in certain specific situations, they often can only achieve local optimization and are difficult to maintain high efficiency in long-term operation.

[0004] In addition, existing energy management systems usually treat fuel cells, energy storage devices, and load demands as independent parts, lacking overall coordination and optimization. This method ignores the mutual influence between the various parts of the system, resulting in unreasonable energy distribution and low overall system efficiency. At the same time, existing systems also have deficiencies in considering the influence of environmental factors and are difficult to adjust energy management strategies in a timely manner according to external conditions such as real-time traffic conditions and weather changes.

[0005] Another significant problem is that existing systems have difficulties in balancing multiple objectives such as energy efficiency, equipment life, and environmental impact. Most methods tend to overly pursue short-term energy efficiency improvement while ignoring the protection of the life of fuel cells and energy storage devices and the consideration of the overall environmental impact. This unbalanced optimization strategy may lead to premature aging of system components and more frequent maintenance requirements, increasing long-term operating costs. Summary of the Invention

[0006] In view of the above problems, the present invention proposes a hydrogen fuel cell energy management system and method based on reinforcement learning. The system realizes adaptive learning and optimization of complex and variable environments by introducing a reinforcement learning algorithm. The system integrates multiple aspects such as environmental information collection, fuel cell control, energy storage system management, and load demand analysis, achieving comprehensive energy coordination and optimization.

[0007] The present invention proposes a hydrogen fuel cell energy management system based on reinforcement learning, comprising:

[0008] An environmental module, for:

[0009] Collect environmental information such as traffic flow, road congestion, weather data, and prediction data;

[0010] Transmit the environmental information to the energy management module;

[0011] A fuel cell, for:

[0012] Convert hydrogen into electrical energy and heat energy;

[0013] Adjust the power output according to the instructions of the energy management module;

[0014] An energy storage system, electrically connected to the fuel cell, for:

[0015] Store the excess electrical energy generated by the fuel cell;

[0016] Perform charge and discharge operations according to the instructions of the energy management module;

[0017] An energy management module, communicatively connected to the environmental module, fuel cell, and energy storage system, for:

[0018] Receive the environmental information sent by the environmental module;

[0019] Based on the reinforcement learning algorithm, manage and optimize the power output and start / stop operations of the fuel cell in real time;

[0020] Monitor the working states of the fuel cell and the energy storage system in real time, and adjust the power output and start / stop instructions according to different situations;

[0021] A vehicle VCU, communicatively connected to the fuel cell, energy storage system, and energy management module, for:

[0022] Transmit and collect data of the entire system;

[0023] Coordinate the work of each module to ensure the overall operation efficiency of the system.

[0024] Preferably, the energy management module includes:

[0025] A communication unit, for collecting the data transmitted from the vehicle VCU and sending control signals, and the data includes the SOC of the power battery, traffic conditions, and weather condition data;

[0026] A calculation unit, connected to the communication unit, for performing real-time calculations on the power generation power and start / stop of the fuel cell and performing real-time calculations on the working state of the energy storage system;

[0027] The battery monitoring unit, connected to the computing unit, is used to monitor in real time the power generation power, start / stop signal, remaining power, operating temperature, and hydrogen storage amount of the fuel cell.

[0028] Preferably, the energy management module further includes:

[0029] The safety prediction unit is used to:

[0030] Preprocess the relevant parameter data of the fuel cell and the energy storage system;

[0031] Perform safety prediction calculations based on the preprocessed data;

[0032] The alarm unit, connected to the safety prediction unit, is used to emit an alarm signal when a potential safety problem is detected.

[0033] Preferably, the energy management module further includes:

[0034] The multi-objective optimization unit is used to:

[0035] Optimize the working efficiency and service life of the fuel cell;

[0036] Optimize the charge and discharge power and service life of the energy storage system;

[0037] Optimize the environmental impact of the system, including reducing the hydrogen consumption of the fuel cell.

[0038] Preferably, the management strategy of the energy storage system includes:

[0039] Dynamically select the slow charging or fast charging mode according to the relationship between the actual SOC, the maximum SOC, and the minimum SOC;

[0040] When the actual SOC is less than the minimum SOC, switch to the fast charging mode;

[0041] When the actual SOC is greater than the maximum SOC, stop charging.

[0042] Preferably, the energy management module uses an LSTM neural network as the policy network for reinforcement learning. The parameters of the LSTM neural network include:

[0043] The input dimension is 15, the output dimension is 3, the number of hidden layers is 3, and each layer has 80 nodes.

[0044] Preferably, the energy management module further includes:

[0045] The reward function calculation unit is used to calculate the reward function r(s,a) based on the Gaussian function, where s represents the system state and a represents the system action;

[0046] A strategy optimization unit, which is used to calculate the reward update function Q(st, a) through the Monte Carlo method to continuously optimize the energy management strategy.

[0047] Preferably, it further includes:

[0048] A load demand grading unit, which is used to divide the load demand into different levels;

[0049] A dynamic power distribution unit, which is connected to the load demand grading unit and is used to dynamically adjust the output power of the fuel cell and the energy storage system according to different load demand levels.

[0050] Preferably, when the system is applied to a heavy truck, it further includes:

[0051] A heavy truck power network, including an AC power network and a DC power network, which is used to provide stable power transmission and distribution;

[0052] A heavy truck energy optimization module, which is connected to the energy management module and is used to optimize the power distribution according to the specific operating environment and requirements of the heavy truck to ensure the minimum service life and environmental impact of the fuel cell.

[0053] A hydrogen fuel cell energy management method based on reinforcement learning includes the following steps:

[0054] S1, collecting environmental information such as traffic flow, road congestion, weather data, and prediction data through an environment module;

[0055] S2, receiving the environmental information through an energy management module and real-time monitoring the working states of the fuel cell and the energy storage system;

[0056] S3, based on the reinforcement learning algorithm, the energy management module performs the following operations:

[0057] Constructing sample pairs containing system states and actions;

[0058] Using an LSTM neural network as a policy network to train the sample pairs;

[0059] Calculating the reward function r(s, a) based on a Gaussian function;

[0060] Calculating the reward update function Q(s t , a t ) through the Monte Carlo method;

[0061] Continuously optimizing the energy management strategy;

[0062] S4, according to the optimized energy management strategy, the energy management module real-time adjusts the power output and start / stop operations of the fuel cell, and at the same time controls the charge and discharge operations of the energy storage system.

[0063] S5. Coordinate the operation of the fuel cell, energy storage system, and energy management module through the vehicle VCU to ensure the overall operating efficiency of the system;

[0064] S6. Repeat steps S1 to S5 to continuously optimize the energy management strategy of the system.

[0065] The beneficial effects of the present invention are mainly reflected in the following aspects:

[0066] While improving the energy utilization efficiency, the system of the present invention also fully considers the equipment life and environmental impact, achieving a balanced optimization of multiple objectives. Through the collection and analysis of real-time environmental information, the system can predict changes in energy demand and make reasonable energy allocation decisions in advance. The application of the reinforcement learning algorithm enables the system to continuously learn and improve, and continuously optimize the energy management strategy.

[0067] From a macroscopic perspective, the system of the present invention significantly improves the overall performance and adaptability of the hydrogen fuel cell system. The system can maintain efficient operation under various complex working conditions, greatly improving the energy utilization rate. At the same time, through intelligent energy allocation and control, the system extends the service life of the fuel cell and energy storage device, and reduces the long-term operation cost.

[0068] From a microscopic level analysis, a high degree of coordination and complementarity is achieved among the various modules of the present invention. The real-time data provided by the environment module provides an important basis for the decision-making of the energy management module, enabling the system to respond to changes in energy demand in advance. Under the coordination of the energy management module, the fuel cell and energy storage system achieve optimal power distribution, meeting the load demand while ensuring the efficient operation of each component.

[0069] It is particularly worth mentioning that the present invention successfully solves the contradiction among efficiency, life, and environmental impact through the multi-objective optimization unit. The system no longer simply pursues short-term efficiency, but on the basis of comprehensively considering multiple factors, achieves long-term optimal operation. This balanced strategy not only improves the overall performance of the system, but also significantly reduces the environmental impact, making an important contribution to the sustainable development of hydrogen energy technology.

[0070] Generally speaking, the system of the present invention successfully solves multiple key problems faced by the traditional hydrogen fuel cell energy management system through innovative technical solutions. It not only improves the energy utilization efficiency of the system, but also realizes an intelligent and adaptive operation mode, providing strong support for the popularization of hydrogen energy technology in various application scenarios. The successful application of this system will make an important contribution to the development of clean energy technology and environmental protection. Brief Description of the Drawings

[0071] Figure 1This is the logic block diagram of the overall system of the present invention.

[0072] Figure 2 This is the logic block diagram of the energy management module of the present invention.

[0073] Figure 3 This is the working flowchart of the system of the present invention.

[0074] Figure 4 This is the system structure diagram under the application scenario of heavy trucks of the present invention. Detailed implementation manners

[0075] Refer to Figures 1-4 The present invention provides a hydrogen energy fuel cell energy management system and method based on reinforcement learning. The system includes an environment module 1, a fuel cell 2, an energy storage system 3, an energy management module 4, and a vehicle VCU 5. These modules work together to achieve intelligent energy management.

[0076] Preferably, the environment module 1 is used to collect environmental information such as traffic flow, road congestion, weather data, and prediction data, and transmit this information to the energy management module 4. For example, the environment module 1 can collect data in real time through devices such as GPS, traffic cameras, and weather sensors. This environmental information is crucial for predicting energy demand and optimizing management strategies.

[0077] The fuel cell 2 of the present invention is responsible for converting hydrogen into electrical energy and heat energy, and adjusting the power output according to the instructions of the energy management module 4. In one embodiment, the rated power of the fuel cell 2 can be 100 kW, and the operating temperature range is 60 - 80 °C. This adjustability of the fuel cell 2 enables the system to flexibly respond to different energy demand situations.

[0078] The energy storage system 3 is electrically connected to the fuel cell 2, and is used to store the excess electrical energy generated by the fuel cell 2, and perform charge and discharge operations according to the instructions of the energy management module 4. Preferably, the energy storage system 3 can adopt a lithium-ion battery with a capacity of 50 kWh, a maximum charging power of 60 kW, and a maximum discharging power of 100 kW. This configuration can effectively balance the energy supply and demand of the system and improve the overall energy utilization efficiency.

[0079] The energy management module 4 is the core of the present invention. It is communicatively connected to the environment module 1, the fuel cell 2, and the energy storage system 3. This module receives the environmental information sent by the environment module 1, and based on the reinforcement learning algorithm, manages and optimizes the power output and start-stop operations of the fuel cell 2 in real time. At the same time, it also monitors the working states of the fuel cell 2 and the energy storage system 3 in real time, and adjusts the power output and start-stop instructions according to different situations.

[0080] In a preferred embodiment of the present invention, the energy management module 4 adopts an LSTM (Long Short-Term Memory) neural network as the policy network for reinforcement learning. The parameters of the LSTM network can be set as follows: the input dimension is 15, the output dimension is 3, the number of hidden layers is 3, and each layer has 80 nodes. This network structure can effectively handle energy management problems related to time series, improving the prediction accuracy and decision-making efficiency of the system.

[0081] The vehicle VCU 5 is communicatively connected to the fuel cell 2, the energy storage system 3, and the energy management module 4, and is responsible for transmitting and collecting data of the entire system, coordinating the work of each module, and ensuring the overall operation efficiency of the system. For example, the vehicle VCU 5 can collect the working data of each module at a frequency of 100 Hz and send control instructions to each module at a frequency of 10 Hz.

[0082] Furthermore, the energy management module 4 includes a communication unit 41, a calculation unit 42, and a battery monitoring unit 43. The communication unit 41 is used to collect data transmitted from the vehicle VCU 5 and send control signals, and these data include the SOC of the power battery, traffic conditions, weather condition data, etc. The calculation unit 42 is connected to the communication unit 41 and performs real-time calculations on the power generation power and start / stop of the fuel cell 2, as well as real-time calculations on the working state of the energy storage system 3. The battery monitoring unit 43 is connected to the calculation unit 42 and monitors in real time parameters such as the power generation power of the fuel cell 2, start / stop signals, remaining power, operating temperature, and hydrogen storage amount.

[0083] The system of the present invention further includes a safety prediction unit 44 and an alarm unit 45. The safety prediction unit 44 preprocesses the relevant parameter data of the fuel cell 2 and the energy storage system 3, and performs safety prediction calculations based on the preprocessed data. The alarm unit 45 is connected to the safety prediction unit 44 and issues an alarm signal when detecting potential safety problems.

[0084] In the safety prediction process, the present invention adopts an innovative prediction algorithm. For example, the following formula can be used to calculate the safety prediction factor:

[0085]

[0086] where S is the safety prediction factor, w i is the weight of the i-th parameter, and p i is the normalized value of the i-th parameter. When S is less than a preset threshold (e.g., 0.8), the system will trigger an alarm.

[0087] This safety prediction and alarm mechanism significantly improves the safety and reliability of the system. By promptly warning of potential safety hazards, the system can take preventive measures to avoid serious safety accidents.

[0088] Through the collaborative work of the above-mentioned modules, the energy management system of the present invention realizes intelligent energy management based on reinforcement learning. This system can optimize the working strategies of fuel cells and energy storage systems in real time according to complex and variable environmental factors and system states, greatly improving the energy utilization efficiency, extending the service life of equipment, and reducing the environmental impact of the system at the same time.

[0089] The energy management module 4 of the present invention further includes a multi-objective optimization unit 46, which is used to optimize the working efficiency and service life of the fuel cell 2, optimize the charge and discharge power and service life of the energy storage system 3, and optimize the environmental impact of the system, including reducing the hydrogen consumption of the fuel cell 2. This multi-objective optimization strategy enables the system not only to improve the energy utilization efficiency, but also to take into account the equipment life and environmental protection, reflecting the comprehensive and sustainable design concept.

[0090] In a preferred embodiment of the present invention, the multi-objective optimization unit 46 adopts a weighted sum method to balance different optimization objectives. Its objective function can be expressed as:

[0091] F = w 1 f 1 + w 2 f 2 + w 3 f 3 ,

[0092] wherein, F is the total objective function, f 1 , f 2 , f 3 represent the efficiency objective, the life objective and the environmental objective respectively, and w 1 , w 2 , w 3 are the corresponding weight coefficients. Preferably, w 1 = 0.4, w 2 = 0.3, w 3 = 0.3 can be set to achieve a balance among efficiency, life and environmental impact.

[0093] The energy storage system 3 of the present invention adopts a refined management strategy. Specifically, the energy storage system 3 dynamically selects the slow charging or fast charging mode according to the actual SOC (State of Charge), the relationship between the maximum SOC and the minimum SOC. When the actual SOC is less than the minimum SOC, the system will automatically switch to the fast charging mode; when the actual SOC is greater than the maximum SOC, the system will stop charging. This dynamic adjustment strategy effectively extends the service life of the energy storage device and improves the energy utilization efficiency of the entire system at the same time.

[0094] In an embodiment of the present invention, the minimum SOC can be set to 20%, and the maximum SOC can be set to 80%. When the actual SOC is between 20% and 80%, the system adopts a slow charging mode, and the charging current can be set to 0.2C (C is the rated capacity of the battery). When the actual SOC is lower than 20%, the system switches to the fast charging mode, and the charging current can be increased to 1C. This setting can achieve a balance between protecting the battery life and meeting the emergency energy demand.

[0095] The energy management module 4 of the present invention uses an LSTM (Long Short-Term Memory) neural network as the policy network for reinforcement learning. The parameters of the LSTM network include: the input dimension is 15, the output dimension is 3, the number of hidden layers is 3, and each layer has 80 nodes. This network structure design fully considers the complexity and time series characteristics of the energy management problem.

[0096] Specifically, the 15-dimensional input can include the environmental information at the current moment (such as traffic flow, weather conditions, etc.), the system state (such as the output power of the fuel cell, the SOC of the energy storage system, etc.), and historical data. The 3-dimensional output corresponds to the three main control decisions of the system: the output power of the fuel cell, the charge and discharge power of the energy storage system, and the selection of the system working mode.

[0097] The 3-layer hidden layer structure enables the network to learn more complex features and patterns, and 80 nodes in each layer provide sufficient model capacity to handle complex energy management problems. Preferably, ReLU (RectifiedLinear Unit) can be used as the activation function to accelerate the training process of the network and alleviate the gradient vanishing problem.

[0098] The energy management module 4 of the present invention further includes a reward function calculation unit 47 and a policy optimization unit 48. The reward function calculation unit 47 is used to calculate the reward function r(s,a) based on the Gaussian function, where s represents the system state and a represents the system action. The policy optimization unit 48 calculates the reward update function Q(st,a) through the Monte Carlo method to continuously optimize the energy management policy.

[0099] In a preferred embodiment of the present invention, the reward function r(s,a) can be expressed as:

[0100]

[0101] where, E t is the system energy consumption at the current moment, For the ideal energy consumption, σ is an adjustable parameter. This reward design based on the Gaussian function enables the system to obtain a higher reward when approaching the ideal energy consumption, thus guiding the system to converge to the optimal energy management strategy. The policy optimization unit 48 uses the Monte Carlo method to calculate Q(s t ,a t ), and its update formula can be expressed as:

[0102] Q(s t ,a t ) = Q(s t ,a t ) + α[G t - Q(s t ,a t )],

[0103] where α is the learning rate, and G t is the discounted return starting from time t. This method can effectively handle the long-term dependencies in the energy management problem and improve the long-term optimization effect of the system.

[0104] Through the above design, the system of the present invention can continuously learn and adapt to the complex and changeable operating environment, continuously optimize the energy management strategy, so as to achieve efficient, reliable and environmentally friendly energy management of hydrogen fuel cells. The system of the present invention further includes a load demand grading unit 49 and a dynamic power distribution unit 50, and the collaborative work of these two units enables the system to manage the energy distribution more precisely.

[0105] The load demand grading unit 49 is used to divide the load demand into different levels. In a preferred embodiment of the present invention, the load demand can be divided into three levels: low load, medium load and high load. Specifically, the following formula can be used for load grading:

[0106] The following formula can be used for load grading:

[0107]

[0108] where L i represents the load level at the i-th moment, P i represents the power demand at this moment, and P max represents the maximum power output of the system. This grading method can help the system better cope with different intensities of energy demand.

[0109] The dynamic power distribution unit 50 is connected to the load demand grading unit 49 and is used to dynamically adjust the output powers of the fuel cell 2 and the energy storage system 3 according to different load demand levels. Preferably, the following strategy can be adopted:

[0110] 1. Under low load conditions, the energy storage system 3 is preferentially used for power supply, and the fuel cell 2 is maintained at the minimum output or standby state.

[0111] 2. Under medium load conditions, the fuel cell 2 provides the basic load, and the energy storage system 3 is used for peak shaving.

[0112] 3. Under high load conditions, the fuel cell 2 and the energy storage system 3 both output the maximum power.

[0113] This dynamic power distribution strategy can effectively balance the system efficiency and the equipment life, while meeting the load requirements of different intensities.

[0114] The system of the present invention is not only applicable to general hydrogen fuel cell systems, but also a special application scheme for heavy trucks is designed. In the heavy truck application scenario, the system of the present invention further includes a heavy truck power network 51 and a heavy truck energy optimization module 52. In addition, it can be used for logistics, buses, and so on.

[0115] The heavy truck power network 51 includes an AC power network and a DC power network, which are used to provide stable power transmission and distribution. In an embodiment of the present invention, the voltage level of the AC power network can be set to 380V / 50Hz, and the voltage level of the DC power network can be set to 750V. This dual-network design can meet the requirements of different electrical equipment on heavy trucks and improve the flexibility and compatibility of the system.

[0116] The heavy truck energy optimization module 52 is connected to the energy management module 4 and is used to optimize the power distribution according to the specific operating environment and requirements of the heavy truck, ensuring the minimum service life and environmental impact of the fuel cell 2. In the heavy truck application, the energy optimization module 52 needs to consider special factors such as sailing status, sea conditions, wind direction, etc. For example, in severe sea conditions, the system may need to reserve more backup energy to cope with emergencies.

[0117] Preferably, the heavy truck energy optimization module 52 can adopt a Model Predictive Control (MPC) algorithm to achieve energy optimization. The objective function of the MPC algorithm can be expressed as:

[0118]

[0119] e(k)∥ 2 ),

[0120] where P fc (k) and respectively represent the actual and desired fuel cell output power, SOC(k and SOC * respectively represent the actual and desired state of charge of the energy storage system, e(k) represents the load power prediction error, w 1, w 2 and w 3 are weight coefficients. By minimizing this objective function, the system can optimize the operating state of the fuel cell and the usage of the energy storage system as much as possible while meeting the load demand. Note that the SOC in the formula * should be understood as the set value of the desired state of charge of the energy storage system. This optimization strategy can effectively improve the overall efficiency and stability of the system.

[0121] Finally, the present invention also provides a hydrogen energy fuel cell energy management method based on reinforcement learning. The method includes the following steps:

[0122] S1. Collect environmental information such as traffic flow, road congestion, weather data, and prediction data through the environmental module 1;

[0123] S2. Receive the environmental information through the energy management module 4 and monitor the operating states of the fuel cell 2 and the energy storage system 3 in real time;

[0124] S3. Based on the reinforcement learning algorithm, the energy management module 4 performs the following operations:

[0125] Construct sample pairs containing system states and actions;

[0126] Use the LSTM neural network as the policy network to train the sample pairs;

[0127] Calculate the reward function r(s, a) based on the Gaussian function;

[0128] Calculate the reward update function Q(s t , a t ) through the Monte Carlo method;

[0129] Continuously optimize the energy management strategy;

[0130] S4. According to the optimized energy management strategy, the energy management module 4 adjusts the power output and start / stop operations of the fuel cell 2 in real time, and at the same time controls the charge and discharge operations of the energy storage system 3;

[0131] S5. Coordinate the operations of the fuel cell 2, the energy storage system 3, and the energy management module 4 through the vehicle VCU 5 to ensure the overall operating efficiency of the system;

[0132] S6. Repeat steps S1 to S5 to continuously optimize the energy management strategy of the system.

[0133] This method realizes the intelligentization and high-efficiency of the energy management of the hydrogen fuel cell system by continuously learning and adapting to the complex and changeable operating environment. In particular, this method can dynamically adjust the energy management strategy according to the real-time environmental information and system status, thereby improving the energy utilization efficiency while extending the service life of system components and reducing the environmental impact.

[0134] It should be noted that the above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A hydrogen fuel cell energy management system based on reinforcement learning, characterized in that: include: Environment modules for: Collect environmental information such as traffic flow, road congestion, weather data and forecast data; transmitting the environmental information to an energy management module; Fuel cells for: converting hydrogen into electricity and heat; Adjust power output according to instructions from the energy management module; An energy storage system, electrically connected to the fuel cell, is used to: storing excess electrical energy generated by the fuel cell; Perform charging and discharging operations according to the instructions of the energy management module; The energy management module is in communication with the environment module, the fuel cell and the energy storage system, and is used to: Receiving environmental information sent by the environmental module; Based on a reinforcement learning algorithm, the power output and start-stop operation of the fuel cell are managed and optimized in real time; Monitor the working status of the fuel cell and energy storage system in real time, and adjust power output and start and stop instructions according to different situations; The vehicle VCU is connected to the fuel cell, energy storage system and energy management module for: Transmit and collect data of the entire system; Coordinate the work of each module to ensure the overall operating efficiency of the system.

2. The system according to claim 1, characterized in that The energy management module comprises: The communication unit is used to collect data transmitted from the vehicle VCU and send control signals, wherein the data includes power battery SOC, traffic conditions, and weather conditions; A calculation unit, connected to the communication unit, for performing real-time calculation of the power generation and start / stop of the fuel cell and real-time calculation of the working state of the energy storage system; The battery monitoring unit is connected to the calculation unit and is used to monitor the power generation power, start / stop signal, remaining power, operating temperature and hydrogen storage capacity of the fuel cell in real time.

3. The system according to claim 1, characterized in that The energy management module also includes: Security prediction unit for: Preprocessing relevant parameter data of the fuel cell and energy storage system; Perform safety prediction calculations based on preprocessed data; An alarm unit is connected to the safety prediction unit and is used to send out an alarm signal when a potential safety problem is detected.

4. The system according to claim 1, characterized in that The energy management module also includes: Multi-objective optimization unit for: Optimizing the operating efficiency and service life of the fuel cell; Optimizing the charging and discharging power and service life of the energy storage system; Optimizing the environmental impact of the system includes reducing the hydrogen consumption of the fuel cell.

5. The system according to claim 1, characterized in that The management strategy of the energy storage system includes: Dynamically select slow charging or fast charging mode based on the relationship between actual SOC, maximum SOC and minimum SOC; When the actual SOC is less than the minimum SOC, switching to the fast charging mode; When the actual SOC is greater than the maximum SOC, charging is stopped.

6. The system according to claim 1, characterized in that The energy management module uses LSTM neural network as the strategy network for reinforcement learning. The parameters of the LSTM neural network include: The input dimension is 15, the output dimension is 3, the number of hidden layers is 3, and each layer has 80 nodes.

7. The system according to claim 1, characterized in that The energy management module also includes: A reward function calculation unit, used to calculate a reward function r(s,a) based on a Gaussian function, where s represents the system state and a represents the system action; The strategy optimization unit is used to calculate the reward update function Q(st,a) through the Monte Carlo method to achieve continuous optimization of the energy management strategy.

8. The system according to claim 1, characterized in that Also includes: A load demand classification unit, used to classify load demands into different levels; The dynamic power allocation unit is connected to the load demand classification unit and is used to dynamically adjust the output power of the fuel cell and the energy storage system according to different load demand levels.

9. The system according to claim 1, characterized in that The system is applied to heavy trucks and also includes: Heavy truck power network, including AC power network and DC power network, is used to provide stable power transmission and distribution; The heavy truck energy optimization module is connected to the energy management module and is used to optimize the distribution of electric energy according to the specific operating environment and requirements of the heavy truck to ensure the service life of the fuel cell and minimize the environmental impact.

10. A hydrogen fuel cell energy management method based on reinforcement learning, using the system according to any one of claims 1 to 9, characterized in that: The following steps are involved: S1, collects environmental information such as traffic flow, road congestion, weather data and forecast data through the environmental module; S2, receiving the environmental information through the energy management module, and monitoring the working status of the fuel cell and the energy storage system in real time; S3, based on the reinforcement learning algorithm, the energy management module performs the following operations: Construct sample pairs containing system states and actions; Using the LSTM neural network as a policy network to train the sample pairs; Calculate the reward function r(s,a) based on the Gaussian function; The reward update function Q(s) is calculated by the Monte Carlo method t ,a t ); Continuously optimize energy management strategies; S4, according to the optimized energy management strategy, the energy management module adjusts the power output and start-stop operation of the fuel cell in real time, and controls the charge and discharge operation of the energy storage system; S5, coordinates the work of the fuel cell, energy storage system and energy management module through the vehicle VCU to ensure the overall operating efficiency of the system; S6, repeating steps S1 to S5 to continuously optimize the energy management strategy of the system.

Images