Photovoltaic hydrogen production real-time dynamic optimization system and method based on reinforcement learning

Abstract

The application provides a photovoltaic hydrogen production real-time dynamic optimization system and method based on reinforcement learning, comprising: a prediction layer module for obtaining short-term prediction results of photovoltaic power and power trend categories; an optimization module for generating floating threshold parameters based on the short-term prediction results, the power trend categories and system state quantities of the photovoltaic hydrogen production system in each optimization cycle, and imposing constraints on the floating threshold parameters; a real-time control module for performing power distribution to obtain a distribution result; and a safety management module for enabling the real-time control module to perform safety override control and trigger a fallback strategy when an abnormal state is detected, and feeding relevant events and operation data to the optimization module to update the floating threshold parameters. Therefore, the application realizes the optimization of suppressing photovoltaic light abandonment, prolonging equipment life and improving system operation robustness by controlling a layered decoupled architecture while ensuring rapid response.

Description

Technical Field

[0001] This invention relates to the field of new energy system scheduling and real-time control technology, specifically to a real-time dynamic optimization system and method for photovoltaic hydrogen production based on reinforcement learning. Background Technology

[0002] With the increasing proportion of renewable energy connected to the grid, using photovoltaic power to directly drive proton exchange membrane (PEM) electrolyzers for hydrogen production has become an important way to achieve a low-carbon energy transition. Photovoltaic hydrogen production systems achieve energy transfer by coupling photovoltaic (PV), proton exchange membrane (PEM), and battery energy storage. However, the system operation faces two key challenges: First, the significant fluctuations and uncertainties in photovoltaic power in the short term can cause frequent start-ups and shutdowns of the electrolyzer and high-amplitude power fluctuations, placing extremely high demands on equipment reliability and control. Second, in order to balance hydrogen production efficiency and equipment lifespan, the scheduling strategy needs to weigh the increase in hydrogen production, the reduction of curtailment rate, and the slowing down of battery degradation.

[0003] Traditional model-based optimization methods (such as linear / nonlinear programming and mixed-integer programming) can theoretically provide a global optimum, but they often rely on high-precision future power prediction and extensive online computation, making it difficult to achieve control responses at the second or minute level. On the other hand, while existing data-driven or intelligent control methods can adapt to complex environments when directly applied to real-time control, their engineering deployment is limited by the large dimensionality of the state and action spaces, high training data requirements, and lack of policy interpretability. Existing hierarchical energy management systems (EMS) typically still couple specific power / modes between layers, resulting in high coupling between the policy and execution sides, and insufficient robustness and portability. Summary of the Invention

[0004] This invention is made to solve the above-mentioned problems, and aims to provide a real-time dynamic optimization system and method for photovoltaic hydrogen production based on reinforcement learning.

[0005] This invention provides a reinforcement learning-based real-time dynamic optimization system for photovoltaic hydrogen production, used for real-time dynamic optimization of photovoltaic hydrogen production systems. The photovoltaic hydrogen production system includes at least a photovoltaic array, batteries, an electrolyzer, and a power conversion device. The reinforcement learning-based real-time dynamic optimization system for photovoltaic hydrogen production includes: a prediction layer module, used to obtain short-term prediction results and power trend categories of photovoltaic power based on input data in each prediction window; an optimization module, used to generate floating threshold parameters based on short-term prediction results, power trend categories, and system state variables of the photovoltaic hydrogen production system in each optimization cycle, and to impose constraints on the floating threshold parameters. The system state variables include real-time photovoltaic power, battery SOC, and operating mode; a real-time control module, used to perform power allocation within the range of the floating threshold parameters and obtain the allocation results; and a safety management module, used to, when an abnormal state is detected, cause the real-time control module to execute safety coverage control and trigger a rollback strategy, while simultaneously feeding back relevant events and operating data to the optimization module to update the floating threshold parameters.

[0006] The photovoltaic hydrogen production real-time dynamic optimization system based on reinforcement learning provided by this invention may also have the following characteristics: the power trend categories include: rising, falling, stable and high fluctuation.

[0007] The real-time dynamic optimization system for photovoltaic hydrogen production based on reinforcement learning provided by this invention may also have the following features: the constraints on the floating threshold parameter include: amplitude boundary constraints, rate of change constraints, and smoothing constraints.

[0008] The photovoltaic hydrogen production real-time dynamic optimization system based on reinforcement learning provided by this invention may also have the following features: the real-time control module further includes: applying hysteresis control, power change rate limit, current change rate limit and minimum start-stop interval constraint to the allocation result.

[0009] The photovoltaic hydrogen production real-time dynamic optimization system based on reinforcement learning provided by the present invention may also have the following features: the real-time control module further includes: setting upper and lower limit thresholds and hysteresis bands for the target power of the battery and the electrolyzer, respectively; and maintaining the operating mode unchanged when the target power of the battery and the electrolyzer is within the hysteresis band.

[0010] The photovoltaic hydrogen production real-time dynamic optimization system based on reinforcement learning provided by this invention may also have the following features: abnormal states include: communication interruption, sensor failure, power conversion device failure, SOC exceeding limit, temperature and voltage exceeding limit.

[0011] The photovoltaic hydrogen production real-time dynamic optimization system based on reinforcement learning provided by this invention may also have the following features: the operating modes include: maximum power hydrogen production mode, maximum power operation and charging mode, shutdown protection mode, standby mode, priority charging mode and priority hydrogen production mode.

[0012] The photovoltaic hydrogen production real-time dynamic optimization system based on reinforcement learning provided by this invention may also have the following features: In the maximum power hydrogen production mode, the electrolyzer operates at the maximum power threshold point; in the maximum power operation and charging mode, the photovoltaic hydrogen production system maximizes hydrogen production and uses all surplus photovoltaic power to charge the battery; in the shutdown protection mode, both the electrolyzer and the battery stop working and enter a standby protection state; in the standby mode, the battery maintains the electrolyzer operating at the minimum power; in the priority charging mode, the photovoltaic hydrogen production system charges the battery, and the electrolyzer operates at the minimum power; in the priority hydrogen production mode, the electrolyzer power follows the photovoltaic power, and all photovoltaic energy is used for hydrogen production.

[0013] The photovoltaic hydrogen production real-time dynamic optimization system based on reinforcement learning provided by this invention may also have the following features: the fallback strategy includes: switching to a static minimum threshold, derated operation, and forced shutdown.

[0014] This invention provides a real-time dynamic optimization method for photovoltaic hydrogen production based on reinforcement learning. It is implemented using any of the reinforcement learning-based real-time dynamic optimization systems for photovoltaic hydrogen production described above, and includes the following features: Step S1, in each prediction window, obtaining short-term prediction results and power trend categories of photovoltaic power based on input data; Step S2, within each optimization cycle, generating floating threshold parameters based on the short-term prediction results, power trend categories, and system state variables of the photovoltaic hydrogen production system, and imposing constraints on the floating threshold parameters. The system state variables include real-time photovoltaic power, battery SOC, and operating mode; Step S3, within the range of the floating threshold parameters, performing power allocation to obtain allocation results; Step S4, when an abnormal state is detected, causing the real-time control module to execute safety coverage control and trigger a rollback strategy, while simultaneously feeding back relevant events and operating data to the optimization module to update the floating threshold parameters.

[0015] The role and effect of invention

[0016] The reinforcement learning-based real-time dynamic optimization system and method for photovoltaic hydrogen production according to the present invention comprises: a prediction layer module, used to obtain short-term prediction results and power trend categories of photovoltaic power based on input data in each prediction window; an optimization module, used to generate floating threshold parameters based on short-term prediction results, power trend categories, and system state variables of the photovoltaic hydrogen production system in each optimization cycle, and to impose constraints on the floating threshold parameters, wherein the system state variables include real-time photovoltaic power, battery SOC, and operating mode; a real-time control module, used to perform power allocation within the range of the floating threshold parameters and obtain allocation results; and a safety management module, used to enable the real-time control module to perform safety coverage control and trigger a fallback strategy when an abnormal state is detected, and to feed back relevant events and operating data to the optimization module to update the floating threshold parameters. Therefore, the reinforcement learning-based real-time dynamic optimization system and method for photovoltaic hydrogen production of the present invention, through a hierarchical decoupled architecture, achieves comprehensive optimization of suppressing photovoltaic curtailment, extending equipment life, and improving system operational robustness while ensuring rapid response. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the module of the photovoltaic hydrogen production real-time dynamic optimization system based on reinforcement learning in an embodiment of the present invention.

[0018] Figure 2 This is a schematic diagram illustrating the working principle of the photovoltaic hydrogen production real-time dynamic optimization system based on reinforcement learning in an embodiment of the present invention.

[0019] Figure 3 This is a structural block diagram of the photovoltaic hydrogen production real-time dynamic optimization system and the photovoltaic hydrogen production system based on reinforcement learning in the embodiments of the present invention.

[0020] Figure 4 This is a schematic diagram of the operation mode determination process of the real-time control module in an embodiment of the present invention.

[0021] Figure 5 This is a flowchart illustrating the training process of the intelligent agent in the optimization module of this invention.

[0022] Figure 6 This is a schematic diagram of the dynamic curves of photovoltaic power generation, electrolytic cell power and battery power under typical day conditions in an embodiment of the present invention. Detailed Implementation

[0023] To make the technical means, creative features, objectives and effects of this invention easy to understand, the following embodiments, in conjunction with the accompanying drawings, specifically illustrate the real-time dynamic optimization system and method for photovoltaic hydrogen production based on reinforcement learning.

[0024] Example

[0025] Figure 1This is a schematic diagram of the module of the photovoltaic hydrogen production real-time dynamic optimization system based on reinforcement learning in an embodiment of the present invention.

[0026] Figure 2 This is a schematic diagram illustrating the working principle of the photovoltaic hydrogen production real-time dynamic optimization system based on reinforcement learning in an embodiment of the present invention.

[0027] Figure 3 This is a structural block diagram of the photovoltaic hydrogen production real-time dynamic optimization system and the photovoltaic hydrogen production system based on reinforcement learning in the embodiments of the present invention.

[0028] like Figure 1 , Figure 2 and Figure 3 As shown, this embodiment provides a real-time dynamic optimization system 100 for photovoltaic hydrogen production based on reinforcement learning, which is used to perform real-time dynamic optimization of a photovoltaic hydrogen production system. The photovoltaic hydrogen production system includes at least a photovoltaic (PV) array, a battery energy storage device (BESS), a proton exchange membrane electrolyzer (PEM), and a power conversion device. The real-time dynamic optimization system 100 for photovoltaic hydrogen production based on reinforcement learning includes: a prediction layer module 10, an optimization module 20, a real-time control module 30, and a safety management module 40.

[0029] The prediction layer module 10 is used to obtain short-term prediction results and power trend categories of photovoltaic power based on the input data in each prediction window. The power trend categories include: rising, falling, stable, and high fluctuation.

[0030] In this embodiment, the input data consists of multidimensional meteorological data and historical operational data. A photovoltaic power prediction model is constructed, which can be ATT, CNN, or BiLSTM, etc. The output is a short-term prediction result, which is a photovoltaic power prediction sequence for the next few steps. The short-term fluctuation pattern of the photovoltaic power prediction sequence is classified by clustering or pattern recognition methods to obtain power trend categories. The power trend categories include sunrise, rising, stable, high fluctuation, falling, convex, concave, and sunset.

[0031] Optimization module 20 is used in each optimization cycle The system generates floating threshold parameters based on short-term forecast results, power trend categories, and system state variables of the photovoltaic hydrogen production system, and imposes constraints on these floating threshold parameters. The system state variables include real-time photovoltaic power, battery SOC, and operating mode. The floating threshold parameter is the upper limit of the battery's maximum charge / discharge rate. and the maximum operating power coefficient of the electrolytic cell .

[0032] The constraints on the floating threshold parameter include: amplitude boundary constraints, rate of change constraints, and smoothing constraints.

[0033] The amplitude boundary constraints are: , .

[0034] The rate of change constraint is: , .

[0035] The smoothing constraint is: , It is obtained by smoothing the previous period value and the current candidate value.

[0036] Figure 4 This is a schematic diagram of the operation mode determination process of the real-time control module in an embodiment of the present invention.

[0037] like Figure 4 As shown, the operating modes include: maximum power hydrogen production mode (Mode1), maximum power operation and charging mode (Mode2), shutdown protection mode (Mode3), standby mode (Mode4), priority charging mode (Mode5), and priority hydrogen production mode (Mode6).

[0038] In maximum power hydrogen production mode When the battery is fully charged and When the cell enters the maximum power hydrogen production mode, the electrolyzer cannot fully absorb the photovoltaic power generation energy, and the battery cannot continue to store electricity. The electrolyzer operates at the maximum power threshold point.

[0039] In maximum power operation and charging mode , When the battery is not fully charged and When the system enters the maximum power operation and charging mode, the photovoltaic hydrogen production system maximizes hydrogen production and uses all the surplus power generated by the photovoltaic system to charge the battery.

[0040] In shutdown protection mode 0. When the photovoltaic power is lower than the lower limit of the electrolyzer's operation and the battery power reaches the minimum value, the system enters the shutdown protection mode. At this time, the photovoltaic hydrogen production system lacks power supply and the battery cannot continue to store electrical energy. Both the electrolyzer and the battery stop working and enter the standby protection state.

[0041] In standby mode, , When the battery level is greater than the minimum value and At this point, it enters standby mode. In this mode, photovoltaic power generation cannot independently support the operation of the electrolyzer. Energy is replenished by battery discharge to maintain the electrolyzer operating at the lowest power.

[0042] In priority charging mode , When the battery level is below the minimum value and At this point, the system enters priority charging mode, where the photovoltaic hydrogen production system charges the battery. In this mode, the photovoltaic hydrogen production system uses most of its available energy to charge the battery, and the electrolyzer operates at the lowest power.

[0043] Under the priority hydrogen production mode When the battery level is greater than the minimum value and At this point, the system enters the priority hydrogen production mode. The battery power is stable, and the electrolyzer power follows the photovoltaic power, using all photovoltaic energy for hydrogen production.

[0044] In this embodiment, the optimization module 20 is based on the Markov decision process and uses the Q-learning algorithm to establish a Markov decision process for the scheduling optimization of the photovoltaic hydrogen production system. ).in Let represent the possible state space of each component in a photovoltaic hydrogen production system. The action space represents the ratio of the maximum power threshold of the electrolytic cell to the maximum charge / discharge power threshold of the battery that the optimization module 20 can output. Let be the state transition probability, describing the state transition probability in the current state. Take action The system then transitions to a new state. The probability distribution. The reward function considers the objectives of maximizing hydrogen production, minimizing battery degradation, and reducing light curtailment rate.

[0045] The flowchart of agent training in optimization module 20 is as follows:

[0046] state space The Q-learning state includes the power trend category, real-time photovoltaic power, and cell SOC. State discretization is defined as follows:

[0047]

[0048] Action space The action is set as a percentage of the rated power of the battery and electrolyzer, meaning that each decision cycle determines only the ratio of the allowed upper power limit of the two critical devices to their rated power. The action and its discretization are defined as follows:

[0049]

[0050] In the formula, and These are the maximum operating power coefficient of the hydrogen generator and the maximum charging rate coefficient of the electrolyzer, respectively. The specific calculation formulas are as follows:

[0051]

[0052] In the formula, Set the maximum power for the electrolytic cell. Set the battery to maximum power. For the battery's rated capacity, This is the rated power of the electrolytic cell.

[0053] reward function The optimization objective is to reduce battery degradation and increase hydrogen production in the photovoltaic hydrogen production system. Therefore, the reward function is set as the sum of the total battery processing power and the cumulative hydrogen production of the electrolyzer.

[0054]

[0055] In the formula, For battery charging and discharging power, The hydrogen production rate of the electrolyzer. These are the weighting coefficients.

[0056] The real-time control module 30 is used to perform power allocation within the range of the floating threshold parameter and obtain the allocation result. The control cycle of the real-time control module 30 is... Smaller than the optimization period .

[0057] In the real-time control module 30, hysteresis control, power change rate limit, current change rate limit, and minimum start-stop interval constraint are applied to the allocation results.

[0058] Upper and lower threshold values ​​and hysteresis bands are set for the target power of the battery and the electrolyzer, respectively. When the target power of the battery and the electrolyzer is within the hysteresis band, their operating mode remains unchanged.

[0059] The safety management module 40 is used to enable the real-time control module to perform safety coverage control and trigger the rollback strategy when an abnormal state is detected. At the same time, it feeds back relevant events and running data to the optimization module to update the floating threshold parameters.

[0060] Abnormal states include: communication interruption, sensor failure, power conversion device failure, SOC exceeding limits, and temperature and voltage exceeding limits.

[0061] The rollback strategies include: switching to a static minimum threshold, reducing the limit, and forced shutdown.

[0062] Figure 5 This is a flowchart illustrating the training process of the intelligent agent in the optimization module of this invention.

[0063] like Figure 5 As shown in this embodiment, the operation process of the photovoltaic hydrogen production real-time dynamic optimization system 100 based on reinforcement learning mainly includes an offline training process and an online operation process.

[0064] The goal of the offline training process is to generate an optimal Q-table for each of the four seasons—spring, summer, autumn, and winter—using multidimensional meteorological data and historical operational data. The training process is based on historical photovoltaic and meteorological datasets, and for each season, the following training iterations are repeated:

[0065] First, prediction and classification are performed: a segment of data is extracted from the seasonal dataset and input into prediction layer module 10. Prediction layer module 10 uses its built-in Attention-CNN-BiLSTM model to predict future photovoltaic power and performs trend classification on the prediction results.

[0066] Next, action selection and simulation execution are performed: the Q-learning agent of optimization module 20 takes the predicted power trend category and the current system state as input, and performs the simulation based on the current Q table and... Explore strategies and select an action, namely a floating threshold parameter, which is also a set of power threshold instructions. , The instruction is then sent to the real-time control module 30.

[0067] Finally, reward calculation and model update are performed: the simulated real-time control module 30 performs power allocation based on the action and calculates the system benefit brought by the decision, i.e., the reward value, which integrates factors such as hydrogen production and battery consumption. The Q-learning agent updates the corresponding values ​​in its Q-table based on the obtained reward value and changes in the system state using a priority experience replay mechanism. Through repeated iterative training on data from the entire season, a convergent optimal Q-table tailored to the characteristics of that season is finally obtained. The above process is repeated for each of the four seasons, ultimately producing four sets of Q-tables for spring, summer, autumn, and winter.

[0068] In the online operation process, the decision-making cycle in this embodiment is 30 minutes, and the execution cycle is 1 minute.

[0069] First, real-time forecasting is performed: the forecasting layer module 10 collects real-time multi-dimensional meteorological data and historical operating data, forecasts and classifies the short-term photovoltaic power, obtains the short-term forecast results and power trend categories, and sends them to the optimization module 20.

[0070] Next, optimization decisions are made. Optimization module 20 first determines the season based on the current date and loads the corresponding Q table. Then, it combines the power trend category from prediction module 10 with the real-time collected photovoltaic power and battery SOC to form the current state. It then searches the Q table and outputs the optimal action for the current state, namely the power threshold command. , And send it to the real-time control module 30.

[0071] Then, after receiving the power threshold command from the optimization module 20, the real-time control module 30 uses this as a boundary condition to perform real-time rule judgment and power allocation in a 1-minute cycle, and issues commands. After determining the operating mode and allocating the target power for the battery and electrolytic cell, the current calculation module and DC-DC control module of the real-time control module 30 will convert the power command into an actual current control signal to drive the equipment to run.

[0072] The key parameters for the simulation environment settings in this embodiment are configured as follows: Photovoltaic array: Rated power: 10657.5 W. Individual cell power: 213.15 W.

[0073] Proton exchange membrane electrolyzer: Rated power: 2000W. Membrane electrode area: 160 cm²; Number of parallel electrolyzers: 6. Operating current density range: 0.1~3.5 A / cm²; Operating voltage range: 18~25 V.

[0074] Battery energy storage system: Battery capacity: 11520Wh. Rated capacity: 120Ah. Rated voltage: 96V.

[0075] Algorithm parameters: learning rate ( Discount factor: 0.2. Greed rate: 0.9. 0.2. Convergence threshold: 1. Maximum number of iterations: 1000.

[0076] This embodiment selects a typical, fluctuating sunny day as the characteristic day for simulation. The photovoltaic output on this day is generally high, but it will fluctuate rapidly at certain times due to factors such as cloud cover, which can fully test the adaptability of the method of the present invention.

[0077] Figure 6 This is a schematic diagram illustrating the dynamic curves of photovoltaic power generation, electrolytic cell power, and battery power under typical conditions according to embodiments of the present invention. Figure 6 As shown in the figure, the maximum available power of the photovoltaic system, the actual operating power of the electrolyzer, and the charging and discharging power of the battery (with positive values ​​for charging and negative values ​​for discharging) change over a 24-hour period of day.

[0078] 1. Nighttime to early morning period (0~5.5 hours)

[0079] During this phase, there is no photovoltaic power generation. The battery is the sole energy source, with an initial SOC of approximately 52.4%. Based on the predicted trend of no photovoltaic power generation at night, the optimization module 20 sets a low electrolyzer power threshold. The goal is to maintain the electrolyzer at a minimum operating level to avoid frequent equipment shutdowns while mitigating battery wear. Upon receiving this threshold instruction, the real-time control module 30, within the threshold boundaries, determines and stably operates in standby mode based on the real-time status.

[0080] 2. Morning solar power ramp-up phase (8.5~10.5 hours)

[0081] The prediction module 10 captures the "rising" trend of photovoltaic power. Based on this trend, the Q-learning model of the optimization module 20 determines that future energy is sufficient and outputs a high electrolyzer power threshold, aiming to maximize hydrogen production. At 8:30, the real-time control module 30 determines that photovoltaic power generation is between the minimum and maximum power thresholds of the electrolyzer and that the battery SOC is high, thus entering the priority hydrogen production mode. The electrolyzer power basically follows the increase of photovoltaic power, while the excess energy is used to charge the battery, at which point the battery charging power reaches a peak of approximately 1907W.

[0082] At 9:00, the photovoltaic power continued to rise to 3700W. At this point, the photovoltaic power had exceeded the maximum power threshold set by the optimization module 20, and the battery was not fully charged. The real-time control module 30 automatically switched to the maximum power operation and charging mode. The electrolytic cell power was limited to the maximum threshold of 2000W set by the optimization module 20, and the remaining 1700W power was used entirely for battery charging.

[0083] At 11:30, the photovoltaic power was approximately 3523W. At this time, the photovoltaic power generation exceeded the set maximum power of the electrolyzer, and the real-time control module 30 determined that the battery was fully charged, triggering the maximum power hydrogen production mode. The electrolyzer maintained operation at the threshold power of 2000W, while the battery charging power dropped to 0, and it no longer absorbed energy.

[0084] 3. The period of sharp power fluctuations in the afternoon (13.5 ~ 15.5 hours)

[0085] During this period, photovoltaic power fluctuates drastically and frequently due to cloud cover. Prediction module 10 categorizes these fluctuations as "concave" or "declining" trends. Upon receiving this signal, optimization module 20 outputs a relatively conservative power threshold to avoid excessive battery consumption when photovoltaic power is insufficient, while also preventing the electrolyzer from shutting down due to a sudden drop in power. The real-time control layer responds rapidly within the boundaries set by the optimization layer in 1-minute increments. When the real-time photovoltaic power is higher than the electrolyzer's lower operating limit, the system primarily operates in priority hydrogen production mode, with the electrolyzer power fluctuating in line with the photovoltaic power. When photovoltaic power drops briefly and photovoltaic power generation is insufficient to supply the electrolyzer alone, the rules of real-time control module 30 immediately initiate shallow discharge of the battery to compensate for the difference, thereby maintaining stable operation of the electrolyzer and avoiding frequent start-ups and shutdowns. Under the same system configuration, price assumptions, and dataset conditions, a fixed threshold implementation and the embodiment of this invention are compared. The results show that under this representative operating condition, the LCOH of the present invention decreased from 32.3804 RMB / kg to 29.3829 RMB / kg, a decrease of approximately 9.26%. The static payback period was shortened from 18 years to 14 years, a reduction of 4 years. The NPV increased from 3882 RMB / kg to 10216 RMB / kg, an increase of approximately 163.16%. The above data are only used to illustrate that the present invention can bring lower hydrogen levelized cost, shorter investment recovery time, and higher net present value.

[0086] This embodiment also provides a real-time dynamic optimization method for photovoltaic hydrogen production based on reinforcement learning, implemented using a real-time dynamic optimization system 100 for photovoltaic hydrogen production based on reinforcement learning as described above, including:

[0087] Step S1: In each prediction window, obtain short-term prediction results of photovoltaic power and power trend category based on the input data.

[0088] Step S2: In each optimization cycle, a floating threshold parameter is generated based on the short-term prediction results, power trend category, and system state variables of the photovoltaic hydrogen production system. Constraints are then imposed on the floating threshold parameter. The system state variables include real-time photovoltaic power, battery SOC, and operating mode.

[0089] Step S3: Within the range of the floating threshold parameter, perform power allocation to obtain the allocation result.

[0090] Step S4: When an abnormal state is detected, the real-time control module executes safety coverage control and triggers a rollback strategy. At the same time, the relevant events and running data are fed back to the optimization module to update the floating threshold parameters.

[0091] The role and effect of the embodiments

[0092] According to the reinforcement learning-based real-time dynamic optimization system and method for photovoltaic hydrogen production involved in this embodiment, it includes: a prediction layer module, used to obtain short-term prediction results and power trend categories of photovoltaic power based on input data in each prediction window; an optimization module, used to generate floating threshold parameters based on short-term prediction results, power trend categories, and system state variables of the photovoltaic hydrogen production system in each optimization cycle, and to impose constraints on the floating threshold parameters, wherein the system state variables include real-time photovoltaic power, battery SOC, and operating mode; a real-time control module, used to perform power allocation within the range of the floating threshold parameters and obtain allocation results; and a safety management module, used to enable the real-time control module to perform safety coverage control and trigger a fallback strategy when an abnormal state is detected, and to feed back relevant events and operating data to the optimization module to update the floating threshold parameters. Therefore, the reinforcement learning-based real-time dynamic optimization system and method for photovoltaic hydrogen production of this invention, through a control layered decoupled architecture, achieves comprehensive optimization of suppressing photovoltaic curtailment, extending equipment life, and improving system operational robustness while ensuring rapid response.

[0093] In this embodiment, the optimization module applies smoothing and rate of change constraints to the threshold. Combined with the hysteresis control, power ramp-up limit, and minimum start-stop interval of the real-time control layer, it significantly suppresses frequent start-stop of the electrolytic cell, power ripple, and current surge, thus extending the equipment life.

[0094] In this embodiment, the optimization module can perform seasonal or time-based training and switching, and introduce an anomaly rollback mechanism to enable the system to better adapt to different climate conditions, fluctuation patterns, and cope with sudden failures, thus ensuring the overall operational efficiency throughout the year.

[0095] Those skilled in the art should understand that this invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to this invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A real-time dynamic optimization system for photovoltaic hydrogen production based on reinforcement learning, used for real-time dynamic optimization of a photovoltaic hydrogen production system, wherein the photovoltaic hydrogen production system includes at least a photovoltaic array, a battery, an electrolyzer, and a power conversion device, characterized in that, The reinforcement learning-based photovoltaic hydrogen production real-time dynamic optimization system includes: The prediction layer module is used to obtain short-term prediction results and power trend categories of photovoltaic power based on the input data in each prediction window; An optimization module is used to generate floating threshold parameters based on the short-term prediction results, the power trend category, and the system state variables of the photovoltaic hydrogen production system in each optimization cycle, and to impose constraints on the floating threshold parameters. The system state variables include real-time photovoltaic power, battery SOC, and operating mode. The real-time control module is used to perform power allocation within the range of the floating threshold parameter and obtain the allocation result; The security management module is used to enable the real-time control module to perform security coverage control and trigger a rollback strategy when an abnormal state is detected. At the same time, it feeds back relevant events and running data to the optimization module to update the floating threshold parameter.

2. The real-time dynamic optimization system for photovoltaic hydrogen production based on reinforcement learning according to claim 1, characterized in that: in, The power trend categories include: rising, falling, stable, and highly volatile.

3. The real-time dynamic optimization system for photovoltaic hydrogen production based on reinforcement learning according to claim 1, characterized in that: in, The constraints on the floating threshold parameter include: amplitude boundary constraints, rate of change constraints, and smoothing constraints.

4. The real-time dynamic optimization system for photovoltaic hydrogen production based on reinforcement learning according to claim 1, characterized in that: in, The real-time control module also includes: applying hysteresis control, power change rate limit, current change rate limit, and minimum start-stop interval constraint to the allocation result.

5. The real-time dynamic optimization system for photovoltaic hydrogen production based on reinforcement learning according to claim 1, characterized in that: in, The real-time control module further includes: setting upper and lower threshold values ​​and hysteresis bands for the target power of the battery and the electrolytic cell respectively; and maintaining their operating mode unchanged when the target power of the battery and the electrolytic cell is within the hysteresis band.

6. The real-time dynamic optimization system for photovoltaic hydrogen production based on reinforcement learning according to claim 1, characterized in that: in, The abnormal states include: communication interruption, sensor failure, power conversion device failure, SOC exceeding limits, and temperature and voltage exceeding limits.

7. The real-time dynamic optimization system for photovoltaic hydrogen production based on reinforcement learning according to claim 1, characterized in that: in, The operating modes include: maximum power hydrogen production mode, maximum power operation and charging mode, shutdown protection mode, standby mode, priority charging mode, and priority hydrogen production mode.

8. The real-time dynamic optimization system for photovoltaic hydrogen production based on reinforcement learning according to claim 7, Its features are: In the maximum power hydrogen production mode, the electrolyzer operates at the maximum power threshold point; In the maximum power operation and charging mode, the photovoltaic hydrogen production system maximizes hydrogen production and uses all surplus power from photovoltaic power generation to charge the battery; In the shutdown protection mode, both the electrolytic cell and the battery stop working and enter standby protection mode; In the standby mode, the battery maintains the electrolytic cell operating at minimum power. In the priority charging mode, the photovoltaic hydrogen production system charges the battery, and the electrolyzer operates at minimum power. In the priority hydrogen production mode, the power of the electrolyzer follows the photovoltaic power, and all photovoltaic energy is used for hydrogen production.

9. The real-time dynamic optimization system for photovoltaic hydrogen production based on reinforcement learning according to claim 1, characterized in that: in, The rollback strategies include: switching to a static minimum threshold, reducing the limit during operation, and forced shutdown.

10. A real-time dynamic optimization method for photovoltaic hydrogen production based on reinforcement learning, implemented using the real-time dynamic optimization system for photovoltaic hydrogen production based on reinforcement learning as described in any one of claims 1-9, characterized in that... include: Step S1: In each prediction window, obtain short-term prediction results and power trend categories of photovoltaic power based on the input data; Step S2: In each optimization cycle, a floating threshold parameter is generated based on the short-term prediction results, the power trend category, and the system state variables of the photovoltaic hydrogen production system, and constraints are applied to the floating threshold parameter. The system state variables include real-time photovoltaic power, battery SOC, and operating mode. Step S3: Within the range of the floating threshold parameter, perform power allocation to obtain the allocation result; Step S4: When an abnormal state is detected, the real-time control module executes safety coverage control and triggers a rollback strategy, while feeding back relevant events and running data to the optimization module to update the floating threshold parameter.

Images