Wind-solar-hydrogen coupling hydrogen production control method and system based on deep reinforcement learning

Abstract

The application discloses a wind-solar-hydrogen coupling hydrogen production control method and system based on deep reinforcement learning, first, wind-solar-hydrogen related data is collected and feature extraction is carried out, and the LSTM model and the LSTM-Transformer model are used for multi-time scale power prediction at the day level, the hour level and the minute level; then, a multi-objective reward function combining hydrogen production income, equipment loss and carbon emission is used in combination with the DDPG algorithm to output an optimal control strategy; finally, multi-time scale control is carried out based on a three-layer scheduling framework of the day level, the hour level and the minute level, and the optimal control strategy is verified in real time by using the PINN. The application is based on wind-solar-hydrogen related data, uses the DDPG algorithm to realize accurate control of a continuous action space, the multi-objective reward function guides the DDPG learning to consider the optimal strategy of economy, life and carbon emission, a three-layer scheduling framework of the day level, the hour level and the minute level is constructed to realize multi-time scale collaborative optimization, and the PINN hybrid modeling is used to reduce simulation delay and meet real-time control requirements.

Description

Technical Field

[0001] This invention belongs to the field of new energy control technology, specifically relating to a wind-solar-hydrogen coupling hydrogen production control method and system based on deep reinforcement learning. Background Technology

[0002] Hydrogen energy, as a zero-carbon energy carrier, occupies a core position in the "dual-carbon" strategy. Green hydrogen, produced through the electrolysis of water using renewable energy, is a key pathway to achieve deep decarbonization. Currently, the global green hydrogen industry is expanding rapidly, with production capacity increasing year by year, and is developing in parallel with multiple technological routes, including high-pressure gaseous hydrogen storage, solid-state hydrogen storage, liquid-state hydrogen storage, and organic liquid-state hydrogen storage, moving beyond a single high-pressure gaseous hydrogen storage approach.

[0003] The core challenge currently facing wind and solar hydrogen production technology is the mismatch between the high volatility of renewable energy and the rigid demand for stable loads from electrolyzers. Wind power fluctuations can range from 0% to 120% of rated value, while solar power fluctuations can reach as high as 80% daily, with frequent minute-level power changes. In contrast, PEM electrolyzers typically allow for load fluctuations of 20%-100%, and frequent start-ups and shutdowns, along with large power variations, significantly shorten equipment lifespan. Therefore, it is necessary to simultaneously optimize multiple objectives, including hydrogen production efficiency, equipment lifespan, wind and solar curtailment rates, and grid electricity prices.

[0004] Existing technologies employ a fixed threshold control strategy, setting a power threshold for the electrolyzer's start-up and shutdown (e.g., starting when wind power exceeds 50% of the rated value), and shutting down for protection when wind and solar power fall below the threshold. This strategy is simple but inefficient, with wind and solar curtailment rates exceeding 15%, resulting in high hydrogen production costs. Traditional methods rely on time series models such as LSTM and ARIMA to predict short-term wind and solar output, achieving smooth power scheduling; however, prediction errors exceed 10% under extreme weather conditions, leading to poor control robustness. Electrolysis water control methods using model predictive control (MPC) algorithms require online optimization problem solving, resulting in high computational complexity and response times exceeding 100ms, failing to meet real-time control requirements. Furthermore, the established third-order linear state-space model of the electrolysis water system cannot accurately describe the nonlinear characteristics of the electrolyzer. Existing technologies construct physical simulation models of electrolyzers and hydrogen storage tanks using digital twin technology, and optimize operation strategies through virtual-real interaction. However, high-fidelity modeling is computationally complex, pure physical modeling requires solving a system of partial differential equations, and the simulation delay is >1s, which cannot meet the requirements of real-time control. At the same time, there is a lack of data-driven correction, and the model accuracy is affected by parameter drift, with a prediction error >5%. Summary of the Invention

[0005] To address the problems existing in the above-mentioned background technology, the purpose of this invention is to provide a wind-solar-hydrogen coupling hydrogen production control method and system based on deep reinforcement learning.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A wind-solar-hydrogen coupling hydrogen production control method based on deep reinforcement learning includes the following steps: S1. Collect wind power generation data, photovoltaic power generation data, water electrolysis hydrogen production data, power grid data, meteorological data, and carbon emission data, and perform data preprocessing; S2. Extract features from the preprocessed data, including historical power features, meteorological features, temporal features, and dimensional features; S3. The extracted time features are concatenated with historical power features through a time coding layer and then input into the LSTM model. The extracted time features are concatenated with meteorological features through a time coding layer and then input into the LSTM-Transformer model. Then, the time cycle pattern recognition (such as the diurnal pattern of photovoltaic power output), seasonal feature extraction (such as the seasonal fluctuation of wind power output), and special pattern marking of holidays are performed for daily prediction. This completes the wind and solar power output prediction at multiple time scales, including daily, hourly, and minute-level predictions, and outputs the power prediction results at multiple time scales. S4. The output multi-timescale power prediction results are used as the state input of the DDPG algorithm. The DDPG algorithm is used to optimize the continuous state space and action space. A multi-objective reward function that integrates hydrogen production revenue, equipment loss and carbon emissions is adopted. The DDPG algorithm learns the optimal control strategy of the wind-solar-hydrogen system through a deep neural network and outputs it. S5. Construct a three-tier scheduling framework: daily, hourly, and minute-level. The daily scheduling uses the MILP algorithm, the hourly scheduling uses the MPC algorithm, and the minute-level scheduling uses the RL algorithm. The following collaborative scheduling mechanisms are used between the levels: Top-down breakdown: Daily plan → Hourly target → Minute setpoint; Bottom-up feedback: minute-level actual values ​​→ correct hourly forecasts → adjust daily plans; Conflict resolution: When there is a conflict between daily planning and real-time control, safety takes the highest priority; Based on the optimal control strategy output in step S4, a three-level scheduling framework of daily-hourly-minute is used to perform daily MILP planning, hourly MPC scheduling, and minute-level RL control. S6. A hybrid modeling approach using Physical Information Neural Network (PINN) is adopted. The electrochemical equations, thermal balance equations, and mass conservation equations of the electrolytic cell in the physical model are used as part of the neural network loss function. The parameters of the physical model are corrected using experimental data, and the parameters of the neural network are updated based on real-time data. The optimal control strategy is verified in real time through the hybrid model of Physical Information Neural Network (PINN) to evaluate the performance and safety of the optimal control strategy.

[0007] Furthermore, in step S1, the wind power generation data includes wind speed, wind direction, pitch angle, power output, and wind turbine status; The photovoltaic power generation data includes irradiance, module temperature, power output, and photovoltaic array status; The water electrolysis hydrogen production data includes electrolyzer voltage, current, temperature, hydrogen production, energy efficiency and membrane resistance, hydrogen storage tank pressure, temperature, hydrogen storage capacity and valve status; The power grid data includes power frequency, voltage, and electricity price; The meteorological data includes temperature, humidity, air pressure, cloud cover, and atmospheric pressure load. The carbon emission data includes real-time grid carbon intensity.

[0008] Furthermore, in step S2, the historical power characteristics include statistical features such as power mean, variance, maximum value, minimum value, and peak-to-valley difference. Meteorological characteristics: including current values, rate of change, and trend characteristics of wind speed, wind direction, temperature, and air pressure; Time characteristics: including periodic time codes such as hours, days of the week, and seasons, as well as characteristics of the same historical period; Dimensional features: Dimensional information of the input data, used for model adaptive processing.

[0009] Furthermore, in step S3, the basis for the multi-timescale wind and solar power output prediction is: Daily forecast: Based on 7-day NWP data, predict the 24-hour wind and solar power output curve; Hourly forecast: Based on data from the past 24 hours, predict power for the next 1-4 hours; Minute-level prediction: Based on data from the past hour, predict power for the next 10-30 minutes.

[0010] Furthermore, in step S4, the state space includes multidimensional continuous variables such as wind and solar status, electrolyzer status, hydrogen storage system, economic signals, historical status, time coding, and safety constraints, and the action space includes three-dimensional continuous variables such as electrolyzer power adjustment coefficient, hydrogen storage tank inlet valve opening degree, and fuel cell discharge coefficient. Specifically, the state space includes 50-dimensional continuous variables: 5-dimensional wind and solar status: real-time wind power, photovoltaic power, prediction error, power change rate, and weather type code; 6-dimensional electrolyzer status: voltage, current, temperature, hydrogen production, energy efficiency, and cumulative operating time; 4-dimensional hydrogen storage system: hydrogen storage tank pressure, temperature, hydrogen storage capacity, and valve opening; 3D economic signals: real-time electricity price, grid carbon intensity, and hydrogen market price; 20-dimensional historical status: the rate of change of wind and solar power and the rate of change of electricity price at 1-minute intervals over the past 10 minutes; 3D time encoding: hour, day of the week, season; Nine-dimensional safety constraints: number of start-stop cycles, number of maximum power changes, temperature anomaly indicators, pressure anomaly indicators, voltage anomaly indicators, current anomaly indicators, electrolyzer membrane humidity anomaly indicators, hydrogen storage tank liquid level anomaly indicators, and hydrogen purity anomaly indicators.

[0011] Furthermore, in step S4, the formula for the multi-objective reward function that integrates hydrogen production revenue, equipment wear and tear, and carbon emissions is as follows: R(t) = w 收益 ×R 收益 (t)-w 成本 ×R 成本 (t)-w 惩罚 ×R 惩罚 (t); R 收益 (t) = Hydrogen production × Hydrogen market price + Peak shaving ancillary service revenue; R 成本 (t) = Electricity purchase cost + Equipment maintenance cost + Carbon emission cost; R 惩罚 (t)=w 弃光 × light waste + w 启停 ×Number of starts and stops + w 波动 ×Power change amplitude + w 寿命 ×Life loss + w 碳 × Carbon emissions exceeded the limit; Where R(t) is the total reward at time step t, w 收益 As a weighting factor for hydrogen production revenue, R 收益 (t) represents the hydrogen production gain at time step t, w 成本 As a cost penalty weight, R 成本 (t) represents the cost of hydrogen production at time step t, w 惩罚 For safety and environmental constraints, R 惩罚 (t) represents the safety and environmental constraints and penalties at time step t, w 弃光 To punish the abandonment of wind and light, w 启停 The weight of the penalty for the number of start-stop cycles, w 波动 w is the penalty weight for the magnitude of power change. 寿命 For lifetime depletion penalty weights, w 碳 Weighting for penalties for exceeding carbon emission limits.

[0012] Furthermore, the w 收益 =1.2, w 成本 =0.8, w 惩罚 =0.5, w 弃光 =0.3, w 启停 =0.2, w 波动 =0.15, w 寿命 =0.2, w 碳=0.15.

[0013] A wind-solar-hydrogen coupling hydrogen production control system based on deep reinforcement learning includes: (1) Multi-source data acquisition and preprocessing module: used to acquire wind power generation data, photovoltaic power generation data, water electrolysis hydrogen production data, power grid data, meteorological data and carbon emission data, and preprocess the acquired data; (2) Wind and solar power output prediction module: The LSTM+Transformer hybrid model is used to extract historical power features, meteorological features, time features and dimensional features from the data output by the multi-source data acquisition and preprocessing module. Based on the extracted features, wind and solar power output prediction is performed at multiple time scales, and the multi-time scale power prediction results are output. (3) Deep reinforcement learning decision engine: The DDPG algorithm and a multi-objective reward function that integrates hydrogen production revenue, equipment loss and carbon emissions are used to process the output results of the wind and solar power output prediction module to obtain the optimal control strategy of the wind and solar hydrogen system. (4) Multi-timescale scheduling module: It has a built-in three-layer scheduling framework of daily-hour-minute. The daily scheduling adopts the MILP algorithm, the hourly scheduling adopts the MPC algorithm, and the minute scheduling adopts the RL algorithm. The multi-timescale scheduling module is used to perform daily MILP planning, hourly MPC scheduling and minute-level RL control based on the optimal control strategy obtained by the deep reinforcement learning decision engine. (5) Digital twin simulation platform: PINN hybrid modeling is used to verify the performance and safety of the optimal control strategy obtained by the deep reinforcement learning decision engine in real time.

[0014] Compared with the shortcomings and deficiencies of existing technologies, the present invention has the following beneficial effects: 1. This invention utilizes a deep reinforcement learning adaptive control strategy to dynamically match the characteristics of wind and solar fluctuations, thereby achieving continuous and smooth adjustment of the electrolytic cell power. 2. This invention achieves multi-timescale collaborative optimization by constructing a three-layer scheduling framework at the daily, hourly, and minute levels; 3. This invention employs a hybrid modeling approach using Physical Information Neural Network (PINN), embedding physical constraints into the neural network to achieve lightweight real-time simulation; 4. This invention designs a multi-objective reward function that integrates hydrogen production revenue, equipment wear and tear, and carbon emissions, achieving multi-objective synergistic optimization and avoiding conflicts between short-term and long-term interests; 5. This invention integrates extreme weather early warning data, improving prediction accuracy under extreme weather conditions and enhancing control robustness under extreme operating conditions; 6. This invention constructs a closed-loop system of "online learning - strategy update - simulation verification", and the control strategy continuously evolves with the accumulation of data, exhibiting high adaptability. Attached Figure Description

[0015] Figure 1 This is an architecture diagram of a wind-solar-hydrogen coupling hydrogen production control system based on deep reinforcement learning provided in an embodiment of the present invention; Figure 2 This is a diagram of the wind and solar power output prediction model architecture provided in this embodiment of the invention; Figure 3 This is a network architecture diagram of the DDPG deep reinforcement learning decision engine provided in an embodiment of the present invention; Figure 4 This is a multi-time-scale scheduling framework diagram provided in an embodiment of the present invention; Figure 5 This is a diagram of the PINN digital twin hybrid modeling architecture provided in an embodiment of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0017] Example 1 This invention proposes a wind-solar-hydrogen coupling hydrogen production control method and system based on deep reinforcement learning, referring to... Figure 1 The system includes several modules: multi-source data acquisition and preprocessing module, wind and solar power output prediction module, deep reinforcement learning decision engine, multi-timescale scheduling module, and digital twin simulation platform.

[0018] 1. Multi-source data acquisition and preprocessing module It is used to collect data on wind power generation, photovoltaic power generation, hydrogen production through water electrolysis, power grid, meteorological data, and carbon emission data, and to preprocess the collected data.

[0019] Wind power generation data includes wind speed, wind direction, pitch angle, power output, and turbine status. Photovoltaic power generation data includes irradiance, module temperature, power output, and photovoltaic array status. Hydrogen production data from water electrolysis includes: electrolyzer voltage, current, temperature, hydrogen production, energy efficiency, and membrane resistance; hydrogen storage tank pressure, temperature, hydrogen storage capacity, and valve status. Power grid data: including power frequency, voltage, and electricity price; Meteorological data: including temperature, humidity, air pressure, cloud cover, and atmospheric pressure load; Carbon emission data: including real-time grid carbon intensity.

[0020] Preprocessing is performed on wind power generation data and photovoltaic power generation data, including noise reduction (Kalman filtering), anomaly detection, and interpolation completion. Trend analysis and normalization are performed on electrolyzer status data. Pressure-temperature compensation and leak detection are performed on hydrogen storage tanks. Time alignment, missing value filling, and outlier removal are performed on grid data, meteorological data, and carbon emission data.

[0021] 2. Wind and Solar Power Output Prediction Module like Figure 2 As shown, an LSTM+Transformer hybrid model is used to extract historical power features, meteorological features, temporal features, and dimensional features from the data output by the multi-source data acquisition and preprocessing module. The specific feature data is as follows: Historical power characteristics: including statistical features such as power mean, variance, maximum value, minimum value, peak-to-valley difference, etc. Meteorological characteristics: current values, rate of change, and trend characteristics of wind speed, wind direction, temperature, and air pressure; Time characteristics: including periodic time codes such as hours, days of the week, and seasons, as well as characteristics of the same historical period; Dimensional features: Dimensional information of the input data, used for model adaptive processing.

[0022] The extracted time features are concatenated with historical power features through a time coding layer and then input into the LSTM model. The extracted time features are concatenated with meteorological features through a time coding layer and then input into the LSTM-Transformer model. Then, time cycle pattern recognition (such as the diurnal pattern of photovoltaic power output), seasonal feature extraction (such as the seasonal fluctuation of wind power output), and special pattern labeling for holidays are performed for daily prediction. This completes the prediction of wind and solar power output at multiple time scales and outputs the power prediction results at multiple time scales.

[0023] The basis for multi-timescale wind and solar power output prediction is: Daily forecast: Based on 7-day NWP data, predict the 24-hour wind and solar power output curve; Hourly forecast: Based on data from the past 24 hours, predict power for the next 1-4 hours; Minute-level prediction: Based on data from the past hour, predict power for the next 10-30 minutes.

[0024] Prediction accuracy metrics: Japanese-level prediction: MAPE < 8%, RMSE < 10% of rated power; Hourly forecast: MAPE < 5%, RMPE < 8% of rated power; Minute-level prediction: MAPE < 3%, RMSE < 5% of rated power.

[0025] 3. Deep reinforcement learning decision engine The DDPG algorithm, combined with a multi-objective reward function that integrates hydrogen production revenue, equipment losses, and carbon emissions, is used to collaboratively process power prediction results across multiple time scales. The DDPG algorithm learns and outputs the optimal control strategy for the wind-solar-hydrogen system through a deep neural network. Its architecture diagram is shown below. Figure 3 As shown, the DDPG algorithm includes an Actor network and a Critic network. The state space variables are input into the Actor network to generate actions and update the reward function. Simultaneously, the state space variables and action space variables are input into the Critic network to output the Q-value; the Actor network is then trained. The DDPG algorithm achieves precise control of the continuous action space, overcoming the limitations of discrete action space or discontinuous optimization in traditional DQN / PPO / MA-AC / OAC algorithms.

[0026] The multi-timescale power prediction results are used as the state input of the DDPG algorithm, and the DDPG algorithm is used to optimize the continuous state space and action space.

[0027] State space structure (50-dimensional continuous variables): 5-dimensional wind and solar status: real-time wind power, photovoltaic power, prediction error, power change rate, and weather type code; 6-dimensional electrolyzer status: voltage, current, temperature, hydrogen production, energy efficiency, and cumulative operating time; 4-dimensional hydrogen storage system: hydrogen storage tank pressure, temperature, hydrogen storage capacity, and valve opening; 3D economic signals: real-time electricity price, grid carbon intensity, and hydrogen market price; 20-dimensional historical status: the rate of change of wind and solar power and the rate of change of electricity price at 1-minute intervals over the past 10 minutes; 3D time encoding: hour, day of the week, season; Nine-dimensional safety constraints: number of start-stop cycles, number of maximum power changes, temperature anomaly indicators, pressure anomaly indicators, voltage anomaly indicators, current anomaly indicators, electrolyzer membrane humidity anomaly indicators, hydrogen storage tank liquid level anomaly indicators, and hydrogen purity anomaly indicators.

[0028] Extensive experimental verification has shown that the aforementioned 50-dimensional state space can adequately describe the operational state of a wind-solar-hydrogen coupled hydrogen production system. Too low a dimension leads to information loss and decreased policy performance; too high a dimension results in training difficulties and slow convergence.

[0029] Action space composition (3-dimensional continuous variables): Electrolytic cell power adjustment coefficient α: takes a value of 0-1, α=1 represents 100% power, α=0 represents shutdown, and α=0.2-1 is the allowable operating range of PEM electrolytic cell; Hydrogen storage tank inlet valve opening β: value 0-1, controls the balance between hydrogen production rate and hydrogen storage rate, β=0 means closed, β=1 means fully open; Fuel cell discharge coefficient γ: takes a value of 0-1, controls the discharge power, participates in peak shaving auxiliary services, γ=0 indicates no discharge.

[0030] Action constraint: |αt-α{t-1}| < 0.1 (to prevent large power fluctuations).

[0031] A multi-objective reward function that integrates hydrogen production revenue, equipment wear and tear, and carbon emissions is adopted to achieve synergistic optimization of economic efficiency, equipment lifespan, and environmental friendliness. The formula for the multi-objective reward function is as follows: R(t) = w 收益 ×R 收益 (t)-w 成本 ×R 成本 (t)-w 惩罚 ×R 惩罚 (t); R 收益 (t) = Hydrogen production × Hydrogen market price + Peak shaving ancillary service revenue; R 成本 (t) = Electricity purchase cost + Equipment maintenance cost + Carbon emission cost; R 惩罚 (t)=w 弃光 × light waste + w 启停 ×Number of starts and stops + w 波动 ×Power change amplitude + w 寿命 ×Life loss + w 碳 × Carbon emissions exceeded the limit; Where R(t) is the total reward at time step t, w 收益 As a weighting factor for hydrogen production revenue, R 收益 (t) represents the hydrogen production gain at time step t, w 成本 As a cost penalty weight, R 成本 (t) represents the cost of hydrogen production at time step t, w 惩罚 For safety and environmental constraints, R 惩罚 (t) represents the safety and environmental constraints and penalties at time step t, w 弃光 To punish the abandonment of wind and light, w 启停 The weight of the penalty for the number of start-stop cycles, w 波动 w is the penalty weight for the magnitude of power change. 寿命 For lifetime depletion penalty weights, w 碳 Weighting for penalties for exceeding carbon emission limits.

[0032] Weight configuration: w 收益 =1.2, w 成本 =0.8, w 惩罚 =0.5, w 弃光 =0.3, w 启停 =0.2, w 波动=0.15, w 寿命 =0.2, w 碳 =0.15.

[0033] By weighting and balancing multiple objectives: reducing wind and solar curtailment rates (economic efficiency), reducing start-up and shutdown frequency (equipment lifespan), smoothing power fluctuations (equipment lifespan), and reducing carbon emissions (environmental friendliness).

[0034] 4. Multi-timescale scheduling module Construct a three-tier scheduling framework: daily, hourly, and minute-level. Figure 4 As shown, it includes daily scheduling (long-term scheduling), hourly scheduling (medium-term scheduling), and minute-level scheduling (short-term scheduling + real-time control).

[0035] The daily scheduling adopts the MILP algorithm: the time granularity is 24 hours, with a decision point every hour, and the decision variables are the electrolyzer power, hydrogen storage capacity, and purchased electricity for each time period; Objective function: maxΣ(hydrogen production × market price - electricity purchase cost - equipment maintenance cost - carbon emission cost); Constraints: (1) Power balance: Wind and solar power + purchased electricity = electrolytic cell power + transmitted electricity; (2) Hydrogen storage constraints: Hydrogen storage capacity ∈ [20%, 95%] capacity; (3) Start-stop constraint: The number of start-stop cycles per day shall be ≤ 3; The hourly scheduling uses the MPC algorithm: the time granularity is 4 hours, with a decision point every 10 minutes; the plan for the next 4 hours is re-optimized every hour.

[0036] Minute-level scheduling employs the RL algorithm: the time granularity is real-time control, the control cycle is 10 seconds, and the electrolyzer power, valve opening, and fuel cell discharge power are adjusted in real time, responding to second-level fluctuations and correcting prediction errors based on real-time measurement data.

[0037] The three-layer scheduling framework employs the following collaborative scheduling mechanisms between its layers: Top-down breakdown: Daily plan → Hourly target → Minute setpoint; Bottom-up feedback: minute-level actual values ​​→ correct hourly forecasts → adjust daily plans; Conflict resolution: When there is a conflict between daily planning and real-time control, safety takes the highest priority; Based on the optimal control strategy output by the deep reinforcement learning decision engine, a three-layer scheduling framework of daily-hourly-minute is used to perform daily MILP planning, hourly MPC scheduling, and minute-level RL control. The hourly MPC scheduling tracks the daily MILP planning.

[0038] 5. Digital Twin Simulation Platform Hybrid modeling using Physical Information Neural Networks (PINN), such as... Figure 5 As shown, the electrochemical equations, thermal balance equations, and mass conservation equations of the physical model are used as part of the neural network loss function (physical constraint embedding). The loss function is as follows: L total = L data + λ physics × L physics ; ; Among them, L total For the total loss, L data For data error, L physics λ is the residual of the physical equation. physics It is an adjustable hyperparameter used to balance the dimensions and importance of data loss and physical loss, f physics For physical equations, such as the electrochemical equations of an electrolytic cell, the heat balance equations, and the mass conservation equations, N is the total number of residual calculation points, and y is the total number of residual calculation points. pred These are the predicted values ​​of the system state variables by the neural network, including electrolyzer voltage, current, temperature, hydrogen production, etc. Experimental data is used to correct physical model parameters, thereby improving accuracy. Neural network parameters are updated based on real-time data to adapt to system changes, enabling deep network fitting, residual learning, and adaptive weight adjustment. The optimal control strategy obtained from the deep reinforcement learning decision engine is then validated in real time. Simulation latency is reduced from 1 second to 50 milliseconds (a 95% reduction), meeting real-time control requirements.

[0039] Example 2 100MW wind-solar hydrogen production station application, system configuration: Renewable energy: 60MW wind power + 40MW solar power Electrolytic cells: 50 units of 2MW PEM electrolytic cells (total 100MW) Hydrogen storage system: 5000m³ / 35MPa hydrogen storage tank (can store approximately 4500kg of hydrogen). Fuel cells: 20MW PEMFC (for peak shaving ancillary services) Data acquisition: Sampling frequency 1Hz (wind and solar power), 10Hz (electrolyte cell status).

[0040] Operating scenario: Typical spring day: Abundant wind and solar resources, with an average daily power output of 80MW. Typical summer day: Solar power dominates, with power output exceeding 120MW at midday. Typical winter day: Wind power dominates, with nighttime power output <30MW Extreme weather: The typhoon caused wind power output to drop sharply to 5MW for 3 hours.

[0041] Table 1 shows a comparison of the operational performance between the control method of this invention and the traditional fixed threshold control method: Table 1 Comparison of running results ; As shown in Table 1, the control method of this invention can intelligently control and dynamically adjust the power of the electrolyzer, fully utilize renewable energy, and reduce the wind and solar curtailment rate from 15.2% to 1.8%; by integrating electricity price signals, hydrogen production is prioritized during off-peak hours (0.25 yuan / kWh), reducing the average electricity purchase cost from 0.5 yuan / kWh to 0.35 yuan / kWh; frequent start-stop cycles are avoided (the number of start-stop cycles is reduced from 150 times per year to 50 times) and large power fluctuations, extending the lifespan by 8,000 hours; participation in grid peak shaving ancillary services increases ancillary service revenue by 15 million yuan per year; and the use of green electricity to produce hydrogen reduces carbon emissions by 43%, meeting the EU CBAM standard.

[0042] Example 3 Robustness verification under extreme weather conditions Scenario description: A typhoon is hitting, and the wind power output drops sharply from 80MW to 5MW within 30 minutes; the electrolytic cell needs to quickly reduce its load to avoid safety risks.

[0043] Control strategy: Minute-level scheduling: The DDPG algorithm reduces the electrolyzer power from 100MW to 25MW within 10 seconds (the power regulation coefficient α is reduced from 1 to 0.25). Start-up and shutdown optimization: predict that wind and solar power will recover to >50MW within the next hour, avoid shutdown, and maintain low load operation of electrolyzers; Hydrogen storage system coordination: Hydrogen is released from the hydrogen storage tank and discharged through the fuel cell to compensate for the power gap.

[0044] Effectiveness evaluation: Response time: 50ms (meets real-time control requirements); Power control accuracy: ±2% (target ±5%); Electrolytic cell safety: No overvoltage, overcurrent, or overtemperature alarms; System stability: No downtime, continuous operation.

[0045] Example 3: Digital Twin Simulation Verification Simulation Scenario: Pre-simulating control strategies under a typhoon scenario The simulation results compare the performance of the DDPG strategy with that of traditional PID control, and are as follows: Simulation speed: Parallel simulation of 1000 operating conditions, total time <5 minutes; Simulation accuracy: State prediction error <2%, highly consistent with actual operating data; Strategy optimization: Simulation results showed that the DDPG strategy can reduce the wind and solar curtailment rate by 8 percentage points.

[0046] The key technical parameters for digital twin simulation verification are shown in Table 2: Table 2 Key Technical Parameters for Digital Twin Simulation Verification .

[0047] This invention employs the DDPG algorithm for continuous action space optimization. Through deep neural networks, it learns the optimal control strategy for the wind-solar-hydrogen system, enabling real-time response to second-level changes in wind and solar power, achieving continuous and smooth adjustment of the electrolyzer power, and thus precise control of the continuous action space. By integrating a multi-objective reward function that considers hydrogen production revenue, equipment wear and tear, and carbon emissions, the electrolyzer lifespan is extended by 15-20%. The PINN digital twin simulation verification platform accelerates strategy exploration, achieving model accuracy >98% (state prediction error <2%), simulation latency <50ms, and a 60% reduction in training time, meeting real-time control requirements.

[0048] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A wind-solar-hydrogen coupling hydrogen production control method based on deep reinforcement learning, characterized in that, Includes the following steps: S1. Collect wind power generation data, photovoltaic power generation data, water electrolysis hydrogen production data, power grid data, meteorological data, and carbon emission data, and perform data preprocessing; S2. Extract features from the preprocessed data, including historical power features, meteorological features, temporal features, and dimensional features; S3. The extracted time features are concatenated with historical power features through a time coding layer and then input into the LSTM model. The extracted time features are concatenated with meteorological features through a time coding layer and then input into the LSTM-Transformer model. Then, time cycle pattern recognition, seasonal feature extraction, and special holiday pattern labeling are performed for daily prediction to complete wind and solar power output prediction at multiple time scales, including daily, hourly, and minute-level predictions, and output multi-time scale power prediction results. S4. The output multi-timescale power prediction results are used as the state input of the DDPG algorithm. The DDPG algorithm is used to optimize the continuous state space and action space. The state space includes multi-dimensional continuous variables such as wind and solar status, electrolyzer status, hydrogen storage system, economic signals, historical status, time coding, and safety constraints. The action space includes three-dimensional continuous variables such as electrolyzer power adjustment coefficient, hydrogen storage tank inlet valve opening, and fuel cell discharge coefficient. A multi-objective reward function that integrates hydrogen production revenue, equipment loss, and carbon emissions is adopted. The DDPG algorithm learns the optimal control strategy of the wind-solar-hydrogen system through a deep neural network and outputs it. S5. Construct a three-tier scheduling framework: daily, hourly, and minute-level. The daily scheduling uses the MILP algorithm, the hourly scheduling uses the MPC algorithm, and the minute-level scheduling uses the RL algorithm. The following collaborative scheduling mechanisms are used between the levels: Top-down breakdown: Daily plan → Hourly target → Minute setpoint; Bottom-up feedback: minute-level actual values ​​→ correct hourly forecasts → adjust daily plans; Conflict resolution: When there is a conflict between daily planning and real-time control, safety takes the highest priority; Based on the optimal control strategy output in step S4, a three-level scheduling framework of daily-hourly-minute is used to perform daily MILP planning, hourly MPC scheduling, and minute-level RL control. S6. A hybrid modeling approach using physical information neural networks is adopted. The electrochemical equations, thermal balance equations, and mass conservation equations of the electrolytic cell in the physical model are used as part of the neural network loss function. The parameters of the physical model are corrected using experimental data, and the parameters of the neural network are updated based on real-time data. The optimal control strategy is verified in real time through the hybrid model of physical information neural networks, and the performance and safety of the optimal control strategy are evaluated.

2. The wind-solar-hydrogen coupling hydrogen production control method based on deep reinforcement learning as described in claim 1, characterized in that, In step S1, the wind power generation data includes wind speed, wind direction, pitch angle, power output, and wind turbine status; The photovoltaic power generation data includes irradiance, module temperature, power output, and photovoltaic array status; The water electrolysis hydrogen production data includes electrolyzer voltage, current, temperature, hydrogen production, energy efficiency and membrane resistance, hydrogen storage tank pressure, temperature, hydrogen storage capacity and valve status; The power grid data includes power frequency, voltage, and electricity price; The meteorological data includes temperature, humidity, air pressure, cloud cover, and atmospheric pressure load. The carbon emission data includes real-time grid carbon intensity.

3. The wind-solar-hydrogen coupling hydrogen production control method based on deep reinforcement learning as described in claim 2, characterized in that, In step S2, the historical power characteristics include statistical characteristics such as power mean, variance, maximum value, minimum value, and peak-to-valley difference. Meteorological characteristics: including current values, rate of change, and trend characteristics of wind speed, wind direction, temperature, and air pressure; Time characteristics: including periodic time codes such as hours, days of the week, and seasons, as well as characteristics of the same historical period; Dimensional features: Dimensional information of the input data, used for model adaptive processing.

4. The wind-solar-hydrogen coupling hydrogen production control method based on deep reinforcement learning as described in claim 1, characterized in that, In step S3, the basis for the multi-timescale wind and solar power output prediction is: Daily forecast: Based on 7-day NWP data, predict the 24-hour wind and solar power output curve; Hourly forecast: Based on data from the past 24 hours, predict power for the next 1-4 hours; Minute-level prediction: Based on data from the past hour, predict power for the next 10-30 minutes.

5. The wind-solar-hydrogen coupling hydrogen production control method based on deep reinforcement learning as described in claim 1, characterized in that, The state space includes multidimensional continuous variables such as wind and solar status, electrolyzer status, hydrogen storage system, economic signals, historical status, time coding, and safety constraints. The action space includes three-dimensional continuous variables such as electrolyzer power adjustment coefficient, hydrogen storage tank inlet valve opening, and fuel cell discharge coefficient.

6. The wind-solar-hydrogen coupling hydrogen production control method based on deep reinforcement learning as described in claim 5, characterized in that, The state space comprises 50 continuous variables: 5-dimensional wind and solar status: real-time wind power, photovoltaic power, prediction error, power change rate, and weather type code; 6-dimensional electrolyzer status: voltage, current, temperature, hydrogen production, energy efficiency, and cumulative operating time; 4-dimensional hydrogen storage system: hydrogen storage tank pressure, temperature, hydrogen storage capacity, and valve opening; 3D economic signals: real-time electricity price, grid carbon intensity, and hydrogen market price; 20-dimensional historical status: the rate of change of wind and solar power and the rate of change of electricity price at 1-minute intervals over the past 10 minutes; 3D time encoding: hour, day of the week, season; Nine-dimensional safety constraints: number of start-stop cycles, number of maximum power changes, temperature anomaly indicators, pressure anomaly indicators, voltage anomaly indicators, current anomaly indicators, electrolyzer membrane humidity anomaly indicators, hydrogen storage tank liquid level anomaly indicators, and hydrogen purity anomaly indicators.

7. The wind-solar-hydrogen coupling hydrogen production control method based on deep reinforcement learning as described in claim 1, 5, or 6, characterized in that, In step S4, the formula for the multi-objective reward function that integrates hydrogen production revenue, equipment wear and tear, and carbon emissions is as follows: R(t)=w 收益 ×R 收益 (t)-w 成本 ×R 成本 (t)-w 惩罚 ×R 惩罚 (t); R 收益 (t) = Hydrogen production × Hydrogen market price + Peak shaving ancillary service revenue; R 成本 (t) = Electricity purchase cost + Equipment maintenance cost + Carbon emission cost; R 惩罚 (t)=w 弃光 × light waste + w 启停 ×Number of starts and stops + w 波动 ×Power change amplitude + w 寿命 ×Life loss + w 碳 × Carbon emissions exceeded the limit; Where R(t) is the total reward at time step t, w 收益 As a weighting factor for hydrogen production revenue, R 收益 (t) represents the hydrogen production gain at time step t, w 成本 As a cost penalty weight, R 成本 (t) represents the cost of hydrogen production at time step t, w 惩罚 For safety and environmental constraints, R 惩罚 (t) represents the safety and environmental constraints and penalties at time step t, w 弃光 To punish the abandonment of wind and light, w 启停 The weight of the penalty for the number of start-stop cycles, w 波动 w is the penalty weight for the magnitude of power change. 寿命 For lifetime depletion penalty weights, w 碳 Weighting for penalties for exceeding carbon emission limits.

8. The wind-solar-hydrogen coupling hydrogen production control method based on deep reinforcement learning as described in claim 7, characterized in that, the w 收益 = 1.2, w 成本 = 0.8, w 惩罚 = 0.5, w 弃光 = 0.3, w 启停 = 0.2, w 波动 = 0.15, w 寿命 = 0.2, w 碳 = 0.

15.

9. A wind-solar-hydrogen coupling hydrogen production control system based on deep reinforcement learning, characterized in that, The method for implementing the deep reinforcement learning-based wind-solar-hydrogen coupling hydrogen production control method according to any one of claims 1-8 includes: (1) Multi-source data acquisition and preprocessing module: used to acquire wind power generation data, photovoltaic power generation data, water electrolysis hydrogen production data, power grid data, meteorological data and carbon emission data, and preprocess the acquired data; (2) Wind and solar power output prediction module: The LSTM+Transformer hybrid model is used to extract historical power features, meteorological features, time features and dimensional features from the data output by the multi-source data acquisition and preprocessing module. Based on the extracted features, wind and solar power output prediction is performed at multiple time scales, and the multi-time scale power prediction results are output. (3) Deep reinforcement learning decision engine: The DDPG algorithm and a multi-objective reward function that integrates hydrogen production revenue, equipment loss and carbon emissions are used to process the output results of the wind and solar power output prediction module to obtain the optimal control strategy of the wind and solar hydrogen system. (4) Multi-timescale scheduling module: It has a built-in three-layer scheduling framework of daily-hour-minute. The daily scheduling adopts the MILP algorithm, the hourly scheduling adopts the MPC algorithm, and the minute scheduling adopts the RL algorithm. The multi-timescale scheduling module is used to perform daily MILP planning, hourly MPC scheduling and minute-level RL control based on the optimal control strategy obtained by the deep reinforcement learning decision engine. (5) Digital twin simulation platform: PINN hybrid modeling is used to verify the performance and safety of the optimal control strategy obtained by the deep reinforcement learning decision engine in real time.

Images