Cooperative control method and device for offshore wind power-energy storage-hydrogen production system and medium

Abstract

The invention relates to an offshore wind power-energy storage-hydrogen production system cooperative control method and device and a medium, and the method comprises the steps: collecting wind power plant operation data, energy storage system data and hydrogen production system data in real time, and obtaining a state vector; the state vector is input into a strategy network, a current action vector is calculated, the strategy network is obtained through training of a double-evaluator reinforcement learning framework, a first evaluator evaluates an economical efficiency target value function, and a second evaluator evaluates a safety target value function; the strategy network outputs a control instruction corresponding to the current action vector, and issues the control instruction to each execution device for adjustment, including adjusting fan operation parameters according to the axial induction factor and the yaw angle; adjusting the electrolytic cell input power according to the electrolytic load power; and controlling the charging and discharging operation of the energy storage system according to the energy storage power. Compared with the prior art, the method has the advantages of high collaboration, high adaptability, safety and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of energy control, in particular to a sea wind power-energy storage-hydrogen production system cooperative control method, device and medium. BACKGROUND

[0002] In the development of new power systems, offshore wind power has become an important part of the renewable energy field due to its abundant resources and high stability. However, the output power of offshore wind power is significantly affected by wind speed fluctuations and weather conditions, with high uncertainty and intermittency, which leads to problems such as high curtailment rate and frequent start-stop of equipment during grid-connected operation. In order to improve the local consumption capacity of offshore wind power, a multi-energy complementary mode driven by wind power to produce hydrogen through electrolysis combined with energy storage systems has been developed in recent years, which realizes flexible storage and utilization of energy by converting wind energy into hydrogen energy.

[0003] Existing offshore wind power-energy storage-hydrogen production systems are usually composed of wind turbines, electrolytic cells and energy storage devices. The wind power output is first allocated to hydrogen production equipment and energy storage systems by a dispatch controller, the electrolytic cell consumes electric energy to produce hydrogen, and the energy storage device is used to balance power fluctuations and realize stable operation of the system. However, the control of this system faces multiple challenges: on the one hand, the rapid changes in wind speed and the wake effect between wind turbines result in uneven distribution of wind power output in space and time, and the operating parameters of the wind turbines need to be adjusted in real time to optimize power output; on the other hand, the hydrogen production equipment has nonlinear operating characteristics such as minimum starting power limit, efficiency interval change, etc., and the energy storage system is also subject to state of charge (SOC) boundary and charge / discharge rate constraints. These factors make the system need to dynamically coordinate multiple objectives such as power balance, equipment efficiency, and operation safety, and the control task is extremely complex.

[0004] Traditional control methods mainly adopt rule-based power distribution strategies or static optimization models. This kind of method usually relies on a pre-set system model, generates a control strategy through offline calculation, and executes according to fixed logic during operation. Although this kind of method is simple to implement, it is difficult to adapt to rapid changes in wind speed and sudden conditions of the system. In addition, traditional methods often use a single objective function, which cannot effectively coordinate the conflicts between different control objectives, for example, improving hydrogen production efficiency may cause frequent start-stop of electrolytic cells, and reducing wind curtailment may cause the energy storage system to enter a limit state, thereby affecting system safety.

[0005] In recent years, reinforcement learning, as a data-driven intelligent control method, has been introduced into the control of complex systems due to its advantages of not requiring precise modeling and its strong adaptability. However, existing reinforcement learning methods typically optimize multiple control objectives by weighting and combining them into a single reward function. This design has significant shortcomings: First, the physical characteristics and numerical scales of different objectives vary greatly, making it difficult for a unified reward function to accurately represent the needs of each objective; second, there are mutual constraints between objectives, and the training process is prone to policy instability or convergence difficulties due to objective interference; finally, for highly sparsity safety constraint objectives, such as the number of start-stop cycles and SOC exceeding limits, their signals in the unified reward function are easily masked by the primary objective, resulting in poor learning performance. In addition, the system involves a large number of continuous control variables, such as wind turbine parameters and energy storage power, and existing methods struggle to achieve fine-grained control and constraint satisfaction in dynamic environments, especially when wind speed fluctuates drastically or the system approaches its operating boundary, further degrading control performance. Therefore, how to avoid training interference from targets with different physical characteristics under the same evaluation path and improve the learning ability for low-frequency event targets in order to achieve efficient, safe and stable operation of offshore wind power-energy storage-hydrogen production systems is a technical problem that needs to be solved. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the existing technology by providing a collaborative control method, equipment and medium for offshore wind power-energy storage-hydrogen production system. An environmental model is constructed based on actual wind speed data, and hydrogen production and safety objectives are optimized separately through a dual evaluator network, avoiding the objective conflict problem in a single evaluator model and improving the convergence speed and training stability of the strategy.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] According to one aspect of the present invention, a method for coordinated control of an offshore wind power-energy storage-hydrogen production system is provided, the specific steps of which include:

[0009] S1. Real-time acquisition of wind farm operation data, energy storage system data, and hydrogen production system data, and preprocessing to obtain a state vector including the inflow wind speed of the wind turbine, the state of charge of the energy storage system, the electrolysis load power, and the energy storage power.

[0010] S2. Input the state vector into the policy network based on the Markov decision process to calculate the current action vector. The policy network is trained through a dual evaluator reinforcement learning framework, wherein the first evaluator evaluates the economic objective value function and the second evaluator evaluates the safety objective value function.

[0011] S3. The strategy network outputs control commands corresponding to the current action vector, including the axial induction factor, yaw angle, electrolytic load power, and energy storage power of the wind turbine; the control commands are sent to each execution device for adjustment, including: adjusting the wind turbine operating parameters according to the axial induction factor and yaw angle; adjusting the input power of the electrolytic cell according to the electrolytic load power; and controlling the charging and discharging operation of the energy storage system according to the energy storage power.

[0012] Furthermore, the first evaluator and the second evaluator have the same structure but independent parameters, and the network training uses separate optimizers.

[0013] Furthermore, the input to the first evaluator is the concatenated features of the state vector and action vector, and the output is a state-action value estimate of the economic objective, that is, the long-term discounted return on hydrogen production incentives after taking the corresponding action in the current state, and the economic objective value function. The expression is:

[0014]

[0015] in, γ is the hydrogen production reward at time step t; γ is the discount factor used to weigh the impact of current rewards against future rewards; E π This represents the mathematical expectation of the future trajectory under the current strategy π; s is the current state, and a is the corresponding action;

[0016] Loss function of the first evaluator for:

[0017]

[0018] Where |B| is the size of the batch sampling data, that is, the number of samples extracted in the experience playback buffer.

[0019] Furthermore, the input to the second evaluator is the concatenated features of the state vector and action vector, and the output is a state-action value estimate of the safety objective, i.e., the long-term expected penalty value of operational stability after taking the corresponding action in the current state. This is used to guide the system to avoid unstable behaviors including frequent start-stops, SOC overruns, and wind curtailment. The safety objective value function... The expression is:

[0020]

[0021] Where, r safe,t The penalty for time step t includes penalties for electrolyzer start-up and shutdown, SOC exceeding limits, and wind curtailment; γ is a discount factor used to weigh the impact of current penalties against future rewards; E π This represents the mathematical expectation of the future trajectory under the current strategy π; s is the current state, and a is the corresponding action;

[0022] Loss function of the second evaluator for:

[0023]

[0024] Where |B| is the size of the batch sampling data, that is, the number of samples extracted in the experience playback buffer.

[0025] Furthermore, the first evaluator and the second evaluator achieve multi-objective fusion control of the policy network through a joint policy optimization mechanism. The optimization objective is to maximize the joint value function, which is expressed as follows:

[0026]

[0027] in, The value of hydrogen production targets, The value function represents the safety objective, where λ is a weighting coefficient. The weighting coefficient is adjusted to balance the optimization objectives between hydrogen production and operational stability.

[0028] Furthermore, the state vector in S1 is obtained from the dynamic model of the offshore wind power-storage-electrolysis hydrogen production system, which includes the wake effect. The dynamic model of the offshore wind power-storage-electrolysis hydrogen production system includes a wind power sub-model, a power flow distribution sub-model, and an energy storage and hydrogen production state sub-model.

[0029] Furthermore, the wind power sub-model is based on the influence of wake interference on the wind turbine inlet velocity, and establishes the wind turbine output power P through actuator disk theory and Gaussian wake model. WT The expression:

[0030]

[0031] Where α is the axial induction factor, γ is the yaw angle, ρ is the air density, and D is the rotor diameter. P is the average wind speed over the rotor surface. P This is an empirically adjusted index.

[0032] The power flow distribution sub-model dynamically allocates power among the three paths of energy storage, electrolysis, and wind curtailment during wind power output in the system, as expressed by:

[0033] P WT (t)=P bat (t)+P H (t)+P dump (t),

[0034] Among them, P bat The charging and discharging power of the energy storage system, with a positive value for charging, P hP is the power of the electrolytic load. dump This refers to the power output of the wind curtailment.

[0035] The energy storage and hydrogen production state sub-model calculates the state of charge (SOC) of the energy storage system, expressed as follows:

[0036]

[0037] Where, η ch ,η dis These represent the charge / discharge efficiency, Δt is the control period, and E... bat It refers to energy storage capacity.

[0038] According to another aspect of the present invention, a coordinated control system for an offshore wind power-energy storage-hydrogen production system is provided, the system comprising a data acquisition module, a dual evaluator module, a strategy decision-making module, and an execution control module.

[0039] The data acquisition module is used to collect wind farm operation data, energy storage system data and hydrogen production system data in real time, and perform preprocessing to obtain a state vector including the inflow wind speed of the wind turbine, the state of charge of the energy storage system, the electrolysis load power and the energy storage power.

[0040] The policy decision module includes a policy network based on a Markov decision process according to the state vector, used to calculate the current action vector. The policy network is trained through a dual evaluator reinforcement learning framework in the dual evaluator module.

[0041] The dual evaluator module includes a first evaluator and a second evaluator. The first evaluator evaluates the economic objective value function, and the second evaluator evaluates the safety objective value function.

[0042] The execution control module is used to send the control commands corresponding to the current action vector, including the axial induction factor, yaw angle, electrolytic load power and energy storage power of the wind turbine, to each execution device for adjustment, including: adjusting the wind turbine operating parameters according to the axial induction factor and yaw angle; adjusting the input power of the electrolytic cell according to the electrolytic load power; and controlling the charging and discharging operation of the energy storage system according to the energy storage power.

[0043] According to a second aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described thereon.

[0044] According to a third aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method described thereon.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] (1) Enhanced multi-objective collaborative optimization capability: Through the dual evaluator reinforcement learning framework, the economic objective, namely hydrogen production, and the safety objective, namely system operation constraints, are modeled as independent value functions, avoiding the interference problem between different objectives in the traditional single evaluator model. Through the joint strategy optimization mechanism, hydrogen production efficiency and system safety are dynamically balanced, enabling the strategy network to optimize multiple conflicting objectives at the same time, thereby improving the overall control performance of the system.

[0047] (2) Enhanced learning effect of sparse constraint targets: For low-frequency but critical constraint signals such as electrolytic cell start-up and shutdown, SOC overrun, etc., this invention performs special optimization through an independent safety evaluator, namely the second evaluator, which solves the problem that sparse reward signals are easily masked by the main target in traditional methods. The safety evaluator enhances the sensitivity and response capability to operational safety indicators by specially processing the penalty signal, ensuring that the system can still meet physical constraints under complex working conditions.

[0048] (3) Improved adaptability to dynamic environment and stability of strategy: The use of wake effect modeling and dual evaluator parallel training mechanism can more accurately characterize dynamic characteristics such as wind speed fluctuation and power distribution. The strategy network can adaptively balance exploration and utilization during training through entropy regularization and dynamic weight adjustment, avoiding the problem of strategy oscillation or constraint failure that is easy to occur under strong disturbances in traditional methods, thereby improving the robustness and engineering applicability of the system in variable scenarios. Attached Figure Description

[0049] Figure 1 A flowchart of a coordinated control method for an offshore wind power-energy storage-hydrogen production system;

[0050] Figure 2 This is a schematic diagram illustrating the effect of the wake effect of the front fan. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0052] Example 1

[0053] In existing control methods for offshore wind power-hydrogen production-energy storage systems, reinforcement learning models are typically constructed using a unified objective function. This approach fails to accurately represent the differences between various objectives, such as hydrogen production efficiency and system operational constraints. Consequently, significant interference between different objectives occurs during policy training, making it difficult to guarantee control accuracy and stability. Furthermore, reinforcement learning methods generally employ a single-policy evaluation path in control structure design, lacking a decoupling mechanism for the physical characteristics of multiple objectives. This is particularly problematic when there are inconsistencies in response frequency and gradient scale between economic objectives (hydrogen production) and safety constraints, easily leading to policy optimization imbalances and degraded system performance. In addition, the system operation involves numerous boundary constraint objectives, such as upper and lower limits of energy storage SOC, electrolysis power thresholds, and start-stop frequency limits. These objective constraints exhibit discontinuous and sparse reward / penalty signals, resulting in poor learning performance under a unified structure. They are often masked by the primary objective, making it difficult to effectively guarantee operational safety.

[0054] In the complex operating conditions of actual offshore wind power-hydrogen production-energy storage systems, when wind speed disturbances are frequent, wind field distribution is heterogeneous, or energy storage is close to its limit, the generalization ability of existing strategy control methods is insufficient, and phenomena such as strategy instability and control variable output jumps are prone to occur, which limits their deployment and promotion in engineering practice.

[0055] like Figure 1 As shown, this embodiment provides a method for coordinated control of an offshore wind power-energy storage-hydrogen production system, the specific steps of which include:

[0056] S1. Real-time acquisition of wind farm operation data, energy storage system data, and hydrogen production system data, and preprocessing to obtain a state vector including the inflow wind speed of the wind turbine, the state of charge of the energy storage system, the electrolysis load power, and the energy storage power.

[0057] S2. Input the state vector into the policy network based on the Markov decision process to calculate the current action vector. The policy network is trained through a dual evaluator reinforcement learning framework, where the first evaluator evaluates the economic objective value function and the second evaluator evaluates the safety objective value function.

[0058] S3. The strategy network outputs control commands corresponding to the current action vector, including the axial induction factor, yaw angle, electrolytic load power, and energy storage power of the wind turbine; the control commands are then sent to each execution device for adjustment, including: adjusting the wind turbine operating parameters according to the axial induction factor and yaw angle; adjusting the input power of the electrolytic cell according to the electrolytic load power; and controlling the charging and discharging operation of the energy storage system according to the energy storage power.

[0059] like Figure 2The diagram shown illustrates the effect of the wake effect of the front-row wind turbines. The state vector in S1 is obtained from the dynamic model of the offshore wind power-storage-electrolysis hydrogen production system, which includes the wake effect. The dynamic model of the offshore wind power-storage-electrolysis hydrogen production system includes a wind power sub-model, a power flow distribution sub-model, and an energy storage and hydrogen production state sub-model.

[0060] The wind power sub-model is based on the influence of wake disturbance on the wind turbine inlet velocity. The wind turbine output power P is established through actuator disk theory and Gaussian wake model. WT The expression:

[0061]

[0062] Where α is the axial induction factor, γ is the yaw angle, ρ is the air density, and D is the rotor diameter. P is the average wind speed over the rotor surface. P This is an empirical correction index. The wind power sub-model reflects the nonlinear response characteristics of wind turbine power to control variables.

[0063] The power flow distribution sub-model dynamically allocates power among the three paths of energy storage, electrolysis, and wind curtailment during wind power output in the system, satisfying the following power balance expression:

[0064] P WT (t)=P bat (t)+P H (t)+P dump (T),

[0065] Among them, P bat The charging and discharging power of the energy storage system, with a positive value for charging, P H P is the power of the electrolytic load. dump This represents the curtailed wind power. The power flow allocation sub-model represents the internal energy path coupling of the system and serves as the basis for constructing action constraints for subsequent control strategies.

[0066] The energy storage and hydrogen production state sub-model calculates the state of charge (SOC) of the energy storage system, and the evolution equation is as follows:

[0067]

[0068] Where, η ch ,η dis These represent the charge / discharge efficiency, Δt is the control period, and E... bat This refers to energy storage capacity. The energy storage and hydrogen production state sub-model is used to track the energy storage state and is the core basis for safety constraint evaluation.

[0069] The formula for calculating the hydrogen production capacity of an electrolyzer based on its electrical energy conversion efficiency is as follows:

[0070]

[0071] Where, η EL For electrolysis efficiency, H HVH This represents the higher heating value of hydrogen. This expression is used to assess the quality of hydrogen production during a control cycle, forming the basis for economic evaluation.

[0072] The dynamic model of the offshore wind power-energy storage-electrolysis hydrogen production system constitutes a unified dynamic modeling platform for the system, possessing continuous doability, control relevance, and physical consistency. It serves both as a realistic environment for intelligent agent training and interaction and as a resolvable control feedback interface.

[0073] Based on the dynamic model of an offshore wind power-energy storage-electrolysis hydrogen production system, the core elements of the Markov decision process corresponding to each control cycle are defined. The state space includes the inflow wind speed of the wind turbine, the state of charge of the energy storage system, the electrolysis load power, and the energy storage power, with the state vector being S. i ={V i SOC i ,P H,i ,P bat,i}, where V i Let be the inflow velocity of the i-th fan, and SOC. i Let P be the state of charge of the i-th wind turbine energy storage system. H,i Let P be the electrolytic load power of the i-th fan. bat,i Let be the energy storage power of the i-th wind turbine. The action space includes the axial induction factor, yaw angle, electrolytic power, and energy storage power of the wind turbine, and the action vector is a. i ={α i ,γ i ,P H,i ,P bat,i}, where α i Let γ be the axial induction factor of the i-th wind turbine. i Let yaw angle be the yaw angle of the i-th wind turbine.

[0074] The reward function includes economic rewards and safety rewards, where the objective of the economic reward is to maximize hydrogen production per unit time. The expression for the economic reward function is as follows:

[0075] r H2 =log(1+E) H ),

[0076] Among them, E H This represents the amount of hydrogen produced by the electrolyzer per unit time.

[0077] Security rewards are used to measure system stability, and are expressed as follows:

[0078] r safe =-(λ1N) shut +λ2N soc+λ3P dump ),

[0079] Where, N shut N is the starting and stopping penalty for the electrolytic cell. soc For the penalty of charge exceeding the limit, P dump Let λ represent the wind curtailment power, and λ1, λ2, and λ3 represent the weighting coefficients of each penalty term.

[0080] The final total reward function consists of two parts: economic reward and security reward.

[0081] r = c1r H2 +c2r safe ,

[0082] Where c1 and c2 are weighting coefficients.

[0083] To enhance the expressive power and convergence efficiency of multi-objective learning, this embodiment constructs a dual-channel state-action value function network structure. This network optimizes the long-term discounted rewards corresponding to the hydrogen production objective and the operational stability objective, respectively. By using two independent value networks, the objectives are decoupled, ensuring that each objective is optimized independently, thus improving the system's learning efficiency and stability. The two independent value networks include a first evaluator and a second evaluator. The first and second evaluators have identical structures but independent parameters, and the network training employs separate optimizers.

[0084] The first evaluator takes as input the concatenated features of the state vector and action vector, and outputs a state-action value estimate of the economic objective, i.e., the long-term discounted return on hydrogen production incentives after taking the corresponding action in the current state, and the economic objective value function. The expression is:

[0085]

[0086] in, γ is the hydrogen production reward at time step t; γ is the discount factor used to weigh the impact of current rewards against future rewards; E π This represents the mathematical expectation of the future trajectory under the current policy π; s is the current state, and a is the corresponding action. This evaluator focuses on estimating the long-term returns related to hydrogen production, maximizing hydrogen production.

[0087] Loss function of the first evaluator Measurement estimate Compared with the actual reward r H2 The difference between them is expressed as:

[0088]

[0089] Where |B| is the size of the batch sampling data, that is, the number of samples extracted in the experience playback buffer.

[0090] The second evaluator takes as input the concatenated features of the state vector and action vector, and outputs a state-action value estimate of the safety objective, i.e., the long-term expected penalty for operational stability after taking the corresponding action in the current state. This value estimate guides the system to avoid unstable behaviors such as frequent start-stops, SOC overruns, and wind curtailment. The safety objective value function is... The expression is:

[0091]

[0092] Where, r safe,t The penalty for time step t includes penalties for electrolyzer start-up and shutdown, SOC exceeding limits, and wind curtailment; γ is a discount factor used to weigh the impact of current penalties against future rewards; E π This represents the mathematical expectation of the future trajectory under the current policy π; s is the current state, and a is the corresponding action. This evaluator aims to avoid system instability and ensure that the system operates within a safe range.

[0093] Loss function of the second evaluator Measurement estimate Compared with the actual reward r safe The difference between them, used to supervise their learning accuracy, is expressed as:

[0094]

[0095] Where |B| is the size of the batch sampling data, that is, the number of samples extracted in the experience playback buffer.

[0096] The dual evaluator network can optimize hydrogen production and safety objectives separately, avoiding the objective conflict problem in the single evaluator model, and improving the convergence speed and training stability of the strategy.

[0097] The first and second evaluators achieve multi-objective fusion control of the policy network through a joint policy optimization mechanism, thereby enhancing the system's multi-objective regulation capability. This mechanism guides the policy network to learn multi-objective trade-offs during the update process by fusing the state-action value functions of the hydrogen production objective and the operational safety objective. The optimization objective is to maximize the joint value function. The expression for the joint value function is:

[0098]

[0099] in, The value of hydrogen production targets, The value function represents the safety objective, where λ is a weighting coefficient. The weighting coefficient is adjusted to balance the optimization objectives between hydrogen production and operational stability.

[0100] To further enhance adaptability, an adaptive adjustment mechanism for the entropy term weights is introduced, with the following update rule:

[0101]

[0102] Where α is the entropy regularization parameter, λ α Let L(α) be the learning rate and L(α) be the entropy objective loss function. Through this mechanism, the system can automatically adjust the exploration intensity of the policy to adapt to the policy learning needs under different operating conditions. Through a joint optimization mechanism, multi-objective fusion is performed under a unified policy framework. The policy network can not only optimize hydrogen production but also ensure the system's operational stability. During the control process, it can effectively regulate various system performance indicators, avoiding the interference and conflict problems between objectives in traditional methods.

[0103] This embodiment constructs an environmental model based on actual wind speed data. Through interaction sampling between the reinforcement learning agent and the system environment, it collects state, action, reward, and transition data, and continuously optimizes the policy network and dual evaluators using an experience replay mechanism. Training data supports both full-year wind speed data and segmented sliding window sample generation to improve the model's generalization ability and stability under different wind conditions. After training and policy convergence, the network weight parameters are fixed and deployed to edge control devices. The system can acquire wind speed data in real time, construct the current state vector, and output corresponding control commands through the policy network to drive the coordinated control of the induction factor, yaw angle, electrolysis load, and energy storage device. It continuously balances random exploration training and experience-based optimization using automatic model parameter tuning until the agent converges to the optimal control strategy. Finally, the trained reinforcement learning agent is deployed to the offshore wind power-energy storage-electrolysis hydrogen production system to monitor the system's dynamic hydrogen production in real time, outputting real-time commands to each control device to increase hydrogen production.

[0104] Example 2

[0105] Based on the method in Example 1, this example also provides a collaborative control system for an offshore wind power-energy storage-hydrogen production system, including a data acquisition module, a dual evaluator module, a strategy decision-making module, and an execution control module.

[0106] The data acquisition module is used to collect wind farm operation data, energy storage system data and hydrogen production system data in real time, and perform preprocessing to obtain a state vector including the inflow wind speed of the wind turbine, the state of charge of the energy storage system, the electrolysis load power and the energy storage power.

[0107] The policy decision module includes a policy network based on the Markov decision process according to the state vector, which is used to calculate the current action vector. The policy network is trained through the dual evaluator reinforcement learning framework in the dual evaluator module.

[0108] The dual evaluator module includes a first evaluator and a second evaluator. The first evaluator evaluates the economic objective value function, and the second evaluator evaluates the safety objective value function.

[0109] The execution control module is used to send control commands to each execution device for adjustment based on the control commands corresponding to the current action vector, including the axial induction factor, yaw angle, electrolytic load power, and energy storage power of the wind turbine. This includes: adjusting the wind turbine operating parameters according to the axial induction factor and yaw angle; adjusting the input power of the electrolytic cell according to the electrolytic load power; and controlling the charging and discharging operation of the energy storage system according to the energy storage power.

[0110] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the described module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0111] The electronic device of this invention includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) or loaded from a storage unit into random access memory (RAM). The RAM may also store various programs and data required for device operation. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0112] Multiple components in the device are connected to an I / O interface, including: input units such as a keyboard, mouse, etc.; output units such as various types of displays, speakers, etc.; storage units such as disks, optical disks, etc.; and communication units such as network interface cards, modems, wireless transceivers, etc. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks. The processing unit performs the various methods and processes described above, such as the method of the present invention. For example, in some embodiments, the method of the present invention may be implemented as a computer software program tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via ROM and / or the communication unit. When the computer program is loaded into RAM and executed by the CPU, one or more steps of the method of the present invention described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute the method of the present invention by any other suitable means (e.g., by means of firmware).

[0113] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0114] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0115] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0116] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for coordinated control of an offshore wind power-energy storage-hydrogen production system, characterized in that, The specific steps include: S1. Real-time acquisition of wind farm operation data, energy storage system data, and hydrogen production system data, and preprocessing to obtain a state vector including the inflow wind speed of the wind turbine, the state of charge of the energy storage system, the electrolysis load power, and the energy storage power. S2. Input the state vector into the policy network based on the Markov decision process to calculate the current action vector. The policy network is trained through a dual evaluator reinforcement learning framework, wherein the first evaluator evaluates the economic objective value function and the second evaluator evaluates the safety objective value function. S3. The strategy network outputs control commands corresponding to the current action vector, including the axial induction factor, yaw angle, electrolytic load power, and energy storage power of the wind turbine; the control commands are sent to each execution device for adjustment, including: adjusting the wind turbine operating parameters according to the axial induction factor and yaw angle; adjusting the input power of the electrolytic cell according to the electrolytic load power; and controlling the charging and discharging operation of the energy storage system according to the energy storage power.

2. The method for coordinated control of an offshore wind power-energy storage-hydrogen production system according to claim 1, characterized in that, The first evaluator and the second evaluator have the same structure but independent parameters, and the network training uses separate optimizers.

3. The method for coordinated control of an offshore wind power-energy storage-hydrogen production system according to claim 2, characterized in that, The first evaluator takes as input the concatenated features of the state vector and action vector, and outputs as a state-action value estimate of the economic objective, i.e., the long-term discounted return on hydrogen production incentives after taking the corresponding action in the current state, and the economic objective value function. The expression is: in, γ is the hydrogen production reward at time step t; γ is the discount factor used to weigh the impact of current rewards against future rewards; E π This represents the mathematical expectation of the future trajectory under the current strategy π; s is the current state, and a is the corresponding action; Loss function of the first evaluator for: Where |B| is the size of the batch sampling data, that is, the number of samples extracted in the experience playback buffer.

4. The method for coordinated control of an offshore wind power-energy storage-hydrogen production system according to claim 2, characterized in that, The second evaluator takes as input the concatenated features of the state vector and action vector, and outputs a state-action value estimate of the safety objective, i.e., the long-term expected penalty value for operational stability after taking the corresponding action in the current state. This value is used to guide the system to avoid unstable behaviors such as frequent start-stops, SOC overruns, and wind curtailment. (Safety objective value function) The expression is: Where, r safe,t The penalty for time step t includes penalties for electrolyzer start-up and shutdown, SOC exceeding limits, and wind curtailment; γ is a discount factor used to weigh the impact of current penalties against future rewards; E π This represents the mathematical expectation of the future trajectory under the current strategy π; s is the current state, and a is the corresponding action; Loss function of the second evaluator for: Where |B| is the size of the batch sampling data, that is, the number of samples extracted in the experience playback buffer.

5. The method for coordinated control of an offshore wind power-energy storage-hydrogen production system according to claim 1, characterized in that, The first evaluator and the second evaluator achieve multi-objective fusion control of the policy network through a joint policy optimization mechanism. The optimization objective is to maximize the joint value function, which is expressed as follows: in, The value of hydrogen production targets, The value function represents the safety objective, where λ is a weighting coefficient. The weighting coefficient is adjusted to balance the optimization objectives between hydrogen production and operational stability.

6. The method for coordinated control of an offshore wind power-energy storage-hydrogen production system according to claim 1, characterized in that, The state vector in S1 is obtained from the dynamic model of the offshore wind power-storage-electrolysis hydrogen production system, which includes the wake effect. The dynamic model of the offshore wind power-storage-electrolysis hydrogen production system includes a wind power sub-model, a power flow distribution sub-model, and an energy storage and hydrogen production state sub-model.

7. The method for coordinated control of an offshore wind power-energy storage-hydrogen production system according to claim 6, characterized in that, The wind power sub-model is based on the influence of wake interference on the wind turbine's inlet velocity. The wind turbine output power P is established using actuator disk theory and a Gaussian wake model. WT The expression: Where α is the axial induction factor, γ is the yaw angle, ρ is the air density, and D is the rotor diameter. P is the average wind speed over the rotor surface. P This is an empirically adjusted index. The power flow distribution sub-model dynamically allocates power among the three paths of energy storage, electrolysis, and wind curtailment during the system's wind power output. The expression is as follows: P WT (t)=P bat (t)+P H (t)+P dump (t), Among them, P bat The charging and discharging power of the energy storage system, with a positive value for charging, P H P is the power of the electrolytic load. dump This refers to the power output of the wind curtailment. The energy storage and hydrogen production state sub-model calculates the state of charge (SOC) of the energy storage system, expressed as follows: Where, η ch ,η dis These represent the charge / discharge efficiency, Δt is the control period, and E... bat It refers to energy storage capacity.

8. A system for the coordinated control method of the offshore wind power-energy storage-hydrogen production system according to any one of claims 1 to 7, characterized in that, The system includes a data acquisition module, a dual evaluator module, a strategy decision-making module, and an execution control module. The data acquisition module is used to collect wind farm operation data, energy storage system data and hydrogen production system data in real time, and perform preprocessing to obtain a state vector including the inflow wind speed of the wind turbine, the state of charge of the energy storage system, the electrolysis load power and the energy storage power. The policy decision module includes a policy network based on a Markov decision process according to the state vector, used to calculate the current action vector. The policy network is trained through a dual evaluator reinforcement learning framework in the dual evaluator module. The dual evaluator module includes a first evaluator and a second evaluator. The first evaluator evaluates the economic objective value function, and the second evaluator evaluates the safety objective value function. The execution control module is used to send the control commands corresponding to the current action vector, including the axial induction factor, yaw angle, electrolytic load power, and energy storage power of the wind turbine, to each execution device for adjustment, including: adjusting the wind turbine operating parameters according to the axial induction factor and yaw angle; and adjusting the input power of the electrolytic cell according to the electrolytic load power. The charging and discharging operation of the energy storage system is controlled according to the energy storage power.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

Images