A method for intelligent scheduling of an offshore energy island system

Abstract

This invention relates to the field of intelligent energy system scheduling technology, and discloses a method for intelligent scheduling of an offshore energy island system. The method includes: constructing a simulated offshore energy island with embedded marine physical constraints, and establishing an objective function aimed at minimizing total cost; jointly modeling multi-source uncertainties based on a diffuse neural network to generate a multi-dimensional dynamic scenario library; training the scheduling system based on a transfer reinforcement learning framework, pre-training the SAC agent using historical operating data to obtain initial schemes and network parameters; using the multi-dimensional dynamic scenario library as a training environment to fine-tune the pre-trained agent; and outputting the optimized scheduling network through a closed-loop iterative mechanism and deploying it to the offshore energy island control system for real-time scheduling. This method solves the problems of difficulty in representing multi-dimensional uncertainties, disconnect between scenarios and policies, low training efficiency, and imbalance in multi-objective collaboration, achieving the goals of improving realism and coverage, and enhancing training efficiency and collaborative optimization capabilities.

Description

Technical Field

[0001] This invention relates to the field of intelligent energy system scheduling technology, and in particular to a method for intelligent scheduling of an offshore energy island system. Background Technology

[0002] Currently, energy system dispatch optimization technology has formed two main approaches: one is scenario-driven stochastic optimization, which generates representative operating scenario sets through historical data clustering or probability sampling, transforming uncertain problems into multi-scenario deterministic optimization problems, and solving dispatch schemes that balance economy and robustness. This approach is widely used in day-ahead planning of power systems and integrated energy system planning. The other is data-driven intelligent decision-making, which integrates short-term prediction models, online optimization algorithms, and real-time feedback mechanisms. This approach models the dispatch process as an interaction between an intelligent agent and the environment, uses deep neural networks to learn state-action mapping strategies, and continuously optimizes long-term cumulative benefits through trial and error mechanisms, demonstrating good adaptability in dynamic load and renewable energy fluctuation scenarios.

[0003] For example, Chinese patent CN120258474A discloses an intelligent scheduling method for a multi-element energy storage system based on deep reinforcement learning. It provides the following technical solutions: establishing a multi-element energy storage system model based on preset multi-element energy storage system information; constructing a dynamic scheduling model based on the requirement that the supply load during peak load demand periods can maximize the economic benefits of the multi-element energy storage system model; transforming the dynamic scheduling of the dynamic scheduling model into a Markov decision process; constructing a deep reinforcement learning network structure suitable for setting the state characteristics of the dynamic scheduling model; and obtaining the maximum reward function through interaction between the deep reinforcement learning network structure and the dynamic scheduling model, thereby achieving robust optimization of the multi-element energy storage system. It focuses on the collaborative optimization problem of electric, thermal, and hydrogen multi-element energy storage systems, modeling the scheduling process as a Markov decision process. The state space includes the state of charge of each energy storage unit, load demand, and renewable energy output; the action space is defined as the charging and discharging power commands of each energy storage device; and the reward function comprehensively considers operational economy and system stability. This method designs a deep reinforcement learning network with a policy and evaluation architecture. During training, an experience replay mechanism is introduced to break the temporal correlation of data, and a dual-objective network structure is adopted to improve training stability. This allows the agent to autonomously learn the collaborative control strategy of energy storage devices through continuous interaction with a high-fidelity simulation environment. However, the aforementioned intelligent scheduling method for multi-element energy storage systems based on deep reinforcement learning cannot handle the multi-dimensional marine environmental uncertainties unique to offshore energy islands (such as wind, wave, and current coupling), lacks the ability to generate high-fidelity extreme event scenarios, and does not employ a transfer learning framework. This results in low policy training efficiency and insufficient adaptability and robustness under complex sea conditions. Summary of the Invention

[0004] This invention addresses the problems of difficulty in representing multi-dimensional uncertainties, disconnect between scenarios and strategies, low training efficiency, and imbalance in multi-objective collaboration in existing technologies. It proposes a method for intelligent scheduling of marine energy island systems, achieving the goals of improving the realism and coverage of scenario generation, as well as improving training efficiency and collaborative optimization capabilities.

[0005] Furthermore, the current optimization of scheduling for offshore energy islands faces challenges in systematically modeling and high-fidelity representation of multi-dimensional uncertainties (including dynamic fluctuations in the marine environment, randomness in equipment operation status, changes in grid interaction boundaries, timing interference from maintenance operations, and extreme events such as typhoons / giant waves). This is particularly pronounced in scenarios requiring frequent repetitive calculations, such as year-round rolling optimization, extreme scenario stress testing, or parameter sensitivity analysis, where the risk of strategy failure is high and computational efficiency is low, making it difficult to simultaneously achieve the multi-objective synergy of safe and stable operation, economic efficiency, and low-carbon operation. This invention aims to propose an intelligent scheduling strategy optimization method for offshore energy islands that considers the predictive uncertainties of multiple scenarios. It generates a high-fidelity multi-dimensional scenario library by constructing a diffuse neural network and integrates a refined offshore energy island simulation with a deep reinforcement learning strategy training mechanism based on transfer learning. This enables adaptive optimization and rapid iteration of scheduling strategies under numerous dynamic scenarios. This method can significantly improve the environmental adaptability and operational robustness of scheduling strategies while effectively shortening strategy training time, meeting the efficient, intelligent, and reliable decision-making needs of offshore energy islands in real-time scheduling, emergency response to extreme events, and long-term planning decisions.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A method for intelligent scheduling of an offshore energy island system includes: Construct a marine energy island simulation model with embedded marine physical constraints, and establish an objective function aimed at minimizing the total cost; A multi-dimensional dynamic scene library is generated by jointly modeling multi-source uncertainties based on diffusion neural networks. The scheduling training is based on the transfer reinforcement learning framework. The SAC agent is pre-trained using historical operation data to obtain the initial scheme and network parameters. A multi-dimensional dynamic scenario library is used as the training environment to fine-tune the pre-trained agent. The optimized scheduling network is output through a closed-loop iterative mechanism and deployed to the marine energy island control system for real-time scheduling.

[0007] By constructing a simulation model with embedded marine physical constraints, generating a high-fidelity scene library using a diffusion neural network, and employing a transfer reinforcement learning training framework, the problems of multi-dimensional uncertainty in the scheduling of marine energy islands, such as difficulty in systematically modeling, high risk of strategy failure, and low computational efficiency, have been solved, achieving multi-objective synergy of safe and stable, economical and efficient, and low-carbon operation.

[0008] Preferably, the construction of the marine energy island simulation body embedded with marine physical constraints specifically includes: determining the equipment composition of the marine energy island, establishing models of each piece of equipment, and constructing equality constraints covering power balance and inequality constraints covering the operating limits of each piece of equipment, thereby forming a simulation operating environment that conforms to marine engineering specifications.

[0009] By accurately modeling the equipment composition, power balance constraints, and operational limit constraints, the simulation can realistically reflect the physical characteristics and operational boundaries of the offshore energy island, providing an accurate and reliable simulation basis for subsequent scheduling strategy optimization and preventing the strategy from deviating from actual physical limitations.

[0010] Preferably, the joint modeling of multi-source uncertainty based on diffusion neural network specifically includes: collecting historical marine environmental data and offshore energy island operation data as training data; constructing a diffusion neural network with a three-layer architecture of encoder, bottleneck layer and decoder, and training the diffusion neural network.

[0011] By training a three-layer diffusion neural network using historical data, it is possible to learn the joint probability distribution of multi-source uncertain data in the marine environment, laying the model foundation for generating high-fidelity scenarios covering both routine operations and extreme events.

[0012] Preferably, the generation of the multi-dimensional dynamic scene library specifically includes: using a trained diffusion neural network to generate scenes; combining given conditional information, using the diffusion neural network to perform multi-step iterative denoising on a random noise vector with a standard Gaussian distribution to generate new scene samples that conform to the joint probability distribution of historical data; evaluating the quality of the generated scene samples, with evaluation indicators including coverage and scene envelope area; the coverage evaluates the degree to which the generated scene covers the actual data range, and the scene envelope area evaluates the fluctuation range of the generated scene; setting a threshold requirement for coverage, and selecting high-quality scenes that meet the fidelity requirements to constitute the multi-dimensional dynamic scene library.

[0013] New scenes that conform to historical distributions are generated through diffusion denoising, and quantitative evaluation and screening are performed using coverage and envelope area to ensure that the generated scenes have high fidelity and complete coverage of extreme events, providing reliable environmental input for policy training.

[0014] Preferably, the pre-training of the SAC agent using historical operating data specifically includes: the agent comprising a policy network and an evaluation network; the policy network takes a state vector containing time steps, new energy output, load, and energy storage status as input, and outputs normalized scheduling actions through a network structure containing multiple fully connected layers; the evaluation network adopts a dual-Q network structure, takes a concatenated vector of state and action as input, and outputs a Q-value estimate; the agent's action space, state space, and reward function are defined; the action space contains scheduling power commands for each controllable device, the state space is the real-time operating state of the system, and the reward function is set as the negative of the objective function value calculated by the simulation agent; samples quadruples consisting of state, action, reward, and next state are sampled from historical operating data, stored in an experience replay buffer, and the network parameters are updated using stochastic gradient descent based on the maximum entropy objective function; after multiple rounds of iterative training, a preliminary scheduling scheme and network parameters are obtained as a pre-trained model.

[0015] By pre-training the SAC agent using historical operational data and obtaining the initial policy network parameters through the maximum entropy objective function, a foundation is provided for subsequent fine-tuning in multiple scenarios, reducing the time and sample size required for training.

[0016] Preferably, the fine-tuning of the pre-trained agent specifically includes: loading the parameters of the pre-trained policy network and evaluation network as initial parameters for fine-tuning training; inputting the multi-dimensional dynamic scene generated by the diffusion neural network into the marine energy island simulation to construct a fine-tuning training environment; the agent interacting with the simulation, outputting scheduling actions according to the current state, obtaining rewards after execution, and transitioning to the next state, storing this interaction experience in a new experience replay buffer; and using a reinforcement learning objective function based on maximum entropy to sample data from the new experience replay buffer and update the parameters of the policy network and evaluation network respectively.

[0017] By using the fine-tuning mechanism in transfer learning, pre-trained knowledge can be quickly adapted to high-fidelity multi-scenario environments, enabling agents to learn robust strategies to cope with various uncertainties, especially extreme events, while avoiding the computational overhead of retraining.

[0018] Preferably, the optimized scheduling network output through a closed-loop iterative mechanism specifically includes: A closed-loop iterative optimization mechanism is constructed. The agent sequentially traverses multiple scenarios generated by the diffusion neural network. Within each consecutive time step of a scenario, the agent outputs scheduling actions based on the current state space vector. The marine energy island simulation agent calculates the reward signal in real time based on the multi-objective cost function and stores the state space, action space, and reward signal in the experience replay pool. After completing a predefined number of time step interactions, experience data is sampled in batches from the experience replay pool, and the policy network and evaluation network parameters are updated synchronously through the gradient descent algorithm. When the policy optimization in a scenario reaches the convergence condition or the preset number of interaction rounds, it automatically switches to the next scenario for policy fine-tuning until all generated scenarios are traversed, and finally outputs a lightweight deep policy network.

[0019] Establish a closed-loop iterative mechanism for scenario generation, strategy fine-tuning, effect evaluation, and scenario updates. This mechanism supports the continuous evolution and dynamic adjustment of strategies across multiple scenarios, overcomes the defects of strategy rigidity, and ensures the continuous adaptability and robustness of scheduling strategies under the long-term evolution of the marine environment and sudden extreme events. At the same time, it outputs a lightweight network for easy real-time deployment.

[0020] Preferably, the objective function is formed by weighting investment and operation and maintenance indicators, renewable energy indicators, reliability indicators, and safety indicators. The economic indicators include equipment investment items, fixed and variable operation and maintenance items, and the net value of hydrogen sales revenue. The renewable energy indicators include wind and solar curtailment penalties. The reliability indicators include power shortage penalties, equipment start-up and shutdown penalties, and energy storage protection items. The energy storage protection items are a linear combination based on battery discharge depth, charge-discharge rate, and cycle count. The safety indicators include overload operation risk items and backup equipment redundancy items. By adjusting the weighting factors before each sub-item, the emphasis on different objectives is adjusted during the scheduling process.

[0021] We construct a weighted multi-objective function that covers four dimensions: economy, low carbon, reliability, and safety. We also introduce refined penalty terms such as energy storage protection and overload risk, so that the agent can simultaneously weigh multiple objectives during the policy optimization process and flexibly adapt to the scheduling priority requirements under different operating scenarios by adjusting the weight factors.

[0022] Preferably, the equipment model specifically includes: the power generation model of offshore wind power equipment is determined based on the relationship between real-time wind speed and cut-in wind speed, rated wind speed, and cut-out wind speed; the power generation model of offshore photovoltaic equipment is determined based on standard irradiance, standard test temperature, and temperature coefficient; the battery energy storage system is determined through charge and discharge efficiency, state of charge constraints, and mutual exclusion constraints of charge and discharge states; and the hydrogen production system is determined by including minimum operating ratio constraints and power ramp-up rate constraints.

[0023] For the wind power, photovoltaic, energy storage and hydrogen production equipment unique to offshore energy islands, a refined mathematical model that conforms to the physical mechanism was established. The minimum operating ratio and power ramp-up rate constraints of the hydrogen production system were included to enable the simulation to accurately reflect the actual operating characteristics and safety boundaries of the offshore equipment.

[0024] Preferably, the encoder uses a downsampling path to perform multi-scale compression on the input multi-source feature vectors, the bottleneck layer uses a self-attention mechanism to capture long-range dependencies between features, and the decoder uses an upsampling path to make skip connections with the corresponding layer of the encoder, gradually recovering features and reconstructing scene data; training the diffusion neural network includes a forward diffusion stage and a reverse denoising stage. In the forward diffusion stage, Gaussian noise is gradually added to the training data until it is transformed into a pure noise distribution. In the reverse denoising stage, the denoising network is trained to predict the added noise from the noisy data, and the network parameters are optimized by minimizing the mean square error between the predicted noise and the real noise.

[0025] The diffusion neural network, which adopts the U-Net structure and the self-attention mechanism, preserves detailed information through multi-scale feature extraction and skip connections, and can reflect the multi-dimensional dynamic coupling characteristics of the marine environment. The training method of forward diffusion and backward denoising enables the model to generate high-fidelity, full-spectrum marine scenes, solving the problem that traditional methods are difficult to handle in low-probability, high-impact extreme events.

[0026] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention constructs a diffuse neural network, using multi-dimensional parameters of the marine environment as conditional information to generate a high-fidelity scenario library covering both routine operations and low-probability, high-impact extreme events. Compared to traditional methods based on historical data clustering or Monte Carlo simulation, this approach can more accurately characterize the coupling relationships of multi-dimensional dynamic characteristics such as wind speed and waves, providing a full-spectrum digital twin marine environment for strategy training. This allows the final generated scheduling strategy to better cope with various uncertainties in real sea conditions.

[0027] 2. This invention constructs a refined simulation entity embedding device safety boundaries and marine physical constraints, and deeply integrates multi-objective cost functions into the deep reinforcement learning training process. This enables the agent to simultaneously weigh and optimize multiple sometimes conflicting objectives when exploring the optimal strategy, effectively solving the safety hazards or multi-objective imbalance problems that may be caused by traditional single-objective optimization, thereby generating a collaborative optimization scheduling scheme that fully satisfies the system's operational constraints.

[0028] 3. This invention employs a training framework based on transfer learning. The agent is first pre-trained on historical data to obtain a basic policy, and then fine-tuned using generated high-fidelity, diverse scenarios. This overcomes the sample efficiency bottleneck in high-dimensional state spaces and accelerates training convergence. Simultaneously, the finally deployed lightweight policy network can respond in an extremely short time, meeting the stringent computational speed requirements of offshore energy islands in real-time scheduling and emergency response to extreme events. Attached Figure Description

[0029] Figure 1 This is an overall flowchart of a method for intelligent scheduling of a marine energy island system according to the present invention.

[0030] Figure 2 This is an initial load data diagram of one embodiment of a method for intelligent scheduling of an offshore energy island system according to the present invention.

[0031] Figure 3 This is a result diagram of load scenario library generation in one embodiment of the intelligent scheduling method for a marine energy island system according to the present invention.

[0032] Figure 4 This is an initial wind speed data diagram of one embodiment of a method for intelligent scheduling of an offshore energy island system according to the present invention.

[0033] Figure 5 This is a wind speed scene library generation result diagram of an embodiment of the intelligent scheduling method for a marine energy island system of the present invention.

[0034] Figure 6 This is an initial data diagram of light intensity from one embodiment of a method for intelligent scheduling of a marine energy island system according to the present invention.

[0035] Figure 7 This is a result image generated from a light intensity scene library in one embodiment of a method for intelligent scheduling of a marine energy island system according to the present invention.

[0036] Figure 8 This is a comparison of convergence curves of different algorithms in one embodiment of the intelligent scheduling method for a marine energy island system according to the present invention. Detailed Implementation

[0037] See Figures 1 to 8 As shown, a method for intelligent scheduling of a marine energy island system includes: Construct a marine energy island simulation model with embedded marine physical constraints, and establish an objective function aimed at minimizing the total cost; A multi-dimensional dynamic scene library is generated by jointly modeling multi-source uncertainties based on diffusion neural networks. The scheduling training is based on the transfer reinforcement learning framework. The SAC agent is pre-trained using historical operation data to obtain the initial scheme and network parameters. A multi-dimensional dynamic scenario library is used as the training environment to fine-tune the pre-trained agent. The optimized scheduling network is output through a closed-loop iterative mechanism and deployed to the marine energy island control system for real-time scheduling.

[0038] like Figure 1 In one embodiment shown, the present invention proposes an intelligent scheduling strategy optimization method for offshore energy islands. The overall process includes the construction of an offshore energy island simulation, the generation of multi-source uncertainty scenarios, and the training and deployment of strategies based on transfer reinforcement learning.

[0039] The specific steps are as follows: First, a diffusion neural network is used to jointly model the multi-source uncertainties in the operation of the offshore energy island. Historical marine environmental data and energy island operation data are collected to construct a diffusion neural network with a three-layer architecture of encoder-bottleneck layer-decoder. Network training includes two stages: forward diffusion and backward denoising. In the forward stage, Gaussian noise is gradually added to the original data until it becomes pure noise. In the backward stage, the denoising network is trained to predict the added noise, and the parameters are optimized by minimizing the mean square error between the predicted noise and the actual noise. After training, starting with random noise of standard Gaussian distribution, combined with given conditional information, new scene samples that conform to the joint distribution of historical data are generated through multi-step iterative denoising. The scene quality is evaluated by coverage and scene envelope area, and a high-fidelity multi-dimensional dynamic scene library is selected.

[0040] Secondly, a simulation of an offshore energy island embedded with marine physical constraints is constructed. The equipment composition is defined as offshore wind power, offshore photovoltaic, battery energy storage, hydrogen energy storage, and energy conversion equipment. Power models and operational constraints for each device are established. Power balance equation constraints and limit inequality constraints for each device are constructed to form a simulation operating environment that conforms to marine engineering specifications. An objective function is set to minimize total cost. This function is composed of weighted economic indicators (investment and operation and maintenance costs, hydrogen sales revenue), renewable energy indicators (wind and solar curtailment penalties), reliability indicators (power shortage penalties, equipment start-up and shutdown penalties, energy storage lifespan protection), and safety indicators (overload operation risk, backup redundancy costs). The weighting factors of each indicator can be dynamically adjusted according to actual operational needs.

[0041] Then, a transfer reinforcement learning framework is used to train the scheduling policy. Training is based on the Soft Actor-Critic (SAC) algorithm, with the agent comprising a policy network and a dual evaluation network. First, the SAC agent is pre-trained using historical operational data: a state space (time step, wind / PV output, load, energy storage state), an action space (scheduling power commands for each controllable device), and a reward function (negative total cost) are defined. Quadruple samples are sampled from historical data and stored in an experience replay buffer. Network parameters are updated using gradient descent based on the maximum entropy objective function, iteratively obtaining the initial policy and network parameters. Fine-tuning is then performed: pre-trained parameters are loaded as initial values, and a multi-dimensional dynamic scenario library generated by a diffusion neural network is used as a new training environment, input into the marine energy island simulation. The agent interacts with the simulation, outputting scheduling actions based on the current state, receiving a reward after execution, and transitioning to a new state. The interaction experience is stored in a new experience replay buffer. The parameters are updated using a fine-tuning objective function, iteratively enhancing the policy's robustness in uncertain scenarios.

[0042] A closed-loop iterative optimization mechanism is constructed during fine-tuning: the agent sequentially traverses multiple scenarios in the scenario library. Within each scenario's consecutive time steps, it outputs scheduling actions based on the current state vector. The agent calculates the reward signal in real time according to the multi-objective cost function and stores the state, action, and reward in the experience replay pool. After completing a certain number of time steps of interaction, experience data is sampled in batches, and network parameters are updated synchronously through gradient descent. When the policy optimization in a scenario reaches convergence or a preset number of interaction rounds are reached, the system automatically switches to the next scenario to continue fine-tuning until all scenarios have been traversed. Finally, a lightweight deep policy network is output.

[0043] Finally, the trained strategy network is deployed to the marine energy island control system. It receives real-time marine environmental data and equipment operating status, encodes them into state vectors, and generates optimal scheduling instructions that balance economy, safety, and low carbon emissions within milliseconds. After execution, the network evaluates the results and continuously updates the strategy as needed, achieving real-time intelligent scheduling.

[0044] In another embodiment, the specific process of the present invention is as follows: Step 1: Construction and Cost Model of the Marine Energy Island Simulation This invention constructs a simulation of a marine energy island, sets a cost model and constraints, and constructs an objective function. This step is fundamental to scheduling strategy optimization, providing a reliable basis for subsequent scheduling optimization by accurately modeling the physical characteristics and operating mechanisms of the marine energy island.

[0045] ① Equipment composition and model: A simulated offshore energy island was constructed, including offshore wind power equipment, offshore photovoltaic equipment, energy storage equipment (battery energy storage system, hydrogen energy storage system), and energy conversion equipment (power electronic converter, water electrolysis hydrogen production equipment). The models of each piece of equipment are as follows: Offshore wind power generation model: When v < cut-in wind speed or v ≥ cut-out wind speed, the current wind turbine power generation is equal to 0; When the cut-in wind speed ≤ v < rated wind speed, the current wind turbine power generation is equal to the installed capacity of the wind power system multiplied by the power supply efficiency of the wind power system, and then multiplied by (v). 3 Subtract the cube of the cut-in wind speed and then divide by (v) rated 3 Subtract v cut-in 3 ); When the rated wind speed ≤ v < cut-out wind speed, the current wind turbine power generation is equal to the installed capacity of the wind power generation system multiplied by the power supply efficiency of the wind power generation system.

[0046] Among them, v t For wind speed, v cut-in To cut off the wind speed, v rated This is the rated wind speed.

[0047] Power generation model of offshore photovoltaic equipment: The current photovoltaic power generation capacity is equal to the installed capacity of the photovoltaic power generation system multiplied by the power supply efficiency of the photovoltaic power generation system, and then multiplied by (1 minus α). cof (T c,t Subtract T STC Then multiply by the current irradiance and divide by the standard irradiance; The standard irradiance unit is 1000 W / m². 2 T c,t T represents the temperature of the photovoltaic panel. STC For the standard test temperature (25°C), α cof This is the temperature coefficient.

[0048] Battery energy storage system charge and discharge model: E BESS t Equal to E BESS t-1 Add the first coefficient and subtract the second coefficient; the first coefficient is the charging efficiency multiplied by P. BESS,ch t Multiply by Δt; the second coefficient is P. BESS,dis t Multiply by Δt and then divide by the discharge efficiency; 0≤E BESS t ≤ Rated capacity of energy storage system; 0 ≤ P BESS,cht ≤ Maximum charge / discharge power of the energy storage system multiplied by the state-of-charge variable; 0≤P BESS,dis t ≤ Maximum charging and discharging power of the energy storage system multiplied by the discharge state variable; Charging state variable plus discharge state variable ≤ 1; Among them, E BESS t P represents the current remaining battery power. BESS,dis t P is the discharge power of the battery energy storage. BESS,ch t The power wasted for battery energy storage.

[0049] Hydrogen production system model: Hydrogen production is equal to the efficiency of the hydrogen production system multiplied by the input electrical power of the hydrogen production system, and then divided by the calorific value of hydrogen. E H2 t Equal to E H2 t-1 Add the third coefficient and subtract the fourth coefficient; the third coefficient is the production efficiency multiplied by the hydrogen production amount and then multiplied by Δt; the fourth coefficient is the hydrogen emission amount multiplied by Δt and then divided by the emission efficiency. 0≤E H2 t ≤ Hydrogen storage tank capacity; Maximum hydrogen production efficiency multiplied by minimum operating ratio of hydrogen production system ≤ Input electrical power of hydrogen production system diagram ≤ Maximum hydrogen production efficiency; 0 ≤ Hydrogen production rate ≤ Maximum hydrogen discharge power; |P H2 t Subtract P H2 t-1 |≤ Hydrogen production system power ramp-up rate multiplied by maximum hydrogen production power; Among them, P H2 t Input electrical power, E, to the hydrogen production system diagram H2 t This represents the current hydrogen reserves.

[0050] If the hydrogen sales cycle is taken into account, Sell T If a separate storage tank is to be built for the hydrogen sales portion, the required capacity of the additional hydrogen storage tank is: Hydrogen sales cycle Sell T The total amount of hydrogen storage required domestically equals , i∈N, i∈[0,T-Sell T Additional hydrogen storage capacity equals the hydrogen sales cycle. T The maximum amount of hydrogen required for internal storage. Where T represents the duration of the simulation, and Sell(n) is the daily hydrogen storage amount.

[0051] Energy balance constraints: Current wind turbine power generation plus current photovoltaic power generation plus P BESS,dis t Adding the power supplied by the grid equals P BESS,ch t The figure includes the input electrical power of the hydrogen production system, the base load, and the discarded electricity.

[0052] ② Cost Model and Objective Function Fixed investment cost C I equals ∑ i r(1 plus r) Ni Multiply by the unit investment cost of equipment i, then multiply by Cap. i Divide by (1 plus r) Ni Subtract 1; where r is the annual depreciation rate, Ni is the operating life of equipment i, and Cap i Let i be the installed capacity of device i.

[0053] Fixed maintenance cost C OM equals ∑ i M i Multiply by Cap i Among them, M i The unit fixed maintenance cost for device i. The variable maintenance cost C. vari equals ∑ i ∑ t T P i t Multiply by the unit variable maintenance cost of device i. The revenue from hydrogen sales is ∑ t T Hydrogen emissions multiplied by the unit price of hydrogen. Penalty for curtailment of wind and solar power (C) cur Equals the power curtailment penalty coefficient multiplied by ∑ t T (Gross energy wasted).

[0054] Power shortage penalty C short equals ∑ t T Power shortage penalty factor multiplied by (max(0, (current load satisfaction rate minus target load satisfaction rate))) 2 Equipment start / stop penalty C s equals ∑ i [(The start / stop penalty coefficient of device i multiplied by the number of start / stop cycles of device i) plus (∑ t T (The square of the power change rate multiplied by the variable power penalty factor). Energy storage protection cost C Blife equals ∑ t T[(First penalty coefficient multiplied by battery discharge depth) plus (second penalty coefficient multiplied by battery charge / discharge rate) plus (third penalty coefficient multiplied by battery cycle count)]

[0055] Overload operation risk cost C overload Equals the overload penalty coefficient multiplied by ∑ t T exp[(Operating power of electrical equipment divided by rated power) minus (Safety power divided by rated power)]. Redundancy cost of standby equipment C. res equals ∑ t T [Redundancy cost factor multiplied by (max(0, (current remaining power minus safe redundancy power)))] 2 】

[0056] objective function minC Total Equals ω1 multiplied by (C) I Add C OM Add C vari Subtract R), then add ω2 multiplied by C cur Plus ω3 multiplied by (C short Add C s Add C Blife ), plus ω4 multiplied by (C overload Add C res ); where ω1, ω2, ω3, and ω4 are weighting factors, corresponding to economic objectives, low-carbon objectives, reliability objectives, and security objectives, respectively, and can be used to adjust the priority of scheduling objectives during actual system operation.

[0057] Step 2: Multi-scenario uncertainty modeling and generation This invention provides a unified model for the multi-source uncertainties present during the operation of offshore energy islands. These multi-source uncertainties include, but are not limited to, marine environmental uncertainties, power equipment operational uncertainties, grid interaction uncertainties, and uncertainties related to operation and maintenance and extreme events. Specifically, marine environmental uncertainties include at least wind fields, waves, ocean currents, tides, and meteorological and hydrological conditions; power equipment operational uncertainties include normal operation, performance degradation, abnormal operation, and fault states; grid interaction uncertainties include grid-connected steady-state operation and grid disturbance states; and extreme event uncertainties include low-probability, high-impact conditions such as typhoons and giant waves.

[0058] Multi-scenario uncertainties can be learned and generated using a diffusion neural network (DNN). First, historical marine environmental data (wind speed, waves, ocean currents, weather, etc.) and offshore energy island operation data (power generation, equipment status, grid interaction, etc.) are collected. By training on historical observation data and simulation data, the diffusion model learns the joint probability distribution of multi-source uncertainty scenarios.

[0059] The diffuse neural network employs a three-layer architecture of encoder-bottleneck layer-decoder, with the U-Net structure at its core. The encoder layer (downsampling path) is responsible for multi-scale feature extraction and compression. It receives marine environmental feature vectors including wind speed, wind direction, wave height, wave period, ocean current speed, ocean current direction, temperature, air pressure, and humidity, as well as energy island system operation feature vectors such as wind power output, photovoltaic power output, electrical load, energy storage status, and grid interaction power. It progressively compresses the feature dimensions through multiple convolutional layers. The bottleneck layer, located at the deepest part of the network, uses a self-attention mechanism to capture long-range dependencies, enhancing the model's ability to perceive global features. The decoder layer (upsampling path) is responsible for feature recovery and scene reconstruction, mapping the denoised latent vectors back to the original data space. Its structure skips connections with the corresponding layer of the encoder, preserving multi-scale feature information to improve denoising accuracy. Finally, it outputs a predicted noise vector for the reverse denoising process.

[0060] The training process of a diffusion neural network consists of two stages: forward diffusion and backward denoising. In the forward diffusion stage, Gaussian noise is progressively added to the training data, gradually transforming the original data into a pure noise distribution over multiple time steps. In the backward denoising stage, the denoising network is trained to predict the noise added at each step, gradually removing noise from the pure noise to reconstruct the original data distribution. At each time step, the denoising network receives noisy data and the current time-step encoding as input, and outputs the mean squared error between the predicted noise and the actual noise as the loss function for backpropagation optimization. After training, the generation stage samples initial noise from a standard Gaussian distribution, and through the iterative denoising process of the denoising network, generates new scene samples that conform to the characteristics of the historical data distribution.

[0061] The specific parameters of the model are shown in Table 1. Table 1:

[0062] Table 1 shows the parameter table for the diffusion neural network model.

[0063] The specific construction process of the model is as follows: ① Forward diffusion process: Noise is gradually added to the original data x0 until it becomes pure noise. The diffusion process is defined as a Markov chain, and Gaussian noise can be used for each step of the noise addition. :x t Equals √(α)t x0 plus √(1-α) t ); where α t equals ∏ i=1 t α i α i =1 minus β i ,β i Let the noise variance be a preset value, t∈[1,T]. After T time steps, the pure noise state x is obtained. T ~N(0,I).

[0064] ② Reverse noise reduction process: From pure noise x T Starting from ~N(0,I), after T time steps of iterative denoising, a new, synthesized scene is finally generated. This process starts from pure noise x T Initially, iteratively perform single-step denoising pθ(x) t-1 |x t c), this process can be represented as: pθ(x t-1 |x t c) equals N(x) t-1 ;μθ(x t ,t,c),β t I), where μθ(x) t ,t,c) and β t I is the mean and covariance parameterized by the neural network, and θ is the trainable parameters of the neural network. In practice, it is usually expressed in the following form: x t-1 Equal to μθ(x) t ,t,c) plus σ t z; where σ t 2 equal to β t (1 minus α) t-1 ) divided by (1-α) t z~N(0,I) is a noise vector randomly drawn from the standard normal distribution, used to introduce randomness to generate diverse scenes.

[0065] μθ(x) t ,t,c) is usually represented as (x t Subtract (β) t Multiply Divide by √(1-α) t ))) Divide by √(α) t );in, is the noise predicted by the neural network, and c is the conditional information (such as marine environment type, equipment operating status, etc.).

[0066] ③ Model training process: The model can be trained using real noise. Noise predicted by the DNN model The mean squared error between them is used as the loss function: L simple (θ) equals minus The training process can be summarized in the following steps: (1) Data loading: Load x0 from historical data (such as monitoring data of wind, solar, and waves over the years, as well as operation data of load, power grid, etc.); (2) Forward diffusion: For each x0, generate a noise dataset. (t equals 1 to T); (3) Reverse process: Input (x) t ,t,c) to a DNN model, output Calculate and predict noise; (4) Loss calculation: Calculate L simple (θ), assess the prediction error; (5) Backpropagation: Gradient-based optimization algorithms, such as Adam or SGD, can be used for training to update parameters θ to minimize L. simple (θ); (6) Scene generation: Generate multi-dimensional scenes based on the trained model. Assess scene quality using coverage (Index1) and scene envelope area (Index2): Index1 equals (∑ n=1 N B n Divide by N, where B is the value of the actual data when the actual data ∈ [min(generated data), max(generated data)]. n It equals 1, otherwise it is 0.

[0067] Index2 equals (∑ n=1 N (max(generated data) minus min(generated data)) divided by N; where N is the number of data points. The scene library typically needs to satisfy Index1 ≥ 95%.

[0068] Step 3: Training deep reinforcement learning strategies based on transfer learning This invention employs a deep reinforcement learning method based on transfer learning to achieve efficient optimization of the scheduling strategy for offshore energy islands through pre-training and fine-tuning. This step is the core of intelligent scheduling, learning the optimal scheduling strategy through deep reinforcement learning algorithms, enabling offshore energy islands to achieve optimal economic, safe, and low-carbon operation under various uncertain scenarios.

[0069] SAC (Soft Actor-Critic) is a reinforcement learning model employing an Actor-Critic architecture, comprising two core modules: a policy network (Actor) and an evaluation network (Critic). The policy network generates the scheduling policy. Its input layer receives state vectors containing time steps, wind power output, photovoltaic power output, electrical load, and energy storage status. These vectors are processed through two fully connected hidden layers, each with 256 neurons and using the ReLU activation function for nonlinear transformation. The output layer, through the Tanh activation function, generates the mean and standard deviation of the actions, ensuring that the action output is normalized to the [-1, 1] interval. The evaluation network uses a dual-Q network design to suppress Q-value overestimation. Its input is a concatenation of the state vector and the action vector, also processed through two fully connected hidden layers with 256 neurons. The output layer is a single neuron used to estimate the Q-value.

[0070] The training of the SAC agent is divided into two stages: pre-training and fine-tuning. During training, an experience replay buffer is constructed. In the pre-training stage, state-action-reward-next state transition samples are sampled from historical running data, and these historical running data samples are represented as quadruples (s...). t ,a t ,r t ,s t+1 The data is stored in an experience replay buffer with a capacity of 20,000. In each training round, 128 samples are randomly selected for mini-batch gradient updates, iterating for 2000 rounds to allow the agent to learn the basic scheduling strategy. During the fine-tuning phase, pre-trained network parameters are loaded as initialization, and multi-scenario data generated by the diffuse neural network is used to continue training for 8000 rounds, enhancing the robustness of the strategy in uncertain scenarios.

[0071] The specific model parameters are shown in Table 2. Table 2:

[0072] Table 2 shows the parameters of the SAC model.

[0073] In the scenario of an offshore energy island system, the state space of the SAC agent needs to encompass time steps, wind power output, photovoltaic output, electrical load, and energy storage status. This data is derived from 8760 hours of historical operating records or a real-time monitoring system, providing a complete representation of the system's operating status. The strategy space needs to include wind power dispatch power, photovoltaic dispatch power, energy storage charging and discharging power, grid interaction power, and electrolyzer operating power. The output range of each action variable is set based on the rated power and operating constraints of the corresponding equipment. The reward function comprehensively considers four objectives: economy, low carbon footprint, reliability, and safety, and adopts a negative form of total operating cost plus constraint violation penalty. A complete Markov decision process is formed between state, action, and reward. The agent outputs the optimal power settings for each device based on current wind and solar power output, load demand, energy storage status, and electricity price signals. After the dispatch decision is executed, the system calculates the total operating cost and the degree of constraint violation and returns a negative reward signal to guide strategy optimization.

[0074] The specific construction process of the model is as follows: ① Agent pre-training: First, define the state space, action space, and reward function of the SAC agent.

[0075] Action space A is [P PV t ,P WT t ,P BESS,ch t ,P BESS,dis t ,P H2,ch t ,P H2,dis t ,P GRID t ]; The state space S is [t, P PV t ,P WT t ,P LOAD t E BESS t E H2 t ]; The reward function R is R(S,A) and equals -C. Total .

[0076] Among them, P PV t P represents the current photovoltaic power generation capacity. WT t P represents the current power generation capacity of the wind turbine. LOAD t Based on the load, E BESSt E represents the current remaining battery power. H2 t This represents the current hydrogen reserves.

[0077] The SAC agent encourages exploration by maximizing entropy while maintaining policy stability. Its objective function is: J(π) equals E. τ~π ∑ t=0 ∞ γ t (r(s t ,a t ) plus αH(π(·∣s t ))】;where π is the optimization strategy, H(π(·∣s t )) is the policy entropy, α is the entropy coefficient, and r(s) t ,a t ) is the reward function, and γ is the discount factor.

[0078] Subsequently, the SAC agent was pre-trained using historical running data to obtain a good initial policy and training network parameters.

[0079] ② Agent fine-tuning: First, high-fidelity scenes generated by a DNN model are used as input for deep learning training. Each scene includes information such as marine environmental parameters, equipment status, and power grid interaction conditions. The generated scene is then encoded as a state space vector S.

[0080] Subsequently, the scene generated by the DNN model is used as the new training environment, and the pre-trained SAC agent is fine-tuned on the new scene. The following fine-tuning objective functions are used to update the policy network and evaluate the network parameters respectively: equal Subtract Q θ (s t ,a t )]; equal minus ; equals r(s) t ,a t ) plus γE st+1 ) [V(s t+1 )]; V(s t+1 )equal (s) t+1 ,a t+1 Subtract αlogπ(s) t+1 ,a t+1 ) ].

[0081] ③ Strategy optimization and deployment: During the training iterations, a policy network is used. Generate action a t (s t Based on the action execution schedule, a reward signal r(s) is obtained. t ,a t The updated information is then stored in the experience replay pool. This iterative process is repeated until the policy converges, ultimately outputting the optimized policy network. .

[0082] The trained strategy is deployed to the marine energy island control system to receive real-time marine environmental data and equipment operating status; the data is encoded into a state vector s, and the trained strategy network is then used. Generate scheduling actions; execute scheduling actions to adjust equipment operating status (such as power generation, energy storage charging and discharging, grid interaction, etc.), evaluate scheduling effects and update strategies, thereby realizing intelligent scheduling strategy optimization of the offshore energy island system.

[0083] The optimization process of the intelligent scheduling strategy for the offshore energy island system is as follows: First, a simulation model of the offshore energy island system is constructed. Based on historical marine environmental data, the SAC agent is pre-trained to obtain the initial SAC network parameters. Then, based on historical marine environment and equipment operation data, N sets of high-fidelity multi-dimensional scenarios are simulated and constructed through a diffuse neural network. Each scenario fully depicts dynamic data such as wind speed, waves, and equipment status within T time steps. Next, the scenarios generated by the DNN model are used as new training environments. The pre-trained SAC agent is fine-tuned in the refined simulation environment of the offshore energy island. The agent sequentially traverses N scenarios, and in each scenario... The system executes interactive decisions over T time steps within a scenario: based on the current state (ocean parameters, SOC, load, etc.), it outputs scheduling actions (wind turbine power, energy storage charging and discharging, hydrogen production power, etc.). The system calculates reward signals in real time based on a multi-objective cost model and stores the state space, action space, and reward signals in an experience playback pool. Through batch sampling and gradient descent, the system continuously optimizes the strategy network to achieve a closed-loop iteration of "scenario generation → strategy fine-tuning → effect evaluation". Finally, it outputs a lightweight deep strategy network that can be deployed to the offshore energy island control system. This network receives the operating status in real time and generates optimal scheduling instructions that balance economy, safety, and low carbon emissions within milliseconds.

[0084] In another embodiment, to empirically verify the technical effectiveness of the present invention, a typical offshore energy island region along the eastern coast of my country was selected for applied research. Based on high-resolution historical data of this region (covering resource parameters such as wind speed and solar irradiance, as well as wind and solar power output time series and regional load curves), after standardized preprocessing, the data was input into a diffusion neural network for conditional diffusion training. This generated a high-fidelity dynamic scene library covering normal operation and extreme events, which was then optimized and filtered using a scene extraction algorithm, such as... Figures 2-7 As shown in Table 3, compared with the traditional Wasserstein Generative Adversarial Network (GAN) method, this invention achieves significant improvements in scene distribution matching, extreme event coverage integrity, and multi-dimensional coupling fidelity. The scene coverage and envelope area indicators show outstanding advantages. The relevant parameters are detailed in Table 3.

[0085] Table 3:

[0086] Table 3 compares the metrics of different generation methods for different scenarios.

[0087] Furthermore, under the same high-fidelity scene library and a refined marine energy island simulation environment, the performance of three optimization algorithms—TD3 (Twin Delayed Deep Deterministic Policy Gradient), SAC (without transfer learning), and the transfer learning SAC algorithm proposed in this invention—was systematically compared. The comparison results are as follows: Figure 8 As shown in Table 4, the results demonstrate that the method of this invention, relying on a transfer learning framework of "historical strategy pre-training + multi-scenario directional fine-tuning," significantly outperforms the comparative algorithms in training convergence speed, effectively alleviating the sample efficiency bottleneck under high-dimensional marine environmental data. Regarding strategy quality, the generated scheduling scheme achieves global optimization of the total cost of the system's entire lifecycle operation. These results fully verify the technological advancement and engineering practicality of the three-in-one framework of "conditional diffusion scenario generation - marine physical constraint simulation - transfer reinforcement learning optimization" in overcoming core challenges in offshore energy island scheduling, such as the disconnect between "scenario and strategy," low efficiency in high-dimensional optimization, and difficulties in multi-objective coordination.

[0088] Table 4:

[0089] Table 4 compares the calculation results of different algorithms.

[0090] All data collection and extraction in this invention are carried out under compliant and legal conditions.

Claims

1. A method for intelligent scheduling of an offshore energy island system, characterized in that, include: Construct a marine energy island simulation model with embedded marine physical constraints, and establish an objective function aimed at minimizing the total cost; A multi-dimensional dynamic scene library is generated by jointly modeling multi-source uncertainties based on diffusion neural networks. Scheduling training is performed based on a transfer reinforcement learning framework, and the SAC agent is pre-trained using historical running data to obtain the initial scheme and network parameters. A multi-dimensional dynamic scenario library is used as a training environment to fine-tune the pre-trained agents; an optimized scheduling network is output through a closed-loop iterative mechanism and deployed to the marine energy island control system for real-time scheduling.

2. The method for intelligent scheduling of a marine energy island system according to claim 1, characterized in that, The construction of the marine energy island simulation system embedded with marine physical constraints specifically includes: determining the equipment composition of the marine energy island, establishing models of each piece of equipment, and constructing equality constraints covering power balance and inequality constraints covering the operating limits of each piece of equipment to form a simulation operating environment that conforms to marine engineering specifications.

3. The method of claim 2, wherein, The joint modeling of multi-source uncertainty based on diffusion neural networks specifically includes: collecting historical marine environmental data and offshore energy island operation data as training data; constructing a diffusion neural network with a three-layer architecture of encoder, bottleneck layer, and decoder; and training the diffusion neural network.

4. The method for intelligent scheduling of a marine energy island system according to claim 3, characterized in that, The generation of the multi-dimensional dynamic scene library specifically includes: using a trained diffusion neural network to generate scenes; combining given conditional information, using the diffusion neural network to perform multi-step iterative denoising on a random noise vector with a standard Gaussian distribution to generate new scene samples that conform to the joint probability distribution of historical data; evaluating the quality of the generated scene samples, with evaluation indicators including coverage and scene envelope area; the coverage evaluates the degree to which the generated scene covers the actual data range, and the scene envelope area evaluates the fluctuation range of the generated scene; setting a threshold requirement for coverage, and selecting high-quality scenes that meet the fidelity requirements to constitute the multi-dimensional dynamic scene library.

5. The method for intelligent scheduling of a marine energy island system according to claim 4, characterized in that, The pre-training of the SAC agent using historical operating data specifically includes: the agent comprises a policy network and an evaluation network. The policy network takes a state vector containing time steps, new energy output, load, and energy storage status as input, and outputs normalized scheduling actions through a network structure containing multiple fully connected layers; the evaluation network adopts a dual-Q network structure, takes a concatenated vector of state and action as input, and outputs a Q-value estimate; the agent's action space, state space, and reward function are defined; the action space contains scheduling power commands for each controllable device, the state space is the real-time operating state of the system, and the reward function is set as the negative of the objective function value calculated by the simulation agent; samples are taken from historical operating data consisting of a quadruple of state, action, reward, and next state, stored in an experience replay buffer, and the network parameters are updated using stochastic gradient descent based on the maximum entropy objective function. After multiple rounds of iterative training, a preliminary scheduling scheme and network parameters are obtained as a pre-trained model.

6. The method for intelligent scheduling of a marine energy island system according to claim 5, characterized in that, The fine-tuning of the pre-trained agent specifically includes: loading the parameters of the pre-trained policy network and evaluation network as the initial parameters for fine-tuning training; inputting the multi-dimensional dynamic scene generated by the diffusion neural network into the marine energy island simulation to construct a fine-tuning training environment; the agent interacting with the simulation, outputting scheduling actions according to the current state, receiving a reward after execution, and transitioning to the next state, storing this interaction experience in a new experience replay buffer; and using a reinforcement learning objective function based on maximum entropy to sample data from the new experience replay buffer and update the parameters of the policy network and evaluation network respectively.

7. The method for intelligent scheduling of a marine energy island system according to claim 6, characterized in that, The optimized scheduling network output through the closed-loop iterative mechanism specifically includes: A closed-loop iterative optimization mechanism is constructed. The agent sequentially traverses multiple scenarios generated by the diffusion neural network. Within each consecutive time step of a scenario, the agent outputs scheduling actions based on the current state space vector. The marine energy island simulation agent calculates the reward signal in real time based on the multi-objective cost function and stores the state space, action space, and reward signal in the experience replay pool. After completing a predefined number of time step interactions, experience data is sampled in batches from the experience replay pool, and the policy network and evaluation network parameters are updated synchronously through the gradient descent algorithm. When the policy optimization in a scenario reaches the convergence condition or the preset number of interaction rounds, it automatically switches to the next scenario for policy fine-tuning until all generated scenarios are traversed, and finally outputs a lightweight deep policy network.

8. A method for intelligent scheduling of a marine energy island system according to claim 1, 5, or 6, characterized in that, The objective function is formed by weighting investment and operation and maintenance indicators, renewable energy indicators, reliability indicators, and safety indicators. The economic indicators include equipment investment items, fixed and variable operation and maintenance items, and the net value of hydrogen sales revenue. The renewable energy indicators include wind and solar curtailment penalties. The reliability indicators include power shortage penalties, equipment start-up and shutdown penalties, and energy storage protection items. The energy storage protection items are a linear combination based on battery discharge depth, charge-discharge rate, and cycle count. The safety indicators include overload operation risk items and backup equipment redundancy items. By adjusting the weighting factors before each sub-item, the emphasis of different objectives is adjusted during the scheduling process.

9. A method for intelligent scheduling of a marine energy island system according to claim 2, characterized in that, The equipment models specifically include: the power generation model of offshore wind power equipment is determined based on the relationship between real-time wind speed and cut-in wind speed, rated wind speed, and cut-out wind speed; the power generation model of offshore photovoltaic equipment is determined based on standard irradiance, standard test temperature, and temperature coefficient; the battery energy storage system is determined through charge and discharge efficiency, state of charge constraints, and mutual exclusion constraints of charge and discharge states; and the hydrogen production system is determined using constraints including minimum operating ratio constraints and power ramp-up rate constraints.

10. The method of claim 3, wherein, The encoder uses a downsampling path to perform multi-scale compression on the input multi-source feature vectors. The bottleneck layer uses a self-attention mechanism to capture long-range dependencies between features. The decoder uses an upsampling path and makes skip connections with the corresponding layer of the encoder to gradually recover features and reconstruct scene data. The training of the diffusion neural network includes a forward diffusion stage and a backward denoising stage. In the forward diffusion stage, Gaussian noise is gradually added to the training data until it is transformed into a pure noise distribution. In the backward denoising stage, the denoising network is trained to predict the added noise from the noisy data, and the network parameters are optimized by minimizing the mean square error between the predicted noise and the real noise.

Images