Method for offshore wind power configuration water and energy storage system to participate in spot electricity market transaction strategy

Abstract

The invention relates to the technical field of power supply and transmission systems, in particular to a method for an offshore wind power configuration water storage and energy storage system to participate in a spot electricity market transaction strategy, comprising the following steps: S1, collecting operation information of an offshore wind power station and operation information of a water storage and energy storage power station; the operation information of the offshore wind power station comprises the total power of the offshore wind power station and the planned workload of a maintenance team, and the operation information of the water storage and energy storage power station comprises the reservoir volume of the water storage and energy storage power station, the power and efficiency of a pump, a motor and a turbine, the power and efficiency of a generator and the loss of a corresponding transmission pipeline; s2, constructing an operation environment of the offshore wind power configuration water storage and energy storage system according to short-term power dispatching requirements of a power grid department, and forming a virtual environment; the short-term power dispatching requirements of the power grid department comprise power generation power, allowable system maximum deviation, allowable system maximum power and allowable system minimum power of an offshore wind power configuration water storage and energy storage system expected by the power grid department.

Description

Technical Field

[0001] The present invention relates to the technical field of power supply and transmission systems, and in particular to a method for an offshore wind power configuration water storage energy storage system to participate in a spot power market trading strategy. Background Art

[0002] Offshore wind power is a renewable energy technology that uses abundant wind energy resources at sea to generate electricity. It has the advantages of stable wind speed, small land occupation and small environmental impact. However, offshore wind power also faces some challenges, including wind power fluctuations, grid access difficulties and high operation and maintenance costs. Especially in the context of power marketization, its economic efficiency and competitiveness are more tested.

[0003] In order to solve these problems, water storage technology has been proposed as an effective supplement. Water storage uses hydroelectric generators for charging and discharging, and has the advantages of large energy storage capacity, long life and high efficiency. It can achieve complementarity and coordination with wind power by smoothing wind power fluctuations, providing frequency and peak regulation services, and participating in electricity market transactions. However, traditional water storage power stations face certain limitations in site selection and water resource utilization efficiency.

[0004] To this end, water storage power stations can be built in the sea area near the offshore wind farm using the seabed topography and artificial structures to form an integrated system of offshore wind power and water storage. By using seawater as an energy storage medium, the construction and operation and maintenance costs of water storage power stations can be greatly reduced. However, the current offshore wind power configuration water storage energy storage system mostly relies on thermal power units to provide a stable power supply, and does not fully consider the volatility of system demand after the marketization of electricity and its application scenarios for participating in electricity market transactions.

[0005] Therefore, the water storage energy storage system can perform flexible charging and discharging operations according to the fluctuations of offshore wind power and changes in market prices, smooth the output power of offshore wind power, and thus improve the power quality and market competitiveness. Summary of the invention

[0006] In order to improve the ability of offshore wind power configuration energy storage system to participate in the market, reduce the impact on the power grid, and increase revenue, it is necessary to develop a method for offshore wind power configuration water storage energy storage system to participate in the spot power market trading strategy, including the following steps:

[0007] S1: Collect the operation information of offshore wind power stations and water storage power stations;

[0008] The offshore wind power station operation information includes the total power of the offshore wind power station and the planned workload of the maintenance team; the water storage power station operation information includes the reservoir volume of the water storage power station, the power and efficiency of the pump, motor, turbine, the power and efficiency of the generator, and the loss of the corresponding transmission pipeline;

[0009] S2: Construct an offshore wind power configuration water storage system operating environment and form a virtual environment based on the short-term power dispatching needs of the power grid department;

[0010] The short-term power dispatch demand of the power grid department includes the power generation power of the offshore wind power configuration water storage energy storage system expected by the power grid department, the maximum allowable system deviation, the maximum allowable system power and the minimum allowable system power. The operating environment of the offshore wind power configuration water storage energy storage system is determined according to the load limit of the active devices of the offshore wind power configuration water storage energy storage system and the load limit of the power grid, including the maximum and minimum power of the pump and motor, turbine and generator and the power grid expected system;

[0011] S3: using a deep double Q network model to train an intelligent agent in the virtual environment, the intelligent agent makes a preliminary decision and participates in short-term spot power market transactions to obtain training results, the deep double Q network model includes a Q Network, a Target Q Network and a Replay buffer, and the deep double Q network model includes the intelligent agent, a virtual environment, a state space, an action space and a reward function;

[0012] S4: Based on the training results and strategies in S3, the deep double Q network model is updated and tested, and the preliminary decision that passes the test is used to determine the final trading decision of the offshore wind power configuration water storage energy storage system participating in the short-term electricity spot market.

[0013] This application also proposes a decision-making model for offshore wind power configuration energy storage to participate in the spot power market, including the following modules:

[0014] A data acquisition module, used to collect the operation information of the offshore wind power station and the water storage power station in S1;

[0015] The virtual environment module is used to execute S2 to build an offshore wind power configuration water storage energy storage system operating environment and form a virtual environment according to the short-term power dispatching needs of the power grid department;

[0016] The algorithm model module is used to execute the deep double Q network model in S3 to train the intelligent agent in the virtual environment, and the intelligent agent makes preliminary decisions and participates in short-term spot power market transactions to obtain training results.

[0017] The test update module is used to execute S4 to update and test the deep dual-Q network model based on the training results and strategies of S3, and use the preliminary decision passed by the test to determine the final trading decision of the offshore wind power configuration water storage energy storage system participating in the short-term electricity spot market.

[0018] The formula for the energy storage capacity of the data acquisition module is:

[0019]

[0020] Where, SoC t Represents the energy storage capacity of the offshore wind power configuration water storage system at time t, SoC k,t represents the capacity of the kth water storage power station at time t, represents the water storage capacity of the kth power station at time t, represents the maximum water storage capacity of the upper reservoir;

[0021] The capacity balance and upper and lower physical constraints of the water storage energy are determined by the following formula:

[0022]

[0023] In the formula, and They represent the water storage capacity of the upper reservoir and the lower reservoir of the kth power station at time t, τ represents the self-loss caused by reservoir leakage or evaporation, and They represent the maximum pumping and maximum discharge of the kth power station at time t respectively;

[0024]

[0025] In the formula, and Respectively represent the charging and discharging instructions of the energy storage power station;

[0026]

[0027] In the formula, and Represent the maximum water storage capacity of the upper reservoir and the lower reservoir respectively, and Represent the minimum water storage capacity of the upper reservoir and the lower reservoir respectively.

[0028] The system energy exchange of the virtual environment module operating environment is determined by the following formula:

[0029]

[0030] In the formula, and They represent the maximum pumping and maximum discharge of the kth power station at time t, and Represent the maximum power of the pump and turbine of the kth energy storage power station, c p represents the pumping coefficient, c g Represents the power generation coefficient.

[0031] η pump =ηp ×η m ×η pipe

[0032] η gen =η TR ×η GR ×η pipe

[0033] Where η p , η m and η pipe Represent the efficiency of the pump, motor and pipeline respectively, η TR , η GR and η pipe represent the efficiency of the turbine, generator, and pipeline respectively;

[0034] The system penalty of the operating environment is determined by the following formula:

[0035] P 1,t =υmax[0,(p total,t -p max )]

[0036] P 2,t =υmax[0,(p min -p total,t )]

[0037]

[0038] P t =P 1,t +P 2,t +P 3,t +P 4,t

[0039] Where P t represents the total penalty of the system at time t, υ represents the over-limit penalty coefficient, and p max and p min Represent the maximum power and minimum power of the expected power grid system respectively, ω represents the deviation penalty coefficient, represents the power generation capacity of the system expected by the power grid department at time t, p allow,t represents the difference between the actual power generation and the expected power allowed by the power grid department at time t, ψ represents the maintenance penalty coefficient, M r,t represents the workload of the maintenance team for the rth wind turbine at time t, It means that the maintenance team has arrived at the rth wind turbine at time t.

[0040] The algorithm model module operation steps are:

[0041] S31: Establish a deep dual Q neural network consisting of QNetwork, Target QNetwork and Replay buffer, and initialize the network according to the system model. The Q Network and Target Q Network structures are completely consistent. The Replay buffer is used to record four pieces of information: the current state, action, reward and next state of the agent.

[0042] S32: The agent moves from action space a based on Q Network at time t t Select an action in the virtual environment, submit the current action to the virtual environment, and calculate the benefit R based on the virtual environment feedback. t , and the state S of the agent at this action t 、Action a t , Reward R t and the state S at the next moment t+1 As a vector group, it is stored in the Replay buffer.

[0043] S33: When the data stored in the Replay buffer reaches the set value, batch sample data is randomly extracted therefrom for Q Network parameter training and the parameters are updated using the optimizer according to the principle of minimizing the loss function. The random extraction of the batch sample data uses a Gaussian distribution sampler, and other samplers may also be used. The optimizer uses a stochastic gradient descent optimizer, and other optimizers may also be used.

[0044] The process of the agent making a preliminary decision is a Markov decision, including: establishing a vector group based on four vectors, the vector group is specifically [S t ,a t ,R t ,S t+1 ], where S t 、a t and R t Represent the state space, action space and reward of the agent at time t, S t+1 Represents the state space of the agent at time t+1, where the reward function R t Represents the agent in S t Execute action a at any time t Change to state S t+1 Instant rewards.

[0045] The state space is determined by the following set of vectors:

[0046] S t =[SoC k,t ,p load,t ,M r,t ]

[0047] In the formula, S t Represents the state space of the system at time t, SoC k,t represents the capacity of the kth water storage power station at time t, p total,t represents the total power transmitted to the grid by the system at time t, M r,t Represents the maintenance workload required by the maintenance team for the rth wind turbine at time t.

[0048] The action space is determined by the following set of vectors:

[0049]

[0050] In the formula, a t represents the action space of the system at time t, and Represent the charging and discharging instructions of the energy storage power station, α t Represents the grid connection coefficient of the offshore wind power station, and They respectively represent that the maintenance team has arrived at the rth wind turbine and is on the way to the rth wind turbine at time t.

[0051] The reward function is determined by the following formula:

[0052]

[0053] In the formula, R t represents the reward of the agent at time t, C t represents the electricity price of the system at time t, p total,t represents the total power transmitted to the grid by the system at time t, and Respectively represent the charging and discharging instructions of the energy storage power station, and Represent the maximum power of the pump and turbine of the kth energy storage power station, ξ k Represents the energy storage aging coefficient of the kth energy storage power station, M r,t represents the workload of the maintenance team for the rth wind turbine at time t, and They represent that the maintenance team has arrived at the rth wind turbine and is on the way to the rth wind turbine at time t, respectively. r Represents the labor remuneration coefficient of wind power station maintenance personnel, P t represents the total penalty of the system at time t, C op Represents other operation and maintenance costs, Represents the safe operation factor of the energy storage system.

[0054] The test update module steps are:

[0055] S41: When the data stored in the Replay buffer reaches the set value, batch sample data is randomly extracted from it for Q Network parameter training and the parameters are updated by the optimizer according to the principle of minimizing the loss function. The random extraction uses a Gaussian distribution sampler, and the optimizer can use a stochastic gradient descent optimizer.

[0056] S42: Synchronously updating the parameters of the Q Network to the Target Q Network after a preset time interval.

[0057] S43: Determine whether the network output meets the expected evaluation criteria; if so, end the training; if not, continue to update the iterative deep reinforcement learning network from S42.

[0058] The loss function in the evaluation criteria is determined by the following formula:

[0059]

[0060] In the formula, L(θ c ) is the loss function, and the loss function calculation method is the root mean square error value of Target Q Network and QNetwork, R t represents the reward of the agent at time t, γ represents the discount factor, Represents the state s of Target Q Network at time t+1 t+1 The greedy strategy is used to select the Q value corresponding to action a, θ c represents the parameters of QNetwork, and θ′ represents the parameters of Target Q Network.

[0061] The present application also proposes a computer device, comprising a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read and execute the computer program to execute the above-mentioned method for an offshore wind power configuration water storage energy storage system to participate in the spot power market trading strategy.

[0062] The present application also proposes a computer-readable storage medium having instructions stored thereon. When the instructions are executed on a computer, the above-mentioned method of configuring an offshore wind power water storage energy storage system to participate in a spot electricity market trading strategy is executed.

[0063] Compared with the prior art, the present invention has the following beneficial effects:

[0064] The present invention provides a method for an offshore wind power configuration water storage energy storage system to participate in spot power market trading strategies. The method fully utilizes the characteristics and advantages of offshore wind power and water storage energy storage to construct an integrated system for offshore wind power configuration water storage energy storage. Taking into account the uncertainty of wind power, grid demand and market price, a deep double-Q network learning method is adopted to realize intelligent decision-making and trading of the offshore wind power configuration water storage energy storage system, improve the operating efficiency and benefits of the system, reduce the operating cost and risk of the system, and improve the adaptability and robustness of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below; in all drawings, similar elements or parts are generally identified by similar figure marks; in the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0066] Figure 1 A flow chart of a method for configuring a water storage energy storage system for offshore wind power to participate in a spot power market trading strategy according to the present invention;

[0067] Figure 2 A decision-making model diagram of offshore wind power configuration energy storage participating in the spot power market according to the present invention;

[0068] Figure 3 It is the energy storage capacity calculation principle diagram of the data acquisition module;

[0069] Figure 4 Run the schematic for testing the updated module. DETAILED DESCRIPTION

[0070] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0071] Reference Figures 1 to 4 , a method for an offshore wind power configuration water storage energy storage system to participate in a spot power market trading strategy, comprising the following steps:

[0072] S1: Collect the operation information of offshore wind power stations and water storage power stations;

[0073] The offshore wind power station operation information includes the total power of the offshore wind power station and the planned workload of the maintenance team; the water storage power station operation information includes the reservoir volume of the water storage power station, the power and efficiency of the pump, motor, turbine, the power and efficiency of the generator, and the loss of the corresponding transmission pipeline;

[0074] S2: Construct an offshore wind power configuration water storage system operating environment and form a virtual environment based on the short-term power dispatching needs of the power grid department;

[0075] The short-term power dispatch demand of the power grid department includes the power generation power of the offshore wind power configuration water storage energy storage system expected by the power grid department, the maximum allowable system deviation, the maximum allowable system power and the minimum allowable system power. The operating environment of the offshore wind power configuration water storage energy storage system is determined according to the load limit of the active devices of the offshore wind power configuration water storage energy storage system and the load limit of the power grid, including the maximum and minimum power of the pump and motor, turbine and generator and the power grid expected system;

[0076] S3: using a deep double Q network model to train an intelligent agent in the virtual environment, the intelligent agent makes a preliminary decision and participates in short-term spot power market transactions to obtain training results, the deep double Q network model includes a Q Network, a Target Q Network and a Replay buffer, and the deep double Q network model includes the intelligent agent, a virtual environment, a state space, an action space and a reward function;

[0077] S4: Based on the training results and strategies in S3, the deep double Q network model is updated and tested, and the preliminary decision that passes the test is used to determine the final trading decision of the offshore wind power configuration water storage energy storage system participating in the short-term electricity spot market.

[0078] This application also proposes a decision-making model for offshore wind power configuration energy storage to participate in the spot power market, including the following modules:

[0079] A data acquisition module, used to collect the operation information of the offshore wind power station and the water storage power station in S1;

[0080] The virtual environment module is used to execute S2 to build an offshore wind power configuration water storage energy storage system operating environment and form a virtual environment according to the short-term power dispatching needs of the power grid department;

[0081] The algorithm model module is used to execute the deep double Q network model in S3 to train the intelligent agent in the virtual environment, and the intelligent agent makes preliminary decisions and participates in short-term spot power market transactions to obtain training results.

[0082] The test update module is used to execute S4 to update and test the deep dual-Q network model based on the training results and strategies of S3, and use the preliminary decision passed by the test to determine the final trading decision of the offshore wind power configuration water storage energy storage system participating in the short-term electricity spot market.

[0083] The formula for the energy storage capacity of the data acquisition module is:

[0084]

[0085] Where, SoC t Represents the energy storage capacity of the offshore wind power configuration water storage system at time t, SoC k,t represents the capacity of the kth water storage power station at time t, represents the water storage capacity of the kth power station at time t, represents the maximum water storage capacity of the upper reservoir;

[0086] The capacity balance and upper and lower physical constraints of the water storage energy are determined by the following formula:

[0087]

[0088] In the formula, and They represent the water storage capacity of the upper reservoir and the lower reservoir of the kth power station at time t, τ represents the self-loss caused by reservoir leakage or evaporation, and They represent the maximum pumping and maximum discharge of the kth power station at time t respectively;

[0089]

[0090] In the formula, and Respectively represent the charging and discharging instructions of the energy storage power station;

[0091]

[0092] In the formula, and Represent the maximum water storage capacity of the upper reservoir and the lower reservoir respectively, and Represent the minimum water storage capacity of the upper reservoir and the lower reservoir respectively.

[0093] The system energy exchange of the virtual environment module operating environment is determined by the following formula:

[0094]

[0095] In the formula, and They represent the maximum pumping and maximum discharge of the kth power station at time t, and Represent the maximum power of the pump and turbine of the kth energy storage power station, c p represents the pumping coefficient, c g Represents the power generation coefficient.

[0096] η pump =η p ×η m ×η pipe

[0097] η gen =η TR ×η GR ×η pipe

[0098] Where η p , η m and η pipe Represent the efficiency of the pump, motor and pipeline respectively, η TR , η GR and η pipe represent the efficiency of the turbine, generator, and pipeline respectively;

[0099] The system penalty of the operating environment is determined by the following formula:

[0100] P 1,t =υmax[0,(p total,t -p max )]

[0101] P 2,t =υmax[0,(p min -p total,t )]

[0102]

[0103] P t =P 1,t +P 2,t +P 3,t +P 4,t

[0104] Where P t represents the total penalty of the system at time t, υ represents the over-limit penalty coefficient, and p max and p min Represent the maximum power and minimum power of the expected power grid system respectively, ω represents the deviation penalty coefficient, represents the power generation capacity of the system expected by the power grid department at time t, p allow,t represents the difference between the actual power generation and the expected power allowed by the power grid department at time t, ψ represents the maintenance penalty coefficient, Mr,t represents the workload of the maintenance team for the rth wind turbine at time t, It means that the maintenance team has arrived at the rth wind turbine at time t.

[0105] The algorithm model module operation steps are:

[0106] S31: Establish a deep dual Q neural network consisting of QNetwork, Target QNetwork and Replay buffer, and initialize the network according to the system model. The Q Network and Target Q Network structures are completely consistent. The Replay buffer is used to record four pieces of information: the current state, action, reward and next state of the agent.

[0107] S32: The agent moves from action space a based on Q Network at time t t Select an action in the virtual environment, submit the current action to the virtual environment, and calculate the benefit R based on the virtual environment feedback. t , and the state S of the agent at this action t 、Action a t , Reward R t and the state S at the next moment t+1 As a vector group, it is stored in the Replay buffer.

[0108] S33: When the data stored in the Replay buffer reaches the set value, batch sample data is randomly extracted therefrom for Q Network parameter training and the parameters are updated using the optimizer according to the principle of minimizing the loss function. The random extraction of the batch sample data uses a Gaussian distribution sampler, and the optimizer uses a stochastic gradient descent optimizer.

[0109] The process of the agent making a preliminary decision is a Markov decision, including: establishing a vector group based on four vectors, the vector group is specifically [S t ,a t ,R t ,S t+1 ], where S t 、a t and R t Represent the state space, action space and reward of the agent at time t, S t+1 Represents the state space of the agent at time t+1, where the reward function R t Represents the agent in S t Execute action a at any time t Change to state S t+1 Instant rewards.

[0110] The state space is determined by the following set of vectors:

[0111] S t =[SoC k,t ,p load,t ,M r,t ]

[0112] In the formula, S t Represents the state space of the system at time t, SoC k,t represents the capacity of the kth water storage power station at time t, p total,t represents the total power transmitted to the grid by the system at time t, M r,t Represents the maintenance workload required by the maintenance team for the rth wind turbine at time t.

[0113] The action space is determined by the following set of vectors:

[0114]

[0115] In the formula, a t represents the action space of the system at time t, and Represent the charging and discharging instructions of the energy storage power station, α t Represents the grid connection coefficient of the offshore wind power station, and They respectively represent that the maintenance team has arrived at the rth wind turbine and is on the way to the rth wind turbine at time t.

[0116] The reward function is determined by the following formula:

[0117]

[0118] In the formula, R t represents the reward of the agent at time t, C t represents the electricity price of the system at time t, p total,t represents the total power transmitted to the grid by the system at time t, and Respectively represent the charging and discharging instructions of the energy storage power station, and Represent the maximum power of the pump and turbine of the kth energy storage power station, ξ k Represents the energy storage aging coefficient of the kth energy storage power station, M r,t represents the workload of the maintenance team for the rth wind turbine at time t, and They represent that the maintenance team has arrived at the rth wind turbine and is on the way to the rth wind turbine at time t, respectively. r Represents the labor remuneration coefficient of wind power station maintenance personnel, Pt represents the total penalty of the system at time t, C op Represents other operation and maintenance costs, Represents the safe operation factor of the energy storage system.

[0119] The test update module steps are:

[0120] S41: When the data stored in the Replay buffer reaches the set value, batch sample data is randomly extracted therefrom for Q Network parameter training and the parameters are updated using the optimizer according to the principle of minimizing the loss function. The random extraction uses a Gaussian distribution sampler, and other samplers can also be used. The optimizer uses a stochastic gradient descent optimizer, and other samplers can also be used.

[0121] S42: Synchronously updating the parameters of the Q Network to the Target Q Network after a preset time interval.

[0122] S43: Determine whether the network output meets the expected evaluation criteria; if so, end the training; if not, continue to update the iterative deep reinforcement learning network from S42.

[0123] The loss function in the evaluation criteria is determined by the following formula:

[0124]

[0125] In the formula, L(θ c ) is the loss function, and the loss function calculation method is the root mean square error value of Target Q Network and QNetwork, R t represents the reward of the agent at time t, γ represents the discount factor, Represents the state s of Target Q Network at time t+1 t+1 The greedy strategy is used to select the Q value corresponding to action a, θ c represents the parameters of QNetwork, and θ′ represents the parameters of Target Q Network.

[0126] The present application also proposes a computer device, comprising a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read and execute the computer program to execute the above-mentioned method for an offshore wind power configuration water storage energy storage system to participate in the spot power market trading strategy.

[0127] The present application also proposes a computer-readable storage medium having instructions stored thereon. When the instructions are executed on a computer, the above-mentioned method of configuring an offshore wind power water storage energy storage system to participate in a spot electricity market trading strategy is executed.

[0128] It should be understood that when used in this specification and the appended claims, the terms “include” and “comprises” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof;

[0129] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention; as used in the present specification and the appended claims, the singular forms "a", "an" and "the" are intended to include plural forms unless the context clearly indicates otherwise;

[0130] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

Claims

1. A method for offshore wind power configuration water storage energy storage system to participate in spot power market trading strategy, comprising the following steps: S1: Collect the operation information of offshore wind power stations and water storage power stations; The offshore wind power station operation information includes the total power of the offshore wind power station and the planned workload of the maintenance team; the water storage power station operation information includes the reservoir volume of the water storage power station, the power and efficiency of the pump, motor, turbine, the power and efficiency of the generator, and the loss of the corresponding transmission pipeline; S2: Construct an offshore wind power configuration water storage system operating environment and form a virtual environment based on the short-term power dispatching needs of the power grid department; The short-term power dispatch demand of the power grid department includes the power generation power of the offshore wind power configuration water storage energy storage system expected by the power grid department, the maximum allowable system deviation, the maximum allowable system power and the minimum allowable system power. The operating environment of the offshore wind power configuration water storage energy storage system is determined according to the load limit of the active devices of the offshore wind power configuration water storage energy storage system and the load limit of the power grid, including the maximum and minimum power of the pump and motor, turbine and generator and the power grid expected system; S3: using a deep double Q network model to train an intelligent agent in the virtual environment, the intelligent agent makes a preliminary decision and participates in short-term spot power market transactions to obtain training results, the deep double Q network model includes a Q Network, a Target QNetwork and a Replay buffer, and the deep double Q network model includes the intelligent agent, a virtual environment, a state space, an action space and a reward function; S4: Based on the training results and preliminary decisions in S3, the deep double Q network model is updated and tested, and the preliminary decision that passes the test is used to determine the final trading decision of the offshore wind power configuration water storage energy storage system participating in the short-term electricity spot market.

2. A method for offshore wind power configuration water storage energy storage system to participate in spot power market trading strategy, characterized by: Includes the following modules: A data acquisition module, used to collect the operation information of the offshore wind power station and the water storage power station in S1; The virtual environment module is used to execute S2 to build an offshore wind power configuration water storage energy storage system operating environment and form a virtual environment according to the short-term power dispatching needs of the power grid department; An algorithm model module, used for executing S3 to train an intelligent agent in the virtual environment using a deep double Q network model, wherein the intelligent agent makes a preliminary decision and participates in short-term spot power market transactions to obtain training results; The test update module is used to execute S4 to update and test the deep dual-Q network model based on the training results and preliminary decisions of S3, and use the preliminary decisions that pass the test to determine the final trading decision of the offshore wind power configuration water storage energy storage system participating in the short-term electricity spot market.

3. A method for offshore wind power configuration water storage energy storage system to participate in spot power market trading strategy according to claim 2, characterized in that: The formula for the energy storage capacity of the data acquisition module is: Where, SoC t Represents the energy storage capacity of the offshore wind power configuration water storage system at time t, SoC k,t represents the capacity of the kth water storage power station at time t, represents the water storage capacity of the kth power station at time t, represents the maximum water storage capacity of the upper reservoir; The capacity balance and upper and lower physical constraints of the water storage energy are determined by the following formula: In the formula, and They represent the water storage capacity of the upper reservoir and the lower reservoir of the kth power station at time t, τ represents the self-loss caused by reservoir leakage or evaporation, and They represent the maximum pumping and maximum discharge of the kth power station at time t respectively; In the formula, and Respectively represent the charging and discharging instructions of the energy storage power station; In the formula, and Represent the maximum water storage capacity of the upper reservoir and the lower reservoir respectively, and Represent the minimum water storage capacity of the upper reservoir and the lower reservoir respectively.

4. A method for offshore wind power configuration water storage energy storage system to participate in spot power market trading strategy according to claim 2, characterized in that: The system energy exchange of the virtual environment module operating environment is determined by the following formula: In the formula, and They represent the maximum pumping and maximum discharge of the kth power station at time t, and Represent the maximum power of the pump and turbine of the kth energy storage power station, c p represents the pumping coefficient, c g represents the power generation coefficient; or pump =the p ×h m ×h pipe or gen =the TR ×h GR ×h pipe Where η p , η m and η pipe Represent the efficiency of the pump, motor and pipeline respectively, η TR , η GR and η pipe represent the efficiency of the turbine, generator, and pipeline respectively; The system penalty of the operating environment is determined by the following formula: P 1,t =υmax[0,(p total,t -p max )] P 2,t =υmax[0,(p min -p total,t )] P t =P 1,t +P 2,t +P 3,t +P 4,t Where P t represents the total penalty of the system at time t, υ represents the over-limit penalty coefficient, and p max and p min Represent the maximum power and minimum power of the expected power grid system respectively, ω represents the deviation penalty coefficient, represents the power generation capacity of the system expected by the power grid department at time t, p allow,t represents the difference between the actual power generation and the expected power allowed by the power grid department at time t, ψ represents the maintenance penalty coefficient, M r,t represents the workload of the maintenance team for the rth wind turbine at time t, It means that the maintenance team has arrived at the rth wind turbine at time t.

5. A method for offshore wind power configuration water storage energy storage system to participate in spot power market trading strategy according to claim 2, characterized in that: The algorithm model module operation steps are: S31: Establish a deep dual Q neural network consisting of QNetwork, Target QNetwork and Replay buffer, and initialize the network according to the system model. The Q Network and Target Q Network structures are completely consistent. The Replay buffer is used to record four pieces of information: the current state, action, reward and next state of the agent. S32: The agent moves from action space a based on Q Network at time t t Select an action in the virtual environment, submit the current action to the virtual environment, and calculate the benefit R based on the virtual environment feedback. t , and the state S of the agent at this action t 、Action a t , Reward R t and the state S at the next moment t+1 As a vector group, store it in the Replay buffer; S33: When the data stored in the Replay buffer reaches the set value, batch sample data is randomly extracted therefrom for QNetwork parameter training and the parameters are updated using the optimizer according to the principle of minimizing the loss function. The random extraction of the batch sample data uses a Gaussian distribution sampler, and the optimizer uses a stochastic gradient descent optimizer.

6. A method for offshore wind power configuration water storage energy storage system to participate in spot power market trading strategy according to claim 2, characterized in that: The process of the agent making a preliminary decision is a Markov decision, including: establishing a vector group based on four vectors, the vector group is specifically [S t ,a t ,R t ,S t+1 ], where S t 、a t and R t Represent the state space, action space and reward of the agent at time t, S t+1 Represents the state space of the agent at time t+1, where the reward function R t Represents the agent in S t Execute action a at any time t Change to state S t+1 Instant rewards; The state space is determined by the following set of vectors: S t =[SoC k,t ,p load,t ,M r,t ] In the formula, S t Represents the state space of the system at time t, SoC k,t represents the capacity of the kth water storage power station at time t, p total,t represents the total power transmitted to the grid by the system at time t, M r,t represents the workload of the maintenance team for the rth wind turbine at time t; The action space is determined by the following set of vectors: In the formula, a t represents the action space of the system at time t, and Represent the charging and discharging instructions of the energy storage power station, α t Represents the grid connection coefficient of the offshore wind power station, and They represent that the maintenance team has arrived at the r-th wind turbine and is on the way to the r-th wind turbine at time t respectively; The reward function is determined by the following formula: In the formula, R t represents the reward of the agent at time t, C t represents the electricity price of the system at time t, p total,t represents the total power transmitted to the grid by the system at time t, and Respectively represent the charging and discharging instructions of the energy storage power station, and Represent the maximum power of the pump and turbine of the kth energy storage power station, ξ k Represents the energy storage aging coefficient of the kth energy storage power station, M r,t represents the workload of the maintenance team for the rth wind turbine at time t, and They represent that the maintenance team has arrived at the rth wind turbine and is on the way to the rth wind turbine at time t, respectively. r Represents the labor remuneration coefficient of wind power station maintenance personnel, P t represents the total penalty of the system at time t, C op Represents other operation and maintenance costs, Represents the safe operation factor of the energy storage system.

7. A method for offshore wind power configuration water storage energy storage system to participate in spot power market trading strategy according to claim 2, characterized in that: The test update module steps are: S41: When the data stored in the Replay buffer reaches the set value, batch sample data is randomly extracted from it for QNetwork parameter training and the parameters are updated by the optimizer according to the principle of minimizing the loss function. The random extraction uses a Gaussian distribution sampler, and the optimizer uses a stochastic gradient descent optimizer. S42: Synchronously updating the parameters of the Q Network to the Target Q Network after a preset time interval; S43: Determine whether the network output meets the expected evaluation criteria; if so, end the training; if not, continue to update the iterative deep reinforcement learning network from S42; The loss function in the evaluation criteria is determined by the following formula: In the formula, L(θ c ) is the loss function, and the loss function calculation method is the root mean square error value of Target Q Network and Q Network, R t represents the reward of the agent at time t, γ represents the discount factor, Represents the state s of Target QNetwork at time t+1 t+1 The greedy strategy is used to select the Q value corresponding to action a, θ c represents the parameters of Q Network, and θ′ represents the parameters of Target Q Network.

8. A computer device, characterized in that: It includes a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read and execute computer programs to execute a method for an offshore wind power configuration water storage energy storage system to participate in a spot power market trading strategy as described in claim 1.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the method of participating in a spot power market trading strategy for an offshore wind power configuration water storage energy storage system as described in claim 1 is executed.

Images