Operation and dispatch methods and related devices for combined energy storage power systems

Abstract

This application provides a method and related apparatus for the operation and scheduling of a combined energy storage power system, relating to the field of power system optimization control. The method includes: acquiring the operating state of the combined energy storage power system; selecting candidate scheduling actions that match the operating state from a preset set of scheduling actions based on the operating state; inputting the operating state and candidate scheduling actions into a pre-trained operation and scheduling generation model to obtain the predicted action value of executing the candidate scheduling actions under the operating state, wherein the operation and scheduling generation model is an online Q-network trained based on a dual-Q network; and scheduling the operation of the combined energy storage power system according to the candidate scheduling action with the highest predicted action value among the selected candidate scheduling actions. This application enables the effective scheduling of the operation of a combined energy storage power system.

Description

Technical Field

[0001] This application relates to the field of power system optimization control, and in particular to an operation and dispatching method and related apparatus for a combined energy storage power system. Background Technology

[0002] Power system optimization and control is a crucial field involving the operation, control, and optimization of power systems. Its core objective is to achieve efficient, stable, and reliable operation of the power system through scientific methods. Supply and demand balance is a vital guarantee for improving the stability and reliability of the power system. Furthermore, with the rapid development of renewable energy sources such as wind and solar power, the power system needs continuous improvement to better address the challenges of supply and demand balance. Energy storage technology, as an effective means, balances the power system's supply and demand by storing excess electrical energy and releasing it when needed.

[0003] Currently, most solutions to the problem of optimizing power system operation and dispatch using energy storage technology employ traditional mathematical methods such as operations research. However, when optimizing combined energy storage power systems, such as wind-storage and solar-storage integrated power systems, traditional mathematical methods suffer from problems such as low operating efficiency, slow response speed, and non-convex optimization operation.

[0004] Therefore, there is an urgent need to provide an optimized scheme for the effective operation and scheduling of combined energy storage power systems. Summary of the Invention

[0005] This application provides a method and related apparatus for the operation and scheduling of a combined energy storage power system, which can be used to effectively schedule the operation of the combined energy storage power system.

[0006] Firstly, this application provides an operation and dispatch method for a combined energy storage power system, including:

[0007] Obtain the operating status of the combined energy storage power system;

[0008] Based on the running status, candidate scheduling actions that match the running status are selected from the preset scheduling actions;

[0009] The running state and candidate scheduling actions are input into a pre-trained running schedule generation model to obtain the predicted action value of executing candidate scheduling actions under the running state. The running schedule generation model is an online Q network trained based on a double Q network.

[0010] Among the selected candidate scheduling actions, the operation of the combined energy storage power system is scheduled based on the candidate scheduling action with the highest predictive action value.

[0011] In one possible implementation, the running state and candidate scheduling actions are input into a pre-trained running schedule generation model to obtain the predicted action value of executing the candidate scheduling actions in the running state, including:

[0012] The running state is input into the front-end network of the running schedule generation model for feature extraction to obtain high-dimensional features of the motion state. The front-end network includes network units, which are used to extract high-dimensional features of the input running state.

[0013] The high-dimensional features are input into the flattening layer of the runtime scheduling generation model to transform them into vectors, resulting in a one-dimensional feature vector corresponding to the high-dimensional features.

[0014] The one-dimensional feature vector is input into the value function network of the runtime scheduling generation model to calculate the state estimation value, thereby obtaining the state estimation value corresponding to the runtime state.

[0015] The one-dimensional feature vector and the candidate scheduling action are input into the concatenation layer of the running scheduling generation model for feature concatenation to obtain a joint feature vector;

[0016] The joint feature vector is input into the dominance function network of the runtime scheduling generation model to calculate the dominance value of the scheduling action, thereby obtaining the dominance value of the candidate scheduling action to be executed in the runtime state.

[0017] The state estimation value and the advantage value of the candidate scheduling action are input into the output layer of the runtime scheduling generation model and summed to obtain the predicted action value corresponding to the execution of the candidate scheduling action in the runtime state.

[0018] In one possible implementation, the pre-trained runtime scheduling generation model is obtained in the following way:

[0019] Obtain historical operational information of the combined energy storage power system. The historical operational information includes the operational status of the combined energy storage power system at time t-1, the scheduling actions performed under the operational status at time t-1, the immediate rewards obtained from performing the scheduling actions, and the operational status at time t.

[0020] Based on historical operational information, a sample dataset containing Markov Decision Process (MDP) data units is generated. The MDP data unit includes the operational state at time t-1, the scheduling action executed in the operational state at time t-1, the immediate reward obtained by executing the scheduling action corresponding to the operational state at time t-1, and the operational state at time t.

[0021] Based on the sample dataset, a runtime scheduling generation model is trained to obtain a pre-trained runtime scheduling generation model.

[0022] In one possible implementation, a sample dataset containing Markov Decision Process (MDP) data units is generated based on historical operational information, including:

[0023] Construct an objective function with the goal of optimizing the operation and scheduling of a combined energy storage power system, and define a set of constraints for the objective function. The set of constraints includes power field output constraints and energy storage power station output constraints.

[0024] Based on the set of constraints, select target scheduling actions that meet the constraints from the executed scheduling actions;

[0025] Based on the objective function, the immediate reward for each objective scheduling action is calculated, where the immediate reward is related to the optimization objective of the objective function;

[0026] The running status at time t-1, the running status at time t, the scheduling actions of each target, and the instantaneous rewards of each scheduling action are encoded into MDP data units.

[0027] Aggregate multiple MDP data units to generate a sample data pool;

[0028] A set number of MDP data units are extracted from the sample data pool to generate a sample dataset.

[0029] In one possible implementation, a predetermined number of MDP data units are extracted from the sample data pool to generate a sample dataset, including:

[0030] A set number of MDP data units are randomly selected from the sample data pool to generate a sample dataset;

[0031] or,

[0032] Determine the priority of each MDP data unit in the sample data pool;

[0033] Based on priority, a set number of MDP data units are extracted from the sample data pool to generate a sample dataset.

[0034] In one possible implementation, the dual-Q network includes an online Q-network and a target Q-network, wherein the online Q-network and the target Q-network have the same network architecture. Based on a sample dataset, a runtime scheduling generation model is trained to obtain a pre-trained runtime scheduling generation model, including:

[0035] The running status at time t-1 is input into the online Q network to determine the predicted action value corresponding to each target scheduling action;

[0036] The running state at time t is input into the target Q network, and the output value of the target Q network is determined based on the immediate reward. The output value is the target action value.

[0037] The mean squared error loss between the predicted action value and the target action value is calculated, and the parameters of the online Q network are updated through an optimization algorithm to minimize the mean squared error loss.

[0038] Based on the preset training termination conditions, and according to the updated parameters of the online Q-network, a pre-trained runtime scheduling generation model is obtained.

[0039] In one possible implementation, based on a preset training termination condition and according to the updated parameters of the online Q-network, a pre-trained runtime scheduling generation model is obtained, including:

[0040] Determine whether the number of times the parameters of the online Q network have been updated has reached the threshold.

[0041] When the number of times the parameters of the online Q network are updated reaches the threshold, the parameters of the online Q network are synchronized to the target Q network based on the update strategy.

[0042] Determine whether the online Q-network is in a convergent state;

[0043] When the online Q-network is in a convergent state, the current online Q-network is used as a pre-trained runtime scheduling generation model;

[0044] If the online Q-network is not in a convergent state, continue iterative training until the online Q-network is in a convergent state.

[0045] In one possible implementation, when the number of parameter updates in the online Q-network reaches a threshold, the parameters of the online Q-network are synchronized to the target Q-network based on an update strategy, including:

[0046] When the number of times the parameters of the online Q network are updated reaches the threshold, the parameters of the online Q network are copied to the target Q network.

[0047] or,

[0048] When the number of times the parameters of the online Q-network are updated reaches the threshold, the parameters of the online Q-network and the parameters of the current target Q-network are weighted and summed according to the set weight coefficients to obtain the parameters of the target Q-network.

[0049] Secondly, this application provides an operation and dispatching device for a combined energy storage power system, comprising:

[0050] The acquisition module is used to acquire the operating status of the combined energy storage power system;

[0051] The selection module is used to select candidate scheduling actions that match the running status from the preset scheduling actions based on the running status.

[0052] The determination module is used to input the running state and candidate scheduling actions into the pre-trained running schedule generation model to obtain the predicted action value of executing the candidate scheduling actions under the running state. The running schedule generation model is an online Q network trained based on a double Q network.

[0053] The scheduling module is used to schedule the operation of the combined energy storage power system based on the candidate scheduling action with the highest predictive action value from among multiple selected candidate scheduling actions.

[0054] In one possible implementation, the determining module is specifically used for:

[0055] The running state is input into the front-end network of the running schedule generation model for feature extraction to obtain high-dimensional features of the motion state. The front-end network includes network units, which are used to extract high-dimensional features of the input running state.

[0056] The high-dimensional features are input into the flattening layer of the runtime scheduling generation model to transform them into vectors, resulting in a one-dimensional feature vector corresponding to the high-dimensional features.

[0057] The one-dimensional feature vector is input into the value function network of the runtime scheduling generation model to calculate the state estimation value, thereby obtaining the state estimation value corresponding to the runtime state.

[0058] The one-dimensional feature vector and the candidate scheduling action are input into the concatenation layer of the running scheduling generation model for feature concatenation to obtain a joint feature vector;

[0059] The joint feature vector is input into the dominance function network of the runtime scheduling generation model to calculate the dominance value of the scheduling action, thereby obtaining the dominance value of the candidate scheduling action to be executed in the runtime state.

[0060] The state estimation value and the advantage value of the candidate scheduling action are input into the output layer of the runtime scheduling generation model and summed to obtain the predicted action value corresponding to the execution of the candidate scheduling action in the runtime state.

[0061] In one possible implementation, the pre-trained runtime scheduling generation model is obtained in the following way:

[0062] Obtain historical operational information of the combined energy storage power system. The historical operational information includes the operational status of the combined energy storage power system at time t-1, the scheduling actions performed under the operational status at time t-1, the immediate rewards obtained from performing the scheduling actions, and the operational status at time t.

[0063] Based on historical operational information, a sample dataset containing Markov Decision Process (MDP) data units is generated. The MDP data unit includes the operational state at time t-1, the scheduling action executed in the operational state at time t-1, the immediate reward obtained by executing the scheduling action corresponding to the operational state at time t-1, and the operational state at time t.

[0064] Based on the sample dataset, a runtime scheduling generation model is trained to obtain a pre-trained runtime scheduling generation model.

[0065] In one possible implementation, the operation and dispatching device of the combined energy storage power system further includes a processing module, which is specifically used for:

[0066] Construct an objective function with the goal of optimizing the operation and scheduling of a combined energy storage power system, and define a set of constraints for the objective function. The set of constraints includes power field output constraints and energy storage power station output constraints.

[0067] Based on the set of constraints, select target scheduling actions that meet the constraints from the executed scheduling actions;

[0068] Based on the objective function, the immediate reward for each objective scheduling action is calculated, where the immediate reward is related to the optimization objective of the objective function;

[0069] The running status at time t-1, the running status at time t, the scheduling actions of each target, and the instantaneous rewards of each scheduling action are encoded into MDP data units.

[0070] Aggregate multiple MDP data units to generate a sample data pool;

[0071] A set number of MDP data units are extracted from the sample data pool to generate a sample dataset.

[0072] In one possible implementation, the processing module is further configured to:

[0073] A set number of MDP data units are randomly selected from the sample data pool to generate a sample dataset;

[0074] or,

[0075] Determine the priority of each MDP data unit in the sample data pool;

[0076] Based on priority, a set number of MDP data units are extracted from the sample data pool to generate a sample dataset.

[0077] In one possible implementation, the dual-Q network includes an online Q network and a target Q network, wherein the online Q network and the target Q network have the same network architecture, and the determining module is specifically used for:

[0078] The running status at time t-1 is input into the online Q network to determine the predicted action value corresponding to each target scheduling action;

[0079] The running state at time t is input into the target Q network, and the output value of the target Q network is determined based on the immediate reward. The output value is the target action value.

[0080] The mean squared error loss between the predicted action value and the target action value is calculated, and the parameters of the online Q network are updated through an optimization algorithm to minimize the mean squared error loss.

[0081] Based on the preset training termination conditions, and according to the updated parameters of the online Q-network, a pre-trained runtime scheduling generation model is obtained.

[0082] In one possible implementation, the determining module is specifically used for:

[0083] Determine whether the number of times the parameters of the online Q network have been updated has reached the threshold.

[0084] When the number of times the parameters of the online Q network are updated reaches the threshold, the parameters of the online Q network are synchronized to the target Q network based on the update strategy.

[0085] Determine whether the online Q-network is in a convergent state;

[0086] When the online Q-network is in a convergent state, the current online Q-network is used as a pre-trained runtime scheduling generation model;

[0087] If the online Q-network is not in a convergent state, continue iterative training until the online Q-network is in a convergent state.

[0088] In one possible implementation, the processing module is specifically used for:

[0089] When the number of times the parameters of the online Q network are updated reaches the threshold, the parameters of the online Q network are copied to the target Q network.

[0090] or,

[0091] When the number of times the parameters of the online Q-network are updated reaches the threshold, the parameters of the online Q-network and the parameters of the current target Q-network are weighted and summed according to the set weight coefficients to obtain the parameters of the target Q-network.

[0092] Thirdly, this application provides an electronic device, including: a memory and a processor;

[0093] The memory stores the instructions that the computer executes;

[0094] The processor executes computer execution instructions stored in memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0095] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible embodiments of the first aspect.

[0096] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0097] The operation and scheduling method and related apparatus for a combined energy storage power system provided in this application relate to power system optimization control. The method includes: acquiring the operating state of the combined energy storage power system; selecting candidate scheduling actions that match the operating state from a set of preset scheduling actions based on the operating state; inputting the operating state and candidate scheduling actions into a pre-trained operation and scheduling generation model to obtain the predicted action value of executing the candidate scheduling actions under the operating state, wherein the operation and scheduling generation model is an online Q-network trained based on a dual-Q network; and scheduling the operation of the combined energy storage power system according to the candidate scheduling action with the highest predicted action value among the selected candidate scheduling actions. After obtaining the operating status of the combined energy storage power system, this application selects candidate scheduling actions that match the operating status based on preset scheduling actions. This selection process avoids interference from invalid scheduling actions on the estimation of predicted action value by constraining the action exploration space. When evaluating the predicted action value of candidate scheduling actions, a dual-Q network collaborative architecture is constructed to separate the coupling relationship between scheduling action selection and predicted action value. The online Q network is used to select candidate scheduling actions with the maximum predicted action value that match the operating status from the candidate scheduling actions, while the other Q network is used to evaluate the target action value of the candidate actions based on updated independent parameters. This division of labor mechanism cuts off the error propagation path. The candidate scheduling action with the maximum predicted action value is selected from multiple candidate scheduling actions, and the operation of the combined energy storage power system is effectively scheduled based on the candidate scheduling action with the maximum predicted action value. Attached Figure Description

[0098] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0099] Figure 1 A flowchart illustrating the operation and dispatch method for the combined energy storage power system provided in this application. Figure 1 ;

[0100] Figure 2 A flowchart illustrating the operation and scheduling method of the wind-solar-storage power system provided in the embodiments of this application. Figure 1 ;

[0101] Figure 3 A schematic diagram of the operation and dispatching device of the combined energy storage power system provided in this application;

[0102] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0103] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0104] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0105] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with relevant laws, regulations, and standards, necessary confidentiality measures have been taken, they do not violate public order and good morals, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0106] Currently, most solutions to the problem of optimizing power system operation and scheduling using energy storage methods employ traditional mathematical methods such as operations research. However, for optimizing wind-storage and solar-storage combined power plants in a market environment, traditional mathematical methods suffer from low operational efficiency, slow response speed, and the possibility of non-convex optimization. To address these challenges, in recent years, artificial intelligence algorithms have begun to be used to optimize the operation of wind-storage and solar-storage combined power plants, achieving promising results. Artificial intelligence applications have been a rapidly developing field in recent years, and Deep Reinforcement Learning (DRL) algorithms are one of the most widely researched and applied algorithms. Using DRL algorithms to optimize the operation and scheduling of wind-storage and solar-storage combined power plants has demonstrated advantages such as high scheduling efficiency, fast response speed, and accurate output decisions, showing promising application prospects.

[0107] However, existing DRL algorithms, such as Proximal Policy Optimization (PPO), Deep Deterministic Policy Gradient (DDPG), and Deep Q-Network (DQN), still face difficulties in handling high-dimensional state and action spaces, which limits the application of DRL algorithms in complex power systems.

[0108] To address the aforementioned issues, this application proposes an operation and scheduling method for a combined energy storage power system. By constructing a dual-Q network collaborative architecture, the coupling relationship between scheduling action selection and predicted action value is separated, and the error propagation path is cut off, thereby achieving effective scheduling of the operation of the combined energy storage power system.

[0109] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0110] Figure 1 A flowchart illustrating the operation and dispatch method for the combined energy storage power system provided in this application. Figure 1 ,like Figure 1 As shown, the method includes:

[0111] S101. Obtain the operating status of the combined energy storage power system.

[0112] Among them, a combined energy storage power system refers to a combination of multiple different types of energy storage technologies or equipment to form a collaborative power energy storage system, which is used to improve the stability, reliability and efficiency of the power system, while meeting diverse power demands.

[0113] The operational status of a combined energy storage power system refers to a comprehensive description of its working conditions and performance at a specific moment. The operational status reflects the real-time operating status of individual energy storage devices, power equipment, and the overall combined energy storage power system. It includes, but is not limited to, the ratio of the current remaining energy of the energy storage devices to the total energy, charging and discharging power, and charging and discharging efficiency.

[0114] S102. Based on the operating status, select candidate scheduling actions that match the operating status from the preset scheduling actions.

[0115] In this step, it can be understood that the preset dispatch actions refer to the operations that the combined energy storage power system can perform. By analyzing the operating status, it is possible to determine which dispatch actions among the preset actions are feasible and effective, and then select candidate dispatch actions that match the operating status. For example, assuming that the state of charge of the energy storage device is low and the grid frequency is low, the preset dispatch action of discharging is selected as the candidate dispatch action.

[0116] S103. Input the running state and candidate scheduling actions into the pre-trained running schedule generation model to obtain the predicted action value of executing candidate scheduling actions under the running state. The running schedule generation model is an online Q network trained based on a double Q network.

[0117] After determining the candidate scheduling actions that match the running state through S102, the determined running state and candidate scheduling actions need to be input into the pre-trained running scheduling generation model to obtain the predicted action value of executing the candidate scheduling actions under the running state.

[0118] For example, the running state and candidate scheduling actions are input into a pre-trained running schedule generation model to obtain the predicted action value of executing the candidate scheduling actions under the running state. This includes: inputting the running state into the front-end network of the running schedule generation model for feature extraction to obtain high-dimensional features of the motion state, wherein the front-end network includes network units used to extract the high-dimensional features of the input running state; inputting the high-dimensional features into the flattening layer of the running schedule generation model for feature-to-vector conversion to obtain a one-dimensional feature vector corresponding to the high-dimensional features; inputting the one-dimensional feature vector into the value function network of the running schedule generation model to calculate the state estimation value to obtain the state estimation value corresponding to the running state; inputting the one-dimensional feature vector and the candidate scheduling actions into the concatenation layer of the running schedule generation model for feature concatenation to obtain a joint feature vector; inputting the joint feature vector into the dominance function network of the running schedule generation model to calculate the dominance value of the scheduling action to obtain the dominance value of executing the candidate scheduling action under the running state; and inputting the state estimation value and the dominance value of the candidate scheduling action into the output layer of the running schedule generation model for summation to obtain the predicted action value corresponding to executing the candidate scheduling action under the running state.

[0119] In this example, it can be understood that the pre-trained runtime scheduling generation model is an online Q-network trained based on a dual Q-network.

[0120] S104. Among the selected candidate scheduling actions, the operation of the joint energy storage power system is scheduled according to the candidate scheduling action with the highest predictive action value.

[0121] In S104, it can be understood that different candidate scheduling actions correspond to different predicted action values. From the predicted action values ​​corresponding to multiple different candidate scheduling actions, the candidate scheduling action with the highest predicted action value is selected, and the candidate scheduling action with the highest predicted action value is used as the operation and scheduling scheme of the combined energy storage power system, thereby realizing the effective scheduling of the operation of the combined energy storage power system.

[0122] After obtaining the operating status of the combined energy storage power system, this embodiment selects candidate scheduling actions that match the operating status based on preset scheduling actions. This selection process avoids interference from invalid scheduling actions on the estimation of predicted action value by constraining the action exploration space. When evaluating the predicted action value of candidate scheduling actions, a dual-Q network collaborative architecture is constructed to separate the coupling relationship between scheduling action selection and predicted action value. The online Q network is used to select candidate scheduling actions with the maximum predicted action value that match the operating status from the candidate scheduling actions, while the other Q network is used to evaluate the target action value of the candidate actions based on updated independent parameters. This division of labor mechanism cuts off the error propagation path. The candidate scheduling action with the maximum predicted action value is selected from multiple candidate scheduling actions, and the operation of the combined energy storage power system is effectively scheduled based on the candidate scheduling action with the maximum predicted action value.

[0123] Based on the above embodiments, the pre-trained operation scheduling generation model is obtained in the following ways: acquiring historical operation information of the combined energy storage power system, including the operation status of the combined energy storage power system at time t-1, the scheduling actions performed in the operation status at time t-1, the immediate rewards obtained from performing the scheduling actions, and the operation status at time t; generating a sample dataset containing MDP data units based on the historical operation information, including the operation status at time t-1, the scheduling actions performed in the operation status at time t-1, the immediate rewards obtained from performing the scheduling actions corresponding to the operation status at time t-1, and the operation status at time t; and training the operation scheduling generation model based on the sample dataset to obtain the pre-trained operation scheduling generation model.

[0124] In this embodiment, it can be understood that before training the operation scheduling generation model, a sample dataset needs to be determined, and the sample dataset is generated based on the historical operation information of the combined energy storage power system.

[0125] Specifically, based on historical operational information, a sample dataset containing MDP data units is generated, including: constructing an objective function aimed at optimizing the operation and scheduling of the joint energy storage power system and defining a set of constraints for the objective function, including power field output constraints and energy storage power station output constraints; selecting target scheduling actions that meet the constraints from the executed scheduling actions based on the set of constraints; calculating the immediate reward for each target scheduling action based on the objective function, where the immediate reward is related to the optimization objective of the objective function; encoding the operating state at time t-1, the operating state at time t, each target scheduling action, and the immediate reward for each target scheduling action into MDP data units; aggregating multiple MDP data units to generate a sample data pool; and extracting a set number of MDP data units from the sample data pool to generate the sample dataset.

[0126] The objective function for optimizing the operation and scheduling of the combined energy storage power system can be constructed according to actual needs.

[0127] In one example, taking a wind-solar-storage combined power system as an example, an objective function is constructed with the goal of minimizing the total operating cost of the wind-solar-storage combined power system. The total cost includes the costs of wind and solar tracking assessments, wind and solar curtailment, and minimizing the operating costs of energy storage. The formula for calculating the total cost is as follows: ; where C wpb C represents the total cost of a combined wind-solar-storage power system. k To assess the cost of wind and solar power tracking, C q For the cost of wind and solar power curtailment, C bt This refers to the operating costs of energy storage.

[0128] The set of constraints includes power output constraints for wind farms, photovoltaic power farms, and energy storage power stations. The power output constraints for wind farms are as follows:

[0129]

[0130] Among them, V wt (t) represents the difference in output power of the wind farm at time t and time t-1; P wt (t) represents the output power of the wind farm at time t-1; V wtmax This represents the maximum theoretical output power of the wind farm.

[0131] The output constraints of the photovoltaic power plant are:

[0132]

[0133] Among them, V pv (t) represents the difference in output power between the photovoltaic electric field at time t and time t-1; P pv (t) represents the output power of the photovoltaic electric field at time t-1; V pvmax This represents the maximum theoretical output power of the photovoltaic electric field.

[0134] The output constraints of the energy storage power station are:

[0135]

[0136] Among them, P btmax H represents the maximum charging and discharging power of the energy storage equipment in the energy storage power station. socmin and H socmax These are the upper and lower limits of the energy storage's State of Charge (SOC); H soc (t) represents the state of charge of the energy storage device in the energy storage power station at time t.

[0137] Furthermore, the operating states at time t-1, the operating states at time t, the scheduling actions of each target, and the immediate rewards of each scheduling action are encoded into MDP data units. An MDP data unit can be represented as a tuple, i.e., S, A, ... R, where S represents the Markov state space; A represents the scheduling action space; R represents the discount factor, which is between 0 and 1 and determines the degree of influence of future rewards; R represents the reward function.

[0138] Establish Markov state S: Track the planned value S of the joint power plant plan Energy storage charging and discharging power S bt S soc Wind power forecasting and processing S wt And the projected output S of photovoltaic power pv The state space for the five input channels is represented as follows: .

[0139] Establish a scheduling action space A: Incremental wind power output A wt Photovoltaic power output increment A pv Energy storage output increment A bt As a scheduling action space, it is represented as: .

[0140] Establish a reward function: For an agent in a certain state Select scheduling action The immediate reward at any time. The cumulative reward function over the entire scheduling period T is: The scheduling cycle refers to the time range within which a wind-solar-energy storage system enters a new state after several scheduling operations, starting from one state. This could be an hour, a day, a time range where the interval between consecutive scheduling actions does not exceed a preset interval, or other time periods of business significance.

[0141] In this embodiment, the process starts from the first scheduling instruction running state s0 and ends in the scheduling end running state s0. n MDP data units are formed sequentially according to n scheduling steps. , The sample data is loaded into the sample data pool. During the entire scheduling period T, the scheduling period reward function for the samples is:

[0142] .

[0143] Where, r + and r - These represent positive and negative reward values, respectively, with tar representing the scheduling target. The scheduling period reward can be used to determine the priority weight of samples.

[0144] In this embodiment, by generating a sample dataset containing MDP data units based on historical operation information, it is possible to reflect the operation rules and dynamic characteristics of the actual combined energy storage power system, making the trained operation scheduling generation model closer to the real scenario and helping to improve the effective operation of the combined energy storage power system.

[0145] Furthermore, after aggregating multiple MDP data units to generate a sample data pool, it is necessary to extract a set number of MDP data units from the sample data pool to generate a sample dataset. Specifically, extracting a set number of MDP data units from the sample data pool to generate the sample dataset includes: randomly selecting a set number of MDP data units from the sample data pool to generate the sample dataset; or, determining the priority of each MDP data unit in the sample data pool; and extracting a set number of MDP data units from the sample data pool according to the priority to generate the sample dataset.

[0146] This means that there are different ways to generate a sample dataset. The first way is to randomly select a set number of MDP data units from the sample data pool. The set number can be set according to actual needs, for example, the number can be set to 3.

[0147] The second implementation method first requires determining the priority of each MDP data unit in the sample data pool. Then, based on the priority, a set number of MDP data units are extracted from the sample data pool. The priority can be calculated using weighting rules, which can be based on a comprehensive evaluation of factors such as the reduction in operating costs, the reduction in wind and solar curtailment rates, and the improvement in energy storage utilization. For example, a sample within a scheduling cycle with significantly reduced operating costs and high energy storage utilization will have a higher priority weight so that it can be sampled more frequently in subsequent training.

[0148] This application embodiment improves the flexibility of the joint energy storage power system operation and scheduling method by providing different methods for generating sample datasets.

[0149] Furthermore, the dual-Q network described in S103 includes an online Q-network and a target Q-network. The online Q-network and the target Q-network have the same network architecture. Based on the sample dataset, a runtime scheduling generation model is trained to obtain a pre-trained runtime scheduling generation model, including: inputting the runtime state at time t-1 into the online Q-network to determine the predicted action value corresponding to each target scheduling action; inputting the runtime state at time t into the target Q-network and determining the output value of the target Q-network based on the immediate reward, the output value being the target action value; calculating the mean squared error loss between the predicted action value and the target action value, and updating the parameters of the online Q-network through an optimization algorithm to minimize the mean squared error loss; and obtaining the pre-trained runtime scheduling generation model based on the updated parameters of the online Q-network according to a preset training termination condition.

[0150] In this embodiment, it can be understood that both the online Q-network and the target Q-network are multi-input channels and dual-output subnets. The number of input channels is the same as the number of channels in the previously established state space S. The dual-output subnet includes a front-end network and an output subnet. The network inputs pass through the front-end network into the output subnet. The output subnet includes a value function network Vn and a dominance function network An. The value function network is responsible for evaluating the value of the running state s, and the dominance function network is responsible for evaluating the merits of each scheduling action in the running state s. The final network value is:

[0151]

[0152] in, Indicates the network parameters of the dual-output subnet; a, Let Vn and An represent the parameters of the value function network and the advantage function network, respectively; A represents the set of all target scheduling actions. This indicates the target scheduling action for the next operational state. .

[0153] The front-end network consists of multiple network units, each of which is composed of convolutional layers, normalization layers, and activation functions. The output subnet is built from fully connected layers, and the front-end network is connected to the output subnet through a planarization layer.

[0154] After determining the structure of the dual-Q network, a runtime scheduling generation model needs to be trained based on the sample dataset to obtain a pre-trained runtime scheduling generation model. Specifically, for each training data point in the sample dataset, the predicted action value and the target action value need to be calculated. That is, the runtime state at time t-1 is input into the online Q network to determine the predicted action value, i.e., the Q value, corresponding to each target scheduling action. The runtime state at time t is input into the target Q network, and the target action value, i.e., the Y value, is obtained by combining it with the immediate reward value r.

[0155] The loss function for predicting action value and target action value is expressed by the following formula:

[0156] .

[0157] in, .

[0158] j is the training step size, E[] 2 This indicates the calculation of the mean square error. This indicates that the online Q-network in a dual-Q network evaluates the next running state s. i+1 The most valuable action is a m ; Indicates the discount factor; This represents the sample data in the sample dataset.

[0159] It should be noted that the online Q-network and the target Q-network need to be initialized before training the dual Q-network.

[0160] After determining the average error loss between the predicted action value and the target action value, the parameters of the online Q-network are updated using an optimization algorithm to minimize the mean squared error loss. Then, based on preset training termination conditions and the updated parameters of the online Q-network, a pre-trained runtime scheduling generation model is obtained.

[0161] This application embodiment constructs a dual-Q network collaborative architecture to separate the coupling relationship between scheduling action selection and predicted action value. The online Q network is used to select candidate scheduling actions that meet the running state and have the maximum predicted action value from the candidate scheduling actions, while the target Q network is used to evaluate the target action value of the candidate actions based on the updated independent parameters. This division of labor mechanism cuts off the error propagation path, thereby improving the accuracy of the running scheduling generation model.

[0162] Furthermore, based on the preset training termination conditions, a pre-trained runtime scheduling generation model is obtained according to the updated parameters of the online Q-network. This includes: determining whether the number of times the parameters of the online Q-network have been updated has reached a threshold; when the number of times the parameters of the online Q-network have been updated has reached the threshold, synchronizing the parameters of the online Q-network to the target Q-network based on the update strategy; determining whether the online Q-network is in a convergent state; when the online Q-network is in a convergent state, using the current online Q-network as the pre-trained runtime scheduling generation model; and when the online Q-network is not in a convergent state, continuing iterative training until the online Q-network is in a convergent state.

[0163] In this embodiment, it can be understood that, in determining the pre-trained runtime scheduling generation model, it is necessary to determine whether the number of times the parameters of the online Q network have been updated has reached a threshold, or it can be understood as determining whether the training frequency of the parameters of the online Q network has reached a frequency threshold.

[0164] When the number of updates to the parameters of the online Q-network reaches a threshold, the parameters of the online Q-network are synchronized to the target Q-network based on the update strategy. For example, synchronizing the parameters of the online Q-network to the target Q-network based on the update strategy includes: copying the parameters of the online Q-network to the target Q-network; or, when the number of updates to the parameters of the online Q-network reaches the threshold, weighting and summing the parameters of the online Q-network and the current parameters of the target Q-network according to a set weight coefficient to obtain the parameters of the target Q-network.

[0165] In this example, the update strategy can be understood to include the following methods: directly copying the parameters of the online Q-network to the target Q-network (hard update), or weighting and summing the parameters of the online Q-network and the current target Q-network according to a set weight coefficient to obtain the parameters of the target Q-network (soft update). This example improves the flexibility of the joint energy storage power system motion dispatch method by providing different parameter update methods to meet different user needs.

[0166] Furthermore, after synchronizing the parameters of the online Q-network to the target Q-network, it is necessary to determine whether the online Q-network is in a convergent state, and to implement corresponding measures based on whether the online Q-network is in a convergent state. When the online Q-network is in a convergent state, the current online Q-network is used as a pre-trained runtime scheduling generation model; when the online Q-network is not in a convergent state, iterative training continues until the online Q-network is in a convergent state.

[0167] Furthermore, determining whether an online Q-network is in a convergent state is equivalent to determining whether it has reached a stable state, meaning that the parameters or output values ​​of the online Q-network no longer change significantly and are close to a stable value. The specific implementation method for determining whether an online Q-network is in a convergent state can be chosen based on the actual scenario.

[0168] In one implementation, the change in the predicted action value of the online Q-network, i.e. the change in the Q-value, is observed. If the Q-value fluctuates very little or tends to stabilize over a period of time, the online Q-network can be considered to have converged.

[0169] In another implementation, the loss function value of the online Q-network is checked. If the loss function value of the online Q-network gradually decreases and tends to stabilize, it can be said that the online Q-network has converged.

[0170] Next, taking a combined energy storage power system as a wind-solar energy storage power system and a dual-Q network as a dueling double DQN (D3QN) deep reinforcement learning network as an example, we will combine... Figure 2 This document explains how to utilize the operation and scheduling method of the combined energy storage power system provided in the embodiments of this application. Figure 2 A flowchart illustrating the operation and scheduling method of the wind-solar-storage power system provided in the embodiments of this application. Figure 1 ,like Figure 2 As shown, the method includes the following steps:

[0171] 1. Collect operational information from the wind-solar combined energy storage system and record it as data;

[0172] 2. Establish an objective function, with the goal of minimizing the cost of wind and solar tracking assessment, the cost of wind and solar curtailment, and the cost of energy storage operation;

[0173] 3. Determine the constraints, including the output constraints of wind farms, photovoltaic farms, and energy storage power stations;

[0174] 4. Based on the objective function and constraints, the operational information collected in step 1 is processed into Markov decision process data units to form a sample data pool. The specific construction principle has been explained in detail in the previous embodiments, so it will not be repeated here.

[0175] 5. Construct an online Q network and a target Q network, where both Q networks are network architectures with multiple input channels and dual output subnets;

[0176] 6. Initialize the online Q-network and the target Q-network;

[0177] 7. Extract several Markov decision process data units from the sample data pool in step 4 to train the online Q network, and copy the parameters of the online Q network to the parameters of the target Q network at a fixed frequency.

[0178] 8. Calculate the predicted action value and the target action value for the sample data in the sample dataset, and update the online Q network parameters by optimizing the algorithm to minimize the mean squared error loss between the predicted action value and the target action value;

[0179] 9. Repeat steps 7 and 8 until the online Q-network converges, then end the model training.

[0180] 10. Call the trained online Q-network as a knowledge network to determine the corresponding predicted action value for multiple candidate scheduling actions. The candidate scheduling action with the highest predicted action value is determined as the operation scheduling method of the wind-solar-energy storage power system and output. In this way, the optimal strategy with the minimum wind-solar tracking assessment cost, wind curtailment cost, and energy storage operation cost is obtained, thereby optimizing the operation efficiency of the wind-solar-energy storage system and improving economic benefits.

[0181] In summary, this application proposes an improved DRL algorithm based on the DQN basic network structure, namely the D3QN model. This model avoids the overestimation problem in the optimization process of traditional algorithms and DQN algorithms, obtains more accurate results, improves operating efficiency, speeds up the operation response, optimizes the scheduling and operation strategy of wind-solar-energy storage systems, reduces operating costs, and improves economic benefits.

[0182] Figure 3 A schematic diagram of the operation and dispatching device of the combined energy storage power system provided in this application is shown below. Figure 3 As shown, the operation and dispatching device 300 for the combined energy storage power system provided in this embodiment includes:

[0183] The acquisition module 301 is used to acquire the operating status of the combined energy storage power system;

[0184] The selection module 302 is used to select candidate scheduling actions that match the running status from the preset scheduling actions based on the running status.

[0185] The determination module 303 is used to input the running state and candidate scheduling actions into the pre-trained running schedule generation model to obtain the predicted action value of executing the candidate scheduling actions under the running state. The running schedule generation model is an online Q network trained based on a double Q network.

[0186] The scheduling module 304 is used to schedule the operation of the combined energy storage power system based on the candidate scheduling action with the highest predictive action value among a number of selected candidate scheduling actions.

[0187] In one possible implementation, the determining module 303 is specifically used for:

[0188] The running state is input into the front-end network of the running schedule generation model for feature extraction to obtain high-dimensional features of the motion state. The front-end network includes network units, which are used to extract high-dimensional features of the input running state.

[0189] The high-dimensional features are input into the flattening layer of the runtime scheduling generation model to transform them into vectors, resulting in a one-dimensional feature vector corresponding to the high-dimensional features.

[0190] The one-dimensional feature vector is input into the value function network of the runtime scheduling generation model to calculate the state estimation value, thereby obtaining the state estimation value corresponding to the runtime state.

[0191] The one-dimensional feature vector and the candidate scheduling action are input into the concatenation layer of the running scheduling generation model for feature concatenation to obtain a joint feature vector;

[0192] The joint feature vector is input into the dominance function network of the runtime scheduling generation model to calculate the dominance value of the scheduling action, thereby obtaining the dominance value of the candidate scheduling action to be executed in the runtime state.

[0193] The state estimation value and the advantage value of the candidate scheduling action are input into the output layer of the runtime scheduling generation model and summed to obtain the predicted action value corresponding to the execution of the candidate scheduling action in the runtime state.

[0194] In one possible implementation, the pre-trained runtime scheduling generation model is obtained in the following way:

[0195] Obtain historical operational information of the combined energy storage power system. The historical operational information includes the operational status of the combined energy storage power system at time t-1, the scheduling actions performed under the operational status at time t-1, the immediate rewards obtained from performing the scheduling actions, and the operational status at time t.

[0196] Based on historical operational information, a sample dataset containing Markov Decision Process (MDP) data units is generated. The MDP data unit includes the operational state at time t-1, the scheduling action executed in the operational state at time t-1, the immediate reward obtained by executing the scheduling action corresponding to the operational state at time t-1, and the operational state at time t.

[0197] Based on the sample dataset, a runtime scheduling generation model is trained to obtain a pre-trained runtime scheduling generation model.

[0198] In one possible implementation, the operation and dispatching device of the combined energy storage power system further includes a processing module (not shown), which is specifically used for:

[0199] Construct an objective function with the goal of optimizing the operation and scheduling of a combined energy storage power system, and define a set of constraints for the objective function. The set of constraints includes power field output constraints and energy storage power station output constraints.

[0200] Based on the set of constraints, select target scheduling actions that meet the constraints from the executed scheduling actions;

[0201] Based on the objective function, the immediate reward for each objective scheduling action is calculated, where the immediate reward is related to the optimization objective of the objective function;

[0202] The running status at time t-1, the running status at time t, the scheduling actions of each target, and the instantaneous rewards of each scheduling action are encoded into MDP data units.

[0203] Aggregate multiple MDP data units to generate a sample data pool;

[0204] A set number of MDP data units are extracted from the sample data pool to generate a sample dataset.

[0205] In one possible implementation, the processing module is further configured to:

[0206] A set number of MDP data units are randomly selected from the sample data pool to generate a sample dataset;

[0207] or,

[0208] Determine the priority of each MDP data unit in the sample data pool;

[0209] Based on priority, a set number of MDP data units are extracted from the sample data pool to generate a sample dataset.

[0210] In one possible implementation, the dual-Q network includes an online Q network and a target Q network, wherein the online Q network and the target Q network have the same network architecture, and the determining module is specifically used for:

[0211] The running status at time t-1 is input into the online Q network to determine the predicted action value corresponding to each target scheduling action;

[0212] The running state at time t is input into the target Q network, and the output value of the target Q network is determined based on the immediate reward. The output value is the target action value.

[0213] The mean squared error loss between the predicted action value and the target action value is calculated, and the parameters of the online Q network are updated through an optimization algorithm to minimize the mean squared error loss.

[0214] Based on the preset training termination conditions, and according to the updated parameters of the online Q-network, a pre-trained runtime scheduling generation model is obtained.

[0215] In one possible implementation, the determining module 303 is specifically used for:

[0216] Determine whether the number of times the parameters of the online Q network have been updated has reached the threshold.

[0217] When the number of times the parameters of the online Q network are updated reaches the threshold, the parameters of the online Q network are synchronized to the target Q network based on the update strategy.

[0218] Determine whether the online Q-network is in a convergent state;

[0219] When the online Q-network is in a convergent state, the current online Q-network is used as a pre-trained runtime scheduling generation model;

[0220] If the online Q-network is not in a convergent state, continue iterative training until the online Q-network is in a convergent state.

[0221] In one possible implementation, the processing module is specifically used for:

[0222] When the number of times the parameters of the online Q network are updated reaches the threshold, the parameters of the online Q network are copied to the target Q network.

[0223] or,

[0224] When the number of times the parameters of the online Q-network are updated reaches the threshold, the parameters of the online Q-network and the parameters of the current target Q-network are weighted and summed according to the set weight coefficients to obtain the parameters of the target Q-network.

[0225] The operation and scheduling device for the combined energy storage power system provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0226] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing element calls; they can be fully implemented in hardware; or some modules can be implemented by processing element calls to software, while others are implemented in hardware. For example, a processing module can be a separate processing element, or it can be integrated into a chip within the above device. Alternatively, it can be stored as program code in the device's memory, and its functions can be called and executed by a processing element within the device. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0227] For example, these modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more Digital Signal Processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). As another example, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a System-On-a-Chip (SOC).

[0228] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4 As shown, the electronic device 400 provided in this application embodiment may include: a processor 401, and a memory 402 communicatively connected to the processor, wherein:

[0229] The memory stores the instructions that the computer executes;

[0230] The processor executes computer execution instructions stored in memory to implement the method described in the foregoing method embodiments.

[0231] It should be understood that processor 401 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor. Memory 402 may include high-speed random access memory (RAM), and may also include non-volatile memory (NVM), such as at least one disk storage device, or a USB flash drive, external hard drive, read-only memory, disk, or optical disc, etc.

[0232] Optionally, the electronic device 400 may also include a communication interface 403. In specific implementations, if the communication interface 403, memory 402, and processor 401 are implemented independently, they can be interconnected via a bus to complete communication. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc., but this does not imply that there is only one bus or one type of bus.

[0233] Optionally, in a specific implementation, if the communication interface 403, memory 402 and processor 401 are integrated on a single chip, then the communication interface 403, memory 402 and processor 401 can communicate through an internal interface.

[0234] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed, are used to implement the methods described in any of the foregoing embodiments.

[0235] It is understood that the computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0236] An exemplary computer-readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the computer-readable storage medium. Of course, the computer-readable storage medium can also be a component of the processor. The processor and the computer-readable storage medium can reside in an ASIC. Alternatively, the processor and the computer-readable storage medium can exist as discrete components in an electronic device.

[0237] The integrated modules implemented as software functional modules described above can be stored in a computer-readable storage medium. These software functional modules, stored in a computer-readable storage medium, include several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application.

[0238] This application also provides a computer program product, including a computer program that, when executed, implements the method described in any of the foregoing embodiments.

[0239] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0240] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0241] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0242] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0243] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for operation scheduling of a combined energy storage power system, characterized in that, The method comprises the following steps: obtaining the operation state of the joint energy storage power system; based on the operation state, selecting a candidate scheduling action that meets the operation state from the preset scheduling action; inputting the operation state into the front-end network of the pre-trained operation scheduling generation model for feature extraction to obtain high-dimensional features of the operation state, wherein the front-end network comprises a network unit for extracting high-dimensional features of the input operation state; inputting the high-dimensional features into the flattening layer of the operation scheduling generation model for feature-to-vector conversion to obtain a one-dimensional feature vector corresponding to the high-dimensional features; inputting the one-dimensional feature vector into the value function network of the operation scheduling generation model for state estimation value calculation to obtain a state estimation value corresponding to the operation state; inputting the one-dimensional feature vector and the candidate scheduling action into the concatenation layer of the operation scheduling generation model for feature concatenation to obtain a joint feature vector; inputting the joint feature vector into the advantage function network of the operation scheduling generation model for scheduling action advantage value calculation to obtain the advantage value of the candidate scheduling action under the operation state; inputting the state estimation value and the advantage value of the candidate scheduling action into the output layer of the operation scheduling generation model for summation calculation to obtain the predicted action value of the candidate scheduling action under the operation state, wherein the operation scheduling generation model is an online Q network trained based on a double Q network. Among the selected multiple candidate scheduling actions, the candidate scheduling action with the maximum predicted action value is used to schedule the operation of the joint energy storage power system.

2. The method of claim 1, wherein, The pre-trained operation scheduling generation model is obtained by the following method: obtaining historical operation information of the joint energy storage power system, wherein the historical operation information comprises the operation state of the joint energy storage power system at t-1 time, the scheduling action executed at t-1 time, the immediate reward obtained by executing the scheduling action, and the operation state at t time; generating a sample data set containing Markov decision process (MDP) data units based on the historical operation information, wherein the MDP data unit comprises the operation state at t-1 time, the scheduling action executed at t-1 time, the immediate reward obtained by executing the scheduling action corresponding to the operation state at t-1 time, and the operation state at t time; training the operation scheduling generation model based on the sample data set to obtain the pre-trained operation scheduling generation model.

3. The method of claim 2, wherein, The method of generating a sample data set containing Markov decision process (MDP) data units based on the historical operation information comprises: constructing an objective function for the operation scheduling optimization of the joint energy storage power system and defining a constraint condition set of the objective function, wherein the constraint condition set comprises power plant output constraint conditions and energy storage power station output constraint conditions; selecting a target scheduling action that meets the constraint conditions from the executed scheduling actions based on the constraint condition set; Based on the target function, the immediate reward of each target scheduling action is calculated, wherein the immediate reward is related to the optimization target of the target function; The running state at the t-1 time, the running state at the t time, each target scheduling action and the immediate reward of each target scheduling action are encoded into an MDP data unit; A plurality of MDP data units are aggregated to generate a sample data pool; A set number of MDP data units are extracted from the sample data pool to generate a sample data set.

4. The method of claim 3, wherein, The extraction of a set number of MDP data units from the sample data pool to generate a sample data set includes: A set number of MDP data units are randomly extracted from the sample data pool to generate a sample data set; Or, The priority of each MDP data unit in the sample data pool is determined; According to the priority, a set number of MDP data units are extracted from the sample data pool to generate a sample data set.

5. The method of claim 3, wherein, The double Q network includes an online Q network and a target Q network, the online Q network and the target Q network have the same network architecture, and the training of the operation scheduling generation model according to the sample data set includes: The running state at the t-1 time is input into the online Q network to determine the predicted action value corresponding to each target scheduling action; The running state at the t time is input into the target Q network, and the output value of the target Q network is determined according to the immediate reward, and the output value is a target action value; The mean square error loss of the predicted action value and the target action value is calculated, and the parameters of the online Q network are updated by an optimization algorithm to minimize the mean square error loss; Based on the preset training end condition, the pre-trained operation scheduling generation model is obtained according to the updated parameters of the online Q network.

6. The method of claim 5, wherein, The pre-trained operation scheduling generation model obtained based on the preset training end condition and the updated parameters of the online Q network includes: Determine whether the number of updates of the parameters of the online Q network reaches a number threshold; When the number of updates of the parameters of the online Q network reaches the number threshold, the parameters of the online Q network are synchronized to the target Q network based on an update strategy; Determine whether the online Q network is in a convergent state; When the online Q network is in a convergent state, the current online Q network is used as a pre-trained operation scheduling generation model; When the online Q network is not in a convergent state, continue to iteratively train until the online Q network is in a convergent state.

7. The method of claim 6, wherein, When the number of updates of the parameters of the online Q network reaches the number threshold, the parameters of the online Q network are synchronized to the target Q network based on an update strategy, which includes: When the number of updates of the parameters of the online Q network reaches the number threshold, the parameters of the online Q network are copied to the target Q network; Or, When the number of times of updating the parameters of the online Q network reaches a number threshold, the parameters of the online Q network and the parameters of the current target Q network are weighted and summed according to a set weight coefficient to obtain the parameters of the target Q network.

8. An operation scheduling device of a combined energy storage power system, the operation scheduling device of the combined energy storage power system being used to implement the operation scheduling method of the combined energy storage power system according to any one of claims 1-7, characterized in that, The method comprises the steps of: An acquisition module is configured to acquire an operating state of the joint energy storage power system. A selection module is configured to select, based on the operating state, a candidate scheduling action that meets the operating state from preset scheduling actions. A determination module is configured to input the operating state and the candidate scheduling action into a pre-trained operation scheduling generation model to obtain a predicted action value of executing the candidate scheduling action under the operating state, the operation scheduling generation model being an online Q network trained based on a double Q network. A scheduling module is configured to schedule operation of the joint energy storage power system according to a candidate scheduling action with the largest predicted action value among the selected candidate scheduling actions.

9. An electronic device, comprising: The method comprises the steps of: A memory and a processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory, so that the processor executes the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the method according to any one of claims 1-7.

11. A computer program product, characterised in that, The computer program is executed by the processor to implement the method according to any one of claims 1-7.

Images