Energy storage system control method and equipment based on dynamic electricity price and medium

Abstract

The embodiment of the invention discloses an energy storage system control method and device based on dynamic electricity price and a medium, and relates to the field of energy storage system control. The method comprises the following steps: acquiring the time-of-use electricity price of at least one time period in the future and the power load of the energy storage system; according to the time-of-use electricity price and the power load, the charging power and the discharging power of the energy storage module in each time period in the future and the electricity selling power and the electricity purchasing power of the power grid interaction module are optimized, so that the total cost of the energy storage system is lowest; wherein the constraint condition in the optimization process comprises that the sum of the power grid exchange power, the photovoltaic power generation power and the wind power generation power is equal to the sum of the charging and discharging power of the energy storage module and the power load. According to the embodiment, the system efficiency and the energy utilization capacity are improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of energy storage system control, and in particular to a method, device and medium for controlling an energy storage system based on dynamic electricity prices. Background Art

[0002] An energy storage system refers to a system that can store energy from the power grid. It is generally managed based on a fixed electricity price strategy, that is, the charging and discharging operations of the energy storage system are determined based on a preset fixed electricity price curve.

[0003] Existing energy storage system control methods usually do not take into account real-time changes in electricity prices, and lack effective coordination of different power sources (such as wind power and photovoltaics), resulting in low overall system efficiency and limited effective use of energy. Summary of the invention

[0004] The embodiments of the present invention provide a method, device and medium for controlling an energy storage system based on dynamic electricity price to solve the above technical problems.

[0005] In a first aspect, an embodiment of the present invention provides an energy storage system control method based on dynamic electricity prices, which is applied to an energy storage system including a photovoltaic power generation module, a wind power generation module, an energy storage module, and a grid exchange module;

[0006] The method comprises:

[0007] Obtaining a time-of-use electricity price for at least one future period and the power load of the energy storage system;

[0008] According to the time-of-use electricity price and power load, the charging power and discharging power of the energy storage module in each future period, as well as the power selling power and power purchasing power of the grid interaction module are optimized to minimize the total cost of the energy storage system;

[0009] Among them, the constraints in the optimization process include: the sum of the grid exchange power, photovoltaic power generation power and wind power generation power is equal to the sum of the charging and discharging power of the energy storage module and the power load.

[0010] In a second aspect, an embodiment of the present invention provides an electronic device, the electronic device comprising:

[0011] one or more processors;

[0012] a memory for storing one or more programs,

[0013] When the one or more programs are executed by the one or more processors, the one or more processors implement the energy storage system control method based on dynamic electricity price described in any embodiment.

[0014] In a third aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the energy storage system control method based on dynamic electricity prices described in any embodiment.

[0015] In summary, the embodiment of the present invention provides a method for controlling an energy storage system based on dynamic electricity prices. By introducing a dynamic electricity price control mechanism, the charging and discharging strategy of the energy storage system is optimized, the overall operating cost of the system is minimized, and the efficiency and energy utilization capacity of the energy storage system are improved. The method establishes detailed power balance conditions, which can be combined with constraints such as charging and discharging restrictions and SOC restrictions to ensure the stable operation of the system. At the same time, it can dynamically adjust the power purchase and sales strategies to adapt to real-time electricity price fluctuations; the system can coordinate wind power, photovoltaic power generation and energy storage systems to maximize the utilization rate of renewable energy; it also takes into account the dynamic changes in power loads, improves the flexibility and response speed of the system, and meets sudden power demands. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 is a flow chart of a method for controlling an energy storage system based on dynamic electricity prices provided by an embodiment of the present invention;

[0018] Figure 2 It is a structural diagram of an Actor-Critic network provided by an embodiment of the present invention;

[0019] Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0021] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0022] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0023] The embodiment of the present invention provides a method for controlling an energy storage system based on a dynamic electricity price. In order to illustrate the method, the energy storage system to which the method is applied is introduced first. Optionally, the energy storage system of the embodiment of the present invention includes an energy storage module, a photovoltaic power generation module, a wind power generation module and a grid exchange module. Among them, the photovoltaic power generation module is used to access photovoltaic power, the wind power generation module is used to access wind power, and the grid exchange module is used to exchange power with the trading grid. The energy storage module may include a battery for storing surplus photovoltaic power, wind power, and power purchased from the trading grid in the system, and the stored power may also be sold to the trading grid. In addition, the energy storage system may also be connected to electrical loads, such as industrial charging piles, etc. The electrical loads will consume the power of the energy storage system, and the consumed power may come from at least one of the photovoltaic power generation module, the wind power generation module and the energy storage module.

[0024] For the above energy storage systems, Figure 1 is a flow chart of a method for controlling an energy storage system based on dynamic electricity prices provided by an embodiment of the present invention. The method dynamically controls the charging and discharging power of the energy storage module according to the real-time electricity price to minimize the cost of the entire energy storage system and achieve real-time response capabilities. The method is executed by an electronic device, such as Figure 1 As shown, specifically including:

[0025] S110, obtaining a time-of-use electricity price for at least one future period and the power load of the energy storage system.

[0026] The time-of-use electricity price can be derived from the electricity trading market. For example, the European electricity trading market will announce the electricity price for the next 24 hours at 14:00 every day. The power load is a prediction of the amount of electricity consumed by the power load in the energy storage system. Optionally, the power load of the same period of the previous day can be used as the power load of the future period. This embodiment periodically obtains these parameters as the data source of the entire method.

[0027] S120. According to the time-of-use electricity price and power load, the charging power and discharging power of the battery in each future period, as well as the power selling power and power purchasing power of the grid interaction module are optimized to minimize the total cost of the energy storage system. The constraints of the optimization process include: the sum of the grid exchange power, photovoltaic power generation power and wind power generation power is equal to the sum of the battery power and the power load, that is, the power balance constraint.

[0028] This embodiment comprehensively considers the real-time dynamic electricity price, as well as the predicted values ​​of power load, photovoltaic power generation and wind power generation, and optimizes the charging and discharging strategy of the energy storage system. The optimized strategy enables the energy storage system to purchase and store electricity in periods with lower electricity prices while meeting its own power load, and sell the stored electricity in periods with higher electricity prices, so as to reduce the total cost of the entire system. Optionally, the charging and discharging strategy can be expressed as the values ​​of the charging power and discharging power of the battery, and the power selling power and power purchasing power of the grid interaction module in each future period. The values ​​of the four powers are arranged in chronological order, so as to obtain the power curve of the battery and the charging and discharging strategy, which is used to guide the charging and discharging actions of the battery and the charging and discharging strategy in multiple future periods.

[0029] In a specific implementation, the constraints of the optimization process can be constructed from the following aspects:

[0030] 1) Battery restraint type:

[0031]

[0032] P t cha ≤P t cs max

[0033] P t dis ≤P t cs max (2)

[0034] Among them, P t cs maxt Indicates the rated power of the battery, P t bat Represents the battery power during period t, P t chaand P t dis They represent the battery charging power and battery discharging power in period t, respectively. t cs max , P t cha and P t dis are all non-negative numbers, and P t cha and P t dis At most one of them is non-zero.

[0035] At the same time, the capacity of the battery meets:

[0036] soc=E t / E max (3)

[0037] E t =E t-1 +P t cha -P t dis (4)

[0038] Among them, SOC represents the charging state of the battery in time period t, which can be read through the battery manager; E max Indicates the rated capacity of the battery, E t and E t-1 They represent the remaining capacity of the battery in time period t and time period t-1 respectively. In formula (4), the unit time for calculating power is taken as a time period.

[0039] Divide both sides of formula (4) by E max , we can get:

[0040]

[0041] and then

[0042] E max SOC = E max ·SOC t-1 +P t cha -P t dis (6)

[0043] The SOC itself needs to meet the following requirements:

[0044] SOC min ≤SOC≤SOC max (7)

[0045] Among them, SOC min and SOCmax The minimum SOC value and the maximum SOC value that the battery should meet for normal operation are respectively represented. Based on formula (7), the at least one time period is accumulated and summed, and the initial energy storage capacity SOC of the first time period is considered. 0 And formula (6), we can get:

[0046] E max (SOC min -SOC 0 ))≤sum(P t cha -P t dis )≤E max (SOC max -SOC 0 ) (8)

[0047] In addition, t in the above variables can also be understood as time t, and formula (8) can be understood as summing up at each time. The following variables are similar and will not be repeated here.

[0048] 2) Grid exchange constraints:

[0049]

[0050] P t buy ≤P t net max

[0051] P t sell ≤P t net max (10)

[0052] Among them, P t net represents the grid exchange power during period t, P t buy and P t sell They represent the power purchased and sold from the grid during period t, P t net max Indicates the maximum exchange power of the power grid. Similarly, P t net max , P t buy and P t sell are all non-negative numbers, and P t buy and P t sell At most one of them is non-zero. Formula (9) has a certain approximation, assuming that when the battery is charged (P tbat Greater than or equal to 0) means purchasing electricity; in the battery discharge state (P t bat Less than 0) means electricity is sold. This approximation will only slightly prolong the convergence time of the optimization process and will not affect the optimization results.

[0053] 3) Power balance constraints:

[0054]

[0055] in, and They represent the wind power generation and photovoltaic power generation in period t respectively, Represents the power load during time period t. The left side of the equation is the power supply of the energy storage system, and the right side is the power consumption of the energy storage system, power supply = power consumption.

[0056] 4) Objective function: Minimum total cost

[0057]

[0058] Among them, F represents the total cost of T time periods, s w 、s pv 、s buy and ess Represent the unit costs of wind power generation, photovoltaic power generation, power purchase, power sales and battery discharge, s sell Represents the unit revenue of electricity sales, which is a fixed constant. buy and sell This is the real-time electricity price during the current period.

[0059] In summary, the overall model is:

[0060] min F

[0061] st(1)-(12)

[0062] In order to solve the optimization problem of the above model, this embodiment provides the following two optional implementations according to the changes in wind turbine output and photovoltaic output:

[0063] The first optional implementation method is suitable for the case where the wind turbine power generation and photovoltaic power generation are relatively stable. and If the system is relatively stable over a long period of time, such as 24 hours or longer, then at the beginning of the stable period, an exhaustive algorithm can be used to traverse each set of solutions in the solution space, and the optimal solution that meets the constraints can be selected as the control strategy for the stable period. Optionally, a recursive algorithm function or an exhaustive algorithm function in the Java language can be called, and the above objective function min F and all constraints are input into the function as known conditions, and the function automatically returns the optimal strategy that meets the conditions. This method has a large amount of calculation, and can be used continuously for a long period of time after one calculation. Therefore, it can be applied when the wind turbine power generation and photovoltaic power generation are relatively stable.

[0064] The second optional implementation is suitable for situations where the wind turbine power generation and photovoltaic power generation are relatively frequent, for example, due to frequent changes in light and / or wind power. and / or The energy storage system changes every hour, or fluctuates at a faster frequency. In this case, if the exhaustive algorithm is executed every time a change occurs, it will cause strategy lag and waste computing resources. Therefore, the Actor-Critic algorithm based on reinforcement learning can be used to dynamically predict the optimal strategy for the next cycle based on the current status and historical strategies of the energy storage system. Optionally, the duration of the at least one period (T periods) can be used as an optimization cycle, which can also be based on and The change frequency is determined by T=2. The following will be described as an example.

[0065] In a specific implementation, first, the action variables and state variables of the Actor-Critic network are constructed. Optionally, the charging power and discharging power of the battery, and the distribution of the power selling power and power purchasing power of the grid interaction module in a certain optimization period can be used as action variables:

[0066]

[0067] Among them, a N represents the action of optimization cycle N, t and t+1 are two time periods in optimization cycle N. In order to reduce the amount of calculation, the action space can be discretized to obtain candidate actions for each discretization. Specifically, the action space is determined by the rated power of the battery and the maximum exchange power of the power grid, as shown in formula (2) and formula (9). The four power variables in the action variable can be and Discretize them at certain intervals, combine the discrete values ​​of each power in each period of an optimization cycle one by one, and then exclude the values ​​that do not meet the "P t buy and P tsell At most one of the P values ​​in the same period is non-zero. t cha and P t dis There is at most one non-zero constraint in the combination (e.g., excluding P t buy ×P t sell Not equal to 0, or P t cha ×P t dis Not equal to 0), we can get each discrete candidate action.

[0068] At the same time, the initial SOC value of the battery, time-of-use electricity price, power load, photovoltaic power generation and wind power generation of an optimization cycle, as well as the action, time-of-use electricity price, power load, photovoltaic power generation and wind power generation of the previous optimization cycle are arranged in sequence and used as the state variables of the optimization cycle. Taking the optimization cycle N as an example, the S N Includes: Initial SOC value Time-of-use electricity price Power load Photovoltaic power generation and wind power generation And the action a of the optimization period N-1 N-1 , Time-of-use electricity price Power load Photovoltaic power generation and wind power generation Wherein, t-2 and t-1 are two time periods in the optimization cycle N-1 respectively.

[0069] Based on the above action variables and state variables, we can construct Figure 2 The Actor-Critic network shown. When this network is used, the state S of the current optimization cycle N can be N Input the Actor network, and the Actor network outputs the probability of selecting each candidate action in the cycle, and the candidate action with the highest probability is taken as the optimal action. Then, the reward value R is calculated according to formula (14), and the Critic network evaluates the value of the candidate action based on the reward value, and updates the parameters of the Actor network and the Critic network based on the value:

[0070]

[0071] Among them, F N and F N-1 Respectively represent the total cost of the current optimization cycle N and the previous optimization cycle N-1; -A means less than -F maxA fixed negative constant, F max represents the maximum fee threshold, F max >0.

[0072] Specifically, the parameters of the Actor network are updated according to the following formula:

[0073]

[0074] in, and They represent the network parameters of the Actor network in the optimization cycle N+1 and cycle N, α represents the learning rate, and π(a N |s N ) means in state s N Select each action a N The probability vector, A(s N ,a N ) indicates state s N and action a N When the advantage value is positive, it means that the current action is feasible, and its probability of being selected is gradually increased through gradient ascent. On the contrary, when the advantage value is negative, the action probability will decrease. A(s N ,a N ) can be obtained by:

[0075] A(s N ,a N )=Q(s N ,a N )-V(s N ) (18)

[0076] Among them, Q(s N ,a N ) represents the state action value, that is, s N with a N The value of the combination; V represents the state value. The Critic network can output V or Q(s) for each action. N ,a N ), which is not specifically limited in this embodiment. Optionally, according to the Bellman equation, the action value Q function can be approximately expressed as:

[0077] Q(s N ,a N )≈r N +γV(s N+1 ) (19)

[0078] Among them, r N Represents the reward value of the current cycle, and γ represents the discount rate, which is a fixed constant.

[0079] Loss function of the Critic network It can be expressed as:

[0080]

[0081] Critic will update the network parameters according to the loss function. The above parameter update process is all existing technology, and more details will not be repeated. In the early stage of network use, the Actor network cannot well screen out the optimal action that minimizes the total cost of the system, but as the network parameters are updated, the Actor network will gradually tend to the action with the smallest total cost, and finally realize the dynamic real-time optimal control of the energy storage system.

[0082] Furthermore, it can be seen that the number of candidate actions in the action space is still relatively large, such as Figure 2 As shown, the number of output nodes of the Actor network is the same as the number of candidate actions, so the amount of computation within the network is still very large, especially when T takes a larger value, the number of candidate actions will increase exponentially. In order to speed up the learning rate of the agent, this embodiment can also further reduce the size of the action space according to the power variation law of the energy storage system and the input-output relationship unique to the Actor-Critic network of this embodiment, so as to reduce the amount of computation of the agent and the complexity of the Actor network.

[0083] In a specific implementation, each candidate action variable can be clustered into multiple clusters based on action similarity. Optionally, each candidate action in the original action space can be arranged in a vector form, and the distance between the candidate actions can be measured by vector similarity; the K-Means clustering algorithm is used to cluster all candidate actions, and candidate actions with closer distances will be clustered into one category.

[0084] Then, according to the change rate threshold of each power in the action variable, determine the action range that each candidate action in the cluster can achieve in the next optimization cycle, and use the actions within each action range as the final multiple candidate actions of each cluster. In the power system, due to the limitations of the device itself or the consideration of stable operation of the system, the input and output power change rates of each module have certain threshold limits, and excessive power mutations should not occur (except for the start or stop state). Therefore, the four power change rate thresholds Pm in the action variable can be obtained respectively, such as Pm = 10% × Pe / min, where Pe represents the rated power of a certain power, and 10% × Pe / min means that the output power change per minute reaches 10% of the rated output power. According to this threshold, for any power, the value in the current time period is P t In the case of t -Pm×⊿t,P t+Pm×⊿t] range, where ⊿t represents the duration of an optimization cycle, the discrete power that meets this range can be used together with the 0 power (corresponding to the next time period stop) and the start power (corresponding to the next time period start) as P t The power value can be reached in the next period. Correspondingly, for any cluster, all candidate actions in the cluster are similar, so the value ranges of the four powers are also relatively concentrated. Assuming that the value range of a certain power is [a, b], [a-Pm×⊿t, b+Pm×⊿t], as well as the set consisting of 0 power and startup power, can be determined as the power range that all candidate actions in the cluster can reach in the next cycle; if the power value range includes multiple intervals [a, b], the interval [a-Pm×⊿t, b+Pm×⊿t] can be determined for each [a, b], and then the union of all intervals is taken, together with 0 power and startup power, to form the power range that can be reached in the next cycle. After performing the above operations on the four powers respectively, the action subspace of the cluster in the next cycle can be obtained from the four power ranges. Of course, the action subspace of each cluster can also be determined by other methods, as long as the power change rate constraint is met, and each action subspace will be much smaller than the original action space.

[0085] After clustering is completed, an Actor network can be constructed for each cluster to make decisions based on the status and actions of each cluster. According to the difference in the Critic network of each cluster, this embodiment provides two optional implementations:

[0086] Optional implementation method A: construct an Actor-Critic network for each cluster, so that the number of output nodes of the Actor network is equal to the size of the action subspace of the corresponding cluster. The number of output nodes of the Critic network can be 1, or it can be equal to the size X of the action subspace of the corresponding cluster. When the number of output nodes of the Critic network is 1, the Critic network inputs the new state brought by the current optimal action, and outputs the value V of the new state, such as Figure 2 As shown, the action state value corresponding to the optimal action is Q = R + γV. When the number of output nodes of the Critic network is X( Figure 2 (not shown in the figure), the Critic network inputs the new state and reward value brought by each candidate action in the subspace, and outputs the value Q of each candidate action combined with the current state.

[0087] Correspondingly, the Actor network and Critic network of each cluster are updated separately. The state and action of a cluster will only update the network parameters of the cluster, and will not affect the network parameters of other clusters. Optionally, when a certain optimization cycle arrives, the target cluster to which the state of the current optimization cycle belongs can be first determined, and the state is input into the Actor network of the target cluster. The Actor network of the target cluster outputs the probability of selecting each candidate action in the action subspace of the target cluster in the cycle, and the action with the highest probability is selected as the optimal action. Then, the reward of the optimal action is calculated according to formula (14), or the reward of all candidate actions in the action subspace of this cluster is calculated; the Critic network of the target cluster evaluates the value of the combination of the optimal action and the current state according to the reward value, or evaluates the value of the combination of all actions in the action subspace of this cluster and the current state respectively, and updates the parameters of the Actor network and Critic network of the target cluster according to the value.

[0088] In this way, the network structure of each cluster is relatively streamlined, the amount of calculation is relatively small, and each cluster network performs training, decision-making and evaluation separately. The time required for stabilization of each Actor network training is longer than that of optional implementation method B.

[0089] Optional implementation method B: construct an Actor network for each cluster, each Actor network of each cluster is independent, and the number of output nodes of each Actor network is equal to the size of the action subspace of the corresponding cluster. At the same time, each cluster shares a Critic network, and the number of output nodes of the Critic network is equal to the size of the union of the action subspaces of all clusters. The input of the Critic network is the new state and reward value brought by each candidate action in the union, and the output is the value Q of the combination of each candidate action and the current action.

[0090] Correspondingly, the Actor network of each cluster is updated separately, and the Critic network is also updated after each Actor network is updated. Optionally, when a certain optimization cycle arrives, the target cluster to which the state of the current optimization cycle belongs can be first determined; the state is input into the Actor network of the target cluster, and the Actor network of the target cluster outputs the probability of selecting each candidate action in the action subspace of the target cluster within the cycle, and the action with the highest probability is selected as the optimal action. Then, the reward value R of each candidate action in the action subspace of the target cluster is calculated according to formula (14), and the reward value of the remaining actions in the union is set to a negative constant less than or equal to -A; the Critic network evaluates the value Q of each action and state combination in the union according to each reward value; and according to the Q corresponding to each action in the action subspace of the target cluster, the Actor network of the target cluster is updated; according to the Q corresponding to each action in the union, the parameters of the Critic network are updated.

[0091] In this way, the Actor network structure is relatively simple and the amount of calculation is relatively small; the Critic network is also simpler than the Critic network that outputs the original action space, but it is more complex than the network structure of optional implementation mode A. The Critic network of this embodiment evaluates the action state of all actions in the union of each cluster action subspace, and the decision of each Actor network will cause the update of the Critic network, which helps the Critic network to more quickly and objectively evaluate the action state value and state value, thereby promoting the training stability of the Actor network.

[0092] The above two optional implementation modes have improved the Actor-Critic network according to the power variation characteristics of the power system, reduced the number of output nodes and the amount of calculation of the network, and can be flexibly selected as needed in practical applications.

[0093] In addition, in order to ensure the stability of power changes in the power system, whether it is the Actor-Critic network of the original action space, the Actor-Critic network of each cluster, or the network structure in which each cluster has an independent Actor network and a shared Critic network, after the Actor network outputs the probability of selecting each candidate action within a certain period, a mask with the same size as the output action space can be constructed, and each mask element corresponds to each candidate action. Specifically, the mask size is different in the above three Actor-Critic networks, and can be the same as the actual number of output nodes of the network.

[0094] Then, according to the change rate threshold of each power in the action variable, determine the multiple actions that can be achieved in the current optimization cycle for the action of the previous optimization cycle, and set the mask elements corresponding to the multiple actions to 1, and the mask elements of other actions to 0. Optionally, the power of the last period of the previous optimization cycle is P t In the case of , the power range that can be achieved in the first period of the current optimization cycle is [P t -Pm×⊿t,P t +Pm×⊿t], the maximum power range that can be achieved in the next period is [P t -Pm×2⊿t,P t +Pm×2⊿t] (it can also be a part of the range, subject to the actual situation), ..., and so on, the power range of each period can be gradually determined, and finally the zero power and the starting power are added to get the action range that the action of the previous cycle can reach in this cycle. This range is smaller than the action subspace described by the current cluster. The mask elements corresponding to the actions within the range are set to 1, and the mask elements corresponding to the actions outside the range are set to 0.

[0095] Finally, the probability of each candidate action is multiplied by the corresponding mask element to obtain the final probability of each candidate action; the candidate action with the highest final probability is taken as the optimal action. Optionally, the mask can be omitted and the candidate action with the highest probability can be directly started from the candidate action, and it can be judged in turn whether it belongs to the action range that the action of the previous cycle can reach in this cycle. The first candidate action that meets the range is the optimal action.

[0096] In summary, this embodiment provides a method for controlling an energy storage system based on dynamic electricity prices. By introducing a dynamic electricity price control mechanism, the charging and discharging strategy of the energy storage system is optimized, the overall operating cost of the system is minimized, and the efficiency and energy utilization capacity of the energy storage system are improved. The method establishes detailed constraints such as power balance, charging and discharging restrictions, and SOC restrictions to ensure the stable operation of the system. At the same time, it can dynamically adjust the power purchase and sales strategies to adapt to real-time electricity price fluctuations; the system can coordinate wind power, photovoltaic power generation, and energy storage systems to maximize the utilization rate of renewable energy; it also takes into account the dynamic changes in power loads, improves the flexibility and response speed of the system, and meets sudden power demands.

[0097] In particular, when the changes in photovoltaic and wind energy are relatively stable, multiple constraints and objective functions can be established to perform global optimization for relatively long periods of time to ensure the high efficiency and economy of system operation. When the changes in photovoltaic and wind energy are relatively frequent, the control strategy for each shorter period can be adaptively updated by the intelligent agent to further improve the intelligent dispatching level of the system; in the process of constructing the intelligent agent, this embodiment provides multiple forms of Actor-Critic networks and matching training methods based on the power stability requirements of the power system and the specific input and output relationship, and provides output masks to further constrain the power change rate, so as to jointly achieve the optimal stable control of the energy storage system.

[0098] Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention is shown in FIG. Figure 3 As shown, the device includes a processor 60, a memory 61, an input device 62 and an output device 63; the number of processors 60 in the device can be one or more. Figure 3 A processor 60 is taken as an example; the processor 60, the memory 61, the input device 62 and the output device 63 in the device can be connected by a bus or other means. Figure 3 The example of connecting through bus is taken in the following.

[0099] The memory 61 is a computer-readable storage medium that can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the energy storage system control method based on dynamic electricity prices in the embodiment of the present invention. The processor 60 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 61, that is, realizing the above-mentioned energy storage system control method based on dynamic electricity prices.

[0100] The memory 61 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 61 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 61 may further include a memory remotely arranged relative to the processor 60, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0101] The input device 62 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 63 may include a display device such as a display screen.

[0102] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the energy storage system control method based on dynamic electricity price of any embodiment.

[0103] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, a device or a device or used in combination with it.

[0104] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0105] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0106] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A method for controlling an energy storage system based on dynamic electricity prices, characterized in that: Applicable to energy storage systems including photovoltaic power generation modules, wind power generation modules, energy storage modules and grid exchange modules; The method comprises: Obtaining a time-of-use electricity price for at least one future period and the power load of the energy storage system; According to the time-of-use electricity price and power load, the charging power and discharging power of the energy storage module in each future period, as well as the power selling power and power purchasing power of the grid interaction module are optimized to minimize the total cost of the energy storage system; Among them, the constraints in the optimization process include: the sum of the grid exchange power, photovoltaic power generation power and wind power generation power is equal to the sum of the charging and discharging power of the energy storage module and the power load.

2. The method according to claim 1, characterized in that The energy storage module includes a battery; The constraints of the optimization process include: P t cha ≤P t cs max P t dis ≤P t cs max (2) From max (SOC min -SOC 0 )≤sum(P t hat -P t dis )≤E max (SOC max -SOC 0 ) (8) P t buy ≤P t net max P t sell ≤P t net max (10) Among them, P t bat Represents the battery power during period t, P t cha and P t dis They represent the battery charging power and battery discharging power in period t, respectively. t cs max Indicates the rated power of the battery; SOC 0 , SOC min and SOC max They represent the initial SOC, minimum SOC and maximum SOC of the battery respectively, and E max Indicates the rated capacity of the battery; P t net represents the grid exchange power during period t, P t buy and P t sell They represent the power purchased and sold from the grid during period t, P t net max Indicates the maximum exchange power of the power grid; represents the power load in period t; F represents the total cost in period T, and They represent the wind power generation and photovoltaic power generation in period t, s w 、s pv 、s buy and ess Represent the unit costs of wind power generation, photovoltaic power generation, electricity purchase and battery discharge, s sell Represents the unit revenue of electricity sales.

3. The method according to claim 2, characterized in that The charging power and discharging power of the battery in each future period, and the selling power and purchasing power of the grid interaction module are optimized according to the time-of-use electricity price and the power load, so that the total cost of the energy storage system is minimized, including: According to constraints (2) and (9), generate a solution space of the charging power and discharging power of the battery in each future period, and the power selling power and power purchasing power of the grid interaction module; An exhaustive algorithm is used to find the optimal solution that meets the constraints from the solution space.

4. The method according to claim 1, characterized in that: The energy storage module includes a battery; Accordingly, according to the time-of-use electricity price and power load, the charging power and discharging power of the battery in each future period, and the power selling power and power purchasing power of the grid interaction module are optimized to minimize the total cost of the energy storage system, including: Taking the duration of the at least one time period as an optimization period; The charging power and discharging power of the battery, and the distribution of the power selling power and power purchasing power of the grid interaction module in an optimization cycle are used as action variables; The initial SOC of the battery, time-of-use electricity price, power load, photovoltaic power generation and wind power generation of an optimization cycle, as well as the action, time-of-use electricity price, power load, photovoltaic power generation and wind power generation of the previous optimization cycle are taken as state variables; Determine an action space according to the rated power of the battery and the maximum exchange power of the power grid, and discretize the action space to obtain candidate actions; Construct an Actor-Critic network and input the state of the current optimization cycle into the Actor network. The Actor network outputs the probability of selecting each candidate action in the cycle, and selects the candidate action with the highest probability as the optimal action. The reward value R is calculated according to formula (14), and the Critic network evaluates the value of the state action according to the reward value, and updates the parameters of the Actor network and the Critic network according to the value: C1: E max (SOC min -SOC 0 )≤sum(P t cha -P t dis )≤E max (SOC max -SOC 0 ) C2： Among them, F N and F N-1 Respectively represent the total cost of the current optimization cycle and the previous optimization cycle; T represents the number of time periods included in each optimization cycle, and They represent the wind power generation and photovoltaic power generation in period t, respectively, t buy and P t sell They represent the power purchased and sold from the grid during period t, P t cha and P t dis They represent the battery charging power and battery discharging power in period t, s w 、s pv 、s buy and ess Represent the unit costs of wind power generation, photovoltaic power generation, power purchase, power sales and battery discharge, s sell represents the revenue from electricity sales, P t net represents the grid exchange power during period t, represents the power load during period t, P t bat Indicates the battery power during period t; SOC min , SOC max and SOC 0 They represent the minimum SOC, maximum SOC, and initial SOC of the battery in the current optimization cycle respectively; E max Indicates the rated capacity of the battery; -A means less than -F max A fixed negative constant, F max represents the maximum fee threshold, F max >0; α and β represent preset weights between [0,1].

5. The method according to claim 4, characterized in that The step of discretizing the action space to obtain candidate actions includes: Discretize each power in the action variable at a certain interval, and combine the discretized values ​​of each power in each time period of an optimization cycle into candidate actions; According to the action similarity, each candidate action variable is clustered into multiple clusters; According to the change rate threshold of each power in the action variable, multiple actions that can be achieved by each candidate action in the cluster in the next optimization cycle are determined as the final multiple candidate actions of each cluster.

6. The method according to claim 5, characterized in that The method of constructing an Actor-Critic network and inputting the state of the current optimization cycle into the Actor network, and having the Actor network output the probability of selecting each candidate action within the cycle, comprises: constructing an Actor-Critic network for each cluster, respectively, so that the number of output nodes of each Actor network is equal to the final number of candidate actions of the corresponding cluster, and the number of output nodes of each Critic network is equal to 1 or equal to the final number of candidate actions of the corresponding cluster; determining the target cluster to which the state of the current optimization cycle belongs; inputting the state into the Actor network of the target cluster, and having the Actor network of the target cluster output the probability of selecting each final candidate action of the target cluster within the cycle; The Critic network evaluates the value of the state action according to the reward value, and updates the parameters of the Actor network and the Critic network according to the value, including: the Critic network of the target cluster evaluates the value of the final state and action combination corresponding to the target cluster according to the reward value, and updates the parameters of the Actor network and the Critic network of the target cluster according to the value.

7. The method according to claim 5, characterized in that The method of constructing an Actor-Critic network and inputting the state of the current optimization cycle into the Actor network, and having the Actor network output the probability of selecting each candidate action within the cycle, includes: constructing an Actor-Critic network for each cluster, wherein the Actor network of each cluster is independent, and the number of output nodes of each Actor network is equal to the number of final candidate actions of the corresponding cluster; each cluster shares a Critic network, and the number of output nodes of the Critic network is equal to the size of the union of the final candidate actions of all clusters; determining the target cluster to which the state of the current optimization cycle belongs; inputting the state into the Actor network of the target cluster, and having the Actor network of the target cluster output the probability of selecting each final candidate action of the target cluster within the cycle; The formula (14) calculates the reward value R, the Critic network evaluates the value of the state action according to the reward value, and updates the parameters of the Actor network and the Critic network according to the value, including: calculating the reward value R of each final candidate action corresponding to the target cluster according to formula (14), and setting the reward value of the remaining actions in the union to a negative constant less than or equal to -A, and the Critic network evaluating the value of each action and state combination in the union according to each reward value; and updating the Actor network of the target cluster according to the value of each final candidate action and state combination corresponding to the target cluster, and updating the parameters of the Critic network according to the value of each action in the union.

8. The method according to claim 4, characterized in that The Actor network outputs the probability of selecting each candidate action in the cycle, and takes the candidate action with the highest probability as the optimal action, including: The Actor network outputs the probability of selecting each candidate action in this cycle; Construct a mask with the same size as the action space, where each mask element corresponds to each candidate action; According to the change rate threshold of each power in the action variable, determine multiple actions that can be achieved in the current optimization cycle for the action of the previous optimization cycle, set the mask elements corresponding to the multiple actions to 1, and set the mask elements of other actions to 0; Multiply the probability of each candidate action by the corresponding mask element to obtain the final probability of each candidate action; The final candidate action with the highest probability is taken as the optimal action.

9. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the energy storage system control method based on dynamic electricity price as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the program is executed by a processor, the energy storage system control method based on dynamic electricity price described in any one of claims 1-8 is implemented.

Images