Energy storage cluster cooperative control method and device considering adjustment cost optimization

Abstract

The invention discloses an energy storage cluster cooperative control method and device considering adjustment cost optimization. The energy storage cluster cooperative control method considering adjustment cost optimization comprises the following steps: 1, defining an active power distribution network adjustment control cost objective function containing an energy storage system; 2, defining active power distribution network safe and economic operation constraint conditions; and 3, on the basis of a reinforcement learning algorithm, node voltage and line power of the power distribution network are optimized. The distributed energy storage facility is utilized, and the network loss cost, the energy storage adjustment cost, the safety out-of-limit cost and the light abandoning and power limiting cost of the distributed energy storage facility are considered. And based on a reinforcement learning algorithm, node voltage and line power of the power distribution network are optimized, so that cost minimization of the active power distribution network under high-proportion distributed photovoltaic access is realized.

Description

Technical Field

[0001] The present invention relates to a method and device for collaborative control of an energy storage cluster taking regulation cost optimization into consideration, and belongs to the technical field of power system operation control. Background Art

[0002] With the continuous progress of human society, the consumption of fossil energy is increasing day by day, but fossil energy is difficult to regenerate, and long-term use will cause certain pollution to the environment. Therefore, it is urgent to find and utilize renewable clean energy. Among them, distributed photovoltaic power generation technology represented by solar energy utilization has become the focus of current research and application. Photovoltaic power generation technology has the advantages of low carbon and environmental protection, easy maintenance and convenient installation. At the same time, affected by solar energy resources, photovoltaic power generation output is very intermittent, random and volatile, which leads to a serious uneven distribution of network flow in time and space when a high proportion of distributed photovoltaic power generation is connected to the distribution network: the utilization rate of network equipment is not high during the valley period; and in some peak periods, there will be problems of network equipment overload or congestion. Therefore, under certain economic cost constraints, how to use energy storage cluster collaborative control to optimize the flow distribution of the distribution network and achieve efficient photovoltaic consumption and safe operation of the system has become a hot issue of current research and concern. Summary of the invention

[0003] The purpose of the present invention is to provide a method and device for collaborative control of energy storage clusters taking into account the optimization of regulation costs, taking into account the charging and discharging costs of energy storage, network loss costs, safety over-limit costs and power curtailment costs, so as to achieve the optimization of node voltage and line power of active distribution network under high proportion of distributed photovoltaic access at the best economic cost.

[0004] The purpose of the present invention is achieved through the following technical solutions:

[0005] A method for coordinated control of an energy storage cluster considering regulation cost optimization includes the following steps:

[0006] Step 1: Define the active distribution network regulation control cost objective function including energy storage system:

[0007] The regulation and control cost of the active distribution network including the energy storage system is:

[0008]

[0009] In the formula, f is the total regulation and control cost of the system, T is the regulation and control period, C1 is the network loss cost of the system, C2 is the energy storage regulation cost of the system, C3 is the safety limit penalty cost, and C4 is the cost of curtailment of solar power and power rationing; where:

[0010] (1) Network loss costs consist of line loss costs and grid equipment operation and maintenance costs, namely:

[0011]

[0012] Where P loss represents the line loss, V represents the electricity price, T represents the regulation control cycle, and C5 represents the operation and maintenance cost of the system;

[0013] (2) The energy storage regulation cost consists of the charging loss cost and the discharging loss cost of the energy storage device, namely:

[0014]

[0015] Where α and β are the energy storage charging loss cost coefficient and energy storage discharging loss cost coefficient respectively; P dis , P sto They are energy storage releasing electric energy and energy storage receiving electric energy;

[0016] (3) The penalty cost of safety over-limit of active distribution network is:

[0017]

[0018] In the formula, γ c is the penalty fee for exceeding the limit, η is the maximum load rate of the system when exceeding the limit;

[0019] (4) The expression of the cost of curtailing solar power and limiting electricity supply is:

[0020]

[0021] In the formula, γ pv Indicates the penalty cost for abandoning light; P f Represents the predicted output of photovoltaic power; P t represents the actual output of photovoltaic power; therefore, the objective function to minimize the total cost is:

[0022]

[0023] Step 2: Define the constraints for safe and economic operation of the active distribution network

[0024] (1) System energy balance constraints:

[0025] P w +P dis +P t =P L +P sto (7)

[0026] In an active distribution network, the power input includes distributed photovoltaic output energy P w , the energy released by the energy storage system P dis , the external public grid provides energy Pt , the power consumption includes the conventional power load P L and the charge capacity P of the energy storage system sto ; The energy balance constraint is shown in formula (7), which is the balance between supply and demand of electric energy;

[0027] (2) Distributed photovoltaic power generation output upper and lower limits:

[0028] P w,min ≤P w ≤P w,max (8)

[0029] (3) Energy storage system charging and discharging power constraints:

[0030] 0≤P dis ≤P dis,max (9)

[0031] 0≤P sto ≤P sto,max (10)

[0032] Where, P dis,max and P sto,max They are the upper limits of the output for discharging and charging of energy storage equipment respectively;

[0033] (4) Constraints on charging and discharging behavior of energy storage systems:

[0034] S0=S T (11)

[0035] P dis ·P sto =0 (12)

[0036] S min ≤S0+P sto -P dis ≤S max (13)

[0037] In the formula, S min With S max are the lower and upper limits of the energy storage device capacity respectively; Formula (11) constrains the storage level S of the energy storage device after one scheduling cycle T It can still return to the initial storage level S0 to prevent insufficient energy supply in the next scheduling cycle; Formula (12) limits the energy storage device from charging and discharging at the same time to prevent it from interfering with the power system frequency; Formula (13) limits the charge state of the energy storage device to avoid excessive charging and discharging that shortens the life of the equipment;

[0038] (5) Bus voltage constraints:

[0039] U min ≤U≤U max (14)

[0040] In the formula, Umin and Umax represent the lower limit and upper limit of the bus voltage respectively, and U is the bus voltage level;

[0041] (6) Line flow constraints:

[0042] 0≤U1≤U' max (15)

[0043] 0≤P1≤P max (16)

[0044] Where U1 is the node voltage, P1 is the line power; U' max , P max They represent the upper limit of line voltage and the upper limit of line power respectively;

[0045] Step 3: Optimize node voltage and line power of the distribution network based on reinforcement learning algorithm

[0046] The reinforcement learning algorithm is described by the Markov decision process, which consists of a five-tuple consisting of the state space S, the action space A, the reward function R, the action probability P and the discount factor γ;

[0047] State space S: includes the charging and discharging state of the energy storage system at a certain moment, the load demand of the distribution network, and the line flow distribution;

[0048] Action space A: includes charging and discharging actions of the energy storage system, energy dispatching actions of the distribution network, and node voltage regulation;

[0049] Reward function R: It is set as the negative value of the total cost of the active distribution network, that is, negative cost is reflected as a positive reward value:

[0050] R(s)=-f (17)

[0051] Action probability P: represents the probability of the active distribution network taking corresponding actions in a certain state, and the action probability is determined by the ε-greedy strategy;

[0052] Discount factor γ: This parameter is in the range of [0, 1] and is used to give the ability to discount rewards. That is, the larger the value of γ, the more the decision-making focuses on long-term rewards. Conversely, the decision-making focuses on immediate rewards.

[0053] The specific steps of the algorithm are as follows:

[0054] (1) Define the action-value function of reinforcement learning

[0055] Using the Q learning algorithm, the action value function Q(S t ,A t) is used to measure the quality of the actions performed in the state following the strategy; in the Q learning phase, each state-action in the distribution network corresponds to a Q value;

[0056]

[0057] Where S t is the distribution network state corresponding to time t, A t represents the action taken by the distribution network at time t, E π represents the expected value of strategy π, γ represents the discount factor, R k is the timely reward obtained after the k-th action, s0, a0 are the initial state and initial action of the distribution network; in the initial stage, the Q value is initialized to 0;

[0058] (2) Using the ε-greedy strategy to select actions

[0059] π * (s) = (1-ε)δ(a * -a)+ε / |A| (19)

[0060] In the formula, π * (s) represents the strategy of the agent to select actions under a given distribution network state S; δ is the Kronecker function, which is equal to 1 when its parameters are equal and 0 otherwise; a* is the action that maximizes the reward value under state S; |A| is the total number of actions in the action set A; δ(a*-a) is 1 when a=a* and 0 otherwise, indicating that the optimal action a* is selected with probability 1-ε; ε / |A| means uniformly randomly selecting any action in the action set A with probability ε;

[0061] (3) Update the Q value iteratively:

[0062] The update iteration rules of Q learning are as follows:

[0063] Q(S t ,A t )=Q(S t ,A t )+α[R t+1 +γmax a' Q(S t+1 ,a')-Q(S t ,A t )] (20)

[0064] In the formula, α∈(0,1) represents the learning rate, R t+1 represents the immediate reward value obtained at time t+1, and a′ represents the value in state S t+1The Q learning algorithm uses the above update iteration rules to obtain rewards by testing different distribution network states, and selects the decision action at time t based on the rewards;

[0065] When the Q-value table finally converges, the learning is completed, and the state of the distribution network is the state that minimizes the distribution network cost.

[0066] A collaborative control device for an energy storage cluster considering adjustment cost optimization includes: a processor and a memory, wherein the processor is connected to the memory, and the memory is used to store executable instructions of the processor, and the processor is configured to execute the aforementioned collaborative control method for an energy storage cluster considering adjustment cost optimization by executing the executable instructions.

[0067] Compared with the prior art, the present invention has the following beneficial effects:

[0068] The present invention utilizes distributed energy storage facilities, taking into account network loss costs, energy storage regulation costs, safety over-limit costs, and power curtailment costs. Based on a reinforcement learning algorithm, the node voltage and line power of the distribution network are optimized, thereby minimizing the cost of the active distribution network under high-proportion distributed photovoltaic access. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 It is a flow chart of the energy storage cluster collaborative control method considering adjustment cost optimization of the present invention. DETAILED DESCRIPTION

[0070] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0071] like Figure 1 As shown, the energy storage cluster collaborative control method considering adjustment cost optimization of the present invention includes the following steps:

[0072] Step 1: Define the active distribution network regulation control cost objective function including energy storage system:

[0073] The regulation and control cost of the active distribution network including the energy storage system is:

[0074]

[0075] In the formula, f is the total regulation and control cost of the system, T is the regulation and control period, C1 is the network loss cost of the system, C2 is the energy storage regulation cost of the system, C3 is the safety limit penalty cost, and C4 is the cost of curtailment of solar power and power rationing; where:

[0076] (1) Network loss costs consist of line loss costs and grid equipment operation and maintenance costs, namely:

[0077]

[0078] Where P loss Represents line loss, V represents electricity price, T represents regulation control cycle, and C5 represents operation and maintenance cost of the system.

[0079] (2) The energy storage regulation cost consists of the charging loss cost and the discharging loss cost of the energy storage device, namely:

[0080]

[0081] Where α and β are the energy storage charging loss cost coefficient and energy storage discharging loss cost coefficient respectively; P dis , P sto They are energy storage releasing electric energy and energy storage receiving electric energy.

[0082] (3) The penalty cost of the active distribution network for safety violations refers to the penalty fee incurred when the system exceeds the safety regulations, that is,

[0083]

[0084] In the formula, γ c is the penalty fee for exceeding the limit, and η is the maximum load rate of the system when it exceeds the limit.

[0085] (4) The cost of curtailment and power rationing refers to the economic loss caused by the fact that the power generated by the photovoltaic power generation system in certain periods exceeds the carrying capacity of the distribution network, or the power generation cannot be immediately connected to the grid due to reasons such as regulation and balance during the operation of the distribution network, and part of the power generation has to be abandoned or limited. The expression of curtailment and power rationing cost is:

[0086]

[0087] In the formula, γ pv Indicates the penalty cost for abandoning light; P f Represents the predicted output of photovoltaic power, P t Indicates the actual output of photovoltaic power.

[0088] Therefore, the objective function to minimize the total cost is:

[0089]

[0090] Step 2: Define the constraints for safe and economic operation of the active distribution network

[0091] (2) System energy balance constraints:

[0092] P w +P dis +P t =P L +P sto (7)

[0093] In the active distribution network, the power input mainly includes distributed photovoltaic output energy P w , the energy released by the energy storage system P dis , the external public grid provides energy P t , and the power consumption mainly includes conventional power load P L and the charge capacity P of the energy storage system sto For active distribution networks, the energy balance constraint is as shown in formula (7), which is the balance between supply and demand of electric energy.

[0094] (2) Distributed photovoltaic power generation output upper and lower limits:

[0095] P w,min ≤P w ≤P w,max (8)

[0096] (3) Energy storage system charging and discharging power constraints:

[0097] 0≤P dis ≤P dis,max (9)

[0098] 0≤P sto ≤P sto,max (10)

[0099] Where, P dis,max and P sto,max They are the upper limits of the discharging and charging outputs of energy storage devices respectively; energy storage devices usually store energy based on physical, chemical, mechanical and other principles, so their energy storage behavior is limited by the characteristics of the equipment.

[0100] (4) Constraints on charging and discharging behavior of energy storage systems:

[0101] S0=S T (11)

[0102] P dis ·P sto =0 (12)

[0103] S min ≤S0+P sto -P dis ≤S max (13)

[0104] In the formula, S min With S max are the lower and upper limits of the energy storage device capacity, respectively. To ensure the normal operation of the energy storage device, it is necessary to constrain the operating state of the energy storage device. Formula (11) constrains the storage level S of the energy storage device after one scheduling cycle. TIt can still return to the initial storage level S0 to prevent insufficient energy supply in the next scheduling cycle; Formula (12) limits the energy storage device from charging and discharging at the same time to prevent it from interfering with the power system frequency; Formula (13) limits the state of charge (SOC) of the energy storage device to avoid excessive charging and discharging that shortens the equipment life.

[0105] (5) Bus voltage constraints:

[0106] U min ≤U≤U max (14)

[0107] Where Umin and Umax represent the lower limit and upper limit of the bus voltage respectively, and U is the bus voltage level.

[0108] (7) Line flow constraints:

[0109] 0≤U1≤U' max (15)

[0110] 0≤P1≤P max (16)

[0111] Where U1 is the node voltage, P1 is the line power; U' max , P max They represent the upper limit of line voltage and the upper limit of line power respectively.

[0112] Step 3: Active distribution network power flow control solution algorithm based on reinforcement learning

[0113] After determining the objective function and constraints, the established node, branch, load, generator and transformer models are substituted into the objective function and constraints. Since the objective function is nonlinear and discontinuous, it is difficult for traditional optimization algorithms to cope with it. The reinforcement learning algorithm (RL) is a goal-oriented algorithm that allows the agent to make decisions based on the environment, accumulate experience through each trial and error, and improve the next action decision to maximize the reward. This environmental interaction process can be described by the Markov decision process, which consists of a five-tuple consisting of a state space S, an action space A, a reward function R, an action probability P and a discount factor γ.

[0114] State space S: includes the charging and discharging state of the energy storage system at a certain moment, the load demand of the distribution network, and the line flow distribution.

[0115] Action space A: includes the charging and discharging actions of the energy storage system, the energy dispatching actions of the distribution network, and the regulation of node voltage.

[0116] Reward function R: is set to the negative value of the total cost of the active distribution network, that is, negative cost is reflected as a positive reward value. This design can make the distribution network move towards reducing costs at every step.

[0117] R(s)=-f (17)

[0118] Action probability P: represents the probability of the active distribution network taking a corresponding action in a certain state. The ε-greedy strategy can be used to determine the action probability, which is used to balance exploration and exploitation. It is widely used in various reinforcement learning algorithms. The ε-greedy strategy selects random actions (i.e., exploration) with probability ε. This action is not necessarily the action that minimizes the regulation and control cost of the distribution network in the current state. The current optimal action (i.e., utilization) is selected with probability 1-ε, that is, the action that minimizes the regulation and control cost of the current distribution network is taken. This can avoid falling into the local optimal solution to a certain extent. At the beginning of learning, a higher exploration probability is usually set. As the exploration proceeds, ε can be gradually reduced so that the agent can make more use of existing knowledge to make decisions.

[0119] Discount factor γ: This parameter is in the range of [0, 1] and is used to give the ability to discount rewards. That is, the larger the value of γ, the more the decision-making focuses on long-term rewards. Conversely, the decision-making focuses on immediate rewards.

[0120] The goal of the reinforcement learning problem is to obtain an optimization strategy π: S→A to maximize the reward value under this strategy.

[0121] The specific steps of the algorithm are as follows:

[0122] (1) Define the action-value function of reinforcement learning

[0123] Q learning is one of the main algorithms of reinforcement learning. It uses the action value function Q(S t ,A t ) is used to measure the quality of the actions performed by the agent in the state it is in following the strategy. In the Q learning phase, each state-action in the distribution network corresponds to a Q value.

[0124]

[0125] Where S t is the distribution network state corresponding to time t, A t represents the action taken by the distribution network at time t, E π represents the expected value of strategy π, γ represents the discount factor, R kis the immediate reward obtained after the k-th action, s0, a0 are the initial state and initial action of the distribution network. In the initial stage, the Q value is initialized to 0.

[0126] (2) Using the ε-greedy strategy to select actions

[0127] π * (s) = (1-ε)δ(a * -a)+ε / |A| (19)

[0128] In the formula, π * (s) represents the strategy of the agent to select actions under a given distribution network state S, that is, the action of the distribution network is determined according to the state S of the distribution network. This embodiment represents the expression of the ε-greedy strategy, which is a probability distribution function about action a. δ is the Kronecker delta function, which is equal to 1 when its parameters are equal, and 0 otherwise. a* is the action that maximizes the reward value under state S. |A| is the total number of actions in the action set A. δ(a*-a) is 1 when a=a*, otherwise it is 0, indicating that the optimal action a* is selected with probability 1-ε. ε / |A| means uniformly randomly selecting any action in the action set A with probability ε.

[0129] (3) Update the Q value iteratively:

[0130] The update iteration rules of Q learning are as follows:

[0131] Q(S t ,A t )=Q(S t ,A t )+α[R t+1 +γmax a' Q(S t+1 ,a')-Q(S t ,A t )] (20)

[0132] In the formula, α∈(0,1) represents the learning rate, R t+1 represents the immediate reward value obtained at time t+1, and a′ represents the value in state S t+1 The Q-learning algorithm uses the above iterative rules to obtain rewards by testing different distribution network states and selects the decision action at time t based on the rewards.

[0133] When the Q-value table finally converges, the agent learning is completed, and the state of the distribution network is the state that minimizes the distribution network cost.

[0134] In a specific implementation, a control device may be provided, including: a processor and a memory, wherein the processor is connected to the memory, and the memory is used to store executable instructions of the processor, and the processor is configured to execute the aforementioned energy storage cluster collaborative control method considering adjustment cost optimization by executing the executable instructions.

[0135] In addition to the above embodiments, the present invention may also have other implementation modes. Any technical solutions formed by equivalent replacement or equivalent transformation shall fall within the protection scope required by the present invention.

Claims

1. A method for coordinated control of energy storage clusters considering adjustment cost optimization, characterized in that: The following steps are involved: Step 1: Define the active distribution network regulation control cost objective function including energy storage system: The regulation and control cost of the active distribution network including the energy storage system is: In the formula, f is the total regulation and control cost of the system, T is the regulation and control period, C1 is the network loss cost of the system, C2 is the energy storage regulation cost of the system, C3 is the safety limit penalty cost, and C4 is the cost of curtailment of solar power and power rationing; where: (1) Network loss costs consist of line loss costs and grid equipment operation and maintenance costs, namely: Where P loss represents the line loss, V represents the electricity price, T represents the regulation control cycle, and C5 represents the operation and maintenance cost of the system; (2) The energy storage regulation cost consists of the charging loss cost and the discharging loss cost of the energy storage device, namely: Where α and β are the energy storage charging loss cost coefficient and energy storage discharging loss cost coefficient respectively; P dis , P sto They are energy storage releasing electric energy and energy storage receiving electric energy; (3) The penalty cost of safety over-limit of active distribution network is: In the formula, γ c is the penalty fee for exceeding the limit, η is the maximum load rate of the system when exceeding the limit; (4) The expression of the cost of curtailing solar power and limiting electricity supply is: In the formula, γ pv Indicates the penalty cost for abandoning light; P f Represents the predicted output of photovoltaic power; P t Indicates the actual output of photovoltaic power; Therefore, the objective function to minimize the total cost is: Step 2: Define the constraints for safe and economic operation of the active distribution network (1) System energy balance constraints: P w +P dis +P t =P L +P sto (7) In an active distribution network, the power input includes distributed photovoltaic output energy P w , the energy released by the energy storage system P dis , the external public grid provides energy P t , power consumption includes conventional power load P L and the charge capacity P of the energy storage system sto ; The energy balance constraint is shown in formula (7), which is the balance between supply and demand of electric energy; (2) Distributed photovoltaic power generation output upper and lower limits: P w,min ≤P w ≤P w,max (8) (3) Energy storage system charging and discharging power constraints: 0≤P dis ≤P dis,max (9) 0≤P sto ≤P sto,max (10) Where, P dis,max and P sto,max They are the upper limits of the output for discharging and charging of energy storage equipment respectively; (4) Constraints on charging and discharging behavior of energy storage systems: S0=S T (11) P dis ·P sto =0 (12) S min ≤S0+P sto -P dis ≤S max (13) In the formula, S min With S max are the lower and upper limits of the energy storage device capacity respectively; Formula (11) constrains the storage level S of the energy storage device after one scheduling cycle T It can still return to the initial storage level S0 to prevent insufficient energy supply in the next scheduling cycle; Formula (12) limits the energy storage device from charging and discharging at the same time to prevent it from interfering with the power system frequency; Formula (13) limits the charge state of the energy storage device to avoid excessive charging and discharging that shortens the life of the equipment; (5) Bus voltage constraints: IN min ≤U≤U max (14) In the formula, Umin and Umax represent the lower limit and upper limit of the bus voltage respectively, and U is the bus voltage level; (6) Line flow constraints: 0≤U1≤U' max (15) 0≤P1≤P max (16) Where U1 is the node voltage, P1 is the line power; U' max , P max They represent the upper limit of line voltage and the upper limit of line power respectively; Step 3: Optimize node voltage and line power of the distribution network based on reinforcement learning algorithm The reinforcement learning algorithm is described by the Markov decision process, which consists of a five-tuple consisting of the state space S, the action space A, the reward function R, the action probability P and the discount factor γ; State space S: includes the charging and discharging state of the energy storage system at a certain moment, the load demand of the distribution network, and the line flow distribution; Action space A: includes charging and discharging actions of the energy storage system, energy dispatching actions of the distribution network, and node voltage regulation; Reward function R: It is set as the negative value of the total cost of the active distribution network, that is, negative cost is reflected as a positive reward value: R(s)=-f (17) Action probability P: represents the probability of the active distribution network taking corresponding actions in a certain state, and the action probability is determined by the ε-greedy strategy; Discount factor γ: This parameter is in the range of [0, 1] and is used to give the ability to discount rewards. That is, the larger the value of γ, the more the decision-making focuses on long-term rewards, and vice versa, the decision-making focuses on immediate rewards; The specific steps of the algorithm are as follows: (1) Define the action-value function of reinforcement learning Using the Q learning algorithm, the action value function Q(S t ,A t ) is used to measure the quality of the actions performed in the state following the strategy; in the Q learning phase, each state-action in the distribution network corresponds to a Q value; Where S t is the distribution network state corresponding to time t, A t represents the action taken by the distribution network at time t, E π represents the expected value of strategy π, γ represents the discount factor, R k is the timely reward obtained after the k-th action, s0, a0 are the initial state and initial action of the distribution network; in the initial stage, the Q value is initialized to 0; (2) Using the ε-greedy strategy to select actions p * (s)=(1-e)δ(a * -a)+ε / |A| (19) In the formula, π * (s) represents the strategy of the agent to select actions under a given distribution network state S; δ is the Kronecker function, which is equal to 1 when its parameters are equal and 0 otherwise; a* is the action that maximizes the reward value under state S; |A| is the total number of actions in the action set A; δ(a*-a) is 1 when a=a* and 0 otherwise, indicating that the optimal action a* is selected with probability 1-ε; ε / |A| means uniformly randomly selecting any action in the action set A with probability ε; (3) Update the Q value iteratively: The update iteration rules of Q learning are as follows: Q(S t ,A t )=Q(S t ,A t )+α[R t+1 +γmax a' Q(S t+1 ,a')-Q(S t ,A t )] (20) In the formula, α∈(0,1) represents the learning rate, R t+1 represents the immediate reward value obtained at time t+1, and a′ represents the value in state S t+1 The Q learning algorithm uses the above update iteration rules to obtain rewards by testing different distribution network states, and selects the decision action at time t based on the rewards; When the Q-value table finally converges, the learning is completed, and the state of the distribution network is the state that minimizes the distribution network cost.

2. A collaborative control device for energy storage clusters considering adjustment cost optimization, characterized in that: include: A processor and a memory, wherein the processor is connected to the memory, and the memory is used to store executable instructions of the processor, and the processor is configured to execute the energy storage cluster collaborative control method considering regulation cost optimization as described in claim 1 by executing the executable instructions.

Images