A nested time-series distributed energy storage two-level planning optimization method

Abstract

This invention belongs to the field of power system operation planning technology and discloses a nested time-series operation distributed energy storage two-level planning optimization method. This invention constructs a nested time-series operation distributed energy storage two-level planning optimization model, which collaboratively considers the differentiated operating costs caused by different time-series operation strategies during the investment planning stage, realizing unified modeling of distributed energy storage system planning and time-series operation, avoiding the problem of planning and operation being separated. A joint solution framework based on HPSO–SAC is proposed. By using the HPSO algorithm at the planning level to optimize the energy storage site selection and capacity configuration scheme with nested operating costs, and using the SAC algorithm at the operation level to optimize the time-series operation strategy of the distributed energy storage system, a collaborative and efficient solution of planning schemes and operation strategies is achieved. This invention provides an effective and practical technical solution for investment planning and operation control of distributed energy storage systems in active distribution networks.

Description

Technical Field

[0001] This invention belongs to the field of power system operation planning technology, and more specifically, relates to a nested time-series operation distributed energy storage two-layer planning optimization method. Background Technology

[0002] The integration of high-penetration distributed renewable energy sources poses significant challenges to the distribution network in terms of both security and economy. Distributed energy storage systems, with their flexible multi-point deployment characteristics and aggregated regulation capabilities, offer a promising regulation method to alleviate these problems. However, the construction investment cost of multi-point connected distributed energy storage systems is high, and their cross-time energy allocation capabilities are highly dependent on the time-series charging and discharging strategies during operation. Therefore, there is an urgent need to study distributed energy storage planning and optimization methods based on nested time-series operation simulations to reasonably reflect the impact of operational behavior on system economy and performance during the planning stage.

[0003] Currently, existing methods for planning and optimizing distributed energy storage systems mostly focus on static planning aimed at minimizing investment costs. During the operation phase, they typically employ pre-set or simplified charging and discharging strategies, resulting in a lack of effective coordination between planning and operation. Regarding the coordinated optimization of planning and operation of distributed energy storage systems, existing technologies mainly suffer from the following problems and shortcomings: (1) Planning and operation still adopt separate modeling, making it difficult to characterize the impact of different operation strategies on the system's economy and performance under different planning schemes; (2) Discrete decision variables such as energy storage site selection need to be jointly optimized with continuous variables such as rated power and capacity configuration. At the same time, the differentiated operation costs caused by the operation strategy also need to be considered. The overall problem presents strong nonlinear and strong nonconvex mixed integer programming characteristics, making it difficult to solve; (3) The cross-time energy coupling characteristics of multi-point access distributed energy storage systems and the uncertainty of wind power and photovoltaic output further exacerbate the complexity of distribution network time-series operation optimization, making it difficult for traditional planning and operation optimization methods to simultaneously take into account computational efficiency and solution accuracy. Summary of the Invention

[0004] To address the above-mentioned deficiencies or improvement needs of existing technologies, this invention provides a nested time-series distributed energy storage two-layer planning and optimization method, which aims to solve the problem of the lack of effective coordination between planning and operation optimization in existing distributed energy storage system planning and optimization methods.

[0005] To achieve the above objectives, this invention provides a nested time-series distributed energy storage two-layer planning optimization method, comprising: A two-layer planning optimization model for a nested time-series distributed energy storage system is constructed, comprising a lower-layer time-series operation optimization model and an upper-layer planning optimization model. The lower-level timing optimization model minimizes costs including charging and discharging costs. Voltage deviation penalty cost With energy storage degradation costs Taking the operating cost of the distributed energy storage system, including the above, as the objective function, As decision variables, As variables for energy storage deployment decisions, Indicates at node Install energy storage units. Indicates at node No energy storage unit is installed; and Representing nodes respectively The rated power and rated capacity of the energy storage units installed on the surface. , This represents the total number of nodes in the distribution network; the upper-level planning and optimization model takes minimizing the total cost of the distributed energy storage system within the planning period as its objective function, and uses the number of nodes as the criterion. The real-time charging and discharging power of the energy storage unit installed on the upper part is the decision variable; wherein, the total cost within the planning period includes the investment cost and the net present value of the operating cost determined by the operating cost fed back by the lower-level time-series operation optimization model; Under the operational constraints satisfied by the distributed energy storage system, the objective function of the upper-level planning optimization model is solved to optimize the planning scheme of the distributed energy storage system. Under the planning scheme, the objective function of the lower-level time-series operation optimization model is solved to optimize the time-series operation strategy of the distributed energy storage system.

[0006] Furthermore, when optimizing the planning scheme and the time-series execution strategy, the HPSO-SAC algorithm is used to jointly optimize the planning scheme and the time-series execution strategy, including: S1. At the current iteration step, the HPSO algorithm is used to solve the objective function of the upper-level planning optimization model to obtain the planning scheme at the current iteration step; wherein, each particle in the HPSO algorithm corresponds to a set of decision variables. To characterize the planning scheme, Indicates the first Particles Corresponding decision variables , Indicates the size of the particle swarm; S2. Under the planning scheme at the current iteration step, the SAC algorithm is used to solve the objective function of the lower-level temporal operation optimization model to obtain the temporal operation strategy at the current iteration step. The state space of the agent in the SAC algorithm is... and action space They are respectively:

[0007]

[0008] In the formula, For nodes Real-time energy status of the energy storage unit installed on it. For nodes At any moment per-unit voltage, For nodes Load demand; and The outputs are wind power and photovoltaic power, respectively. Time-of-use pricing; Represents a node The real-time charging and discharging power of the energy storage unit installed on it represents the timing operation strategy; Represents a node The energy storage unit installed on it is at all times Rated power; The charging and discharging cost Voltage deviation penalty cost With energy storage degradation costs By using weighted summation as the immediate reward function, the temporal execution strategy at the current iteration step is obtained when the agent converges. ; S3. Timing execution strategy based on the current iteration step. Calculate the current time-series operating cost and convert it into the net present value of the operating cost, then feed it back to the upper-level planning and optimization model to update the objective function of the upper-level planning and optimization model; S4. Determine whether the preset number of iterations has been reached. If not, return to step S1 to proceed to the next iteration. If yes, output the final planning scheme and timing strategy.

[0009] Furthermore, the HPSO algorithm is used to solve the objective function of the upper-level planning optimization model to obtain the planning scheme at the current iteration step, including: S11, based on the particles at the current moment Update the objective function of the upper-level planning and optimization model, and update the particles based on the objective function. The velocity and position of the updated particles are determined, and crossover and mutation operations are performed on the updated particles to obtain the particles for the next time step. S12. Determine whether the planning scheme represented by the particle at the next moment satisfies the running constraints. If not, take the particle at the next moment as the particle at the current moment and return to step S11 to update the particle again. If yes, obtain the planning scheme under the current iteration step.

[0010] Furthermore, particles The speed update method is as follows:

[0011] in, Represents the internal particles of the HPSO algorithm At the present moment speed, , Indicates by Affected particles velocity components, Indicates by Affected particles velocity components, Indicates by Affected particles velocity components; Inertial weight; Indicates the current time particles ; , These are cognitive learning factors and social learning factors, , A random number uniformly distributed within the interval [0,1]. Indicates the local optimal position. Indicates the globally optimal position. and The objective function value is determined based on the current value of the upper-level planning and optimization model; Indicates the particle at the next moment speed; particle The location update method is as follows:

[0012]

[0013]

[0014] in, , , They represent particles respectively At the present moment of , , ; , , They represent particles respectively In the next moment , , ; , , They represent respectively by , , Affected particles The velocity component at the next moment; This represents the threshold function.

[0015] Furthermore, the objective function of the upper-level planning optimization model is:

[0016] in, This represents the total cost over the planning period. Indicates the installation node of the energy storage unit. This indicates the rated power of the energy storage unit. This indicates the rated capacity of the energy storage unit. This represents the investment cost. The net present value of the operating costs is obtained by discounting the operating costs fed back from the lower-level time-series operation optimization model to the beginning of the planning period and accumulating them over the planning period; the investment costs for:

[0017] In the formula, Investment cost for rated power of energy storage, The investment cost for the rated capacity of energy storage; As variables for energy storage deployment decisions, Indicates at node Install energy storage units. Indicates at node No energy storage unit is installed; and Representing nodes respectively The rated power and rated capacity of the energy storage units installed on the surface. This indicates the total number of nodes in the distribution network.

[0018] Furthermore, the objective function of the lower-level time-series operation optimization model is:

[0019] in, This represents the operating cost of a distributed energy storage system. Represents a node Real-time charging and discharging power of the energy storage unit installed on it; charging and discharging cost. Voltage deviation penalty cost With energy storage degradation costs They are respectively:

[0020] In the formula, Indicates charging. Indicates discharge; , They are respectively The charging price and the discharge grid connection price at different times. and They are respectively at the node The charging and discharging efficiency of the energy storage unit installed on it. Indicates the scheduling time step. Total operating cycle;

[0021] In the formula, For nodes At any moment per-unit voltage, , These are the upper and lower limits of voltage for normal operation of the distribution network, respectively. This is the cost factor for voltage offset penalty;

[0022] In the formula, For nodes The cost per unit capacity of the energy storage units installed on the site, For nodes The rated cycle life of the energy storage unit installed on it.

[0023] Furthermore, the operational constraints include energy storage operation frequency constraints:

[0024] in, Indicates the maximum permissible number of energy storage operation cycles. For binary indicator variables, defined as:

[0025] In the formula, It is the introduced power change threshold.

[0026] The present invention also provides a nested time-series distributed energy storage two-level planning optimization system, including a computer-readable storage medium and a processor; The computer-readable storage medium is used to store executable instructions; The processor is used to read executable instructions stored in the computer-readable storage medium and execute the nested timing-based distributed energy storage two-level planning optimization method described above.

[0027] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the nested time-series distributed energy storage two-level planning optimization method as described in any of the preceding claims.

[0028] The present invention also provides a computer program product, including a computer program that, when the computer program is run on a computer, causes the computer to execute the nested time-series distributed energy storage two-level planning optimization method described above.

[0029] In summary, the above-described technical solutions conceived in this invention can achieve the following beneficial effects: (1) This invention constructs a nested time-series operation distributed energy storage two-layer planning optimization model. When optimizing the planning scheme, it considers the differentiated operating costs caused by different time-series operation strategies, realizing unified modeling of distributed energy storage system planning and time-series operation, avoiding the problem of planning and operation being separated. Specifically, this invention uses the upper-layer planning optimization model to model the investment cost of the distributed energy storage system and the net present value of the operating cost fed back by the lower-layer time-series operation optimization model into the total cost within the planning period, and optimizes the planning scheme of the distributed energy storage system with the goal of minimizing it. The lower-layer time-series operation optimization model considers differentiated operating costs, including time-of-use electricity price arbitrage (representation of charging and discharging cost), voltage regulation effect (representation of voltage deviation penalty cost), and lifetime loss (representation of energy storage degradation cost), optimizes the time-series operation strategy of the distributed energy storage system with the goal of minimizing it, and feeds back the corresponding operating cost discount to the upper-layer planning optimization model, realizing unified modeling of energy storage system planning nested time-series operation, avoiding the separation of planning and operation, and realizing the collaborative solution of planning decision and time-series operation decision.

[0030] (2) Furthermore, this invention proposes a two-layer planning optimization joint solution framework based on HPSO–SAC. By using the HPSO algorithm at the planning layer to optimize the energy storage site selection and capacity configuration scheme with nested operating costs, under the current planning scheme, the distribution network time-series operation scenario under the condition of high proportion of distributed renewable energy access is modeled as a Markov decision process at the operation layer to characterize the dynamic behavior of energy allocation across time periods of the distributed energy storage system in a stochastic environment. The SAC algorithm is used to optimize the time-series operation strategy of the distributed energy storage system, thereby realizing the coordinated and efficient solution of the planning scheme and the operation strategy.

[0031] (3) Preferably, the operation constraints of the present invention also include energy storage operation frequency constraints, which can ensure that the number of energy storage operation cycles is reduced and the battery degradation is delayed while meeting the optimization objectives.

[0032] In summary, this invention addresses the problem of ineffective coordination and difficulty in solving existing distributed energy storage system planning and optimization methods, which often lack effective coordination between planning and operation optimization. Therefore, in scenarios with a high proportion of distributed renewable energy integration, it coordinates the relationship between the investment scale, operational economy, and voltage regulation performance of distributed energy storage systems, thereby improving the utilization efficiency of distribution network assets and the integration and absorption capacity of distributed renewable energy. This invention provides an effective and practical technical solution for the investment planning and operation control of distributed energy storage systems in active distribution networks. Attached Figure Description

[0033] Figure 1 This is the overall flowchart of the distributed energy storage two-layer planning optimization model with nested time-series operation according to an embodiment of the present invention.

[0034] Figure 2 This is a flowchart of the bi-level programming optimization joint solution framework based on HPSO–SAC in this embodiment of the invention.

[0035] Figure 3 This is a comparative analysis chart of the economic effects in embodiments of the present invention.

[0036] Figure 4 This is a comparative analysis chart of the operational security effects in embodiments of the present invention. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0038] In this invention, the terms "first," "second," etc., used in the invention and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0039] Example 1 like Figure 1 As shown, this embodiment of the invention provides a nested time-series two-layer planning optimization method for multi-point access distributed energy storage systems, including the following steps: (1) Construction of a two-layer planning optimization model for nested time-series operation of distributed energy storage systems: The upper-layer model unifies the investment cost of the distributed energy storage system with the net present value of operating costs fed back from the lower-layer model as the total cost within the planning period, and optimizes the planning scheme of the distributed energy storage system with the goal of minimizing it. The lower-layer model considers differentiated operating costs, including time-of-use electricity price arbitrage, voltage regulation effect, and lifespan depreciation, and optimizes the time-series operation strategy of the distributed energy storage system with the goal of minimizing it. The corresponding operating costs are discounted and fed back to the upper-layer model, realizing the unified modeling of nested time-series operation of energy storage system planning. Specifically, it includes: (1-1) Construct a nested time-series operation upper-level planning optimization model. The upper-level planning optimization model unifies the investment cost of the distributed energy storage system with the net present value of the operating cost fed back from the lower-level model (lower-level time-series operation optimization model) as the total cost within the planning period, and constructs an objective function with the goal of minimizing it, thereby optimizing the planning decision of the distributed energy storage system:

[0040] in, This represents the total cost over the planning period. Indicates the installation node of the energy storage unit. This indicates the rated power of the energy storage unit. This indicates the rated capacity of the energy storage unit. This indicates the investment cost of a distributed energy storage system. This represents the net present value of the operating costs fed back from the lower-level model.

[0041] (1-1-1) Investment Costs. The investment costs of distributed energy storage systems mainly consist of the location of energy storage units, rated power, rated capacity, and other factors, expressed as follows:

[0042] in, Investment cost for rated power of energy storage, This refers to the investment cost for the rated capacity of energy storage. As variables for energy storage deployment decisions, Indicates at node Install energy storage units. Indicates at node No energy storage unit is installed; and Representing nodes respectively The rated power and rated capacity of the energy storage units installed on the surface. This indicates the total number of nodes in the distribution network. , and These respectively characterize the location, capacity, and power configuration of a distributed energy storage system, representing the planning scheme for the distributed energy storage system.

[0043] (1-1-2) Operating Costs. The operating costs reported by the lower-level model are determined by the real-time charging and discharging power of the distributed energy storage system. To align these costs with investment costs on both the time and value scales, they are discounted to the beginning of the planning period and accumulated over the planning cycle to obtain the net present value (NPV) of the operating costs, expressed as follows:

[0044] in, The operating cost reported by the lower-level model. Indicates at node Real-time charging and discharging power of the energy storage unit installed on it, and real-time charging and discharging power at various times. This represents the timing operation strategy of a distributed energy storage system; The number of days in a year. The annual discount rate is 10%. The planning period is in years.

[0045] (1-2) Lower-level time-series operation optimization model. The cross-time-period energy allocation of distributed energy storage systems needs to comprehensively consider the time-of-use electricity price arbitrage benefits caused by real-time charging and discharging behavior, voltage control effectiveness, and lifespan reduction caused by charging and discharging. The lower-level model is constructed based on charging and discharging costs. Voltage deviation penalty cost With energy storage degradation costs The operating cost function of the distributed energy storage system is defined, and the timing operation strategy of the energy storage is optimized with the goal of minimizing this cost function. The expression is as follows:

[0046] in, This represents the operating cost of a distributed energy storage system.

[0047] (1-2-1) Charging and discharging costs:

[0048] in, Indicates charging. Indicates discharge; , They are respectively The charging price and the discharge grid connection price at different times. and They are respectively at the node The charging and discharging efficiency of the energy storage unit installed on it. Indicates the scheduling time step. This refers to the total operating cycle.

[0049] (1-2-2) Voltage deviation penalty cost:

[0050] in, For nodes At any moment per-unit voltage, , These are the upper and lower limits of voltage for normal operation of the distribution network, respectively. This is the voltage offset penalty cost factor.

[0051] (1-2-3) Energy storage degradation costs:

[0052] In the formula, For the node The cost per unit capacity of the energy storage units installed on the site, For the node The rated cycle life of the energy storage unit installed on it.

[0053] (1-3) Constraints. The two-layer optimization framework must meet the following constraints to ensure the technical feasibility and economic rationality of the distributed energy storage system.

[0054] (1-3-1) Budget Constraints:

[0055] in, This indicates the total investment budget.

[0056] (1-3-2) Power and capacity constraints:

[0057] in, , Indicates at node The upper limit of the rated power and rated capacity of the energy storage unit installed on it.

[0058] (1-3-3) Power flow constraint:

[0059] in, and They are nodes The amount of active power and reactive power injected. For nodes and The phase angle difference between them. and These represent the admittance and admittance of the branch admittance matrix of the distribution network, respectively.

[0060] (1-3-4) Node voltage constraints:

[0061] in, , Representing nodes respectively The lower and upper limits of the voltage.

[0062] (1-3-5) State of Energy (SOC) Constraints:

[0063]

[0064] in, Indicates at node Real-time energy status of the energy storage units installed on the surface; This indicates the duration of each scheduling cycle within the scheduling period, also known as the scheduling time step.

[0065] (1-3-6) Charge and discharge power constraints:

[0066] (1-3-7) Constraints on the frequency of energy storage operation:

[0067] in, Indicates the maximum permissible number of energy storage operation cycles. For binary indicator variables, defined as:

[0068] in, It is an introduced power change threshold (empirical value) used to filter out small fluctuations and avoid overcounting trivial control actions.

[0069] By constraining the frequency of energy storage operations, we can ensure that the number of energy storage cycles is reduced while meeting the optimization objectives, thus slowing down battery degradation.

[0070] (2) A joint solution framework for a bilevel programming optimization model based on HPSO–SAC. For example... Figure 2 As shown. Within the two-layer optimization framework, the upper-layer model uses the Hybrid Particle Swarm Optimization (HPSO) algorithm to optimize the site selection and capacity configuration scheme of the distributed energy storage system; the lower-layer model formalizes the time-series operation process of the distribution network as a Markov Decision Process (MDP), and uses the SoftActor-Critic (SAC) algorithm to optimize the time-series operation strategy of the distributed energy storage system within the MDP, achieving collaborative solution between the planning and operation layers. Specifically, this includes: (2-1) Upper-level planning optimization based on nested time-series execution of HPSO (Hyperjective Particle Swarm Optimization). Within the HPSO algorithm framework, each particle... This corresponds to a set of planning and decision variables for distributed energy storage systems, including discrete variables related to site selection. and rated power With capacity configuration For continuous variables, the expression is:

[0071] Among them, particles The merits of the plan are evaluated based on the total cost over the planning period. Minimum is the optimization objective for particle Updates enable the timing sequence to directly participate in the optimization of planning decisions; This indicates the size of the particle swarm.

[0072] (2-1-1) The particle velocity is updated as follows:

[0073] in, Represents the internal particles of the HPSO algorithm At the present moment speed, , Indicates by Affected particles At the present moment velocity components, Indicates by Affected particles At the present moment velocity components, Indicates by Affected particles At the present moment velocity components; The inertial weight is determined empirically. Indicates the current time particles ; , These are cognitive learning factors and social learning factors, with empirical values. , A random number uniformly distributed within the interval [0,1]. Indicates the local optimal position. Represents the globally optimal position, where, and The objective function value is determined based on the upper-level planning and optimization model.

[0074] (2-1-2) Particle position update: (2-1-2-1) To ensure the particles Discrete variables of the distribution points Preserve binary properties, apply a threshold function :

[0075] in, Represents particles At the present moment Discrete variables of the distribution points , Represents particles Discrete variables at the next time step , Indicates by Affected particles The velocity component at the next moment.

[0076] (2-1-2-2) Continuous variables such as rated power and capacity are updated using an additive method:

[0077] in, Represents particles At the present moment of , Represents particles In the next moment , Indicates by Affected particles The velocity component at the next moment; Indicates the current time of , Indicates the next moment , Indicates by Affected particles The velocity component at the next moment.

[0078] (2-1-3) Considering the problem of local optima in the planning scheme due to high nonlinearity and nonconvexity, the particle update process is performed on the particles. Embedded crossover and mutation operations enhance the algorithm's ability to explore different distributed energy storage system planning schemes.

[0079] (2-1-3-1) Crossover operation

[0080] in, Represents the particle after the crossover operation At the node Planning and decision-making on the basis of Indicates the first candidate for updating the planning decision. This indicates the second candidate for updating the planning decision; For crossover probability, Let the random numbers be uniformly distributed in the interval [0,1]; that is, if for a particle At the node Random numbers generated above Then the particle At the node Planning and decision-making Update the planning decision along the first candidate path; otherwise... Cross-update the planning decision towards the second candidate.

[0081] (2-1-3-2) Mutation operation

[0082] in, Particles before mutation operation At the node Planning and decision-making on the basis of This represents the particle after the mutation operation. At the node Planning and decision-making on the basis of For random variables, [ , ] is a node The permissible range of variables for energy storage planning decisions. To balance the mutation probability of exploration and convergence.

[0083] (2-2) Optimization of lower-level time-series operation strategy based on SAC algorithm (including Actor and Critic parts). Under the current planning scheme, the distribution network time-series operation scenario under the condition of high proportion of distributed renewable energy access is modeled as a Markov decision process (MDP) to characterize the dynamic behavior of distributed energy storage system in cross-time period energy allocation under stochastic environment. Then, the SAC algorithm is used to solve the MDP to optimize the time-series operation strategy of distributed energy storage system. ).

[0084] (2-2-1) The MDP includes a state space, an action space, a state transition function, and an immediate reward function. Its mathematical model is as follows: (2-2-1-1) State Space. Under the MDP framework, the time-series state of the distribution network is composed of multiple variables, including node-side operating information, distributed generation output characteristics, and external market signals. The system state is defined as:

[0085] in, For nodes The load demand, and The output is renewable energy power generation, specifically wind power and photovoltaic power generation. It is a time-of-use electricity price.

[0086] (2-2-1-2) Action Space. At each moment, the agent determines the charging and discharging power of each energy storage unit based on the current system state. Its action space is defined as:

[0087] when When >0, it indicates that the energy storage unit is charging; when When <0, it indicates that discharge is in progress; while =0 indicates an idle running state; This represents the current feedback from the upper-level model. .

[0088] (2-2-1-3) State transition function. At each moment, the agent transitions based on the current state. choose The system transitions to the next state according to the power flow evolution pattern. Its state transition process is described as follows:

[0089] (2-2-1-4) Immediate Reward Function. The charging / discharging cost, voltage deviation penalty cost, and energy storage degradation cost in the operating costs of a distributed energy storage system are weighted and constructed into an immediate reward function, defined as follows:

[0090] in, , , These are the corresponding weighting coefficients, and their empirical values. Based on the feedback from the upper-level model, the current... calculate.

[0091] (2-2-2) The SAC algorithm quantitatively evaluates the operating costs of cross-time energy allocation in distributed energy storage systems through the state-action value function and the state value function, which are defined as follows:

[0092]

[0093] in, State-action value function Represents the state value function. Indicates the expectation. The time-reward function is represented. This is a discount factor, ranging from 0 to 1, used to balance the importance of immediate and future rewards. The closer the value is to 1, the more the algorithm values ​​long-term cumulative rewards; the closer it is to 0, the more it focuses on immediate rewards. Indicates the next state The expected value is to be solved, where Follows the environmental transition probability distribution P That is, the agent changes state Execute action Then, it will transition to the next state based on the randomness of the environment. Indicates the action a The expected value is to be solved, where the action is... a Follow the current strategy π The distribution of .

[0094] (2-2-3) The SAC algorithm maximizes The optimal operating strategy for a distributed energy storage system is defined as follows:

[0095] in, Indicates the state The optimal strategy under these circumstances; This represents the optimal state-action value function.

[0096] This formula describes the strategy that optimizes the operating cost of a distributed energy storage system under a given system state, and is the core basis for the SAC algorithm to search for an approximately optimal operating strategy.

[0097] (2-2-4) Considering that distributed energy storage systems with multiple access points have continuously controllable charging and discharging power characteristics, the SAC algorithm uses an Actor network to... By performing function approximation, the continuous control actions of the distributed energy storage system are directly output, as defined below:

[0098] (2-2-5) To avoid premature convergence during training, the SAC algorithm introduces policy entropy as a regularization term in the policy optimization objective. This ensures optimal operating costs while maintaining appropriate policy exploration capabilities. Its Actor network loss function... Defined as:

[0099] in, The temperature coefficient is used to balance the trade-off between the exploratory capabilities of the operating strategy and the optimization effect of operating costs; These are the parameters of the Actor network; Represents the expectation operator, From the experience replay pool M The state of mid-sampling, In the current strategy πθ Below, from the state The sampled actions, the whole expression represents the calculation of the average loss over all state-action pairs in the experience replay pool; This is the state-action value function output by the Critic network. These are the parameters of the Critic network.

[0100] (2-2-6) To make this trade-off adaptive, the SAC algorithm further introduces an automatic adjustment mechanism for the temperature parameter, and its loss function... Defined as:

[0101] in, It is the target entropy.

[0102] (2-2-7) is approximately optimal. The SAC algorithm introduces a dual-critic network to approximate the value function, and its Bellman objective is defined as: =

[0103] in, The target value for the Critic network; This is the discount factor, with a value range of (0,1). This represents the state-action value of the k-th Critic network output; the "min" operator mitigates the accumulation of positive bias. By taking the smaller value of the double-Q network output, the overestimation of operating costs can be significantly suppressed, thereby improving the reliability of the operating strategy optimization.

[0104] (2-2-8) To prevent parameter oscillations during deep network training, the SAC algorithm uses a soft update method to update the target network:

[0105]

[0106] in, Control the update rate, This represents the updated Actor network parameters. This represents the updated k-th Critic network parameter.

[0107] Specifically, the above solution process includes the following steps: S1. At the current iteration step, the HPSO algorithm is used to solve the upper-level planning optimization model to obtain the planning scheme at the current iteration step, specifically including: Based on the particles in the particle swarm at the current moment The objective function of the upper-level planning optimization model is updated to update the particle's velocity and position. Then, crossover and mutation operations are performed on the updated particle position vectors to enhance population diversity and obtain the particles for the next time step. Based on the particles for the next time step, it is determined whether the constraints are met. If not, the particles are updated again; otherwise, the planning decision for the currently obtained particles is... This serves as the planning scheme for the current number of iterations.

[0108] S2. Under the planning scheme at the current iteration step, a reinforcement learning algorithm is used to solve the lower-level temporal operation optimization model to obtain the temporal operation strategy at the current iteration step. The state space and action space of the agent in the reinforcement learning algorithm are as follows:

[0109]

[0110] The charging and discharging costs, voltage deviation penalty costs, and energy storage degradation costs in the operating costs of a distributed energy storage system are weighted and constructed into an immediate reward function. When the agent converges, the time-series operation strategy for the current iteration step is obtained. .

[0111] S3. Timing execution strategy based on the current iteration step. Obtain the current operating cost And convert it into the net present value of operating costs. Feedback is given to the upper-level planning and optimization model to update the objective function of the upper-level planning and optimization model, and the process returns to step S1 for the next round of iteration until the preset number of iterations is reached, and the final energy storage planning scheme and time-series operation strategy are output. In this embodiment of the invention, under the constraints, the objective function of the upper-level planning and optimization model and the objective function of the lower-level time-series operation optimization model are minimized, and the preset number of iterations is reached.

[0112] (3) Comprehensive technical effect analysis.

[0113] (3-1) Based on the IEEE 33-node distribution system, a high-proportion distributed renewable energy (photovoltaic and wind power) access scenario is constructed. The nested time-series operation distributed energy storage two-layer planning optimization method proposed in this invention is adopted to realize the coordinated optimization of planning and operation of multi-point distributed energy storage.

[0114] (3-2) To verify the technical effect of the method of the present invention, it is compared with two existing typical schemes: Scheme 1 only aims to minimize the investment cost of energy storage and adopts a fixed charging and discharging strategy during the operation phase; Scheme 2 only aims to minimize the operating cost and performs time-series operation optimization, and uses the maximum required energy storage configuration as the planning result.

[0115] (3-3) Economic Comparison. Based on the planning results of distributed energy storage systems under different schemes and their corresponding time-series operation strategies, the initial investment cost and annualized operating cost of the energy storage systems are calculated respectively, and the overall return on investment is calculated accordingly. The specific results are as follows: Figure 3 As shown. Compared with the comparative scheme, the method of the present invention introduces runtime layer feedback information during the planning stage, i.e. This approach achieves synergistic optimization between energy storage investment scale and operating costs. With a moderate increase in investment scale, it can significantly reduce long-term system operating costs, thereby achieving a higher overall return on investment and effectively improving the overall utilization efficiency of distribution network assets.

[0116] (3-4) Comparison of Operational Safety. Under a medium level of energy storage configuration, the variation characteristics of the average voltage at each node of the distribution network under different distributed energy storage time-series operation strategies are analyzed to compare their operational safety levels. The results are as follows: Figure 4 As shown. Compared with the comparative scheme, the method of the present invention can achieve continuous regulation and fluctuation suppression of node voltage under a medium level of energy storage configuration, effectively improving the voltage deviation problem. While ensuring the safe and stable operation of the distribution network voltage, it further enhances the grid connection and absorption capacity of distributed renewable energy, achieving a synergistic improvement in operational safety and economy.

[0117] Example 2 This invention provides a nested time-series distributed energy storage two-layer planning optimization system, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the nested time-series distributed energy storage two-layer planning optimization method in Embodiment 1 above.

[0118] The relevant technical solutions are the same as above, and will not be repeated here.

[0119] Example 3 This invention provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the steps of the nested time-series distributed energy storage two-layer planning optimization method in Embodiment 1 above.

[0120] Specifically, the memory may include high-speed random access memory, as well as non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital (SD) cards, flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0121] The relevant technical solutions are the same as above, and will not be repeated here.

[0122] Example 4 This invention provides a computer program product, including a computer program that, when run on a computer, causes the computer to execute the steps of the nested time-series distributed energy storage two-layer planning optimization method described in Embodiment 1 above.

[0123] The relevant technical solutions are the same as above, and will not be repeated here.

[0124] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A nested time-series distributed energy storage two-level planning optimization method, characterized in that, include: A two-layer planning optimization model for a nested time-series distributed energy storage system is constructed, comprising a lower-layer time-series operation optimization model and an upper-layer planning optimization model. The lower-level timing optimization model minimizes costs including charging and discharging costs. Voltage deviation penalty cost With energy storage degradation costs Taking the operating cost of the distributed energy storage system, including the above, as the objective function, As decision variables, As variables for energy storage deployment decisions, Indicates at node Install energy storage units. Indicates at node No energy storage unit is installed; and Representing nodes respectively The rated power and rated capacity of the energy storage units installed on the surface. , This represents the total number of nodes in the distribution network; the upper-level planning and optimization model takes minimizing the total cost of the distributed energy storage system within the planning period as its objective function, and uses the number of nodes as the criterion. The real-time charging and discharging power of the energy storage unit installed on the upper part is the decision variable; wherein, the total cost within the planning period includes the investment cost and the net present value of the operating cost determined by the operating cost fed back by the lower-level time-series operation optimization model; Under the operational constraints satisfied by the distributed energy storage system, the objective function of the upper-level planning optimization model is solved to optimize the planning scheme of the distributed energy storage system. Under the planning scheme, the objective function of the lower-level time-series operation optimization model is solved to optimize the time-series operation strategy of the distributed energy storage system.

2. The nested time-series distributed energy storage two-layer planning optimization method according to claim 1, characterized in that, When optimizing the planning scheme and the time-series execution strategy, the HPSO-SAC algorithm is used to achieve joint optimization of the planning scheme and the time-series execution strategy, including: S1. At the current iteration step, the HPSO algorithm is used to solve the objective function of the upper-level planning optimization model to obtain the planning scheme at the current iteration step; wherein, each particle in the HPSO algorithm Corresponding to a set of decision variables To characterize the planning scheme, Indicates the first Particles Corresponding decision variables , Indicates the size of the particle swarm; S2. Under the planning scheme at the current iteration step, the SAC algorithm is used to solve the objective function of the lower-level temporal operation optimization model to obtain the temporal operation strategy at the current iteration step. The state space of the agent in the SAC algorithm is... and action space They are respectively: In the formula, For nodes Real-time energy status of the energy storage unit installed on it. For nodes At any moment per-unit voltage, For nodes Load demand; and The outputs are wind power and photovoltaic power, respectively. Time-of-use pricing; Represents a node The real-time charging and discharging power of the energy storage unit installed on it represents the timing operation strategy; Represents a node The energy storage unit installed on it is always Rated power; The charging and discharging cost Voltage deviation penalty cost With energy storage degradation costs By using weighted summation as the immediate reward function, the temporal execution strategy at the current iteration step is obtained when the agent converges. ; S3. Timing execution strategy based on the current iteration step. Calculate the current time-series operating cost and convert it into the net present value of the operating cost, then feed it back to the upper-level planning and optimization model to update the objective function of the upper-level planning and optimization model; S4. Determine whether the preset number of iterations has been reached. If not, return to step S1 to proceed to the next iteration. If yes, output the final planning scheme and timing strategy.

3. The nested time-series distributed energy storage two-layer planning optimization method according to claim 2, characterized in that, The HPSO algorithm is used to solve the objective function of the upper-level planning optimization model to obtain the planning scheme at the current iteration step, including: S11, based on the particles at the current moment Update the objective function of the upper-level planning and optimization model, and update the particles based on the objective function. The velocity and position of the updated particles are determined, and crossover and mutation operations are performed on the updated particles to obtain the particles for the next time step. S12. Determine whether the planning scheme represented by the particle at the next moment satisfies the running constraints. If not, take the particle at the next moment as the particle at the current moment and return to step S11 to update the particle again. If yes, obtain the planning scheme under the current iteration step.

4. The nested time-series distributed energy storage two-layer planning optimization method according to claim 3, characterized in that, particle The speed update method is as follows: in, Represents the internal particles of the HPSO algorithm At the present moment speed, , Indicates by Affected particles velocity components, Indicates by Affected particles velocity components, Indicates by Affected particles velocity components; Inertial weights; Indicates the current time particles ; , These are cognitive learning factors and social learning factors, respectively. , A random number uniformly distributed within the interval [0,1]. Indicates the local optimal position. Indicates the globally optimal position. and The objective function value is determined based on the current value of the upper-level planning and optimization model; Indicates the particle at the next moment speed; particle The location update method is as follows: in, , , They represent particles respectively At the present moment of , , ; , , They represent particles respectively In the next moment , , ; , , They represent respectively by , , Affected particles The velocity component at the next moment; This represents the threshold function.

5. The nested time-series distributed energy storage two-level planning optimization method according to any one of claims 1-4, characterized in that, The objective function of the upper-level planning optimization model is: in, This represents the total cost over the planning period. Indicates the installation node of the energy storage unit. This indicates the rated power of the energy storage unit. This indicates the rated capacity of the energy storage unit. This represents the investment cost. The net present value of the operating costs is obtained by discounting the operating costs fed back from the lower-level time-series operation optimization model to the beginning of the planning period and accumulating them over the planning period; the investment costs for: In the formula, Investment cost for rated power of energy storage, The investment cost for the rated capacity of energy storage; As variables for energy storage deployment decisions, Indicates at node Install energy storage units. Indicates at node No energy storage unit is installed; and Representing nodes respectively The rated power and rated capacity of the energy storage units installed on the surface. This indicates the total number of nodes in the distribution network.

6. The nested time-series distributed energy storage two-layer planning optimization method according to claim 5, characterized in that, The objective function of the lower-level time-series optimization model is: in, This represents the operating cost of a distributed energy storage system. Represents a node Real-time charging and discharging power of the energy storage unit installed on it; charging and discharging cost. Voltage deviation penalty cost With energy storage degradation costs They are respectively: In the formula, Indicates charging. Indicates discharge; , They are respectively The charging price and the discharge grid connection price at different times. and They are respectively at the node The charging and discharging efficiency of the energy storage unit installed on it. Indicates the scheduling time step. Total operating cycle; In the formula, For nodes At any moment per-unit voltage, , These are the upper and lower limits of voltage for normal operation of the distribution network, respectively. This is the cost factor for voltage offset penalty; In the formula, For nodes The cost per unit capacity of the energy storage units installed on the site, For nodes The rated cycle life of the energy storage unit installed on it.

7. The nested time-series distributed energy storage two-layer planning optimization method according to claim 6, characterized in that, The operational constraints include energy storage operation frequency constraints: in, Indicates the maximum permissible number of energy storage operation cycles. For binary indicator variables, defined as: In the formula, It is the introduced power change threshold.

8. A nested, time-series distributed energy storage two-level planning and optimization system, characterized in that, Includes computer-readable storage media and processors; The computer-readable storage medium is used to store executable instructions; The processor is used to read executable instructions stored in the computer-readable storage medium and execute the nested time-series distributed energy storage two-layer planning optimization method according to any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the nested time-series distributed energy storage two-level planning optimization method as described in any one of claims 1-7.

10. A computer program product, characterized in that, The method includes a computer program that, when run on a computer, causes the computer to execute the nested, time-series distributed energy storage two-layer planning optimization method as described in any one of claims 1-7.

Images