Day-ahead multi-stage stochastic optimization method and system for active power distribution network, medium and equipment

Abstract

The invention relates to a day-ahead multi-stage stochastic optimization method and system for an active power distribution network, a medium and equipment, and belongs to the technical field of power systems, and the method comprises the steps: constructing a day-ahead multi-stage optimization model of the active power distribution network; and solving the constructed day-ahead multi-stage optimization model of the active power distribution network to realize random optimization of the active power distribution network. According to the method, the day-ahead multi-period active and reactive power coordinated optimization problem of the active power distribution network is constructed into a multi-stage stochastic programming model, flexible adjustment potential of adjustable and controllable resources is mined by using a deep reinforcement learning algorithm, renewable energy sources can be consumed as far as possible, peak load shifting and power exchange with a superior power grid plan can be realized, and the optimal power distribution network can be obtained. And optimal economical efficiency of active power distribution network operation is pursued. According to the method, the flexible adjustment potential of the energy storage device and the grouped switched capacitor bank can be better exerted to cope with the uncertainty of renewable energy sources and loads, the deep reinforcement learning algorithm is adaptively modified, and the reliability and safety of the deep reinforcement learning algorithm are improved.

Description

Technical Field

[0001] The invention relates to a day-ahead multi-stage random optimization method, system, medium and equipment for an active power distribution network, belonging to the technical field of power systems. Background Art

[0002] The purpose of optimizing the day-ahead operation of active distribution networks is to implement active management and coordinated control of controllable resources such as distributed generation and energy storage, so as to absorb renewable energy locally as much as possible, reduce the pressure of regulation by the upper-level power grid, and pursue the optimal economic operation of the active distribution network.

[0003] In order to ensure global optimality within the daily optimization cycle, the day-ahead optimization of active distribution networks requires the day-ahead determination of the operation plans of adjustable resources such as energy storage devices and grouped switching capacitors with coupling constraints between time periods. However, it is difficult to give full play to the flexible adjustment capabilities of energy storage by limiting the working state of energy storage devices with a set of fixed day-ahead plans. At the same time, discrete control devices such as grouped switching capacitors also have the potential to be dynamically adjusted according to the power fluctuations of renewable energy during the day. The difficulty lies in how to ensure that the constraints on the number of daily adjustments are met.

[0004] In order to fully tap the flexible adjustment potential of controllable resources in the operation of active distribution networks, deep reinforcement learning, as a representative of the new generation of artificial intelligence algorithms, has been used in many studies to optimize and control the operation of active distribution networks, but there are still many problems to be solved in terms of reliability and safety. Therefore, adaptive modification of deep reinforcement learning algorithms based on the characteristics of specified power system optimization problems is the key to improving the reliability and safety of deep reinforcement learning algorithms. Summary of the invention

[0005] In response to the above problems, the present invention provides a multi-stage random optimization method, system, medium and equipment for an active distribution network a day ahead. The method of the present invention constructs the multi-period active and reactive coordinated optimization problem of the active distribution network a day ahead as a multi-stage random programming model, which can better exert the flexible adjustment potential of energy storage devices and grouped switching capacitor groups to cope with the uncertainty of renewable energy and loads, and adaptively modify the deep reinforcement learning algorithm, thereby improving the reliability and safety of the deep reinforcement learning algorithm.

[0006] The technical solution of the present invention is as follows:

[0007] A multi-stage stochastic optimization method for active distribution network day-ahead, comprising:

[0008] Construct a multi-stage optimization model for active distribution network;

[0009] Solve the constructed multi-stage optimization model of the active distribution network to achieve the stochastic optimization of the active distribution network.

[0010] Preferably, the stochastic optimization process is:

[0011] Firstly, the multi-stage stochastic programming problem in the multi-stage optimization model of the active distribution network is constructed as a Markov decision process to achieve the decoupling of decision-making time periods; then, according to the characteristics of the day-ahead optimization problem of the active distribution network, the power flow model and the uncertainty of renewable energy are decoupled, and the power flow model of the active distribution network is fitted with the help of neural network; then, the strategy function neural network is introduced to assist in solving the nonlinear optimization in the iterative process of the proposed improved value iteration algorithm; finally, in order to improve the stability of the proposed value iteration algorithm, the improved algorithm is improved by introducing engineering practice techniques that have been verified in practice, so that the algorithm can handle the coexisting discrete and continuous control quantities.

[0012] Preferably, the multi-stage optimization model of the active distribution network includes an objective function and constraints, and the objective function is:

[0013]

[0014] Where t represents the number of time periods, and t = 1, ..., T; x t is the decision amount in period t, And x 1:t =(x1,…,x t );ξ t is the uncertainty in period t, and the uncertainty And 1:t =(ξ1,...,ξ t ); x t is the decision quantity x t The definition domain of is related to the decisions made at all previous moments because the group switching capacitor bank has a daily adjustment limit. For the policy function π, use a deterministic policy function C T is the objective function of stage T; it can be seen that for the active power distribution network day-ahead active and reactive power coordinated optimization problem of the present invention, the stages correspond to the time periods one by one, the t stage corresponds to the t time period, and the decision quantity x of the t time period is t Need to be determined at time t-1 to deal with the uncertainty ξ in period t t .

[0015] Preferably, the constraints include branch flow model, safe operation constraints, renewable energy operation constraints, power constraints for interaction with the upper power grid, energy storage operation constraints, and group switching capacitor bank operation constraints.

[0016] Preferably, based on the historical measurement data of the power grid, a neural network is used to parameterize the power flow mapping relationship of the distribution network, so as to calculate the reward function, and then a value iteration algorithm is used to solve a multi-stage stochastic programming problem of a certain day-ahead prediction generation scenario tree;

[0017] And the reward function in the Markov decision process is optimized based on the power flow mapping relationship of the active distribution network power flow model fitted by the neural network.

[0018] Preferably, the strategy function aims to maximize the state-action value function Q.

[0019] Preferably, the process of improving the algorithm by introducing engineering practice techniques includes periodic updating of learning objectives, truncated double Q learning, and target strategy smoothing regularization;

[0020] The phased update learning objectives include: introducing an auxiliary neural network θ with the same structure as the neural networks θ and φ q and φ q , and synchronize the parameters with phased time differences, so that the algorithm training process becomes an optimization problem of fixed update target at each stage, which in disguise fixes the update target;

[0021] In order to achieve the phased time difference parameter update, the auxiliary neural network θ q and φ q The parameters of are updated using soft updates, and the update of the policy function is delayed, that is, the update frequency of φ is lower than that of θ;

[0022] Truncated double Q learning includes: introducing two parameters of the neural network, estimating the update target in parallel with the original parameters, and taking the larger of the two update targets as the actual update target, which can effectively reduce the estimation error;

[0023] The target strategy smoothing regularization includes: updating the target with the help of this regularized target strategy can suppress the overfitting of the state action value function Q and reduce the variance.

[0024] A multi-stage stochastic optimization system for an active power distribution network, comprising:

[0025] A building module for constructing a multi-stage optimization model for active distribution networks;

[0026] The optimization module is used to solve the constructed multi-stage optimization model of the active distribution network and realize the stochastic optimization of the active distribution network.

[0027] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned multi-stage random optimization method for active power distribution network day-ahead.

[0028] An electronic device comprises a memory, a processor and a computer program that can be run on the processor, wherein the computer program is stored thereon, and when the processor executes the computer program, the day-ahead multi-stage random optimization method for an active distribution network as described in any one of claims 1-7 is implemented.

[0029] For any details not provided in the present invention, please refer to the prior art.

[0030] The beneficial effects of the present invention are:

[0031] (1) The present invention proposes a multi-stage optimization model for active distribution network with coordinated optimization of active and reactive power in multiple time periods a day ahead, and an improved value iteration algorithm. By using a deep reinforcement learning algorithm to explore the flexible adjustment potential of adjustable resources, the economic optimization of active distribution network operation can be pursued while maximizing the absorption of renewable energy, shaving peaks and filling valleys, and exchanging power with the upper power grid plan.

[0032] (2) The multi-stage optimization model of the active distribution network of the present invention can more effectively tap the flexible adjustment potential of adjustable resources. The proposed algorithm can realize the transfer of the adjustable margin of the energy storage device between time periods, so as to better cope with the power fluctuations of renewable energy and load, and improve the reliability and economy of the operation of the active distribution network.

[0033] (3) The proposed improved value iteration algorithm uses a neural network to fit the active distribution network power flow model to achieve the decoupling of the system model and the uncertainty realization process, so that the information acquisition process of the deep reinforcement learning algorithm "exploration and trial and error" is realized with the help of the data-driven active distribution network power flow model, which improves the reliability and security of the algorithm.

[0034] (4) The improved value iteration algorithm of the present invention can process both continuous and discrete decision variables, and the introduced engineering practice techniques improve the stability of the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings in the specification, which constitute a part of the present application, are used to provide further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0036] Figure 1 It is a flow chart of the multi-stage random optimization method for active power distribution network of the present invention;

[0037] Figure 2 It is a schematic diagram of the multi-stage stochastic optimization system of the active power distribution network of the present invention. DETAILED DESCRIPTION

[0038] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of the present invention are clearly and completely described below in conjunction with the drawings in the implementation of this specification, but are not limited to this. Anything not fully described in the present invention shall be based on the conventional technology in the art.

[0039] Example 1

[0040] A multi-stage stochastic optimization method for active distribution networks, such as Figure 1 As shown, including:

[0041] Construct a multi-stage optimization model for active distribution network;

[0042] Solve the constructed multi-stage optimization model of the active distribution network to achieve the stochastic optimization of the active distribution network.

[0043] (I) The multi-stage optimization model of the active distribution network includes the objective function and constraints. The objective function is:

[0044]

[0045] Where t represents the number of time periods, and t = 1, ..., T; x t is the decision amount in period t, And x 1:t =(x1,…,x t );ξ t is the uncertainty in period t, and the uncertainty And 1:t =(ξ1,...,ξ t ); x t is the decision quantity x t The definition domain of is related to the decisions made at all previous moments because the group switching capacitor bank has a daily adjustment limit. For the policy function π, use a deterministic policy function C T is the objective function of stage T; it can be seen that for the active power distribution network day-ahead active and reactive power coordinated optimization problem of the present invention, the stages correspond to the time periods one by one, the t stage corresponds to the t time period, and the decision quantity x of the t time period is t Need to be determined at time t-1 to deal with the uncertainty ξ in period t t .

[0046] Preferably, the constraints include branch flow model, safe operation constraints, renewable energy operation constraints, power constraints for interaction with the upper power grid, energy storage operation constraints, and group switching capacitor bank operation constraints.

[0047] (1) Branch flow model:

[0048] The present invention generates sample data required by the deep learning algorithm by simulating the branch flow model. Based on Ohm's law and Kirchhoff's current law, and assuming that each phase of the active distribution network is balanced, the branch flow model of the active distribution network can be derived as shown in formula (2).

[0049]

[0050] Where P ij,t , Q ij,t , I ij,t are the active power, reactive power and current amplitude of branch ij at time t respectively; R ij , X ij are the resistance and reactance of branch ij respectively; U i,t is the voltage amplitude of node i at time t; is the reactive power of renewable energy at node i at time t; are the active and reactive power of the load at node i at time t respectively; is the energy storage active power of node i at time t; is the reactive power of the grouped switched capacitor group of node i at time t; i:i→j represents the node combination of all nodes connected to node j whose power flows to node j.

[0051] (2) Safety operation constraints:

[0052] The system safety operation constraints considered in the present invention include node voltage constraints and branch current constraints.

[0053]

[0054] In the formula, U are the upper and lower limits of the node voltage amplitude respectively; is the upper limit of the current amplitude of branch ij.

[0055] (3) Renewable energy operation constraints:

[0056] The reactive power output of the renewable energy generator set of the present invention takes into account the limitations of inverter capacity and power factor.

[0057]

[0058] In the formula, is the capacity of the renewable energy inverter at node i; cosφ i,t is the renewable energy output power factor of node i at time t.

[0059] (4) Interaction power constraints with the upper grid:

[0060]

[0061] In the formula, P G are the upper and lower limits of the active power transmitted from the upper power grid to the distribution network during period t; Q t G is the reactive power transmitted from the upper power grid to the distribution network during period t; Q G They are the upper and lower limits of reactive power transmitted from the upper power grid to the distribution network during period t.

[0062] (5) Energy storage operation constraints:

[0063] The present invention takes into account the capacity of energy storage, charging and discharging efficiency, maximum output power constraint and energy storage power constraint at the end moment.

[0064]

[0065]

[0066]

[0067]

[0068] In the formula, is the energy storage capacity of node i at time t; η i C , η i D is the energy storage charging and discharging efficiency of node i; are the upper and lower limits of the energy storage active output of node i respectively; are the upper and lower limits of energy storage capacity of node i respectively; The energy storage capacity of node i at the end of the moment; The energy storage value of node i at the last moment.

[0069] (6) Operation constraints of group switching capacitor banks:

[0070] The present invention takes into account the constraints of the maximum number of groups of grouped switching capacitors put into operation and the maximum number of daily adjustments.

[0071]

[0072]

[0073]

[0074] In the formula, is the number of capacitor groups in operation at node i at time t; is the reactive output of each group of capacitors at node i; is the upper limit of the number of capacitor groups put into operation at node i; The maximum number of times that can be adjusted within a day.

[0075] (II) Solve the constructed multi-stage optimization model of the active distribution network to achieve the stochastic optimization of the active distribution network. The stochastic optimization process is:

[0076] Firstly, the multi-stage stochastic programming problem in the multi-stage optimization model of the active distribution network is constructed as a Markov decision process to achieve the decoupling of decision-making time periods; then, according to the characteristics of the day-ahead optimization problem of the active distribution network, the power flow model and the uncertainty of renewable energy are decoupled, and the power flow model of the active distribution network is fitted with the help of neural network; then, the strategy function neural network is introduced to assist in solving the nonlinear optimization in the iterative process of the proposed improved value iteration algorithm; finally, in order to improve the stability of the proposed value iteration algorithm, the improved algorithm is improved by introducing engineering practice techniques that have been verified in practice, so that the algorithm can handle the coexisting discrete and continuous control quantities.

[0077] 1. Constructing the multi-stage stochastic programming problem as a Markov decision process

[0078] 1) State quantity s t :

[0079] For the multi-stage stochastic programming model of formula (1), the initial state variables include time period t and Among them, the state quantity The existence of makes it possible to solve the energy storage decision variables in stage t It is not necessary to know the historical decisions in the [1, t-1] stage Therefore, the same treatment is carried out on the group switching capacitor bank, and the state variable is introduced Used to record the number of adjustments used by the capacitor bank in the [1, t-1] phase. Introducing state quantity So that the energy storage decision variables in stage T are solved No need to know historical decisions Similarly, introduce state variables This means that there is no need to know historical decision results when solving the decision-making quantity of group switching capacitors.

[0080]

[0081] At the same time, the uncertainty ξ t Introduce into the state quantity. The reconstructed state quantity is:

[0082]

[0083] The state quantity is mainly composed of two parts. One part is used to determine the realization of the uncertainty quantity in stage t. t, and the other part is an auxiliary state used to reflect the impact of stage [1, t-1] on the decision of stage t and

[0084] 2) Movement amount x t :

[0085] Movement amount x t The variables involved are the same as the decision variables involved in the multi-stage stochastic programming problem.

[0086] 3) State transition probability

[0087] For the coordinated optimization problem of active and reactive power in active distribution network, the uncertainty ξ t The realization of is not affected by the decision. Therefore, the state transition probability Directly corresponds to the uncertainty ξ t The probability distribution of .

[0088] 4) Reward function r t :

[0089] The safe operation constraints and the final energy storage capacity constraints are converted into penalty functions as reward functions r t Part of. Construct the reward function r t as follows:

[0090]

[0091]

[0092]

[0093]

[0094] Where (x) + =max(0,x); μ1, μ2 and μ3 are penalty function scaling factors.

[0095] 5) Value function:

[0096] Dynamic programming decouples the stage before the current decision node by transforming the multi-stage stochastic programming problem into a Markov decision process, but it is still coupled with the stage after the current decision node. To this end, dynamic programming introduces the concept of value function to balance the cost of the current stage and the future stage, and organize and structure the strategy search. The value function V corresponding to a given strategy π t π The definition is as follows:

[0097]

[0098] Value function V t π It indicates the objective function value of the decision tree sub-branch of the decision node corresponding to the state s when the policy function is given. Another commonly used value function form is the state-action value function. The state-action value function Q corresponding to a given policy π is t π The definition is as follows:

[0099]

[0100] In the formula, (s, x) is a given state-action pair, corresponding to a given uncertainty realization node in the decision tree. State-action value function Q t π It means that when the policy function is given, the uncertainty corresponding to the state-action pair (s, x) realizes the objective function value of the node decision tree sub-branch.

[0101] 2. Fitting the active distribution network power flow model with the help of neural network

[0102] A data-driven solution method based on an improved value iteration algorithm is proposed for the multi-stage optimization model of the active distribution network. This method decouples the system model and the probability distribution of uncertain quantities, and uses a neural network to parameterize the distribution network flow mapping relationship, so that the information acquisition of deep reinforcement learning "exploration and trial and error" is realized through the power system day-ahead forecast and the distribution network flow model fitted by the neural network. This method does not need to solve the entire multi-stage stochastic programming problem in a data-driven way. It only needs to use a neural network to parameterize the distribution network flow mapping relationship based on the historical measurement data of the power grid to calculate the reward function, and then use the value iteration algorithm to solve the multi-stage stochastic programming problem of a certain day-ahead forecast to generate a scenario tree.

[0103] The proposed active distribution network day-ahead optimization problem uses the power flow equation to solve the network loss power P t loss , node voltage U i,t and line current I ij,t . Since the solution is obtained by deep reinforcement learning, the nonlinear power flow relationship can be directly fitted by neural network to avoid the error caused by linearization. At the same time, because Δt = 1h, the proposed active distribution network day-ahead optimization problem satisfies the following assumptions: 1) The uncertainty ξ t Phase independence; 2) The power flow does not consider transient processes. Therefore, it can be considered that the power flow mapping relationship is independent of the time period t. The power flow equation approximated by the neural network f parameterized by ψ is as follows:

[0104]

[0105] With the help of this parameterized power flow mapping relationship, the reward function r t Rewritten as:

[0106] r t (s t ,x t |ψ) (22)

[0107] 3. Fitting the policy function with the help of neural network

[0108] Select the borrowed value iteration algorithm, use the state action value function Q to solve the control strategy, and use the neural network parameterized by θ The approximate state-action-value function is as follows:

[0109]

[0110] The following neural network π approximate policy function parameterized by φ is introduced to determine the optimal strategy under different states and achieve the update target Fast calculation of .

[0111]

[0112] With the help of the strategy function approximated by the neural network π, equations (24)-(25) are rewritten as follows (27)-(28):

[0113]

[0114]

[0115]

[0116]

[0117] By introducing the policy function, the Actor-Critic structure in deep reinforcement learning is formed. However, the policy function of the present invention is only used to assist in solving nonlinear optimization. Therefore, the policy function update does not use the policy gradient method, but aims to maximize the state action value function Q. The update iteration formula of the policy function parameter φ is given as follows:

[0118]

[0119] 4. Introduce common engineering techniques to improve algorithms

[0120] The innovations of the series of value iteration algorithms represented by DQN, DDPG and TD3 are mostly proven engineering practice techniques. Therefore, some proven engineering practice techniques are introduced to increase the stability of the algorithm.

[0121] 1) Update learning objectives periodically

[0122] The update method of forward sampling iteration in reinforcement learning makes the update target of neural network parameters change in each iteration. This variable target optimization process seriously affects the stability of the algorithm. To this end, an auxiliary neural network θ is introduced, which has the same structure as neural networks θ and φ. q and φ q , and synchronize the parameters with phased time difference, so that the algorithm training process becomes an optimization problem of fixed update target at each stage, which in disguise fixes the update target. q and φ q The parameterized neural network is used to calculate the update target of the state action value function Q

[0123]

[0124] In order to achieve the phased time difference parameter update, the auxiliary neural network θ q and φ q The parameters of are updated using soft updates. At the same time, in order to reduce the correlation between the state action value function Q and the policy function update, the policy function update is usually delayed, that is, the update frequency of φ is lower than θ. Therefore, m is used to represent the parameter θ q ,φ q The number of iterations of φ and θ is used to distinguish the update frequency different from that of the parameter θ. q and φ q The update iteration formula is as follows:

[0125] θ q,m =τθ m +(1-τ)θ q,m-1 (31)

[0126] φ q,m =τφ m +(1-τ)φ q,m-1 (32)

[0127] Where τ is the soft update rate. The proposed algorithm is a different strategy implementation method, which can learn the control strategy with the help of historical measurement data. Therefore, a memory library is established, and the storage-random sampling method is adopted to break the correlation between data, which meets the requirements of deep learning for data independence and identical distribution. While improving data utilization, the algorithm stability is further improved. Therefore, the update iteration formula of the state action value function parameter θ and the strategy function parameter φ can be improved as follows:

[0128]

[0129]

[0130] In the formula, The set of samples extracted from the memory bank for the nth iteration; and Respectively represent The state, decision and update target of the jth memory sample in β m-1 is the update step size of the policy function.

[0131] 2) Truncated Double Q Learning

[0132] Introduce another set of neural networks θ2 and With the original parameters θ1 and Parallel estimation updates the target q t n , taking the larger of the two update targets as the actual update target can effectively reduce the estimation error. Rewritten as:

[0133]

[0134] Neural network θ2 and The parameter update method is the same as θ1 and θ1 q Same, and uses the same update target

[0135] 3) Target strategy smoothing regularization

[0136] A regularization strategy is introduced to smooth the target strategy, which can suppress the overfitting of the state action value function Q and reduce the variance. With this regularized target strategy, the target is updated. Rewritten as:

[0137]

[0138]

[0139] Where ε is the noise of the smoothing target strategy; The table shows random numbers generated by a normal distribution with an expected value of 0 and a standard deviation of σ1; clip means that the noise ε is truncated within the range of [-c, c].

[0140] 5. Dealing with the coexistence of discrete and continuous control quantities

[0141] In the strategy function parameter update method of formula (32), The existence of makes this parameter update method only valid for continuous control variables. is a discrete variable and cannot be used to make decisions with the help of a policy function. To solve this problem, the control variable x t Discrete Control Quantity With continuous control The two parts are as follows:

[0142]

[0143]

[0144]

[0145] In the formula, is the control variable x t The discrete control quantity part of is the control variable x t The continuous control part of the neural network. The policy function parameterized by the neural network φ is only used to output the continuous control quantity. Therefore, the policy function parameterized by the neural network φ is rewritten as:

[0146]

[0147] Using this continuous control variable x t p The strategy function of , rewrite equations (24) and (25) as:

[0148]

[0149]

[0150]

[0151] At the same time, the update iteration formula of the policy function parameter φ is rewritten as follows:

[0152]

[0153] In the formula, represent The discrete control amount of the jth memory sample in .

[0154] The multi-stage optimization model of the active distribution network of the present invention is a discrete model. The Markov decision theory can be referred to. The scheduling problem of the entire model is decomposed into stages one by one. A decision can be made at each stage, that is, the original complex problem is divided into simple sub-problems. After the division, the scheduling problem is easier to solve. The Markov decision process mainly includes these elements: decision moment, decision stage state, decision strategy, transition probability, reward and criterion. In the decision-making process, the decision maker needs to make the best decision at the decision moment, which in turn affects the transition probability of the system and makes the operation of the system meet the predetermined criteria.

[0155] The invention constructs the multi-stage stochastic programming problem as a Markov decision process, which is to construct the optimization scheduling problem (1)-(12) into a Markov decision process, so as to achieve decoupling between decision time periods and facilitate solution.

[0156] The purpose of fitting the active distribution network power flow model with the help of neural network is to decouple the power flow model from the uncertainty of renewable energy according to the characteristics of the day-ahead optimization problem of the active distribution network, and to approximate the active distribution network power flow model with the help of neural network, and to rewrite the reward function in the Markov decision process accordingly.

[0157] The role of introducing the strategy function neural network is to assist in solving the nonlinear optimization in the iterative process of the proposed improved value iteration algorithm.

[0158] Finally, in order to improve the stability of the proposed value iteration algorithm, some engineering practice techniques that have been verified in practice are introduced, and the algorithm is enabled to handle coexisting discrete and continuous control quantities.

[0159] Example 2

[0160] A multi-stage stochastic optimization system for active distribution networks, such as Figure 2 , including a building block for constructing a day-ahead multi-stage optimization model for active distribution networks;

[0161] The optimization module is used to solve the constructed multi-stage optimization model of the active distribution network and realize the stochastic optimization of the active distribution network.

[0162] Example 3

[0163] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements a multi-stage random optimization method for an active distribution network a day ahead. The detailed steps of the optimization method are the same as those in Example 1 and are not repeated here.

[0164] Example 4

[0165] An electronic device includes a memory, a processor and a computer program that can be run on the processor, wherein the computer program is stored thereon, and when the processor executes the computer program, the multi-stage random optimization method for the active distribution network according to Example 1 is implemented.

[0166] The present invention constructs the multi-period active and reactive power coordinated optimization problem of the active distribution network as a multi-stage stochastic programming model. It models the process of gradually determining the actual output and actual load size of renewable energy over time as a scenario tree, and provides multiple groups of alternative strategies for energy storage devices and group switching capacitor groups that meet the energy storage power constraints and the daily adjustment number constraints of group switching capacitor groups, so that its day-ahead operation plan can be dynamically adjusted with the help of online measurement to adapt to the power fluctuation of renewable energy and loads. Its decision results are more accurate and targeted, and can better give play to the flexible adjustment potential of energy storage devices and group switching capacitor groups to cope with the uncertainty of renewable energy and loads; in view of the fact that the power fluctuation of renewable energy and loads in the day-ahead operation optimization problem of active distribution networks is not affected by the operation plan of adjustable resources, the system model and the realization process of uncertainty are decoupled, so that the information acquisition process of "exploration and trial and error" of the deep reinforcement learning algorithm is realized with the help of data-driven active distribution network power flow model, which improves the reliability and security of the algorithm; an improved value iteration algorithm is proposed, which improves the stability of the algorithm by introducing mature engineering practice techniques such as phased parallel time difference parameter update and target strategy smoothing regularization, and enables the algorithm to process continuous and discrete decision variables at the same time.

[0167] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A multi-stage random optimization method for active distribution network, characterized in that: include: Construct a multi-stage optimization model for active distribution network; Solve the constructed multi-stage optimization model of the active distribution network to achieve the stochastic optimization of the active distribution network.

2. The multi-stage random optimization method for active power distribution network according to claim 1, characterized in that: The stochastic optimization process is: Firstly, the multi-stage stochastic programming problem in the multi-stage optimization model of the active distribution network is constructed as a Markov decision process to achieve the decoupling of decision-making time periods; then, according to the characteristics of the day-ahead optimization problem of the active distribution network, the power flow model and the uncertainty of renewable energy are decoupled, and the power flow model of the active distribution network is fitted with the help of neural network; then, the strategy function neural network is introduced to assist in solving the nonlinear optimization in the iterative process of the proposed improved value iteration algorithm; finally, in order to improve the stability of the proposed value iteration algorithm, the improved algorithm is improved by introducing engineering practice techniques that have been verified in practice, so that the algorithm can handle the coexisting discrete and continuous control quantities.

3. The multi-stage random optimization method for active distribution network according to claim 2 is characterized in that: The multi-stage optimization model of the active distribution network includes the objective function and constraints. The objective function is: Where t represents the number of time periods, and t = 1, ..., T; x t is the decision amount in period t, And x 1:t =(x1,...,x t );C T is the objective function of stage T; ξ t is the uncertainty in period t, and the uncertainty And 1:t =(ξ1,...,ξ t ); x t is the decision quantity x t The definition domain of is related to the decisions made at all previous moments because the group switching capacitor bank has a daily adjustment limit. For the policy function π, use a deterministic policy function 4. The multi-stage random optimization method for active power distribution network according to claim 3 is characterized in that: The constraints include branch power flow model, safe operation constraints, renewable energy operation constraints, power constraints for interaction with the upper-level power grid, energy storage operation constraints, and group switching capacitor bank operation constraints.

5. The multi-stage random optimization method for active power distribution network according to claim 4 is characterized in that: Based on the historical measurement data of the power grid, a neural network is used to parameterize the power flow mapping relationship of the distribution network to calculate the reward function, and then a value iteration algorithm is used to solve the multi-stage stochastic programming problem of a certain day-ahead prediction generation scenario tree; And the reward function in the Markov decision process is optimized based on the power flow mapping relationship of the active distribution network power flow model fitted by the neural network.

6. The multi-stage random optimization method for active power distribution network according to claim 5, characterized in that: The policy function aims to maximize the state-action value function Q.

7. The multi-stage random optimization method for active power distribution network according to claim 6, characterized in that: The process of improving the algorithm by introducing engineering practice techniques includes periodic updating of learning objectives, truncated double Q learning, and target strategy smoothing regularization; The phased update learning objectives include: introducing an auxiliary neural network θ with the same structure as the neural networks θ and φ q and φ q , and synchronize the parameters with phased time differences, so that the algorithm training process becomes an optimization problem of fixed update target at each stage, which in disguise fixes the update target; In order to achieve the phased time difference parameter update, the auxiliary neural network θ q and φ q The parameters of are updated using soft updates, and the update of the policy function is delayed, that is, the update frequency of φ is lower than that of θ; Truncated double Q learning includes: introducing two parameters of the neural network, estimating the update target in parallel with the original parameters, and taking the larger of the two update targets as the actual update target, which can effectively reduce the estimation error; The target strategy smoothing regularization includes: updating the target with the help of this regularized target strategy can suppress the overfitting of the state action value function Q and reduce the variance.

8. A multi-stage random optimization system for active distribution network, characterized in that: include: A building module for constructing a multi-stage optimization model for active distribution networks; The optimization module is used to solve the constructed multi-stage optimization model of the active distribution network and realize the stochastic optimization of the active distribution network.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the day-ahead multi-stage random optimization method for an active power distribution network as described in any one of claims 1 to 7 is implemented.

10. An electronic device comprising a memory, a processor and a computer program that can be run on the processor, wherein the computer program is stored on the processor, characterized in that: When the processor executes the computer program, the multi-stage random optimization method for the active distribution network according to any one of claims 1-7 is implemented.

Images