High-voltage distribution network multi-stage stochastic programming method based on approximate dynamic programming

Abstract

The invention discloses a high-voltage distribution network multi-stage stochastic programming method based on approximate dynamic programming, which comprises the following steps of: firstly, constructing a multi-stage stochastic programming model based on a Markov decision process, and solving by adopting an approximate dynamic programming algorithm; the multi-stage stochastic programming model takes the minimum total cost as the target, the investment cost and the operation maintenance cost are considered, and the system reliability cost is also considered in the operation cost except the maintenance cost, the electricity purchase cost and the like. The model divides a planning period into a plurality of stages, each stage comprises several years, a plurality of load growth uncertainty scenes are generated, and a planning model considering the multi-stage load development uncertainty is established; the solving method adopts an approximate dynamic programming algorithm based on a state after decision, and an approximate value function table is obtained by iteratively simulating conversion between scenes in different stages. Then, the problem is converted into a single-stage optimization problem to be solved, and a multi-stage planning scheme is obtained.

Description

Technical Field

[0001] The invention belongs to the technical field of multi-stage stochastic programming of high-voltage distribution networks, and in particular relates to a multi-stage stochastic programming method of high-voltage distribution networks based on approximate dynamic programming. Background Art

[0002] In recent years, the development of distribution systems has received increasing attention. After continuous construction and transformation, the development level of distribution network has been greatly improved. However, there is still a lot of room for improvement in reliability and economy. In the context of building a new power system, for the new grid structure of high-voltage distribution network proposed recently, considering the future development needs of cities, the trend of large-scale access to renewable energy, and the development and application of new technologies and equipment, it is also necessary to study the planning method of the grid to provide theoretical support for the large-scale application of the grid. Summary of the invention

[0003] In order to solve the above problems, the present invention provides a multi-stage stochastic planning method for a high-voltage distribution network based on approximate dynamic programming, which can continuously adjust and optimize decision-making strategies according to actual data and feedback, and better cope with dynamically changing systems and environments.

[0004] The present invention is specifically a multi-stage stochastic programming method for a high-voltage distribution network based on approximate dynamic programming, and the multi-stage stochastic programming method for a high-voltage distribution network comprises the following steps:

[0005] Step (1): Establish a model based on the Markov decision process to generate an investment decision tree;

[0006] Step (2): Considering the randomness of load development and the economic efficiency of distribution system investment and operation, a multi-stage stochastic programming model is established, and relevant constraints are established. The objective function aims to minimize the total cost, which includes investment and operation costs.

[0007] Step (3): Use an approximate dynamic programming algorithm based on the post-decision state to solve.

[0008] The first part of the objective function in step (2) is the investment cost, including feeder construction cost, substation construction and expansion cost, transformer construction cost, new energy construction cost and energy storage construction cost; the second part of the objective function is the operating cost, including the cost of purchasing electricity from the superior power grid, distributed energy generation, equipment maintenance, load shedding, abandoned light and reliability, among which the reliability cost is obtained by calculating the amount of electricity that cannot be supplied to users due to system component failure.

[0009] For different equipment in the system, it is assumed that the annual investment cost is the same during the normal operation period. The calculation formula is shown in formula (1), where γ is the inflation rate and LT is the equipment life:

[0010]

[0011] After calculating the annual cost, the annual cost is converted to the current period through the present value factor. The calculation formula is shown in formula (2), where u and t represent the current stage and the number of years in the current stage respectively, and T represents the total number of years in each stage.

[0012]

[0013] On this basis, the objective function is defined as shown in formula (3): The optimization goal is to minimize the present value of the total cost, including the investment cost in the tth year of stage u Operation and maintenance costs in the sth quarter of the tth year of stage u Production costs Cost of power outage Reliability Cost

[0014]

[0015] To make the model more practical, the planning period is divided into U stages, each stage contains Y years, each year contains S quarters, and 12 typical operation scenarios including daytime and nighttime are selected in each quarter; it is assumed that only one investment will be made in each stage within Y years, that is, the investment cost in the same stage is the same; and the investment decision in each stage is related to the specific load situation, so the investment cost in different stages is different; the construction investment cost includes the investment cost of the line, substation, renewable energy, and transformer, as shown in formula (4), where Ω u ,Ω t ,Ω s They are respectively the planning stage set, the year set included in each stage, and the quarter set. are the number of lines, substations, distributed generation and transformers to be built, respectively, L is the line construction variable, x DG is the variable for distributed generation construction, x sub is the substation construction variable, x TR For transformer construction variables, L n is the length of the nth line, C I L is the line investment cost per unit length, C I sub is the substation investment cost, C I DG is the distributed generation investment cost, C I TR The investment cost of the transformer.

[0016]

[0017] The operation and maintenance cost is related to the load demand and the operating status of distributed power in the specific scenario. In the planning model, the system operating status of the typical scenario is first obtained, and then the annual operation and maintenance cost can be obtained according to the duration of different scenarios. The operation and maintenance cost is shown in formula (5), including the maintenance cost of the line, the maintenance cost of the new transformer, and the maintenance cost of the distributed power.

[0018] Where N L is the number of all lines, y + ,y - It is a 0-1 variable, indicating whether the direction of the power flowing through the line is forward or reverse.

[0019]

[0020] Formula (6) is the production cost, where C L M is the line maintenance cost per unit length, C TR M is the transformer maintenance cost, C DG M is the maintenance cost of distributed generation, C pp is the electricity purchase cost, LD is the current scenario hours, and LF is the substation power factor; the production cost includes the substation's electricity purchase cost and photovoltaic power generation cost. Since the electricity purchase cost is calculated based on the power flowing into the network through the substation, the network loss cost is already included in the production cost and is no longer listed and calculated separately.

[0021]

[0022] Formula (7) shows the power shortage cost, including load shedding cost and solar curtailment cost. node is the number of nodes, is the abandoned light power, Scutload is the load shedding power, is the substation power, C P E They are the abandoned light cost and the load interruption cost respectively.

[0023]

[0024] The reliability cost quantifies the annual amount of electricity that is not provided due to network failures, and can be expressed as the cost of not being able to supply electricity to users due to failures in feeder and other components within the system. For the high-voltage distribution network studied in this paper, the lines between the switch stations of the backbone network can be regarded as feeder segments. Since the entire line is equipped with circuit breakers and distribution system automation equipment, after a feeder segment fails, the upstream load is directly transferred without power outage, and the downstream is restored through power transfer. The reliability cost calculation can be simplified to only consider the impact of the downstream of the fault. The load that needs to be transferred downstream of the fault is the power transmitted on the feeder segment before the fault, the power outage time downstream of the fault is the transfer time, and the power outage probability downstream of the fault is the failure probability of the current fault feeder segment. The reliability cost is specifically shown in formula (8).

[0025]

[0026] where λ n is the failure rate of the nth line, μ n is the maintenance or transfer time of the nth line, P l,n is the transmission power of the nth line, LD is the current scenario hour, C P E is the load interruption cost. The reliability index calculation during the planning period can be obtained by the reliability cost obtained by formula (8). First, the regional annual total electricity consumption minus the power supply loss due to the fault is calculated to obtain the regional annual supply electricity, and then divided by the regional annual total electricity consumption to obtain the average power supply reliability rate ASAI, as shown in formula (9):

[0027]

[0028] Where SAIDI is the average annual power outage time for users, T total P is the annual power supply time, generally 8760 hours. ave is the average annual power consumption of users, N cus is the number of users in the region, and E total is the total electricity consumption of the region.

[0029] For the multi-stage dynamic programming model of the distribution network of the present invention, an equivalent calculation method of converting annual costs to the project start-up year is adopted to convert various costs generated each year to the project start-up year. In the process of calculating the total cost, the model must consider the time value of various costs in the entire planning cycle. For the construction cost of a one-time investment in various electrical equipment, it is necessary to first depreciate the assets and convert them into annual fixed investments, and then formulate a suitable equivalent calculation plan based on the bank interest rate together with the production, maintenance, and power shortage costs generated each year.

[0030] The constraints in step (2) include construction constraints, operation constraints, upper and lower limit constraints of components, and network topology constraints. The specific constraints are as follows:

[0031] 1) Construction constraints

[0032] The construction constraints are to ensure that each construction variable meets the actual requirements. Once the electrical equipment is built in a certain stage, it will exist in the following stages. Formula (10) means that each line can only be built once in all stages. Once it is built in a certain stage, it will exist in the following stages. Formula (11) means that only one type of construction can be selected. The construction constraints corresponding to each type of electrical equipment are as follows.

[0033]

[0034] Among them, Ω a is the number of line types, x L is a 0-1 variable for line construction, u is the stage number, n is the line number, a is the number of line types, N stage is the total number of stages.

[0035] Formula (12) indicates that a transformer can only be built once at most. Once built in a certain stage, it will exist in the following stages. Formula (13) indicates that each node can only select one type of transformer for construction. Formula (14) indicates that a transformer can only be installed after the substation is newly built.

[0036]

[0037] Among them, Ω sub is the number of substation nodes, Ω TR is the number of transformers, x TR Construct a 0-1 variable for the transformer, x sub is a 0-1 variable for substation construction, u is the stage number, n is the transformer number, i is the node to be selected for the substation, G i is the maximum number of transformers installed in the substation, a is the number of transformer types, N stage is the total number of stages.

[0038] If a line is to be built, in order to prevent the line end from connecting to an empty node, the substation it is connected to must have been built, that is, only after the substation is built can the construction of the line connected to the substation be considered. The specific formula is shown in formula (15).

[0039]

[0040] Among them, x sub is a 0-1 variable for substation construction, u is the stage number, i is the substation, n is the line number connected to substation i, a is the number of transformer types, N stage is the total number of stages.

[0041] For distributed power sources, formula (16) indicates that once distributed power sources are built in a certain stage, they will exist in the following stages. Although distributed power sources have many advantages, the number of them connected to the distribution network also needs to be limited. Considering the uncertainty of renewable energy output, the access of a large number of renewable energy sources will have an adverse effect on the stability of the power system. The capacity constraint of distributed power generation construction is shown in formula (17). The access capacity of candidate nodes that can be connected to distributed power sources cannot exceed the maximum capacity limit, and the maximum capacity is limited by the load level and the penetration rate of renewable energy.

[0042]

[0043] Where Ω DG is the candidate distributed generation node set, N DG is the number of candidate distributed power generation nodes, a is the number of candidate distributed power generation types, n is the candidate distributed power generation node, N DG is the number of distributed generation connections, x DG For distributed generation construction variables, S DG is the rated capacity of distributed power supply, is the penetration rate of distributed generation, N node is the number of load nodes, S load is the load of each node.

[0044] In order to smooth the output power, distributed energy is matched with energy storage. Formula (18) indicates that the energy storage constructed in the previous stage will continue to exist in the next stage. Formula (19) indicates that the capacity of energy storage should be within the allowable upper and lower limits. Formulas (20)-(21) give the calculation formulas for the upper and lower limits of energy storage capacity at a certain node.

[0045]

[0046] Among them, Ω ESS is the energy storage node set, x ESS is the energy storage access variable, a is the number of candidate energy storage types, n is the energy storage candidate node, and u is the number of stages; Cp ESS,n is the energy storage access capacity of node n, is the minimum access capacity of energy storage for node n, The maximum access capacity of energy storage for node n; The minimum storage ratio required for photovoltaic operation. The maximum energy storage ratio configured for PV operation, S load,n is the load of node n.

[0047] 2) Operational constraints

[0048] This section gives the system operation constraints, including power balance constraints, voltage balance constraints, second-order cone constraints, line power constraints, distributed generation and energy storage constraints. For all formulas in this section, They represent the current stage, year, and quarter respectively. The operation constraints are met on typical days in each quarter and are not listed one by one for simplicity.

[0049] Power balance constraints include active power balance and reactive power balance constraints. According to Kirchhoff's voltage and current law and the law of conservation of energy, the power of each node in the system should be balanced, that is, the power injected by the line, substation, and distributed power supply to the node should be balanced with the load demand. The power balance constraints are as follows:

[0050]

[0051] Formula (22) represents the active power balance constraint, where n is the node number, u is the stage number, t is the year in the stage, and PF load is the power factor, is the load node power, The active power injected into the line, The active power injected by DG, Active power injected into energy storage, Inject active power of substation nodes into the upper power grid, or cut load active power.

[0052]

[0053] Formula (23) is the reactive power balance constraint, where n is the node number, u is the stage number, t is the year in the stage, and PF load is the power factor, The reactive power injected into the line, The reactive power injected by DG, Reactive power injected into energy storage, Inject reactive power of substation nodes into the upper power grid, or cut load reactive power.

[0054] For the operational lines, the voltage change at both ends of the line is related to the line voltage loss. The voltage difference vector at the beginning and end of the power line can be decomposed into the voltage difference transverse component ΔU and the longitudinal component δU. Since the value of the voltage difference transverse component ΔU is small, it can be omitted for further simplification, and only the longitudinal component δU is retained. For the problems in this section, only the lines through which power flows are meaningful for discussion, so the specific form of the line voltage drop balance constraint is shown in formula (24):

[0055]

[0056] Among them, i and j represent the starting and ending nodes of line n respectively, R n and X n denote the resistance and reactance of line n respectively, It represents the square of the voltage at the node i at the beginning of the line. It represents the square of the current in line n, Respectively represent the active and reactive power of line n.

[0057] The line current expression is shown in equation (25), where is the square of the current in line n, is the square of the voltage at node i at the beginning of line n. is the active and reactive power of the line.

[0058]

[0059] The second-order cone constraint is a non-convex and nonlinear constraint. The feasible domain is the surface of the second-order cone, which is a non-convex optimization model. In order to transform the problem into a mixed integer second-order cone programming problem for solution, the constraint can be relaxed as follows:

[0060]

[0061] The transmission power of the line should not exceed the rated capacity of the line. At the same time, the line operation variable is introduced to constrain the power of the unoperated line to zero. Only the operational line can have transmission power. The line power constraints are shown in equations (27)-(28).

[0062]

[0063] in represents the rated capacity of line n, Indicates whether line n is put into operation. They represent the active and reactive power flowing through line n respectively.

[0064] For energy storage, equation (29) indicates that the power of energy storage is less than the maximum charge and discharge power, and the energy storage should satisfy the discharge balance equation as shown in equation (30).

[0065]

[0066] Where P ESS,n is the energy storage charging and discharging power of node n, The minimum energy storage power for node n, The maximum power stored for node n; Indicates the energy storage charging power, Indicates the energy storage discharge power; η ch , η dis Indicates the charging and discharging efficiency of energy storage.

[0067] In order to ensure that the access of distributed power generation does not affect the stable operation of the power grid, the output power of distributed power generation must meet certain constraints, that is, the output of distributed power generation should be higher than the minimum value, as shown in equations (31)-(32), where χ is the minimum distributed power generation utilization rate of the system, It is the minimum utilization power of distributed power source.

[0068]

[0069]

[0070] During system operation, power transmission depends on the construction of the grid. Some lines may be built but not put into use. The uniqueness constraint of line power flow is shown in equation (33), and the constraint that power only exists in the built lines is shown in equation (34):

[0071]

[0072] in To indicate whether line n is in operation in the sth quarter of the tth year in phase u, For negative delivery, is the line construction variable, a is the line selection; Formula (33) prevents the situation where the power flow is both positive and negative at the same time, because at any time, the direction of power flow is certain; Constraint (34) indicates that the state of line operation within the stage should be consistent with the line construction status of the current stage, limiting the power flow to only exist on the built line.

[0073] 3) Upper and lower limit constraints

[0074] Upper and lower limit constraints refer to the working range that the system should limit during operation. Upper and lower limit constraints play a vital role in ensuring the safe and reliable operation of the distribution network, improving the power quality, and extending the service life of electrical equipment. The constraints are shown in equations (35)-(39), where u is the planning stage, n is the node number, t is the year of the current stage, and s is the current quarter.

[0075]

[0076] Formula (35) is the upper and lower limit constraints of the node voltage, indicating that the node voltage should be limited within the upper and lower limits. is the lower limit of node voltage, is the upper limit of node voltage.

[0077] Formula (36) is the line current constraint. The line current should be within the line bearing capacity. The upper limit of the line current.

[0078] Formula (37) is the output constraint of distributed generation. The output of distributed generation should meet the power constraint and should not exceed the capacity limit. is the maximum output of the distributed generation at node n.

[0079] Formula (38) is the load shedding power constraint. The maximum load shedding power does not exceed the node load. is the load at node n; Equation (39) is the distributed generation power removal constraint, where is the maximum removal capacity of distributed generation, is the capacity of the distributed generation connected to node n.

[0080] 4) Network topology constraints

[0081] To ensure the safe and stable operation of the power distribution system, the network should have a radial topology when in operation, that is, a closed loop connected to an open loop. Secondly, it should also prevent the distributed power supply from being connected to the network, so that part of the load is only powered by the distributed power supply, thus being separated from the main grid to form an island, resulting in a decrease in the safety and reliability of operation.

[0082] According to the characteristics of the power distribution system, the radial topology belongs to a tree structure. Since the power is transmitted from top to bottom, the power flows out of the substation and enters the lower-level power grid. The substation node has no parent node, and there is no node that transmits power to it, as shown in formula (40). In order to ensure the power supply of the load node and the stability of the system, the load node has only one parent node to supply power to it, as shown in formula (41), where are the forward and reverse operation states of the line, respectively, .j is the number of nodes connected to node j, Ω l is the number of lines, S li,u The parent node of the node exists.

[0083]

[0084] At the same time, in the planning process, in order to prevent the power supply from losing connection with the main grid when it is put into operation in isolation, and to prevent the load from being separated from the main grid, it is necessary to introduce anti-islanding constraints, introduce the concept of virtual power, and build a virtual power system. Anti-islanding constraints include two parts: construction and operation, as follows.

[0085] The anti-islanding constraint during the construction process considers that the virtual injection power of each substation is greater than 0, and the virtual injection power of the node without a substation is 0, as shown in formula (42).

[0086]

[0087] At the same time, the virtual power demand of the nodes in the system defines the virtual load If the node has load demand or there is renewable energy power injection, the virtual load is 1, and the virtual load of other nodes is 0, as shown in formula (43), where is the virtual power flowing through line ij. According to the node power balance equation, the node balance equation of virtual power is established, as shown in equation (44). are the virtual power of distributed power sources and energy storage, are the virtual powers flowing into and out of node j respectively.

[0088]

[0089] The upper and lower limits of virtual power are shown in equations (45) to (46).

[0090]

[0091] For the network topology constraints of the operation plan, the line commissioning state variables are introduced, and the variables under the construction state in formula (46) are replaced by the variables under the operation state. The specific constraints are shown in formula (47).

[0092]

[0093] Network topology constraints can ensure that the currently operating distribution system is a radial network, avoiding closed-loop connection of lines. The system may contain multiple subnetworks powered by a single substation, avoiding the isolated operation of load points or distributed power sources.

[0094] In step (3), the approximate dynamic programming algorithm based on the post-decision state is used to solve the problem, which specifically includes:

[0095] (31) Approximation of the value function based on the post-decision state;

[0096] First, rewrite the above multi-stage planning model into the Bellman equation form, as shown in equation (48). t (S t ) is state S t The value function represents the sum of the costs from the current stage to the final stage, C t (S t ,A t ) represents the current stage of action A t The total cost, E[V t+1 (S t+1 |S t ,A t ,λ t )] represents the expected total cost of all future stages, which is a recursive equation.

[0097] V t (S t)=min{C t (S t ,A t )+E[V t+1 (S t+1 |S t ,A t ,λ t )]} (48)

[0098] The approximate dynamic programming algorithm based on value function approximation introduces the post-decision state That is, the system state after the decision maker has taken action and has not yet introduced the influence of the external environment, namely, the load forecast error. Its expression is shown in formula (49), which can be used to replace the sum of the expected costs of all future stages in formula (48). The replaced Bellman equation is shown in formula (50).

[0099]

[0100] The key to solving the above multi-stage planning model by using the approximate dynamic programming algorithm through the value function approximation method is to know the state after the decision In this way, the multi-stage planning model can be transformed into a single-stage deterministic planning problem. First, the mapping between the actions in each stage and the expected future costs is established. After the training value function, we can get The approximate value of , and then the optimal decision for each stage can be obtained by calculating formula (51).

[0101]

[0102] Formula (49) needs to obtain the approximate value of the state value function after the decision in each stage. The action space of each stage is certain. If the state space of each stage is large, that is, the number of load development forecast errors considered is large, it will still lead to calculation difficulties. Therefore, in order to simplify the calculation process, the number of state spaces of the above model is the same as the number of load development forecast error intervals, that is, each load forecast error maps a system state space, and each load forecast error can be solved by the above distribution system planning model, and then the approximate value function can be obtained by iteration.

[0103] (32) Value function iteration process

[0104] First, we need to calculate the approximate value of the value function. Solving the value function is an iterative process. The approximate dynamic programming algorithm recursively calculates the value function through the value function after the decision of the previous iteration, as shown in formula (52), where Indicates state S t The value function of the nth iteration and the optimal decision can be obtained by formula (53), and the value function after the decision in the previous stage can be obtained by formula (54).

[0105]

[0106] Where ε is the iteration step length, and its value range is (0,1). In multiple iterations, each state transition path is calculated, and in each iteration, the state passed in the previous stage will update the value function through equation (54). Through the above iterations, each state S of the system can be obtained t The corresponding value function approximation is obtained, thus converting the problem into a single-stage deterministic planning problem.

[0107] (33) Approximate dynamic programming solution process

[0108] The approximate dynamic programming algorithm based on the approximation of the post-decision state value function includes four parts: parameter reading, solving the action space, value function approximation, and finally solving the planning solution.

[0109] The action space can be solved directly using commercial solvers such as gurobi and Cplex, so that the optimal action under each load growth state can be obtained. Then, the randomness of load growth is simulated by the value function approximation method to obtain the approximate value of the value function. Finally, by solving the single-stage planning problem, the multi-stage planning results of the grid structure are obtained. When planning, decision makers can determine the optimal one based on the current load growth information obtained step by step, considering the uncertainty of future load growth.

[0110] Compared with the prior art, the beneficial effects are as follows: the multi-stage random programming method for high-voltage distribution networks addresses the problem of exponential growth of state space and action space that traditional planning methods may face when dealing with large-scale high-voltage distribution networks, resulting in high computational difficulty and high time cost. Approximate dynamic programming can effectively deal with high-dimensional and complex state spaces through approximate strategies and approximation functions, thereby maintaining high computational efficiency in large-scale high-voltage distribution network problems. In addition, in the face of environmental uncertainties in high-voltage distribution networks, traditional methods may need to pre-assume simpler random models or perform complex calculations to solve uncertainty problems, while approximate dynamic programming methods can deal with these uncertainties through learning and optimization, providing more robust solutions. In addition, the approximate dynamic programming method of this patent can continuously adjust and optimize decision-making strategies based on actual data and feedback, and can better cope with dynamically changing systems and environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0111] Figure 1 This is a schematic diagram of the investment decision tree of the present invention;

[0112] Figure 2 It is a schematic diagram of the iteration of the value function of the present invention;

[0113] Figure 3 This is a flow chart of the approximate dynamic programming algorithm of the present invention. DETAILED DESCRIPTION

[0114] The specific implementation method of a multi-stage stochastic programming method for a high-voltage distribution network based on approximate dynamic programming of the present invention is described in detail below in conjunction with the accompanying drawings.

[0115] In traditional planning methods, the future load development is certain. However, under the influence of multiple driving factors such as new technologies, new policies, and new concepts, the uncertainty of load development in the future distribution system will also increase greatly, and the grid structure also needs to be adjusted dynamically. For this random dynamic process, the present invention first establishes a model based on the Markov decision process to generate an investment decision tree; considering the randomness of load development and the economy of investment and operation of the distribution system, a multi-stage random programming model is established, and related constraints are established. The objective function takes the minimum total cost as the goal. The total cost includes investment and operating costs, and the system reliability cost is considered in the operating cost; in order to solve the "curse of dimensionality" problem caused by too many state variables and decision variables, an approximate dynamic programming algorithm based on the post-decision state is used to solve it; the example analysis is based on the typical structure of the high-voltage distribution network. Through the established multi-stage random programming model, a multi-stage planning scheme for the development of the regional distribution network to the high-voltage distribution network is obtained, and the effectiveness of the multi-stage random programming model is verified.

[0116] 1. Multi-stage planning model considering load development uncertainty:

[0117] Traditional distribution system planning mainly focuses on network topology and system internal lines and other components. With the large-scale access of new energy sources such as photovoltaic power generation, wind power generation, and energy storage, the complexity of distribution system planning models and the difficulty of solving them will increase. In view of the uncertainty of multi-stage load development in high-voltage distribution networks, this paper models the multi-stage dynamic decision-making process in distribution system planning based on the Markov decision process, and establishes a multi-stage stochastic planning model for high-voltage distribution networks with the goal of minimizing total cost.

[0118] The model takes into account the uncertainty of load development and the long time span. In order to improve the flexibility of the model, the planning period is divided into multiple stages. In each stage, decision makers will make investment decisions based on the specific load development. Therefore, there is uncertainty in the grid construction decisions in each stage in the future. The superposition of multiple decision possibilities will result in an investment decision tree. Figure 1 The figure shows a schematic diagram of the investment decision tree, where circles represent the state of each stage and lines represent possible state transition paths. It can be seen that the investment decision of each stage is uncertain under different load growth conditions. The decision maker first obtains the possible load growth information in stage 1, and then makes the optimal decision considering the current state and future state. After the decision maker implements the planning decision, it enters stage 2 to repeat the decision-making process of stage 1 and continue dynamic planning until the planning decision of all stages is completed.

[0119] Although only one decision variable is shown in the figure, a huge number of state conditions will evolve as the stages develop. The growth of state variables is exponential. If certain methods are not taken to reduce variables and simplify the model, it will lead to dimensional disasters, making the model solution time longer or even impossible to solve. For the above distribution network planning problem considering the uncertainty of multi-stage load development, the Markov decision process will be used to establish a mathematical model.

[0120] The Markov decision process is a mathematical programming method used to model real-time decision-making of dynamic multi-stage systems under random conditions. Each decision is made by the decision maker according to the time sequence after continuously observing the dynamic system with randomness.

[0121] 1.1 Objective Function

[0122] The multi-stage planning model established based on the above Markov decision process takes the minimum total cost as the optimization goal. The first part of the objective function is the investment cost, including the feeder construction cost, substation construction and expansion cost, transformer construction cost, new energy construction cost and energy storage construction cost; the second part of the objective function is the operating cost, including the cost of purchasing electricity from the superior power grid, distributed energy generation, equipment maintenance, load shedding, abandoned light and reliability, among which the reliability cost is obtained by calculating the amount of electricity that cannot be supplied to users due to system component failure.

[0123] Since the time span of this planning model is relatively long, it is necessary to use two parameters, capital recovery factor and present value factor, to convert future costs to the current stage. The capital recovery factor can calculate the present value of a certain amount of money, and the present value factor converts the cost of each future payment to the current year.

[0124] For different equipment in the system, it is assumed that the annual investment cost is the same during the normal operation period. The calculation formula is shown in formula (1), where γ is the inflation rate and LT is the equipment life:

[0125]

[0126] After calculating the annual cost, the annual cost is converted to the current period through the present value factor. The calculation formula is shown in formula (2), where u and t represent the current stage and the number of years in the current stage respectively, and T represents the total number of years in each stage.

[0127]

[0128] On this basis, the objective function is defined as shown in formula (3): The optimization goal is to minimize the present value of the total cost, including the investment cost in the tth year of stage u Operation and maintenance costs in the sth quarter of the tth year of stage u Production costs Cost of power shortage Reliability Cost

[0129]

[0130] To make the model more practical, the planning period is divided into U stages, each stage contains Y years, each year contains S quarters, and each quarter selects 12 typical operation scenarios including daytime and nighttime. It is assumed that only one investment will be made in each stage within Y years, that is, the investment cost in the same stage is the same. The investment decision in each stage is related to the specific load situation, so the investment cost in different stages is different. The construction investment cost includes the investment cost of equipment such as lines, substations, renewable energy, and transformers, as shown in formula (4), where Ω u ,Ω t ,Ω s They are respectively the planning stage set, the year set included in each stage, and the quarter set. are the number of lines, substations, distributed generation and transformers to be built, respectively, L is the line construction variable, x DG is the variable for distributed generation construction, x sub is the substation construction variable, x TR For transformer construction variables, L n is the length of the nth line, C I L is the line investment cost per unit length, C I sub is the substation investment cost, C I DG is the distributed generation investment cost, C I TR The investment cost of the transformer.

[0131]

[0132] The operation and maintenance cost is related to the load demand and the operating status of distributed power in the specific scenario. In the planning model, the system operating status of the typical scenario is first obtained, and then the annual operation and maintenance cost can be obtained according to the duration of different scenarios. The operation and maintenance cost is shown in formula (5), including the maintenance cost of the line, the maintenance cost of the new transformer, and the maintenance cost of the distributed power.

[0133] Where N L is the number of all lines, y + ,y - It is a 0-1 variable, indicating whether the direction of the power flowing through the line is forward or reverse.

[0134]

[0135] Formula (6) is the production cost, where C L M is the line maintenance cost per unit length, C TR M is the transformer maintenance cost, C DG M is the maintenance cost of distributed generation, C pp is the electricity purchase cost, LD is the current scenario hours, and LF is the substation power factor. The production cost includes the substation's electricity purchase cost and photovoltaic power generation cost. Since the electricity purchase cost is calculated based on the power flowing into the network through the substation, the network loss cost is already included in the production cost and is no longer listed and calculated separately.

[0136]

[0137] Formula (7) shows the power shortage cost, including load shedding cost and solar curtailment cost. node is the number of nodes, is the abandoned optical power, S cutload is the load shedding power, is the substation power, C P E They are the cost of abandoned light and the cost of load interruption respectively.

[0138]

[0139] The reliability cost quantifies the annual amount of electricity that is not provided due to network failures, and can be expressed as the cost of failing to supply electricity to users due to failures in feeder and other components within the system. For the high-voltage distribution network studied in this invention, the lines between the switch stations of the backbone network can be regarded as feeder segments. Since the circuit breakers and distribution system automation equipment are configured throughout the line, after a feeder segment fails, the upstream load is directly transferred without power outage, and the downstream is restored through transfer. The reliability cost calculation can be simplified to only consider the impact of the downstream of the fault. The load that needs to be transferred downstream of the fault is the power transmitted on the feeder segment before the fault, the power outage time downstream of the fault is the transfer time, and the power outage probability downstream of the fault is the failure probability of the current fault feeder segment. The reliability cost is specifically shown in formula (8).

[0140]

[0141] where λ n is the failure rate of the nth line, μ n is the maintenance or transfer time of the nth line, P l,n is the transmission power of the nth line, LD is the current scenario hour, C P Eis the load interruption cost. The reliability index calculation during the planning period can be obtained by the reliability cost obtained by formula (8). First, the regional annual total electricity consumption minus the power supply loss due to the fault is calculated to obtain the regional annual supply electricity, and then divided by the regional annual total electricity consumption to obtain the average power supply reliability rate ASAI, as shown in formula (9):

[0142]

[0143] Where SAIDI is the average annual power outage time for users, T total P is the annual power supply time, generally 8760 hours. ave is the average annual power consumption of users, N cus is the number of users in the region, and E total is the total electricity consumption of the region.

[0144] For the multi-stage dynamic programming model of the distribution network of the present invention, an equivalent calculation method of converting annual costs to the project start-up year is adopted to convert various costs generated each year to the project start-up year. In the process of calculating the total cost, the model must consider the time value of various costs in the entire planning cycle. For the construction cost of a one-time investment in various electrical equipment, it is necessary to first depreciate the assets and convert them into annual fixed investments, and then formulate a suitable equivalent calculation plan based on the bank interest rate together with the production, maintenance, and power shortage costs generated each year.

[0145] 1.2 Constraints

[0146] For the above multi-stage planning model, constraints that are compatible with the objective function are proposed to ensure the feasibility of the planning model. The constraints include construction constraints, operation constraints, upper and lower limit constraints of components, and network topology constraints. The specific constraints are as follows.

[0147] 1) Construction constraints

[0148] The construction constraints are to ensure that each construction variable meets the actual requirements. Once the electrical equipment is built in a certain stage, it will exist in the following stages. Formula (10) means that each line can only be built once in all stages. Once it is built in a certain stage, it will exist in the following stages. Formula (11) means that only one type of construction can be selected. The construction constraints corresponding to each type of electrical equipment are as follows.

[0149]

[0150] Among them, Ω a is the number of line types, x L is a 0-1 variable for line construction, u is the stage number, n is the line number, a is the number of line types, N stage is the total number of stages.

[0151] Formula (12) indicates that a transformer can only be built once at most. Once built in a certain stage, it will exist in the following stages. Formula (13) indicates that each node can only select one type of transformer for construction. Formula (14) indicates that a transformer can only be installed after the substation is newly built.

[0152]

[0153] Among them, Ω sub is the number of substation nodes, Ω TR is the number of transformers, x TR Construct a 0-1 variable for the transformer, x sub is a 0-1 variable for substation construction, u is the stage number, n is the transformer number, i is the node to be selected for the substation, G i is the maximum number of transformers installed in the substation, a is the number of transformer types, N stage is the total number of stages.

[0154] If a line is to be built, in order to prevent the line end from connecting to an empty node, the substation it is connected to must have been built, that is, only after the substation is built can the construction of the line connected to the substation be considered. The specific formula is shown in formula (15).

[0155]

[0156] Among them, x sub is a 0-1 variable for substation construction, u is the stage number, i is the substation, n is the line number connected to substation i, a is the number of transformer types, N stage is the total number of stages.

[0157] For distributed power sources, formula (16) indicates that once distributed power sources are built in a certain stage, they will exist in the following stages. Although distributed power sources have many advantages, the number of them connected to the distribution network also needs to be limited. Considering the uncertainty of renewable energy output, the access of a large number of renewable energy sources will have an adverse effect on the stability of the power system. The capacity constraint of distributed power generation construction is shown in formula (17). The access capacity of candidate nodes that can be connected to distributed power sources cannot exceed the maximum capacity limit, and the maximum capacity is limited by the load level and the penetration rate of renewable energy.

[0158]

[0159] Where Ω DG is the candidate distributed generation node set, N DG is the number of candidate distributed power generation nodes, a is the number of candidate distributed power generation types, n is the candidate distributed power generation node, N DG is the number of distributed generation accesses, x DG For distributed generation construction variables, S DGis the rated capacity of distributed power supply, is the penetration rate of distributed generation, N node is the number of load nodes, S load is the load of each node.

[0160] In order to smooth the output power, distributed energy is matched with energy storage. Formula (18) indicates that the energy storage constructed in the previous stage will continue to exist in the next stage. Formula (19) indicates that the capacity of energy storage should be within the allowable upper and lower limits. Formulas (20)-(21) give the calculation formulas for the upper and lower limits of energy storage capacity at a certain node.

[0161]

[0162] Among them, Ω ESS is the energy storage node set, x ESS is the energy storage access variable, a is the number of candidate energy storage types, n is the energy storage candidate node, and u is the number of stages; Cp ESS,n is the energy storage access capacity of node n, is the minimum access capacity of energy storage for node n, The maximum access capacity of energy storage for node n; The minimum storage ratio required for photovoltaic operation. The maximum energy storage ratio configured for photovoltaic operation, S load,n is the load of node n.

[0163] 2) Operational constraints

[0164] This section gives the system operation constraints, including power balance constraints, voltage balance constraints, second-order cone constraints, line power constraints, distributed generation and energy storage constraints. For all formulas in this section, They represent the current stage, year, and quarter respectively. The operation constraints are met on typical days in each quarter and are not listed one by one for simplicity.

[0165] Power balance constraints include active power balance and reactive power balance constraints. According to Kirchhoff's voltage and current law and the law of conservation of energy, the power of each node in the system should be balanced, that is, the power injected by the line, substation, and distributed power supply to the node should be balanced with the load demand. The power balance constraints are as follows:

[0166]

[0167] Formula (22) represents the active power balance constraint, where n is the node number, u is the stage number, t is the year in the stage, and PF load is the power factor, is the load node power, The active power injected into the line, The active power injected by DG, Active power injected into energy storage, Inject active power of substation nodes into the upper power grid, or cut load active power.

[0168]

[0169] Formula (23) is the reactive power balance constraint, where n is the node number, u is the stage number, t is the year in the stage, PFload is the power factor, The reactive power injected into the line, The reactive power injected by DG, Reactive power injected into energy storage, Inject reactive power of substation nodes into the upper power grid, or cut load reactive power.

[0170] For the lines in operation, the voltage change at both ends of the line is related to the line voltage loss. The voltage difference vector at the beginning and end of the power line can be decomposed into the voltage difference transverse component ΔU and the longitudinal component δU. Since the value of the voltage difference transverse component ΔU is small, it can be omitted for further simplification, and only the longitudinal component δU is retained. In the present invention, only the lines through which power flows are meaningful for discussion, so the specific form of the line voltage drop balance constraint is shown in formula (24):

[0171]

[0172] Among them, i and j represent the starting and ending nodes of line n respectively, R n and X n denote the resistance and reactance of line n respectively, It represents the square of the voltage at the node i at the beginning of the line. It represents the square of the current in line n, Respectively represent the active and reactive power of line n.

[0173] The line current expression is shown in equation (25), where is the square of the current in line n, is the square of the voltage at node i at the beginning of line n. is the active and reactive power of the line.

[0174]

[0175] The second-order cone constraint is a non-convex and nonlinear constraint. The feasible domain is the surface of the second-order cone, which is a non-convex optimization model. In order to transform the problem into a mixed integer second-order cone programming problem for solution, the constraint can be relaxed as follows:

[0176]

[0177] The transmission power of the line should not exceed the rated capacity of the line. At the same time, the line operation variable is introduced to constrain the power of the unoperated line to zero. Only the operated line can have transmission power. The line power constraints are shown in equations (27)-(28).

[0178]

[0179] in represents the rated capacity of line n, Indicates whether line n is put into operation. They represent the active and reactive power flowing through line n respectively.

[0180] For energy storage, equation (29) indicates that the power of energy storage is less than the maximum charge and discharge power, and the energy storage should satisfy the discharge balance equation as shown in equation (30).

[0181]

[0182] Where P ESS,n is the energy storage charging and discharging power of node n, The minimum energy storage power for node n, The maximum power stored for node n; Indicates the energy storage charging power, Indicates the energy storage discharge power; η ch , η dis Indicates the charging and discharging efficiency of energy storage.

[0183] In order to ensure that the access of distributed power generation does not affect the stable operation of the power grid, the output power of distributed power generation must meet certain constraints, that is, the output of distributed power generation should be higher than the minimum value, as shown in equations (31)-(32), where χ is the minimum distributed power generation utilization rate of the system, It is the minimum utilization power of distributed power source.

[0184]

[0185]

[0186] During system operation, power transmission depends on the construction of the grid. Some lines may be built but not put into use. The uniqueness constraint of line power flow is shown in equation (33), and the constraint that power only exists in the built lines is shown in equation (34):

[0187]

[0188] in To indicate whether line n is in operation in the sth quarter of the tth year in phase u, For negative delivery, is the line construction variable, and a is the line selection. Formula (33) prevents the situation where the power flow is both positive and negative at the same time, because at any time, the direction of power flow is certain. Constraint (34) indicates that the state of line operation in the stage should be consistent with the line construction status of the current stage, limiting the power flow to only exist on the built line.

[0189] 3) Upper and lower limit constraints

[0190] Upper and lower limit constraints refer to the working range that the system should limit during operation. Upper and lower limit constraints play a vital role in ensuring the safe and reliable operation of the distribution network, improving the power quality, and extending the service life of electrical equipment. The constraints are shown in equations (35)-(39), where u is the planning stage, n is the node number, t is the year of the current stage, and s is the current quarter.

[0191]

[0192] Formula (35) is the upper and lower limit constraints of the node voltage, indicating that the node voltage should be limited within the upper and lower limits. is the lower limit of node voltage, is the upper limit of node voltage.

[0193] Formula (36) is the line current constraint. The line current should be within the line bearing capacity. The upper limit of the line current.

[0194] Formula (37) is the output constraint of distributed generation. The output of distributed generation should meet the power constraint and should not exceed the capacity limit. is the maximum output of the distributed generation at node n.

[0195] Formula (38) is the load shedding power constraint. The maximum load shedding power does not exceed the node load. is the load at node n. Equation (39) is the distributed generation power removal constraint, where is the maximum removal capacity of distributed generation, is the capacity of the distributed generation connected to node n.

[0196] 4) Network topology constraints

[0197] To ensure the safe and stable operation of the power distribution system, the network should have a radial topology when in operation, that is, a closed loop connected to an open loop. Secondly, it should also prevent the distributed power supply from being connected to the network, so that part of the load is only powered by the distributed power supply, thus being separated from the main grid to form an island, resulting in a decrease in the safety and reliability of operation.

[0198] According to the characteristics of the distribution system, the radial topology belongs to a tree structure. Since the electric energy is transmitted from top to bottom, the power flows out of the substation and enters the lower-level power grid. The substation node has no parent node, and there is no node that transmits power to it, as shown in formula (40). In order to ensure the power supply of the load node and the stability of the system, the load node has only one parent node to supply power to it, as shown in formula (41), where are the forward and reverse operation states of the line, respectively, .j is the number of nodes connected to node j, Ω l is the number of lines, S li,u The parent node of the node exists.

[0199]

[0200] At the same time, in the planning process, in order to prevent the power supply from losing connection with the main grid when it is put into operation in isolation, and to prevent the load from being separated from the main grid, it is necessary to introduce anti-islanding constraints, introduce the concept of virtual power, and build a virtual power system. Anti-islanding constraints include two parts: construction and operation, as follows.

[0201] The anti-islanding constraint during the construction process considers that the virtual injection power of each substation is greater than 0, and the virtual injection power of the node without a substation is 0, as shown in formula (42).

[0202]

[0203] At the same time, the virtual power demand of the nodes in the system defines the virtual load If the node has load demand or there is renewable energy power injection, the virtual load is 1, and the virtual load of other nodes is 0, as shown in formula (43), where is the virtual power flowing through line ij. According to the node power balance equation, the node balance equation of virtual power is established, as shown in equation (44). are the virtual power of distributed power sources and energy storage, are the virtual powers flowing into and out of node j respectively.

[0204]

[0205] The upper and lower limits of virtual power are shown in equations (45) to (46).

[0206]

[0207] For the network topology constraints of the operation plan, the line commissioning state variables are introduced, and the variables under the construction state in formula (46) are replaced by the variables under the operation state. The specific constraints are shown in formula (47).

[0208]

[0209] Network topology constraints can ensure that the currently operating distribution system is a radial network, avoiding closed-loop line connection operation. The system may contain multiple subnetworks powered by a single substation, avoiding the island operation of load points or distributed power sources.

[0210] 2. Solution algorithm based on approximate dynamic programming

[0211] In the process of solving the above-mentioned multi-stage planning model, a large number of Markov states in each stage and a huge action space with a large dimension will bring about the "curse of dimensionality" problem. An approximate dynamic programming algorithm can be used to solve this problem. The approximate dynamic programming algorithm is also called an adaptive dynamic programming algorithm. The core is to find the global optimal strategy. The decisions made by decision makers at each stage based on the currently available information will not only affect the current stage, but also all subsequent stages. For this influence, a commonly used approximation method is the value function approximation strategy, including neural network approximation, table function approximation, piecewise linear approximation, etc. The present invention uses an approximate dynamic programming algorithm based on table function approximation to solve the above-mentioned multi-stage stochastic programming problem.

[0212] 2.1 Value function approximation based on post-decision state

[0213] First, rewrite the above multi-stage planning model into the Bellman equation form, as shown in equation (48). t (S t ) is state S t The value function represents the sum of the costs from the current stage to the final stage, C t (S t ,A t ) represents the current stage of action A t The total cost, E[V t+1 (S t+1 |S t ,A t ,λ t )] represents the expected total cost of all future stages, which is a recursive equation.

[0214] V t (S t )=min{C t (S t ,A t )+E[V t+1 (S t+1 |S t ,A t ,λ t )]} (48)

[0215] The approximate dynamic programming algorithm based on value function approximation introduces the post-decision state That is, the system state after the decision maker has taken action and has not yet introduced the influence of the external environment, namely, the load forecast error. Its expression is shown in formula (49), which can be used to replace the sum of the expected costs of all future stages in formula (48). The replaced Bellman equation is shown in formula (50).

[0216]

[0217] The key to solving the above multi-stage planning model by using the approximate dynamic programming algorithm through the value function approximation method is to know the state after the decision In this way, the multi-stage planning model can be transformed into a single-stage deterministic planning problem. First, the mapping between the actions in each stage and the expected future costs is established. After the training value function, we can get The approximate value of , and then the optimal decision for each stage can be obtained by calculating formula (51).

[0218]

[0219] Formula (49) needs to obtain the approximate value of the state value function after the decision in each stage. The action space of each stage is certain. If the state space of each stage is large, that is, the number of load development forecast errors considered is large, it will still lead to calculation difficulties. Therefore, in order to simplify the calculation process, the number of state spaces of the above model is the same as the number of load development forecast error intervals, that is, each load forecast error maps a system state space, and each load forecast error can be solved by the above distribution system planning model, and then the approximate value function can be obtained by iteration.

[0220] 2.2 Value function iteration process

[0221] To solve this problem, we must first calculate the approximate value of the value function. The solution of the value function is an iterative process. The approximate dynamic programming algorithm recursively calculates the value function through the value function after the decision in each iteration, as shown in formula (52), where Indicates state S t The value function of the nth iteration and the optimal decision can be obtained by formula (53), and the value function after the decision in the previous stage can be obtained by formula (54).

[0222]

[0223] Where ε is the iteration step, and its value range is (0,1). The iteration of the value function is as follows Figure 2As shown, each box represents all states of each stage of each cycle, the circles represent all states of the current stage, and the arrows represent the state transition path of the system. It can be seen that each state transition path has been calculated in multiple iterations, and the state passed in the previous stage in each iteration will update the value function through formula (55). Through the above iterations, each state S of the system can be obtained t The corresponding value function approximation is obtained, thus converting the problem into a single-stage deterministic planning problem.

[0224] 2.3 Approximate dynamic programming solution process

[0225] The flowchart of using the approximate dynamic programming algorithm based on the post-decision state value function approximation to solve the above distribution system multi-stage planning problem is as follows: Figure 3 As shown, it includes four parts: parameter reading, solving the action space, value function approximation, and finally solving the planning solution.

[0226] The action space can be solved directly using commercial solvers such as gurobi and Cplex, so that the optimal action under each load growth state can be obtained. Then, the randomness of load growth is simulated by the value function approximation method to obtain the approximate value of the value function. Finally, by solving the single-stage planning problem, the multi-stage planning results of the grid structure are obtained. When planning, decision makers can determine the optimal planning scheme based on the current load growth information obtained step by step, considering the uncertainty of future load growth.

[0227] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit the present invention. A person skilled in the art should understand that the specific implementation of the present invention can be modified or replaced by equivalents, but these modifications or changes are within the scope of protection of the pending claims.

Claims

1. A multi-stage stochastic programming method for high-voltage distribution network based on approximate dynamic programming, characterized in that: The multi-stage stochastic programming method for high-voltage distribution network includes the following steps: Step (1): Establish a model based on the Markov decision process to generate an investment decision tree; Step (2): Considering the randomness of load development and the economic efficiency of distribution system investment and operation, a multi-stage stochastic programming model is established, and relevant constraints are established. The objective function aims to minimize the total cost, which includes investment and operation costs. Step (3): Use an approximate dynamic programming algorithm based on the post-decision state to solve.

2. A multi-stage stochastic programming method for high-voltage distribution network based on approximate dynamic programming according to claim 1, characterized in that: The first part of the objective function in step (2) is the investment cost, including feeder construction cost, substation construction and expansion cost, transformer construction cost, new energy construction cost and energy storage construction cost; the second part of the objective function is the operating cost, including the cost of purchasing electricity from the superior power grid, distributed energy generation, equipment maintenance, load shedding, abandoned light and reliability, among which the reliability cost is obtained by calculating the amount of electricity that cannot be supplied to users due to system component failure.

3. A multi-stage stochastic programming method for high-voltage distribution network based on approximate dynamic programming according to claim 2, characterized in that: The objective function in step (2) specifically includes: For different equipment in the system, it is assumed that the annual investment cost is the same during the normal operation period. The calculation formula is shown in formula (1), where γ is the inflation rate and LT is the equipment life: After calculating the annual cost, the annual cost is converted to the current period through the present value factor; the calculation formula is shown in formula (2), where u and t represent the current period and the number of years in the current period, respectively, and T represents the total number of years in each period; On this basis, the objective function is defined as shown in formula (3): The optimization goal is to minimize the present value of the total cost, including the investment cost in the tth year of stage u Operation and maintenance costs in the sth quarter of the tth year of stage u Production costs Cost of power outage Reliability Cost To make the model more practical, the planning period is divided into U stages, each stage contains Y years, each year contains S quarters, and 12 typical operation scenarios including daytime and nighttime are selected in each quarter; it is assumed that only one investment will be made in each stage within Y years, that is, the investment cost in the same stage is the same; and the investment decision in each stage is related to the specific load situation, so the investment cost in different stages is different; the construction investment cost includes the investment cost of the line, substation, renewable energy, and transformer, as shown in formula (4), where Ω u ,Ω t ,Ω s They are respectively the planning stage set, the year set included in each stage, and the quarter set. are the number of lines, substations, distributed generation and transformers to be built, respectively, L is the line construction variable, x DG is the variable for distributed generation construction, x sub is the substation construction variable, x TR For transformer construction variables, L n is the length of the nth line, C I L is the line investment cost per unit length, C I sub is the substation investment cost, C I DG is the distributed generation investment cost, C I TR The investment cost of the transformer; The operation and maintenance cost is related to the load demand and the operating status of distributed power in the specific scenario. In the planning model, the system operating status of the typical scenario is first obtained, and then the annual operation and maintenance cost can be obtained according to the duration of different scenarios. The operation and maintenance cost is shown in formula (5), including the maintenance cost of the line, the maintenance cost of the new transformer, and the maintenance cost of the distributed power. L is the number of all lines, y + ,y - It is a 0-1 variable, indicating whether the direction of the power flowing through the line is forward or reverse; Formula (6) is the production cost, where C L M is the line maintenance cost per unit length, C TR M is the transformer maintenance cost, C DG M is the maintenance cost of distributed generation, C pp is the electricity purchase cost, LD is the current scenario hours, and LF is the substation power factor; the production cost includes the electricity purchase cost of the substation and the photovoltaic power generation cost; Formula (7) shows the power shortage cost, including load shedding cost and solar curtailment cost; where N node is the number of nodes, is the abandoned optical power, S cutload is the load shedding power, is the substation power, C P E They are the abandoned light cost and the load interruption cost respectively; The lines between the switch stations of the backbone network are regarded as feeder segments. Since the circuit breakers and distribution system automation equipment are configured throughout the line, after a feeder segment fails, the upstream load is directly transferred without power outage, and the downstream restores power supply through transfer. The reliability cost calculation is simplified to only consider the impact of the downstream of the fault. The load that needs to be transferred downstream of the fault is the power transmitted on the feeder segment before the fault, the power outage time downstream of the fault is the transfer time, and the power outage probability downstream of the fault is the failure probability of the current fault feeder segment. The reliability cost is specifically shown in formula (8): where λ n is the failure rate of the nth line, μ n is the maintenance or transfer time of the nth line, P l,n is the transmission power of the nth line, LD is the current scenario hour, C P E is the load interruption cost; for the calculation of the reliability index during the planning period, the reliability cost can be obtained by the reliability cost obtained by formula (8). First, the regional annual total electricity consumption is calculated minus the power supply loss due to the fault to obtain the regional annual supply electricity, and then divided by the regional annual total electricity consumption to obtain the average power supply reliability rate ASAI, as shown in formula (9): Where SAIDI is the average annual power outage time for users, T total P is the annual power supply time, generally 8760 hours. ave is the average annual power consumption of users, N cus is the number of users in the region, and E total is the total electricity consumption of the region.

4. A multi-stage stochastic programming method for high-voltage distribution network based on approximate dynamic programming according to claim 3, characterized in that: The constraints in step (2) include construction constraints, operation constraints, upper and lower limit constraints of components, and network topology constraints.

5. A multi-stage stochastic programming method for high-voltage distribution network based on approximate dynamic programming according to claim 4, characterized in that: The construction constraints are to ensure that each construction variable meets the actual requirements. Once the electrical equipment is built in a certain stage, it will exist in the following stages. Formula (10) indicates that each line can only be built once in all stages. Once it is built in a certain stage, it will exist in the following stages. Formula (11) indicates that only one type of construction can be selected. The construction constraints corresponding to each type of electrical equipment are as follows: Among them, Ω a is the number of line types, x L is a 0-1 variable for line construction, u is the stage number, n is the line number, a is the number of line types, N stage is the total number of stages; Formula (12) indicates that a transformer can only be built once at most. Once built in a certain stage, it will exist in the following stages. Formula (13) indicates that each node can only select one type of transformer for construction. Formula (14) indicates that a transformer can only be installed after the substation is newly built. Among them, Ω sub is the number of substation nodes, Ω TR is the number of transformers, x TR Construct a 0-1 variable for the transformer, x sub is a 0-1 variable for substation construction, u is the stage number, n is the transformer number, i is the node to be selected for the substation, G i is the maximum number of transformers installed in the substation, a is the number of transformer types, N stage is the total number of stages; If a line is to be built, in order to prevent the line end from connecting to an empty node, the substation it is connected to must have been built. That is, only after the substation is built can the construction of the line connected to the substation be considered; as shown in formula (15); Among them, x sub is a 0-1 variable for substation construction, u is the stage number, i is the substation, n is the line number connected to substation i, a is the number of transformer types, N stage is the total number of stages; For distributed power sources, formula (16) indicates that once distributed power sources are built in a certain stage, they will exist in the following stages; the capacity constraint of distributed power generation construction is shown in formula (17), and the access capacity of candidate nodes that can be connected to distributed power sources cannot exceed the maximum capacity limit, and the maximum capacity is limited by the load level and the penetration rate of renewable energy; Where Ω DG is the candidate distributed generation node set, N DG is the number of candidate distributed power generation nodes, a is the number of candidate distributed power generation types, n is the candidate distributed power generation node, N DG is the number of distributed generation connections, x DG For distributed generation construction variables, S DG is the rated capacity of distributed power supply, is the penetration rate of distributed generation, N node is the number of load nodes, S load is the load of each node; In order to smooth the output power, distributed energy is matched with energy storage. Formula (18) indicates that the energy storage constructed in the previous stage will continue to exist in the next stage. Formula (19) indicates that the capacity of energy storage should be within the allowable upper and lower limits. Formulas (20)-(21) give the calculation formulas for the upper and lower limits of energy storage capacity at a certain node. Among them, Ω ESS is the energy storage node set, x ESS is the energy storage access variable, a is the number of candidate energy storage types, n is the energy storage candidate node, and u is the number of stages; Cp ESS,n is the energy storage access capacity of node n, is the minimum access capacity of energy storage for node n, The maximum access capacity of energy storage for node n; The minimum storage ratio required for photovoltaic operation. The maximum energy storage ratio configured for PV operation, S load,n is the load of node n.

6. A multi-stage stochastic programming method for high-voltage distribution network based on approximate dynamic programming according to claim 5, characterized in that: The operation constraints include power balance constraints, voltage balance constraints, second-order cone constraints, line power constraints, and distributed power and energy storage constraints; The power balance constraints include active power balance and reactive power balance constraints. According to Kirchhoff's voltage and current law and the law of conservation of energy, the power of each node in the system should be balanced, that is, the power injected by the line, substation, and distributed power supply to the node and the load demand should be balanced; the power balance constraints are as follows: Formula (22) represents the active power balance constraint, where n is the node number, u is the stage number, t is the year in the stage, and PF load is the power factor, is the load node power, The active power injected into the line, The active power injected by DG, Active power injected into energy storage, Inject active power of substation nodes into the upper power grid, or cut load active power; Formula (23) is the reactive power balance constraint, where n is the node number, u is the stage number, t is the year in the stage, and PF load is the power factor, The reactive power injected into the line, The reactive power injected by DG, Reactive power injected into energy storage, Inject reactive power of substation nodes into the upper power grid, or cut load reactive power; For the operational lines, the voltage change at both ends of the line is related to the line voltage loss. The voltage difference vector at the beginning and end of the power line is decomposed into the voltage difference transverse component ΔU and the longitudinal component δU. Since the value of the voltage difference transverse component ΔU is small, it is omitted for further simplification and only the longitudinal component δU is retained. The specific form of the line voltage drop balance constraint is shown in formula (24): Among them, i and j represent the starting and ending nodes of line n respectively, R n and X n denote the resistance and reactance of line n respectively, It represents the square of the voltage at the node i at the beginning of the line. It represents the square of the current in line n, Respectively represent the active and reactive power of line n; The line current expression is shown in equation (25), where is the square of the current in line n, is the square of the voltage at node i at the beginning of line n. is the active and reactive power of the line; The second-order cone constraint is a non-convex and nonlinear constraint. The feasible domain is the surface of the second-order cone. It is a non-convex optimization model. In order to transform the problem into a mixed integer second-order cone programming problem for solution, the constraint is relaxed as follows: The transmission power of the line should not exceed the rated capacity of the line. At the same time, the line operation variable is introduced to constrain the power of the unoperated line to zero. Only the operational line can have transmission power. The line power constraint is shown in equations (27)-(28): in represents the rated capacity of line n, Indicates whether line n is put into operation. They represent the active and reactive power flowing through line n respectively. For energy storage, equation (29) indicates that the power of energy storage is less than the maximum charge and discharge power, and at the same time, the energy storage should satisfy the discharge balance equation as shown in equation (30); Where P ESS,n is the energy storage charging and discharging power of node n, The minimum energy storage power for node n, The maximum power stored for node n; Indicates the energy storage charging power, Indicates the energy storage discharge power; η ch , η dis Indicates the charging and discharging efficiency of energy storage; In order to ensure that the access of distributed power generation does not affect the stable operation of the power grid, the output power of distributed power generation must meet certain constraints, that is, the output of distributed power generation should be higher than the minimum value, as shown in equations (31)-(32), where χ is the minimum distributed power generation utilization rate of the system, The minimum utilization power of distributed power sources; During system operation, power transmission depends on the construction of the grid; the uniqueness constraint of line power flow is shown in equation (33), and the constraint that power only exists in the constructed line is shown in equation (34): in To indicate whether line n is in operation in the sth quarter of the tth year in phase u, For negative delivery, is the line construction variable, a is the line selection; Formula (33) prevents the situation where the power flow is both positive and negative at the same time, because at any time, the direction of power flow is certain; Constraint (34) indicates that the state of line operation within the stage should be consistent with the line construction status of the current stage, limiting the power flow to only exist on the built line.

7. A multi-stage stochastic programming method for high-voltage distribution network based on approximate dynamic programming according to claim 6, characterized in that: The upper and lower limit constraints of the components refer to the working range that the system should limit during operation. The upper and lower limit constraints of the components play a vital role in ensuring the safe and reliable operation of the distribution network, improving the power quality, and extending the service life of electrical equipment. The constraints are specifically shown in equations (35)-(39), where u is the planning stage, n is the node number, t is the year of the current stage, and s is the current quarter. Formula (35) is the upper and lower limit constraints of the node voltage, indicating that the node voltage should be limited within the upper and lower limits. is the lower limit of node voltage, is the upper limit of node voltage; Formula (36) is the line current constraint. The line current should be within the line bearing capacity. is the upper limit of line current; Formula (37) is the output constraint of distributed generation. The output of distributed generation should meet the power constraint and should not exceed the capacity limit. is the maximum output of the distributed generation at node n; Formula (38) is the load shedding power constraint. The maximum load shedding power does not exceed the node load. is the load at node n; Equation (39) is the power cut-off constraint of distributed generation, where is the maximum removal capacity of distributed generation, is the capacity of the distributed generation connected to node n.

8. A multi-stage stochastic programming method for high-voltage distribution network based on approximate dynamic programming according to claim 7, characterized in that: The network topology constraints are specifically: According to the characteristics of the distribution system, the radial topology belongs to a tree structure. Since the electric energy is transmitted from top to bottom, the power flows out of the substation and enters the lower-level power grid. The substation node has no parent node, and there is no node that transmits power to it, as shown in formula (40). In order to ensure the power supply of the load node and the stability of the system, the load node has only one parent node to supply power to it, as shown in formula (41), where are the forward and reverse operation states of the line, respectively, .j is the number of nodes connected to node j, Ω l is the number of lines, S li,u The parent node of the node exists. At the same time, in the planning process, in order to prevent the power supply from losing connection with the main grid when it is put into operation in isolation, and to prevent the load from being separated from the main grid, it is necessary to introduce anti-islanding constraints, introduce the concept of virtual power, and build a virtual power system; anti-islanding constraints include construction and operation, as follows: The anti-islanding constraint during the construction process considers that the virtual injection power of each substation If it is greater than 0, the virtual injection power of the node without substation is 0, as shown in formula (42): At the same time, the virtual power demand of the nodes in the system defines the virtual load If the node has load demand or there is renewable energy power injection, the virtual load is 1, and the virtual load of other nodes is 0, as shown in formula (43), where is the virtual power flowing through line ij. According to the node power balance equation, the node balance equation of virtual power is established, as shown in equation (44); are the virtual power of distributed power sources and energy storage, are the virtual powers flowing into and out of node j respectively; The upper and lower limits of virtual power are shown in equations (45) to (46): For the network topology constraints of the operation plan, the line commissioning state variables are introduced, and the variables under the construction state in formula (46) are replaced by the variables under the operation state. The specific constraints are shown in formula (47): Network topology constraints can ensure that the currently operating distribution system is a radial network, avoiding closed-loop connection of lines. The system may contain multiple subnetworks powered by a single substation, avoiding the isolated operation of load points or distributed power sources.

9. A multi-stage stochastic programming method for high-voltage distribution network based on approximate dynamic programming according to claim 8, characterized in that: In step (3), the approximate dynamic programming algorithm based on the post-decision state is used to solve the problem, which specifically includes: (31) Approximation of the value function based on the post-decision state; First, the above multi-stage planning model is rewritten into the Bellman equation form, as shown in Equation (48), where V t (S t ) is state S t The value function represents the sum of the costs from the current stage to the final stage, C t (S t ,A t ) represents the current stage of action A t The total cost, E[V t+1 (S t+1 |S t ,A t ,λ t )] represents the expected total cost of all future stages, which is a recursive equation; V t (S t )=min{C t (S t ,A t )+E[V t+1 (S t+1 |S t ,A t ,λ t )]} (48) The approximate dynamic programming algorithm based on value function approximation introduces the post-decision state That is, the system state after the decision maker has taken action and has not yet introduced the influence of the external environment, namely, the load forecast error. Its expression is shown in formula (49), which can be used to replace the sum of the expected costs of all future stages in formula (48). The replaced Bellman equation is shown in formula (50): The key to solving the above multi-stage planning model by using the approximate dynamic programming algorithm through the value function approximation method is to know the state after the decision In this way, the multi-stage planning model can be transformed into a single-stage deterministic planning problem. First, the mapping between the actions in each stage and the expected future costs is established. After the training value function, we can get The approximate value of , and then the optimal decision of each stage can be obtained by calculating formula (51); Formula (49) needs to obtain the approximate value of the state value function after the decision in each stage. The action space of each stage is certain. If the state space of each stage is large, that is, the number of load development prediction errors considered is large, it will still lead to calculation difficulties. Therefore, in order to simplify the calculation process, the number of state spaces of the above model is the same as the number of load development prediction error intervals, that is, each load prediction error maps a system state space, and each load prediction error can be solved by the above distribution system planning model, and then the approximate value function can be obtained by iteration; (32) Value function iteration process First, we need to calculate the approximate value of the value function. Solving the value function is an iterative process. The approximate dynamic programming algorithm recursively calculates the value function through the decision-making value function of the previous iteration, as shown in formula (52), where Indicates state S t The value function of the nth iteration, the optimal decision can be obtained by formula (53), and the value function after the decision in the previous stage can be obtained by formula (54); Where ε is the iteration step length, and its value range is (0,1); in multiple iterations, each state transition path is calculated, and in each iteration, the state passed in the previous stage will be updated through equation (54) to update the value function; through the above iterations, each state S of the system can be obtained t The corresponding value function approximation is used to transform the problem into a single-stage deterministic planning problem; (33) Approximate dynamic programming solution process The process of solving the above-mentioned multi-stage planning problem of the distribution system by using the approximate dynamic programming algorithm based on the approximate state value function after decision making includes four parts: parameter reading, solving the action space, approximating the value function, and finally solving the planning scheme; The action space can be solved directly using commercial solvers such as gurobi and Cplex, so that the optimal action under each load growth state is obtained; then, the randomness of load growth is simulated through the value function approximation method to obtain the approximate value of the value function; finally, by solving the single-stage planning problem, the multi-stage planning result of the grid structure is obtained; when making plans, the decision maker determines the optimal planning scheme based on the current load growth information obtained step by step, considering the uncertainty of future load growth.

Images