Optimal Method and Device for Transmission Network Expansion Planning Considering Extreme Scenarios

Abstract

This application relates to the field of electrical engineering technology, and proposes an optimization method and device for the expansion planning of a power grid framework considering extreme scenarios. The method includes: First, obtain historical data of extreme disaster scenarios of the power grid and perform preliminary processing to obtain a set of extreme scenarios; Then, construct an optimization objective function for the expansion of the power grid framework, and the optimization objective function for the expansion of the power grid framework includes economic indicators and stability indicators; Then, set constraint conditions based on the optimization objective function for the expansion of the power grid framework; Finally, use the TD3 algorithm improved based on the multi-task learning algorithm to solve the new energy multi-infeed power grid framework expansion planning model based on the set of extreme scenarios and the optimization objective function for the expansion of the power grid framework. It enhances the overall risk resistance ability of the power grid, helps to enhance the structural toughness of the power grid, comprehensively improves the safe operation level of the power grid under extreme weather conditions, and enhances the emergency response ability under extreme weather. It improves the stability, reliability and risk resistance ability of the power grid.

Description

Technical Field

[0001] The present application relates to the field of electrical engineering technology, and in particular to a method and device for optimizing transmission network expansion planning taking into account extreme scenarios. Background Art

[0002] The new energy system is an inevitable choice to ensure national energy security. my country's energy distribution and load center show inverse distribution characteristics, and the development of renewable energy is mainly concentrated in natural resource-rich areas such as the west and north. As global climate change has intensified the frequency of extreme weather, the above-mentioned regions are constrained by geographical location, climate conditions, and the continuous increase in the proportion of renewable energy, resulting in a significant reduction in the voltage support strength of their AC transmission lines, and the balance of power and electricity is facing major challenges. The extreme scenarios brought about by climate change have put forward new requirements for the planning and construction of the transmission network.

[0003] At present, the main research methods for the planning of transmission networks in which new energy replaces conventional units are numerical solutions, heuristic learning methods, and reinforcement learning methods. Among them, the numerical solution mainly uses robust optimization technology, multi-scenario technology, and collaborative planning technology for modeling research, and its specific application scenario is determined by the objective function of grid expansion planning. The planning scenario of robust optimization technology emphasizes the safety of the solution, while the transmission network planning based on multi-scenario technology focuses more on the economy of the solution. The collaborative planning technology is suitable for the scenario where a variety of renewable energy sources are concentratedly connected to the power grid. However, when solving the planning and construction scheme of large-scale power grids, the numerical solution has the problem of difficulty in migrating scenarios due to the high computational complexity. Based on this, some scholars have proposed to use heuristic learning algorithms to avoid the defects of numerical solutions, simulate transmission network planning scenarios in a data-driven way, and reduce dependence on physical models. Although the heuristic algorithm has a high solution efficiency, the process of solving the optimal solution is separated from the physical model, and the interpretability of the power grid planning operation is poor. The reinforcement learning algorithm optimizes the decision-making strategy through interaction with the environment, guides the active learning process and mines the optimal solution with a reward mechanism, and shows higher adaptability and robustness in a complex and dynamically changing environment.

[0004] At present, the research on the optimization of transmission network expansion with multiple feed-in of renewable energy is relatively conservative, and the application scenarios are mainly normal operation under universal conditions. However, with the intensification of global climate change, the operation scenarios of power systems have become more extreme. Natural disasters such as cold waves, typhoons, and floods have caused serious damage to the transmission network, while continuous hot and windless weather and extremely cold and lightless weather have a strong impact on renewable energy. The extreme scenarios faced by the expansion planning of the new power system grid can be mainly divided into three categories: low output scenarios of generator sets, surge load scenarios, and disconnection scenarios of transmission networks, which are mainly affected by extreme meteorological disasters. At present, there is little research on the planning of transmission networks with multiple feed-in of renewable energy under extreme scenarios.

[0005] Therefore, in the related art, there is an urgent need for a method to improve the emergency response ability and stability of the power grid in complex environments. Summary of the Invention

[0006] Based on this, in view of the above technical problems, it is necessary to provide an optimized method and device for the expansion planning of a transmission grid framework considering extreme scenarios, which can improve the emergency response ability and stability of the power grid in complex environments.

[0007] In a first aspect, the present application provides an optimized method for the expansion planning of a transmission grid framework considering extreme scenarios. The method includes:

[0008] Obtain historical data of extreme disaster scenarios of the power grid, and perform preliminary processing to obtain an extreme scenario set;

[0009] Construct an optimization objective function for the expansion of the transmission grid framework, where the optimization objective function for the expansion of the transmission grid framework includes economic indicators and stability indicators;

[0010] Set constraint conditions based on the optimization objective function for the expansion of the transmission grid framework;

[0011] Use the TD3 algorithm improved based on the multi-task learning algorithm to solve and obtain a new energy multi-infeed transmission grid framework expansion planning model based on the extreme scenario set and the optimization objective function for the expansion of the transmission grid framework.

[0012] Optionally, in an embodiment of the present application, the obtaining of the historical data of extreme disaster scenarios of the power grid and performing preliminary processing to obtain an extreme scenario set includes:

[0013] Classify and extract based on the historical data of extreme disaster scenarios of the power grid to obtain extreme scenario samples;

[0014] For extreme scenario samples of different categories, generate an extreme scenario set based on their dynamic characteristics using a simulation method.

[0015] Optionally, in an embodiment of the present application, the obtaining of the historical data of extreme disaster scenarios of the power grid and performing preliminary processing to obtain an extreme scenario set further includes:

[0016] Based on the system power and electricity balance relationship, perform secondary screening on the historical data of extreme disaster scenarios of the power grid, and extract a supplementary scenario set with power and electricity imbalance.

[0017] Optionally, in an embodiment of the present application, the constructing of the optimization objective function for the expansion of the transmission grid framework includes:

[0018] Determine the construction, operation, and maintenance costs of the transmission grid framework based on the construction investment costs, operation and maintenance costs, and additional costs for new energy grid connection of the transmission grid;

[0019] Determine the voltage support strength margin of the transmission grid framework based on the short-circuit ratio and critical short-circuit ratio at the new energy grid connection point;

[0020] Determine the expansion and optimization objective function of the transmission grid framework based on the construction, operation, and maintenance costs of the transmission grid framework and the voltage support strength margin of the transmission grid framework.

[0021] Optionally, in an embodiment of the present application, the constraint conditions include node power balance constraints, power flow balance constraints, line transmission capacity constraints, voltage support strength margin constraints, and N-1 security constraints.

[0022] Optionally, in an embodiment of the present application, the TD3 algorithm improved based on the multi-task learning algorithm includes:

[0023] Use the action network in the TD3 algorithm to select the execution action for changing the state of the transmission line;

[0024] Use the multi-task learning algorithm to improve the action network and the corresponding target action;

[0025] Screen the key transmission lines through the dual evaluation network, and calculate the reward value generated by the execution action based on the dual target network.

[0026] Optionally, in an embodiment of the present application, the algorithm formula for using the multi-task learning algorithm to improve the action network and the corresponding target action is:

[0027]

[0028]

[0029]

[0030]

[0031] Among them, represents the probability of each line executing the action, represents the current line state set, 、 and respectively represent the expert network, the gating network, and the tower network, 、 and respectively represent the parameters of the expert network, the gating network, and the tower network, represents the weight vector generated by the gating network, represents the weighted action probability set, represents the execution action.

[0032] Second aspect, the present application also provides a transmission network framework expansion planning optimization device considering extreme scenarios. The device includes:

[0033] An extreme scenario set acquisition module, configured to acquire historical data of extreme disaster scenarios of the power grid and perform preliminary processing to obtain an extreme scenario set;

[0034] A target function construction module, configured to construct a transmission network framework expansion optimization target function, where the transmission network framework expansion optimization target function includes an economic index and a stability index;

[0035] A constraint condition setting module, configured to set constraint conditions based on the transmission network framework expansion optimization target function;

[0036] A transmission network framework expansion planning optimization module, configured to use the TD3 algorithm improved based on the multi-task learning algorithm to solve the new energy multi-infeed transmission network framework expansion planning model based on the extreme scenario set and the transmission network framework expansion optimization target function.

[0037] The above-mentioned transmission network framework expansion planning optimization method considering extreme scenarios. First, acquire historical data of extreme disaster scenarios of the power grid and perform preliminary processing to obtain an extreme scenario set; then, construct a transmission network framework expansion optimization target function, where the transmission network framework expansion optimization target function includes an economic index and a stability index; then, set constraint conditions based on the transmission network framework expansion optimization target function; finally, use the TD3 algorithm improved based on the multi-task learning algorithm to solve the new energy multi-infeed transmission network framework expansion planning model based on the extreme scenario set and the transmission network framework expansion optimization target function. That is to say, through the simulation generation of the extreme scenario set, as the interaction environment of the new energy multi-infeed transmission network framework expansion planning model, it helps to optimize the structure design and operation mode of the power grid, reduce the dependence of the system on a single device or line, thereby enhancing the overall risk resistance ability of the power grid, contributing to enhancing the structural toughness of the power grid, and comprehensively improving the safe operation level of the power grid under extreme weather conditions and enhancing the emergency response ability under extreme weather. At the same time, based on the economy and stability of the grid framework expansion planning, considering constraint conditions, construct a grid framework expansion planning model to ensure that the transmission network framework can still operate normally after a failure under extreme weather conditions, thereby supporting the safe dispatching and control of the entire power grid, and fundamentally improving the stability, reliability and risk resistance ability of the power grid. And use the TD3 algorithm improved based on the multi-task learning algorithm to solve the transmission network planning scheme that takes into account economy and stability. On the basis of the excellent performance of the TD3 algorithm, ensure that the intelligent agent can share the same expert experience while independently extracting other information, so that the intelligent agent can better adapt to different environmental changes, thereby improving the generalization ability and calculation accuracy. Description of the Drawings

[0038] Figure 1It is an application environment diagram of an optimized method for transmission network framework expansion planning considering extreme scenarios in an embodiment;

[0039] Figure 2 It is a schematic flowchart of an optimized method for transmission network framework expansion planning considering extreme scenarios in an embodiment;

[0040] Figure 3 It is a schematic structural diagram of the TD3 algorithm in an embodiment;

[0041] Figure 4 It is a schematic diagram of the improvement of the action network by the multi-task learning algorithm in an embodiment;

[0042] Figure 5 It is a schematic flowchart of the specific steps of an optimized method for transmission network framework expansion planning considering extreme scenarios in an embodiment;

[0043] Figure 6 It is a structural block diagram of an optimized device for transmission network framework expansion planning considering extreme scenarios in an embodiment;

[0044] Figure 7 It is an internal structure diagram of a computer device in an embodiment. Specific embodiments

[0045] In order to make the objectives, technical solutions, and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0046] The optimized method for transmission network framework expansion planning considering extreme scenarios provided by the embodiments of the present application can be applied to an application environment as Figure 1 shown. Among them, the terminal 102 communicates with the server 104 through a network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed in the cloud or other network servers. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0047] In one embodiment, as Figure 2 shown, an optimized method for transmission network framework expansion planning considering extreme scenarios is provided. Taking the application of this method to the Figure 1 server as an example, the method includes the following steps:

[0048] S201: Obtain the historical data of the extreme disaster scenarios of the power grid, and perform preliminary processing to obtain an extreme scenario set.

[0049] In the embodiments of the present application, first, obtain the historical data of the extreme disaster scenarios of the power grid, and perform preliminary processing on the historical data to obtain an extreme scenario set. The extreme scenarios faced in the expansion planning of the new power system grid mainly refer to the new energy units, traditional generating units, and the transmission grid directly or indirectly affected by severe weather, resulting in an imbalance between the power supply and demand of the power grid source load and a continuous new energy consumption gap, and even causing events such as large-scale disconnection of new energy units and disconnection of the transmission grid, thus leading to large-scale power outages. The coupling between the extreme scenarios of the new power system and severe meteorological disasters is strong, and the intensity and type of meteorological disasters have obvious spatio-temporal distribution characteristics, and the generation and development process of meteorological disasters are difficult to predict. Therefore, extreme scenarios have significant spatio-temporal and random characteristics. Simulate and generate different extreme scenarios respectively to construct an extreme scenario set with typical characteristics.

[0050] Specifically, in an embodiment of the present application, the obtaining the historical data of the extreme disaster scenarios of the power grid and performing preliminary processing to obtain an extreme scenario set includes:

[0051] S301: Classify and extract based on the historical data of the extreme disaster scenarios of the power grid to obtain extreme scenario samples.

[0052] S303: For extreme scenario samples of different categories, generate an extreme scenario set using a simulation method based on their dynamic characteristics.

[0053] In an embodiment of the present application, for the low output scenario of the generating unit and the load surge scenario, through comprehensive analysis of the new energy output data and load data of a certain regional power grid, the threshold method is used to identify and extract extreme scenario samples. According to the electrical quantities such as the generating unit structure, grid line topology, and load dynamic characteristics of the system, set the lower limits of the output of conventional units and new energy units and the upper limit of the load, identify and screen the scenarios in the historical scenario sample set where the output of the generating unit is lower than the threshold and the load level exceeds the load limit, and extract typical extreme scenarios to construct a sparse extreme scenario set. The criteria for the low output scenario of the generating unit and the load surge scenario are as follows:

[0054]

[0055] Wherein, represents the total output of all generators at node , represents the total maximum output of all generators at node ; represents the load value of node , Represents the maximum load value of the node ; and respectively represent the low output coefficient of the generating unit and the load growth coefficient. When the output power of the generating unit is less than the low output threshold, it is determined that the unit enters the low output state; when the regional load exceeds the limit load margin, it is determined that the power grid operation scenario is an extreme load surge scenario.

[0056] The simulation and expansion of the sparse scenario set are realized by using the meteorological model simulation method and the Monte Carlo sampling method. Establish the coupling relationship between the sparse scenario set and the regional meteorological conditions, map the extreme climate condition scenarios through the output data and load data of the generating units in the sample set, simulate the regional extreme meteorology based on the climate model, and then reconstruct the sparse scenario set sequence, recalculate the generator output data and load data, and simulate and generate more extreme scenarios. For the generated discrete extreme scenario sample set, use the Monte Carlo sampling method to sample and simulate to obtain a continuous time-series extreme scenario sample set, and construct a probabilistic model of the extreme scenario set based on the generator and load data generated by the meteorological simulation model, so as to sample and generate a complete extreme scenario set.

[0057] In addition, in an embodiment of the present application, the obtaining of the historical data of the extreme disaster scenarios of the power grid and the preliminary processing to obtain the extreme scenario set further includes:

[0058] Based on the system power and electricity balance relationship, perform secondary screening on the historical data of the extreme disaster scenarios of the power grid, and extract the supplementary scenario set with power and electricity imbalance.

[0059] In an embodiment of the present application, for the severe extreme scenario of the transmission grid framework disconnection, its occurrence probability is extremely small, and it is difficult to find a scenario with typical characteristics. Therefore, for the simulation of the disconnection scenario, considering from different directions of typhoon invasion, ignoring the transient process, only considering the transmission grid framework optimization scheme that the system can still be put into operation when some nodes are off-grid. After the grid framework has a disconnection fault, it can be expressed as:

[0060]

[0061] Among them, represents the disconnection probability factor, which is a set of random arrays between 0 and 1, and is used to represent the possibility of random disconnection of the transmission grid framework under typhoon invasion, and respectively represent the set of transmission lines before and after the disconnection occurs.

[0062] S203: Construct an objective function for the expansion and optimization of the transmission grid framework, and the objective function for the expansion and optimization of the transmission grid framework includes economic indicators and stability indicators.

[0063] In the embodiments of the present application, to solve the problem of transmission expansion planning for large-scale grid connection of new energy in extreme scenarios, it is necessary to consider the construction and operation costs of the transmission grid framework, and improve the voltage support intensity of the transmission grid framework on the premise of considering economy to cope with the large-capacity access of new energy. Therefore, the construction, operation and maintenance costs of the transmission grid framework are used as economic indicators, and the short-circuit ratio of the new energy grid connection point of the grid framework is used as a stability indicator, and an expansion optimization objective function of the transmission grid framework is constructed by taking into account economy and stability.

[0064] In one embodiment of the present application, the construction of the expansion optimization objective function of the transmission grid framework includes:

[0065] S401: Determine the construction, operation and maintenance costs of the transmission grid framework based on the construction investment cost, operation and maintenance cost of the transmission grid and the additional cost of new energy grid connection.

[0066] S403: Determine the voltage support intensity margin of the transmission grid framework based on the short-circuit ratio and critical short-circuit ratio of the new energy grid connection point.

[0067] S405: Determine the expansion optimization objective function of the transmission grid framework based on the construction, operation and maintenance costs of the transmission grid framework and the voltage support intensity margin of the transmission grid framework.

[0068] In one embodiment of the present application, the construction and operation costs of the transmission grid framework and the voltage support intensity of the grid framework are used as optimization objectives. Through the construction investment cost of the transmission grid , operation and maintenance cost and the additional cost of new energy grid connection The comprehensive total cost composed of is used as an indicator to measure the economy of grid framework construction. The short-circuit ratio margin of the new energy grid connection point is used as an indicator to measure the stability of grid framework construction, and an optimization objective function of the transmission grid framework expansion planning model with multi-infeed of new energy in extreme scenarios is constructed.

[0069] The construction investment cost of the transmission grid mainly considers the line construction cost between grid framework nodes and the line construction cost of new energy units accessing the grid framework. Its expression is as follows:

[0070]

[0071] Among them, represents the construction investment cost of the transmission grid, and respectively represent the line construction status sets between grid framework nodes and for new energy units accessing the grid, while and respectively represent the [0,1] construction status variables therein; represents the line The investment price of construction.

[0072] The operation and maintenance cost of the power transmission network includes the power generation costs of synchronous machines and new energy units, the line power loss cost, and the inspection and maintenance cost of the power transmission network. Its expression is as follows:

[0073]

[0074] Among them, represents the operation and maintenance cost of the power transmission network, and respectively represent the total power generation of synchronous machines and new energy units, while and correspond to their operation cost coefficients respectively, and respectively represent the line loss cost coefficient and the line maintenance cost coefficient, represents the set of constructed lines. Since constructing one line corresponds to two grid nodes, therefore and respectively represent the phase angles of the two nodes connected by the line, represents the resistance value of this line, represents the total length of all constructed lines.

[0075] Due to the volatility and intermittency of new energy unit power generation, additional costs will be incurred during the grid connection process, mainly including power fluctuation penalty costs and curtailment of wind and solar penalty costs. Their expression is as follows:

[0076]

[0077] Among them, represents the additional cost of new energy grid connection, represents the actual power fluctuation caused by new energy unit power generation to the line, while represents the line power flow under normal operating conditions. When the power fluctuation is small, the curtailment of wind and solar of new energy units is not serious, so the penalty cost is not considered; and respectively represent the power fluctuation penalty cost coefficient and the curtailment of wind and solar penalty cost coefficient. When the power fluctuation is obvious, the new energy grid connection penalty cost is imposed on the grid planning according to the power fluctuation amount.

[0078] The operation, maintenance cost of the power transmission network framework, that is, the comprehensive total cost, can be expressed as:

[0079]

[0080] When planning the transmission grid, the static voltage stability of the grid is approximately analyzed by the static analysis method. The multi-infeed model of new energy is equivalent to Thevenin, and the equivalent decoupling analysis of the AC grid and new energy is realized based on the superposition theorem. The voltage characteristics of the transmission grid remain unchanged before and after the equivalence. The stability of the grid expansion plan is measured by the short-circuit ratio index of the new energy connection point, and the calculation formula is as follows:

[0081]

[0082] Among them, represents the short-circuit ratio, represents the nominal voltage of the new energy connection point, represents the equivalent electromotive force of the connection point, represents the offset voltage of the connection point, represents the node voltage of the new energy connection point.

[0083] The critical short-circuit ratio is the short-circuit ratio corresponding to the critical stable state of the system, which can be used to evaluate the strength of the transmission grid. The calculation formula of the critical short-circuit ratio of the new energy connection point is as follows:

[0084]

[0085] Among them, represents the critical short-circuit ratio, is the short-circuit capacity provided by the synchronous machine to the new energy connection point and corresponds to the maximum transmission power of the line when the connection point is in the critical stable state, is the reactive power corresponding to the imaginary part of the equivalent grid-connected capacity of new energy in the grid-connected capacity.

[0086] The difference between the short-circuit ratio of the connection point and the critical short-circuit ratio is used as the voltage support strength margin of the transmission grid. The greater the voltage support strength, the stronger the voltage support strength of the system. The calculation formula of the stability margin is as follows:

[0087]

[0088] Therefore, the optimization objective function of the transmission grid expansion planning model with multi-infeed of new energy under extreme scenarios can be expressed as:

[0089]

[0090] Among them, represents the objective function. Since the stability margin needs to be maximized, it is expressed as a negative value here.

[0091] S205: Set the constraint conditions based on the above transmission grid expansion optimization objective function.

[0092] In the embodiments of the present application, constraint conditions are set based on the optimization objective function, that is, a constraint set is set for the transmission grid expansion planning model of multi-feed of new energy under extreme scenarios constructed.

[0093] Specifically, in an embodiment of the present application, the constraint conditions include node power balance constraints, power flow balance constraints, line transmission capacity constraints, voltage support strength margin constraints, and N-1 security constraints.

[0094] In an embodiment of the present application, the expression of the node power balance constraint is as follows:

[0095]

[0096] Among them, represents the total output of all synchronous machines at node , represents the total output of all new energy units at node , represents the difference between the incoming and outgoing power of the line at node , represents the total load at node .

[0097] The expression of the power flow balance constraint is as follows:

[0098]

[0099] Among them, represents the susceptance value of line , and respectively represent the phase angles of the two nodes connected by the line.

[0100] The expression of the line transmission capacity constraint is as follows:

[0101]

[0102] Among them, represents the maximum transmission power of the line.

[0103] The expression of the voltage support strength margin constraint is as follows:

[0104]

[0105] The expression of the N-1 security constraint is as follows:

[0106]

[0107] Among them, represents the node under the N-1 condition The total output of all synchronous machines above, indicating the node under the N-1 condition The total output of all new energy units above, indicating the node under the N-1 condition The difference between the incoming and outgoing power of the line at, indicating the node under the N-1 condition The total load at. When the agent performs a new action, it will evaluate the previous action and return a reward value, and substitute the power grid into the N-1 scenario to check the voltage stability of the grid framework, so as to guide the next action.

[0108] S207: The new energy multi-infeed power grid framework expansion planning model is solved by using the TD3 algorithm improved based on the multi-task learning algorithm based on the extreme scenario set and the power grid framework expansion optimization objective function.

[0109] In the embodiment of the present application, the TD3 algorithm is a reinforcement learning algorithm based on the action-evaluation framework, which can be used to solve the continuous action space problem of power grid planning under extreme scenarios. The TD3 algorithm is used to realize the state conversion of transmission lines. Based on the current line state, the action network selects an action to execute, while the evaluation network evaluates the executed action. By executing the action, the line state is converted and a reward value is obtained until the agent finds a planning scheme that takes into account both the economy and reliability of the power grid framework. The action value function Q is independently estimated by two evaluation networks, and the smaller value of the estimated values is taken to calculate the target Q value, effectively reducing the overestimation problem caused by the maximization operation. At the same time, since the target network updates parameters more slowly than the evaluation network, the process of calculating the target Q value is more stable, and the bootstrap problem is solved to a certain extent. As Figure 3 shown, it is a schematic structural diagram of the TD3 algorithm.

[0110] Since the TD3 algorithm takes the line state space as the input of the algorithm when solving the power grid planning problem, it is difficult to take into account the characteristics of each line by calculating through a single Actor network when calculating the action probability of each line, and it is affected by different experiences. Therefore, the TD3 agent is improved by using the multi-task learning algorithm (MMOE). The parameter sharing between tasks is realized through the expert network, which improves the ability of the agent to extract the same knowledge; while the multi-gating mechanism and the specific task tower can independently solve the action probability of each line, and extract information other than the same knowledge from multiple perspectives, so as to ensure that the agent can share the expert experience and calculate independently in a differentiated manner when solving tasks, effectively improving the solving efficiency of the model and achieving better policy learning in different environments. As Figure 4 shown, it is a schematic diagram of the improvement of the action network by the multi-task learning algorithm, where represents the probability of selecting a certain action, Indicates the execution of an action, and Tower represents multiple tower networks.

[0111] Specifically, in an embodiment of the present application, the improved TD3 algorithm based on the multi-task learning algorithm includes:

[0112] S501: Select the execution action for changing the state of the transmission line using the action network in the TD3 algorithm.

[0113] S503: Improve the action network and the corresponding target action using the multi-task learning algorithm.

[0114] S505: Screen the key transmission lines through the dual evaluation network, and calculate the reward value generated by the execution action based on the dual target network.

[0115] In an embodiment of the present application, the action network in the TD3 algorithm is used to select the execution action for changing the state of the transmission line. The key transmission lines are screened through the dual evaluation network, and the reward value generated by the execution action is calculated based on the dual target network. According to the current line state set and the action network, select and execute the action to obtain the next state and the reward , constituting the algorithm state space as follows:

[0116]

[0117] First, based on the state and the target action network, generate the execution action , then based on the idea of the dual network, estimate the target Q value through the relatively smaller value in the two target evaluation networks, and finally use the gradient descent algorithm to minimize the error between the evaluation value and the target value, thereby updating the parameters in the dual evaluation network. The calculation formula is as follows:

[0118]

[0119] Among them, represents the target action network, represents the target action network parameters, represents the target comment network parameters, represents the comment network parameters, represents the dual target comment network, represents the profit factor, represents the target Q value, represents the loss function of the comment network, represents the number of samples, represents the action generated by the action network.

[0120] The TD3 algorithm introduces a delayed policy update mechanism, that is, the action network is updated only after a certain number of updates to the critic network, so as to provide more stable feedback for the action network. The update of the action network is implemented based on the gradient ascent method, and its loss function and gradient are as follows:

[0121]

[0122] Improve the Actor network and the corresponding target Actor based on the idea of the multi-gated mixture-of-experts network, realizing the improvement from calculating the action probabilities of multiple lines by a single Actor network to calculating the action probabilities of each line by a multi-task tower network respectively.

[0123] In an embodiment of the present application, the algorithm formula for improving the action network and the corresponding target action by using the multi-task learning algorithm is:

[0124]

[0125]

[0126]

[0127]

[0128] Among them, represents the probability of performing an action on each line, represents the set of current line states, 、 and represent the expert network, the gating network, and the tower network respectively, 、 and represent the parameters of the expert network, the gating network, and the tower network respectively, represents the weight vector generated by the gating network, represents the set of weighted action probabilities, represents performing an action.

[0129] In the above method for optimizing the transmission network framework expansion plan considering extreme scenarios, first, historical data of the extreme disaster scenarios of the power grid is obtained and preliminarily processed to obtain an extreme scenario set; then, an objective function for optimizing the transmission network framework expansion is constructed, and the objective function for optimizing the transmission network framework expansion includes economic indicators and stability indicators; then, constraint conditions are set based on the objective function for optimizing the transmission network framework expansion; finally, the TD3 algorithm improved based on the multi-task learning algorithm is used to solve the new energy multi-infeed transmission network framework expansion plan model based on the extreme scenario set and the objective function for optimizing the transmission network framework expansion. That is to say, through the simulation generation of the extreme scenario set, as the interaction environment of the new energy multi-infeed transmission network framework expansion plan model, it helps to optimize the structural design and operation mode of the power grid, reduce the dependence of the system on a single device or line, thereby enhancing the overall risk resistance ability of the power grid, contributing to enhancing the structural toughness of the power grid, and comprehensively improving the safe operation level of the power grid under extreme weather conditions and enhancing the emergency response ability under extreme weather. At the same time, based on the economy and stability of the transmission network framework expansion plan, considering the constraint conditions, a transmission network framework expansion plan model is constructed to ensure that the transmission network framework can still operate normally after a failure under extreme weather conditions, thereby supporting the safe dispatching and control of the entire power grid, and fundamentally improving the stability, reliability and risk resistance ability of the power grid. And the TD3 algorithm improved based on the multi-task learning algorithm is used to solve the transmission network planning scheme that takes into account economy and stability. Based on the excellent performance of the TD3 algorithm, it is ensured that while the intelligent agent shares the same expert experience, it can also independently extract other information, enabling the intelligent agent to better adapt to different environmental changes, thereby improving the generalization ability and calculation accuracy.

[0130] The following uses a specific embodiment to illustrate the specific implementation steps of the method for optimizing the transmission network framework expansion plan considering extreme scenarios of the present application. As Figure 5 shown, first, in S601, historical data of the extreme disaster scenarios of the power grid is obtained and preliminarily processed to obtain an extreme scenario set. Specifically, in S603 - S607, extreme scenario samples are extracted based on the classification of the historical data of the extreme disaster scenarios of the power grid; for different types of extreme scenario samples, an extreme scenario set is generated by using a simulation method based on their dynamic characteristics; based on the power balance relationship of the system, the historical data of the extreme disaster scenarios of the power grid is screened again to extract a supplementary scenario set with power and electricity imbalance.

[0131] After that, in S609, an objective function for optimizing the expansion of the power transmission network framework is constructed, and the objective function for optimizing the expansion of the power transmission network framework includes economic indicators and stability indicators. Specifically, in S611 - S615, the operation and maintenance cost of the power transmission network framework is determined based on the construction investment cost, operation and maintenance cost of the power transmission network, and additional cost for new energy grid connection; the voltage support strength margin of the power transmission network framework is determined based on the short - circuit ratio and critical short - circuit ratio of the new energy grid connection point; the objective function for optimizing the expansion of the power transmission network framework is determined based on the operation and maintenance cost of the power transmission network framework and the voltage support strength margin of the power transmission network framework.

[0132] After that, in S617, constraint conditions are set based on the objective function for optimizing the expansion of the power transmission network framework. In S619, the constraint conditions include node power balance constraint, power flow balance constraint, line transmission capacity constraint, voltage support strength margin constraint, and N - 1 security constraint.

[0133] Finally, in S621, a new energy multi - feed - in power transmission network framework expansion planning model is solved using the TD3 algorithm improved based on the multi - task learning algorithm based on the extreme scenario set and the objective function for optimizing the expansion of the power transmission network framework. Specifically, in S623 - S627, the execution action for changing the state of the transmission line is selected by the action network in the TD3 algorithm; the action network and the corresponding target action are improved using the multi - task learning algorithm; the key transmission lines are screened through the dual evaluation network, and the reward value generated by the execution action is calculated based on the dual target network. In S629, the algorithm formula for improving the action network and the corresponding target action using the multi - task learning algorithm is:

[0134]

[0135]

[0136]

[0137]

[0138] Among them, represents the probability of the execution action for each line, represents the current line state set, 、 and represent the expert network, the gating network, and the tower network respectively, 、 and represent the parameters of the expert network, the gating network, and the tower network respectively, represents the weight vector generated by the gating network, represents the weighted action probability set, represents the execution action.

[0139] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are sequentially shown as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless specifically stated herein, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in rotation with at least a part of other steps or steps or stages in other steps.

[0140] Based on the same inventive concept, an embodiment of the present application further provides an extreme-scenario-considering transmission grid expansion planning optimization device for implementing the above-mentioned extreme-scenario-considering transmission grid expansion planning optimization method. The solution provided by this device for solving problems is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the extreme-scenario-considering transmission grid expansion planning optimization device provided below can refer to the limitations on the extreme-scenario-considering transmission grid expansion planning optimization method in the above text, and will not be repeated here.

[0141] In one embodiment, as Figure 6 shown, an extreme-scenario-considering transmission grid expansion planning optimization device 600 is provided, including: an extreme scenario set acquisition module 601, a target function construction module 603, a constraint condition setting module 605, and a transmission grid expansion planning optimization module 607, where:

[0142] The extreme scenario set acquisition module 601 is configured to acquire historical data of grid extreme disaster scenarios and perform preliminary processing to obtain an extreme scenario set.

[0143] The target function construction module 603 is configured to construct a transmission grid expansion optimization target function, and the transmission grid expansion optimization target function includes economic indicators and stability indicators.

[0144] The constraint condition setting module 605 is configured to set constraint conditions based on the transmission grid expansion optimization target function.

[0145] The transmission grid expansion planning optimization module 607 is configured to use the TD3 algorithm improved based on the multi-task learning algorithm to solve the new energy multi-feed-in transmission grid expansion planning model based on the extreme scenario set and the transmission grid expansion optimization target function.

[0146] In an embodiment of the present application, the extreme scenario set acquisition module is further configured to:

[0147] Classify and extract the historical data of the extreme disaster scenarios of the power grid to obtain extreme scenario samples;

[0148] For extreme scenario samples of different categories, generate an extreme scenario set using a simulation method based on their dynamic characteristics.

[0149] In an embodiment of the present application, the extreme scenario set acquisition module is further configured to:

[0150] Based on the power and electricity balance relationship of the system, perform secondary screening on the historical data of the extreme disaster scenarios of the power grid, and extract a supplementary scenario set with power and electricity imbalance.

[0151] In an embodiment of the present application, the objective function construction module is further configured to:

[0152] Determine the construction, operation, and maintenance costs of the transmission grid framework based on the construction investment costs, operation and maintenance costs of the transmission grid, and additional costs for new energy grid connection;

[0153] Determine the voltage support strength margin of the transmission grid framework based on the short-circuit ratio and critical short-circuit ratio of the new energy grid connection point;

[0154] Determine the expansion and optimization objective function of the transmission grid framework based on the construction, operation, and maintenance costs of the transmission grid framework and the voltage support strength margin of the transmission grid framework.

[0155] In an embodiment of the present application, the constraint conditions include node power balance constraints, power flow balance constraints, line transmission capacity constraints, voltage support strength margin constraints, and N-1 security constraints.

[0156] In an embodiment of the present application, the TD3 algorithm improved based on the multi-task learning algorithm includes:

[0157] Use the action network in the TD3 algorithm to select the execution action for changing the state of the transmission line;

[0158] Use the multi-task learning algorithm to improve the action network and the corresponding target action;

[0159] Screen key transmission lines through a dual evaluation network, and calculate the reward value generated by the execution action based on the dual target network.

[0160] In an embodiment of the present application, the algorithm formula for using the multi-task learning algorithm to improve the action network and the corresponding target action is:

[0161]

[0162]

[0163]

[0164]

[0165] Among them, represents the probability of each line executing an action, represents the set of current line states, , and respectively represent the expert network, the gating network, and the tower network, , and respectively represent the parameters of the expert network, the gating network, and the tower network, represents the weight vector generated by the gating network, represents the set of weighted action probabilities, represents the executed action.

[0166] Each module in the above transmission network framework expansion planning optimization device considering extreme scenarios can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in the processor in the computer device in hardware form or independent of it, or stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0167] In one embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 7 shown. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it realizes a method for optimizing the transmission network framework expansion planning considering extreme scenarios. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, a touchpad, or a mouse, etc.

[0168] Those skilled in the art can understand, Figure 7The structure shown is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0169] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memories can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., and are not limited thereto. The processors involved in the embodiments provided in this application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., and are not limited thereto.

[0170] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

[0171] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. An optimization method for transmission network expansion planning considering extreme scenarios, characterized in that, The method includes: Obtaining historical data of extreme disaster scenarios of the power grid and performing preliminary processing to obtain an extreme scenario set; Constructing an optimization objective function for the expansion of the transmission grid framework, where the optimization objective function for the expansion of the transmission grid framework includes economic indicators and stability indicators; Setting constraint conditions based on the optimization objective function for the expansion of the transmission grid framework; Using the TD3 algorithm improved based on the multi-task learning algorithm MMoE to solve for the new energy multi-feed-in transmission grid framework expansion planning model based on the extreme scenario set and the optimization objective function for the expansion of the transmission grid framework; The obtaining of historical data of extreme disaster scenarios of the power grid and performing preliminary processing to obtain an extreme scenario set includes: Classifying and extracting based on the historical data of extreme disaster scenarios of the power grid to obtain extreme scenario samples; For extreme scenario samples of different categories, generating an extreme scenario set using a simulation method based on their dynamic characteristics, where the dynamic characteristics refer to the mapping relationship between generator output data and load data and extreme climate condition scenarios; The obtaining of historical data of extreme disaster scenarios of the power grid and performing preliminary processing to obtain an extreme scenario set further includes: Based on the system power and electricity balance relationship, performing secondary screening on the historical data of extreme disaster scenarios of the power grid and extracting a supplementary scenario set with power and electricity imbalance; The TD3 algorithm improved based on the multi-task learning algorithm MMoE includes: Using the action network in the TD3 algorithm to select the execution action for changing the state of the transmission line; Using the multi-task learning algorithm MMoE to improve the action network and the corresponding target action; Screening key transmission lines through a dual evaluation network and calculating the reward value generated by the execution action based on the dual target network; The algorithm formula for using the multi-task learning algorithm MMoE to improve the action network and the corresponding target action is: Among them, represents the probability of each line performing an action, represents the set of current line states, , and respectively represent the expert network, the gating network, and the tower network, , and respectively represent the parameters of the expert network, the gating network, and the tower network, represents the weight vector generated by the gating network, represents the set of weighted action probabilities, represents the execution action, is the expert network index, representing the k-th expert network, is the number of expert networks.

2. The method for optimizing the transmission network framework expansion plan considering extreme scenarios according to claim 1, characterized in that The constructing of the optimization objective function for the expansion of the transmission grid framework includes: Determining the construction, operation, and maintenance costs of the transmission grid framework based on the construction investment costs, operation and maintenance costs, and additional costs for new energy grid connection of the transmission grid; Determining the voltage support strength margin of the transmission grid framework based on the short-circuit ratio and critical short-circuit ratio of the new energy grid connection point; Determining the optimization objective function for the expansion of the transmission grid framework based on the construction, operation, and maintenance costs of the transmission grid framework and the voltage support strength margin of the transmission grid framework.

3. The optimized method for the expansion planning of the transmission network framework considering extreme scenarios according to claim 1, characterized in that The constraint conditions include node power balance constraints, power flow balance constraints, line transmission capacity constraints, voltage support strength margin constraints, and N-1 security constraints.

4. An optimization device for the transmission network framework expansion planning considering extreme scenarios, characterized in that, The device includes: An extreme scenario set acquisition module for obtaining historical data of extreme disaster scenarios of the power grid and performing preliminary processing to obtain an extreme scenario set; A target function construction module for constructing an optimization objective function for the expansion of the transmission grid framework, where the optimization objective function for the expansion of the transmission grid framework includes economic indicators and stability indicators; A constraint condition setting module for setting constraint conditions based on the optimization objective function for the expansion of the transmission grid framework; A transmission grid framework expansion planning optimization module for using the TD3 algorithm improved based on the multi-task learning algorithm MMoE to solve for the new energy multi-feed-in transmission grid framework expansion planning model based on the extreme scenario set and the optimization objective function for the expansion of the transmission grid framework; The obtaining of historical data of extreme disaster scenarios of the power grid and performing preliminary processing to obtain an extreme scenario set includes: Based on the classification and extraction of the historical data of the extreme disaster scenarios of the power grid, extreme scenario samples are obtained; For extreme scenario samples of different categories, an extreme scenario set is generated by a simulation method based on their dynamic characteristics, where the dynamic characteristics refer to the mapping relationship between the output data of the generating units and the load data and the extreme climate condition scenarios; The obtaining of the historical data of the extreme disaster scenarios of the power grid and the preliminary processing to obtain the extreme scenario set further includes: Based on the power and electricity balance relationship of the system, the historical data of the extreme disaster scenarios of the power grid is screened for the second time, and a supplementary scenario set with power and electricity imbalance is extracted; The improved TD3 algorithm based on the multi-task learning algorithm MMoE includes: An execution action for changing the state of the transmission line is selected by the action network in the TD3 algorithm; The multi-task learning algorithm MMoE is used to improve the action network and the corresponding target action; The key transmission lines are screened through a dual evaluation network, and the reward value generated by the execution action is calculated based on the dual target network; The algorithm formula for improving the action network and the corresponding target action by using the multi-task learning algorithm MMoE is: Among them, represents the probability of each line performing an action, represents the set of current line states, , and respectively represent the expert network, the gating network, and the tower network, , and respectively represent the parameters of the expert network, the gating network, and the tower network, represents the weight vector generated by the gating network, represents the set of weighted action probabilities, represents the action to be executed, is the expert network index, representing the k-th expert network, is the number of expert networks.

Images