A data-driven multi-mobile emergency power supply resilience optimization scheduling method

Abstract

The present invention relates to the technical field of distribution network resilience restoration, and specifically discloses a data-driven multi-mobile emergency power resilience optimization scheduling method, which specifically includes the following steps: first, mathematically modeling the optimization problem, constructing a multi-level load shedding agent model, and training the multi-level load shedding agent model based on the least squares support vector machine regression algorithm; then constructing an optimization decision model based on multi-agent deep reinforcement learning, converting the collaborative optimization problem involving multiple MPSs into a partially observable Markov process, and embedding the agent model therein to accurately estimate the reward value of the agent during the training process, and finally, solving the MASAC algorithm through a centralized training-distributed execution framework. The present invention adopts the above-mentioned data-driven multi-mobile emergency power resilience optimization scheduling method to achieve real-time collaborative optimization of MPSs and power grid topology transformation decisions while ensuring the privacy of the distribution network, thereby effectively improving the resilience of the power grid.

Description

Technical Field

[0001] The present invention relates to the technical field of distribution network resilience restoration, and in particular to a data-driven multi-mobile emergency power resilience optimization scheduling method. Background Art

[0002] The increasingly frequent disturbances and shocks of extreme events have become the "new normal" facing the power system. Building a strong local power grid that is resilient and can recover quickly and improving the power grid's ability to resist natural disasters and prevent and control risks are inevitable requirements for building a resilient urban power grid.

[0003] In the face of extreme events, the topological structure of the power grid is vulnerable to damage, which will cause the load inside the distribution network to lose connection with the main grid. In the new distribution network, by integrating distributed generation, emergency mobile resources and energy storage systems to build local power grids or microgrids, the power balance and load demand in the large power grid can be borne by the formed local power supply area. In order to achieve the maximum recoverable load and reduce the duration of power outages, it is necessary to develop a planning and operation strategy based on the resilience of deployed distributed energy, and use mobile emergency power sources with significant flexibility advantages, including path planning, energy configuration, and charging and discharging scheduling. Resilience enhancement strategies.

[0004] With the continuous development of artificial intelligence technology, reinforcement learning has been used in the research of distribution network optimization scheduling. This method realizes the search for the optimal strategy in the decision space by continuously "trial and error" in a complex environment. Through offline training, the nonlinear fitting relationship of model mapping is learned from historical data to improve the generalization ability in practical applications, and the system is optimized in real time through online deployment. Wang Zihao et al. transformed the problem of improving the resilience of the distribution network into a multi-stage Markov decision process based on the proximal strategy optimization algorithm, constructed the action space through the switching of the tie switch, and interacted with the post-disaster distribution network environment to optimize the load power failure rate to achieve the improvement of the resilience of the distribution network. Huang Yuxiong et al. proposed a distribution network load recovery method considering the participation of microgrids, regarded the switching of the segmented switches between microgrids as the action space, constructed the Markov decision process considering the constraints such as user satisfaction and microgrid operation, and explored the dispatching strategy that optimizes the load shedding in the constructed fault simulation environment. Dehghani et al. modeled the problem of improving the aging of the distribution system facing hurricane risk as a sequential decision, and used the dominant action-evaluation method based on the risk ranking strategy to solve the intelligent agent, realizing the resilience optimization of the system to cope with various random extreme events during the life cycle.

[0005] Centralized control optimization strategies rely on real-time acquisition of global information. Communication interruptions and signal fluctuations will bring risks to the decision-making process. In addition, complex power systems require coordinated scheduling and optimization among components. Nie et al. proposed a dual-agent reinforcement learning model based on DQN and DDPG to achieve optimal control of power generation equipment and loads in isolated islands after extreme events. The goal is to optimize the source-load matching decision problem. However, the use of two independent algorithms to train the agent may lead to instability in the training performance during the strategy interaction process. Kamruzzaman et al. explored the advantages of reactive compensation equipment in resisting extreme events and multi-component failures, divided the power system into multiple control areas and trained based on the MASAC algorithm to achieve optimal deployment of reactive compensation equipment. Although the strategy is trained under the same algorithm framework, it does not study the synergistic effect of mobile emergency resources participating in resilience optimization. Wang et al. modeled each mobile emergency power source and maintenance worker as an agent participating in the recovery of isolated island loads, and constructed a sequential decision framework integrating multi-source collaborative support, line repair and emergency supply. By solving it in a multi-agent method, it can enhance the flexible collaboration of multiple distributed resources in the resilience recovery of the power system.

[0006] Although a variety of reinforcement learning algorithms are used to build single-agent or multi-agent optimization models for improving the resilience of power systems, after extreme events, the resilience optimization process of the distribution network faces the risks of complex line failures and real-time state transmission of the power grid. The real distribution network topology and line parameters are private and difficult to obtain accurately in real time through physical models. The focus of the research is to use the local information of multiple flexible emergency equipment to achieve rapid collaborative scheduling and optimization. As a cyber-physical fusion system, extreme events can easily lead to information transmission blockage. In order to explore the nonlinear relationship of power grid data and fully evaluate the system status, it is necessary to carry out data-driven resilience collaborative strategy research in scenarios where partial distribution network information is observable. Summary of the invention

[0007] The purpose of the present invention is to provide a data-driven multi-mobile emergency power supply resilience optimization scheduling method. Aiming at the problem that line parameters and topological structures in urban distribution networks are difficult to obtain, a data-driven resilience recovery strategy based on agent model-multi-agent is proposed. The proposed strategy realizes real-time collaborative optimization of MPS and grid topology transformation decision-making while ensuring the privacy of the distribution network, thereby effectively improving the resilience of the grid.

[0008] To achieve the above object, the present invention provides a data-driven multi-mobile emergency power resilience optimization scheduling method, comprising the following steps:

[0009] S1. Mathematical modeling of optimization problems;

[0010] S2, building a multi-level load shedding agent model;

[0011] S3. Construct an optimization decision-making model based on multi-agent deep reinforcement learning.

[0012] Preferably, in step S1, the specific steps of mathematical modeling for the optimization problem are as follows:

[0013] S11. The goal of distribution network resilience recovery is to optimize the load reduction of key system loads. Based on this, the mathematical model of the objective function is:

[0014]

[0015] In the formula, and They represent the load shedding cost at node i and the load shedding amount at time t respectively; and is the reduction of wind turbine and photovoltaic active power at time t; α is a weight coefficient used to adjust the weighted load reduction target and the proportion of the reduction target in the total target; B represents the node set in the entire distribution system, B pv , B wt It is a collection of photovoltaic and wind turbine equipment;

[0016] S12. Establish constraint conditions, including component operation model, distribution network radiation topology constraints and distribution network dynamic load recovery constraints.

[0017] Preferably, in step S12, the motion model of the component is as follows:

[0018]

[0019]

[0020]

[0021] In the formula, and Respectively represent the actual output of wind turbine and photovoltaic; η pv With η wt Respectively represent the maximum reduction range; and Respectively and simultaneously represent the active and reactive conversion limits of wind turbines and photovoltaic inverters, and constraints (2) to (4) restrict the active and reactive output of renewable energy;

[0022] The distribution network radiation topology constraints are as follows:

[0023] The single commodity flow method is introduced to construct a virtual network with the same topology and connectivity as the original power network. The virtual power flow must satisfy the power flow balance constraint:

[0024]

[0025]

[0026]

[0027]

[0028]

[0029]

[0030]

[0031] In the formula, the 0-1 binary variable c b,p,t represents the connection status of line (b, p) at time t; L is the line set; B source is the source node set; f p,b,t is the virtual power flow of line (b, p); represents the virtual power provided by the node with energy supply capability in the distribution network; δ(b) and π(b) represent the parent node and child node set at node b, respectively; x b,t is the unit virtual load at node b at time t; β b,p,t is a binary state variable representing the fault state of the line (b, p) at time t; L init represents the set of non-damaged initial lines without tie switches;

[0032] The dynamic load recovery constraints of the distribution network are as follows:

[0033] Based on the DistFlow power flow equation, the power balance including wind turbines, photovoltaics, MPS and controllable loads is shown in equations (12) to (15):

[0034]

[0035]

[0036]

[0037]

[0038] In the formula, and are the charging and discharging capacities of the MPS at the site respectively; in constraints (14) and (15), the on-off state variable c of the line (b, p) in the virtual power flow calculation is b,p,t , limiting the line flow balance including the tie switch.

[0039] Preferably, in step S2, constructing a multi-level load shedding agent model specifically includes the following steps:

[0040] S21. Component operation model

[0041] According to the optimization model in step S11, the optimal power flow calculation is performed to obtain the reduction data of each level of load and the total load of the distribution network, and then the multi-level load reduction agent model estimates the current power grid load reduction situation based on the real-time MPS, renewable energy and load data after supervised learning training using the obtained real data;

[0042] In supervised learning, the minimum-maximum normalization method is used to map each input and output variable to the interval [0,1], and remove the unsolvable data rows in the obtained results. The normalization and denormalization formulas of the data are as follows:

[0043]

[0044] Y real =Y predict ·(Y max -Y min )+Y min (17)

[0045] Where, X norm is the value after data normalization; X i represents the feature sequence that needs to be normalized; X max With X min Respectively represent the maximum and minimum values ​​in the feature sequence; Y predict is the normalized predicted value output by the model; Y real is the true value with the dimension of the original data after denormalization;

[0046] S22. Least Squares Support Vector Principle

[0047] The least squares support vector machine LSSVM uses the least squares method to solve and transform the inequality constraints of the above optimization model into equality constraints;

[0048] The purpose of the LSSVM regression problem is to find the mapping φ(x) of the sample from the original space to the high-dimensional feature space, that is, the prediction model is f(x) = w·φ(x) + b. Therefore, the slack variable ξ is introduced in LSSVM to measure the outlier error. The LSSVM parameter solution optimization problem is described as:

[0049]

[0050] In formula (18), γ is the regularization parameter that measures the error tolerance;

[0051] The Lagrange multiplier is introduced to transform the optimization problem into a single parameter:

[0052]

[0053] Under the KKT condition, the partial derivative of the parameters is obtained, and the obtained α and b are brought back to the original problem, and the LSSVM regression function is obtained as follows:

[0054]

[0055] In the formula, K(x,x i ) is the kernel function, as shown in formula (21):

[0056]

[0057] S23, constructing a proxy model and prediction process based on GA-LSSVM;

[0058] Genetic algorithm GA is used to optimize the hyperparameters, and the solution of the optimization problem is converted into screening chromosomes that adapt to the environment from individuals. Through replication, crossover and mutation genetic operations, the population that adapts to the environment is continuously generated according to the principle of survival of the fittest. The fitness function is used to evaluate the quality of individuals in the population. The fitness function expression is:

[0059]

[0060] In the formula, y i and are the predicted value and true value of the sample respectively; γ min With γ max are the lower and upper limits of the regularization parameter respectively; σ min With σ max The lower and upper limits of kernel function parameters;

[0061] The prediction process based on the GA-LSSVM agent model is as follows:

[0062] First, according to the constructed proxy model dataset, the training sample and test sample sets are selected, the search range of the hyperparameters γ and σ is set, and the hyperparameters are encoded.

[0063] Secondly, the LSSVM is trained based on the initialized hyperparameter values, and the fitness function constructed by formula (22) is used to determine whether the optimization goal is met;

[0064] Finally, when the optimized parameters make the optimization target reach the predetermined stopping condition, the optimal hyperparameters γ and σ are brought into the LSSVM model for training, and the model is analyzed using the test sample set.

[0065] Preferably, in step S3, constructing an optimization decision model based on multi-agent deep reinforcement learning specifically includes the following steps:

[0066] S31. Establish a Dec-POMDP framework based on resilience improvement, including observation space, action space, state transition and reward function;

[0067] S32. Use the MASAC algorithm to solve Dec-POMDP.

[0068] Preferably, in step S31, the observation space is:

[0069] At each time step t, the local observation value of the i-th MPS is o i.t , which is represented as a vector: This includes the on / off status of the distribution network lines c bp,t , the power generation of wind turbines and photovoltaics at time t The load demand of the distribution network at time t is the location where the MPS is connected in the distribution network, and the battery capacity state of the i-th MPS at time t

[0070] Introducing Observations To reflect the impact of the actions taken by the i-th MPS on the resilience recovery of the power grid:

[0071]

[0072] In formula (23), the denominator represents the total net recovery of the load by the MPS, and the numerator represents the discharge amount of the i-th MPS. The larger the value, the greater the contribution of the MPS to resilience recovery;

[0073] The action space is:

[0074] The action set of each MPS in a single iteration is a i,t , which is represented as a two-dimensional vector: in, is the charge and discharge amount of the i-th MPS at time t, The site location selected for it is actually scheduled For discrete variables, by rounding up and normalizing, it will be transformed from the position set Select from;

[0075] The state transition is:

[0076] In the scheduling period t∈[1,T], the state transition function between each time step t and t+1 is s t+1 =T(o 1:I,t ,a 1:I,t,w t ), where the random quantity affected by environmental factors at time t is

[0077] The MPS state is affected by its own actions and its position state By Action Considering the traffic flow, network topology, capacity status By Action At maximum charge and discharge power The mathematical expression of the MPS operation safety model is as follows:

[0078]

[0079]

[0080]

[0081] Equations (24) and (25) represent the charge and discharge action limits of MPS; Equation (26) represents the capacity The state transition, and are the charging and discharging efficiencies respectively; each MPS agent in the MADRL algorithm selects its own position With power The original optimization problem is equivalent to the optimal power flow problem of the distribution network after fixing the MPS position and power, thus obtaining the load reduction at each level;

[0082] The reward function is:

[0083] The agent transmits the actions of MPS to the agent model, and each agent i receives a reward r i.t , the negative value of the load reduction cost in the optimization objective is used as the team reward value;

[0084] Taking the critical load reduction as the criterion, we further construct a step reward for each MPS. Finally, we sum the individual reward and the team reward to generate a new individual reward function, so that the original sparse reward is transformed into a dense reward to provide effective feedback for each action of the agent. The reward function of the i-th MPS agent in a single time step is shown in formula (27), which consists of the following three parts: weighted load reduction cost r pcurl , Charging penalty items and discharge penalty

[0085]

[0086]

[0087]

[0088]

[0089]

[0090] Formula (28) is the weighted load reduction; Formula (29) represents the primary load reduction ratio, which is obtained indirectly through the proxy model fitting; Equations (30) and (31) represent the MPS charging and discharging penalties, respectively.

[0091] Preferably, in step S32, the MASAC algorithm is used to solve Dec-POMDP, and the specific process is as follows:

[0092] The MASAC algorithm extends the maximum entropy deep reinforcement learning algorithm SAC by adopting the CTDE framework. The SAC algorithm is based on the Actor-Critic framework and learns by simultaneously training the policy-based Actor network and the value-based Critic network. The Actor network is used to generate actions, and the Critic network evaluates the value of actions by approximating the behavior value function.

[0093] In the centralized training phase, the MPS agent i obtains the current state s through the proxy model. i,t 、Action a i,t , the states of other agents s -i,t 、Action a -i,t The information is input into the value network for parameter update, and the policy network updates the state s of the agent. i,t Perform parameter updates; the strategy and value networks in the MASAC algorithm are fitted through a deep neural network, and the minimum value output from the two Q-value networks is selected to avoid overestimation of the state value;

[0094] In the execution phase, the agents select actions based on their respective strategy networks and pass them to the proxy model, which then gives feedback r on the reward of agent i at time t. i,t ;

[0095] In the routing and power scheduling of N MPS agents, the strategy set is expressed as The action network is represented as in is the action strategy function of agent i, φ i Represents the parameters to be optimized of the policy network. SAC introduces the entropy function into the reward function. The policy needs to maximize the entropy value to optimize the policy. The objective function is expressed as follows:

[0096]

[0097] In the formula, T and r represent the number of time steps and the single-step reward respectively; a i,t with a -i,t Respectively represent the actions selected by agent i and other collaborators; α i ∈(0,1) is the temperature coefficient, which determines the relative importance of the entropy term and the reward term. The larger the α, the more exploration is achieved to achieve more randomness in the optimization strategy. For strategy The induced trajectory of Indicates state s t The state entropy under .

[0098] Preferably, in step S32, a soft value function is defined It is used to measure the strategic value of agent i, and its expression is:

[0099]

[0100] In the formula, ψ i is the network parameter of the value function of agent i; Q θ (s t ,a i,t ,a -i,t ) is the evaluation function of the strategy; θ is the network parameter of the Q-value function;

[0101] MASAC introduces two Q-value functions Q with the same structure θi,1 With Q θi,2 , the network loss function of the Q value function is as follows:

[0102]

[0103]

[0104] Among them, D is the experience buffer used to store previous experience data; is the target Q value function network parameter, and the gradient calculation of optimizing the Q value function is as follows:

[0105]

[0106] MASAC continuously obtains new Q-value functions and guides the strategy to update in a way of maximum return to obtain the optimal strategy. The strategy network is trained by minimizing the Kullback-Leibler divergence. The strategy update expression is as follows:

[0107]

[0108] In the formula, Z θ (s t) is the normalized distribution of Q value. In the calculation process, the reparameterization technique is introduced to convert a i,t Parameterized as

[0109] The strategy update target of agent i is as follows:

[0110]

[0111] In the formula, Parameterize strategies for other collaborators; i,t is a noise vector sampled from a Gaussian distribution;

[0112] SAC algorithm for temperature coefficient α i Adjustment is performed using a hyperparameter equivalent constant vector The objective function representing the entropy of the target policy is shown in formula (39):

[0113]

[0114] Therefore, the present invention adopts the above-mentioned data-driven multi-mobile emergency power resilience optimization scheduling method, and the beneficial effects are as follows:

[0115] (1) In the distribution network failure scenario, the present invention establishes a multi-MPS resilience collaborative optimization model under the optimal load reduction target, and achieves the optimal matching of island source and load through the scheduling of wind turbines, photovoltaics, and mobile power systems and distribution network reconstruction decisions.

[0116] (2) The present invention collects the historical data of distribution network sources and loads in the optimal power flow calculation, obtains the nonlinear mapping relationship between the power inflow and load reduction of the distribution network after the disaster through the GA-LSSVM regression algorithm, and constructs a multi-level load reduction agent model that can replace the physical model of the power grid.

[0117] (3) The present invention constructs the collaborative optimization problem in which MPS participates as a partially observable Markov process, and embeds a multi-level load shedding agent model into it to estimate the reward value during the agent training process; and solves the MASAC algorithm through a centralized training-distributed execution framework. While MPS achieves autonomous decision-making, it ensures the privacy of distribution network line parameters and topological structure.

[0118] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0119] Figure 1 It is a GA-LSSVM prediction flow chart of an embodiment of a data-driven multi-mobile emergency power resilience optimization scheduling method of the present invention;

[0120] Figure 2 It is a data-driven resilience recovery framework diagram of an embodiment of a data-driven multi-mobile emergency power resilience optimization scheduling method of the present invention;

[0121] Figure 3 It is a schematic diagram of a Dec-POMDP framework based on resilience improvement in an embodiment of a data-driven multi-mobile emergency power resilience optimization scheduling method of the present invention;

[0122] Figure 4 It is an overall algorithm framework diagram of an embodiment of a data-driven multi-mobile emergency power resilience optimization scheduling method of the present invention;

[0123] Figure 5 It is a schematic diagram of an improved IEEE 33-node power distribution system according to an embodiment of a data-driven multi-mobile emergency power supply resilience optimization scheduling method of the present invention;

[0124] Figure 6 It is a schematic diagram of MPS scheduling results in scenario 1 of an embodiment of a data-driven multi-mobile emergency power supply resilience optimization scheduling method of the present invention, wherein (a) represents the MPS1 scheduling result, (b) represents the MPS2 scheduling result, and (c) represents the MPS3 scheduling result;

[0125] Figure 7 It is the result of distribution network topology change in scenario 1 of an embodiment of a data-driven multi-mobile emergency power supply resilience optimization scheduling method of the present invention. DETAILED DESCRIPTION

[0126] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0127] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.

[0128] As shown in the figure, a data-driven multi-mobile emergency power resilience optimization scheduling method is characterized by comprising the following steps:

[0129] S1. For radial distribution networks, line failures after extreme events will cause the network to be decoupled and then decomposed into several sub-networks. In this scenario, the primary task of distribution network operators is to prioritize the supply of critical loads through limited resources. On this basis, when the power generation resources (wind turbines, photovoltaics) in the microgrid are affected by the environment and show a high output trend, it is necessary to weigh non-critical loads and MPS charging needs to generate the optimal strategy and mathematically model the optimization problem. The specific steps are as follows:

[0130] S11. The goal of distribution network resilience recovery is to optimize the load shedding of key system loads, reduce the active power reduction of renewable energy, and increase the consumption of new energy. Based on this, the mathematical model of the objective function is:

[0131]

[0132] In the formula, and They represent the load shedding cost at node i and the load shedding amount at time t respectively; and is the reduction of wind turbine and photovoltaic active power at time t; α is a weight coefficient used to adjust the weighted load reduction target and the proportion of the reduction target in the total target; B represents the node set in the entire distribution system, B pv , B wt It is a collection of photovoltaic and wind turbine equipment;

[0133] S12. Establish constraint conditions, including component operation model, distribution network radiation topology constraints and distribution network dynamic load recovery constraints.

[0134] The component mathematical model used is as follows:

[0135]

[0136]

[0137]

[0138] In the formula, and Respectively represent the actual output of wind turbine and photovoltaic; η pv With η wt Respectively represent the maximum reduction range; and Respectively and simultaneously represent the active and reactive conversion limits of wind turbines and photovoltaic inverters, and constraints (2) to (4) restrict the active and reactive output of renewable energy;

[0139] The distribution network radiation topology constraints are as follows:

[0140] Unlike the transmission system, the distribution network topology is radial. In the coordinated optimization of the present invention, the reconfiguration of the power network is achieved through the dynamic operation of intelligent soft switches, and the radiality of the distribution network topology must be guaranteed. In the dynamic load recovery process, the network radiality must meet two main conditions: the total number of connected lines is equal to the number of non-fault lines minus the number of microgrids, and for each microgrid, the source node and the key load node are connected, and there is a unique recovery path.

[0141] Based on this, in order to avoid forming a ring microgrid, the present invention introduces a single commodity flow method to construct a virtual network with the same topology and connectivity as the original power network. It is worth noting that the virtual network does not contain real power flow, load and source node information, and only serves as a mapping method of the actual network to assist power flow calculation.

[0142] The virtual power flow must satisfy the power flow balance constraint:

[0143]

[0144]

[0145]

[0146]

[0147]

[0148]

[0149]

[0150] In the formula, the 0-1 binary variable c b,p,t represents the connection status of line (b, p) at time t; L is the line set; B source is the source node set; f p,b,t is the virtual power flow of line (b, p); represents the virtual power provided by the node with energy supply capability in the distribution network; δ(b) and π(b) represent the parent node and child node set at node b respectively; x b,t is the unit virtual load at node b at time t, and is usually taken as 1 in actual calculations; β b,p,t is a binary state variable representing the fault state of the line (b, p) at time t; L init represents the set of non-damaged initial lines without tie switches;

[0151] In each microgrid, one of the nodes is regarded as a virtual source node, and the rest are virtual load nodes. The above condition (1) is satisfied in constraint (5), and condition (2) is satisfied in constraints (6) to (9). Among them, constraint (6) represents the virtual power flow balance at the load node; constraint (7) is the virtual power flow balance at the source node; constraints (8) and (9) use the Big-M method to forcibly limit the output range of the line virtual power flow and the virtual source node under different connectivity states. According to constraints (10) and (11), the fault branch always remains in the off state at time t, and at the initial moment of single-step optimization, the undamaged line will maintain the original on-off state;

[0152] The dynamic load recovery constraints of the distribution network are as follows:

[0153] Based on the DistFlow power flow equation, the power balance including wind turbines, photovoltaics, MPS and controllable loads is shown in equations (12) to (15):

[0154]

[0155]

[0156]

[0157]

[0158] In the formula, and are the charging and discharging capacities of the MPS at the site respectively; in constraints (14) and (15), the on-off state variable c of the line (b, p) in the virtual power flow calculation is b,p,t , limiting the power flow balance of the line including the tie switch; when c b,p,t When it is 0, the line (b, p) tie switch is in the off state, that is, through the tie switch operation, the dynamic reconstruction of the power grid can be realized to realize the dynamic distribution of line flow and load. The quadratic term constraint included in the optimal power flow calculation of the present invention is convexified by the second-order cone relaxation constraint.

[0159] S2. Construct a multi-level load shedding agent model, which specifically includes the following steps:

[0160] S21. Component operation model

[0161] In the agent model used, the optimal power flow calculation is performed according to the optimization model detailed in step S11 based on the given input conditions such as the MPS position, MPS charging and discharging power, renewable energy output, and load demand, to obtain the reduction of each level of load and the total load of the power grid under the input conditions. After supervised learning training using the real data obtained in the above process, the multi-level load reduction agent model estimates the current power grid load reduction based on real-time MPS, renewable energy and load data;

[0162] By randomly selecting values ​​for the input data within the feasible range, the training set and test set are constructed and represented as (X, Y) and (X * ,Y * ), where the training set and test set input are represented by X∈R n*9 and The output is represented by Y∈R n*3 and Some samples of the obtained data are shown in Table 1, and the above data contain different dimensional features. For example, the MPS position (L-1, L-2, L-3) is within 3, the output values ​​of wind turbines (WT) and photovoltaics (PV) are in the hundreds to thousands, and the load demand (Load) in different periods is used as the input value of its proportion in the total load. The reduction of total load (L_T), secondary load (L_2) and tertiary load (L_3) is in the order of thousands. At the same time, the unsolvable situation in the optimal power flow calculation will cause abnormal output values. For example, when the MPS charging demand and load demand are much higher than the energy supply, the optimal power flow calculation will have no solution (as shown in the bold line in the last row).

[0163] Table 1. Sample training data of multi-level load shedding of proxy model

[0164] L-1 P-1 L-2 P-2 L-3 P-3 PV WT Load L_T L_2 L_3 3 77.79 3 -110.9 1 84.07 190.6 181.6 0.98 3879.7 2383.6 1201.1 2 77.52 1 -75.41 3 104.41 15.6 1324.6 0.50 251.6 0 251.1 1 29.52 1 -33.63 1 -9.97 195.3 187.3 0.97 4049.4 2451.7 1199.2 1 50.76 2 -97.7 2 -31.64 130.1 184.1 0.63 2511.2 0 773.5 1 -103.3 2 -8.14 3 23.05 176.8 175.1 0.82 3271.1 2015.1 1015.6 3 96.48 2 70.03 1 93.15 23.6 1332.4 0.52 280.2 0 280.7 1 109.1 1 73.5 1 83.34 197.1 189.8 0.97 2000 2000 2000

[0165] In supervised learning, data quality has a crucial impact on the accuracy of the proxy model, and the input and output data need to be preprocessed. To address the above problems, the present invention adopts the minimum-maximum normalization method to map each input and output variable to the interval [0,1] and remove the unsolvable data rows in the obtained results. The normalization and denormalization formulas of the data are as follows:

[0166]

[0167] Y real =Y predict ·(Y max -Y min )+Y min (17)

[0168] Where, X norm is the value after data normalization; X i represents the feature sequence that needs to be normalized; X max With X min Respectively represent the maximum and minimum values ​​in the feature sequence; Y predict is the normalized predicted value output by the model; Y real It is the true value with the dimension of the original data after denormalization.

[0169] S22. Least Squares Support Vector Principle

[0170] Faced with large-scale data sets, the complexity of quadratic programming increases accordingly. The training speed based on support vectors and kernel functions is slow. At the same time, the adjustment of hyperparameters such as kernel function selection and balance factor in SVR makes model tuning more difficult. Least squares support vector machine LSSVM (least squares support vector machines) uses the least squares method to solve, converting the inequality constraints of the above optimization model into equality constraints, which has a faster solution speed. LSSVM has strong interpretability and better generalization ability and stability than linear regression and neural networks.

[0171] The purpose of the LSSVM regression problem is to find the mapping φ(x) of the sample from the original space to the high-dimensional feature space, that is, the prediction model is f(x) = w·φ(x) + b. Similarly, the slack variable ξ is introduced in LSSVM to measure the outlier error. The LSSVM parameter solution optimization problem is described as:

[0172]

[0173] In formula (18), γ is a regularization parameter that measures the degree of error tolerance, which is used to balance the hyperplane and sample deviation. Compared with SVR, LSSVM transforms the original problem into an equality constraint.

[0174] Furthermore, the Lagrange multiplier is introduced to transform the optimization problem into a single parameter:

[0175]

[0176] Under the KKT condition, the partial derivative of the parameters is obtained, and the obtained α and b are brought back to the original problem, and the LSSVM regression function is obtained as follows:

[0177]

[0178] In the formula, K(x,x i ) is the kernel function; the sample data used in the above introduction is completely or partially linear. For nonlinear data sets that may exist, the kernel function is introduced to map the input vector in the original feature space to the high-dimensional feature vector space. Commonly used kernel functions include linear kernel functions, polynomial kernel functions, Sigmoid kernel-multilayer perceptron kernel functions, etc.

[0179] The kernel function used in the present invention is a Gaussian radial basis kernel function with small deviation and high generalization ability as shown in formula (21):

[0180]

[0181] S23, constructing a proxy model and prediction process based on GA-LSSVM;

[0182] Through the above theoretical analysis, it can be known that the stability and accuracy of the LSSVM model in practical applications depend on the selection of the kernel function parameter σ and the regularization parameter γ. Only through trial and error, ergodic method and empirical method will consume a lot of computing resources, and it is difficult to determine the hyperparameters that make the model most accurate. Therefore, the present invention uses genetic algorithm GA (genetic Algorithm) to optimize the hyperparameters.

[0183] Genetic algorithm is a search algorithm based on the theory of natural evolution, which originates from the simulation of biological genetics and evolution process. The core concept of GA is to transform the solution of optimization problem into the selection of chromosomes that adapt to the environment from individuals, and continuously generate populations that adapt to the environment by following the principle of survival of the fittest through genetic operations of replication, crossover and mutation. The fitness function in the present invention is:

[0184]

[0185] In the formula, y i and are the predicted value and true value of the sample respectively; γ min With γ max are the lower and upper limits of the regularization parameter respectively; σ min With σ max The lower and upper limits of kernel function parameters;

[0186] The prediction process based on the GA-LSSVM agent model is as follows:

[0187] like Figure 1 As shown, first, according to the constructed proxy model dataset, the training sample and the test sample set are selected, the search range of the hyperparameters γ and σ is set, and the hyperparameters γ and σ are encoded;

[0188] Secondly, the LSSVM is trained based on the initialized hyperparameter values, and the fitness function constructed by formula (22) is used to determine whether the optimization goal is met. When the stopping condition is not met, the dominant individuals are found according to the crossover and mutation probabilities set in the GA algorithm, a new population is constructed, and the fitness function is calculated again.

[0189] Finally, when the optimized parameters make the optimization target reach the predetermined stopping condition, the optimal hyperparameters γ and σ are brought into the LSSVM model for training, and the model is analyzed using the test sample set.

[0190] S3. The optimization decision model based on multi-agent deep reinforcement learning specifically includes the following steps:

[0191] S31. Establish a Dec-POMDP framework based on resilience improvement, including observation space, action space, state transition and reward function;

[0192] like Figure 2 As shown, at each time step t, the environment is in a specific state s t And send out a set of joint observations O t , each agent i receives the local observation related to the state at the current time step, and according to its local observation value o i,t According to the strategy π(a i,t |o i,t )Select action a i,t Among them, the interactive environment is based on the state transfer function T(s,a 1:I ,w) enter the next state and obtain the reward value r of the agent in this state i,t This process uses the centralized training and decentralized execution (CTDE) framework, and the agent will maximize its cumulative reward value. In the present invention, the reward value corresponding to each step size is equally important in strategy evaluation, so γ is taken as 1. The specific components of the proposed Dec-POMDP are as follows:

[0193] (1) Observation space:

[0194] At each time step t, the local observation value of the i-th MPS is o i.t , which is represented as a vector: This includes the on / off status of the distribution network lines c bp,t , the power generation of wind turbines and photovoltaics at time t The load demand of the distribution network at time t is the location where the MPS is connected in the distribution network, and the battery capacity state of the i-th MPS at time t

[0195] Due to the complex scenario with multiple faulty lines and random output fluctuations of new energy, and the inability of direct information exchange between different MPS entities, there may be instability problems if only relying on the above local observations. Even if the intelligent agent is very likely to fall into the local optimal solution during the exploration process (MPS only takes conservative actions of neither charging nor discharging), it is difficult to improve the cumulative reward value. Therefore, the present invention introduces the observation value To reflect the impact of the actions taken by the i-th MPS on the resilience recovery of the power grid:

[0196]

[0197] In formula (23), the denominator represents the total net recovery of the load by the MPS, and the numerator represents the discharge amount of the i-th MPS. The larger the value, the greater the contribution of the MPS to resilience recovery;

[0198] pass The internal observation values ​​of the power grid and the local observation results of other MPS are implicitly included. In this process, the MPS obtains information through the power grid trusted platform (agent model) without knowing the activities of other MPS and power grid parameters, which fully protects the privacy of MPS and operators.

[0199] (2) Action Space:

[0200] The action set of each MPS in a single iteration is a i,t , which is represented as a two-dimensional vector: in, is the charge and discharge amount of the i-th MPS at time t, The site location selected for it is actually scheduled For discrete variables, by rounding up and normalizing, it will be transformed from the position set Select from;

[0201] (3) State transfer:

[0202] In the scheduling period t∈[1,T], the state transition function between each time step t and t+1 is s t+1 =T(o 1:I,t ,a 1:I,t ,w t ), where the random quantity affected by environmental factors at time t is These variables are related to the external environment such as radiation intensity, wind speed, and user behavior and are not controlled by the agent.

[0203] The data-driven DRL algorithm can effectively learn its probabilistic characteristics by gaining experience from the historical data of the environment. In addition to the state changes of the environment, the MPS state is affected by its own actions, and its position state By Action Considering the traffic flow, network topology, capacity status By Action At maximum charge and discharge power The mathematical expression of the MPS operation safety model is as follows:

[0204]

[0205]

[0206]

[0207] Formula (24) and (25) represent the charge and discharge action limits of MPS. When the selected action exceeds the upper and lower limits of the MPS capacity, the action will be performed according to the maximum charge and discharge amount within its current capacity. Formula (26) represents the capacity The state transition, and are the charging and discharging efficiencies respectively; each MPS agent in the MADRL algorithm selects its own position With power The original optimization problem is equivalent to the optimal power flow problem of the distribution network after fixing the MPS position and power, thus obtaining the load reduction at each level;

[0208] (4) Reward function:

[0209] The agent transmits the actions of MPS to the agent model, and each agent i receives a reward r i.t ,The goal of resilience recovery in the present invention is to minimize the load reduction of the key load of the distribution network, therefore, the negative value of the load reduction cost in the optimization ,objective is used as the team reward value;

[0210] However, in the actual resilience optimization process, the cumulative reward can only be obtained after a series of actions are executed and the optimization task is completed. It is easy to be affected by the actions selected in a few time steps, which can easily cause "sparse rewards" and "reward signal delay" problems. Specifically in the training process, if only team rewards are used as the optimization goal, the load reduction reward value under different inputs will be discontinuous, and the agent will need to spend more time to know the effectiveness of the current action sequence. On the other hand, since multiple agents share the environment and seek optimization under the same goal, the actions taken by each agent will also affect the current state of the environment and the actions of other agents, causing non-steady-state problems.

[0211] The present invention takes the critical load reduction as the criterion, further constructs a ladder reward for each MPS, and finally, sums the individual reward with the team reward to generate a new individual reward function, so that the original sparse reward is transformed into a dense reward to provide effective feedback for each action of the agent. The reward function of the i-th MPS agent in a single time step is shown in formula (27), which consists of the following three parts: weighted load reduction cost r pcurl , Charging penalty items and discharge penalty

[0212]

[0213]

[0214]

[0215]

[0216] Formula (28) is the weighted load reduction; Formula (29) represents the reduction ratio of the primary load, which is indirectly obtained by fitting the proxy model; Equations (30) and (31) represent the MPS charging and discharging penalties, respectively. It should be noted that MPS should be charged when there is no or little reduction in the primary load, and should choose to discharge when the reduction is large to help the overall recovery of the system.

[0217] In the above formula, for different ranges Segmented rewards are set, and penalties are set for time periods with more reductions but no power generation and less reductions but no charging. In addition, the location where the MPS is connected is also one of the factors that affect the system recovery effect. Therefore, in formula (27), the team reward weight κ1 is the highest and dominates the overall reward to avoid excessive attention to the MPS charging and discharging behavior and fall into local optimal decisions;

[0218] S32. Use MASAC algorithm to solve Dec-POMDP. The specific process is as follows:

[0219] Since the resilience optimization decision problem belongs to the field of continuous control, and the system model is a high-dimensional, linear optimization problem. Traditional reinforcement learning methods for solving this type of problem, such as deep deterministic policy gradient (DDPG), twin delayed DDPG (TD3) and other algorithms, are all deterministic strategies. They have weak exploration capabilities for power grid resilience restoration tasks in extreme environments, and are prone to falling into local optimal solutions during the training of intelligent agents. In order to avoid premature convergence of the agent during the optimization process and ensure the stability of the algorithm, the present invention uses the MASAC algorithm with stronger exploration capabilities to solve Dec-POMDP.

[0220] In traditional multi-agent training and execution, agents face difficulties in obtaining observation and action information from other agents, and learning instability is caused by the constant change of strategies. The MASAC algorithm uses the CTDE framework to theoretically extend the maximum entropy deep reinforcement learning algorithm SAC (soft actor-critic) in the multi-agent field. The SAC algorithm is based on the Actor-Critic framework and learns by simultaneously training the policy-based Actor network and the value-based Critic network. The Actor network is used to generate actions, and the Critic network evaluates the value of actions by approximating the behavior value function. The proposed algorithm framework is shown in Figure 3.

[0221] The load shedding amount, renewable energy output, MPS location and power data of the post-disaster power grid will be provided to the minimum support vector machine model. The minimum support vector machine model is optimized through the GA algorithm, and the fitted data is trained through supervised learning to train the multi-level load shedding agent model. In the centralized training stage, the MPS agent i obtains the current state s through the agent model. i,t 、Action a i,t , the states of other agents s -i,t 、Action a -i,t The information is input into the value network for parameter update, and the policy network updates the state s of the agent. i,t Perform parameter updates; the strategy and value networks in the MASAC algorithm are fitted through a deep neural network, and the minimum value output from the two Q-value networks is selected to avoid overestimation of the state value;

[0222] In the execution phase, the agents select actions based on their respective strategy networks and pass them to the proxy model, which then gives feedback r on the reward of agent i at time t. i,t ;

[0223] In the routing and power scheduling of N MPS agents, the strategy set is expressed as The action network is represented as in is the action strategy function of agent i, φ i Represents the parameters to be optimized of the policy network. As a random strategy, SAC introduces the entropy function in the reward value function. The strategy needs to maximize the entropy value to optimize the strategy. The objective function is expressed as follows:

[0224]

[0225] In the formula, T and r represent the number of time steps and the single-step reward respectively; a i,t with a -i,t Respectively represent the actions selected by agent i and other collaborators; α i ∈(0,1) is the temperature coefficient, which determines the relative importance of the entropy term and the reward term. The larger the α, the more exploration is achieved to achieve more randomness in the optimization strategy. For strategy The induced trajectory of Indicates state s t The state entropy under

[0226] Defining the soft value function It is used to measure the strategic value of agent i, and its expression is:

[0227]

[0228] In the formula, ψ i is the network parameter of the value function of agent i; Q θ (s t ,a i,t ,a -i,t ) is the evaluation function of the strategy; θ is the network parameter of the Q-value function;

[0229] In order to reduce the problem of overestimation of Q value caused by maximization of Bellman equation, two Q value functions with the same structure are introduced in MASAC and The network loss function of the Q value function is as follows:

[0230]

[0231]

[0232] Among them, D is the experience buffer used to store previous experience data; is the target Q value function network parameter, and the gradient calculation of optimizing the Q value function is as follows:

[0233]

[0234] To achieve strategy improvement, MASAC continuously obtains new Q-value functions and guides the strategy to update in a way of maximum return to obtain the optimal strategy. The strategy network is trained by minimizing the Kullback-Leibler divergence. The strategy update expression is as follows:

[0235]

[0236] In the formula, Z θ (s t ) is the normalized distribution of Q value. In order to reduce the variance of action network estimation during training and facilitate gradient calculation, the reparameterization technique is introduced in the calculation process. i,t Parameterized as

[0237] The strategy update target of agent i is as follows:

[0238]

[0239] In the formula, Parameterize strategies for other collaborators; i,t is a noise vector sampled from a Gaussian distribution,

[0240] The SAC algorithm uses the temperature coefficient α i Adjustments are made to ensure that the agent fully explores the environment and selects the best strategy, using a hyperparameter equivalent constant vector The objective function representing the target policy entropy is shown in formula (39):

[0241]

[0242] In order to verify the effectiveness of the proposed resilience recovery strategy, the proposed method was applied to the IEEE 33-node power distribution system for comparative experiments. Multiple regression algorithms were used to compare and demonstrate the advantages of the proxy model, and multiple optimization algorithms were used to compare the performance of the proposed method. Finally, the resilience recovery optimization effect of the data-driven control strategy proposed in the present invention was verified in the test system.

[0243] The case study evaluates the routing and scheduling process of three MPSs located in the test system, such as Figure 4 As shown. In this system, two photovoltaic units are connected to nodes 16 and 21 respectively, three wind turbines are connected to nodes 12, 24, and 33 respectively, and three MPS accessible nodes are located at nodes 3, 9, and 30 respectively. Lines 25-29, 21-8, 9-15, 22-12, and 33-18 in the system correspond to interconnection lines that can be closed for extreme events. They are equipped with remote controllable switches to achieve dynamic network reconstruction. Assuming that the disaster occurs at 3:00, the distribution system is disconnected from the upper power grid, and lines 2-19, 6-26, 8-9, 29-30, 14-15, 17-18, and 21-2 fail. The estimated downtime is 12 hours.

[0244] The regression effects of multi-level load shedding training under different environments are shown in Table 2. All four algorithms have high accuracy, among which the proposed R-square and MAE are the best in the prediction of three types of load shedding. The parameters in the SVM model are determined by manual experience, so the deviation from the true value is the largest during the prediction process. The effect of fitting with deep neural network is better than LR and SVM. However, it is difficult to train the neural network to the optimal value with more parameters to be optimized in a limited sample. In scenario 2, due to the small photovoltaic output, the low-level loads in the isolated island formed by it will have more load shedding, and the load shedding data samples based on the optimal power flow calculation will have more complex nonlinear mapping relationships. Therefore, the prediction effects of the three comparison algorithms for three-level load shedding all decline. In the proposed method, GA is used to optimize the regularization parameters and kernel parameters. The model can quickly extract the nonlinear mapping relationship between power injection and load shedding in a limited sample. The prediction accuracy of the three-level load shedding is 2.81%, 2.46%, and 2.17% higher than that of the LR, SVM, and DNN methods, respectively.

[0245] Table 2 Comparison of regression effects

[0246] method LR SVM DNN GA-LSSVM R square (total load reduction) 0.9964 0.9913 0.9995 0.9997 MAE (total load reduction) 65.832 101.9 21.148 17.706 R square (secondary load reduction) 0.9929 0.9905 0.9979 0.9988 MAE (secondary load shedding) 67.156 75.884 32.53 24.7412 R square (three-level load reduction) 0.9867 0.989 0.9942 0.9976 MAE (Level 3 load shedding) 24.76 25.842 14.337 10.1185

[0247] S2. To verify the superiority of the proposed method in MPS routing and scheduling, it is compared in detail with a variety of optimization algorithms based on reinforcement learning during the training process, focusing on the evolution of the cumulative reward. The heuristic comparison algorithm used is as follows:

[0248] (1) MASAC: MPS adopts a multi-agent method based on the SAC algorithm. The difference from the method proposed in this paper is that its reward value is not fed back through the grid agent.

[0249] (2) MATD3: Each MPS is modeled as an agent based on the TD3 algorithm. According to the Dec-POMDP model constructed by the present invention, the agent makes decisions based on local observation information.

[0250] (3) SAC: A centralized control method is adopted to model the distribution network operator as a SAC agent. In this setting, the action space becomes 6 dimensions, and the positions and charge and discharge amounts of the three MPSs are determined by a policy network based on global information.

[0251] (4) TD3: Compared with SAC, both adopt the Actor-Critic framework. The TD3 algorithm uses a dual Q network to estimate the action value function. Similarly, a centralized control method is used for decision making.

[0252] After the training is completed, the uncertainty is introduced to randomly generate wind, light, and load scenarios. The five methods including the proposed method are verified in the real power grid environment. The test results are shown in Table 3, which lists the load reduction at each level, the cumulative load reduction rate, the operator cost, and the training time of the model. It can be seen from Table 3 that in scenario 1, due to the centralized training method used by SAC and TD3, there is a lack of coordination between MPSs, and the recovery of more critical loads in the region cannot be achieved. Based on the local observation information of the intelligent agent, the MASAC and MATD3 methods achieved better load reduction optimization effects.

[0253] The proposed method uses a proxy model to feedback the system status during the training process, and the cumulative load recovery rate is close to other methods. The first-level load reduction is 8.14% / 15.52% higher than MATD3 / MASAC, but 20.08% / 7.58% lower than TD3 / SAC. At the same time, since the grid proxy feedback reward based on the regression model is embedded in the Markov decision, there is no need to obtain complex grid topology and line parameters for optimal power flow calculation, which significantly improves the training time and reduces the computing power required for model training.

[0254] Table 3 Algorithm test results comparison

[0255] method TD3 SAC MATD3 MASAC Proposed method First level load reduction (kW) 541.90 474.48 405.07 379.732 438.51 Secondary load reduction capacity (kW) 14030.16 14281.13 14236.574 14103.01 14316.49 Level 3 load reduction (kW) 9158.94 8982.32 8953.69 8794.530 8930.11 Cumulative load reduction rate 0.3926 0.3927 0.3925 0.3937 0.3948 Operator cost (yuan) 371.92 363.44 339.47 290.988 359.158 Training duration 19h38m 22h38m 19h10m 24h25m 6h38m Does it depend on the physical model? yes yes yes yes no

[0256] The dispatch results of S3 and 3 MPS are as follows Figure 5 As shown in the figure, the bars represent the charging / discharging active power, and the purple line on the right Y-axis represents the routing process of MPS. The total capacity of MPS is 800kWh, and the present invention sets its initial capacity to 600kWh. It can be seen from the figure that MPS is connected to nodes 9 and 30 at the initial moment. Since the wind turbine power generation resources are abundant in scenario 1, MPS takes charging operations at the initial moment to prepare for subsequent resilience recovery.

[0257] At T=4:00, since the primary loads at nodes 9, 11, 13, and 14 will be unloaded, MPS2 will be moved to node 9 to provide support for the critical loads in this area together with MPS1. At this time, all primary loads can be guaranteed, so MPS3 goes to node 3 to supply the secondary load at node 21.

[0258] At T=5:00, since the wind turbine power generation in the area where node 3 is located is large, all MPS will go to node 3 for charging. Following this routing mode, MPS will go to areas with abundant power generation resources to charge when the SOC level is low, while giving priority to the supply of primary loads and discharging in areas with more critical load reductions.

[0259] At T=10:00, the wind turbine output fluctuates greatly, the primary load at nodes 31 and 33 is greatly reduced, and MPS1 goes to node 30 to discharge.

[0260] At T=11:00, MPS1 goes to node 9 with abundant renewable energy to charge. At this time, to ensure the continuity of load supply of nodes 31 and 33, MPS3 moves to node 30 to discharge. Similarly, MPS3 goes to node 9 to charge at T=12:00, while MPS1 and MPS2 go to node 30 to discharge. The operation of MPS alternating discharge and charge in the same area in different time periods demonstrates the advantages of coordinated control of multiple MPSs.

[0261] In addition to MPS coordination, dynamic reconfiguration of switches based on remote lines can also help enhance system resilience. Figure 6As shown in the figure, after the line damage occurs in the system, the connecting lines 25-29, 33-18, 9-15, and 22-12 are closed to restore the primary and secondary loads on nodes 28, 18, 15, and 22. At this moment, the photovoltaic power generation at node 21 is 0. Due to the limited output of the wind turbine at node 24, the connecting line 8-21 will remain open, and the load at node 21 will be reduced. In addition, as the photovoltaic output increases, the connecting line 8-21 is closed at T=10:00 to ensure the supply of the loads at nodes 3 and 5, and the line switches 11-12 are opened, using photovoltaics and wind turbines to form two different island areas, respectively, to achieve the optimal matching of source and load. Through the intelligent control of the lines and connecting lines, the coordinated optimization of dynamic network reconstruction and MPS can be achieved.

[0262] Therefore, the present invention adopts the above-mentioned data-driven multi-mobile emergency power resilience optimization scheduling method to achieve the optimal matching of island sources and loads through the scheduling of wind turbines, photovoltaics, and mobile power systems and the reconstruction decision of the distribution network; the nonlinear mapping relationship between the power inflow and the load reduction amount of the post-disaster distribution network is obtained through the GA-LSSVM regression algorithm, and a multi-level load reduction agent model that can replace the physical model of the power grid is constructed; while ensuring the privacy of the distribution network, the real-time collaborative optimization of MPS and the power grid topology transformation decision are realized, effectively improving the resilience of the power grid.

[0263] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A data-driven multi-mobile emergency power resilience optimization scheduling method, characterized in that: The following steps are involved: S1. Mathematical modeling of optimization problems; S2, construct a multi-level load shedding agent model; S3. Construct an optimization decision model based on multi-agent deep reinforcement learning; In step S1, the specific steps of mathematical modeling for the optimization problem are as follows: S11. The goal of distribution network resilience recovery is to optimize the load reduction of key system loads. Based on this, the mathematical model of the objective function is: In the formula, and They represent the load shedding cost at node i and the load shedding amount at time t respectively; and is the reduction of wind turbine and photovoltaic active power at time t; α is a weight coefficient used to adjust the weighted load reduction target and the proportion of the reduction target in the total target; B represents the node set in the entire distribution system, B pv , B wt It is a collection of photovoltaic and wind turbine equipment; S12. Establishing constraint conditions, including component operation model, distribution network radiation topology constraint and distribution network dynamic load recovery constraint; In step S12, the operation model of the component is as follows: In the formula, and Respectively represent the actual output of wind turbines and photovoltaics; η pv With η wt Respectively represent the maximum reduction range; and Respectively and simultaneously represent the active and reactive conversion limits of wind turbines and photovoltaic inverters, and constraints (2) to (4) restrict the active and reactive output of renewable energy; The distribution network radiation topology constraints are as follows: The single commodity flow method is introduced to construct a virtual network with the same topology and connectivity as the original power network. The virtual power flow must satisfy the power flow balance constraint: In the formula, the 0-1 binary variable c b,p,t represents the connection status of line (b, p) at time t; L is the line set; B source is the source node set; f p,b,t is the virtual power flow of line (b, p); represents the virtual power provided by the node with energy supply capability in the distribution network; δ(b) and π(b) represent the parent node and child node set at node b, respectively; x b,t is the unit virtual load at node b at time t; β b,p,t is a binary state variable representing the fault state of the line (b, p) at time t; L init represents the set of non-damaged initial lines without tie switches; The dynamic load recovery constraints of the distribution network are as follows: Based on the DistFlow power flow equation, the power balance including wind turbines, photovoltaics, MPS and controllable loads is shown in equations (12) to (15): In the formula, and are the charging and discharging capacities of the MPS at the site respectively; in constraints (14) and (15), the on-off state variable c of the line (b, p) in the virtual power flow calculation is b,p,t , limiting the power flow balance of the line including the tie switch; In step S2, constructing a multi-level load shedding agent model specifically includes the following steps: S21. Component operation model According to the optimization model in step S11, the optimal power flow calculation is performed to obtain the reduction data of each level of load and the total load of the distribution network, and then the multi-level load reduction agent model estimates the current power grid load reduction situation based on the real-time MPS, renewable energy and load data after supervised learning training using the obtained real data; In supervised learning, the minimum-maximum normalization method is used to map each input and output variable to the interval [0,1], and remove the unsolvable data rows in the obtained results. The normalization and denormalization formulas of the data are as follows: AND real =And predict ·(AND max -AND min )+Y min (17) Where, X norm is the value after data normalization; X i represents the feature sequence that needs to be normalized; X max With X min Respectively represent the maximum and minimum values ​​in the feature sequence; Y predict is the normalized predicted value output by the model; Y real is the true value with the dimension of the original data after denormalization; S22. Least Squares Support Vector Principle The least squares support vector machine LSSVM uses the least squares method to solve and transform the inequality constraints of the above optimization model into equality constraints; The purpose of the LSSVM regression problem is to find the mapping φ(x) of the sample from the original space to the high-dimensional feature space, that is, the prediction model is f(x) = w·φ(x) + b. Therefore, the slack variable ξ is introduced in LSSVM to measure the outlier error. The LSSVM parameter solution optimization problem is described as: s.t.y i [w·φ(x i )+b]=1-ξ In formula (18), γ is the regularization parameter that measures the error tolerance; The Lagrange multiplier is introduced to transform the optimization problem into a single parameter: Under the KKT condition, the partial derivative of the parameters is obtained, and the obtained α and b are brought back to the original problem, and the LSSVM regression function is obtained as follows: In the formula, K(x,x i ) is the kernel function, as shown in formula (21): S23, constructing a proxy model and prediction process based on GA-LSSVM; Genetic algorithm GA is used to optimize the hyperparameters, and the solution of the optimization problem is converted into screening chromosomes that adapt to the environment from individuals. Through replication, crossover and mutation genetic operations, the population that adapts to the environment is continuously generated according to the principle of survival of the fittest. The fitness function is used to evaluate the quality of individuals in the population. The fitness function expression is: stγ∈[γ min ,c max ],σ∈[σ min ,s max ] In the formula, y i and are the predicted value and true value of the sample respectively; γ min With γ max are the lower and upper limits of the regularization parameter respectively; σ min With σ max The lower and upper limits of kernel function parameters; The prediction process based on the GA-LSSVM agent model is as follows: First, according to the constructed proxy model dataset, the training sample and test sample sets are selected, the search range of the hyperparameters γ and σ is set, and the hyperparameters are encoded. Secondly, the LSSVM is trained based on the initialized hyperparameter values, and the fitness function constructed by formula (22) is used to determine whether the optimization goal is met; Finally, when the optimized parameters make the optimization target reach the predetermined stop condition, the optimal hyperparameters γ and σ are brought into the LSSVM model for training, and the model is analyzed using the test sample set; In step S3, an optimization decision model based on multi-agent deep reinforcement learning is constructed, which specifically includes the following steps: S31. Establish a Dec-POMDP framework based on resilience improvement, including observation space, action space, state transition and reward function; S32, using MASAC algorithm to solve Dec-POMDP; In step S31, the observation space is: At each time step t, the local observation value of the i-th MPS is o i.t , which is represented as a vector: This includes the on / off status of the distribution network lines c bp,t , the power generation of wind turbines and photovoltaics at time t The load demand of the distribution network at time t is the location where the MPS is connected in the distribution network, and the battery capacity state of the i-th MPS at time t Introducing Observations To reflect the impact of the actions taken by the i-th MPS on the resilience recovery of the power grid: In formula (23), the denominator represents the total net recovery of the load by the MPS, and the numerator represents the discharge amount of the i-th MPS. The larger the value, the greater the contribution of the MPS to resilience recovery; The action space is: The action set of each MPS in a single iteration is a i,t , which is represented as a two-dimensional vector: in, is the charge and discharge amount of the i-th MPS at time t, The site location selected for it is actually scheduled For discrete variables, by rounding up and normalizing, it will be transformed from the position set Select from; The state transition is: In the scheduling period t∈[1,T], the state transition function between each time step t and t+1 is s t+1 =T(o 1:I,t ,a 1:I,t ,w t ), where the random quantity affected by environmental factors at time t is The MPS state is affected by its own actions and its position state By Action Considering the traffic flow, network topology, capacity status By Action At the maximum charge and discharge power P i mps,max The mathematical expression of the MPS operation safety model is as follows: Equations (24) and (25) represent the charge and discharge action limits of MPS; Equation (26) represents the capacity The state transition, and are the charging and discharging efficiencies respectively; each MPS agent in the MADRL algorithm selects its own position With power The original optimization problem is equivalent to the optimal power flow problem of the distribution network after fixing the MPS position and power, thus obtaining the load reduction at each level; The reward function is: The agent transmits the actions of MPS to the agent model, and each agent i receives a reward r i.t , the negative value of the load reduction cost in the optimization objective is used as the team reward value; Taking the critical load reduction as the criterion, we further construct a step reward for each MPS. Finally, we sum the individual reward and the team reward to generate a new individual reward function, so that the original sparse reward is transformed into a dense reward to provide effective feedback for each action of the agent. The reward function of the i-th MPS agent in a single time step is shown in formula (27), which consists of the following three parts: weighted load reduction cost r pcurl , Charging penalty items and discharge penalty Formula (28) is the weighted load reduction; Formula (29) represents the primary load reduction ratio, which is obtained by indirect fitting of the proxy model; Equations (30) and (31) represent the MPS charging and discharging penalties respectively; In step S32, the MASAC algorithm is used to solve Dec-POMDP, and the specific process is as follows: The MASAC algorithm extends the maximum entropy deep reinforcement learning algorithm SAC by adopting the CTDE framework. The SAC algorithm is based on the Actor-Critic framework and learns by simultaneously training the policy-based Actor network and the value-based Critic network. The Actor network is used to generate actions, and the Critic network evaluates the value of actions by approximating the behavior value function. In the centralized training phase, the MPS agent i obtains the current state s through the proxy model. i,t 、Action a i,t , the states of other agents s -i,t 、Action a -i,t The information is input into the value network for parameter update, and the policy network updates the state s of the agent. i,t Perform parameter updates; the strategy and value networks in the MASAC algorithm are fitted through a deep neural network, and the minimum value output from the two Q-value networks is selected to avoid overestimation of the state value; In the execution phase, the agents select actions based on their respective strategy networks and pass them to the proxy model, which then gives feedback r on the reward of agent i at time t. i,t ; In the routing and power scheduling of N MPS agents, the strategy set is expressed as The action network is represented as in is the action strategy function of agent i, φ i Represents the parameters to be optimized of the policy network. SAC introduces the entropy function in the reward value function. The policy needs to maximize the entropy value to optimize the policy. The objective function is expressed as follows: In the formula, T and r represent the number of time steps and the single-step reward respectively; a i,t with a -i,t Respectively represent the actions selected by agent i and other collaborators; α i ∈(0,1) is the temperature coefficient, which determines the relative importance of the entropy term and the reward term. The larger the α, the more exploration is achieved to achieve more randomness in the optimization strategy. For strategy The induced trajectory of Indicates state s t The state entropy under In step S32, a soft value function is defined It is used to measure the strategic value of agent i, and its expression is: In the formula, ψ i is the network parameter of the value function of agent i; Q θ (s t ,a i,t ,a -i,t ) is the evaluation function of the strategy; θ is the network parameter of the Q-value function; MASAC introduces two Q-value functions Q with the same structure θi,1 With Q θi,2 , the network loss function of the Q value function is as follows: Among them, D is the experience buffer used to store previous experience data; is the target Q value function network parameter, and the gradient calculation of optimizing the Q value function is as follows: MASAC continuously obtains new Q-value functions and guides the strategy to update in a way of maximum return to obtain the optimal strategy. The strategy network is trained by minimizing the Kullback-Leibler divergence. The strategy update expression is as follows: In the formula, Z θ (s t ) is the normalized distribution of Q. In the calculation process, the reparameterization technique is introduced to convert a i,t Parameterized as The strategy update target of agent i is as follows: In the formula, Parameterize strategies for other collaborators; i,t is a noise vector sampled from a Gaussian distribution; SAC algorithm for temperature coefficient α i Adjustment is performed using a hyperparameter equivalent constant vector The objective function representing the entropy of the target policy is shown in formula (39):

Images