Comprehensive energy multi-agent collaborative optimization scheduling method and system under extreme event

Abstract

The invention provides an integrated energy multi-agent collaborative optimization scheduling method and system under an extreme event, and belongs to the technical field of integrated energy system scheduling. The method comprises the following steps: extracting spatial-temporal characteristics of multiple agents of the comprehensive energy system, performing knowledge distillation lightweight, setting the spatial-temporal characteristics to be in a global system state, constructing an agent lightweight general model in combination with Nash equilibrium, and solving the model; obtaining a global system state and an optimal action strategy of each agent in the global system state; combining the optimal action strategy with a built power failure probability model of each extreme event, a built extreme event driven electricity-gas-heat-storage multi-energy disturbance propagation model solving model and a generated adjustment demand matrix; adopting a deep reinforcement learning method to construct a multi-agent collaborative strategy model; and inputting the real-time monitoring data of the comprehensive energy multi-agent into the multi-agent collaborative strategy model to obtain a comprehensive energy multi-agent collaborative optimization scheduling strategy under the extreme event. According to the invention, the regulation and control precision and safety of the integrated energy system under extreme events are improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of comprehensive energy system scheduling, and more particularly to a multi-agent collaborative optimization scheduling model of a comprehensive energy system under the influence of extreme events. BACKGROUND

[0002] Extreme events, including natural disasters such as typhoons and thunderstorms, and man-made disasters such as terrorist attacks and large-scale power outages, bring double severe challenges to the comprehensive energy system: first, the risk of energy supply interruption, such events often damage energy infrastructure, causing regional power outages, affecting residents' lives and industrial production; second, the problem of system instability, extreme events disrupt the balance of system operation, causing imbalances between power generation and power consumption in the power system, resulting in large fluctuations in frequency and voltage, and in severe cases, system collapse; the heat and gas systems also experience abnormal heating and gas supply due to unstable pipe network pressure and heat source gas supply, and even cause safety accidents, endangering life and property.

[0003] As a complex energy supply system with multi-energy flow coupling, the scheduling optimization technology of the comprehensive energy system has become a key support for energy security. The current mainstream optimization scheduling model adopts a centralized decision-making architecture, which can achieve energy supply and demand balance under normal working conditions, but has significant limitations under extreme events: first, a single decision center is vulnerable to chain-like failures caused by disasters, and the system topology reconstruction capability is insufficient to cope with sudden conditions such as equipment damage and communication interruption; second, existing models focus more on steady-state analysis of the coupling mechanism of electricity-gas-heat multi-energy flow, and lack detailed modeling of dynamic response under extreme disturbances, resulting in weak multi-time scale coordination capabilities; third, traditional optimization objectives focus on economic indicators, and do not adequately consider safety elements such as energy supply reliability and fault recovery speed in extreme scenarios.

[0004] In summary, given the key position of the comprehensive energy system in modern energy supply and the severe challenges posed by extreme events, and considering the many shortcomings of existing technologies in dealing with extreme events, there is an urgent need for an innovative multi-agent collaborative optimization scheduling method for the comprehensive energy system to improve the collaborative optimization capabilities of the comprehensive energy system under extreme events and ensure the safety, reliability, efficiency, and sustainability of energy supply.

[0005] Prior art document 1 (CN119539424A) discloses a resource regulation method and device for park-level adjustable load, equipment and medium. Its shortcomings are that it cannot cope with extreme event disturbances, relies on a centralized communication architecture, and lacks a dynamic safety risk quantification mechanism. SUMMARY

[0006] To solve the problems in the prior art, the application provides a space-time feature extraction method based on mutual information and fast Fourier transform, key variables are screened through mutual information weight, and adaptive dimension reduction processing of input data is realized by fusing frequency domain features; a power outage probability model of multi-factor coupling is established for four types of extreme events, i.e., typhoon, extreme cold, high temperature and thunderstorm, to quantize the nonlinear correlation between meteorological parameters and energy system vulnerability; a four-dimensional dynamic energy flow coupling differential equation system of electricity-gas-heat-storage is proposed to accurately depict the cross-medium propagation mechanism and space-time evolution law of multi-energy flow under extreme disturbance; and a risk perception action selection mechanism based on MADDPG (Multi Agent Deep Deterministic Policy Gradient, multi-agent deep reinforcement learning framework) is constructed to embed the dynamic adjustment demand matrix as a constraint condition into the agent decision-making process, and distributed collaborative optimization of safety constraints and economic targets is realized through Nash equilibrium solution.

[0007] The application adopts the technical scheme as follows.

[0008] The first aspect of the application provides a comprehensive energy multi-agent collaborative optimization scheduling method under extreme events, including the following steps: The space-time features of the comprehensive energy system multi-agent are extracted through mutual information and fast Fourier transform, and a lightweight general model of the agent is constructed after lightening through knowledge distillation and solving, to obtain the global system state and the optimal action strategy of each agent under the global system state; A power outage probability model of each extreme event is constructed, and the extreme events include typhoon event, extreme cold event, high temperature event and thunderstorm event; An extreme event driven electricity-gas-heat-storage multi-energy disturbance propagation model is constructed according to the basic power loss of the power system, the basic power loss of the gas system, the basic power loss of the heat system and the basic power loss of the energy storage system caused by the extreme event, the model is solved, and an adjustment demand matrix is generated; The global system state and the optimal action strategy of each agent under the global system state are combined with the power outage probability model of each extreme event and the adjustment demand matrix, and a multi-agent collaborative strategy model is constructed by using a multi-agent deep reinforcement learning method; Real-time monitoring data of the comprehensive energy multi-agent are input into the multi-agent collaborative strategy model, to obtain a comprehensive energy multi-agent collaborative optimization scheduling strategy under extreme events.

[0009] Preferably, the construction of the lightweight general model of the agent and the solving include: The energy physical devices of the comprehensive energy system are divided into multi-type agents according to the functional attributes, to obtain a comprehensive energy system multi-type agent cluster; Obtaining data of a plurality of types of agent clusters of a comprehensive energy system, performing time-space feature extraction through mutual information and fast Fourier transform to obtain time-space features; Migrating knowledge in a teacher model to a student model through knowledge distillation to obtain lightweight student model data; Setting the lightweight student model data as a global system state, combining a multi-agent Nash equilibrium to construct an agent lightweight general model, and solving the agent lightweight general model to obtain an optimal action strategy of each agent under the global system state.

[0010] Preferably, the solving of the agent lightweight general model to obtain the optimal action strategy of each agent under the global system state comprises: Constructing a reward function of each agent according to the action of each agent itself and the actions of other agents; Setting the lightweight student model data as the global system state, inputting a safety constraint function, setting a safety constraint relaxation term as an output result of the safety constraint function if the safety constraint function is violated, or setting the safety constraint relaxation term as 0 otherwise; Subtracting a product of a balance coefficient and the safety constraint relaxation term from the reward function of each agent to obtain an optimal action strategy of a current agent; Summing the optimal action strategies of each agent to obtain the optimal action strategy of each agent under the global system state.

[0011] Preferably, the constructing of the power outage probability model of each extreme event comprises: Solving a probability of power outage directly caused by precipitation, a cumulative risk probability corresponding to a wind speed deviation degree, and a joint risk degree of precipitation and wind speed through wind speed and precipitation, and constructing a power outage probability model of a typhoon event according to a solving result; Solving a probability of power outage caused by an extremely cold temperature and a probability of power outage caused by an extremely cold event duration through temperature and duration, and constructing a power outage probability model of an extremely cold event according to a solving result; Solving a probability of power outage caused by high temperature through temperature, and constructing a power outage probability model of a high temperature event in combination with an air conditioner load rate; Solving a probability of power outage caused by lightning density and a nonlinear saturation risk of rainfall through lightning density and rainfall, and constructing a power outage probability model of a thunderstorm event according to a solving result.

[0012] Preferably, the generating of the adjustment demand matrix comprises: constructing an energy flow model of the power system according to a basic power loss of the power system caused by the extreme event, constructing an energy flow model of the gas system according to a basic power loss of the gas system caused by the extreme event, constructing an energy flow model of the heat supply system according to a basic power loss of the heat supply system caused by the extreme event, and constructing an energy flow model of the energy storage system according to a basic power loss of the energy storage system caused by the extreme event; constructing a multi-energy disturbance propagation model of electricity-gas-heat-storage according to the energy flow model of the power system, the energy flow model of the gas system, the energy flow model of the heat supply system and the energy flow model of the energy storage system; solving the multi-energy disturbance propagation model of electricity-gas-heat-storage to generate the power required to be supplemented by the power system, the power required to be supplemented by the gas system, the heat power required to be supplemented by the heat supply system and the charge-discharge demand of the energy storage system of the heat supply system, and constructing an adjustment demand matrix according to the generated results.

[0013] Preferably, the constructing the multi-agent collaborative strategy model comprises: obtaining a basic state space and a basic action space of the multi-agent according to the global system state and the optimal action strategy of each agent under the global system state, and solving an extreme event state space and an extreme event action space in combination with a power outage probability model of the extreme event; solving a single reward value according to the cost benefit, the global voltage safety state and the calculation results of the adjustment efficiency, and constructing a risk-sensitive reward function in combination with the power outage probability model of each extreme event and the adjustment demand matrix; constructing a multi-agent collaborative optimization model by using a multi-agent deep reinforcement learning algorithm according to the state space of the extreme event, the action space of the extreme event and the risk-sensitive reward function.

[0014] Preferably, the extreme event action space comprises: determining the load loss risk quantitative score of the action by differentiating the action through the power outage probability model, multiplying the change rate of the power outage probability after the action is executed by a corresponding weight coefficient and summing them up to obtain the load loss risk quantitative score of the action; setting the load loss risk quantitative score of the action corresponding to the action located in the basic action space to be less than a set threshold to obtain the extreme event action space.

[0015] Preferably, the constructing the risk-sensitive reward function comprises: multiplying the risk-sensitive coefficient by the power outage probability model to obtain a risk penalty term, taking a negative value of the risk penalty term and taking an exponential function as a base to obtain an exponential decay function, and performing a minus 1 operation to obtain a linear penalty value of the power outage; The absolute error of the actual power adjustment amount generated by the computing agent action and the adjustment demand matrix is divided by the sum of the adjustment demand matrix and the prevention of zero parameter to obtain a relative error rate, and the relative error rate is subtracted from 1 to obtain a generated adjustment accuracy score; The outage penalty value and the adjustment accuracy score are multiplied by the corresponding weight respectively, and the multiplication result is summed, and then added to a single reward value to obtain a risk-sensitive reward function.

[0016] Preferably, the multi-agent collaborative optimization model is constructed, comprising: An Actor network of each agent is constructed according to an extreme event state space for the multi-agent, and a Critic network of each agent is constructed according to an extreme event action space and the extreme event state space; The action output from the Actor network of each agent is summed with exploration noise to obtain an agent action satisfying the extreme event action space; A joint action of the agents is constructed according to the agent action, the joint action of the agents is executed, the risk-sensitive reward of the multi-agent and a new multi-agent state are obtained, and a transition tuple is constructed according to the extreme event state space and the joint action of the agents, and is stored in an experience pool; The transition tuple is obtained from the experience pool, and the multi-agent collaborative strategy model is trained to obtain a preliminary multi-agent collaborative strategy model; The action strategy of the multi-agent output by the preliminary multi-agent collaborative strategy model is subjected to Nash equilibrium to obtain a multi-agent collaborative strategy, and a multi-agent collaborative strategy model is constructed.

[0017] The second aspect of the application provides a comprehensive energy multi-agent collaborative optimization scheduling system under extreme events, which runs the method for comprehensive energy multi-agent collaborative optimization scheduling under extreme events. A multi-agent state extraction module is configured to extract the spatiotemporal characteristics of the multi-agent of the comprehensive energy system through mutual information and fast Fourier transform, construct a lightweight general model of the agent through knowledge distillation, and obtain the global system state and the optimal action strategy of each agent under the global system state by solving the lightweight general model of the agent. A power outage probability solving module is configured to construct a power outage probability model of each extreme event, wherein the extreme event includes a typhoon event, an extremely cold event, a high temperature event, and a thunderstorm event. A demand solving module is configured to construct an extreme event driven electric-gas-heat-storage multi-energy disturbance propagation model according to the basic power loss of the power system, the basic power loss of the gas system, the basic power loss of the heat system, and the basic power loss of the energy storage system caused by the extreme event, solve the model, and generate an adjustment demand matrix. The cooperative strategy model construction module is used for combining the global system state, the optimal action strategy of each intelligent agent under the global system state, the power outage probability model of each extreme event and the adjustment demand matrix, and constructing a multi-intelligent agent cooperative strategy model by using a multi-intelligent agent deep reinforcement learning method; The output module is used for inputting the real-time monitoring data of the comprehensive energy multi-intelligent agent into the multi-intelligent agent cooperative strategy model to obtain a comprehensive energy multi-intelligent agent cooperative optimization scheduling strategy under an extreme event.

[0018] Compared with the prior art, the beneficial effects of the present application at least include: The present application uses a knowledge distillation technology, supports edge deployment, and can still independently operate when an extreme event causes a communication interruption, thereby improving the safety of the comprehensive energy system scheduling, using a Nash equilibrium method of a distributed decision architecture to reward and punish each intelligent agent, forcing the intelligent agent to consider the actions of other intelligent agents when pursuing its own benefits, reducing the multi-intelligent agent strategy conflict, and improving the efficiency of the comprehensive energy system multi-intelligent agent cooperative optimization; The present application enhances the basic state space of the multi-intelligent agent by constructing the power outage probability model of the extreme event, improves the risk perception ability of the intelligent agent under the extreme event, restricts the multi-intelligent agent basic action space through the power outage probability model of the extreme event, limits the intelligent agent action selection of the comprehensive energy system to the extreme event, can constrain the high-risk action under the extreme event, and improves the risk adaptive ability and safety of the comprehensive energy system under the extreme event; The present application generates the scheduling demand by constructing the electric-gas-heat-storage multi-energy disturbance propagation model, generates the risk sensitive reward function of the extreme event by combining the scheduling demand with the power outage probability model of the extreme event, reduces the load loss of the comprehensive energy system under the extreme event, and improves the economy of the comprehensive energy system regulation and control; The present application balances the multi-intelligent agent strategy by Nash equilibrium, reduces the conflict rate of the comprehensive energy system under the multi-intelligent agent strategy regulation and control, and improves the safety of the comprehensive energy system under the multi-intelligent agent strategy regulation and control. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is a schematic diagram of the comprehensive energy multi-intelligent agent cooperative optimization scheduling under an extreme event provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. The described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the spirit of the present application, all other embodiments obtained by those skilled in the art without making creative efforts fall within the protection scope of the present application.

[0021] As Figure 1 shown, embodiment 1 of the present application provides a method for collaborative optimization and scheduling of multi-agent of integrated energy system under extreme events, comprising the following steps: Step 1, divide the multi-agent according to the functional attributes of the equipment, extract the spatio-temporal characteristics of the multi-agent of the integrated energy system through mutual information and fast Fourier transform, and construct a lightweight general model of the agent through knowledge distillation after lightweight, and solve to obtain the global system state and the optimal action strategy of each agent under the global system state.

[0022] In the preferred but non-limiting embodiments of the present application, step 1 comprises: Step 1.1, the energy physical equipment of the integrated energy system such as generator set, electric hydrogen production device, gas storage tank and industrial flexible load is divided into four types of agents of energy production, energy conversion, energy storage and adjustable load according to the functional attributes of the energy physical equipment, and a multi-type agent cluster of the integrated energy system is obtained.

[0023] Further preferably, step 1.1 comprises: The energy production agent (PA) includes all energy production units, realizes the conversion of primary energy to secondary energy such as electricity / heat / cold, and mainly includes photovoltaic array, gas turbine generator set, industrial waste heat boiler, solar heat collection tower, combined cooling heating and power system, etc.; The energy conversion agent (CA) includes all energy form conversion units, realizes multi-energy coupling and energy flow direction regulation, mainly involving electric boiler for electric-heat conversion, absorption chiller for electric-cold conversion, gas boiler for gas-heat conversion, and heat pump system for multi-directional coupling conversion, etc.; The energy storage agent (SA) includes all energy storage units, realizes energy time shift and power buffer across time scales, including lithium ion battery pack, heat storage tank, gas storage tank, etc.; The adjustable load agent (ALA) covers all energy terminal use units, has demand response capability and energy use mode optimization function, including industrial load, commercial load, residential load, transportation load and special load.

[0024] Step 1.2, obtain the real-time monitoring data, prediction and scheduling data, system coupling state and environmental and event signals of the multi-type agent cluster of the integrated energy system in step 1.1, extract the spatio-temporal characteristics through mutual information and fast Fourier transform, and obtain the spatio-temporal characteristics, which can be expressed as follows: (1) In the formula, H t represents t spatiotemporal features at time k, Concat represents concatenation, k represents a sliding index within a time window, which ranges from the time window start time k=t-T to the time window end time k=t, t represents the end time, x k represents input data at time k, including real-time monitoring data such as but not limited to power and temperature, prediction data such as but not limited to weather prediction, scheduling data such as but not limited to scheduling instructions, system coupling state such as but not limited to SOC and health state; T represents the adaptive sliding window length, dynamically adjusts the time range, and sets the time range to 5-30 minutes; represents the feature sequence of the input data within the time window from to t; FFT(.) represents the Fourier transform, MutualInfo(x, y) represents the mutual information of feature x and target variable y, which is used to calculate the feature importance weight, feature x represents the input item in Table 1, and target variable y represents the output item in Table 1, including optimized control instructions, state information participating in cooperation, performance indicators, etc., wherein the performance indicators include economic performance, cooperation performance, and safety performance, the economic performance includes but is not limited to local scheduling plan and market bidding capacity, the cooperation performance includes but is not limited to cooperation boundary constraint and cross-agent coordination parameter, and the safety performance includes but is not limited to emergency regulation strategy and safety overrun alarm.

[0025] Table 1 Input and output data of the lightweight general model of the intelligent agent

[0026] Based on the mutual information, the key variable selection retains the features strongly related to the target control, such as but not limited to the irradiance-power nonlinear relationship feature of the photovoltaic intelligent agent, the fast Fourier transform processing is performed on x within the time window, and the frequency domain features such as but not limited to periodicity and fluctuation intensity are extracted.

[0027] Notably, the present application can better capture the timing rules of energy load by screening the time domain correlation features strongly related to the multi-agent control target through mutual information and extracting the frequency domain features of periodicity in multi-agent data through fast Fourier, and is more comprehensive and efficient than simply using time domain or frequency domain methods.

[0028] Step 1.3, the spatiotemporal features obtained in step 1.2 are subjected to knowledge distillation to transfer the knowledge in the teacher model to the student model for lightweight processing, and lightweight student model data is obtained.

[0029] Further preferably, step 1.3 comprises: The LSTM (Long Short Term Memory) compression model based on teacher-student distillation is adopted, and the knowledge in the complex teacher model constructed by BiGRU (Bidirectional Gated Recurrent Unit) is transferred to the lightweight student model constructed by PrunedLSTM (Pruned Long Short Term Memory) through knowledge distillation, such as but not limited to output distribution and hidden state, so that the student model can maintain high performance while reducing the number of parameters. The BiGRU is used to extract the timing characteristics of the full-size teacher model, which has large parameter size and high performance. The lightweight student model reduces the number of LSTM parameters through pruning, reduces the calculation amount and storage requirement, and realizes lightweight, which can be expressed as follows:

[0030] In the formula, is the hidden state of the teacher model at time t, t is the hidden state of the teacher model at time t-1, t H t is the spatio-temporal feature at time t, t θ tea is the weight of the teacher model, is the hidden state of the student model at time t, t is the hidden state of the student model at time t-1, t θ stu is the weight of the student model; L distill is the distillation loss function, which is used to ensure that the lightweight model can still approximate the performance of the original model; λ is the distillation weight, which is used to balance the contribution of knowledge distillation and task loss; KL (·||·) represents the KL divergence, which is used to measure the difference between the output distribution of the teacher and the student; σ () is the Softmax function, which converts the hidden state into a probability distribution; τ is the temperature coefficient, which is used to soften the probability distribution and enhance the knowledge transfer effect; L task is the loss of the student model on the target task.

[0031] Step 1.4, set the lightweight student model data as the global system state, combine the multi-agent Nash equilibrium to construct an agent lightweight general model, and solve the agent lightweight general model to obtain the optimal action strategy of each agent under the global system state. ​​​​​

[0032] More preferably, step 1.4 includes: Nash equilibrium is suitable for non-cooperative game scenarios, aiming to maximize the interests of each agent. It more closely reflects the decentralized decision-making needs of real-world multi-agent systems. Nash equilibrium enables distributed collaborative optimization of the economics and security of multiple agents, solving for the optimal action strategies of each agent in the global system state, including: Construct a reward function for each agent based on its own actions and the actions of other agents; Set the lightweight student model data as the global system state, input the security constraint function, and if the security constraint function is violated, set the security constraint relaxation term to the output of the security constraint function; otherwise, set the security constraint relaxation term to 0. The optimal action strategy of the current agent is obtained by subtracting the product of the balance coefficient and the safety constraint relaxation term from the reward function of each agent. Summing the optimal action policies of each agent yields the optimal action policies of each agent in the global system state. The lightweight general model of agents is expressed by the following formula: (5) In the formula, N The number of intelligent agents; in this invention, N is 4. For the first i The reward function of each agent depends on its own actions. Actions with other intelligent agents Margin(g) j ( S )) represents a safety constraint relaxation term, such as, but not limited to, voltage over-limit penalties. When the safety constraint function g j ( S ) is violated, that is g j ( S A penalty is incurred when )>0. j ( S ))=g j ( S Otherwise, Margin(g) j ( S ))=0;g j ( S ) indicates the first j The specific form of the safety constraint function is determined by the physical safety boundary of the integrated energy system, such as, but not limited to, voltage over-limit constraints in power systems and temperature constraints in thermal systems. S The global system state, containing the basic state space of all agents, is the lightweight student model data output from step 1.3. , set as the input state of Nash equilibrium solution, contains compressed spatiotemporal features, and is used to describe the global system state, such as but not limited to power, temperature, etc. γ is a balance coefficient, used to adjust the weight of safety constraints and economic rewards, improve γ ↑ indicates stricter safety constraints, prioritizes system stability, and reduces γ ↓ indicates more concern for economy, allowing a certain degree of safety risk.

[0033] Step 2, constructing outage probability models for each extreme event, wherein the extreme events include typhoon events, extreme cold events, high temperature events, and thunderstorm events.

[0034] In a preferred but non-limiting embodiment of the present application, step 2 includes: Step 2.1, solving the probability of power outage directly caused by precipitation, the cumulative risk probability corresponding to the deviation degree of wind speed, and the joint risk degree of precipitation and wind speed through wind speed and precipitation, and constructing a power outage probability model for typhoon events according to the solving results, for quantifying the power outage risk caused by typhoon events.

[0035] Further preferably, step 2.1 includes: The risk weighting value of precipitation is obtained by multiplying the sensitivity of controlling precipitation impact and the amount of precipitation, and the risk weighting value of precipitation is taken as the index of an exponential function with the negative of the risk weighting value of precipitation as the index, to simulate the saturation effect of precipitation impact, and obtain the attenuation trend of precipitation impact outage, and the probability of power outage directly caused by precipitation is obtained by subtracting the attenuation trend of precipitation impact outage from 1, when the amount of precipitation R is small, such as light rain, the risk weighting value of precipitation is small, at this time the impact of precipitation on the probability of power outage is approximately linear, and the power outage risk increases slowly with the increase of rainfall, when the amount of precipitation is large, such as heavy rain, the risk weighting value of precipitation is large, tends to 1, the contribution of precipitation to the probability of power outage reaches saturation, describing the scenario that the power outage risk caused by excessive rainfall has approached the maximum value, and the increase of rainfall has limited effect on the risk; The actual wind speed is subtracted from the wind speed threshold and then divided by the dispersion degree of the wind speed risk distribution to obtain the standardized deviation degree of the actual wind speed relative to the wind speed threshold, and the cumulative distribution function of the standard normal distribution is input to obtain the cumulative risk probability corresponding to the deviation degree of wind speed; The joint action intensity of precipitation and wind speed is obtained by multiplying the amount of precipitation and the actual wind speed, and the joint risk degree of precipitation and wind speed is obtained by normalization through the normalization coefficient; The probability of power outage directly caused by precipitation is multiplied by the precipitation weight, the cumulative risk probability corresponding to the deviation degree of wind speed is multiplied by the wind speed weight, and the joint risk degree of precipitation and wind speed is multiplied by the joint weight of precipitation and wind speed, and the sum of the multiplied results is obtained to obtain the power outage probability of typhoon events. The power outage probability model of typhoon events is expressed by the following formula:

[0036] where P outage,1 represents the outage probability of typhoon event; R is the precipitation, unit mm / h, used to quantify the direct impact of rainfall on power facilities, such as but not limited to short circuit, flood; V is the actual wind speed, unit m / s, used to reflect the physical damage of wind force on power facilities, such as but not limited to pole falling, wire breaking; Φ(·) is the cumulative distribution function of standard normal distribution, used to describe the risk steep increase effect when wind speed exceeds the threshold V 0; α 1, β 1, γ 1are the precipitation weight, wind speed weight and precipitation and wind speed combined action weight respectively, calibrated based on historical data, used to reflect the relative importance of precipitation, wind speed and their combined action; k 1, V 0, σ , k 2are the sensitivity of controlling precipitation impact (h / mm), wind speed threshold (m / s) and the dispersion degree of wind speed risk distribution (m / s) and the normalization coefficient .

[0037] Step 2.2, solve the probability of power outage caused by extremely cold temperature and the probability of power outage caused by extremely cold event duration through temperature and duration, and construct the power outage probability model of extremely cold event according to the solving results, used to quantify the impact of extremely cold weather on power system.

[0038] Further preferably, step 2.2 comprises: obtain the temperature deviation by subtracting the actual extremely cold temperature from the extremely cold temperature threshold and dividing by the dispersion degree parameter of temperature distribution, input the cumulative distribution function of standard normal distribution to obtain the probability of power outage caused by extremely cold temperature; obtain the risk weighting value of extremely cold event duration by multiplying the decay rate of duration with the extremely cold event duration, take the negative number of the risk weighting value of extremely cold event duration as the index of the exponential function, obtain the decay trend of extremely cold duration impact on power outage by taking the negative number of the risk weighting value of extremely cold event duration as the index of the exponential function, obtain the probability of power outage caused by extremely cold event duration by subtracting the decay trend of extremely cold duration impact on power outage from 1; multiply the probability of power outage caused by extremely cold temperature with the weight coefficient of temperature action, multiply the probability of power outage caused by extremely cold event duration with the weight coefficient of duration action, sum the multiplication results to obtain the power outage probability of extremely cold event, the power outage probability model of extremely cold event is expressed as follows:

[0039] where P outage,2power outage probability of extreme cold event; α2, β2 are weight coefficients of temperature effect and weight coefficients of duration effect, respectively; Φ is the cumulative distribution function of standard normal distribution, which is used to quantify the risk when the temperature is lower than the threshold; T is the actual extreme cold temperature, unit: ℃; T c is the extreme cold temperature threshold, unit: ℃, and the actual temperature lower than the extreme cold temperature threshold is determined as the actual extreme cold temperature, T c is-5℃ in the present application; σ T is the dispersion degree parameter of temperature distribution, unit: ℃; t is the duration of extreme cold event, unit: h; k t is the decay rate of duration, unit: h -1 ; is the exponential decay function, which describes the dynamic influence of duration on risk.

[0040] Step 2.3, the probability of power outage caused by high temperature is solved by temperature, and the power outage probability model of high temperature event is constructed by combining the air conditioner load rate.

[0041] Further preferably, step 2.3 comprises: The actual high temperature temperature is subtracted from the high temperature threshold, multiplied by the temperature sensitivity coefficient, to obtain the temperature deviation degree, and the S-shaped function is input to solve the probability of power outage caused by high temperature, wherein the S-shaped function is used to describe the smooth transition of temperature overrun, avoiding temperature threshold mutation false alarm; The square of the air conditioner load rate is multiplied by the weight of the air conditioner load rate, the probability of power outage caused by high temperature is multiplied by the weight coefficient of the high temperature event, and the sum of the multiplication results is obtained to obtain the power outage probability of the high temperature event, wherein the square of the air conditioner load rate is used to describe the surge of air conditioner leading to overload, and to identify the real overload risk. The power outage probability model of high temperature event is expressed in the following formula:

[0042] In the formula, P outage,3 represents the power outage probability of high temperature event; k T is the sensitivity coefficient of temperature risk, unit: ℃ -1 ; α3, β3 are weight coefficients of high temperature event and weight coefficients of air conditioner load rate, respectively; T’ is the actual high temperature temperature, unit: ℃, which directly affects the heat dissipation of transformer and the efficiency of photovoltaic; T 0 ’ is the high temperature threshold, unit: ℃, and the actual temperature greater than the high temperature threshold is determined as the actual high temperature temperature; L AC is the air conditioner load rate, which reflects the load pressure of air conditioner use on power grid, sigmoid( . ) is the S-shaped function, sigmoid( x )=1 / (1+e -x ), for smoothing the steep increase effect of temperature on risk.

[0043] Step 2.4, solving the nonlinear saturation risk of lightning density causing outage probability and rainfall by lightning density and rainfall, and constructing the outage probability model of thunderstorm event according to the solving results.

[0044] Further preferably, step 2.4 comprises: normalizing the lightning density by multiplying the normalized coefficient of lightning density to obtain the risk weighted value of lightning density, taking the negative number of the risk weighted value of lightning density as the index of the exponential function to obtain the attenuation trend of lightning density affecting outage, and subtracting 1 from the attenuation trend of lightning density affecting outage to obtain the probability of lightning density causing outage, for describing the cumulative effect of lightning; setting the hyperbolic tangent function as the rainstorm saturation function, normalizing the rainfall by the normalized coefficient of rainfall, and inputting the hyperbolic tangent function to obtain the nonlinear saturation risk of rainfall; multiplying the nonlinear saturation risk of rainfall by the weight of rainfall acting on the thunderstorm event, multiplying the probability of lightning density causing outage by the weight of lightning density acting, and summing the multiplication results to obtain the outage probability of thunderstorm event, the outage probability model of thunderstorm event being expressed as follows:

[0045] In the formula, P outage,4 represents the outage probability of thunderstorm event; and α4 and β4 are the weight coefficients of lightning density acting and the weight coefficient of rainfall acting on the thunderstorm event, respectively. λ is the lightning density, with the unit of times / km2·h, k λ is the normalized coefficient of lightning density, with the unit of km 2 ·h / time; R is the rainfall, with the unit of mm / h, k R is the normalized coefficient of rainfall, with the unit of h / mm; is the exponential decay function, describing the cumulative effect of lightning causing line trip, when λ →∞, 1- →1, i.e. the outage probability tends to 100% when lightning is extremely frequent, and when λ =0, the lightning part risk is 0, tanh( k R R ) is the hyperbolic tangent function, simulating the nonlinear saturation effect of rainfall on outage risk.

[0046] Step 3, according to the extreme event caused by the power system base power loss, gas system base power loss, heat system base power loss and energy storage system base power loss.

[0047] In a preferred but non-limiting embodiment of the present application, step 3 comprises: Step 3.1, according to the power system energy flow model, gas system energy flow model, heat system energy flow model and energy storage system energy flow model, the electricity-gas-heat-storage multi-energy disturbance propagation model is constructed. The electricity-gas-heat-storage multi-energy disturbance propagation model includes the power system energy flow model, the gas system energy flow model, the heat system energy flow model and the energy storage system energy flow model.

[0048] Further preferably, step 3.1 comprises: Step 3.1.1, according to the extreme event caused by the power system base power loss, the power system energy flow model is constructed, including: By multiplying the voltage diffusion coefficient G e With the voltage gradient The power flux density is obtained, the divergence of the power flux density is solved and the negative value is taken, and the power change amount driven by the voltage diffusion ; By multiplying the gas-to-electricity conversion efficiency with the gas input power, the power contribution amount of the gas input converted into electric energy ; The heat-to-electricity conversion efficiency is multiplied by the heat input power to obtain the contribution amount of the heat input converted into electric energy ; The discharge efficiency of energy storage to electric energy is multiplied by the discharge power to obtain the output power contribution amount when the energy storage is discharged ; The charging efficiency of electric-to-energy storage is multiplied by the charging power to obtain the input power contribution amount when the energy storage is charged ; The power system extreme event vulnerability coefficient is multiplied by the power system base power loss caused by the extreme event to obtain the power loss caused by the power system extreme event; The power change amount driven by the voltage diffusion is summed with the power contribution amount of the gas input converted into electric energy, the contribution amount of the heat input converted into electric energy and the output power contribution amount when the energy storage is discharged, and then the input power contribution amount when the energy storage is charged and the power loss caused by the power system extreme event are subtracted to obtain the power that needs to be supplemented by the power system. The power system energy flow model is represented by the following formula:

[0049] In the formula, dQ e / dt Psupp is the power supply for the power system, used to describe the energy stored in the power system Q e is the rate of change over time; G e is the voltage diffusion coefficient, used to describe the power diffusion driven by the voltage gradient; V is the voltage, in V; is the gradient operator; is the divergence operator; P g is the gas input power, in W; η eg is the gas-to-electricity conversion efficiency, η eh is the heat-to-electricity conversion efficiency; Q h is the heat input power, in W; η sd is the discharge efficiency of the energy storage to electricity, reflecting the conversion loss from energy storage to electricity; η se is the charging efficiency of electricity to energy storage, reflecting the conversion loss from electricity to energy storage; is the charging power, in W, input from the power grid to the energy storage; is the discharging power, in W, output from the energy storage to the power grid; ξ e is the extreme event vulnerability coefficient of the power system; is the power system base power loss caused by extreme events, in W.

[0050] Step 3.1.2, constructing a gas system energy flow model according to the gas system base power loss caused by extreme events, including: the heat transfer power is obtained by multiplying the gas density, constant-pressure specific heat capacity, flow characteristic cross-sectional area, and characteristic temperature difference, and dividing by the characteristic length of gas flow; the enthalpy flow density vector is obtained by multiplying the gas density, gas unit mass enthalpy, and gas flow rate, and the convective transfer power is obtained by integrating the volume of the gas system control body and multiplying the spatial divergence of the enthalpy flow density vector obtained by the gradient operator; the electricity-to-gas efficiency is multiplied by the electricity input power to obtain the electricity-to-gas power; the gas system extreme event vulnerability coefficient is multiplied by the gas system base power loss caused by extreme events of the gas system to obtain the gas system extreme event loss; the gas system energy flow model is obtained by summing the heat transfer power and the electricity-to-gas power, subtracting the convective transfer power and the gas system extreme event loss, and is represented by the following formula: (11) In the formula, dQ g / dt The gas system needs additional power. Q g ρ represents the total energy of the gas system. g This refers to the density of the gas, expressed in kg / m³. 3 C pg This refers to the specific heat capacity of the gas at constant pressure, expressed in J / (kg·K). A The characteristic cross-sectional area of ​​the gas flow is expressed in meters (m²). 2 ; This refers to the gas temperature difference, expressed in Kelvin (K). L The characteristic length of the gas flow is given in meters (m); h is the enthalpy of the gas per unit mass (J / kg); v g V represents the gas flow velocity, measured in m / s. g The volume of the gas system control unit is in meters (m). 3 ; η ge Indicates the efficiency of electricity-to-gas conversion; P e For electrical input power; ξ g This represents the vulnerability coefficient of the gas system to extreme events. This represents the basic power loss of the gas system caused by extreme events, measured in watts (W).

[0051] Step 3.1.3: Construct a thermal system energy flow model based on the fundamental power loss of the thermal system caused by extreme events, including: Integrating the volume of the gas system control volume, and multiplying the integral result by the Laplace operator of thermal conductivity and temperature field, we obtain the heat diffusion power caused by temperature unevenness inside the gas system. Multiplying the electro-thermal conversion efficiency by the electrical input power yields the power that directly converts electrical energy into heat energy. Multiplying the energy storage heat conversion efficiency by the energy storage input power yields the power released by the energy storage system to the heating network. The difference between the system average temperature and the ambient temperature is multiplied by the thermal conductivity between the heating network and the environment to obtain the irreversible heat loss from the heating network to the environment through the equipment. The power loss caused by extreme events in a thermal system is obtained by multiplying the vulnerability coefficient of the thermal system to extreme events by the base power loss of the thermal system. The required heat power to be replenished by the gas system is obtained by summing the power of heat diffusion due to uneven temperature within the gas system, the power of direct conversion of electrical energy into heat energy, and the power released from the energy storage system to the heating network, and then subtracting the irreversible heat loss from the heating network to the environment through equipment and the power loss caused by extreme events in the thermal system. The energy flow model of the thermal system is expressed by the following formula: (12) In the formula, dQ h / dt The heat power that needs to be supplemented to the thermal system, Q h The total energy of the thermal system; α h Thermal conductivity, in units of ;T h The system average temperature; The Laplace operator for the temperature field, with units of K / m. 2 V h The volume of the gas system control unit is in meters (m). 3 ; η he P represents the electro-thermal conversion efficiency; s This refers to the energy storage input power, measured in watts (W). η hs Indicates energy storage to heat conversion efficiency; β h T0 represents the thermal conductivity between the heating network and the environment, measured in W / K; T0 represents the ambient temperature, measured in K. ξ h This represents the vulnerability coefficient of a thermal system to extreme events. This represents the basic power loss of a thermal system caused by extreme events, measured in W.

[0052] Step 3.1.4: Construct an energy flow model for the energy storage system based on the base power loss caused by extreme events, including: The charging efficiency of the electro-to-energy storage is multiplied by the charging power to obtain the input power contribution during energy storage charging. The discharge efficiency of energy storage to electrical energy is multiplied by the discharge power to obtain the output power contribution during energy storage discharge. The energy storage self-loss coefficient is multiplied by the total energy of the energy storage system to obtain the self-loss of the energy storage body; The extreme event vulnerability coefficient of an energy storage system is multiplied by the base power loss of the energy storage system caused by an extreme event to obtain the power loss of the energy storage system caused by an extreme event. The charging or discharging demand of a thermal energy storage system is obtained by subtracting the output power contribution during energy storage discharge, the self-loss of the energy storage unit, and the power loss caused by extreme events in the energy storage system from the input power contribution during energy storage charging. The energy flow model of the energy storage system is expressed by the following formula:

[0053] In the formula, d Q s / dt Q is needed to meet the charging and discharging requirements of thermal energy storage systems. sis the total energy of the energy storage system, with the unit of J; γ s is the energy storage self-loss coefficient, with the unit of s -1 ; ξ s is the energy storage system extreme event vulnerability coefficient; is the energy storage system basic power loss caused by the extreme event, with the unit of W.

[0054] Step 3.2, the power system power supplement, the gas system power supplement, the heat power supplement of the heat system and the charge-discharge demand of the energy storage system of the heat system are obtained by solving the electric-gas-heat-energy storage multi-energy disturbance propagation model, and the adjustment demand matrix is generated according to the solving result, which is expressed as follows:

[0055] In the formula, indicates the adjustment demand matrix.

[0056] Step 4, the global system state solved in step 1 and the optimal action strategy of each agent under the global system state are combined with the outage probability model obtained in step 2 and the adjustment demand matrix obtained in step 3, and a multi-agent collaborative strategy model is constructed by using MADDPG.

[0057] In the preferred but non-limiting embodiments of the application, step 4 includes: Step 4.1, the basic state space and the basic action space of the multi-agent are obtained according to the global system state solved in step 1 and the optimal action strategy of each agent under the global system state, as shown in Table 2, and the state space of the extreme event and the action space of the extreme event are solved by combining the outage probability model of the extreme event.

[0058] Further preferably, step 4.1 includes: Step 4.1.1, the basic state space and the basic action space of the multi-agent are set according to the global system state solved in step 1 and the optimal action strategy of each agent under the global system state, wherein the global system state contains the basic state space of each multi-agent, and the optimal action strategy of each agent under the global system state contains the basic action space of each agent, as shown in Table 2.

[0059] Table 2 Basic state-action space

[0060] Step 4.1.2, the basic state space of the multi-agent is connected with the outage probability model obtained in step 2, the change rate of the outage probability with time and the spatial change rate of the outage probability, and the extreme event state space is solved, which is expressed as follows:

[0061] In the formula, represents the state space of extreme events, used to describe the state space after increasing the characteristics of extreme event regulation requirements, represents the basic state space of multi-agent, , containing the basic state space of all agents, is the power outage probability at the current time, obtained by the power outage probability model of extreme events is calculated in real time, represents the rate of change of power outage probability with time, which is the time derivative of power outage probability, is the spatial rate of change of power outage probability, used to describe high-risk areas.

[0062] Step 4.1.3, the action a i is differentiated to solve the executed action a i The rate of change of power outage probability after the action is executed a i The rate of change of power outage probability after the action is executed a i The load amount cut off after the action is executed is multiplied by the corresponding weight coefficient respectively and summed up to obtain the load loss risk quantitative score of the action Γ(P outage a i ) , which is expressed as follows:

[0063] In the formula, α represents the weight coefficient, which is between 0 and 1, used to adjust the weight proportion of load reduction and risk sensitive term, LoadShed( a i ) is the amount of load cut off after the action is executed; a i is the marginal effect of the action on power outage probability, calculated by differentiating the power outage probability model of extreme events.

[0064] Step 4.1.4, set the load loss risk quantitative score of the action corresponding to the action located in the basic action space to be less than the set threshold value, obtain the extreme event action space, realize risk adaptive action selection, which is expressed as follows:

[0065] In the formula, represents the extreme event action space, represents the basic action space, γ th represents the threshold value, the upper limit of the constraint function Γ, which determines whether the action is allowed. ​​

[0066] Step 4.2, solve a single reward value according to the cost-effectiveness, global voltage safety state and adjustment performance calculation results, and combine the power outage probability model of each extreme event obtained in step 2 and the adjustment demand matrix obtained in step 3 to construct a risk-sensitive reward function.

[0067] Further preferably, step 4.2 includes: Step 4.2.1, obtain the cost-effectiveness calculation result by dividing the change amount of the current operating cost relative to the benchmark value by the benchmark operating cost; obtain the absolute deviation of the actual voltage from the standard value by subtracting the rated voltage from the actual voltage value of each monitoring point and taking the absolute value of the difference, divide the voltage risk degree by the voltage safety threshold, obtain the positive evaluation index of the voltage safety state by subtracting the voltage risk degree from 1, and obtain the global voltage safety state comprehensive calculation result by summing the positive evaluation indexes of the voltage safety states of all monitoring points; obtain the adjustment performance calculation result by the ratio of the actual response amount of the adjustable load to the maximum adjustable capacity; multiply the cost-effectiveness, global voltage safety state and adjustment performance calculation results by the corresponding weight coefficients to obtain a single reward value, and unify the economic, safety and adjustment performance of the three targets into a single reward value R t , which is expressed as follows:

[0068] In the formula, R t represents a single reward value, Δ C op is the change amount of the current operating cost relative to the benchmark value, Δ C op <0, the cost is reduced, the reward is increased, and vice versa. C base is the benchmark operating cost; V i is the actual voltage value of the i i th monitoring point, V nom is the rated voltage, V th is the voltage safety threshold, Q DR is the actual response amount of the adjustable load, Q max is the maximum adjustable capacity.

[0069] Step 4.2.2, multiply the risk sensitivity coefficient with the power outage probability model to obtain a risk penalty term, take the negative value of the risk penalty term and take the exponential function as the base to obtain an exponential decay function, which is used to convert the power outage probability model into an attenuation factor in the interval [0, 1], and then subtract one to offset the exponential decay function to the interval [-1, 0] to obtain a linear penalty value of power outage; The absolute error between the actual power adjustment amount generated by the agent action and the adjustment demand matrix is calculated by the norm, and the relative error rate is obtained by dividing the sum of the adjustment demand matrix and the zero prevention parameter. The generation adjustment accuracy score is equal to 1 minus the relative error rate; The linear penalty value of power outage and the adjustment accuracy score are multiplied by the corresponding weights respectively and the multiplication results are summed, and then added with a single reward value to obtain a risk-sensitive reward. The risk-sensitive reward function is represented by the following formula:

[0070] In the formula, The risk-sensitive reward is represented by Δ P x,act The actual power adjustment amount generated by the multi-agent action is β The risk sensitivity coefficient is high, and the penalty index grows when the power outage probability is high; w 1~ w 5 is the weight coefficient, which needs to satisfy w 1+ w 2+ w 3+ w 4+ w 5=1, which reflects the priority of different goals; ε The zero prevention parameter is .

[0071] Step 4.3, according to the state space of extreme events and the action space of extreme events obtained in step 4.1 and the risk-sensitive reward function obtained in step 4.2, a multi-agent deep reinforcement learning algorithm is used to construct a multi-agent collaborative optimization model to obtain a multi-agent collaborative strategy of comprehensive energy under extreme events, and realize the multi-agent collaborative optimization scheduling of comprehensive energy under extreme events.

[0072] Further preferably, step 4.3 includes: Step 4.3.1, respectively according to the extreme event state space, the Actor network of each agent is constructed for multi-agent (PA / CA / SA / ALA), and the Critic network of each agent is constructed according to the optimal action strategy of each agent under the global power grid state and the global system state, and the multi-agent Actor-Critic network of the multi-agent collaborative optimization model is initialized.

[0073] More preferably, step 4.3.1 includes: Actor network of the i-th agent is constructed, and the action policy function is set as μ i( ; ), the extreme event state space of the i-th agent is input , , the weight parameters of the i-th agent Actor network are represented, and the action is output a i .

[0074] Critic network of the i-th agent is constructed, and the action value evaluation function is set as Q i ( , a 1,... a N ; ), the global extreme event state space containing all agents is input and the joint action a 1,... a N , wherein the joint action a 1,... a N represents the action set of all agents, which conforms to the extreme event action space , , the weight parameters of the i-th agent Critic network are represented, and the action value is output Q .

[0075] Step 4.3.2, according to the sum of the action output by the Actor network of each agent in step 4.3.1 and the exploration noise, select the agent action from the extreme event action space, and obtain the agent action that satisfies the extreme event action space to the following formula:

[0076] , wherein represents the agent action, N t is the exploration noise, and Clip() represents the Clip function, which is used to ensure that the action satisfies the feasible region constraint.

[0077] Step 4.3.3, construct the agent joint action according to the agent action obtained in step 4.3.2, and execute the agent joint action a ={ a 1,multi , ... a N,multi}, obtain the risk-sensitive reward of the multi-agent and the new global multi-agent state The transition tuple (S, A, R, S') is stored in the experience pool. , a , , .

[0078] Step 4.3.4. Obtain the transition tuple from the experience pool in step 4.3.3 to train the centralized network multi-agent collaborative strategy model, and solve to obtain a preliminary multi-agent collaborative strategy model.

[0079] More preferably, step 4.3.4 includes: Sampling a batch of data from the experience pool, updating as follows: For each agent i , update the Critic network by minimizing the time difference error, which is expressed as follows:

[0080] In the formula, represents the loss function of the Critic network, represents the expectation, Q i represents the current Q value of the Critic network evaluating the joint action, y i represents the time difference target Q value, which is expressed as follows:

[0081] In the formula, r i The risk-sensitive reward is set as the immediate reward , which reflects the effect of the action in real time, Q i ’ represents the Q value at the next time evaluated by the target network, represents the estimated action vector of the output agent N in the next time state of the agent N, , is the discount factor.

[0082] Update the Actor network by policy gradient ascent method to maximize the expected return, which is expressed as follows:

[0083] In the formula, represents the change direction of the expected return J when adjusting , represents the gradient of the Q value output by the Critic network with respect to the action ; Actor network output Parameter Gradient.

[0084] Update the target network parameters, expressed as follows:

[0085] In the formula, The weight parameters of the Critic target network of the first i Intelligent agent at the next time, The weight parameters of the Actor target network of the first i Intelligent agent at the next time, Soft update coefficient.

[0086] The weight parameters of the first i Intelligent agent Critic target network of the updated target network at the next time are minimized by gradient descent to minimize the time difference error , and a new action a i ’= μ i( ; ), the cumulative reward is maximized by policy gradient ; Store the updated and , if in the last N iterations, Satisfy , The L2 norm of , determine the network convergence, enter step 4.3.5; otherwise, return to step 4.3.2, and generate actions based on the optimized network.

[0087] Step 4.3.5, Nash equilibrium is performed on the action strategy of the multi-agent output by the preliminary multi-agent cooperative strategy model obtained in step 4.3.4, and the multi-agent cooperative strategy is solved to obtain the multi-agent cooperative strategy model, expressed as follows:

[0088] Where, The updated strategy of the first i Intelligent agent, The expected cumulative reward of the intelligent agent i , indicates the policy coordination strength weight, The policy function of the first Intelligent agent i μ i( ;​ ), for describing the optimal action policy output after step 4.3.4 training, which can be regarded as the optimal action of the i-th agent after training, and KL() is the KL divergence used to measure the difference between the agent i policy π i and the average policy π avg , π avg =∑ π j / N is the average policy, which promotes policy consensus.

[0089] Step 5, input the real-time monitoring data of the integrated energy multi-agent into the multi-agent collaborative strategy model constructed in step 4 to obtain the integrated energy multi-agent collaborative optimization scheduling strategy under extreme events, and realize the integrated energy multi-agent collaborative optimization scheduling under extreme events.

[0090] Embodiment 2 of the present application provides an integrated energy multi-agent collaborative optimization scheduling system under extreme events, which runs the integrated energy multi-agent collaborative optimization scheduling method under extreme events described in embodiment 1, comprising: A multi-agent state extraction module is used to extract the spatio-temporal characteristics of the integrated energy system multi-agent through mutual information and fast Fourier transform, and to construct a lightweight general model of the agent through knowledge distillation and solve it to obtain the global system state and the optimal action policy of each agent under the global system state. A power outage probability solving module is used to construct a power outage probability model of each extreme event, and the extreme events include typhoon events, extreme cold events, high temperature events and thunderstorm events. A demand solving module is used to construct an extreme event driven electric-gas-heat- storage multi-energy disturbance propagation model according to the basic power loss of the power system, the basic power loss of the gas system, the basic power loss of the heat system and the basic power loss of the energy storage system caused by extreme events, solve the model and generate an adjustment demand matrix. A collaborative strategy model construction module is used to combine the global system state and the optimal action policy of each agent under the global system state with the power outage probability model of each extreme event and the adjustment demand matrix, and to construct a multi-agent collaborative strategy model by using a multi-agent deep reinforcement learning method. An output module is used to input the real-time monitoring data of the integrated energy multi-agent into the multi-agent collaborative strategy model to obtain the integrated energy multi-agent collaborative optimization scheduling strategy under extreme events.

[0091] Compared with the prior art, the present application has at least the following advantages: The application uses a knowledge distillation technology, supports edge deployment, and enables the integrated energy system to still operate independently when an extreme event causes a communication interruption, improves the safety of integrated energy system scheduling, uses a Nash equilibrium method of a distributed decision architecture to reward and punish each agent, forces the agent to consider the actions of other agents when pursuing its own benefits, reduces multi-agent strategy conflicts, and improves the efficiency and accuracy of integrated energy system multi-agent collaborative optimization. The application enhances the basic state space of the multi-agent by constructing a power outage probability model of the extreme event, improves the risk perception ability of the agent under the extreme event, and restricts the basic action space of the multi-agent through the power outage probability model of the extreme event, so that the action selection of the integrated energy system is limited to the extreme event and high-risk actions under the extreme event are constrained, thereby improving the adaptive ability and safety of the integrated energy system under the extreme event. The application generates a scheduling demand by constructing an electricity-gas-heat-storage multi-energy disturbance propagation model, generates a risk-sensitive reward function of the extreme event by combining the scheduling demand with the power outage probability model of the extreme event, reduces the load loss of the integrated energy system under the extreme event, and improves the economy of the integrated energy system regulation. The application balances multiple agent strategies through Nash equilibrium, reduces the conflict rate of the integrated energy system under multi-agent strategy regulation, and improves the safety of the integrated energy system under multi-agent strategy regulation.

[0092] The present disclosure can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions loaded thereon for causing a processor to implement various aspects of the present disclosure.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit it, although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the specific embodiments of the present application can be modified or replaced, without departing from the spirit and scope of the present application, any modification or equivalent replacement, which should be covered within the protection scope of the claims of the present application.

Claims

1. An integrated energy multi-agent collaborative optimization scheduling method under extreme events, characterized in that, The method comprises the following steps: extracting spatiotemporal features of the multi-agent of the integrated energy system through mutual information and fast Fourier transform, and constructing a lightweight general model of the agent through knowledge distillation and solving the model to obtain the global system state and the optimal action strategy of each agent under the global system state; constructing a power outage probability model of each extreme event, including a typhoon event, an extreme cold event, a high temperature event, and a thunderstorm event; constructing an extreme event-driven electric-gas-thermal-storage multi-energy disturbance propagation model according to the basic power loss of the power system, the basic power loss of the gas system, the basic power loss of the heat supply system, and the basic power loss of the energy storage system caused by the extreme event, solving the model, and generating an adjustment demand matrix; combining the global system state and the optimal action strategy of each agent under the global system state with the power outage probability model of each extreme event and the adjustment demand matrix, and constructing a multi-agent collaborative strategy model by using a multi-agent deep reinforcement learning method; inputting real-time monitoring data of the integrated energy multi-agent into the multi-agent collaborative strategy model to obtain a collaborative optimization scheduling strategy of the integrated energy multi-agent under the extreme event.

2. The method according to claim 1, wherein the construction of the lightweight general model of the agent and the solving of the model comprise: dividing the energy physical devices of the integrated energy system into multi-type agents according to functional attributes to obtain a multi-type agent cluster of the integrated energy system; obtaining data of the multi-type agent cluster of the integrated energy system, extracting spatiotemporal features through mutual information and fast Fourier transform, and obtaining the spatiotemporal features; migrating knowledge in a teacher model to a student model through knowledge distillation to obtain lightweight student model data; setting the lightweight student model data as a global system state, combining a multi-agent Nash equilibrium to construct a lightweight general model of the agent, and solving the lightweight general model of the agent to obtain the optimal action strategy of each agent under the global system state.

3. The method according to claim 2, wherein the solving of the lightweight general model of the agent to obtain the optimal action strategy of each agent under the global system state comprises: constructing a reward function of each agent according to the action of each agent itself and the actions of other agents; setting the lightweight student model data as the global system state, inputting a safety constraint function, setting a safety constraint relaxation term as an output result of the safety constraint function if the safety constraint function is violated, or setting the safety constraint relaxation term as 0 otherwise; subtracting a product of a balance coefficient and the safety constraint relaxation term from the reward function of each agent to obtain the optimal action strategy of the current agent; summing the optimal action strategies of each agent to obtain the optimal action strategy of each agent under the global system state.

4. The method according to claim 1, wherein the construction of the power outage probability model of each extreme event comprises: ​ ​ ​ The probability of power outage caused by precipitation, the cumulative risk probability corresponding to the degree of wind speed deviation, and the joint risk of precipitation and wind speed are solved by wind speed and precipitation, and a power outage probability model of typhoon events is constructed according to the solving results; The probability of power outage caused by extreme cold temperature and the probability of power outage caused by the duration of extreme cold event are solved by temperature and duration, and a power outage probability model of extreme cold event is constructed according to the solving results; The probability of power outage caused by high temperature is solved by temperature, and a power outage probability model of high temperature event is constructed by combining air conditioning load rate; The probability of power outage caused by lightning density and the nonlinear saturation risk of rainfall are solved by lightning density and rainfall, and a power outage probability model of thunderstorm event is constructed according to the solving results.

5. The method according to claim 1, wherein: The generation of the adjustment demand matrix comprises: Including: An energy flow model of the power system is constructed according to the basic power loss of the power system caused by the extreme event, an energy flow model of the gas system is constructed according to the basic power loss of the gas system caused by the extreme event, an energy flow model of the heat system is constructed according to the basic power loss of the heat system caused by the extreme event, and an energy flow model of the energy storage system is constructed according to the basic power loss of the energy storage system caused by the extreme event; An electric-gas-heat-storage multi-energy disturbance propagation model is constructed according to the energy flow models of the power system, the gas system, the heat system and the energy storage system; The electric-gas-heat-storage multi-energy disturbance propagation model is solved to generate the power system power supplement demand, the gas system power supplement demand, the heat system heat power supplement demand and the charge-discharge demand of the energy storage system of the heat system, and an adjustment demand matrix is constructed according to the generated results.

6. The method according to claim 1, wherein: The construction of the multi-agent collaborative strategy model comprises: The basic state space and the basic action space of the multi-agent are obtained according to the global system state and the optimal action strategy of each agent under the global system state, and the extreme event state space and the extreme event action space are solved by combining the power outage probability model of the extreme event; A single reward value is solved according to the cost benefit, the global voltage safety state and the adjustment efficiency calculation result, and a risk sensitive reward function is constructed by combining the power outage probability model of each extreme event and the adjustment demand matrix; A multi-agent collaborative optimization model is constructed by using a multi-agent deep reinforcement learning algorithm according to the state space of the extreme event and the action space of the extreme event and the risk sensitive reward function.

7. The method according to claim 6, wherein: The extreme event action space comprises: The rate of change of the power outage probability after the action is executed is solved by differentiating the action through the power outage probability model, the rate of change of the power outage probability after the action is executed and the amount of load cut off after the action is executed are multiplied by corresponding weight coefficients respectively and summed to obtain the load loss risk quantitative score of the action; The load loss risk quantification score of the action corresponding to the action located in the basic action space is less than a set threshold, and an extreme event action space is obtained.

8. The integrated energy multi-agent collaborative optimization scheduling method under extreme events according to claim 6, characterized in that: the construction of the risk-sensitive reward function comprises: the risk-sensitive coefficient is multiplied by the power outage probability model to obtain a risk penalty term, the risk penalty term is taken as a negative value and is based on an exponential function to obtain an exponential decay function, and then a minus 1 operation is performed to obtain a linear penalty value of power outage; the absolute error of the actual power adjustment amount generated by the agent action and the adjustment demand matrix is calculated, and the absolute error is divided by the sum of the adjustment demand matrix and the zero prevention parameter to obtain a relative error rate, and the relative error rate is subtracted by 1 to obtain a generation adjustment accuracy score; the linear penalty value of power outage and the adjustment accuracy score are multiplied by the corresponding weights respectively, and the multiplication results are summed, and then added to a single reward value to obtain a risk-sensitive reward function.

9. The integrated energy multi-agent collaborative optimization scheduling method under extreme events according to claim 6, characterized in that: the construction of the multi-agent collaborative optimization model comprises: the Actor network of each agent is constructed according to the extreme event state space for the multi-agent, and the Critic network of each agent is constructed according to the extreme event action space and the extreme event state space; the sum of the action output from the Actor network of each agent and the exploration noise is obtained to obtain the agent action satisfying the extreme event action space; the joint action of the agents is constructed according to the agent action, the joint action of the agents is executed, the risk-sensitive reward of the multi-agent and the new multi-agent state are obtained, and the transition tuple is constructed according to the extreme event state space and the joint action of the agents, and is stored in the experience pool; the transition tuple is obtained from the experience pool, and the multi-agent collaborative strategy model is trained to obtain a preliminary multi-agent collaborative strategy model; the action strategy of the multi-agent output by the preliminary multi-agent collaborative strategy model is subjected to Nash equilibrium to obtain the multi-agent collaborative strategy, and the multi-agent collaborative strategy model is constructed.

10. An integrated energy multi-agent collaborative optimization scheduling system under extreme events, which runs the integrated energy multi-agent collaborative optimization scheduling method under extreme events according to any one of claims 1-9, characterized in that: a multi-agent state extraction module is configured to extract the space-time characteristics of the multi-agent of the integrated energy system through mutual information and fast Fourier transform, construct a lightweight general model of the agent through knowledge distillation and light weighting, and solve the model to obtain the global system state and the optimal action strategy of each agent under the global system state; a power outage probability solving module is configured to construct a power outage probability model of each extreme event, and the extreme events include typhoon events, extreme cold events, high temperature events and thunderstorm events; a demand solving module is configured to construct an extreme event driven electric-gas-heat-storage multi-energy disturbance propagation model according to the basic power loss of the power system, the basic power loss of the gas system, the basic power loss of the heat system and the basic power loss of the energy storage system caused by the extreme event, solve the model and generate an adjustment demand matrix. The cooperative strategy model construction module is configured to combine the global system state and the optimal action strategy of each intelligent agent under the global system state with the power cut probability model of each extreme event and the adjustment demand matrix, and construct a multi-intelligent agent cooperative strategy model by using a multi-intelligent agent deep reinforcement learning method. The output module is configured to input real-time monitoring data of the comprehensive energy multi-intelligent agent into the multi-intelligent agent cooperative strategy model, and obtain a comprehensive energy multi-intelligent agent cooperative optimization dispatching strategy under the extreme event.

Images