Power system intelligent optimization scheduling method and system in energy transmission limited environment

Abstract

The invention discloses an intelligent optimal scheduling method and system for a power system in an energy transmission limited environment, and the method comprises the steps: constructing a power supply party-power utilization party energy management model, and enabling the optimal scheduling target of the model to be a minimum power supply party energy management target function and a minimum power utilization party energy management target function; and converting the optimal scheduling target into a shared reward function maximization form of a multi-agent system, and solving the distributed energy management model based on a multi-agent reinforcement learning method to obtain a scheduling scheme of the mobile energy storage vehicle. According to the design, on the basis of meeting the load demand of the power consumer, a power supplier energy management objective function is constructed by taking the minimum transportation cost and the renewable energy source discarding amount as the target, and the power consumer energy management objective function is constructed by taking the minimum controllable load amount for maintaining supply and demand balance removal as the target; therefore, the effect of maximizing the wind and light utilization rate is achieved on the basis of ensuring the independent, safe and stable operation of the renewable energy system in the energy transmission limited area.

Description

Technical Field

[0001] The present invention belongs to the technical field of energy system optimization decision-making, and specifically relates to an intelligent optimization scheduling method and system for an electric power system under an energy transmission restricted environment. Background Art

[0002] Compared with the energy supply system in cities and areas with convenient transportation, the development and utilization of energy in areas with limited energy transmission is relatively scarce. The energy supply in these limited areas often relies more on traditional energy supply methods, such as independent power generation systems such as small diesel generators, but traditional energy supply methods bring high operating costs. At the same time, these areas with limited energy transmission often contain abundant renewable energy, such as solar energy and wind energy, so renewable energy generation technology will provide a new energy supply channel for areas with limited energy transmission. At the same time, these resources have significant advantages of large reserves, wide distribution, clean and renewable, and have become an important direction to alleviate energy shortages, reduce energy costs, and promote green development.

[0003] However, due to the complex terrain of energy transmission restricted areas and the uneven distribution of energy between different energy transmission restricted areas, the energy transmission between power suppliers and power users is limited. Mobile energy storage vehicles can flexibly connect power suppliers and power users, develop and utilize renewable energy, and promote sustainable development in energy transmission restricted areas. How to optimize the scheduling of mobile energy storage vehicles, maximize the utilization rate of wind and solar power, and reduce wind and solar power abandonment on the basis of ensuring the independent, safe and stable operation of renewable energy systems in energy transmission restricted areas, is the main research direction. Summary of the invention

[0004] The purpose of the present invention is to address the above-mentioned problems existing in the prior art and to provide a method and system for intelligent optimization and scheduling of power systems in energy transmission restricted environments, which can maximize the utilization rate of wind and solar power while ensuring the independent, safe and stable operation of renewable energy systems in energy transmission restricted areas.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] In a first aspect, the present invention provides a method for intelligent optimization and scheduling of a power system under an energy transmission-constrained environment, the optimization and scheduling method comprising the following steps:

[0007] S1. Construct a wind-solar-hydrogen coupled energy supply system model and a mobile energy storage vehicle-road resistance model in areas with restricted energy transmission; construct a power supplier-power user energy management model in areas with restricted energy transmission based on the wind-solar-hydrogen coupled energy supply system model and the mobile energy storage vehicle-road resistance model; the optimal scheduling goal of the power supplier-power user energy management model is to minimize the power supplier energy management objective function and the power user energy management objective function, wherein the power supplier energy management objective function is to minimize the transportation cost and the amount of renewable energy discarded on the basis of meeting the load demand of the power user; the power user energy management objective function is to minimize the amount of controllable load removed to maintain the balance between supply and demand;

[0008] S2. Convert the optimal scheduling objective of the power supplier-user energy management model into the form of maximizing the shared reward function of the multi-agent system, solve the distributed energy management model based on the multi-agent reinforcement learning method, and obtain the scheduling plan for the mobile energy storage vehicle.

[0009] The energy management objective function of the power supplier is:

[0010]

[0011] E w,i,t +E pv,i,t =P w t+P pv tC ij,t E mes ;

[0012]

[0013] In the above formula, F g represents the energy management objective function of the power supplier; λ1 and λ2 are penalty factors; E w,i,t 、E pv,i,t , are the wind power, solar power and hydrogen abandonment of the power supplier in time period t; d ij is the travel distance of the mobile energy storage vehicle from the i-th power supplier to the j-th power user; E mes Storing energy for mobile energy storage vehicles; C ij,t is the number of mobile energy storage vehicles dispatched from the i-th power supplier to the j-th power user in time period t. The mobile energy storage vehicles transmit discrete energy in the form of loaded energy packages; V mes 、V non are the loading space volume and non-energy storage space volume of the mobile energy storage vehicle respectively; V ep 、E ep are the volume and storage capacity of a single energy pack respectively; P w , P pv are the theoretical output powers of wind turbines and photovoltaic arrays respectively; is the hydrogen production of the electrolyzer; n hst (t) is the hydrogen storage capacity of the hydrogen tank in time period t; N and M are the total number of power suppliers and power users respectively; T is the total cycle;

[0014] The energy management objective function of the electricity user is:

[0015]

[0016] E cut,j,t =E j,t -I j,t E mes ;

[0017]

[0018] E i,t =P w t+P pv t;

[0019] E j,t =P equip t;

[0020] In the above formula, F u represents the energy management objective function of the electricity user; λ3 is the penalty factor for load shedding; p c E is the contribution of the i-th power supplier to the energy demand of the j-th power user; cut,j,t is the controllable load removed by the jth electricity user in time period t; E i,t 、E j,t are the power that the i-th power supplier can supply and the power demand of the j-th power user in time period t; P equip I is the operating power of the electricity user’s electrical equipment; j,t is the total number of mobile energy storage vehicles received by the jth electricity user from each power supplier in time period t;

[0021] The shared reward function is:

[0022]

[0023] Φ=β1F g +β2F u ;

[0024] In the above formula, r n represents the shared reward function value of agent n; is the adjustment coefficient; Φ represents the intermediate calculation amount; β1 and β2 are both adjustment weights.

[0025] The wind-solar-hydrogen coupled energy supply system model includes:

[0026] Wind turbine energy model:

[0027]

[0028] In the above formula, P w , P wr are the theoretical output power and rated power of the wind turbine respectively; v, v w are the actual wind speed and rated wind speed respectively; v in 、v out They are cut-in wind speed and cut-out wind speed respectively;

[0029] Photovoltaic array energy model:

[0030]

[0031] In the above formula, P pv , P pr are the theoretical output power and rated power of the photovoltaic array respectively; α is the rated conversion efficiency of the photovoltaic panel; G and G0 are the actual light radiation intensity and the reference light radiation intensity respectively; k p is the temperature power coefficient; T pv , T pr They are the actual temperature and rated temperature of the photovoltaic panel respectively;

[0032] Hydrogen energy system model:

[0033]

[0034] In the above formula, are the hydrogen production of the electrolyzer and the fuel cell respectively; P ec , P pc are the input power of the electrolyzer and the output power of the fuel cell respectively; η ec , η pc are the hydrogen production efficiency of the electrolyzer and the operating efficiency of the fuel cell respectively; n hst (t), n hst (t0) are the hydrogen storage capacity of the hydrogen tank in time period t and time period t0, respectively, t=t0+Δt, Δt is the unit time interval; is the net hydrogen storage rate of the hydrogen storage tank in time period t; dt represents the differential of t; h ec The power consumption of the electrolyzer to produce unit volume of hydrogen; μ pc The amount of hydrogen consumed to generate a unit of electricity for the fuel cell; are the hydrogen storage rate and hydrogen consumption rate of the hydrogen storage tank in time period t respectively;

[0035] The mobile energy storage vehicle-road resistance model includes:

[0036] w ij,t =E ij (t);

[0037]

[0038] P mes,c =F e v ts ;

[0039] F e =F f +F w +F r +F a +F b ;

[0040] In the above formula, w ij (t) represents the impact value of the environmental disturbance on the mobile energy storage vehicle in time period t; E ij (t) is the energy consumption of the mobile energy storage vehicle to overcome environmental disturbances in time period t; C ij,t P is the number of mobile energy storage vehicles dispatched from the i-th power supplier to the j-th power user in time period t; mes,c is the driving power of the cth mobile energy storage vehicle; F e Indicates the traction force of the mobile energy storage vehicle; v ts is the actual driving speed of the mobile energy storage vehicle; F f 、F w 、F r 、F a 、F b They are respectively the rolling resistance, air resistance, transmission system resistance, slope resistance and vehicle acceleration resistance of the mobile energy storage vehicle.

[0041] The step S2 is specifically as follows:

[0042] S21. Construct a partially observable Markov decision process, treat each power supplier as an independent intelligent agent, each of which includes a policy network and a value network; define a multi-tuple POMDP = (N, S, A, O, R, P) to represent the energy scheduling process in the energy transmission restricted area, where N is the number of power suppliers, i.e. the number of intelligent agents, and S = {s 1 ,…,s i ,…,s N},s i represents the global state space of the environment, including the state, action and interaction relationship of agent i with other agents, O = {o 1 ,…,o i ,…,o N},o i represents the local observation space of the environment by agent i; A = {a 1 ,…,a i ,…,a N}, a i represents the action of agent i;

[0043] S22, the agent includes a policy network and a value network; in the centralized training phase, the agent i obtains local observations o from the environment i = {P equip ,P w ,P pv ,P ec ,P pc} as the input of the policy network, which i Make an action i ={C ij,t} and get shared rewards i ; After all agents complete their actions, the state information, action information, and shared rewards of each agent are input into the value network of each agent. The value network calculates the joint advantage function, and the loss function of the value network and the policy network is calculated based on the joint advantage function. The network parameters of the value network and the policy network are updated by minimizing the loss function of the value network and the policy network. In the decentralized execution stage, each agent inputs local observations into its own policy network to make decisions.

[0044] The calculation formula of the joint advantage function is:

[0045]

[0046] In the above formula, is the joint advantage function; are the time series difference errors at time t, time t+1, and time T-1 respectively; γ is the discount rate; φ is the smoothing coefficient; For agent n in strategy Next, from state s t+1 Execute action a t+1 The expected return that can be obtained in the future; For agent n in strategy Next, from state s t Start executing action a t The expected return that can be obtained in the future; represents the shared reward obtained by agent n at time t; represents the old network parameters of the value network;

[0047] The expression of the loss function of the value network is:

[0048]

[0049] In the above formula, Represents the loss function of the value network; For agent n in strategy Next, from state s t+1 Start executing action at+1 The expected return that can be obtained in the future; For agent n in strategy Next, from state s t Start executing action a t The expected return that can be obtained in the future; Represents the current network parameters of the value network;

[0050] The loss function of the policy network is expressed as:

[0051]

[0052] In the above formula, loss(θ n ) represents the loss function of the policy network; χ(θ n ) is the ratio of the current strategy of the policy network to the old strategy; is the current policy of the policy network; is the old policy of the policy network; clip represents the clipping function; ε represents the clipping parameter; θ n Represents the current network parameters of the policy network; Represents the old network parameters of the policy network.

[0053] In a second aspect, the present invention provides an intelligent optimization and dispatching system for a power system under an energy transmission-constrained environment, wherein the optimization and dispatching system includes a model building module and a model solving module;

[0054] The model building module is used to build a wind-solar-hydrogen coupled energy supply system model and a mobile energy storage vehicle-road resistance model in an area with restricted energy transmission, and to build a power supplier-power user energy management model in an area with restricted energy transmission based on the wind-solar-hydrogen coupled energy supply system model and the mobile energy storage vehicle-road resistance model; the optimal scheduling goal of the power supplier-power user energy management model is to minimize the power supplier energy management objective function and the power user energy management objective function, and the power supplier energy management objective function is to minimize the transportation cost and the amount of renewable energy discarded on the basis of meeting the load demand of the power user; the power user energy management objective function is to minimize the controllable load removed to maintain the balance between supply and demand;

[0055] The model solving module is used to convert the optimal scheduling target of the power supplier-power user energy management model into the maximization form of the shared reward function of the multi-agent system, solve the distributed energy management model based on the multi-agent reinforcement learning method, and obtain the scheduling plan of the mobile energy storage vehicle.

[0056] The energy management objective function of the power supplier is:

[0057]

[0058] Ew,i,t +E pv,i,t =P w t+P pv tC ij,t E mes ;

[0059]

[0060] In the above formula, F g represents the energy management objective function of the power supplier; λ1 and λ2 are penalty factors; E w,i,t 、E pv,i,t , are the wind power, solar power and hydrogen abandonment of the power supplier in time period t; d ij is the travel distance of the mobile energy storage vehicle from the i-th power supplier to the j-th power user; E mes Storing energy for mobile energy storage vehicles; C ij,t is the number of mobile energy storage vehicles dispatched from the i-th power supplier to the j-th power user in time period t. The mobile energy storage vehicles transmit discrete energy in the form of loaded energy packages; V mes 、V non are the loading space volume and non-energy storage space volume of the mobile energy storage vehicle respectively; V ep 、E ep are the volume and storage capacity of a single energy pack respectively; P w , P pv are the theoretical output powers of wind turbines and photovoltaic arrays respectively; is the hydrogen production of the electrolyzer; n hst (t) is the hydrogen storage capacity of the hydrogen tank in time period t; N and M are the total number of power suppliers and power users respectively; T is the total cycle;

[0061] The energy management objective function of the electricity user is:

[0062]

[0063] E cut,j,t =E j,t -I j,t E mes ;

[0064]

[0065] E i,t =P w t+P pv t;

[0066] E j,t =P equip t;

[0067] In the above formula, F urepresents the energy management objective function of the electricity user; λ3 is the penalty factor for load shedding; p c E is the contribution of the i-th power supplier to the energy demand of the j-th power user; cut,j,t is the controllable load removed by the jth electricity user in time period t; E i,t 、E j,t are the power that the i-th power supplier can supply and the power demand of the j-th power user in time period t; P equip I is the operating power of the electricity user’s electrical equipment; j,t is the total number of mobile energy storage vehicles received by the jth electricity user from each power supplier in time period t;

[0068] The shared reward function is:

[0069]

[0070] Φ=β1F g +β2F u ;

[0071] In the above formula, r n represents the shared reward function value of agent n; is the adjustment coefficient; Φ represents the intermediate calculation amount; β1 and β2 are both adjustment weights.

[0072] The wind-solar-hydrogen coupled energy supply system model includes:

[0073] Wind turbine energy model:

[0074]

[0075] In the above formula, P w , P wr are the theoretical output power and rated power of the wind turbine respectively; v, v w are the actual wind speed and rated wind speed respectively; v in 、v out They are cut-in wind speed and cut-out wind speed respectively;

[0076] Photovoltaic array energy model:

[0077]

[0078] In the above formula, P pv , P pr are the theoretical output power and rated power of the photovoltaic array respectively; α is the rated conversion efficiency of the photovoltaic panel; G and G0 are the actual light radiation intensity and the reference light radiation intensity respectively; k p is the temperature power coefficient; T pv , T pr They are the actual temperature and rated temperature of the photovoltaic panel respectively;

[0079] Hydrogen energy system model:

[0080]

[0081] In the above formula, are the hydrogen production of the electrolyzer and the fuel cell respectively; P ec , P pc are the input power of the electrolyzer and the output power of the fuel cell respectively; η ec , η pc are the hydrogen production efficiency of the electrolyzer and the operating efficiency of the fuel cell respectively; n hst (t), n hst (t0) are the hydrogen storage capacity of the hydrogen tank in time period t and time period t0, respectively, t=t0+Δt, Δt is the unit time interval; is the net hydrogen storage rate of the hydrogen storage tank in time period t; dt represents the differential of t; h ec The power consumption of the electrolyzer to produce unit volume of hydrogen; μ pc The amount of hydrogen consumed to generate a unit of electricity for the fuel cell; are the hydrogen storage rate and hydrogen consumption rate of the hydrogen storage tank in time period t respectively;

[0082] The mobile energy storage vehicle-road resistance model includes:

[0083] w ij,t =E ij (t);

[0084]

[0085] P mes,c =F e v ts ;

[0086] F e =F f +F w +F r +F a +F b ;

[0087] In the above formula, w ij (t) represents the impact value of the environmental disturbance on the mobile energy storage vehicle in time period t; E ij (t) is the energy consumption of the mobile energy storage vehicle to overcome environmental disturbances in time period t; C ij,t P is the number of mobile energy storage vehicles dispatched from the i-th power supplier to the j-th power user in time period t; mes,c is the driving power of the cth mobile energy storage vehicle; F e Indicates the traction force of the mobile energy storage vehicle; v tsis the actual driving speed of the mobile energy storage vehicle; F f 、F w 、F r 、F a 、F b They are respectively the rolling resistance, air resistance, transmission system resistance, slope resistance and vehicle acceleration resistance of the mobile energy storage vehicle.

[0088] The model solving step is used to solve the distributed energy management model according to the following steps to obtain a scheduling plan for the mobile energy storage vehicle:

[0089] S21. Construct a partially observable Markov decision process, treat each power supplier as an independent agent, each agent includes a strategy network and a value network; define a multi-tuple POMDP = (N, S, A, O, R, P) to represent the energy scheduling process in the energy transmission restricted area, N is the number of power suppliers, that is, the number of agents, S = {s 1 ,…,s i ,…s, N},s i represents the global state space of the environment, including the state, action and interaction relationship of agent i with other agents, O = {o 1 ,…,o i ,…,o N},o i represents the local observation space of the environment by agent i; A = {a 1 ,…,a i ,…,a N}, a i represents the action of agent i;

[0090] S22, centralized training phase, agent i obtains local observations o from the environment i = {P equip ,P w ,P pv ,P ec ,P pc} as the input of the policy network, which i Make an action i ={C ij,t} and get shared rewards i ; After all agents complete their actions, the state information, action information, and shared rewards of each agent are input into the value network of each agent. The value network calculates the joint advantage function, and the loss function of the value network and the policy network is calculated based on the joint advantage function. The network parameters of the value network and the policy network are updated by minimizing the loss function of the value network and the policy network. In the decentralized execution stage, each agent inputs local observations into its own policy network to make decisions.

[0091] The calculation formula of the joint advantage function is:

[0092]

[0093] In the above formula, is the joint advantage function; are the time series difference errors at time t, time t+1, and time T-1 respectively; γ is the discount rate; φ is the smoothing coefficient; For agent n in strategy Next, from state s t+1 Execute action a t+1 The expected return that can be obtained in the future; For agent n in strategy Next, from state s t Start executing action a t The expected return that can be obtained in the future; represents the shared reward obtained by agent n at time t; represents the old network parameters of the value network;

[0094] The expression of the loss function of the value network is:

[0095]

[0096] In the above formula, Represents the loss function of the value network; For agent n in strategy Next, from state s t+1 Start executing action a t+1 The expected return that can be obtained in the future; For agent n in strategy Next, from state s t Start executing action a t The expected return that can be obtained in the future; Represents the current network parameters of the value network;

[0097] The loss function of the policy network is expressed as:

[0098]

[0099]

[0100] In the above formula, loss(θ n ) represents the loss function of the policy network; χ(θ n ) is the ratio of the current strategy of the policy network to the old strategy; is the current policy of the policy network; is the old policy of the policy network; clip represents the clipping function; ε represents the clipping parameter; θn Represents the current network parameters of the policy network; Represents the old network parameters of the policy network.

[0101] Compared with the prior art, the present invention has the following beneficial effects:

[0102] The present invention provides an intelligent optimization and scheduling method for a power system in an environment with limited energy transmission. The method first constructs the optimal scheduling target of the power supplier-power user energy management model, minimizes the power supplier's energy management objective function and the power user's energy management objective function, and then converts the optimal scheduling target into a shared reward function maximization form of a multi-agent system. The distributed energy management model is solved based on a multi-agent reinforcement learning method to obtain a scheduling plan for a mobile energy storage vehicle. This design constructs the power supplier's energy management objective function with the goal of minimizing transportation costs and the amount of renewable energy discarded on the basis of meeting the load demand of the power user, and constructs the power user's energy management objective function with the goal of minimizing the amount of controllable load removed to maintain a balance between supply and demand. This maximizes the utilization rate of wind and solar power while ensuring the independent, safe and stable operation of the renewable energy system in the area with limited energy transmission, and promotes the sustainable development of the area with limited energy transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0103] Figure 1 The present invention is a flowchart of the method.

[0104] Figure 2 This is a schematic diagram of the wind-solar-hydrogen coupled energy supply system model.

[0105] Figure 3 It is a schematic diagram of the structure of the system of the present invention. DETAILED DESCRIPTION

[0106] The present invention is further described in detail below in conjunction with specific implementations and drawings.

[0107] Embodiment 1:

[0108] This embodiment is as follows Figure 2 The energy transmission restricted area shown in the figure is taken as the research object. There are four main bodies in this energy transmission restricted area, each of which has the ability to operate independently and work together. Under the condition of restricted energy transmission, they can flexibly adjust the roles of power suppliers and power users to ensure the stability and reliability of energy supply in the area. In addition, the four main bodies are equipped with electric poles and towers, and the discrete transmission of energy packages between power suppliers and power users is achieved through mobile energy storage vehicles. At the same time, continuous real-time energy transmission can be achieved within each main body through electric poles and towers.

[0109] See also Figure 1 , an intelligent optimization dispatching method for power system under energy transmission limited environment, is carried out in the following steps:

[0110] S1. Construct a wind-solar-hydrogen coupled energy supply system model and a mobile energy storage vehicle-road resistance model in areas with limited energy transmission

[0111] A wind-solar-hydrogen coupled energy supply system model is designed to describe the energy pattern in areas with limited energy transmission. In the wind-solar-hydrogen coupled energy supply system model, a renewable energy power system is composed of a wind turbine energy model and a photovoltaic array energy model. In order to alleviate the problem of wind and solar abandonment caused by the intermittent and random nature of wind and solar power, a hydrogen energy system model including an electrolyzer, a hydrogen storage tank and a fuel cell is built. A high-pressure hydrogen storage tank is used to store hydrogen and it is assumed to be a lossless closed system. Wind turbines, photovoltaic arrays and electrolyzers are coupled. When wind and solar resources are abundant, wind turbines and photovoltaic arrays work together to stably transmit electricity to the external system. Excess electricity is converted into hydrogen by electrolyzing water to produce hydrogen for storage. On the contrary, when wind and solar resources are insufficient to support load demand, the previously stored hydrogen will be converted into electricity through fuel cell technology to provide necessary supplementary electricity for the external system, ensuring the stability and continuity of power supply.

[0112] The energy model of wind turbines is as follows:

[0113]

[0114] In the above formula, P w , P wr are the theoretical output power and rated power of the wind turbine respectively; v, v w are the actual wind speed and rated wind speed respectively; v in 、v out They are cut-in wind speed and cut-out wind speed respectively;

[0115] The photovoltaic array energy model is as follows:

[0116]

[0117] In the above formula, P pv , P pr are the theoretical output power and rated power of the photovoltaic array respectively; α is the rated conversion efficiency of the photovoltaic panel; G and G0 are the actual light radiation intensity and the reference light radiation intensity respectively; k p is the temperature power coefficient; T pv , T pr They are the actual temperature and rated temperature of the photovoltaic panel respectively;

[0118] Hydrogen energy system model:

[0119]

[0120] In the above formula, are the hydrogen production of the electrolyzer and the fuel cell respectively; P ec , P pc are the input power of the electrolyzer and the output power of the fuel cell respectively; η ec , η pc are the hydrogen production efficiency of the electrolyzer and the operating efficiency of the fuel cell respectively; n hst (t), n hst (t0) are the hydrogen storage capacity of the hydrogen tank in time period t and time period t0, respectively, t=t0+Δt, Δt is the unit time interval; is the net hydrogen storage rate of the hydrogen storage tank in time period t; dt represents the differential of t; h ec The power consumption of the electrolyzer to produce unit volume of hydrogen; μ pc The amount of hydrogen consumed to generate a unit of electricity for the fuel cell; are the hydrogen storage rate and hydrogen consumption rate of the hydrogen storage tank in time period t respectively;

[0121] S2. Constructing a mobile energy storage vehicle-road resistance model

[0122] The traffic network model is the basis for studying energy transmission in energy transmission restricted areas. Based on the real-time environmental information of energy transmission restricted areas, a dynamic road network model G = (V, E, T, W) is established by using graph theory analysis method, where V is the set of all traffic nodes; E is the set of road sections in the traffic network; T is the time period of a day, and in this embodiment, the whole day is divided into 24 time periods; W is the set of road section weights, i.e., road resistance, which is used to characterize the impact of environmental disturbances on the transportation of energy by mobile energy storage vehicles; the road resistance is modeled and described as follows:

[0123] w ij,t =E ij (t);

[0124]

[0125] P mes,c =F e v ts ;

[0126] F e =F f +F w +F r +F a +F b ;

[0127] In the above formula, w ij (t) represents the impact value of the environmental disturbance on the mobile energy storage vehicle in time period t; E ij (t) is the energy consumption of the mobile energy storage vehicle to overcome environmental disturbances in time period t; C ij,tP is the number of mobile energy storage vehicles dispatched from the i-th power supplier to the j-th power user in time period t; mes,c is the driving power of the cth mobile energy storage vehicle; F e Indicates the traction force of the mobile energy storage vehicle; v ts is the actual driving speed of the mobile energy storage vehicle; F f 、F w 、F r 、F a 、F b They are the rolling resistance, air resistance, transmission resistance, slope resistance, and vehicle acceleration resistance of the mobile energy storage vehicle, which can be expressed as follows:

[0128] F f =m mes gf;

[0129]

[0130] F r =F r0 +uF ru ;

[0131] F a =m mes gu;

[0132] F b =m mes b;

[0133] In the above formula, m mes is the mass of the car; g is the acceleration of gravity; f is the rolling resistance coefficient; W is the air resistance coefficient; H is the windward area; ρ is the air density; F r0 is the actual driving speed v of the mobile energy storage vehicle ts The relevant transmission system resistance coefficient; u is the slope of the road; F ru is the transmission system resistance coefficient related to speed; b is the acceleration; v c is the relative speed, v in the absence of wind c is the driving speed of the mobile energy storage vehicle; when there is wind, v c The calculation involves the actual driving speed v of the mobile energy storage vehicle ts and wind speed v air Specifically, if the wind direction is the same as the direction of travel of the mobile energy storage vehicle, then v c =v ts +v air ; If the wind direction is opposite to the direction of the mobile energy storage vehicle, then v c =v ts -v air ; In practical applications, it is also necessary to consider the angle ι between the wind direction and the driving direction of the mobile energy storage vehicle, that is, Where cosι is the cosine value of the angle between the mobile energy storage vehicle speed vector and the wind speed vector;

[0134] S3. Construct an energy management model for power suppliers and users in energy transmission restricted areas based on the wind-solar-hydrogen coupled energy supply system model and the mobile energy storage vehicle-road resistance model.

[0135] The optimal scheduling objective of the power supplier-power user energy management model is to minimize the power supplier energy management objective function and the power user energy management objective function. The power supplier energy management objective function is to minimize the transportation cost and the amount of renewable energy discarded on the basis of meeting the load demand of the power user. Specifically,

[0136]

[0137] E w,i,t +E pv,i,t =P w t+P pv tC ij,t E mes ;

[0138]

[0139] In the above formula, F g represents the energy management objective function of the power supplier; λ1 and λ2 are penalty factors; E w,i,t 、E pv,i,t , are the wind power, solar power and hydrogen abandonment of the power supplier in time period t; d ij is the travel distance of the mobile energy storage vehicle from the i-th power supplier to the j-th power user; E mes Storing energy for mobile energy storage vehicles; C ij,t is the number of mobile energy storage vehicles dispatched from the i-th power supplier to the j-th power user in time period t. The mobile energy storage vehicles transmit discrete energy in the form of loaded energy packages; V mes 、V non are the loading space volume and non-energy storage space volume of the mobile energy storage vehicle respectively; V ep 、E ep are the volume and storage capacity of a single energy pack respectively; P w , P pv are the theoretical output powers of wind turbines and photovoltaic arrays respectively; is the hydrogen production of the electrolyzer; n hst (t) is the hydrogen storage capacity of the hydrogen tank in time period t; N and M are the total number of power suppliers and power users respectively; T is 24h;

[0140] The energy management objective function of the power user is to minimize the controllable load removed to maintain the balance between supply and demand, specifically:

[0141]

[0142] E cut,j,t =E j,t -I j,t E mes ;

[0143] E i,t =P w t+P pv t;

[0144] E j,t =P equip t;

[0145] In the above formula, F u represents the energy management objective function of the electricity user; λ3 is the penalty factor for load shedding; p c E is the contribution of the i-th power supplier to the energy demand of the j-th power user; cut,j,t is the controllable load removed by the jth electricity user in time period t; E i,t 、E j,t are the power that the i-th power supplier can supply and the power demand of the j-th power user in time period t; P equip I is the operating power of the electricity user’s electrical equipment; j,t is the total number of mobile energy storage vehicles received by the jth electricity user from each power supplier in time period t, I j,t can be described as:

[0146]

[0147] S2. Solve the distributed energy management model based on multi-agent reinforcement learning method and obtain the dispatching plan of mobile energy storage vehicles

[0148] The present invention relies on a multi-agent system with centralized training and distributed execution to efficiently handle the energy optimization scheduling problem in areas with limited energy transmission. When solving the problem, the optimal scheduling target of the power supplier-power user energy management model is converted into a shared reward function maximization form of the multi-agent system, and the shared reward function is:

[0149]

[0150] Φ=β1F g +β2F u ;

[0151] In the above formula, r n represents the shared reward function value of agent n; is the adjustment coefficient; Φ represents the intermediate calculation amount; β1 and β2 are both adjustment weights;

[0152] The traditional shared reward function provides a unified reward for all agents, regardless of the personalized needs and goals of each agent. The shared reward function designed in the present invention provides a unified reward for all agents, regardless of the personalized needs and goals of each agent. The shared reward function designed in the present invention provides a reasonable reward based on the distance matrix between the power supplier and the power user, the amount of renewable energy waste and the degree of contribution, while the optimization goals of each agent are consistent, and can meet the personalized needs of the agent.

[0153] The specific solution process is as follows:

[0154] S21. Build a deep reinforcement learning algorithm structure with centralized training and distributed execution based on the Actor-Critic framework, construct a partially observable Markov decision process, regard each power supplier as an independent intelligent agent, and each intelligent agent includes a policy network and a value network; define the multi-tuple POMDP = (N, S, A, O, R, P) to represent the energy scheduling process in the energy transmission restricted area, where N is the number of power suppliers, that is, the number of intelligent agents, and S = {s 1 ,…,s i ,…,s N},s i represents the global state space of the environment, including the state, action and interaction relationship of agent i with other agents, O = {o 1 ,…,o i ,…,o N},o i represents the local observation space of the environment by agent i; A = {a 1 ,…,a i ,…,a N}, a i represents the action of agent i; the joint action A of all agents acts on the environment, and the environment moves from the current state to the next state and obtains a shared reward;

[0155] S22, centralized training phase: Agent i obtains local observations o from the environment i = {P equip ,P w ,P pv ,P ec ,P pc} as the input of the policy network, which i Make an action i ={C ij,t} and get shared rewards i After all agents complete their actions, the state information, action information, and shared rewards of each agent are input into the value network of each agent. The value network calculates the joint advantage function, which is designed using global information. The specific calculation formula is:

[0156]

[0157] In the above formula, is the joint advantage function; are the time series difference errors at time t, time t+1, and time T-1 respectively; γ is the discount rate; φ is the smoothing coefficient; For agent n in strategy Next, from state s t+1 Execute action a t+1 The expected return that can be obtained in the future; For agent n in strategy Next, from state s t Start executing action a t The expected return that can be obtained in the future; represents the shared reward obtained by agent n at time t; represents the old network parameters of the value network;

[0158] The loss functions of the value network and the policy network are calculated based on the joint advantage function, and the network parameters of the value network and the policy network are updated by minimizing the loss functions of the value network and the policy network; the expression of the loss function of the value network is:

[0159]

[0160] In the above formula, Represents the loss function of the value network; For agent n in strategy Next, from state s t+1 Start executing action a t+1 The expected return that can be obtained in the future; For agent n in strategy Next, from state s t Start executing action a t The expected return that can be obtained in the future; Represents the current network parameters of the value network;

[0161] The loss function of the policy network is expressed as:

[0162]

[0163] In the above formula, loss(θ n ) represents the loss function of the policy network; χ(θ n ) is the ratio of the current strategy of the policy network to the old strategy; is the current policy of the policy network; is the old policy of the policy network; clip represents the clipping function; ε represents the clipping parameter; θ n Represents the current network parameters of the policy network; Represents the old network parameters of the policy network.

[0164] Decentralized execution phase: Each agent inputs local observations into its own policy network to make decisions and ultimately outputs a scheduling plan for the mobile energy storage vehicle.

[0165] Embodiment 2:

[0166] See also Figure 2 , an intelligent optimization and dispatching system for a power system in an energy transmission-constrained environment, the optimization and dispatching system comprising a model building module and a model solving module; the model building module is used to build a wind-solar-hydrogen coupled energy supply system model and a mobile energy storage vehicle-road resistance model in an energy transmission-constrained area, and to build a power supplier-power user energy management model in an energy transmission-constrained area based on the wind-solar-hydrogen coupled energy supply system model and the mobile energy storage vehicle-road resistance model; the wind-solar-hydrogen coupled energy supply system model comprises:

[0167] Wind turbine energy model:

[0168]

[0169] In the above formula, P w , P wr are the theoretical output power and rated power of the wind turbine respectively; v, v w are the actual wind speed and rated wind speed respectively; v in 、v out They are cut-in wind speed and cut-out wind speed respectively;

[0170] Photovoltaic array energy model:

[0171]

[0172] In the above formula, P pv , P pr are the theoretical output power and rated power of the photovoltaic array respectively; α is the rated conversion efficiency of the photovoltaic panel; G and G0 are the actual light radiation intensity and the reference light radiation intensity respectively; k p is the temperature power coefficient; T pv , T pr They are the actual temperature and rated temperature of the photovoltaic panel respectively;

[0173] Hydrogen energy system model:

[0174]

[0175] In the above formula, are the hydrogen production of the electrolyzer and the fuel cell respectively; P ec , P pcare the input power of the electrolyzer and the output power of the fuel cell respectively; η ec , η pc are the hydrogen production efficiency of the electrolyzer and the operating efficiency of the fuel cell respectively; n hst (t), n hst (t0) are the hydrogen storage capacity of the hydrogen tank in time period t and time period t0, respectively, t=t0+Δt, Δt is the unit time interval; is the net hydrogen storage rate of the hydrogen storage tank in time period t; dt represents the differential of t; h ec The power consumption of the electrolyzer to produce unit volume of hydrogen; μ pc The amount of hydrogen consumed to generate a unit of electricity for the fuel cell; are the hydrogen storage rate and hydrogen consumption rate of the hydrogen storage tank in time period t respectively;

[0176] The mobile energy storage vehicle-road resistance model includes:

[0177] w ij,t =E ij (t);

[0178]

[0179] P mes,c =F e v ts ;

[0180] F e =F f +F w +F r +F a +F b ;

[0181] In the above formula, w ij (t) represents the impact value of the environmental disturbance on the mobile energy storage vehicle in time period t; E ij (t) is the energy consumption of the mobile energy storage vehicle to overcome environmental disturbances in time period t; C ij,t P is the number of mobile energy storage vehicles dispatched from the i-th power supplier to the j-th power user in time period t; mes,c is the driving power of the cth mobile energy storage vehicle; F e Indicates the traction force of the mobile energy storage vehicle; v ts is the actual driving speed of the mobile energy storage vehicle; F f 、F w 、F r 、F a 、F b They are rolling resistance, air resistance, transmission system resistance, slope resistance, and vehicle acceleration resistance of the mobile energy storage vehicle;

[0182] The optimal scheduling objective of the power supplier-power user energy management model is to minimize the power supplier energy management objective function and the power user energy management objective function. The power supplier energy management objective function is to minimize the transportation cost and the amount of renewable energy discarded on the basis of meeting the load demand of the power user. Specifically,

[0183]

[0184] E w,i,t +E pv,i,t =P w t+P pv tC ij,t E mes ;

[0185]

[0186] In the above formula, F g represents the energy management objective function of the power supplier; λ1 and λ2 are penalty factors; E w,i,t 、E pv,i,t , are the wind power, solar power and hydrogen abandonment of the power supplier in time period t; d ij is the travel distance of the mobile energy storage vehicle from the i-th power supplier to the j-th power user; E mes Storing energy for mobile energy storage vehicles; C ij,t is the number of mobile energy storage vehicles dispatched from the i-th power supplier to the j-th power user in time period t. The mobile energy storage vehicles transmit discrete energy in the form of loaded energy packages; V mes 、V non are the loading space volume and non-energy storage space volume of the mobile energy storage vehicle respectively; V ep 、E ep are the volume and storage capacity of a single energy pack respectively; P w , P pv are the theoretical output powers of wind turbines and photovoltaic arrays respectively; is the hydrogen production of the electrolyzer; n hst (t) is the hydrogen storage capacity of the hydrogen tank in time period t; N and M are the total number of power suppliers and power users respectively; T is the total cycle;

[0187] The energy management objective function of the power user is to minimize the controllable load removed to maintain the balance between supply and demand, specifically:

[0188]

[0189] E cut，j，t =E j,t -I j,t E mes ;

[0190]

[0191] E i,t =P w t+P pv t;

[0192] E j,t =P equip t;

[0193] In the above formula, F u represents the energy management objective function of the electricity user; λ3 is the penalty factor for load shedding; p c E is the contribution of the i-th power supplier to the energy demand of the j-th power user; cut,j,t is the controllable load removed by the jth electricity user in time period t; E i,t 、E j,t are the power that the i-th power supplier can supply and the power demand of the j-th power user in time period t; P equip I is the operating power of the electricity user’s electrical equipment; j,t is the total number of mobile energy storage vehicles received by the jth electricity user from each power supplier in time period t;

[0194] The model solving step is used to solve the distributed energy management model based on the multi-agent reinforcement learning method to obtain a scheduling plan for the mobile energy storage vehicle; the solving steps for solving the distributed energy management model based on the multi-agent reinforcement learning method are:

[0195] S21. Construct a partially observable Markov decision process, design a shared reward function, and transform the optimal scheduling objective of the power supplier-user energy management model into a shared reward function maximization form of the multi-agent system; the shared reward function is:

[0196]

[0197] Φ=β1F g +β2F u ;

[0198] In the above formula, r n represents the shared reward function value of agent n; is the adjustment coefficient; Φ represents the intermediate calculation amount; β1 and β2 are both adjustment weights;

[0199] Each power supplier is regarded as an independent intelligent agent, and a multi-tuple POMDP = (N, S, A, O, R, P) is defined to represent the energy scheduling process in the energy transmission restricted area, where N is the number of power suppliers, i.e. the number of intelligent agents, and S = {s 1 ,…,s i ,…s, N},s irepresents the global state space of the environment, including the state, action and interaction relationship of agent i with other agents, O = {o 1 ,…,o i ,…,o N},o i represents the local observation space of the environment by agent i; A = {a 1 ,…,a i ,…,a N}, a i represents the action of agent i; the joint action A of all agents acts on the environment, and the environment moves from the current state to the next state and obtains a shared reward;

[0200] S22, centralized training phase: Agent i obtains local observations o from the environment i = {P equip ,P w ,P pv ,P ec ,P pc} as the input of the policy network, which i Make an action i ={C ij,t} and get shared rewards i After all agents complete their actions, the state information, action information, and shared rewards of each agent are input into the value network of each agent. The value network calculates the joint advantage function, which is designed using global information. The specific calculation formula is:

[0201]

[0202] In the above formula, is the joint advantage function; are the time series difference errors at time t, time t+1, and time T-1 respectively; γ is the discount rate; φ is the smoothing coefficient; For agent n in strategy Next, from state s t+1 Execute action a t+1 The expected return that can be obtained in the future; For agent n in strategy Next, from state s t Start executing action a t The expected return that can be obtained in the future; represents the shared reward obtained by agent n at time t; represents the old network parameters of the value network;

[0203] The loss functions of the value network and the policy network are calculated based on the joint advantage function, and the network parameters of the value network and the policy network are updated by minimizing the loss functions of the value network and the policy network; the expression of the loss function of the value network is:

[0204]

[0205] In the above formula, Represents the loss function of the value network; For agent n in strategy Next, from state s t+1 Start executing action a t+1 The expected return that can be obtained in the future; For agent n in strategy Next, from state s t Start executing action a t The expected return that can be obtained in the future; Represents the current network parameters of the value network;

[0206] The loss function of the policy network is expressed as:

[0207]

[0208] In the above formula, loss(θ n ) represents the loss function of the policy network; χ(θ n ) is the ratio of the current strategy of the policy network to the old strategy; is the current policy of the policy network; is the old policy of the policy network; clip represents the clipping function; ε represents the clipping parameter; θ n Represents the current network parameters of the policy network; represents the old network parameters of the policy network;

[0209] Decentralized execution phase: Each agent inputs local observations into its own policy network to make decisions and ultimately outputs a scheduling plan for the mobile energy storage vehicle.

[0210] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0211] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0212] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0213] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0214] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. An intelligent optimization dispatching method for a power system under an energy transmission-constrained environment, characterized by: The optimization scheduling method comprises the following steps: S1. Construct a wind-solar-hydrogen coupled energy supply system model and a mobile energy storage vehicle-road resistance model in areas with restricted energy transmission; construct a power supplier-power user energy management model in areas with restricted energy transmission based on the wind-solar-hydrogen coupled energy supply system model and the mobile energy storage vehicle-road resistance model; the optimal scheduling goal of the power supplier-power user energy management model is to minimize the power supplier energy management objective function and the power user energy management objective function, wherein the power supplier energy management objective function is to minimize the transportation cost and the amount of renewable energy discarded on the basis of meeting the load demand of the power user; the power user energy management objective function is to minimize the amount of controllable load removed to maintain the balance between supply and demand; S2. Convert the optimal scheduling objective of the power supplier-user energy management model into the form of maximizing the shared reward function of the multi-agent system, solve the distributed energy management model based on the multi-agent reinforcement learning method, and obtain the scheduling plan for the mobile energy storage vehicle.

2. The method for intelligent optimization and dispatching of a power system under an energy transmission-constrained environment according to claim 1 is characterized in that: The energy management objective function of the power supplier is: E w,i,t +E pv,i,t =P w t+P pv t-C ij,t E mes ; In the above formula, F g represents the energy management objective function of the power supplier; λ1 and λ2 are penalty factors; E w,i,t 、E pv,i,t , are the wind power, solar power and hydrogen abandonment of the power supplier in time period t; d ij is the travel distance of the mobile energy storage vehicle from the i-th power supplier to the j-th power user; E mes Storing energy for mobile energy storage vehicles; C ij,t is the number of mobile energy storage vehicles dispatched from the i-th power supplier to the j-th power user in time period t. The mobile energy storage vehicles transmit discrete energy in the form of loaded energy packages; V mes 、V non are the loading space volume and non-energy storage space volume of the mobile energy storage vehicle respectively; V ep 、E ep are the volume and storage capacity of a single energy pack respectively; P w , P pv are the theoretical output powers of wind turbines and photovoltaic arrays respectively; is the hydrogen production of the electrolyzer; n hst (t) is the hydrogen storage capacity of the hydrogen tank in time period t; N and M are the total number of power suppliers and power users respectively; T is the total cycle; The energy management objective function of the electricity user is: E cut,j,t =E j,t -I j,t E mes ; E i,t =P w t+P pv t; E j,t =P equip t; In the above formula, F u represents the energy management objective function of the electricity user; λ3 is the penalty factor for load shedding; p c E is the contribution of the i-th power supplier to the energy demand of the j-th power user; cut,j,t is the controllable load removed by the jth electricity user in time period t; E i,t 、E j,t are the power that the i-th power supplier can supply and the power demand of the j-th power user in time period t; P equip I is the operating power of the electricity user’s electrical equipment; j,t is the total number of mobile energy storage vehicles received by the jth electricity user from each power supplier in time period t; The shared reward function is: r n =-lΦ; Φ=β1F g +β2F u ; In the above formula, r n represents the shared reward function value of agent n; l is the adjustment coefficient; Φ represents the intermediate calculation amount; β1 and β2 are both adjustment weights.

3. The method for intelligent optimization and dispatching of a power system under an energy transmission-constrained environment according to claim 2 is characterized in that: The wind-solar-hydrogen coupled energy supply system model includes: Wind turbine energy model: In the above formula, P w , P wr are the theoretical output power and rated power of the wind turbine respectively; v, v w are the actual wind speed and rated wind speed respectively; v in 、v out They are cut-in wind speed and cut-out wind speed respectively; Photovoltaic array energy model: In the above formula, P pv , P pr are the theoretical output power and rated power of the photovoltaic array respectively; α is the rated conversion efficiency of the photovoltaic panel; G and G0 are the actual light radiation intensity and the reference light radiation intensity respectively; k p is the temperature power coefficient; T pv , T pr They are the actual temperature and rated temperature of the photovoltaic panel respectively; Hydrogen energy system model: In the above formula, are the hydrogen production of the electrolyzer and the fuel cell respectively; P ec , P pc are the input power of the electrolyzer and the output power of the fuel cell respectively; η ec , η pc are the hydrogen production efficiency of the electrolyzer and the operating efficiency of the fuel cell; n hst (t), n hst (t0) are the hydrogen storage capacity of the hydrogen tank in time period t and time period t0, respectively, t=t0+Δt, Δt is the unit time interval; is the net hydrogen storage rate of the hydrogen storage tank in time period t; dt represents the differential of t; h ec The power consumption of the electrolyzer to produce unit volume of hydrogen; μ pc The amount of hydrogen consumed to generate a unit of electricity for the fuel cell; are the hydrogen storage rate and hydrogen consumption rate of the hydrogen storage tank in time period t respectively; The mobile energy storage vehicle-road resistance model includes: w ij,t =E ij (t); P mes,c =F e v ts ; F e =F f +F w +F r +F a +F b ; In the above formula, w ij (t) represents the impact value of the environmental disturbance on the mobile energy storage vehicle in time period t; E ij (t) is the energy consumption of the mobile energy storage vehicle to overcome environmental disturbances in time period t; C ij,t P is the number of mobile energy storage vehicles dispatched from the i-th power supplier to the j-th power user in time period t; mes,c is the driving power of the cth mobile energy storage vehicle; F e Indicates the traction force of the mobile energy storage vehicle; v ts is the actual driving speed of the mobile energy storage vehicle; F f 、F w 、F r 、F a 、F b They are respectively the rolling resistance, air resistance, transmission system resistance, slope resistance and vehicle acceleration resistance of the mobile energy storage vehicle.

4. The method for intelligent optimization and dispatching of a power system under an energy transmission-constrained environment according to claim 1 is characterized in that: The step S2 is specifically as follows: S21. Construct a partially observable Markov decision process, treat each power supplier as an independent intelligent agent, each of which includes a strategy network and a value network; define a multi-tuple POMDP = (N, S, A, O, R, P) to represent the energy scheduling process in the energy transmission restricted area, where N is the number of power suppliers, i.e. the number of agents, and S = {s 1 ,…,s i ,…s, N },s i represents the global state space of the environment, including the state, action and interaction relationship of agent i with other agents, O = {o 1 ,…,o i ,…,o N },o i represents the local observation space of the environment by agent i; A = {a 1 ,…,ai,…,a N }, a i represents the action of agent i; S22, centralized training phase, agent i obtains local observations o from the environment i = {P equip ,P w ,P pv ,P ec ,P pc } as the input of the policy network, which i Make an action i ={C ij,t } and get shared rewards i ; After all agents complete their actions, the state information, action information, and shared rewards of each agent are input into the value network of each agent. The value network calculates the joint advantage function, and the loss function of the value network and the policy network is calculated based on the joint advantage function. The network parameters of the value network and the policy network are updated by minimizing the loss function of the value network and the policy network. In the decentralized execution stage, each agent inputs local observations into its own policy network to make decisions.

5. The method for intelligent optimization and dispatching of power systems under energy transmission-constrained environments according to claim 4 is characterized in that: The calculation formula of the joint advantage function is: In the above formula, is the joint advantage function; are the time series difference errors at time t, time t+1, and time T-1 respectively; γ is the discount rate; φ is the smoothing coefficient; For agent n in strategy Next, from state s t+1 Execute action a t+1 The expected return that can be obtained in the future; For agent n in strategy Next, from state s t Start executing action a t The expected return that can be obtained in the future; represents the shared reward obtained by agent n at time t; represents the old network parameters of the value network; The expression of the loss function of the value network is: In the above formula, Represents the loss function of the value network; For agent n in strategy Next, from state s t+1 Start executing action a t+1 The expected return that can be obtained in the future; For agent n in strategy Next, from state s t Start executing action a t The expected return that can be obtained in the future; Represents the current network parameters of the value network; The loss function of the policy network is expressed as: In the above formula, loss(θ n ) represents the loss function of the policy network; χ(θ n ) is the ratio of the current strategy of the policy network to the old strategy; is the current policy of the policy network; is the old policy of the policy network; clip represents the clipping function; ε represents the clipping parameter; θ n Represents the current network parameters of the policy network; Represents the old network parameters of the policy network.

6. Intelligent optimization dispatching system for power system under energy transmission restricted environment, characterized by: The optimization scheduling system includes a model building module and a model solving module; The model building module is used to build a wind-solar-hydrogen coupled energy supply system model and a mobile energy storage vehicle-road resistance model in an area with restricted energy transmission, and to build a power supplier-power user energy management model in an area with restricted energy transmission based on the wind-solar-hydrogen coupled energy supply system model and the mobile energy storage vehicle-road resistance model; the optimal scheduling goal of the power supplier-power user energy management model is to minimize the power supplier energy management objective function and the power user energy management objective function, and the power supplier energy management objective function is to minimize the transportation cost and the amount of renewable energy discarded on the basis of meeting the load demand of the power user; the power user energy management objective function is to minimize the controllable load removed to maintain the balance between supply and demand; The model solving module is used to convert the optimal scheduling target of the power supplier-power user energy management model into the maximization form of the shared reward function of the multi-agent system, solve the distributed energy management model based on the multi-agent reinforcement learning method, and obtain the scheduling plan of the mobile energy storage vehicle.

7. The intelligent optimization and dispatching system for power systems in an energy transmission-constrained environment according to claim 6 is characterized in that: The energy management objective function of the power supplier is: E w,i,t +E pv,i,t =P w t+P pv t-C ij,t E mes ; In the above formula, F g represents the energy management objective function of the power supplier; λ1 and λ2 are penalty factors; E w,i,t 、E pv,i,t , are the wind power, solar power and hydrogen abandonment of the power supplier in time period t; d ij is the travel distance of the mobile energy storage vehicle from the i-th power supplier to the j-th power user; E mes Storing energy for mobile energy storage vehicles; C ij,t is the number of mobile energy storage vehicles dispatched from the i-th power supplier to the j-th power user in time period t. The mobile energy storage vehicles transmit discrete energy in the form of loaded energy packages; V mes 、V non are the loading space volume and non-energy storage space volume of the mobile energy storage vehicle respectively; V ep 、E ep are the volume and storage capacity of a single energy pack respectively; P w , P pv are the theoretical output powers of wind turbines and photovoltaic arrays respectively; is the hydrogen production of the electrolyzer; n hst (t) is the hydrogen storage capacity of the hydrogen tank in time period t; N and M are the total number of power suppliers and power users respectively; T is the total cycle; The energy management objective function of the electricity user is: E cut,j,t =E j,t -I j,t E mes ; E i,t =P w t+P pv t; E j,t =P equip t; In the above formula, F u represents the energy management objective function of the electricity user; λ3 is the penalty factor for load shedding; p c E is the contribution of the i-th power supplier to the energy demand of the j-th power user; cut,j,t is the controllable load removed by the jth electricity user in time period t; E i,t 、E j,t are the power that the i-th power supplier can supply and the power demand of the j-th power user in time period t; P equip I is the operating power of the electricity user’s electrical equipment; j,t is the total number of mobile energy storage vehicles received by the jth electricity user from each power supplier in time period t; The shared reward function is: r n =-lΦ; Φ=β1F g +β2F u ; In the above formula, r n represents the shared reward function value of agent n; l is the adjustment coefficient; Φ represents the intermediate calculation amount; β1 and β2 are both adjustment weights.

8. The intelligent optimization and dispatching system for power systems in an energy transmission-constrained environment according to claim 7 is characterized in that: The wind-solar-hydrogen coupled energy supply system model includes: Wind turbine energy model: In the above formula, P w , P wr are the theoretical output power and rated power of the wind turbine respectively; v, v w are the actual wind speed and rated wind speed respectively; v in 、v out They are cut-in wind speed and cut-out wind speed respectively; Photovoltaic array energy model: In the above formula, P pv , P pr are the theoretical output power and rated power of the photovoltaic array respectively; α is the rated conversion efficiency of the photovoltaic panel; G and G0 are the actual light radiation intensity and the reference light radiation intensity respectively; k p is the temperature power coefficient; T pv , T pr They are the actual temperature and rated temperature of the photovoltaic panel respectively; Hydrogen energy system model: In the above formula, are the hydrogen production of the electrolyzer and the fuel cell respectively; P ec , P pc are the input power of the electrolyzer and the output power of the fuel cell respectively; η ec , η pc are the hydrogen production efficiency of the electrolyzer and the operating efficiency of the fuel cell; n hst (t), n hst (t0) are the hydrogen storage capacity of the hydrogen tank in time period t and time period t0, respectively, t=t0+Δt, Δt is the unit time interval; is the net hydrogen storage rate of the hydrogen storage tank; dτ is Differentiate the parameter τ; h ec The power consumption of the electrolyzer to produce unit volume of hydrogen; μ pc The amount of hydrogen consumed to generate a unit of electricity for the fuel cell; are the hydrogen storage rate and hydrogen consumption rate of the hydrogen storage tank respectively; The mobile energy storage vehicle-road resistance model includes: w ij,t =E ij (t); P mes,c =F e v ts ; F e =F f +F w +F r +F a +F b ; In the above formula, w ij (t) represents the impact value of the environmental disturbance on the mobile energy storage vehicle in time period t; E ij (t) is the energy consumption of the mobile energy storage vehicle to overcome environmental disturbances in time period t; C ij,t P is the number of mobile energy storage vehicles dispatched from the i-th power supplier to the j-th power user in time period t; mes,c is the driving power of the cth mobile energy storage vehicle; F e Indicates the traction force of the mobile energy storage vehicle; v ts is the actual driving speed of the mobile energy storage vehicle; F f 、F w 、F r 、F a 、F b They are respectively the rolling resistance, air resistance, transmission system resistance, slope resistance and vehicle acceleration resistance of the mobile energy storage vehicle.

9. The intelligent optimization and dispatching system for power systems in an energy transmission-constrained environment according to claim 6 is characterized in that: The model solving step is used to solve the distributed energy management model according to the following steps to obtain a scheduling plan for the mobile energy storage vehicle: S21. Construct a partially observable Markov decision process, treat each power supplier as an independent agent, each agent includes a policy network and a value network; define a multi-tuple POMDP = (N, S, A, O, R, P) to represent the energy scheduling process in the energy transmission restricted area, N is the number of power suppliers, that is, the number of agents, s i represents the global state space of the environment, including the state, action and interaction relationship of agent i with other agents, O = {o 1 ,…,o i ,…,o N },o i represents the local observation space of the environment by agent i; A = {a 1 ,…,a i ,…,a N }, ai represents the action of agent i; S22, centralized training phase, agent i obtains local observations o from the environment i = {P equip ,P w ,P pv ,P ec ,P pc } as the input of the policy network, which i Make an action i ={C ij,t } and get shared rewards i ; After all agents complete their actions, the state information, action information, and shared rewards of each agent are input into the value network of each agent. The value network calculates the joint advantage function, and the loss function of the value network and the policy network is calculated based on the joint advantage function. The network parameters of the value network and the policy network are updated by minimizing the loss function of the value network and the policy network. In the decentralized execution stage, each agent inputs local observations into its own policy network to make decisions.

10. The intelligent optimization and dispatching system for power systems in an energy transmission-constrained environment according to claim 9, characterized in that: The calculation formula of the joint advantage function is: In the above formula, is the joint advantage function; are the time series difference errors at time t, time t+1, and time T-1 respectively; γ is the discount rate; φ is the smoothing coefficient; For agent n in strategy Next, from state s t+1 Execute action a t+1 The expected return that can be obtained in the future; For agent n in strategy Next, from state s t Start executing action a t The expected return that can be obtained in the future; represents the shared reward obtained by agent n at time t; represents the old network parameters of the value network; The expression of the loss function of the value network is: In the above formula, Represents the loss function of the value network; For agent n in strategy Next, from state s t+1 Start executing action a t+1 The expected return that can be obtained in the future; For agent n in strategy Next, from state s t Start executing action a t The expected return that can be obtained in the future; Represents the current network parameters of the value network; The loss function of the policy network is expressed as: In the above formula, loss(θ n ) represents the loss function of the policy network; χ(θ n ) is the ratio of the current strategy of the policy network to the old strategy; is the current policy of the policy network; is the old policy of the policy network; clip represents the clipping function; ε represents the clipping parameter; θ n Represents the current network parameters of the policy network; Represents the old network parameters of the policy network.

Images