Comprehensive energy system optimization method for sharing energy storage and wind and light output uncertainty

Abstract

According to the comprehensive energy system optimization method for the shared energy storage and wind and light output uncertainty, the shared energy storage mathematical model is established, the investment cost at the earlier stage of energy storage is reduced, meanwhile, the flexibility of the energy storage system is improved, and efficient utilization of resources and rapid recovery of the energy storage investment cost are achieved. Secondly, considering the uncertainty of wind power and photovoltaic output to establish a wind power and photovoltaic output uncertainty model; and finally, by taking the lowest total investment and operation cost of the system as an optimization target and considering factors such as one-time investment cost, operation and maintenance cost, life cycle, renewable energy consumption rate, wind curtailment and light curtailment penalty cost of the shared energy storage, establishing a comprehensive energy system optimization model considering the shared energy storage and wind and light output uncertainty, and solving, thereby reducing the investment cost. And high-proportion consumption of renewable energy sources and efficient utilization of resources are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of integrated energy systems, and in particular to an optimization method for an integrated energy system with shared energy storage and wind and solar power output uncertainty. Background Art

[0002] Under the background of the vigorous development of "dual carbon" and "integrated energy system", as a large number of new energy sources are continuously connected to the power system, the capacity configuration, dispatch optimization and benefit distribution of energy storage participating in the integrated energy system have become very complicated. Shared energy storage has comprehensively improved the economic affordability of energy storage in the integrated energy system by realizing the sharing of idle and abandoned energy storage resources. However, there are great uncertainties in the power generation of distributed energy on the supply side of the integrated energy system and the energy load demand such as gas, electricity, cold and heat. There are extremely complex coupling characteristics between different energy sources. These uncertainties and complexities have brought certain risks to the stable and effective operation of shared energy storage participating in the integrated energy system. In addition, the access and configuration of excessive distributed new energy equipment will make it impossible to absorb huge energy. Therefore, the optimized operation of the integrated energy system considering shared energy storage and new energy output has become a hot issue in current research. Summary of the invention

[0003] The present invention proposes a method for optimizing an integrated energy system with shared energy storage and wind and solar power output uncertainty, which can solve at least one of the technical problems in the background technology.

[0004] To achieve the above object, the present invention adopts the following technical solutions:

[0005] A method for optimizing an integrated energy system with shared energy storage and wind and solar power output uncertainty, characterized in that the specific steps are as follows:

[0006] S100, establishing a mathematical model for shared energy storage;

[0007] S200, combining the uncertainty of wind power and photovoltaic output, establish a wind power and photovoltaic output uncertainty model;

[0008] S300, based on the uncertainty model of wind power and photovoltaic output and the mathematical model of shared energy storage, with the minimum total system investment and operation cost as the optimization goal, taking into account factors such as the one-time investment cost, operation and maintenance cost, life cycle, renewable energy absorption rate, wind and solar abandonment penalty costs of shared energy storage, an integrated energy system optimization model considering the uncertainty of shared energy storage and wind and solar output is established and solved.

[0009] Furthermore, the method for building the shared energy storage mathematical model in S100 of the present invention is as follows:

[0010] S110, determining the research cycle time of shared energy storage;

[0011] Divide the research cycle of shared energy storage into n time periods, and the formula is:

[0012]

[0013] Where: RC is the research cycle time of shared energy storage, u t-1 and u t are the upper and lower boundary values ​​of the t period, respectively, where u t =RC, u0=0;

[0014] S120, within the research cycle time, establish a shared energy storage stage t output power model;

[0015]

[0016] Where: is the output power of the shared energy storage at stage t; Discharge efficiency for shared energy storage; is the average load power predicted by generator i participating in shared energy storage in stage t; The power purchased by power generator i participating in shared energy storage from the large power grid at stage t; is the average power generation of distributed wind power and distributed photovoltaic enterprise i on the supply side in stage t; is the maximum discharge power of the shared energy storage battery; is the minimum discharge power of the shared energy storage battery;

[0017] S130, establishing a shared energy storage charging power model in stage t that can store excess electric energy in the battery based on the shared energy storage output power model in stage t;

[0018]

[0019]

[0020] Where: is the input charging power of the large power grid to the shared energy storage at stage t; Charging power for shared energy storage batteries; The maximum charging power of the shared energy storage battery; is the minimum charging power of the shared energy storage battery;

[0021] S140, calculating a power capacity expression of a shared energy storage battery used by a UPS function according to a charging power model of the shared energy storage at stage t;

[0022] When the main power supply equipment on the supply side fails to supply power, the UPS backup battery power capacity must ensure that the core control system and emergency alarm can work simultaneously for more than 3 hours. Taking 3 hours as the UPS power supply time, the power capacity expression of the shared energy storage battery used by the UPS function is:

[0023]

[0024] Where: P i UPS The total power of the UPS equipment that the i-th generator needs to use for shared energy storage;

[0025] S150. Establishing a total energy storage device capacity model

[0026]

[0027] SOC min ≤SOC t ≤SOC max Formula (11)

[0028] SOC0=SOC uN Formula (12)

[0029] Where: S ALL is the total power stored by the shared energy storage battery, S min The lower limit of the shared energy storage battery power, indicating the minimum remaining power of the shared energy storage device, is to avoid over-discharge of the shared energy storage battery, which may shorten the battery life, and meet the requirements of UPS power backup; SOC t S is the state of charge of the shared energy storage at stage t; t is the total power stored in the shared energy storage battery in stage t; low is the duration of the valley time on the demand side; β is the charging efficiency of the shared energy storage battery; SOC min SOC is the minimum state of charge of the shared energy storage device; max It is the maximum value of the state of charge of the shared energy storage device.

[0030] Furthermore, the method for building the wind power and photovoltaic output uncertainty model in S200 of the present invention is as follows:

[0031] S210. Establishing an uncertain output model for wind power generation

[0032] Fuzzy chance constraint programming is used to describe wind power output. The fuzzy chance constraint is expressed as

[0033] M[g(x,ξ)≤0]≥αFormula (13)

[0034] Where: α is the confidence level; g is the set of constraints; ξ is the fuzzy variable; x is the decision variable; M is the measurement function;

[0035] The credibility measure of fuzzy event A is expressed as:

[0036] M(A)=C r (A) Formula (14)

[0037] And there is the following formula:

[0038]

[0039] Where: μ is the membership function of the fuzzy variable ξ, sup is the upper bound;

[0040] Credibility Measure C r It has self-duality, assuming that an event with a credibility of 1 will definitely occur, and an event with a credibility of 0 will definitely not occur, thus avoiding the decision-making confusion that may be caused by traditional membership calculation;

[0041] The uncertainty of wind turbine output is transformed into the uncertainty of output prediction error. That is, assuming that the predicted value of wind turbine output is certain data, the prediction error is modeled using fuzzy theory from a new perspective. The uncertainty of wind turbine output is indirectly described by the uncertainty of wind turbine output prediction error. Assuming that the percentage of wind turbine output prediction error is ε wind , the function expression is as follows:

[0042]

[0043] Where p′ wind 、p wind Represent the predicted output and actual output of wind power respectively; the prediction error has two forms: if the actual output is higher than the predicted output, it is a positive error; if the actual output is lower than the predicted output, it is a negative error;

[0044] Its membership μ can be expressed by Cauchy distribution wind :

[0045]

[0046] Where: represent the statistical average of positive error and negative error respectively; σ is the weight. According to formula (3), for ε wind ∈R, the credibility measure of the prediction error ξ is

[0047]

[0048] The credibility distribution function value refers to the fuzzy variable ξ not greater than ε windThe credibility of can be compared to the probability distribution function in probability theory;

[0049] S220. Establishing an output uncertainty model for photovoltaic power generation

[0050] Firstly, the historical data of PV output is statistically analyzed, a certain time period is selected, and a continuous range covering all PV output scenarios is selected within the time period, thereby generating an uncertainty set under a certain radius constraint;

[0051]

[0052] Where: is the uncertain set of photovoltaic power output; is the actual output of the photovoltaic unit in the T period; and They represent the minimum and maximum output of the photovoltaic unit in the T period respectively; T is the number of periods in a day. This paper takes 1 hour as the unit and decomposes each day into 24 periods; Λ is the robustness factor with a value range of [0, 24], which is used to control the interval range of the uncertainty set. The larger the value, the larger the range covered by the uncertainty set.

[0053] Furthermore, the method for constructing an integrated energy system optimization model considering the uncertainty of shared energy storage and wind and solar power output in S300 of the present invention is as follows:

[0054] S310, establishing the objective function of the integrated energy system optimization operation model;

[0055] S320, establish constraints for the optimal operation model of the integrated energy system;

[0056] S330. Optimize and solve the integrated energy system operation model.

[0057] Furthermore, the objective function in S310 of the present invention is specifically:

[0058]

[0059]

[0060]

[0061]

[0062] Where: F is the total operating cost of the integrated energy system, including the power generation cost of the natural gas cooling-heating-generator unit and the penalty for deviation from the planned output, the power purchase cost, the penalty for deviation from the planned power purchase, the cost of wind and solar power abandonment, and the annual investment and operation and maintenance cost of the shared energy storage system; N rMT is the number of natural gas cooling-heating-generator units in area r; is the power output of the i-th natural gas cooling-heating-generator unit in region r at time t under scenario s; is the thermal output of the i-th natural gas cooling-heating-generator unit in region r at time t under scenario s; is the purchased power of the i-th natural gas cooling-heating-generating unit in area r at time t under scenario s; is the cost function of the natural gas cooling-heating-electrical unit; It is the abandoned wind and solar power; is the power that can be generated by distributed wind power; is the power that can be generated by distributed photovoltaics; The grid-connected power of distributed wind power; is the grid-connected power of distributed photovoltaics; is the heating power of distributed wind power; is the heating power of distributed photovoltaics; is the unit price of electricity purchase; ρ1 is the penalty price for the natural gas cooling-heating-generator group deviating from the planned output; ρ2 is the penalty price for the purchased power deviating from the planned purchased power; ρ3 is the penalty price for abandoning wind and solar power; λ is the rate of return of funds; N SES The total number of waste batteries invested and collected for shared energy storage operators; is the unit power investment coefficient of shared energy storage; is the unit capacity investment coefficient of shared energy storage; is the annual maintenance cost coefficient of shared energy storage; is the operating power of shared energy storage; y is the full life cycle of shared energy storage; D is the number of typical days in a year; T d is the number of days corresponding to a typical day in a year.

[0063] Furthermore, the constraints of the model in S320 of the present invention are specifically as follows:

[0064] (1) Electric power balance constraints

[0065] The electric power output needs to meet the following constraints:

[0066]

[0067] Where: L r is the set of contact lines between area r and other areas; is the transmission power of tie line l; It indicates that the tie line l inputs power into the area at time t; It means that at time t, the tie line l transmits power outside the area; is the electrical load power; is the power of the electric refrigerator;

[0068] (2) Constraints of natural gas cooling-heating-generator units

[0069] Thermoelectric ratio constraints, output upper and lower limit constraints and ramp constraints of natural gas cooling-heating-generator units:

[0070]

[0071] Where: K hp,r,t is the heat-to-electricity ratio of the i-th natural gas cooling-heating-generating unit in region r; is the lower limit of the power output of the i-th natural gas cooling-heating-generating unit in region r; The upper limit of the power output of the i-th natural gas cooling-heating-generating unit in region r; is the maximum increase in power per unit time of the i-th natural gas cooling-heating-generating unit in area r;

[0072] (3) Power purchase power constraints

[0073] Each demand-side functional area is only allowed to purchase electricity and gas, and is not allowed to sell electricity and gas:

[0074]

[0075] Where: The capacity of the substation at the grid connection point; The power of purchased gas; Capping the capacity of natural gas pipelines;

[0076] (4) Distributed wind power and distributed photovoltaic constraints

[0077] Distributed wind power and distributed photovoltaic output need to meet the following constraints:

[0078]

[0079]

[0080] (5) Heating system constraints

[0081] The electric boiler in area r only consumes the wind power and photovoltaic power in area r, satisfying the electric-heat conversion relationship constraints of the electric boiler and the output constraints of the electric boiler:

[0082]

[0083] Where: is the heat output of the electric boiler in area r at time t under scenario s; η EB,r is the electricity conversion efficiency of the electric boiler in area r; The output limit of the electric boiler in area r;

[0084] (6) Start and stop time constraints

[0085] The device start and stop time constraints must meet the following constraints:

[0086]

[0087] Where: The continuous operation time of the natural gas cooling-heating-electricity unit; The continuous downtime of the natural gas cooling-heating-electricity unit; The continuous operation time of the electric boiler equipment; The continuous downtime of the electric boiler equipment; The minimum start-up time for natural gas cooling-heating-electricity; Minimum downtime for natural gas cooling, heating and electricity; The minimum start-up time for electric boiler equipment; is the minimum downtime of the electric boiler equipment; u MT,r,t-1 is the operating status of the natural gas cooling-heating-generator unit at time t-1, which is a 00-1 variable; u EB,r,t-1 is the operating status of the electric boiler equipment at time t-1, which is a 0-1 variable;

[0088] (7) Transport channel capacity constraints

[0089] The transport channel capacity meets the following constraints:

[0090]

[0091] Where: is the line capacity between nodes i and j at the initial moment; is the newly added line capacity between nodes i and j at time t; is the maximum capacity of the transmission line between nodes i and j; M is the penalty coefficient, so that hour, is 0, when When the , is large enough so that there is enough space to choose a new line; is a Boolean variable, a “switch” for a continuous constraint, producing a capacity greater than or equal to the maximum capacity allowed by the line upgrade;

[0092] (8) Backup constraints

[0093] The spare capacity meets the following constraints:

[0094]

[0095]

[0096] In the formula: r1 is the upper reserve coefficient of the load; r2 is the upper reserve coefficient of distributed wind power; r3 is the upper reserve coefficient of distributed photovoltaic; r4 is the lower reserve coefficient of distributed wind power; r5 is the lower reserve coefficient of distributed photovoltaic;

[0097] (9) Shared energy storage capacity configuration constraints

[0098] The shared energy storage configuration constraint is used to determine the total shared energy storage capacity invested in the integrated energy system. If the shared energy storage capacity is configured too large, the utilization rate of the shared energy storage of the entire integrated energy system will be low, and the return on investment will be low; if the shared energy storage capacity is configured too small, the optimization space for the operation and scheduling of the integrated energy system will be reduced.

[0099] 1. Shared energy storage configuration ratio constraints:

[0100] Q SES =ηP SES Formula (50)

[0101] Where: η is the energy ratio of shared energy storage, and the total rated power of shared energy storage is proportional to the total capacity;

[0102] 2. Shared energy storage charge state constraints:

[0103] SOC SES,d,t =(1-ω)SOC SES,d,t-1 Formula (51)

[0104] Where: ω is the self-discharge rate of shared energy storage; SOC SES,d,t is the state of charge of the shared energy storage at time period t on the dth typical day;

[0105] 3. Shared energy storage capacity power state constraints:

[0106]

[0107] Q SES =T SES P SES Formula (53)

[0108] Where: The maximum capacity power value allowed for shared energy storage; T SES The total time of charging and discharging shared energy storage; P SES The capacity power planned for shared energy storage; Q SES Energy capacity planned for shared energy storage.

[0109] Furthermore, the optimization solution method of the integrated energy system operation model in S330 of the present invention is as follows:

[0110] In the process of interaction between the agent and the environment, the sequence from the starting state to the end state is called trajectory χ; when the strategy parameter ω is given, the probability of each χ existing is expressed as p ω (χ), the calculation process is as follows:

[0111]

[0112] Where: p ω (m t |c t ) means in c t Select action m in state t The probability of p(c t+1 |c t ,m t ) means in state c t Apply action m t The environment state changes to c t+1 The probability of the probability is determined by the environment and cannot be controlled manually. In the continuous interaction between the agent and the environment, several trajectories χ are generated. The sum of the instant rewards corresponding to each trajectory χ is recorded as R(χ), and the expected cumulative reward is recorded as It is expressed by the following formula:

[0113]

[0114] The goal of reinforcement learning is to achieve trajectory cumulative rewards The maximization of can be solved by using the policy gradient algorithm; in the solution process, the target gradient is first required

[0115]

[0116] In the formula express Actions in status The direction in which the probability is most affected by ω; R(χ n ) represents the step size and direction of the parameter ω update. The solution process can be simply explained as when the reward R(χ n ) increases, the state Next action The probability increases, otherwise the execution of the action is inhibited.

[0117] On the other hand, the present invention further discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the above method.

[0118] On the other hand, the present invention further discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.

[0119] It can be seen from the above technical solutions that the present invention establishes a mathematical model for shared energy storage, and considers the uncertainty of wind power and photovoltaic output to establish wind power and photovoltaic output uncertainty models respectively. Taking the lowest total system investment and operation cost as the optimization goal, the one-time investment cost, operation and maintenance cost, life cycle, renewable energy absorption rate, wind abandonment, and light abandonment penalty costs of shared energy storage are considered to establish and solve the optimization model of the comprehensive energy system considering the uncertainty of shared energy storage and wind and solar output, reduce investment costs, achieve a high proportion of renewable energy absorption, and efficient use of resources, which has a guiding role in the stable, economical, and efficient operation of the comprehensive energy system.

[0120] The present invention establishes an optimized operation method for an integrated energy system that takes into account the uncertainty of shared energy storage and wind and solar output, establishes a mathematical model for shared energy storage, effectively integrates and redistributes fragmented and dispersed idle energy storage resources, alleviates the problem of idle shared energy storage resources, and further explores the economic value of shared energy storage. Considering the uncertainty of wind power and photovoltaic output, an uncertainty model for wind power and photovoltaic output is established. Taking the lowest total investment and operating cost of the system as the optimization goal, a comprehensive energy system optimization model that takes into account the uncertainty of shared energy storage and wind and solar output is established and solved by considering factors such as the one-time investment cost, operation and maintenance cost, life cycle, renewable energy absorption rate, wind abandonment, and solar abandonment penalty costs of shared energy storage, thereby reducing investment costs, achieving a high proportion of renewable energy absorption, and efficient use of resources, which is of great significance to the stable and economic operation of the integrated energy system. BRIEF DESCRIPTION OF THE DRAWINGS

[0121] Figure 1 It is a flowchart of the present invention;

[0122] Figure 2 This is a schematic diagram of the optimization strategy flow of the present invention. DETAILED DESCRIPTION

[0123] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0124] like Figure 1 As shown, the method for optimizing the integrated energy system with shared energy storage and wind and solar power output uncertainty described in this embodiment is to execute the following steps through a computer device:

[0125] S100, establishing a mathematical model for shared energy storage;

[0126] S200, combining the uncertainty of wind power and photovoltaic output, establish a wind power and photovoltaic output uncertainty model;

[0127] S300, based on the uncertainty model of wind power and photovoltaic output and the mathematical model of shared energy storage, taking the minimum total investment and operation cost of the system as the optimization goal, considering the one-time investment cost, operation and maintenance cost, life cycle, renewable energy absorption rate, wind abandonment, and photovoltaic abandonment penalty costs of shared energy storage, establish and solve the comprehensive energy system optimization model that takes into account the uncertainty of shared energy storage and wind and photovoltaic output;

[0128] The following are specific instructions:

[0129] S100, establishing a mathematical model for shared energy storage;

[0130] The method for building the shared energy storage mathematical model is as follows:

[0131] S110, determining the research cycle time of shared energy storage;

[0132] Divide the research cycle of shared energy storage into n time periods, and the formula is:

[0133]

[0134] Where: RC is the research cycle time of shared energy storage, u t-1 and u t are the upper and lower boundary values ​​of the t period, respectively, where u t =RC, u0=0.

[0135] S120, within the research cycle time, establish a shared energy storage stage t output power model;

[0136]

[0137] Where: is the output power of the shared energy storage at stage t; Discharge efficiency for shared energy storage; is the average load power predicted by generator i participating in shared energy storage in stage t; The power purchased by power generator i participating in shared energy storage from the large power grid at stage t; is the average power generation of distributed wind power and distributed photovoltaic enterprise i on the supply side in stage t; is the maximum discharge power of the shared energy storage battery; is the minimum discharge power of the shared energy storage battery.

[0138] S130, establishing a shared energy storage charging power model in stage t that can store excess electric energy in the battery based on the shared energy storage output power model in stage t;

[0139]

[0140]

[0141] Where: is the input charging power of the large power grid to the shared energy storage at stage t; Charging power for shared energy storage batteries; The maximum charging power of the shared energy storage battery; is the minimum charging power of the shared energy storage battery.

[0142] S140, calculating a power capacity expression of a shared energy storage battery used by a UPS function according to a charging power model of the shared energy storage at stage t;

[0143] When the main power supply equipment on the supply side fails to supply power, the UPS backup battery power capacity must ensure that the core control system and emergency alarm can work simultaneously for more than 3 hours. Taking 3 hours as the UPS power supply time, the power capacity expression of the shared energy storage battery used by the UPS function is:

[0144]

[0145] Where: P i UPS The total power of the UPS equipment that the i-th generator needs to use for shared energy storage.

[0146] S150. Establishing a total energy storage device capacity model

[0147]

[0148]

[0149] Where: S ALL is the total power stored by the shared energy storage battery, S min The lower limit of the shared energy storage battery power, indicating the minimum remaining power of the shared energy storage device, is to avoid over-discharge of the shared energy storage battery, which may shorten the battery life, and meet the requirements of UPS power backup; SOC t S is the state of charge of the shared energy storage at stage t; t is the total power stored in the shared energy storage battery in stage t; low is the duration of the valley time on the demand side; β is the charging efficiency of the shared energy storage battery; SOC min SOC is the minimum state of charge of the shared energy storage device; maxIt is the maximum value of the state of charge of the shared energy storage device.

[0150] S200, based on the shared energy storage mathematical model and combined with the uncertainty of wind power and photovoltaic output, establish a wind power and photovoltaic output uncertainty model;

[0151] The method for building the wind power and photovoltaic processing uncertainty model is as follows:

[0152] S210. Establishing an uncertain output model for wind power generation

[0153] Fuzzy chance constrained programming is used to describe wind power output. When there are fuzzy random variables in the constraints, and the decision must be made before the random variables are predicted to be realized, the decision result may not meet the constraints. In this case, the decision result can be allowed to not meet the constraints to a certain extent, but the probability that the constraints are satisfied is not less than an acceptable confidence level. The fuzzy chance constraint is expressed in the form of

[0154] M[g(x,ξ)≤0]≥αFormula (13)

[0155] Where: α is the confidence level; g is the set of constraints; ξ is the fuzzy variable; x is the decision variable; M is the measurement function.

[0156] The credibility measure of fuzzy event A is expressed as:

[0157] M(A)=C r (A) Formula (14)

[0158] And there is the following formula:

[0159]

[0160] Where μ is the membership function of the fuzzy variable ξ, and sup is the upper bound.

[0161] Credibility Measure C r It has self-duality, assuming that an event with a credibility of 1 will definitely occur, and an event with a credibility of 0 will definitely not occur, thus avoiding the decision-making confusion that may be caused by traditional membership calculation.

[0162] The uncertainty of wind turbine output is transformed into the uncertainty of output prediction error. That is, assuming that the predicted value of wind turbine output is certain data, from a new perspective, fuzzy theory is used to model the prediction error, and the uncertainty of wind turbine output is indirectly described through the uncertainty of wind turbine output prediction error. Assuming that the percentage of wind turbine output prediction error is ε wind , the function expression is as follows.

[0163]

[0164] Where p′ wind 、p wind Represent the predicted output and actual output of wind power respectively. There are two forms of prediction error: if the actual output is higher than the predicted output, it is a positive error; if the actual output is lower than the predicted output, it is a negative error.

[0165] Its membership μ can be expressed by Cauchy distribution wind :

[0166]

[0167] Where: represent the statistical average of positive error and negative error respectively; σ is the weight. According to formula (3), for ε wind ∈R, the credibility measure of the prediction error ξ is

[0168]

[0169] The credibility distribution function value refers to the fuzzy variable ξ not greater than ε wind The credibility of can be compared to the probability distribution function in probability theory.

[0170] S220. Establishing an output uncertainty model for photovoltaic power generation

[0171] First, it is necessary to conduct a statistical analysis on the historical data of PV output, select a certain time period, and select a continuous range covering all PV output scenarios within the period to generate an uncertainty set under a certain radius constraint.

[0172]

[0173] Where: is the uncertain set of photovoltaic power output; is the actual output of the photovoltaic unit in the T period; and They represent the minimum and maximum output of the photovoltaic unit in the T period respectively; T is the number of periods in a day. This paper takes 1 hour as the unit and decomposes each day into 24 periods; Λ is the robustness factor with a value range of [0, 24], which is used to control the interval range of the uncertainty set. The larger the value, the larger the range covered by the uncertainty set.

[0174] S300, based on the uncertainty model of wind power and photovoltaic output and the mathematical model of shared energy storage, with the minimum total system investment and operation cost as the optimization goal, taking into account factors such as the one-time investment cost, operation and maintenance cost, life cycle, renewable energy absorption rate, wind and solar abandonment penalty costs of shared energy storage, an integrated energy system optimization model considering the uncertainty of shared energy storage and wind and solar output is established and solved.

[0175] S310. Establish the objective function of the integrated energy system optimization operation model.

[0176]

[0177] Where: F is the total operating cost of the integrated energy system, including the power generation cost of the natural gas cooling-heating-generator unit and the penalty for deviation from the planned output, the power purchase cost, the penalty for deviation from the planned power purchase, the cost of wind and solar power abandonment, and the annual investment and operation and maintenance cost of the shared energy storage system; N rMT is the number of natural gas cooling-heating-generator units in area r; is the power output of the i-th natural gas cooling-heating-generator unit in region r at time t under scenario s; is the thermal output of the i-th natural gas cooling-heating-generator unit in region r at time t under scenario s; is the purchased power of the i-th natural gas cooling-heating-generating unit in area r at time t under scenario s; is the cost function of the natural gas cooling-heating-electrical unit; It is the abandoned wind and solar power; is the power that can be generated by distributed wind power; is the power that can be generated by distributed photovoltaics; The grid-connected power of distributed wind power; is the grid-connected power of distributed photovoltaics; is the heating power of distributed wind power; is the heating power of distributed photovoltaics; is the unit price of electricity purchase; ρ1 is the penalty price for the natural gas cooling-heating-generator group deviating from the planned output; ρ2 is the penalty price for the purchased power deviating from the planned purchased power; ρ3 is the penalty price for abandoning wind and solar power; λ is the rate of return of funds; N SES The total number of waste batteries invested and collected for shared energy storage operators; is the unit power investment coefficient of shared energy storage; is the unit capacity investment coefficient of shared energy storage; is the annual maintenance cost coefficient of shared energy storage; is the operating power of shared energy storage; y is the full life cycle of shared energy storage; D is the number of typical days in a year; T d is the number of days corresponding to a typical day in a year.

[0178] S320. Establish constraint conditions for the optimal operation model of the integrated energy system.

[0179] (1) Electric power balance constraints

[0180] The electric power output needs to meet the following constraints:

[0181]

[0182] Where: L r is the set of contact lines between area r and other areas; is the transmission power of tie line l; It indicates that the tie line l inputs power into the area at time t; It means that at time t, the tie line l transmits power outside the area; is the electrical load power; is the power of the electric refrigerator.

[0183] (2) Constraints of natural gas cooling-heating-generator units

[0184] Thermoelectric ratio constraints, output upper and lower limit constraints and ramp constraints of natural gas cooling-heating-generator units.

[0185]

[0186] Where: K hp,r,t is the heat-to-electricity ratio of the i-th natural gas cooling-heating-generating unit in region r; is the lower limit of the power output of the i-th natural gas cooling-heating-generating unit in region r; The upper limit of the power output of the i-th natural gas cooling-heating-generating unit in region r; It is the maximum power increase per unit time of the i-th natural gas cooling-heating-generating unit in area r.

[0187] (3) Power purchase power constraints

[0188] Each demand-side functional area is only allowed to purchase electricity and gas, and is not allowed to sell electricity and gas.

[0189]

[0190] Where: The capacity of the substation at the grid connection point; The power of purchased gas; The upper limit of natural gas pipeline transmission capacity.

[0191] (4) Distributed wind power and distributed photovoltaic constraints

[0192] Distributed wind power and distributed photovoltaic output need to meet the following constraints.

[0193]

[0194] (5) Heating system constraints

[0195] The electric boiler in area r only consumes the wind power and photovoltaic power in area r, satisfying the constraints of the electric-heat conversion relationship and the output constraints of the electric boiler.

[0196]

[0197] Where: is the heat output of the electric boiler in area r at time t under scenario s; η EB,r is the electricity conversion efficiency of the electric boiler in area r; It is the upper limit of the output of the electric boiler in area r.

[0198] (6) Start and stop time constraints

[0199] The equipment start and stop time constraints must meet the following constraints.

[0200]

[0201] Where: The continuous operation time of the natural gas cooling-heating-electricity unit; The continuous downtime of the natural gas cooling-heating-electricity unit; The continuous operation time of the electric boiler equipment; The continuous downtime of the electric boiler equipment; The minimum start-up time for natural gas cooling-heating-electricity; Minimum downtime for natural gas cooling, heating and electricity; The minimum start-up time for electric boiler equipment; is the minimum downtime of the electric boiler equipment; u MT,r,t-1 is the operating status of the natural gas cooling-heating-generator unit at time t-1, which is a 00-1 variable; u EB,r,t-1 is the operating status of the electric boiler equipment at time t-1, which is a 0-1 variable.

[0202] (7) Transport channel capacity constraints

[0203] The transport channel capacity satisfies the following constraints.

[0204]

[0205] Where: is the line capacity between nodes i and j at the initial moment; is the newly added line capacity between nodes i and j at time t; is the maximum capacity of the transmission line between nodes i and j; M is the penalty coefficient, so that hour, is 0, when When the , is large enough so that there is enough space to choose a new line; is a Boolean variable, a "switch" of a continuous constraint, which produces a capacity greater than or equal to the maximum capacity allowed by the line upgrade. (8) Reserve constraint

[0206] The spare capacity meets the following constraints.

[0207]

[0208] In the formula: r1 is the upper reserve coefficient of the load; r2 is the upper reserve coefficient of distributed wind power; r3 is the upper reserve coefficient of distributed photovoltaic; r4 is the lower reserve coefficient of distributed wind power; r5 is the lower reserve coefficient of distributed photovoltaic.

[0209] (9) Shared energy storage capacity configuration constraints

[0210] The shared energy storage configuration constraint is used to determine the total shared energy storage capacity invested in the integrated energy system. If the shared energy storage capacity is configured too large, it will lead to low utilization of the shared energy storage of the entire integrated energy system and low investment return rate; if the shared energy storage capacity is configured too small, it will lead to a reduction in the optimization space for the operation and scheduling of the integrated energy system.

[0211] 1. Shared energy storage configuration ratio constraints:

[0212] Q SES =ηP SES Formula (50)

[0213] Where: η is the energy ratio of shared energy storage, and the total rated power of shared energy storage is proportional to the total capacity.

[0214] 2. Shared energy storage charge state constraints:

[0215] SOC SES,d,t =(1-ω)SOC SES,d,t-1 Formula (51)

[0216] Where: ω is the self-discharge rate of shared energy storage; SOC SES,d,t is the state of charge of the shared energy storage at time period t on the dth typical day.

[0217] 3. Shared energy storage capacity power state constraints:

[0218]

[0219] Q SES =T SES P SES Formula (53)

[0220] Where: The maximum capacity power value allowed for shared energy storage; T SES The total time of charging and discharging shared energy storage; P SES The capacity power planned for shared energy storage; Q SES Energy capacity planned for shared energy storage.

[0221] S330. Optimize and solve the integrated energy system operation model.

[0222] In the process of interaction between the agent and the environment, the sequence from the starting state to the end state is called trajectory χ. When the policy parameter ω is given, the probability of each χ existing is expressed as p ω (χ), the calculation process is as follows:

[0223]

[0224] Where: p ω (m t |c t ) means in c t Select action m in state t The probability of p(c t+1 |c t ,m t ) means in state c t Apply action m t The environment state changes to c t+1 The probability of the probability is determined by the environment and cannot be controlled manually. In the continuous interaction between the agent and the environment, several trajectories χ are generated. The sum of the instant rewards corresponding to each trajectory χ is recorded as R(χ), and the expected cumulative reward is recorded as It is expressed by the following formula:

[0225]

[0226] The goal of reinforcement learning is to achieve trajectory cumulative rewards The maximization of can be solved using the policy gradient algorithm. In the solution process, the target gradient is first obtained

[0227]

[0228] Where: express Actions in status The direction in which the probability is most affected by ω; R(χ n ) represents the step size and direction of the parameter ω update. The solution process can be simply explained as when the reward R(χ n ) increases, the state Next action The probability increases, otherwise the execution of the action is inhibited.

[0229] The specific steps of the final optimization strategy are: initialize the parameter ω in the strategy, the agent follows the given strategy and continuously interacts with the environment to collect several trajectories; calculate the policy gradient based on the collected data And update the parameter ω; use the strategy with updated parameters to continue to repeat the above steps to collect data, and iteratively update the strategy to the optimal strategy. The specific process is as follows Figure 2 shown.

[0230] In summary, the present invention establishes a mathematical model for shared energy storage, and considers the uncertainty of wind power and photovoltaic output to establish wind power and photovoltaic output uncertainty models respectively. Taking the lowest total system investment and operation cost as the optimization goal, the one-time investment cost, operation and maintenance cost, life cycle, renewable energy absorption rate, wind abandonment, and light abandonment penalty costs of shared energy storage are considered to establish and solve the optimization model of the comprehensive energy system considering the uncertainty of shared energy storage and wind and solar output, reduce investment costs, achieve a high proportion of renewable energy absorption, and efficient use of resources, which has a guiding role in the stable, economical, and efficient operation of the comprehensive energy system.

[0231] In another embodiment provided in the present application, a computer program product comprising instructions is also provided. When the computer is run on a computer, the computer executes any of the methods for optimizing an integrated energy system with shared energy storage and wind and solar power output uncertainty in the above embodiments.

[0232] It is understandable that the system, device and storage medium provided in the embodiments of the present invention correspond to the method provided in the embodiments of the present invention, and the explanation, examples and beneficial effects of the relevant contents can refer to the corresponding parts in the above methods.

[0233] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0234] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0235] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0236] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for optimizing an integrated energy system with shared energy storage and wind and solar output uncertainty, characterized in that: The specific steps are as follows: S100, establishing a mathematical model for shared energy storage; S200, combining the uncertainty of wind power and photovoltaic output, establish a wind power and photovoltaic output uncertainty model; S300. Based on the uncertainty model of wind power and photovoltaic output and the mathematical model of shared energy storage, with the minimum total system investment and operation cost as the optimization goal, the one-time investment cost, operation and maintenance cost, life cycle, renewable energy absorption rate, wind and solar abandonment penalty cost factors of shared energy storage are considered to establish and solve the comprehensive energy system optimization model considering the uncertainty of shared energy storage and wind and solar output.

2. The method for optimizing a comprehensive energy system with shared energy storage and wind and solar power output uncertainty according to claim 1, characterized in that: The method for building the shared energy storage mathematical model in S100 is as follows: S110, determining the research cycle time of shared energy storage; Divide the research cycle of shared energy storage into n time periods, and the formula is: Where: RC is the research cycle time of shared energy storage, u t-1 and u t are the upper and lower boundary values ​​of the t period, respectively, where u t =RC, u0=0; S120, within the research cycle time, establish a shared energy storage stage t output power model; Where: is the output power of the shared energy storage at stage t; Discharge efficiency for shared energy storage; is the average load power predicted by generator i participating in shared energy storage in stage t; The power purchased by power generator i participating in shared energy storage from the large power grid at stage t; is the average power generation of distributed wind power and distributed photovoltaic enterprise i on the supply side in stage t; is the maximum discharge power of the shared energy storage battery; is the minimum discharge power of the shared energy storage battery; S130, establishing a shared energy storage charging power model in stage t that can store excess electric energy in the battery based on the shared energy storage output power model in stage t; Where: is the input charging power of the large power grid to the shared energy storage at stage t; Charging power for shared energy storage batteries; The maximum charging power of the shared energy storage battery; is the minimum charging power of the shared energy storage battery; S140, calculating a power capacity expression of a shared energy storage battery used by a UPS function according to a charging power model of the shared energy storage at stage t; When the main power supply equipment on the supply side fails to supply power, the UPS backup battery power capacity must ensure that the core control system and emergency alarm can work simultaneously for more than 3 hours. Taking 3 hours as the UPS power supply time, the power capacity expression of the shared energy storage battery used by the UPS function is: Where: P i UPS The total power of the UPS equipment that the i-th generator needs to use for shared energy storage; S150. Establishing a total energy storage device capacity model SOC min ≤SOC t ≤SOC max Formula (11) SOC0=SOC uN Formula (12) Where: S ALL is the total power stored by the shared energy storage battery, S min The lower limit of the shared energy storage battery power, indicating the minimum remaining power of the shared energy storage device, is to avoid over-discharge of the shared energy storage battery, which may shorten the battery life, and meet the requirements of UPS power backup; SOC t S is the state of charge of the shared energy storage at stage t; t is the total power stored in the shared energy storage battery in stage t; low is the duration of the valley time on the demand side; β is the charging efficiency of the shared energy storage battery; SOC min SOC is the minimum state of charge of the shared energy storage device; max It is the maximum value of the state of charge of the shared energy storage device.

3. The method for optimizing a comprehensive energy system with shared energy storage and wind and solar power output uncertainty according to claim 1 is characterized in that: The method for building the wind power and photovoltaic output uncertainty model in S200 is as follows: S210. Establishing an uncertain output model for wind power generation Fuzzy chance constraint programming is used to describe wind power output. The fuzzy chance constraint is expressed as M[g(x,ξ)≤0]≥αFormula (13) Where: α is the confidence level; g is the set of constraints; ξ is the fuzzy variable; x is the decision variable; M is the measurement function; The credibility measure of fuzzy event A is expressed as: M(A) = C r (Formula (14) of (A)) And there is the following formula: Where: μ is the membership function of the fuzzy variable ξ, sup is the upper bound; Credibility Measure C r It has self-duality, assuming that an event with a credibility of 1 will definitely occur, and an event with a credibility of 0 will definitely not occur, thus avoiding the decision-making confusion that may be caused by traditional membership calculation; The uncertainty of wind turbine output is transformed into the uncertainty of output prediction error. That is, assuming that the predicted value of wind turbine output is certain data, the prediction error is modeled using fuzzy theory from a new perspective. The uncertainty of wind turbine output is indirectly described by the uncertainty of wind turbine output prediction error. Assuming that the percentage of wind turbine output prediction error is ε wind , the function expression is as follows: Where p' wind 、p wind Respectively represent the predicted output and actual output of wind power; the prediction error has two forms: if the actual output is higher than the predicted output, it is a positive error; if the actual output is lower than the predicted output, it is a negative error; Its membership μ can be expressed by Cauchy distribution wind : Where: Represent the statistical average of positive error and negative error respectively; σ is the weight. According to formula (3), for ε wind ∈R, the credibility measure of the prediction error ξ is The credibility distribution function value refers to the fuzzy variable ξ not greater than ε wind The credibility of can be compared to the probability distribution function in probability theory; S220. Establishing an output uncertainty model for photovoltaic power generation Firstly, the historical data of PV output is statistically analyzed, a certain time period is selected, and a continuous range covering all PV output scenarios is selected within the time period, thereby generating an uncertainty set under a certain radius constraint; Where: is the uncertain set of photovoltaic power output; is the actual output of the photovoltaic unit in the T period; and They represent the minimum and maximum output of the photovoltaic unit in the T period respectively; T is the number of periods in a day. This paper takes 1 hour as the unit and divides each day into 24 periods; Λ is the robustness factor, with a value range of [0, 24], which is used to control the interval range of the uncertainty set. The larger the value, the larger the range covered by the uncertainty set.

4. The method for optimizing a comprehensive energy system with shared energy storage and wind and solar power output uncertainty according to claim 1, characterized in that: The method for building an integrated energy system optimization model considering the uncertainty of shared energy storage and wind and solar power output in S300 is as follows: S310, establishing the objective function of the integrated energy system optimization operation model; S320, establish constraints for the optimal operation model of the integrated energy system; S330. Optimize and solve the integrated energy system operation model.

5. The method for optimizing a comprehensive energy system with shared energy storage and wind and solar power output uncertainty according to claim 4 is characterized in that: The objective function in S310 is specifically: Where: F is the total operating cost of the integrated energy system, including the power generation cost of the natural gas cooling-heating-generator unit and the penalty for deviation from the planned output, the power purchase cost, the penalty for deviation from the planned power purchase, the cost of wind and solar power abandonment, and the annual investment and operation and maintenance cost of the shared energy storage system; N rMT is the number of natural gas cooling-heating-generator units in area r; is the power output of the i-th natural gas cooling-heating-generator unit in region r at time t under scenario s; is the thermal output of the i-th natural gas cooling-heating-generator unit in region r at time t under scenario s; is the purchased power of the i-th natural gas cooling-heating-generating unit in area r at time t under scenario s; is the cost function of the natural gas cooling-heating-electrical unit; It is the abandoned wind and solar power; is the power that can be generated by distributed wind power; is the power that can be generated by distributed photovoltaics; The grid-connected power of distributed wind power; is the grid-connected power of distributed photovoltaics; is the heating power of distributed wind power; is the heating power of distributed photovoltaics; is the unit price of electricity purchase; ρ1 is the penalty price for the natural gas cooling-heating-generator group deviating from the planned output; ρ2 is the penalty price for the purchased power deviating from the planned purchased power; ρ3 is the penalty price for abandoning wind and solar power; λ is the rate of return of funds; N SES The total number of waste batteries invested and collected for shared energy storage operators; is the unit power investment coefficient of shared energy storage; is the unit capacity investment coefficient of shared energy storage; is the annual maintenance cost coefficient of shared energy storage; is the operating power of shared energy storage; y is the full life cycle of shared energy storage; D is the number of typical days in a year; T d is the number of days corresponding to a typical day in a year.

6. The method for optimizing a comprehensive energy system with shared energy storage and wind and solar power output uncertainty according to claim 4, characterized in that: The constraints of the model in S320 are specifically as follows: (1) Electric power balance constraints The electric power output needs to meet the following constraints: Where: L r is the set of contact lines between area r and other areas; is the transmission power of tie line l; It indicates that the tie line l inputs power into the area at time t; It means that at time t, the tie line l transmits power outside the area; is the electrical load power; is the power of the electric refrigerator; (2) Constraints of natural gas cooling-heating-generator units Thermoelectric ratio constraints, output upper and lower limit constraints and ramp constraints of natural gas cooling-heating-generator units: Where: K hp,r,t is the heat-to-electricity ratio of the i-th natural gas cooling-heating-generating unit in region r; is the lower limit of the power output of the i-th natural gas cooling-heating-generating unit in region r; The upper limit of the power output of the i-th natural gas cooling-heating-generating unit in region r; is the maximum increase in power per unit time of the i-th natural gas cooling-heating-generating unit in area r; (3) Power purchase power constraints Each demand-side functional area is only allowed to purchase electricity and gas, and is not allowed to sell electricity and gas: Where: The capacity of the substation at the grid connection point; The power of purchased gas; Capping the capacity of natural gas pipelines; (4) Distributed wind power and distributed photovoltaic constraints Distributed wind power and distributed photovoltaic output need to meet the following constraints: (5) Heating system constraints The electric boiler in area r only consumes the wind power and photovoltaic power in area r, satisfying the electric-heat conversion relationship constraints of the electric boiler and the output constraints of the electric boiler: Where: is the heat output of the electric boiler in area r at time t under scenario s; η EB,r is the electricity conversion efficiency of the electric boiler in area r; The output limit of the electric boiler in area r; (6) Start and stop time constraints The device start and stop time constraints must meet the following constraints: Where: The continuous operation time of the natural gas cooling-heating-electricity unit; The continuous downtime of the natural gas cooling-heating-electricity unit; The continuous operation time of the electric boiler equipment; The continuous downtime of the electric boiler equipment; The minimum start-up time for natural gas cooling-heating-electricity; Minimum downtime for natural gas cooling, heating and electricity; The minimum start-up time for electric boiler equipment; The minimum downtime of the electric boiler equipment; u MT,r,t-1 is the operating status of the natural gas cooling-heating-generator unit at time t-1, which is a 00-1 variable; u EB,r,t-1 is the operating status of the electric boiler equipment at time t-1, which is a 0-1 variable; (7) Transport channel capacity constraints The transport channel capacity meets the following constraints: Where: is the line capacity between nodes i and j at the initial moment; is the newly added line capacity between nodes i and j at time t; is the maximum capacity of the transmission line between nodes i and j; M is the penalty coefficient, so that hour, is 0, when When the , is large enough so that there is enough space to choose a new line; is a Boolean variable, a "switch" of a continuous constraint, which produces a capacity greater than or equal to the maximum capacity allowed by the line upgrade; (8) Backup constraints The spare capacity meets the following constraints: In the formula: r1 is the upper reserve coefficient of the load; r2 is the upper reserve coefficient of distributed wind power; r3 is the upper reserve coefficient of distributed photovoltaic; r4 is the lower reserve coefficient of distributed wind power; r5 is the lower reserve coefficient of distributed photovoltaic; (9) Shared energy storage capacity configuration constraints The shared energy storage configuration constraint is used to determine the total shared energy storage capacity invested in the integrated energy system. If the shared energy storage capacity is configured too large, the utilization rate of the shared energy storage of the entire integrated energy system will be low, and the return on investment will be low; if the shared energy storage capacity is configured too small, the optimization space for the operation and scheduling of the integrated energy system will be reduced. The shared energy storage configuration constraints are as follows:

1. Shared energy storage configuration ratio constraints: Q SES = ηP SES Formula (50) Where: η is the energy ratio of shared energy storage, and the total rated power of shared energy storage is proportional to the total capacity; 2. Shared energy storage charge state constraints: SOCIETY SES,d,t =(1-ω)SOC SES,d,t-1 formula (51) Where: ω is the self-discharge rate of shared energy storage; SOC SES,d,t is the state of charge of the shared energy storage at time period t on the dth typical day; 3. Shared energy storage capacity power state constraints: Q SES =T SES P SES Formula (53) Where: The maximum capacity power value allowed for shared energy storage; T SES The total time of charging and discharging shared energy storage; P SES The capacity power planned for shared energy storage; Q SES Energy capacity planned for shared energy storage.

7. The method for optimizing a comprehensive energy system with shared energy storage and wind and solar power output uncertainty according to claim 4, characterized in that: The optimization solution method of the integrated energy system operation model in S330 is as follows: In the process of interaction between the agent and the environment, the sequence from the starting state to the end state is called trajectory χ; when the strategy parameter ω is given, the probability of each χ existing is expressed as p ω (χ), the calculation process is as follows: Where: p ω (m t |c t ) means in c t Select action m in state t The probability of p(c t+1 |c t ,m t ) means in state c t Apply action m t The environment state changes to c t+1 The probability of the probability is determined by the environment and cannot be controlled artificially. In the continuous interaction between the agent and the environment, several trajectories χ are generated. The sum of the immediate rewards corresponding to each trajectory χ is recorded as R(χ), and the expected cumulative reward is recorded as It is expressed by the following formula: The goal of reinforcement learning is to achieve the trajectory cumulative reward R ω The maximization of can be solved by using the policy gradient algorithm; in the solution process, the target gradient is first required Where: express Actions in status The direction in which the probability is most affected by ω; R(χ n ) represents the step size and direction of the parameter ω update. The solution process can be simply explained as when the reward R(χ n ) increases, the state Next action The probability increases, otherwise the execution of the action is inhibited.

Images