Decision-making method for light storage and charging multifunctional charging station participating in peak shaving auxiliary service based on reinforcement learning

Abstract

The invention discloses a decision-making method for a light storage and charging multifunctional charging station to participate in peak regulation auxiliary service based on reinforcement learning, and relates to the field of power system analysis and calculation. Establishing a cost objective function through comprehensive analysis of charging service income, electricity purchasing and selling cost, peak regulation response income and peak regulation deviation punishment cost; obtaining a peak regulation operation constraint condition based on confluence analysis of an electric power and electric quantity balance constraint, a peak regulation capacity constraint, an energy storage system charging and discharging constraint, an electric vehicle charging constraint and a spinning reserve capacity constraint; and establishing a peak regulation auxiliary service decision model according to the cost objective function and the peak regulation operation constraint condition, and training through a reinforcement learning algorithm to obtain an energy storage charging and discharging strategy. According to the method, cost-benefit factors such as system operation and maintenance cost and carbon emission reduction benefit are considered, and optimization of peak regulation cost benefit can be realized when the light storage and charging multifunctional charging station participates in peak regulation auxiliary service.

Description

Technical Field

[0001] The present invention relates to the field of power system analysis and calculation, and in particular to a decision-making method for a photovoltaic storage and charging multifunctional charging station to participate in peak load auxiliary services based on reinforcement learning. Background Art

[0002] As the global climate change problem becomes increasingly prominent, countries have stepped up their support for the new energy industry. The development of the electric vehicle industry has become an important direction for the global energy revolution and low-carbon transformation. As an important carrier of clean energy consumption, electric vehicles can not only effectively reduce carbon emissions from traditional fuel vehicles, but also participate in grid peak regulation as mobile energy storage units, providing flexible load regulation capabilities for the power system. my country's electric vehicle industry is developing rapidly, the construction of charging infrastructure is constantly improving, and the scale of the charging network continues to expand. However, with the continuous growth of the number of electric vehicles, the matching problem of charging load and grid load has become increasingly prominent. How to efficiently participate in the peak regulation of the power system through advanced charging facilities and achieve the coordinated optimization of economic and environmental benefits has become a key issue of concern to the industry.

[0003] In order to match the charging load with the grid load, the industry has proposed a solution for a photovoltaic storage and charging multifunctional charging station that integrates energy storage systems and photovoltaic power generation equipment. This solution can effectively respond to user charging needs through clean power generation from photovoltaic power generation equipment and flexible adjustment of the energy storage system. At present, the industry has explored the operation modes of various photovoltaic storage and charging multifunctional charging stations. Especially in the context of the growing demand for peak load regulation in the power system, the importance of charging stations as a new type of peak load regulation resource has been continuously highlighted, and the decision-making optimization research on their participation in peak load regulation auxiliary services is particularly important.

[0004] However, existing photovoltaic, storage and charging multifunctional charging stations still face many challenges when participating in the peak-shaving auxiliary service market: first, existing decision-making methods often only focus on a single economic benefit indicator, lack of comprehensive consideration of system operation and maintenance costs, which may lead to inaccurate cost estimates in the actual operation process; second, the existing optimization schemes do not take into account the operating benefits and carbon emission reduction benefits when evaluating the participation of charging stations in peak-shaving, and cannot accurately measure the comprehensive benefits of charging stations in the peak-shaving process; in addition, the current decision-making methods have obvious deficiencies in cost-benefit analysis and have failed to establish a complete cost-benefit evaluation system, which makes it difficult for charging stations to achieve the optimal benefit balance when participating in peak-shaving auxiliary services; the existence of these problems has seriously affected the enthusiasm of photovoltaic, storage and charging multifunctional charging stations to participate in peak-shaving auxiliary services, and restricted them from playing a greater regulatory role in the power system.

[0005] Currently, no effective solution has been proposed for the problems in the related technologies. Summary of the invention

[0006] In response to the problems in the related technology, the present invention proposes a decision-making method for the participation of photovoltaic, storage and charging multifunctional charging stations in peak-shaving auxiliary services based on reinforcement learning. The method has the advantages of taking into account cost-effectiveness factors such as system operation and maintenance costs, operating benefits and carbon emission reduction benefits, and achieving the optimization of peak-shaving cost-effectiveness in the participation of photovoltaic, storage and charging multifunctional charging stations in peak-shaving auxiliary services. It further solves the problem that the prior art only focuses on a single economic benefit indicator, lacks comprehensive consideration of system operation and maintenance costs, and the market mechanism for photovoltaic, storage and charging multifunctional charging stations to participate in peak-shaving is still imperfect, lacks reasonable decision-making methods, and affects the enthusiasm of photovoltaic, storage and charging multifunctional charging stations to participate in peak-shaving auxiliary services.

[0007] To this end, the specific technical solution adopted by the present invention is as follows:

[0008] According to one aspect of the present invention, a method for making a decision on a photovoltaic storage and charging multifunctional charging station to participate in a peak-shaving auxiliary service based on reinforcement learning is provided. The method for making a decision on a photovoltaic storage and charging multifunctional charging station to participate in a peak-shaving auxiliary service based on reinforcement learning comprises the following steps:

[0009] S1. Based on the photovoltaic operation and maintenance costs and energy storage operation and maintenance costs, the operating power and carbon emissions of different types of facilities are analyzed to obtain the operation and maintenance costs and carbon emission reduction benefits. Through a comprehensive analysis of charging service revenue, electricity purchase and sales costs, peak-shaving response revenue and peak-shaving deviation penalty fees, a cost objective function for the participation of photovoltaic storage and charging multifunctional charging stations in peak-shaving is established;

[0010] S2. Based on different types of facilities and photovoltaic storage and charging multifunctional charging stations, establish power balance constraints and peak-shaving capacity constraints, and construct energy storage system charging and discharging constraints and electric vehicle charging constraints. Through integrated analysis with spinning reserve capacity constraints, obtain the peak-shaving operation constraints of photovoltaic storage and charging multifunctional charging stations;

[0011] S3. According to the cost objective function of the multifunctional charging station for photovoltaic storage and charging participating in peak load regulation and the peak load regulation operation constraints, a peak load regulation auxiliary service decision model is established using a recurrent neural network, and the peak load regulation auxiliary service decision model is trained through a reinforcement learning algorithm to obtain the energy storage charging and discharging strategy.

[0012] Furthermore, according to the photovoltaic operation and maintenance costs and energy storage operation and maintenance costs, the operation and maintenance costs and carbon emission reduction benefits are obtained by analyzing the operating power and carbon emissions of different types of facilities. Through a comprehensive analysis of charging service revenue, electricity purchase and sales costs, peak-shaving response revenue, and peak-shaving deviation penalty fees, the cost objective function of the photovoltaic storage and charging multifunctional charging station participating in peak-shaving is established, which includes the following steps:

[0013] S11. Based on the photovoltaic operation and maintenance cost and the energy storage operation and maintenance cost, the fixed purchase cost of the multifunctional charging station is solved, and the operation and maintenance cost of the photovoltaic storage multifunctional charging station is obtained through a comprehensive analysis of the fixed purchase cost and the variable dynamic cost;

[0014] S12. Calculate the carbon reduction benefits of different types of facilities based on the operating power of photovoltaic power generation, battery energy storage, public power grid and charging system, combined with the corresponding carbon emissions analysis;

[0015] S13. Calculate the charging service income based on the charging fee and the charging load power, analyze the purchase and sale costs of the charging service, and solve the peak response income according to the adjustment amount of the photovoltaic storage and charging multifunctional integrated charging station in different time periods;

[0016] S14. Calculate the penalty fee for peak-shaving deviation by using the deviation assessment segmentation threshold, and establish the cost objective function for the participation of the photovoltaic storage and charging multifunctional charging station in peak-shaving by combining the operation and maintenance cost of the photovoltaic storage and charging multifunctional charging station, the carbon emission reduction benefits of different types of facilities, the charging service income, the electricity purchase and sales cost of the charging service and the peak-shaving response income.

[0017] Furthermore, based on the photovoltaic operation and maintenance cost and the energy storage operation and maintenance cost, the fixed purchase cost of the multifunctional charging station is solved, and the operation and maintenance cost of the photovoltaic storage multifunctional charging station is obtained through a comprehensive analysis of the fixed purchase cost and the variable dynamic cost, including the following steps:

[0018] S111. Calculate the photovoltaic operation and maintenance cost based on the photovoltaic system, and obtain the fixed purchase cost of the photovoltaic storage and charging multifunctional charging station by analyzing the fixed cost and variable cost of energy storage operation and maintenance;

[0019] S112. Calculate the variable dynamic cost of the photovoltaic storage and charging multifunctional charging station based on different facility types;

[0020] S113. Calculate the operation and maintenance cost of the photovoltaic-storage-charging multifunctional charging station based on the fixed purchase cost and variable dynamic cost of the photovoltaic-storage-charging multifunctional charging station.

[0021] Furthermore, the photovoltaic operation and maintenance costs are calculated based on the photovoltaic system, and the fixed purchase cost of the photovoltaic storage and charging multifunctional charging station is obtained by analyzing the fixed cost of energy storage operation and maintenance and the variable cost of energy storage operation and maintenance, including the following steps:

[0022] S1111. Calculate the photovoltaic operation and maintenance cost based on the unit operation and maintenance cost and maximum power of the photovoltaic system and the discount rate of photovoltaic equipment;

[0023] S1112. Calculate the fixed cost of energy storage operation and maintenance based on the unit capacity operation and maintenance cost of energy storage and the rated capacity, combined with the discount rate of energy storage equipment;

[0024] S1113. Calculate the variable cost of energy storage operation and maintenance based on the charging power and discharging power of the energy storage system, and analyze the energy storage operation and maintenance cost in combination with the fixed cost of energy storage operation and maintenance;

[0025] S1114. Based on the photovoltaic operation and maintenance cost and the energy storage operation and maintenance cost, solve the fixed purchase cost of the multi-functional charging station.

[0026] Furthermore, the expression of photovoltaic operation and maintenance cost is:

[0027]

[0028] In the formula, C PV,OM The cost of photovoltaic operation and maintenance; PV is the unit operation and maintenance cost of the photovoltaic system; is the maximum photovoltaic power; r PV is the discount rate of photovoltaic equipment; PV The service life of photovoltaic equipment;

[0029] The expression of energy storage operation and maintenance cost is:

[0030]

[0031] In the formula, C ES,OM is the operation and maintenance cost of battery energy storage; θ E is the operation and maintenance cost per unit capacity of energy storage; is the rated capacity of energy storage; r ES is the discount rate of energy storage equipment; ES is the service life of the energy storage device; θ ES is the dynamic cost per unit of electricity; is the charging power of energy storage during period t; is the discharge power of energy storage during period t;

[0032] The expression for the fixed acquisition cost of a multifunctional charging station is:

[0033] C in =C PV,OM +C ES,OM

[0034] In the formula, C in Fixed acquisition cost.

[0035] Furthermore, according to the operating power of photovoltaic power generation, battery energy storage, public power grid and charging system, combined with the corresponding carbon emissions analysis, the carbon emission reduction benefits of different types of facilities are calculated, including the following steps:

[0036] S121. Based on the photovoltaic power generation power and the carbon emissions of coal-fired power generation with the same power, calculate the carbon emission reduction benefits for photovoltaic power generation;

[0037] S122. Calculate the carbon emission reduction benefits of battery energy storage based on the battery energy storage power and the carbon emissions of coal burning for battery energy storage at the same power level;

[0038] S123. Calculate the carbon emission reduction benefits for the public power grid based on the load power of the public power grid and the carbon emissions of coal-fired power generation under the same power consumption;

[0039] S124. Calculate the carbon emission reduction benefits of the charging pile based on the charging load power and the carbon emissions of fuel at the same power level.

[0040] Furthermore, based on different types of facilities and the photovoltaic storage and charging multifunctional charging station, power balance constraints and peak load capacity constraints are established, and energy storage system charging and discharging constraints and electric vehicle charging constraints are constructed. Through integrated analysis with the spinning reserve capacity constraints, the peak load operation constraints of the photovoltaic storage and charging multifunctional charging station are obtained, including the following steps:

[0041] S21. Establishing power balance constraints based on the power supply of the public grid, the power of photovoltaic power generation, and the power of energy storage charging and discharging;

[0042] S22. Constructing a peak-shaving capacity constraint according to the available capacity of the photovoltaic storage and charging multifunctional charging station in the peak-shaving process;

[0043] S23. Based on the energy storage capacity change equation, establish the energy storage system charge and discharge constraints;

[0044] S24. Constructing charging constraints for electric vehicles by analyzing random load demands and dispatchable load demands of electric vehicles;

[0045] S25. Based on the requirements of power supply reliability, a spinning reserve capacity constraint is established, and by integrating it with the power balance constraint, peak-shaving capacity constraint, energy storage system charging and discharging constraint, and electric vehicle charging constraint, the peak-shaving operation constraint of the photovoltaic storage and charging multifunctional charging station is obtained.

[0046] Furthermore, based on the energy storage capacity change equation, establishing the energy storage system charging and discharging constraints includes the following steps:

[0047] S231, establishing an energy storage capacity change equation according to the initial state of the energy storage system and the rated capacity of the battery;

[0048] S232, solving the energy storage capacity change equation to obtain the charge state of the energy storage system;

[0049] S233. Construct charging and discharging constraints for the energy storage system based on the charging and discharging power of the energy storage system and the state of charge of the energy storage system.

[0050] Furthermore, according to the cost objective function of the photovoltaic storage and charging multifunctional charging station participating in peak load regulation and the peak load regulation operation constraints, a peak load regulation auxiliary service decision model is established using a recurrent neural network, and the peak load regulation auxiliary service decision model is trained by a reinforcement learning algorithm. The acquisition of the energy storage charging and discharging strategy includes the following steps:

[0051] S31, pre-processing the historical electricity consumption data of the photovoltaic storage and charging multifunctional charging station to obtain a basic peak-shaving data set, and establishing a peak-shaving auxiliary service decision model based on a recurrent neural network;

[0052] S32. Based on the reinforcement learning algorithm, according to the cost objective function of the photovoltaic storage and charging multifunctional charging station participating in peak load regulation and the peak load regulation operation constraints, the basic peak load regulation data set is used to train the peak load regulation auxiliary service decision model;

[0053] S33. Based on the trained peak-shaving auxiliary service decision model, the peak-shaving target and environmental information are input, and the energy storage charging and discharging strategy is output.

[0054] Furthermore, based on the reinforcement learning algorithm, according to the cost objective function of the photovoltaic storage and charging multifunctional charging station participating in peak load regulation and the peak load regulation operation constraints, the peak load regulation auxiliary service decision model is trained using the basic peak load regulation data set, including the following steps:

[0055] S321. Establish a state space vector of a peak-shaving auxiliary service decision model based on photovoltaic power generation, charging load power, time-of-use electricity price, energy storage system charging time, state of charge, and peak-shaving response price;

[0056] S322, based on the public grid exchange power and the energy storage system charge and discharge power, establish the action space vector of the peak load auxiliary service decision model, and use the energy storage system charge and discharge constraints to optimize the action space vector;

[0057] S323, calculating the state transition probability according to the state space vector and the action space vector of the peak load auxiliary service decision model;

[0058] S324. Based on the carbon emission reduction benefits, electricity purchase and sales costs, and charging service revenue of the photovoltaic storage and charging multifunctional charging station, a profit reward is constructed, and a penalty function is established according to the power balance constraints and the upper and lower limits of the energy storage system reserves. The profit reward and penalty function are integrated to obtain the reward function of the peak-shaving auxiliary service decision-making model.

[0059] The beneficial effects of the present invention are:

[0060] (1) The present invention utilizes the photovoltaic, storage and charging multifunctional charging station facilities, takes into account the system operation and maintenance costs, operating benefits, carbon emission reduction benefits and the purchase and sale costs of electricity for charging services, and optimizes the internal resource scheduling strategy based on the reinforcement learning algorithm. It can adapt to the ever-changing market environment and user electricity consumption behavior by continuously learning and updating price strategies, and realize the optimization of the peak-shaving cost-effectiveness of the photovoltaic, storage and charging multifunctional charging station.

[0061] (2) The present invention proposes a decision-making method for the participation of a photovoltaic, storage and charging multifunctional charging station in peak-shaving auxiliary services based on reinforcement learning. The photovoltaic, storage and charging station is used as a unit to participate in peak-shaving. When establishing the cost objective function, multiple influencing factors such as operation and maintenance costs, carbon emission reduction benefits, charging service income, electricity purchase and sales costs, peak-shaving response income and peak-shaving deviation penalty fees are fully considered, thereby realizing a systematic evaluation of the market cost of charging stations participating in peak-shaving. At the same time, by constructing power balance constraints, peak-shaving capacity constraints, energy storage system charging and discharging constraints, electric vehicle charging constraints and rotating reserve capacity constraints, the safe and economical operation of the photovoltaic, storage and charging multifunctional charging station is ensured. The peak-shaving auxiliary service decision-making model established based on the recurrent neural network, combined with the training optimization process of the reinforcement learning algorithm, can effectively improve the accuracy and adaptability of the decision, and ultimately obtain the optimal energy storage charging and discharging strategy.

[0062] (3) The present invention establishes a complete cost objective function, comprehensively considers operation and maintenance costs, operational benefits and carbon emission reduction benefits, and solves the problem that existing methods only focus on economic benefits and inaccurate cost estimates; by constructing a multi-dimensional operation constraint system, it achieves comprehensive guarantee for the safe and stable operation of the photovoltaic storage and charging multifunctional charging station; by combining recurrent neural networks and reinforcement learning algorithms, it establishes and trains a peak-shaving auxiliary service decision-making model, which effectively solves the shortcomings of existing decision-making methods in cost-benefit analysis, significantly improves the enthusiasm of photovoltaic storage and charging multifunctional charging stations to participate in peak-shaving auxiliary services, and provides strong support for them to play a greater regulatory role in the power system. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0064] Figure 1 It is a flowchart of a method for making a decision on a photovoltaic storage and charging multifunctional charging station participating in a peak load auxiliary service based on reinforcement learning according to an embodiment of the present invention;

[0065] Figure 2 It is a schematic diagram of peak load response price in a decision method for a photovoltaic storage and charging multifunctional charging station to participate in a peak load auxiliary service based on reinforcement learning according to an embodiment of the present invention;

[0066] Figure 3 It is a schematic diagram of a reinforcement learning training process in a method for making a decision based on reinforcement learning for a photovoltaic storage and charging multifunctional charging station to participate in a peak load auxiliary service according to an embodiment of the present invention;

[0067] Figure 4It is a schematic diagram of energy storage charging and discharging strategy in a decision method for a photovoltaic storage and charging multifunctional charging station to participate in a peak load auxiliary service based on reinforcement learning according to an embodiment of the present invention;

[0068] Figure 5 It is an algorithm flow chart of a method for making decisions on a photovoltaic storage and charging multifunctional charging station participating in peak load auxiliary services based on reinforcement learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0069] To further illustrate each embodiment, the present invention provides drawings, which are part of the disclosure of the present invention and are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in the field should be able to understand other possible implementations and advantages of the present invention. The components in the figures are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0070] According to an embodiment of the present invention, a decision-making method for a photovoltaic storage and charging multifunctional charging station to participate in peak load auxiliary services based on reinforcement learning is provided.

[0071] The present invention is further described with reference to the accompanying drawings and specific embodiments. Figure 1 As shown, according to one embodiment of the present invention, a method for making a decision on a photovoltaic storage and charging multifunctional charging station to participate in a peak-shaving auxiliary service based on reinforcement learning is provided. The method for making a decision on a photovoltaic storage and charging multifunctional charging station to participate in a peak-shaving auxiliary service based on reinforcement learning comprises the following steps:

[0072] S1. Based on the photovoltaic operation and maintenance costs and energy storage operation and maintenance costs, the operating power and carbon emissions of different types of facilities are analyzed to obtain the operation and maintenance costs and carbon emission reduction benefits. Through a comprehensive analysis of charging service revenue, electricity purchase and sales costs, peak-shaving response revenue and peak-shaving deviation penalty fees, a cost objective function for the participation of photovoltaic storage and charging multifunctional charging stations in peak-shaving is established;

[0073] S2. Based on different types of facilities and photovoltaic storage and charging multifunctional charging stations, establish power balance constraints and peak-shaving capacity constraints, and construct energy storage system charging and discharging constraints and electric vehicle charging constraints. Through integrated analysis with spinning reserve capacity constraints, obtain the peak-shaving operation constraints of photovoltaic storage and charging multifunctional charging stations;

[0074] S3. According to the cost objective function of the multifunctional charging station for photovoltaic storage and charging participating in peak load regulation and the peak load regulation operation constraints, a peak load regulation auxiliary service decision model is established using a recurrent neural network, and the peak load regulation auxiliary service decision model is trained through a reinforcement learning algorithm to obtain the energy storage charging and discharging strategy.

[0075] In one embodiment, according to the photovoltaic operation and maintenance costs and energy storage operation and maintenance costs, the operating power and carbon emissions of different types of facilities are analyzed to obtain the operation and maintenance costs and carbon emission reduction benefits, and through a comprehensive analysis of charging service revenue, electricity purchase and sales costs, peak-shaving response revenue and peak-shaving deviation penalty fees, a cost objective function for the participation of a photovoltaic storage and charging multifunctional charging station in peak-shaving is established, including the following steps:

[0076] S11. Based on the photovoltaic operation and maintenance cost and the energy storage operation and maintenance cost, the fixed purchase cost of the multifunctional charging station is solved, and the operation and maintenance cost of the photovoltaic storage multifunctional charging station is obtained through a comprehensive analysis of the fixed purchase cost and the variable dynamic cost;

[0077] Specifically, based on the photovoltaic operation and maintenance cost and the energy storage operation and maintenance cost, the fixed purchase cost of the multifunctional charging station is solved, and the operation and maintenance cost of the photovoltaic storage multifunctional charging station is obtained through a comprehensive analysis of the fixed purchase cost and the variable dynamic cost. The following steps are included:

[0078] S111. Calculate the photovoltaic operation and maintenance cost based on the photovoltaic system, and obtain the fixed purchase cost of the photovoltaic storage and charging multifunctional charging station by analyzing the fixed cost and variable cost of energy storage operation and maintenance;

[0079] Specifically, the photovoltaic operation and maintenance costs are calculated based on the photovoltaic system, and the fixed purchase cost of the photovoltaic storage and charging multifunctional charging station is obtained by analyzing the fixed cost and variable cost of energy storage operation and maintenance. The following steps are included:

[0080] S1111. Calculate the photovoltaic operation and maintenance cost based on the unit operation and maintenance cost and maximum power of the photovoltaic system and the discount rate of photovoltaic equipment;

[0081] Specifically, the expression of photovoltaic operation and maintenance cost is:

[0082]

[0083] In the formula, C PV,OM The cost of photovoltaic operation and maintenance; PV is the unit operation and maintenance cost of the photovoltaic system; is the maximum photovoltaic power; r PV is the discount rate of photovoltaic equipment; PV The service life of photovoltaic equipment.

[0084] S1112. Calculate the fixed cost of energy storage operation and maintenance based on the unit capacity operation and maintenance cost of energy storage and the rated capacity, combined with the discount rate of energy storage equipment;

[0085] Specifically, the energy storage operation and maintenance cost consists of two parts: the fixed cost of energy storage operation and maintenance and the variable cost of energy storage operation and maintenance. Among them, the variable cost of energy storage operation and maintenance is mainly affected by the charging and discharging amount of the energy storage system, that is, it changes with the change of charging and discharging amount.

[0086] Specifically, the expression of energy storage operation and maintenance fixed cost is:

[0087]

[0088] S1113. Calculate the variable cost of energy storage operation and maintenance based on the charging power and discharging power of the energy storage system, and analyze the energy storage operation and maintenance cost in combination with the fixed cost of energy storage operation and maintenance;

[0089] Specifically, the expression of variable cost of energy storage operation and maintenance is:

[0090]

[0091] Specifically, the expression of energy storage operation and maintenance cost is:

[0092]

[0093] In the formula, C ES,OM is the operation and maintenance cost of battery energy storage; θ E is the operation and maintenance cost per unit capacity of energy storage; is the rated capacity of energy storage; r ES is the discount rate of energy storage equipment; ES is the service life of the energy storage device; θ ES is the dynamic cost per unit of electricity; is the charging power of energy storage during period t; is the discharge power of the energy storage during period t.

[0094] S1114. Based on the photovoltaic operation and maintenance cost and the energy storage operation and maintenance cost, solve the fixed purchase cost of the multi-functional charging station.

[0095] The expression for the fixed acquisition cost of a multifunctional charging station is:

[0096] C in =C PV,OM +C ES,OM

[0097] In the formula, C in Fixed acquisition cost.

[0098] S112. Calculate the variable dynamic cost of the photovoltaic storage and charging multifunctional charging station based on different facility types;

[0099] S113. Calculate the operation and maintenance cost of the photovoltaic-storage-charging multifunctional charging station based on the fixed purchase cost and variable dynamic cost of the photovoltaic-storage-charging multifunctional charging station.

[0100] Specifically, the operation and maintenance costs of the photovoltaic storage and charging multifunctional charging station are mainly composed of the maintenance costs of key subsystems (including photovoltaic systems and energy storage systems in this embodiment). The size of these costs mainly depends on the operating power status of each subsystem, equipment depreciation and service life.

[0101] Specifically, the expression of the operation and maintenance cost of the photovoltaic storage and charging multifunctional charging station is:

[0102] C iOM =C in +C op

[0103]

[0104] C in =C PV,OM +C ES,OM

[0105] In the formula, C iOM The operation and maintenance cost of the photovoltaic storage and charging multifunctional charging station; C in The fixed purchase cost of the photovoltaic storage multifunctional charging station; C op The variable dynamic cost of a multifunctional charging station for photovoltaic storage and charging; θ i t is the variable dynamic cost for different facility types; P i t is the dynamic power of different types of facilities; Δt is the scheduling period.

[0106] S12. Calculate the carbon reduction benefits of different types of facilities based on the operating power of photovoltaic power generation, battery energy storage, public power grid and charging system, combined with the corresponding carbon emissions analysis;

[0107] Specifically, the carbon emission reduction benefit of electric vehicle charging is reflected in the fact that after the charging station provides charging services for electric vehicles, the CO2 emissions reduced compared to fuel vehicles. 2 The carbon emission reduction benefit of charging piles is calculated by multiplying the emissions by the price determined by the carbon emission trading market; similarly, the carbon emission reduction benefit of photovoltaic power generation is calculated by comparing the carbon emissions generated by coal-fired power generation under the same amount of electricity, and then multiplying it by the price of the carbon emission trading market.

[0108] Specifically, according to the operating power of photovoltaic power generation, battery energy storage, public power grid and charging system, combined with the corresponding carbon emissions analysis, the carbon emission reduction benefits of different types of facilities are calculated, including the following steps:

[0109] S121. Based on the photovoltaic power generation power and the carbon emissions of coal-fired power generation with the same power, calculate the carbon emission reduction benefits for photovoltaic power generation;

[0110] Specifically, the expression for the carbon emission reduction benefits of photovoltaic power generation is:

[0111] C 1O2 =P PV Δt·Q PV κ t

[0112] Where P PV is the photovoltaic power generation power; Q PV is the carbon emissions from coal-fired power generation under the same amount of electricity; κ t is the carbon trading price; Δt is the scheduling period.

[0113] S122. Calculate the carbon emission reduction benefits of battery energy storage based on the battery energy storage power and the carbon emissions of coal burning for battery energy storage at the same power level;

[0114] Specifically, the expression for the carbon emission reduction benefits of battery energy storage is:

[0115] C 2O2 =P ES Δt·Q ES κ t

[0116] Where P ES is the battery storage power; Q ES is the carbon emissions generated by coal combustion with battery energy storage under the same amount of electricity; t is the carbon trading price; Δt is the scheduling period.

[0117] S123. Calculate the carbon emission reduction benefits for the public power grid based on the load power of the public power grid and the carbon emissions of coal-fired power generation under the same power consumption;

[0118] Specifically, the expression for the carbon emission reduction benefits of the public power grid is:

[0119] C 3O2 =P PG Δt·Q PG κ t

[0120] Where P PG is the load power of the public power grid; Q PG is the carbon emission of coal-fired power generation under the same amount of electricity; κ t is the carbon trading price; Δt is the scheduling period.

[0121] S124. Calculate the carbon emission reduction benefits of the charging pile based on the charging load power and the carbon emissions of fuel at the same power level.

[0122] Specifically, the expression for the carbon emission reduction benefits of charging piles is:

[0123] C 4O2 =P EV Δt·Q EV κ t

[0124] Where P EV is the charging load power; Q EV is the carbon emissions generated by fuel under the same mileage; t is the carbon trading price; Δt is the scheduling period.

[0125] Specifically, the total carbon emission reduction benefit is the sum of the carbon emission reduction benefits of different types of facilities, expressed as:

[0126] C iO2 =C 1O2 +C 2O2 +C 3O2 +C 4O2

[0127] S13. Calculate the charging service income based on the charging fee and the charging load power, analyze the purchase and sale costs of the charging service, and solve the peak response income according to the adjustment amount of the photovoltaic storage and charging multifunctional integrated charging station in different time periods;

[0128] Specifically, the expression of charging service revenue is:

[0129]

[0130] In the formula, ρ s (t) is the charging cost of the electric vehicle in the t period; P EV (t) is the charging load power of electric vehicles in the tth period; Δt is the scheduling period.

[0131] Specifically, the cost of electricity purchase and sale mainly includes two parts: the purchase of electricity from the power grid and the sale of electricity to users. These two parts together constitute the main components of the cost of electricity purchase and sale of charging stations, which have an important impact on the operating economic benefits of charging stations.

[0132] Specifically, the cost of purchasing and selling electricity for charging services is expressed as:

[0133]

[0134] In the formula, Γ t sell is the electricity price for on-grid electricity sales during period t; Γ t buy P is the electricity price for purchasing electricity in period t; EG is the exchange power between the charging station and the public power grid. In this embodiment, it is negative when the charging station purchases electricity and positive when it sells electricity; Δt is the scheduling period.

[0135] Specifically, the expression of peak load response income is:

[0136] C pr =ΔP pr ·Δt·χ

[0137] ΔP pr =|P′ EG -P EG |

[0138] In the formula, C pr The peak response income of the photovoltaic storage and charging multifunctional integrated charging station in the t period; ΔP pr is the regulation amount of the t-th period of the photovoltaic storage and charging multifunctional integrated charging station; χ is the peak load response price; P EG '(t) is the actual value of the power supply of the public power grid; P EG (t) is the predicted value of the public power grid power supply.

[0139] S14. Calculate the penalty fee for peak-shaving deviation by using the deviation assessment segmentation threshold, and establish the cost objective function for the participation of the photovoltaic storage and charging multifunctional charging station in peak-shaving by combining the operation and maintenance cost of the photovoltaic storage and charging multifunctional charging station, the carbon emission reduction benefits of different types of facilities, the charging service income, the electricity purchase and sales cost of the charging service and the peak-shaving response income.

[0140] Specifically, in the peak-shaving market, if the actual response volume fails to meet the standard, the integrated photovoltaic storage and charging station will face the expenditure of response deviation penalty fees. In order to reasonably regulate this fee, a segmented setting method is adopted: when the deviation value is lower than the segment threshold, the penalty fee will decrease significantly according to the quadratic function relationship. Specifically for the photovoltaic storage and charging multifunctional charging station, the calculation method of its response deviation penalty fee is based on the above principle.

[0141] Specifically, the expression of the peak load deviation penalty fee is:

[0142]

[0143] W SD =|P EG '(t)-P″ EG (t)|Δt

[0144] R SD =μ·ΔP pr

[0145] In the formula, C SD Penalty fee for peak load deviation; c U1 and c U2 W is the deviation penalty cost coefficient of the photovoltaic storage and charging multifunctional integrated charging station; SD is the peak load deviation power; R SD is the deviation assessment segmentation threshold; P″EG (t) is the planned value of the public power grid power supply; μ is the response deviation assessment ratio.

[0146] Specifically, the cost objective function minF of the photovoltaic storage and charging multifunctional charging station participating in peak load regulation is expressed as:

[0147]

[0148] Where i represents different types of facilities (in this embodiment, it includes four types: photovoltaic power generation, battery energy storage, public power grid, and charging system); T is the scheduling period (in this embodiment, a day is divided into 24 time periods, that is, T = 24); C SD Penalty fee for peak load deviation; C pr The peak load response income of the photovoltaic storage and charging multifunctional integrated charging station in the t period; C iOM The operation and maintenance cost of the photovoltaic storage and charging multifunctional charging station; C io2 is the total carbon emission reduction benefit; C tb is the operating income of charging service in period t; C tv The cost of purchasing and selling electricity for charging service in period t.

[0149] In one embodiment, based on different types of facilities and photovoltaic storage and charging multifunctional charging stations, power balance constraints and peak load capacity constraints are established, and energy storage system charging and discharging constraints and electric vehicle charging constraints are constructed. Through integrated analysis with spinning reserve capacity constraints, peak load operation constraints of photovoltaic storage and charging multifunctional charging stations are obtained, including the following steps:

[0150] S21. Establishing power balance constraints based on the power supply of the public grid, the power of photovoltaic power generation, and the power of energy storage charging and discharging;

[0151] Specifically, the expression of power balance constraint is:

[0152] P EG (t)+P PV (t)+P ES (t) = P EV (t)+P RE (t)+P EM (t)

[0153] Where P EG (t) is the power supply of the public power grid; P PV (t) is the photovoltaic power generation power; P ES (t) is the energy storage charging and discharging power (discharging is positive, charging is negative); P EV (t) is the charging load power; P RE (t) is the spinning reserve capacity power; P EM (t) is the station power load.

[0154] S22. Constructing a peak-shaving capacity constraint according to the available capacity of the photovoltaic storage and charging multifunctional charging station in the peak-shaving process;

[0155] Specifically, the available capacity of the photovoltaic storage and charging multifunctional charging station during the peak load regulation process is constrained by a series of internal resource response capabilities.

[0156] Specifically, the expression of peak load capacity constraint is:

[0157]

[0158] In the formula, is the maximum response power of facility i (here the facilities are mainly four types: photovoltaic power generation, battery energy storage, public power grid, and charging system); is the basic power consumption of facility i; is the minimum value of the power consumption of facility i; is the maximum value of the power consumption of facility i.

[0159] S23. Based on the energy storage capacity change equation, establish the energy storage system charge and discharge constraints;

[0160] Specifically, based on the energy storage capacity change equation, establishing the energy storage system charging and discharging constraints includes the following steps:

[0161] S231, establishing an energy storage capacity change equation according to the initial state of the energy storage system and the rated capacity of the battery;

[0162] Specifically, the expression of the energy storage capacity change equation is:

[0163]

[0164] In the formula, SOC 0 is the initial state of the energy storage system; E ES is the rated capacity of the battery; P(t) is the charging and discharging power of the energy storage battery at the tth moment.

[0165] S232, solving the energy storage capacity change equation to obtain the charge state of the energy storage system;

[0166] S233. Construct charging and discharging constraints for the energy storage system based on the charging and discharging power of the energy storage system and the state of charge of the energy storage system.

[0167] Specifically, the expression of the energy storage system charging and discharging constraint is:

[0168] -P ES,rate ≤P ES (t)≤P ES,rate

[0169] SOCmin ≤SOC(t)≤SOC max

[0170] Where P ES (t) is the charging and discharging power of the energy storage system; P ES,rate is the rated power of the energy storage system; SOC(t) is the state of charge of the energy storage system; SOC max SOC is the upper limit of the state of charge of the energy storage system; min It is the lower limit of the charge state of the energy storage system. Usually the battery takes SOC min =0.1~0.2, SOC max = 0.8 ~ 0.9. At the same time, in order to ensure that charging and discharging can be done at the beginning, the initial value of the remaining power SOC is usually taken 0 =0.5~0.6.

[0171] S24. Constructing charging constraints for electric vehicles by analyzing random load demands and dispatchable load demands of electric vehicles;

[0172] Specifically, the electric vehicle charging load includes two parts: random load and dispatchable load, which must meet the following constraints:

[0173] P EV (t) = P EV1 (t)+P EV2 (t)

[0174] P E ' V (t) = P E ' V1 (t)+P E ' V2 (t)

[0175]

[0176] 0≤E≤E max

[0177] Where P EV1 is the random load demand; P EV2 is the dispatchable load demand; P′ EV1 is the predicted value of random load demand; P′ EV2 is the forecast value of dispatchable load demand; P ES (t) is the load transferred at time t for charging electric vehicles; S max is the change in power caused by the transfer of loads within the scheduling period. In this embodiment, S max =0 means that all the reduced loads are transferred to other time periods, and the power consumption does not change during the dispatch cycle; E is the power of the dispatchable load; E 0 is the amount of electricity that can be dispatched; Emax It is the maximum amount of electricity that can be transferred by the dispatchable load.

[0178] S25. Based on the requirements of power supply reliability, a spinning reserve capacity constraint is established, and by integrating it with the power balance constraint, peak-shaving capacity constraint, energy storage system charging and discharging constraint, and electric vehicle charging constraint, the peak-shaving operation constraint of the photovoltaic storage and charging multifunctional charging station is obtained.

[0179] Specifically, the spinning reserve capacity constraint is mainly to meet the requirements for power supply reliability under load forecast errors or system failure conditions.

[0180] Specifically, the expression of the spinning reserve capacity constraint is:

[0181] P RE =r%·P EV (t)

[0182] Where r% is the spinning reserve capacity factor.

[0183] In one embodiment, according to the cost objective function of the photovoltaic storage and charging multifunctional charging station participating in peak load regulation and the peak load regulation operation constraints, a peak load regulation auxiliary service decision model is established using a recurrent neural network, and the peak load regulation auxiliary service decision model is trained by a reinforcement learning algorithm. The acquisition of the energy storage charging and discharging strategy includes the following steps:

[0184] S31, pre-processing the historical electricity consumption data of the photovoltaic storage and charging multifunctional charging station to obtain a basic peak-shaving data set, and establishing a peak-shaving auxiliary service decision model based on a recurrent neural network;

[0185] Specifically, in the operation and management of the photovoltaic storage and charging multifunctional charging station, the reinforcement learning method is used to optimize the internal resource scheduling strategy, such as Figure 5 As shown, the present invention adapts to the ever-changing market environment and user electricity consumption behavior by continuously learning and updating the price strategy.

[0186] Specifically, the peak-shaving management architecture of the photovoltaic storage and charging multifunctional charging station based on reinforcement learning includes: Based on a recurrent neural network, the historical electricity consumption data of the internal proxy resources of the charging station is collected, covering the user responses under different electricity prices and environments. After preprocessing, a high-quality sample set is formed to improve the generalization ability of the model. The selected samples are trained offline to build a peak-shaving auxiliary service decision model that predicts the peak-shaving response of internal resources. The model learns the user's sensitivity to price and environmental changes, providing a basis for the formulation of peak-shaving strategies.

[0187] S32. Based on the reinforcement learning algorithm, according to the cost objective function of the photovoltaic storage and charging multifunctional charging station participating in peak load regulation and the peak load regulation operation constraints, the basic peak load regulation data set is used to train the peak load regulation auxiliary service decision model;

[0188] Specifically, based on the response optimization decision driven by reinforcement learning, the photovoltaic storage and charging multifunctional charging station selects actions according to the peak-shaving target and environmental information through the decision-making process of the peak-shaving auxiliary service decision model, and makes an over-limit judgment to determine the reward value function; this is the core of reinforcement learning strategy formulation, which aims to maximize long-term cumulative returns; finally, the system saves operating data, conducts result analysis, evaluates the effectiveness of the pricing strategy, and adjusts and improves the reinforcement learning model to adapt to the complex and dynamic market environment in the future.

[0189] Specifically, based on the reinforcement learning algorithm, according to the cost objective function of the photovoltaic storage and charging multifunctional charging station participating in peak load regulation and the peak load regulation operation constraints, the peak load regulation auxiliary service decision model is trained using the basic peak load regulation data set, including the following steps:

[0190] S321. Establish a state space vector of a peak-shaving auxiliary service decision model based on photovoltaic power generation, charging load power, time-of-use electricity price, energy storage system charging time, state of charge, and peak-shaving response price;

[0191] Specifically, the state space is defined; the state space vector S is the feedback signal about the state change that the peak-shaving auxiliary service decision model obtains from the environment when making behavior selection. In detail, this vector reflects the changes in the environmental state of the photovoltaic storage and charging multifunctional integrated charging station represented by the peak-shaving auxiliary service decision model under different behaviors, providing an important basis for the subsequent decision-making of the peak-shaving auxiliary service decision model.

[0192] Specifically, the state space vector expression of the peak load auxiliary service decision model is:

[0193] S=[P PV (t),P EV (t),Γ(t),SOC,χ(t)]

[0194] Where P PV (t) is the photovoltaic power generation power; P EV (t) is the charging load power; Γ(t) is the time-of-use electricity price; Δt c represents the charging time of the energy storage system; SOC is the state of charge of the energy storage system; χ(t) is the peak load response price.

[0195] S322, based on the public grid exchange power and the energy storage system charge and discharge power, establish the action space vector of the peak load auxiliary service decision model, and use the energy storage system charge and discharge constraints to optimize the action space vector;

[0196] Specifically, the action space is defined; when the peak-shaving auxiliary service decision model of the deep Q network observes the information in the state space, it will act according to the current strategy set θ tSelect an action in the action space. For the photovoltaic storage and charging multifunctional charging station, it mainly exchanges power with the public power grid and charges and discharges power of the battery energy storage system.

[0197] Specifically, the expression of the action space vector of the peak load auxiliary service decision model is:

[0198] A=[P EG ,P ES ]

[0199] In order to reduce the computational complexity, the action space vector A can be optimized, the energy storage output is set as the action space, and the action space range is limited by the constraint condition, thereby optimizing the action space vector. The expression is:

[0200]

[0201] In the formula, a t It is the energy storage output action in period t.

[0202] S323, calculating the state transition probability according to the state space vector and the action space vector of the peak load auxiliary service decision model;

[0203] Specifically, given the current state S and taking action a, the possibility or probability of transferring to the next state S' is calculated. The expression of the state transition probability P is:

[0204]

[0205] In the formula, is the possibility or probability of transitioning from state S to the next state S′; S t+1 is the state signal of the next moment; s′ is the next state signal; S t is the state signal at time t; A t Take energy storage output action a at time t.

[0206] S324. Based on the carbon emission reduction benefits, electricity purchase and sales costs, and charging service revenue of the photovoltaic storage and charging multifunctional charging station, a profit reward is constructed, and a penalty function is established according to the power balance constraints and the upper and lower limits of the energy storage system reserves. The profit reward and penalty function are integrated to obtain the reward function of the peak-shaving auxiliary service decision-making model.

[0207] Specifically, with the economic operation of the photovoltaic, storage and charging multifunctional charging station as the goal, the cost of purchasing and selling electricity, the carbon emission reduction benefits and the charging service income obtained by the photovoltaic, storage and charging station are taken as optimization targets, and C is the income reward of the photovoltaic, storage and charging multifunctional charging station.

[0208] Specifically, the expression of the revenue reward C is:

[0209]

[0210] Where Γ(t) is the time-of-use electricity price per unit of electricity in the public power grid; T is the total number of time periods corresponding to the time period; P x is the power purchased by the system in the i-th time period; ρ s (t) is the charging cost of the electric vehicle in the t period; P EV (t) is the charging load power in the tth period; I t SD is the peak regulation deviation.

[0211] Specifically, during the iteration process, each variable should satisfy the system-level balance constraints. A penalty function is set to include power and electricity balance constraints, and the energy storage system should not be overcharged or over-discharged. Therefore, the upper and lower limit penalties of the energy storage system reserves are added to the penalty function.

[0212] Specifically, the expression of the penalty function D is:

[0213] D=D p +D SOC =λ(|d p,i |+|d SOC,i |)

[0214] Where D p Penalty for unbalanced power in the power system; D SOC is the penalty for over-discharging or over-charging the energy storage system; λ is the penalty for the unit difference in power; d p,i is the unbalanced amount of power system in the i-th time period; d SOC,i is the over-discharge or over-charge amount of the energy storage system in the i-th period.

[0215] Specifically, by integrating the revenue reward and penalty functions, we can obtain the reward function R of the peak load auxiliary service decision model, which is expressed as:

[0216] R=-(C+D)

[0217] S33. Based on the trained peak-shaving auxiliary service decision model, the peak-shaving target and environmental information are input, and the energy storage charging and discharging strategy is output.

[0218] In order to facilitate understanding of the above technical solution of the present invention, a specific description is given below by taking a photovoltaic storage and charging multifunctional charging station in a certain place as an example:

[0219] The present invention selects the data of a photovoltaic storage and charging multifunctional charging station system in a certain place for detailed simulation research. The charging station system is equipped with a 600kW photovoltaic power generation equipment and a battery energy storage system with a rated capacity of 160kW / 320kWh. In addition, the system also includes 30 DC charging piles with a rated power of 30kW, with a total charging capacity of up to 900kW. Through this configuration, the system has a dispatchable charging load of 200kWh, which provides strong data support for verifying the peak-shaving strategy.

[0220] like Figure 2 As shown, according to the conventional load power consumption characteristics, the present invention considers the peak power consumption period (8:00-12:00, 17:00-21:00) in a day in the above calculation example, the early morning (0:00-8:00) is the valley period, and the other periods are flat periods. The selected peak-shaving period is the afternoon period (12:00-17:00), which corresponds to the upward peak-shaving period, and the evening period (21:00-24:00), which corresponds to the downward peak-shaving period. The peak-valley period peak-shaving price setting is shown in Table 1.

[0221] Table 1 Peak-valley period peak-shaving price setting

[0222] Time period Peak shaving response price 0:00-8:00 (valley section) 175 8:00-12:00 (peak period) 195 12:00-17:00 (peak period) 195 17:00-21:00 (peak period) 195 21:00-24:00 (Flat section) 175

[0223] Based on the basic peak-shaving data, the present invention starts reinforcement learning training. In this process, in order to stabilize the model parameter update, the software update factor is set to 0.01. In the optimization stage, the efficient Adam optimizer is used to improve the accuracy and convergence speed of model training. After careful settings, the model completed 2000 rounds of training.

[0224] In order to explore the performance of the action network in terms of learning efficiency and greediness, the present invention conducted six targeted trainings to find the best balance between learning and exploration. Figure 3 As shown in the figure, the peak-shaving auxiliary service decision model behavior in the peak-shaving period reinforcement learning training. In the early stage, the peak-shaving auxiliary service decision model is highly exploratory and the curve fluctuates greatly; as the training deepens, the peak-shaving auxiliary service decision model gradually learns to choose the optimal action, and the operating income is significantly improved. Finally, the model converges, providing a stable decision-making basis for peak-shaving auxiliary services.

[0225] In the peak load auxiliary service market, the present invention uses deep reinforcement learning technology to develop a charging and discharging strategy for the energy storage system, such as Figure 4As shown. The adjustment of energy storage charging and discharging strategy should be based on the change of electricity demand and the fluctuation of electricity price. During the non-peak period, that is, from 5:00 am to 11:00 am and from 21:00 pm to 1:00 am the next day, due to the relatively flat electricity demand, the energy storage system can maintain a stable energy state and hardly perform charging and discharging operations. During the low electricity price period, that is, from 1:00 am to 5:00 am and from 12:00 pm to 16:00 pm, the energy storage system actively captures the opportunity of low-priced electricity and performs charging operations to reserve energy for the electricity demand during the peak period. When entering the peak electricity price period, that is, from 11:00 am to 12:00 pm and from 16:00 pm to 21:00 pm, the energy storage system will start the discharge mode in time to meet the electricity demand during the peak period. Through this charging and discharging strategy, the energy storage system can not only effectively reduce the operating cost, but also improve the power supply reliability of the power grid and achieve the optimal allocation of power resources.

[0226] In summary, with the help of the above-mentioned technical scheme of the present invention, by utilizing the photovoltaic storage and charging multifunctional charging station facilities, considering its system operation and maintenance costs, operating benefits, carbon emission reduction benefits and the purchase and sale costs of electricity for charging services, and based on the reinforcement learning algorithm, optimizing the internal resource scheduling strategy, it is possible to adapt to the ever-changing market environment and user electricity consumption behavior by continuously learning and updating the price strategy, and realize the optimization of the peak-shaving cost-effectiveness of the photovoltaic storage and charging multifunctional charging station; the present invention proposes a decision-making method for the participation of photovoltaic storage and charging multifunctional charging stations in peak-shaving auxiliary services based on reinforcement learning, and takes the photovoltaic storage and charging charging station as a unit to participate in peak-shaving. When establishing the cost objective function, it comprehensively considers multiple influencing factors such as operation and maintenance costs, carbon emission reduction benefits, charging service income, purchase and sale costs of electricity, peak-shaving response income and peak-shaving deviation penalty fees, and realizes a systematic evaluation of the market cost of charging stations participating in peak-shaving. At the same time, by constructing power and electricity balance constraints, peak-shaving capacity constraints, energy storage system charging and discharging constraints, electric vehicle charging constraints and rotating standby capacity constraints, the safe and economical operation of the photovoltaic storage and charging multifunctional charging station is ensured. The peak-shaving auxiliary service decision-making model established based on the recurrent neural network, combined with the training optimization process of the reinforcement learning algorithm, can effectively improve the accuracy and adaptability of the decision, and ultimately obtain the optimal energy storage charging and discharging strategy; the present invention establishes a complete cost objective function to comprehensively consider the operation and maintenance costs, operating benefits and carbon emission reduction benefits, thereby solving the problem that the existing methods only focus on economic benefits and inaccurate cost estimates; by constructing a multi-dimensional operation constraint system, comprehensive guarantees are achieved for the safe and stable operation of the photovoltaic storage and charging multifunctional charging station; by combining the recurrent neural network and the reinforcement learning algorithm, a peak-shaving auxiliary service decision-making model is established and trained, which effectively solves the shortcomings of the existing decision-making methods in cost-benefit analysis, significantly improves the enthusiasm of the photovoltaic storage and charging multifunctional charging station to participate in the peak-shaving auxiliary service, and provides strong support for it to play a greater regulatory role in the power system.

[0227] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A decision-making method for a photovoltaic storage and charging multifunctional charging station to participate in peak load auxiliary service based on reinforcement learning, characterized in that: The decision-making method of the photovoltaic storage and charging multifunctional charging station participating in the peak load auxiliary service based on reinforcement learning includes the following steps: S1. Based on the photovoltaic operation and maintenance costs and energy storage operation and maintenance costs, the operating power and carbon emissions of different types of facilities are analyzed to obtain the operation and maintenance costs and carbon emission reduction benefits. Through a comprehensive analysis of charging service revenue, electricity purchase and sales costs, peak-shaving response revenue and peak-shaving deviation penalty fees, a cost objective function for the participation of photovoltaic storage and charging multifunctional charging stations in peak-shaving is established; S2. Based on different types of facilities and photovoltaic storage and charging multifunctional charging stations, establish power balance constraints and peak-shaving capacity constraints, and construct energy storage system charging and discharging constraints and electric vehicle charging constraints. Through integrated analysis with spinning reserve capacity constraints, obtain the peak-shaving operation constraints of photovoltaic storage and charging multifunctional charging stations; S3. According to the cost objective function of the multifunctional charging station for photovoltaic storage and charging participating in peak load regulation and the peak load regulation operation constraints, a peak load regulation auxiliary service decision model is established using a recurrent neural network, and the peak load regulation auxiliary service decision model is trained through a reinforcement learning algorithm to obtain the energy storage charging and discharging strategy.

2. According to the reinforcement learning-based decision-making method for the photovoltaic storage and charging multifunctional charging station to participate in the peak load auxiliary service, the method is characterized in that: According to the photovoltaic operation and maintenance costs and energy storage operation and maintenance costs, the operating power and carbon emissions of different types of facilities are analyzed to obtain the operation and maintenance costs and carbon emission reduction benefits, and through a comprehensive analysis of charging service income, electricity purchase and sales costs, peak-shaving response income and peak-shaving deviation penalty fees, the cost objective function of the photovoltaic storage and charging multifunctional charging station participating in peak-shaving is established, which includes the following steps: S11. Based on the photovoltaic operation and maintenance cost and the energy storage operation and maintenance cost, the fixed purchase cost of the multifunctional charging station is solved, and the operation and maintenance cost of the photovoltaic storage multifunctional charging station is obtained through a comprehensive analysis of the fixed purchase cost and the variable dynamic cost; S12. Calculate the carbon reduction benefits of different types of facilities based on the operating power of photovoltaic power generation, battery energy storage, public power grid and charging system, combined with the corresponding carbon emissions analysis; S13. Calculate the charging service income based on the charging fee and the charging load power, analyze the purchase and sale costs of the charging service, and solve the peak response income according to the adjustment amount of the photovoltaic storage and charging multifunctional integrated charging station in different time periods; S14. Calculate the penalty fee for peak-shaving deviation by using the deviation assessment segmentation threshold, and establish the cost objective function for the participation of the photovoltaic storage and charging multifunctional charging station in peak-shaving by combining the operation and maintenance cost of the photovoltaic storage and charging multifunctional charging station, the carbon emission reduction benefits of different types of facilities, the charging service income, the electricity purchase and sales cost of the charging service and the peak-shaving response income.

3. The decision-making method for participating in peak load auxiliary service of a photovoltaic storage and charging multifunctional charging station based on reinforcement learning according to claim 2 is characterized in that: The method of solving the fixed purchase cost of the multifunctional charging station based on the photovoltaic operation and maintenance cost and the energy storage operation and maintenance cost, and obtaining the operation and maintenance cost of the photovoltaic storage and charging multifunctional charging station through comprehensive analysis of the fixed purchase cost and the variable dynamic cost includes the following steps: S111. Calculate the photovoltaic operation and maintenance cost based on the photovoltaic system, and obtain the fixed purchase cost of the photovoltaic storage and charging multifunctional charging station by analyzing the fixed cost and variable cost of energy storage operation and maintenance; S112. Calculate the variable dynamic cost of the photovoltaic storage and charging multifunctional charging station based on different facility types; S113. Calculate the operation and maintenance cost of the photovoltaic-storage-charging multifunctional charging station based on the fixed purchase cost and variable dynamic cost of the photovoltaic-storage-charging multifunctional charging station.

4. The decision-making method for participating in peak load auxiliary service of a photovoltaic storage and charging multifunctional charging station based on reinforcement learning according to claim 3 is characterized in that: The method of calculating the photovoltaic operation and maintenance cost according to the photovoltaic system and obtaining the fixed purchase cost of the photovoltaic storage and charging multifunctional charging station by analyzing the energy storage operation and maintenance fixed cost and the energy storage operation and maintenance variable cost includes the following steps: S1111. Calculate the photovoltaic operation and maintenance cost based on the unit operation and maintenance cost and maximum power of the photovoltaic system and the discount rate of photovoltaic equipment; S1112. Calculate the fixed cost of energy storage operation and maintenance based on the unit capacity operation and maintenance cost of energy storage and the rated capacity, combined with the discount rate of energy storage equipment; S1113. Calculate the variable cost of energy storage operation and maintenance based on the charging power and discharging power of the energy storage system, and analyze the energy storage operation and maintenance cost in combination with the fixed cost of energy storage operation and maintenance; S1114. Based on the photovoltaic operation and maintenance cost and the energy storage operation and maintenance cost, solve the fixed purchase cost of the multi-functional charging station.

5. According to the reinforcement learning-based decision-making method for the participation of a multifunctional photovoltaic storage and charging station in peak load auxiliary services as claimed in claim 4, it is characterized in that: The expression of photovoltaic operation and maintenance cost is: In the formula, C PV,OM The cost of photovoltaic operation and maintenance; PV is the unit operation and maintenance cost of the photovoltaic system; is the maximum photovoltaic power; r PV is the discount rate of photovoltaic equipment; PV The service life of photovoltaic equipment; The expression of the energy storage operation and maintenance cost is: In the formula, C ES,OM is the operation and maintenance cost of battery energy storage; θ E is the operation and maintenance cost per unit capacity of energy storage; is the rated capacity of energy storage; r ES is the discount rate of energy storage equipment; ES is the service life of the energy storage device; θ ES is the dynamic cost per unit of electricity; is the charging power of energy storage during period t; is the discharge power of energy storage during period t; The expression of the fixed acquisition cost of the multifunctional charging station is: C in =C PV,OM +C ES,OM In the formula, C in Fixed acquisition cost.

6. The decision-making method for participating in peak load auxiliary service of a photovoltaic storage and charging multifunctional charging station based on reinforcement learning according to claim 2 is characterized in that: The calculation of the carbon reduction benefits of different types of facilities based on the operating power of photovoltaic power generation, battery energy storage, public power grid and charging system, combined with the corresponding carbon emissions analysis, includes the following steps: S121. Based on the photovoltaic power generation power and the carbon emissions of coal-fired power generation with the same power, calculate the carbon emission reduction benefits for photovoltaic power generation; S122. Calculate the carbon emission reduction benefits of battery energy storage based on the battery energy storage power and the carbon emissions of coal burning for battery energy storage at the same power level; S123. Calculate the carbon emission reduction benefits for the public power grid based on the load power of the public power grid and the carbon emissions of coal-fired power generation under the same power consumption; S124. Calculate the carbon emission reduction benefits of the charging pile based on the charging load power and the carbon emissions of fuel at the same power level.

7. The decision-making method for participating in peak load auxiliary service of a photovoltaic storage and charging multifunctional charging station based on reinforcement learning according to claim 1 is characterized in that: The method of establishing power balance constraints and peak load capacity constraints based on different types of facilities and the photovoltaic storage and charging multifunctional charging station, and constructing energy storage system charging and discharging constraints and electric vehicle charging constraints, and obtaining peak load operation constraints of the photovoltaic storage and charging multifunctional charging station through integrated analysis with the spinning reserve capacity constraints includes the following steps: S21. Establishing power balance constraints based on the power supply of the public grid, the power of photovoltaic power generation, and the power of energy storage charging and discharging; S22. Constructing a peak-shaving capacity constraint according to the available capacity of the photovoltaic storage and charging multifunctional charging station in the peak-shaving process; S23. Based on the energy storage capacity change equation, establish the energy storage system charge and discharge constraints; S24. Constructing charging constraints for electric vehicles by analyzing random load demands and dispatchable load demands of electric vehicles; S25. Based on the requirements of power supply reliability, a spinning reserve capacity constraint is established, and by integrating it with the power balance constraint, peak-shaving capacity constraint, energy storage system charging and discharging constraint, and electric vehicle charging constraint, the peak-shaving operation constraint of the photovoltaic storage and charging multifunctional charging station is obtained.

8. The method for decision-making of a photovoltaic storage and charging multifunctional charging station participating in peak load auxiliary service based on reinforcement learning according to claim 7 is characterized in that: The establishment of the energy storage system charge and discharge constraints based on the energy storage quantity change equation includes the following steps: S231, establishing an energy storage capacity change equation according to the initial state of the energy storage system and the rated capacity of the battery; S232, solving the energy storage capacity change equation to obtain the charge state of the energy storage system; S233. Construct charging and discharging constraints for the energy storage system based on the charging and discharging power of the energy storage system and the state of charge of the energy storage system.

9. The decision-making method for participating in peak load auxiliary service of a photovoltaic storage and charging multifunctional charging station based on reinforcement learning according to claim 7 is characterized in that: According to the cost objective function of the multifunctional charging station for photovoltaic storage and charging participating in peak load regulation and the peak load regulation operation constraints, a peak load regulation auxiliary service decision model is established by using a recurrent neural network, and the peak load regulation auxiliary service decision model is trained by a reinforcement learning algorithm to obtain the energy storage charging and discharging strategy, which includes the following steps: S31, pre-processing the historical electricity consumption data of the photovoltaic storage and charging multifunctional charging station to obtain a basic peak-shaving data set, and establishing a peak-shaving auxiliary service decision model based on a recurrent neural network; S32. Based on the reinforcement learning algorithm, according to the cost objective function of the photovoltaic storage and charging multifunctional charging station participating in peak load regulation and the peak load regulation operation constraints, the basic peak load regulation data set is used to train the peak load regulation auxiliary service decision model; S33. Based on the trained peak-shaving auxiliary service decision model, the peak-shaving target and environmental information are input, and the energy storage charging and discharging strategy is output.

10. The method for decision-making of a photovoltaic storage and charging multifunctional charging station participating in peak load auxiliary service based on reinforcement learning according to claim 9 is characterized in that: Based on the reinforcement learning algorithm, according to the cost objective function of the photovoltaic storage and charging multifunctional charging station participating in peak load regulation and the peak load regulation operation constraints, the peak load regulation auxiliary service decision model is trained using the basic peak load regulation data set, including the following steps: S321. Establish a state space vector of a peak-shaving auxiliary service decision model based on photovoltaic power generation, charging load power, time-of-use electricity price, energy storage system charging time, state of charge, and peak-shaving response price; S322, based on the public grid exchange power and the energy storage system charge and discharge power, establish the action space vector of the peak load auxiliary service decision model, and use the energy storage system charge and discharge constraints to optimize the action space vector; S323, calculating the state transition probability according to the state space vector and the action space vector of the peak load auxiliary service decision model; S324. Based on the carbon emission reduction benefits, electricity purchase and sales costs, and charging service revenue of the photovoltaic storage and charging multifunctional charging station, a profit reward is constructed, and a penalty function is established according to the power balance constraints and the upper and lower limits of the energy storage system reserves. The profit reward and penalty function are integrated to obtain the reward function of the peak-shaving auxiliary service decision-making model.

Images