Virtual power plant dual-target reinforcement learning scheduling method and system based on carbon capture

Abstract

The invention provides a double-target reinforcement learning scheduling method and system for a virtual power plant based on carbon capture. The method comprises the following steps: establishing a scheduling model optimization target function for the virtual power plant containing a carbon capture power plant and P2G equipment; constructing a double-target virtual power plant optimization scheduling algorithm based on a PPO algorithm; and a Pareto optimization algorithm is utilized, and the dual-objective function is optimized by dynamically adjusting weight parameters. The system comprises an objective function algorithm module used for establishing a scheduling model optimization objective function for a virtual power plant containing a carbon capture power plant and P2G equipment; the PPO algorithm module is used for constructing a dual-target virtual power plant optimization scheduling algorithm based on a PPO algorithm; and the Pareto optimization algorithm module is used for optimizing the dual-objective function by dynamically adjusting the weight parameters.

Description

Technical Field

[0001] The present invention belongs to the field of virtual power plants, and in particular relates to a dual-objective reinforcement learning scheduling optimization method and system for virtual power plants based on carbon capture. Background Art

[0002] The transformation of energy consumption structure, the development and utilization of new energy, and the decarbonization of energy utilization have become widely discussed and urgent topics. In order to cope with the increasingly severe energy and environmental challenges, distributed energy has attracted people's attention due to its high efficiency and cleanliness. Virtual power plant (VPP) is a comprehensive distributed power system that combines various types of distributed energy sources. By utilizing the complementarity of multiple energy sources, VPP promotes the integration of renewable energy and overcomes the challenges brought by the diverse number, dispersed location and intermittent power generation of distributed energy sources. This enhances their competitiveness in the energy market.

[0003] Power-to-gas (P2G) technology is an effective way to convert excess renewable energy into natural gas storage, achieve carbon emission reduction and renewable energy integration. The carbon dioxide used as raw material in this process comes from carbon capture power plants (CCPPs), which have been transformed with carbon capture and storage technology. The study proposed a VPP optimization scheduling model based on P2G and hierarchical carbon trading, and introduced an incentive mechanism for P2G to participate in the carbon market. In addition, a flexible operation model within a VPP based on CCPP is proposed, which includes P2G and solar thermal power plants, and relieves peak load pressure by utilizing solar thermal power generation.

[0004] However, most of the existing literature focuses on the CCPP-P2G collaborative operation framework in the electricity-gas interconnection system, without considering the coupling of multiple energy forms such as cold, heat, and electricity. On the other hand, under the background of the dual-carbon strategy, the traditional optimization model that simply pursues economic goals is no longer sufficient to meet the optimization requirements. In order to achieve a unified optimization result, it is necessary to incorporate carbon emissions into the optimization scheduling model. At present, more and more researchers use intelligent optimization algorithms to solve multi-objective optimization problems in VPPs. However, when solving these problems, heuristic algorithms often face challenges such as slow convergence, falling into local optimality, and difficulty in balancing multiple objectives. In contrast, reinforcement learning (RL) has shown greater effectiveness in solving such problems. Reinforcement learning has been successfully applied to many multi-objective optimization problems. However, research on using RL for dual-objective optimization in VPPs is still limited, especially when integrating two key objectives: economic efficiency and low carbon dioxide emissions. At the same time, in multi-objective optimization, the Pareto weights of each objective need to be determined before optimization, but there may be problems such as inaccurate weight setting or human preference. The accuracy and objectivity of the weight value cannot be guaranteed. At the same time, it is too cumbersome to adjust the weight value multiple times to find the optimal solution. How to determine the optimal weight value is an urgent problem to be solved.

[0005] The existing technologies are mainly divided into the following two types: Technical Document 1 (CN117436588A, 2024.01.23) proposes a low-carbon park optimization configuration method, which minimizes the comprehensive cost and carbon emissions through a two-layer planning model. The upper model aims to minimize costs and takes into account the investment and operating costs of equipment such as carbon capture, electrolyzers, and hydrogen storage tanks; the lower model aims to minimize operating costs and uses operating expenses such as carbon trading costs as constraints to obtain a low-carbon operation plan, thereby achieving the dual goals of reducing costs and carbon emissions. Technical Document 2 (CN114865631B, 2022.09.20) proposes a comprehensive energy system scheduling method for source-load coordinated carbon reduction, constructs a carbon emission model, and establishes a joint operation framework and demand response mechanism on the source side and the load side respectively. The distributed robust optimization method is applied to achieve a balance between robustness and economy, realize the multi-objective scheduling of virtual power plants, and improve the ability to cope with complex environments.

[0006] The present invention greatly improves the dispatching capability through multiple technical innovations and enhances the adaptability and stability of the system in a complex energy environment. First, the present invention introduces a dynamic adjustment weight value ω i The mechanism enables the system to adapt to the changing operating environment in real time, which not only improves the scheduling efficiency, but also enhances the system's ability to respond quickly to emergencies, especially in the case of energy supply fluctuations or demand surges, ensuring the continuity and reliability of energy supply. Secondly, the present invention adopts a dual-objective reinforcement learning algorithm to achieve an efficient balance between economic and carbon emission targets. Unlike traditional methods that can only optimize a single method, the algorithm achieves a balance between economic benefits and environmental responsibilities through continuous learning and optimization, further promoting the low-carbon and sustainable development of the system. Finally, based on the distributionally robust optimization theory of non-precise probabilistic opportunity constraints, the present invention overcomes the subjective limitations of traditional distributionally robust optimization in the scheduling scheme, comprehensively considers the robustness and economy of the scheduling scheme, obtains the optimal scheduling scheme under complex conditions, and achieves a balance between economy and robustness. Through the above innovations, the present invention effectively improves the multi-objective scheduling capability of the virtual power plant under a changing environment, enabling the system to achieve low-carbon, efficient and stable optimized scheduling in a complex energy environment. Summary of the invention

[0007] In the multi-energy optimization scheduling of VPP, traditional research focuses on the coordinated operation of the electricity-gas interconnected system, without fully considering the coupling complexity of multiple energy forms such as electricity, gas, heat, and cold. At the same time, under the background of the dual-carbon strategy, the optimization model that only pursues economic goals can no longer meet actual needs. The balance between economic benefits and carbon emission minimization has become a key challenge. In the existing multi-objective optimization methods, weight setting relies on subjective experience, which is difficult to ensure accuracy and objectivity, and the process of adjusting weights multiple times is also cumbersome and inefficient. In addition, traditional heuristic algorithms face problems such as slow convergence, easy to fall into local optimality, and poor target balance when solving multi-objective optimization problems. Although reinforcement learning (RL) has shown certain advantages in multi-objective optimization, research on the dual-objective optimization of economy and low carbon in VPP is still scarce. To this end, this study proposes the BiPPO algorithm, which takes the minimization of operating costs and carbon emissions as the core goals, introduces a dynamic weight adjustment mechanism, and automatically adjusts the target weights during the optimization process to avoid the inaccuracy of human settings. At the same time, by introducing carbon capture technology and P2G equipment, it absorbs excess output of new energy, reduces gas purchase costs, and further optimizes economic benefits. The BiPPO algorithm effectively combines the dynamic adaptability of reinforcement learning and the advantages of multi-objective optimization. It dynamically adjusts the optimization weight values ​​to reduce constraint inconsistency, improves optimization efficiency, and ultimately achieves unified optimization of the supply and demand balance and dual objectives of the multi-energy system.

[0008] In addition, this study proposed a dynamic learning strategy based on an adaptive feedback mechanism in the algorithm design, which can autonomously optimize the adjustment rate and direction of the weight value according to the environmental changes in different optimization stages, thereby further improving the robustness and global search capabilities of the algorithm, especially in solving the volatility and uncertainty of new energy in the energy system. The dynamic optimization strategy can more efficiently respond to real-time changes in energy demand and carbon emission targets. In addition, combined with the deep integration of carbon capture and P2G equipment, this study explored a new model for new energy consumption and internal resource recycling, and provided a systematic technical solution for achieving the coordination of economy and low carbon under the "dual carbon" goal. This innovative framework not only promotes the advancement of VPP optimization methods in theory, but also provides a new idea for multi-objective optimization scheduling problems in practical applications.

[0009] The present invention adopts the following technical solution.

[0010] The present invention provides a dual-objective reinforcement learning scheduling method for a virtual power plant based on carbon capture, comprising:

[0011] Establish a dispatch model to optimize the objective function for a virtual power plant containing carbon capture power plants and P2G equipment;

[0012] Construct a dual-objective virtual power plant optimization scheduling algorithm based on the PPO algorithm;

[0013] The Pareto optimization algorithm is used to optimize the dual objective function by dynamically adjusting the weight parameters.

[0014] Preferably, the objective function includes:

[0015] Objective function 1: Minimize the economic cost of the virtual power plant,

[0016]

[0017] Among them, C grid and C gas Cost of purchasing electricity and gas for the virtual power plant; C CO2 is the carbon trading cost; C F is the fuel cost of the carbon capture power plant; C CS is the carbon sequestration cost; C P2G is the cost of P2G equipment; C K Equipment operation and maintenance costs for virtual power plants;

[0018] Objective function 2: Minimize carbon emissions of virtual power plants,

[0019]

[0020] Among them, E c is the actual carbon emissions of the virtual power plant, δ is the carbon emission coefficient of purchased electricity, P t grid Refers to the amount of electricity purchased from the grid at time t.

[0021] Preferably, the construction of a dual-objective virtual power plant optimization scheduling algorithm based on the PPO algorithm includes:

[0022] Set the state space of the system,

[0023] S=[P PV ,P WT ,P load ,Q c,load ,Q load ,S bat ,S st ,P buy ,P sell ]

[0024] Where P PV is the photovoltaic output, P WT is the fan output, P load is the power load, Q c,load is the cooling load, Q load is the heat load, S bat is the battery status, S st is the carbon capture output power, P buy ,P sellThe buying and selling prices of time-of-use electricity prices;

[0025] Set the action space of the system,

[0026] A=[P MT ,P FC ,P grid-buy ,P grid-sell ,P load ,Q AC ]

[0027] Where P MT is the gas turbine output, P FC is the output of the fuel cell, P grid-buy and P grid-sell is the amount of electricity purchased and sold from the grid, P load is the load demand, Q AC Thermoelectricity required for users and absorption chillers.

[0028] Preferably, the construction of a dual-objective virtual power plant optimization scheduling algorithm based on the PPO algorithm also includes:

[0029] Reward Function

[0030] Among them, ω i is the target weight, i=1,2, and f is the target function.

[0031] Preferably, the construction of a dual-objective virtual power plant optimization scheduling algorithm based on the PPO algorithm also includes:

[0032] The optimal action space A is sampled using the random strategy π(a|s), where a and s refer to the action space and state space at a random moment, and the generalized advantage function is:

[0033]

[0034] in represents the real expected return, B i (s) represents the expected return;

[0035] During training, the PPO algorithm uses the random policy π(a|s) while collecting trajectories τ from their interactions with the environment. The training is performed according to the following expression:

[0036]

[0037] where Φ p are the policy network parameters;

[0038] Value Network The training is performed according to the following expression:

[0039]

[0040] Among them, θ v is the parameter of the value network, V(s t ) is the true expected value function, is through the parameter θ v The estimated i-th value function, s t Indicates the current state of t.

[0041] Preferably, the optimizing the dual objective function by dynamically adjusting weight parameters using the Pareto optimization algorithm includes:

[0042]

[0043] Where η represents H i The parameters of (η), the target weight ω i The optimal solution is

[0044]

[0045] in, H i The gradient of (η).

[0046] Preferably, the optimizing the dual objective function by dynamically adjusting the weight parameters using the Pareto optimization algorithm also includes:

[0047] The projection process is introduced to adjust the preliminary optimization solution to the feasible solution set containing all constraints, the objective function of minimizing economic cost and the objective function of minimizing carbon emissions are re-evaluated, and the final optimization solution is calculated by non-negative least squares method.

[0048] The present invention also provides a dual-objective reinforcement learning scheduling system for a virtual power plant based on carbon capture, the system being a system used by the aforementioned dual-objective reinforcement learning scheduling method for a virtual power plant based on carbon capture, comprising:

[0049] The objective function algorithm module is used to establish a scheduling model to optimize the objective function of the virtual power plant containing carbon capture power plants and P2G equipment;

[0050] PPO algorithm module, used to build a dual-objective virtual power plant optimization scheduling algorithm based on the PPO algorithm;

[0051] The Pareto optimization algorithm module is used to optimize the dual objective function by dynamically adjusting the weight parameters.

[0052] The present invention also provides a terminal, including a processor and a storage medium;

[0053] The storage medium is used to store instructions;

[0054] The processor is configured to operate according to the instructions to execute the steps according to the aforementioned method.

[0055] The present invention also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the above method are implemented.

[0056] The beneficial effect of the present invention is that, compared with the prior art, the present invention significantly improves the multi-objective optimal scheduling capability of virtual power plants in complex environments through a number of technical innovations. First, by dynamically adjusting the weight value ωi, the system can adapt to complex and changeable operating environments in real time. This flexibility not only improves the scheduling efficiency, but also enhances the ability to respond to emergencies, allowing the system to quickly adjust in the face of energy supply fluctuations or demand surges, ensuring the stability and reliability of energy supply. The dual-objective reinforcement learning algorithm efficiently balances the economic and carbon emission targets. Traditional methods often find it difficult to balance economic benefits and environmental responsibilities, while the reinforcement learning algorithm can achieve the best compromise between the two through learning and optimization under different conditions, promoting sustainable development while improving the economic benefits of the overall system.

[0057] Secondly, by organically combining carbon capture technology with P2G equipment, the system can not only effectively absorb excess renewable energy output, but also convert it into natural gas for storage and internal recycling. This innovation greatly reduces the gas purchase cost of VPP and significantly reduces carbon emissions, fully improving energy utilization efficiency and the economy of the system. Compared with traditional energy management methods, the present invention provides a more environmentally friendly and economical solution, greatly enhancing the system's ability to absorb wind power and photovoltaic power generation.

[0058] In addition, the internal recycling of natural gas generated by abandoned wind and solar power not only significantly reduces energy costs, but also reduces dependence on external energy, further improving the energy self-sufficiency and sustainability of the virtual power plant. This efficient energy management path provides a strong reference for the design and optimization of other energy systems.

[0059] Finally, the present invention monitors the supply and demand status in real time and dynamically adjusts the scheduling strategy to ensure that the supply and demand balance is always within a reasonable range, and combines the supply and demand balance with the dual optimization goals of operating costs and carbon emissions to ensure that the overall performance of the system can be comprehensively improved under complex operating conditions. Through this dual optimization strategy, the virtual power plant can not only meet system requirements, but also maximize economic benefits and environmental protection goals, achieving the unity of stability and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1It is a flow chart of the dual-objective reinforcement learning scheduling method for a virtual power plant based on carbon capture in the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical scheme and advantages of the present invention clearer, the technical scheme of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. The embodiments described in this application are only part of the embodiments of the present invention, not all of them. Based on the spirit of the present invention, other embodiments obtained by ordinary technicians in this field without creative work are all within the scope of protection of the present invention.

[0062] like Figure 1 As shown, embodiment 1 of the present invention provides a dual-objective reinforcement learning scheduling method for a virtual power plant based on carbon capture, comprising:

[0063] Firstly, a scheduling model is established to optimize the objective function of the virtual power plant (VPP) containing carbon capture power plants and P2G equipment.

[0064] Objective function 1: Minimization of VPP economic costs. Minimization of VPP economic costs aims to optimize the scheduling and utilization of various energy resources during the operation of virtual power plants and reduce overall operating costs as much as possible. The main components of economic costs include electricity and gas purchase costs, carbon trading costs, fuel costs of carbon capture power plants, carbon storage costs, P2G equipment costs, and VPP equipment operation and maintenance costs.

[0065]

[0066] Where: C grid and C gas The cost of purchasing electricity and gas for VPP; C CO2 is the carbon trading cost; C F is the fuel cost of the carbon capture power plant; C CS is the carbon sequestration cost; C P2G is the cost of P2G equipment; C K It is the operating and maintenance cost of VPP equipment.

[0067] Objective function 2: Minimize VPP carbon emissions. Minimizing VPP carbon emissions aims to reduce greenhouse gas emissions during system operation by optimizing the production, transmission and use of energy. VPP carbon emissions mainly come from CO2 directly emitted into the atmosphere without carbon capture and storage in CCPP and the carbon generated by purchased electricity. The main sources of carbon emissions include direct emissions from fossil fuel combustion and indirect emissions corresponding to purchased electricity. The objective function can be expressed as:

[0068]

[0069] Where: E c is the actual carbon emission of VPP, δ is the carbon emission coefficient of purchased electricity, P t grid Refers to the amount of electricity purchased from the grid at time t.

[0070] Secondly, a Markov decision process is set for the dual-objective VPP optimization scheduling algorithm.

[0071] (1) State variables. The VPP model used in this algorithm takes the following elements as components of the state space: photovoltaic output, wind turbine output, power load, heat load, cooling load, thermal storage status, battery status, time-of-use electricity price and carbon capture output power. The system state space is:

[0072] S=[P PV ,P WT ,P load ,Q c,load ,Q load ,S bat ,S st ,P buy ,P sell ] (3)

[0073] Where P PV is the photovoltaic output, P WT is the fan output, P load is the power load, Q c,load is the cooling load, Q load is the heat load, S bat is the battery status, S st is the carbon capture output power, P buy ,P sell It is the buying price and selling price of time-of-use electricity.

[0074] (2) Action variables. When VPP observes information in the state space, it selects an action from the action space based on the current policy. The main action space of the algorithm is:

[0075] A=[P MT ,P FC ,P grid-buy ,P grid-sell ,P load ,Q AC ] (4)

[0076] Where P MT is the gas turbine output, P FC is the output of the fuel cell, P grid-buy and P grid-sell is the amount of electricity purchased and sold from the grid, P load is the load demand, Q AC Thermoelectricity required for users and absorption chillers.

[0077] (3) Reward Function

[0078] The objective function of the present invention consists of two parts: economic efficiency and low carbon emissions. The reward function R is defined as:

[0079]

[0080] where ω i is the target weight, i=1,2, f i is the objective function.

[0081] A dual-objective VPP optimization scheduling algorithm based on proximal policy optimization (PPO) is constructed. For the modeling of the action space A, the action network generates a series of Gaussian distributions. These distributions produce corresponding means and standard deviations, and the optimal action space A is sampled using a random strategy π(a|s), where a and s refer to the action space and state space at a random moment. A random strategy is a strategy that determines the selection of actions through probability distribution. Unlike a deterministic strategy, it outputs the probability distribution of actions instead of fixed actions when a state S is given. Here, it represents the adjustment plan corresponding to the current output according to the state of each device. This method solves the instability in the policy gradient algorithm by creating a probability ratio between different action spaces A and then restricting it to a stable range. In this way, the PPO policy update is saved in the trust region. The generalized advantage function can be expressed as:

[0082]

[0083] in represents the real expected return, B i (s) represents the expected return.

[0084] During training, the PPO algorithm uses the random policies π(a|s) while collecting trajectories τ from their interactions with the environment. With the goal of maximizing the reward value, training is performed according to the following expression:

[0085]

[0086] where Φ p is the policy network parameter. The p-th power represents a certain form of operation in the policy probability distribution. The policy network directly outputs the selection probability of each action. The purpose of is to improve the accuracy of state value estimation. The state value refers to the estimated value of the expected cumulative reward that the agent can obtain in the future under a specific state. Here, it refers to the estimated value of the cumulative return under the current device condition. The method of updating the value network parameters is to minimize the mean square error of the value loss:

[0087]

[0088] where θ v is the parameter of the value network. V(s t ) is the true expected value function, is through the parameter θ v The estimated i-th value function, s t Represents the state at the current state t. Finally, the Adam algorithm is used to optimize the parameters through the gradient descent method to obtain the update of the policy network parameters and the value network parameters.

[0089] In order to achieve dual optimization of economy and environment, this study adopts the Pareto optimization algorithm, and jointly optimizes the economic objective function and the environmental objective function through scalarization technology to ensure that the importance of both is fully reflected in the optimization process. The economic goal emphasizes reducing operating costs by optimizing the energy scheduling and resource utilization of VPP, while the environmental goal focuses on reducing greenhouse gas emissions and achieving low-carbon operation. In the optimization framework, these two goals are set as equal core goals, and each round of optimization can adaptively balance the relative priority between economy and environment by dynamically adjusting the weight parameter ωi. The dual-objective optimization algorithm includes two steps: first, updating the model parameters αΦ and αθ, and then updating ωi. These two steps alternate and influence each other to achieve the best effect. In this algorithm, H(i) is defined as the strategy loss and value loss of the dual objectives, where η represents the parameter of H(i). The economic goal directly improves the economy of VPP by reducing the cost of electricity purchase, fuel cost and operation and maintenance cost; the environmental objective function minimizes carbon emissions by optimizing the energy structure (such as giving priority to the scheduling of new energy and introducing carbon capture technology). In order to dynamically balance the impact of these two goals during the optimization process, the optimization of the weight parameter ωi becomes the key. By adjusting ωi, the algorithm can adaptively give higher priority to economic or environmental goals according to the optimization needs of the current stage. For example, when the volatility of new energy is large, it may be prioritized to reduce punitive expenditures related to economic costs, while in scenarios with high carbon emission pressure, the weight adjustment is biased towards environmental goals.

[0090]

[0091] Where η represents H i Considering the Karush-Kuhn-Tucker (KKT) conditions of the model parameters, the optimal solution of ωi is obtained by minimizing H(η):

[0092]

[0093] in, H i The gradient of (η). When this condition is met, ω can be obtained i . However, it is not easy to solve this problem using its quadratic programming form. To this end, a two-step algorithm is proposed as a solver for the Pareto efficiency condition. In the preliminary optimization, the problem is simplified by relaxing the equality constraints, the policy gradients of economic and environmental objectives are decomposed into independent parts, and the optimal solution is preliminarily generated according to the current system state. First, only the equality constraints are considered to relax the problem; then, a valid solution is generated from the feasible set containing all constraints by introducing a projection process.

[0094] The projection process is introduced to adjust the preliminary optimization solution to a feasible solution set that contains all constraints (such as non-negativity and weight constraints). In this process, the relationship between economic and environmental goals is re-evaluated, and the final optimization solution is calculated by non-negative least squares method. This method ensures that the optimization solution achieves low-carbon operation while taking into account economic benefits, so that the priorities of the two goals are dynamically balanced under specific constraints.

[0095]

[0096] This problem is exactly a non-negative least squares problem, which can be easily solved using the active set method. It also provides a more comprehensive evaluation of the interrelationships and mutual influences between objectives. At this point, we can get the optimal weight ωi. During the optimization process, the VPP state is reconfigured through the exploration mechanism of reinforcement learning and action A, the weight parameter ωi is dynamically adjusted, and the results of the two objective functions are optimized. When the total reward R of the optimization result no longer increases or the target state remains unchanged, it is considered that the Pareto optimal solution has been found in the current stage. In this process, the importance of economic goals is reflected in the immediate effect of reducing electricity purchase and operating costs during the optimization process, while the importance of environmental goals provides a guarantee for the long-term sustainable operation of the VPP by continuously reducing carbon emissions. By dynamically adjusting ω i This study not only reasonably balances economic and environmental goals at different times, but also highlights the importance of a certain goal under specific conditions, such as favoring economic efficiency when operating cost pressure is high, and favoring environmental goals when environmental protection policies are strict or the demand for new energy consumption is high. This flexible and dynamic optimization strategy enables VPP to achieve the best balance between economic benefits and low carbon emissions under the dual carbon background.

[0097] Embodiment 2 of the present invention provides a dual-objective reinforcement learning scheduling optimization system for a virtual power plant based on carbon capture. The system is a system used by the aforementioned dual-objective reinforcement learning scheduling optimization method for a virtual power plant based on carbon capture, and includes:

[0098] The objective function algorithm module is used to establish a scheduling model to optimize the objective function of the virtual power plant containing carbon capture power plants and P2G equipment;

[0099] PPO algorithm module, used to build a dual-objective virtual power plant optimization scheduling algorithm based on the PPO algorithm;

[0100] The Pareto optimization algorithm module is used to optimize the dual objective function by dynamically adjusting the weight parameters.

[0101] Example 3

[0102] Dynamic scheduling and optimization of virtual power plants:

[0103] The BiPPO algorithm shows high adaptability in the multi-objective optimal scheduling of virtual power plants, especially for application scenarios with dynamic environmental changes. For example, in areas where the output of renewable energy such as wind and solar energy fluctuates greatly, the BiPPO algorithm can adjust the system weight value ω in real time. i , thus ensuring that the system can still achieve the dual optimality of operating costs and carbon emissions under different load conditions. This feature makes the BiPPO algorithm the preferred solution for the mixed operation of renewable energy power plants and traditional power plants.

[0104] Engineering application example: A coastal city built a virtual power plant that combines wind power, photovoltaic power generation and natural gas power generation. In the case of large fluctuations in wind power generation, the BiPPO algorithm can effectively adjust the system weights, achieve rapid response, and ensure the dual optimization of power generation costs and carbon emissions. Simulation results show that the convergence speed of the BiPPO algorithm under different climatic conditions is more than 30% higher than that of traditional algorithms, significantly reducing the operating costs and carbon emissions of the system.

[0105] Example 4

[0106] Application of carbon capture and P2G technology in energy storage and conversion:

[0107] The integrated application of carbon capture technology and P2G equipment is particularly suitable for areas with insufficient energy consumption capacity, such as remote islands or areas with low power load but high renewable energy production. In these scenarios, the system can convert excess wind power and photovoltaic power generation into natural gas storage, thereby providing energy during low-load periods and reducing carbon emissions through carbon capture technology. This integrated solution is an important and preferred option for the construction of future low-carbon energy systems.

[0108] Engineering application example: In a certain island microgrid system, excess wind energy is converted into natural gas through P2G technology and released during peak hours of electricity consumption, greatly alleviating the problem of imbalance between energy supply and demand on the island. After a year of simulation testing, the system's gas purchase cost was reduced by 20% and carbon emissions were reduced by 15%. This integrated technology provides a sustainable energy solution for islands far from the mainland.

[0109] Example 3

[0110] Internal recycling of abandoned wind and solar natural gas:

[0111] In areas where wind and solar power curtailment is a serious problem, the combination of the BiPPO algorithm and P2G technology can effectively reduce energy waste and achieve energy self-sufficiency through internal recycling of natural gas. This solution is particularly suitable for large industrial parks or remote areas, and can reduce dependence on external energy.

[0112] Engineering application example: An industrial park faced a serious problem of wind curtailment due to excess wind power. Through BiPPO algorithm optimization and combined with P2G equipment, the industrial park achieved internal recycling of natural gas and significantly reduced the purchase of external natural gas. The simulation results showed that energy utilization efficiency increased by 25% and energy costs decreased by 15%.

[0113] The beneficial effect of the present invention is that, compared with the prior art, the present invention significantly improves the multi-objective optimal scheduling capability of virtual power plants in complex environments through a number of technical innovations. First, by dynamically adjusting the weight value ωi, the system can adapt to complex and changeable operating environments in real time. This flexibility not only improves the scheduling efficiency, but also enhances the ability to respond to emergencies, allowing the system to quickly adjust in the face of energy supply fluctuations or demand surges, ensuring the stability and reliability of energy supply. The dual-objective reinforcement learning algorithm efficiently balances the economic and carbon emission targets. Traditional methods often find it difficult to balance economic benefits and environmental responsibilities, while the reinforcement learning algorithm can achieve the best compromise between the two through learning and optimization under different conditions, promoting sustainable development while improving the economic benefits of the overall system.

[0114] Secondly, by organically combining carbon capture technology with P2G equipment, the system can not only effectively absorb excess renewable energy output, but also convert it into natural gas for storage and internal recycling. This innovation greatly reduces the gas purchase cost of VPP and significantly reduces carbon emissions, fully improving energy utilization efficiency and the economy of the system. Compared with traditional energy management methods, the present invention provides a more environmentally friendly and economical solution, greatly enhancing the system's ability to absorb wind power and photovoltaic power generation.

[0115] In addition, the internal recycling of natural gas generated by abandoned wind and solar power not only significantly reduces energy costs, but also reduces dependence on external energy, further improving the energy self-sufficiency and sustainability of the virtual power plant. This efficient energy management path provides a strong reference for the design and optimization of other energy systems.

[0116] Finally, the present invention monitors the supply and demand status in real time and dynamically adjusts the scheduling strategy to ensure that the supply and demand balance is always within a reasonable range, and combines the supply and demand balance with the dual optimization goals of operating costs and carbon emissions to ensure that the overall performance of the system can be comprehensively improved under complex operating conditions. Through this dual optimization strategy, the virtual power plant can not only meet system requirements, but also maximize economic benefits and environmental protection goals, achieving the unity of stability and efficiency.

[0117] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0118] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0119] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0120] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A dual-objective reinforcement learning scheduling method for a virtual power plant based on carbon capture, characterized in that: include: Establish a dispatch model to optimize the objective function for a virtual power plant containing carbon capture power plants and P2G equipment; Construct a dual-objective virtual power plant optimization scheduling algorithm based on the PPO algorithm; The Pareto optimization algorithm is used to optimize the dual objective function by dynamically adjusting the weight parameters.

2. The dual-objective reinforcement learning scheduling method for a virtual power plant based on carbon capture according to claim 1 is characterized by: The objective function includes: Objective function 1: Minimize the economic cost of the virtual power plant, Among them, C grid and C gas Cost of purchasing electricity and gas for the virtual power plant; C CO2 is the carbon trading cost; C F is the fuel cost of the carbon capture power plant; C CS is the carbon sequestration cost; C P2G is the cost of P2G equipment; C K Equipment operation and maintenance costs for virtual power plants; Objective function 2: Minimize carbon emissions of virtual power plants, Among them, E c is the actual carbon emissions of the virtual power plant, δ is the carbon emission coefficient of purchased electricity, P t grid Refers to the amount of electricity purchased from the grid at time t.

3. The dual-objective reinforcement learning scheduling method for a virtual power plant based on carbon capture according to claim 2 is characterized in that: The dual-objective virtual power plant optimization scheduling algorithm based on the PPO algorithm is constructed, including: Set the state space of the system, S=[P PV ,P WT ,P load ,Q c,load ,Q load ,S bat ,S st ,P buy ,P sell ] Where P PV is the photovoltaic output, P WT is the fan output, P load is the power load, Q c,load is the cooling load, Q load is the heat load, S bat is the battery status, S st is the carbon capture output power, P buy ,P sell The buying and selling prices of time-of-use electricity prices; Set the action space of the system, A=[P MT ,P FC ,P grid-buy ,P grid-sell ,P load ,Q AC ] Where P MT is the gas turbine output, P FC is the output of the fuel cell, P grid-buy and P grid-sell is the amount of electricity purchased and sold from the grid, P load is the load demand, Q AC Thermoelectricity required for users and absorption chillers.

4. The dual-objective reinforcement learning scheduling method for a virtual power plant based on carbon capture according to claim 3 is characterized by: The dual-objective virtual power plant optimization scheduling algorithm based on the PPO algorithm also includes: Reward Function Among them, ω i is the target weight, i=1,2, f i is the objective function.

5. The dual-objective reinforcement learning scheduling method for a virtual power plant based on carbon capture according to claim 4 is characterized in that: The dual-objective virtual power plant optimization scheduling algorithm based on the PPO algorithm also includes: The optimal action space A is sampled using the random strategy π(a|s), where a and s refer to the action space and state space at a random moment, and the generalized advantage function is: in represents the real expected return, B i (s) represents the expected return; During training, the PPO algorithm uses the random policy π(a|s) while collecting trajectories τ from their interactions with the environment. The training is performed according to the following expression: where Φ p are the policy network parameters; Value Network The training is performed according to the following expression: Among them, θ v is the parameter of the value network, V(s t ) is the true expected value function, is through the parameter θ v The estimated i-th value function, s t Indicates the current state of t.

6. The dual-objective reinforcement learning scheduling method for a virtual power plant based on carbon capture according to claim 5 is characterized by: The method of optimizing the dual objective function by dynamically adjusting the weight parameters using the Pareto optimization algorithm includes: Where η represents H i The parameters of (η), the target weight ω i The optimal solution is in, H i The gradient of (η).

7. The dual-objective reinforcement learning scheduling method for a virtual power plant based on carbon capture according to claim 6 is characterized by: The method of optimizing the dual objective function by dynamically adjusting the weight parameters using the Pareto optimization algorithm also includes: The projection process is introduced to adjust the preliminary optimization solution to the feasible solution set containing all constraints, the objective function of minimizing economic cost and the objective function of minimizing carbon emissions are re-evaluated, and the final optimization solution is calculated by non-negative least squares method.

8. A dual-objective reinforcement learning scheduling system for a virtual power plant based on carbon capture, the system being a system used in a dual-objective reinforcement learning scheduling method for a virtual power plant based on carbon capture according to any one of claims 1 to 7, comprising: The objective function algorithm module is used to establish a scheduling model to optimize the objective function of the virtual power plant containing carbon capture power plants and P2G equipment; PPO algorithm module, used to build a dual-objective virtual power plant optimization scheduling algorithm based on the PPO algorithm; The Pareto optimization algorithm module is used to optimize the dual objective function by dynamically adjusting the weight parameters.

9. A terminal comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images