New energy power system low-carbon scheduling optimization method based on deep reinforcement learning

Abstract

The invention relates to the technical field of power system dispatching optimization, in particular to a new energy power system low-carbon dispatching optimization method based on deep reinforcement learning. The method comprises the following steps: firstly, constructing a carbon capture unit model and a load side demand response resource model, wherein the load side demand response resource model comprises an excitation type demand response model and a price type demand response model; then, taking the optimal comprehensive cost of the new energy power system as a target function, and determining system imbalance constraints and system balance constraints, so as to construct a demand response and carbon capture coordinated new energy power system low-carbon scheduling model; finally, determining a state space, an action space and a reward function of the intelligent agent; and solving the optimal day-ahead scheduling strategy by applying a deep reinforcement learning method based on a deep deterministic strategy gradient. Scheduling optimization potential is mined from the source side and the load side, the wind and light absorption capacity is improved through an intelligent scheduling strategy, the low-carbon performance of the system is enhanced, and the operation cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of power system dispatch optimization, and specifically to a low-carbon dispatch optimization method for a new energy power system based on deep reinforcement learning. Background Art

[0003] Carbon capture and storage (CCS) technology can significantly reduce CO 2 Emissions have become one of the key technologies to achieve emission reduction targets. However, the carbon capture process will affect the unit output characteristics, causing the unit output power to change synchronously with the carbon capture energy consumption, which undoubtedly increases the complexity and difficulty of power system dispatch. At the same time, although new energy power generation, especially wind power and photovoltaic power, has the advantages of clean, environmentally friendly and carbon-free, its output is highly volatile, which has an adverse effect on the stable operation of the power system and the quality of power.

[0004] In order to meet this challenge, power demand response (DR) as an effective means of load adjustment has gradually attracted widespread attention in the industry. By guiding users to adjust their electricity consumption behavior according to electricity price signals or incentive mechanisms, DR can promote the consumption of renewable energy such as wind power, while achieving the goal of energy conservation and emission reduction. In the optimization methods for wind power uncertainty scheduling, although model-based stochastic optimization, robust optimization and other methods have achieved certain results, there are still problems such as difficulty in modeling environmental uncertainty. The data-driven deep reinforcement learning algorithm avoids the uncertainty modeling of the environment, has strong adaptability to the random volatility of renewable energy, and provides a new idea for the optimization of power system scheduling. Summary of the invention

[0005] In view of the above problems, the purpose of this invention is to provide a low-carbon dispatch optimization method for a new energy power system based on deep reinforcement learning, tap the dispatch optimization potential from both the source and the load sides, improve the wind and solar power absorption capacity through intelligent dispatch strategies, enhance the low-carbon performance of the system, and reduce operating costs. The technical solution is as follows:

[0006] Step 1: construct a carbon capture unit model and a load-side demand response resource model, wherein the load-side demand response resource model includes an incentive-based demand response model and a price-based demand response model;

[0007] Step 2: Based on the carbon capture unit model and the load-side demand response resource model, taking the optimal comprehensive cost of the new energy power system as the objective function, and determining the system imbalance constraint and the system balance constraint, a low-carbon dispatch model of the new energy power system with coordinated demand response and carbon capture is constructed;

[0008] Step 3: Based on the low-carbon dispatch model of the new energy power system, under the reinforcement learning framework, the Markov decision process is used to represent the interaction between the agent and the environment, and the state space, action space and reward function of the agent are determined; then the deep reinforcement learning method based on deep deterministic policy gradient is applied to solve the optimal day-ahead dispatch strategy.

[0009] The beneficial effects of the present invention are as follows: the present invention comprehensively considers the coordination of carbon capture units, dispatchable resource solar thermal power stations, energy storage batteries and wind power on the source side, and considers the characteristics of DR resources on the load side; taps the scheduling optimization potential from both the source and load sides, improves the wind and solar absorption capacity through intelligent scheduling strategies, enhances the low-carbon performance of the system, and reduces operating costs; and uses data-based reinforcement learning methods to better deal with the uncertainty factors in the scheduling process. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is the convergence graph of the DDPG (Deep Deterministic Policy Gradient) algorithm of the present invention.

[0011] Figure 2 This is the scheduling result diagram for Case 1.

[0012] Figure 3 This is the scheduling result diagram for Case 2. DETAILED DESCRIPTION

[0013] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0014] The present invention comprehensively considers the coordination of carbon capture units, dispatchable resource CSP (Concentrating SolarPower CSP), energy storage batteries and wind power on the source side, and considers the characteristics of DR resources on the load side, and proposes a low-carbon dispatch optimization method for new energy power systems based on deep reinforcement learning algorithms. The specific steps are as follows:

[0015] 1. Carbon capture unit and load-side DR resource model

[0016] 1. Carbon capture unit model

[0017] Carbon capture output includes net output and carbon capture energy consumption, and its mathematical model is:

[0018]

[0019] Where: P CCS,i,t , P CGS,i,t , P B,i,t and P CGS,i,maxare the net output, total output power, operating energy consumption and maximum power of carbon capture unit i at time t; P D,i is the fixed energy consumption of the carbon capture unit; e gi and δ i are the carbon emission intensity and flue gas split ratio of carbon capture unit i: E GS,i,t and E CCS,i,t are the CO generated by carbon capture unit i at time t 2 Total amount and total carbon capture; β is carbon capture efficiency: η is the maximum working state coefficient of the regeneration tower and compressor; λ CCS CO captured per unit 2 energy consumption.

[0020] 2. Load-side demand response resource model

[0021] Demand response resources play an important role in the power system. By flexibly adjusting the load at the user end, it can effectively reduce the peak-to-valley difference, alleviate the contradiction between supply and demand, and thus reduce the operating cost of the distribution network. Demand response resources are mainly divided into two types: incentive demand response (IDR) and price demand response (PDR).

[0022] (1) Incentive Demand Response Model

[0023] IDR is a dispatching method in which power companies use policy measures to encourage large users and large loads to sign agreements, and compensate users according to the agreement by changing their electricity consumption habits. The common type of IDR is interruptible load, which directly cuts the appropriate amount of electricity consumption during peak load periods and compensates users.

[0024] The specific expression is:

[0025] P load1,t =P load,t -P IDR,t

[0026] Where: P load,t is the load at time t before IDR; P load1,t is the load at time t after IDR; P IDR,t The amount of load reduction for IDR.

[0027] (2) Price-based demand response model

[0028] The PDR program is a mechanism that encourages electricity consumers to dynamically manage their electricity consumption according to real-time changing electricity prices to reduce peak electricity demand. This demand response mechanism uses time-of-use electricity prices to influence user electricity consumption.

[0029] The electricity price elasticity coefficient model is used to describe the impact of electricity price changes on load changes. The specific expression is:

[0030] λ=E·d -1

[0031] Where: λ is the load change rate matrix, which indicates the impact of electricity price changes on load changes; E is the time-of-use electricity price demand elasticity matrix, which indicates the proportional relationship between load changes and electricity price changes; d is the electricity price change rate matrix, which indicates the changes in electricity prices.

[0032] 2. Low-carbon dispatch model of power system with coordinated demand response and carbon capture

[0033] 1. Objective Function

[0034] The model optimizes the system comprehensive cost as the objective function and establishes a low-carbon dispatch model for the power system. The objective function is as follows:

[0035] F=min(C PV +C WD +C A +C CSP +C ESS +C GS +C CGS +C T +C IDR )

[0036] Where: F is the total cost; C PV is the operating cost of the photovoltaic power station; C WD is the fan operating cost; C A C is the cost of curtailing wind and solar power; CSP is the operating cost of the CSP plant; C ESS is the operating cost of the energy storage battery; C GS is the operating cost of thermal power units; C CGS is the operating cost of the carbon capture unit; C T is the carbon trading cost; C IDR Invoke costs for incentivized demand response.

[0037] Specifically,

[0038]

[0039] Where: k PV , k WD , k CSP and k ESS are the power generation cost coefficients of photovoltaic, wind turbine, solar thermal power station and energy storage battery respectively; α 1 and α 2 They are the penalty coefficients for abandoning solar power and abandoning wind power respectively; and are the wind turbine power generation and photovoltaic power generation at time t; P PV,t , P WD,t , P CSP,t and P ESS,t t is the grid-connected power of photovoltaic, wind turbine, solar thermal power station and energy storage battery at the moment; N is the number of carbon capture units; a ci , b ci and c ci are the coal burning cost coefficients of carbon capture unit i; U Ci,t and S Ci,t are the start-up and shutdown status and start-up cost of carbon capture unit i at time t; M is the number of thermal power units; a j , b j and c j are the coal cost coefficients of thermal power unit j; P GS,j,t is the output of thermal power unit j at time t; U j,t and S j,t are the start-up and shutdown status and start-up cost of thermal power unit j at time t; k T is the carbon trading price coefficient; E c CO generated by thermal power plants 2 Total amount; h is the carbon emission quota coefficient; k IDR Incentive demand response calling cost coefficient; P IDR,t is the load reduction value of the incentive-based demand response at time t.

[0040] 2. Constraints

[0041] The main system imbalance constraints include: capacity constraints of conventional thermal power units, ramp constraints, and start-stop constraints.

[0042] P GS,j,min ≤P GS,j,t ≤P GS,j,max

[0043] P CGS,i,min ≤P CGS,i,t ≤P CGS,i,max

[0044]

[0045] P GS,j,down ≤P GS,j,t -P GS,j,t-1 ≤P GS,j,up

[0046] P CGS,i,down ≤P CGS,i,t -P CGS,i,t-1 ≤P CGS,i,up

[0047] 0≤P IDR,t≤u i,t P IDR,max

[0048] Where: T on,j,t-1 and T off,j,t-1 are the continuous start-up time and continuous shutdown time of thermal power unit j in period t-1; T on,j and T off,j are the minimum start-up time and minimum shutdown time of thermal power unit j respectively; P GS,j,down and P GS,j,up are the downward climbing power and upward climbing power of thermal power unit j respectively; P CGS,i,down and P CGS,i,up are the downward climbing power and upward climbing power of carbon capture unit i respectively; P IDR,max P is the maximum load reduction value of incentive demand response; GS,j,max and P GS,j,min The maximum and minimum output values ​​of thermal power unit j respectively; P CGS,i,max and P CGS,i,min The maximum and minimum output of carbon capture unit i respectively; u i,t is the interruption state of the load at time t.

[0049] System balance constraints include: energy storage battery constraints; CSP power generation system constraints, energy storage system constraints, collector field constraints, PDR constraints and power balance constraints.

[0050]

[0051] Where: SOC t is the state of charge at time t; η cha and η dis E is the charging efficiency and discharging efficiency respectively; m is the battery capacity; P cha,t and P dis,t are the charging power and discharging power at time t respectively; f cha,t and f dis,t They are binary symbols for energy storage charging and discharging, and charging and discharging cannot be performed at the same time; P bes,t is the energy storage charging and discharging power at time t.

[0052]

[0053] q tes,t =q tes,t-1 +η 3 -q st,t -q tp,t

[0054] q sf,t =q sp,t +q tp,t

[0055]

[0056]

[0057] Where: P CSP,t Represents the output of the CSP station; P sp,t , P tp,t are the electric power provided by the heat collection system and the heat storage system at time t; q sp,t ,q tp,t Thermal power output of the system; η 1 and η 2 are the thermoelectric conversion efficiency of the power generation system; q tes,t is the capacity of the heat storage device at time t; q st,t is the thermal power input to the solar collector system at time t; η 3 is the conversion efficiency of the heat storage system; q sf,t is the total thermal power output of the collector field at time t; P t , P PDR,t are the load power before and after PDR, respectively. The total load power before and after PDR remains unchanged. 3. Low-carbon dispatch solution for new energy power system based on Deep Deterministic Policy Gradient (DDPG) reinforcement learning

[0058] In the framework of reinforcement learning, the interaction between the agent and the environment is generally represented by the Markov decision process (MDP), which is used to describe a series of state transition processes and is mainly composed of a five-tuple (S, A, R, P, γ). S represents the state space of the agent; A represents the action space taken by the agent; R represents the reward function; P is the state transition probability; and γ represents the discount factor.

[0059] 1) Determine the state of the environment.

[0060] The state space of deep reinforcement learning usually refers to the physical quantities that the agent can observe in the environment. In this paper, the state space S is expressed as:

[0061]

[0062] Where: P L,t , S PDR,t and S IDR,t They are load power at time t, price-based demand response status, and incentive-based demand response status.

[0063] 2) Determine the agent action.

[0064] The action space of deep reinforcement learning refers to the variables controlled by the agent in the system. The system action space A is expressed as

[0065] a t =[P GS,j,t ,P CGS,i,t ,P PV,t ,P WD,t ,P bes,t ,P csp,t ,P IDR,t ]

[0066] Where: P GS,j,t ,P CGS,i,t ,P PV,t ,P WD,t ,P bes,t ,P csp,t ,P IDR,t They respectively represent the output of thermal power unit j at time t, the total output power of carbon capture unit i at time t, the grid-connected power of photovoltaic power plant at time t, the grid-connected power of wind turbine at time t, the energy storage charging and discharging power at time t, the grid-connected power of solar thermal power station at time t and the load reduction of incentive-type demand response at time t.

[0067] 3) Reward function.

[0068] The low-carbon economic dispatching goal of the wind power power system is to minimize the total system cost. The present invention transforms the total cost minimization problem of the new energy power system into the form of maximizing the cumulative return of deep reinforcement learning rewards, and defines the reward function r in the process of interaction between the intelligent agent and the environment t for

[0069] r t (s t ,a t )=-(F+F punish )

[0070] Where: r t is the agent in state s at time t t Next, perform action a t The reward obtained after punish Punishment for exceeding the limit.

[0071] Once the state space, action space, and reward function are clear, deep reinforcement learning methods can be applied to solve them.

[0072] DDPG solves the optimal day-ahead scheduling strategy as follows:

[0073] Set the number of training rounds according to historical data, set one hour as the time step for the day-ahead scheduling task, and set 24 time steps as one training round.

[0074] Initialize the neural network and use random network parameters ω and θ to initialize the Critic network Q ω(s,a), Actor network μ θ (s) and noise η.

[0075] Copy the same parameters in the Critic network and the Actor network, and initialize the target network separately.

[0076] The agent selects a random scheduling strategy in the action space and determines the next state s t+1 , the selection formula is as follows:

[0077] a t =μ θ (s t )+η

[0078] The experience (s t ,a t ,r t ,s t+1 ) is stored in the experience pool R.

[0079] K samples are randomly collected from the experience pool R. The sample collection formula is as follows:

[0080] {(s k ,a k ,r k ,s k+1 )} k＝1,...,K

[0081] The target network is used to calculate the Q value of each sample. The calculation formula is as follows:

[0082]

[0083] Where: y k is the target Q value of the kth action state sample, r k is the instant reward of the k-th action state sample, γ is the discount factor, is the next state s of the kth action state sample k+1 The Q value when taking action a.

[0084] The loss function is calculated based on the Q value of each sample and the Critic network is updated. The calculation formula is as follows:

[0085]

[0086] Among them, L(ω) is the update loss function, Q ω (s k , a k ) is the current state s k Take action a k Q value at

[0087] Calculate the gradient of the Actor network and update the Actor network. The calculation formula is as follows:

[0088]

[0089] in, represents the derivation of strategy J, Indicates the gradient of the Actor network, θ is the derivative parameter, Indicates the derivation of the Q value, a is the state s k The action taken by the Actor main network.

[0090] Update the target network. The calculation formula for updating the target network is as follows:

[0091]

[0092] Among them, ω - , ω are the parameters of the Critic target network and the main network respectively, θ - , θ are the Actor target network and main network parameters respectively, and σ is the soft update parameter.

[0093] The above steps are repeated until convergence reaches the preset accuracy, and the result of the current round of training is obtained. After the training is completed, the parameters are saved for real-time scheduling to obtain the scheduling result.

[0094] 4. Example Analysis

[0095] When transforming thermal power units, dispatchable resources CSP and energy storage batteries are introduced to form a low-carbon dispatch model of the power system with dual demand-side response together with non-dispatchable resources wind power and photovoltaics.

[0096] This article sets up four case studies to build models:

[0097] 1) Equipped with conventional thermal power units and carbon capture units, without considering DR resources;

[0098] 2) Equipped with conventional thermal power generation units for capture, taking into account PDR and IDR.

[0099] like Figure 1 As shown in the figure, the algorithm begins to converge after about 600 rounds of training and obtains the optimal scheduling result.

[0100] The unit dispatch results of Case 1 and Case 2 are as follows: Figure 2 and Figure 3 , and as shown in Table 1.

[0101] Table 1 Scheduling results for different cases

[0102] project Case 1 Case 2 Carbon emissions / t 4400 2900 Amount of wind and solar power abandoned / % 10 0 Comprehensive cost / yuan 601000 581000

[0103] After the introduction of carbon capture units, the efficiency of wind and solar energy consumption has been greatly improved and optimized. This remarkable progress is mainly due to the coordinated work of carbon capture units and wind and solar energy output, which together constitute the main output source of the power system. In this process, traditional pure thermal power units have gradually transformed into auxiliary roles, which not only promotes the better consumption of wind and solar energy, but also effectively promotes the transformation and development of the entire power system towards low carbonization.

[0104] In Case 2, the power system innovatively implemented a dual demand-side response strategy based on the combined effects of price demand response (PDR) and incentive demand response (IDR) on the user side (load side). During the low-power consumption period from 01:00 to 03:00 in the middle of the night, the IDR mechanism successfully reduced some non-essential loads through economic incentives, providing valuable output opportunities for low-carbon and environmentally friendly generators. This load reduction approach is essentially equivalent to introducing a zero-carbon emission virtual power source to replace the high-carbon emission units that were originally required to operate for power generation. As the power of the replaced high-carbon units gradually increases, the number of starts and stops is also reduced accordingly, thereby effectively reducing carbon emissions.

[0105] At the same time, the PDR mechanism further smoothes the load curve by guiding users to transfer load power for free during a specific period of time. Unlike the paid load reduction of IDR, PDR relies more on the guidance of price signals, encouraging users to increase electricity consumption when electricity prices are low and reduce electricity consumption when electricity prices are high, so as to achieve flexible adjustment of load. When PDR and IDR are used in conjunction with each other, the two can produce significant synergistic effects, making the load curve smoother, while also further expanding the substitution effect of zero-carbon emission virtual power on high-carbon units, which is fully in line with the core concept of low-carbon development of the power system and deeply taps the low-carbon potential of the system.

[0106] From the data comparison in Table 1, we can clearly see that Case 2 successfully achieved full consumption of wind power within the dispatch cycle by considering and implementing the dual demand-side response strategies of PDR and IDR. Compared with Case 1, Case 2 not only achieved a significant reduction in carbon emissions, but also reduced comprehensive costs, which fully demonstrated the effectiveness of this strategy in promoting the low-carbonization of the power system, improving energy efficiency and reducing operating costs.

Claims

1. A low-carbon dispatch optimization method for a new energy power system based on deep reinforcement learning, characterized in that: The following steps are involved: Step 1: construct a carbon capture unit model and a load-side demand response resource model, wherein the load-side demand response resource model includes an incentive-based demand response model and a price-based demand response model; Step 2: Based on the carbon capture unit model and the load-side demand response resource model, taking the optimal comprehensive cost of the new energy power system as the objective function, and determining the system imbalance constraint and the system balance constraint, a low-carbon dispatch model of the new energy power system with coordinated demand response and carbon capture is constructed; Step 3: Based on the low-carbon dispatch model of the new energy power system, under the reinforcement learning framework, the Markov decision process is used to represent the interaction between the agent and the environment, and the state space, action space and reward function of the agent are determined; then the deep reinforcement learning method based on deep deterministic policy gradient is applied to solve the optimal day-ahead dispatch strategy.

2. The low-carbon dispatch optimization method for new energy power system based on deep reinforcement learning according to claim 1 is characterized in that: The step 1 is specifically as follows: Step 1.1: Build a carbon capture plant model: Carbon capture output includes net output and carbon capture energy consumption, and its mathematical model is: Where: P CCS,i,t , P CGS,i,t , P B,i,t and P CGS,i,max are the net output, total output power, operating energy consumption and maximum power of carbon capture unit i at time t; P D,i is the fixed energy consumption of the carbon capture unit; e gi and δ i are the carbon emission intensity and flue gas split ratio of carbon capture unit i; E GS,i,t and E CCS,i,t are the total amount of CO2 produced and the total amount of carbon captured by carbon capture unit i at time t, respectively; β is the carbon capture efficiency; η is the maximum working state coefficient of the regeneration tower and compressor; λ CCS is the energy consumption per unit CO2 captured; Step 1.2: Construct load-side demand response resource model: The type of incentive-based demand response is interruptible load, that is, directly reducing power consumption during peak load periods and compensating users. The specific expression is: P load1,t =P load,t -P IDR,t Where: P load,t is the load at time t before the incentive demand response; P load1,t is the load at time t after the incentive demand response; P IDR,t is the load reduction at time t for the incentive demand response; Step 1.3: Construct a price-based demand response model: The price-based demand response mechanism uses time-of-use electricity prices to influence user electricity consumption. The electricity price elasticity coefficient model is used to describe the impact of electricity price changes on load changes. The specific expression is: λ=E·d -1 Where: λ is the load change rate matrix, which indicates the impact of electricity price changes on load changes; E is the time-of-use electricity price demand elasticity matrix, which indicates the proportional relationship between load changes and electricity price changes; d is the electricity price change rate matrix, which indicates the changes in electricity prices.

3. The low-carbon dispatch optimization method for new energy power system based on deep reinforcement learning according to claim 2 is characterized in that: Step 2 is as follows: Step 2.1: Determine the objective function of the low-carbon dispatch model of the new energy power system as: F=min(C PV +C WD +C A +C CSP +C ESS +C GS +C CGS +C T +C IDR ) Where: F is the total cost; C PV is the operating cost of the photovoltaic power station; C WD is the fan operating cost; C A C is the cost of curtailing wind and solar power; CSP is the operating cost of the CSP plant; C ESS is the operating cost of the energy storage battery; C GS is the operating cost of thermal power units; C CGS is the operating cost of the carbon capture unit; C T is the carbon trading cost; C IDR The cost of incentivized demand response is as follows: Where: k PV , k WD , k CSP and k ESS are the power generation cost coefficients of photovoltaic, wind turbine, solar thermal power station and energy storage battery respectively; α1 and α2 are the penalty coefficients for abandoning solar power and abandoning wind respectively; and are the wind turbine power generation and photovoltaic power generation at time t; P PV,t , P WD,t , P CSP,t and P ESS,t t is the grid-connected power of photovoltaic, wind turbine, solar thermal power station and energy storage battery at the moment; N is the number of carbon capture units; a ci 、b ci and c ci are the coal burning cost coefficients of carbon capture unit i; U Ci,t and S Ci,t are the start-up and shutdown status and start-up cost of carbon capture unit i at time t; M is the number of thermal power units; a j 、b j and c j are the coal cost coefficients of thermal power unit j; P GS,j,t is the output of thermal power unit j at time t; U j,t and S j,t are the start-up and shutdown status and start-up cost of thermal power unit j at time t; k T is the carbon trading price coefficient; E c is the total amount of CO2 produced by thermal power units; δ h is the carbon emission quota coefficient; k IDR Incentive demand response calling cost coefficient; P IDR,t is the load reduction value of the incentive-based demand response at time t; Step 2.2: Determine the system imbalance constraints, including the capacity constraints, ramp constraints and start-stop constraints of conventional thermal power units, and the capacity constraints and ramp constraints of carbon capture units are shown in the following formulas: P GS,j,min ≤P GS,j,t ≤P GS,j,max P CGS,i,min ≤P CGS,i,t ≤P CGS,i,max P GS,j,down ≤P GS,j,t -P GS,j,t-1 ≤P GS,j,up P CGS,i,down ≤P CGS,i,t -P CGS,i,t-1 ≤P CGS,i,up 0≤P IDR,t ≤u i,t P IDR,max Where: T on,j,t-1 and T off,j,t-1 are the continuous start-up time and continuous shutdown time of thermal power unit j in period t-1 respectively; T on,j and T off,j are the minimum start-up time and minimum shutdown time of thermal power unit j respectively; P GS,j,down and P GS,j,up are the downward climbing power and upward climbing power of thermal power unit j respectively; P CGS,i,down and P CGS,i,up are the downward climbing power and upward climbing power of carbon capture unit i respectively; P IDR,max The maximum load reduction value for incentivized demand response; P GS,j,max and P GS,j,min The maximum and minimum output values ​​of thermal power unit j respectively; P CGS,i,max and P CGS,i,min The maximum and minimum output of carbon capture unit i respectively; u i,t is the interruption state of the load at time t; Step 2.3: Determine the system balance constraints including: energy storage battery constraints; power generation system constraints of the CSP plant, energy storage system constraints, collector field constraints, price-based demand response constraints and power balance constraints, as shown in the following formula: Where: SOC t is the state of charge at time t; η cha and η dis E is the charging efficiency and discharging efficiency respectively; m is the battery capacity; P cha,t and P dis,t are the charging power and discharging power at time t respectively; f cha,t and f dis,t They are binary symbols for energy storage charging and discharging, and charging and discharging cannot be performed at the same time; P bes,t is the energy storage charging and discharging power at time t; q tes,t =q tes,t-1 +η3-q st,t -q tp,t q sf,t =q sp,t +q tp,t Where: P CSP,t Represents the output of the CSP power station; P sp,t and P tp,t are the electric power provided by the heat collection system and the heat storage system at time t; q sp,t and q tp,t are the thermal power output of the heat collection system and the heat storage system at time t; η1 and η2 are the thermoelectric conversion efficiency of the power generation system; q tes,t is the capacity of the heat storage device at time t; q st,t is the thermal power input to the heat collection system at time t; η3 is the conversion efficiency of the heat storage system; q sf,t is the total thermal power output of the collector field at time t; P t and P PDR,t They are the load power before and after price-type demand response respectively.

4. The low-carbon dispatch optimization method for new energy power system based on deep reinforcement learning according to claim 3 is characterized in that: The step 3 is specifically as follows: Step 3.1: Determine the state of the environment: The state space of deep reinforcement learning is the physical quantity that the agent can observe in the environment, expressed as: Where: P L,t , S PDR,t and S IDR,t They are load power, price-based demand response state, and incentive-based demand response state at time t respectively; Step 3.2: Determine the agent action: The action space of deep reinforcement learning is the variables controlled by the agent in the system, expressed as: a t =[P GS,j,t ,P CGS,i,t ,P PV,t ,P WD,t ,P bes,t ,P csp,t ,P IDR,t ] Where: P GS,j,t ,P CGS,i,t ,P PV,t ,P WD,t ,P bes,t ,P csp,t ,P IDR,t They represent the output of thermal power unit j at time t, the total output power of carbon capture unit i at time t, the grid-connected power of photovoltaic power plant at time t, the grid-connected power of wind turbine at time t, the energy storage charging and discharging power at time t, the grid-connected power of CSP power station at time t, and the load reduction of incentive-type demand response at time t. Step 3.3: Determine the reward function: The total cost minimization problem of the new energy power system is transformed into the form of maximizing the cumulative return of deep reinforcement learning rewards, and the reward function r in the process of interaction between the agent and the environment is defined t for: r t (s t ,a t )=-(F+F punish ) Where: r t is the agent in state s at time t t Next, perform action a t The reward obtained after punish Punishment for exceeding the limit; Step 3.4: Use the deep deterministic policy gradient method to solve the optimal day-ahead scheduling strategy: Step 3.4.1: Set the number of training rounds according to historical data, and set the time step of the day-ahead scheduling task; Initialize the neural network and use random network parameters ω and θ to initialize the Critic network Q ω (s,a), Actor network μ θ (s) and noise η; copy the same parameters in the Critic network and the Actor network, and initialize the target network separately; Step 3.4.2: The agent selects a random scheduling strategy in the action space and determines the next state s in time sequence t+1 , the selection formula is as follows: a t =μ θ (s t )+η The experience (s t ,a t ,r t ,s t+1 ) is stored in the experience pool R; K action state samples are randomly collected from the experience pool R. The sample collection formula is as follows: {(s k ,a k ,r k ,s k+1 )} k＝1,...,K The target network is used to calculate the Q value of each action state sample. The calculation formula is as follows: Where: y k is the target Q value of the kth action state sample, r k is the instant reward of the k-th action state sample, γ is the discount factor, is the next state s of the kth action state sample k+1 The Q value when taking action a; Step 3.4.3: Calculate the loss function based on the Q value of each sample and update the Critic network. The calculation formula is as follows: Among them, L(ω) is the update loss function, Q ω (s k ,a k ) is the current state s k Take action a k Q value at ; Calculate the gradient of the Actor network and update the Actor network. The calculation formula is as follows: in, represents the derivation of strategy J, Indicates the gradient of the Actor network, θ is the derivative parameter, Indicates the derivation of the Q value, a is the state s k The action taken by the Actor main network; Step 3.4.4: Update the target network. The calculation formula for updating the target network is as follows: Among them, ω - , ω are the parameters of the Critic target network and the main network respectively, θ - , θ are the Actor target network and main network parameters respectively, σ is the soft update parameter; Step 3.4.5: Repeat steps 3.4.2 to 3.4.4 until convergence reaches the preset accuracy and the result of the current round of training is obtained; after the training is completed, save the parameters for real-time scheduling to obtain the scheduling result.

Images