Industrial park comprehensive energy system low-carbon planning method considering production process

Abstract

The invention discloses an industrial park comprehensive energy system low-carbon planning method considering a production process, and the method comprises the following steps: 1, obtaining the production line condition of an industrial park, and constructing a production line model; 2, determining to-be-planned equipment and parameters, and constructing a lower-layer planning operation model considering short-term uncertainty; 3, determining medium and long term uncertainty parameters, and constructing a medium and long term uncertainty set; step 4, determining a multi-objective optimization model; 5, searching a Pareto leading edge by using multi-target reinforcement learning based on a decomposition technology; and 6, selecting a proper Pareto frontier solution, and substituting the Pareto frontier solution into the lower-layer planning operation model to obtain a corresponding planning result. The invention aims to realize low-carbon transformation of an integrated energy system (IRIES) in an industrial area so as to cope with carbon reduction, emission reduction and challenges in a carbon dense area.

Description

Technical Field

[0001] This invention relates to a planning method for integrated energy systems in industrial parks, and particularly to a low-carbon planning method for integrated energy systems in industrial parks that takes into account the production process. Background Technology

[0002] The increasing severity of global warming and the resulting extreme natural disasters have prompted growing attention to the comprehensive low-carbon transformation of human activities. As major sources of global carbon dioxide emissions, the low-carbon transformation of the energy and industrial sectors has become a key focus for addressing climate change and achieving the 1.5°C global temperature control target set in the Paris Agreement. Integrated energy systems (IES), as the core framework for providing electricity, natural gas, and heat to various energy consumers within an operating area, are a primary means of achieving complementary use of different energy forms, improving overall energy efficiency, and reducing carbon emissions, especially in industrial areas where multiple energy consumption and productive carbon dioxide emissions are highly concentrated. However, the core of an integrated energy system typically consists of combined heat and power (CHP) units, boilers, and other energy supply equipment that relies on fossil fuels as the primary energy source, inevitably leading to additional carbon emissions. Therefore, low carbon has become an important aspect of IES planning.

[0003] The promising low-carbon measures for integrated energy systems (IES) implemented or proposed in the current research can be summarized as follows: (1) renewable energy substitution; (2) low-carbon transformation of power sources; (3) demand-side interaction mechanisms; and (4) inclusion in carbon market compliance management.

[0004] However, the introduction of these measures has also brought new problems. First, there is some conflict between the introduction of carbon capture, utilization, and storage (CCUS) and other low-carbon measures. On the one hand, the operation of CCUS requires additional energy consumption to capture large amounts of carbon dioxide. On the other hand, other carbon reduction measures in integrated energy systems (IES), such as renewable energy substitution, may affect the availability of captureable carbon dioxide, thereby weakening the economic viability of CCUS.

[0005] Secondly, the presence of CCUS further strengthens the coupling between material and energy flows, which is particularly evident in integrated energy systems (IES) within industrial areas. The interaction of industrial loads relies on production line-level load modeling, and the energy consumption of a production line is highly correlated with the production and transport of intermediate products in various processes. Industrial load modeling based on resource-task networks reflects this coupling characteristic between material and energy. Furthermore, the introduction of CCUS allows carbon dioxide emissions from production lines and fossil fuel units to enter the industrial energy and product production cycle through carbon utilization. Therefore, considering the emission characteristics of the production process also becomes necessary.

[0006] Third, the implementation of carbon markets and the introduction of emerging technologies have exacerbated the uncertainty surrounding the planning and operation of IES (Engineering, Procurement, and Construction). The uncertainties affecting IES include not only factors such as energy price fluctuations, demand and supply volatility, but also the deployment costs of new equipment and carbon prices. Carbon trading is a key driver of the negative externalities of carbon emissions in the pricing market, encouraging carbon emitters to actively adopt low-carbon technologies. Although carbon prices fluctuate daily according to market trading conditions, compliance with carbon allowances is typically measured annually. This means that, excluding arbitrage trading, companies can purchase emission rights at prices close to the annual average carbon price. Therefore, unlike energy prices, which fluctuate hourly or daily, carbon price fluctuations primarily affect medium- to long-term timescales. Summary of the Invention

[0007] The purpose of this invention is to provide a low-carbon planning method for integrated energy systems in industrial parks that takes into account the production process, aiming to achieve a low-carbon transformation of integrated energy systems in industrial regions (IRIES) to address the challenges of carbon reduction and emission reduction in these carbon-intensive areas.

[0008] To achieve the above objectives, the solution of the present invention is:

[0009] A low-carbon planning method for an integrated energy system in an industrial park that considers the production process includes the following steps:

[0010] Step 1: Obtain information on the production lines in the industrial park and build a production line model;

[0011] Step 2: Determine the equipment and parameters to be planned, and construct a lower-level planning and operation model that takes into account short-term uncertainties;

[0012] Step 3: Determine the medium- and long-term uncertainty parameters and construct the medium- and long-term uncertainty set;

[0013] Step 4: Determine the multi-objective optimization model;

[0014] Step 5: Use multi-objective reinforcement learning based on decomposition techniques to search for the Pareto frontier;

[0015] Step 6: Select an appropriate Pareto front solution and substitute it into the lower-level planning and running model to obtain the corresponding planning results.

[0016] In step 1 above, the constructed production line model includes,

[0017] The coupling relationship of the r-th production line in the l-th process at scheduling time t is described as follows:

[0018]

[0019]

[0020] Among them, TP r,l,t For the current throughput of process l, B r,ι,t This refers to the output volume of the buffer zone containing upstream products on the production line. Φ is the conversion ratio coefficient between upstream products and process throughput on the production line. P (l) represents the set of upstream processes connected to process l; These are the active power demand for electricity, the thermal power demand, and the fuel demand for process l, respectively. The corresponding electrical power, thermal power, and natural gas energy consumption coefficients are given. The corresponding standby power consumption is given, and OF represents the fuel oxidation rate. and These are the relative molecular masses of carbon dioxide and methane, respectively. x is the carbon emission factor for process l; r,l,t This is a 0-1 variable for starting / stopping the production line, where 0 represents stopping and 1 represents starting. TP r,l These represent the upper and lower limits of the throughput of production line r in process l;

[0021] The cache stores middleware output from the previous production process, and its model is as follows:

[0022] BC r,l,t =(TP) r,l,t -B r,l,t )ΔT,t=1

[0023] BC r,l,t =BC r,l,t-1 +(TP r,l,t -B r,l,t )ΔT,t≥2

[0024]

[0025] x γ,ι,t ≤B r,l,t ,(γ,ι)∈Φ S (r,l)

[0026] Among them, BC r,l,t For the storage level of the cache, B r,l,t For the amount to be transported to the next process, Φ is the upper limit of cache storage, ΔT is the scheduling interval, and Φ is the upper limit of cache storage. S (r,l) represents the subsequent stage of stage (r,l);

[0027] The production line's output target is:

[0028]

[0029] in, For the daily output target of production line r, Φ L (r) represents the set of final stages of production line r;

[0030] Production volume is adjusted based on production conditions, described as follows:

[0031]

[0032] in, To transfer production, This represents the upper limit of transferable production.

[0033] In step 2 above, the lower-level planning operation model that considers short-term uncertainties includes:

[0034] The planning objects of the lower-level planning and operation model include photovoltaic (PV), wind power (WT), energy storage (ES), and carbon capture, utilization and storage (CCUS). Among them, CCUS includes carbon capture system (CCS) and power-to-gas (P2G).

[0035] The objective function of the lower-level planning and operation model is:

[0036]

[0037] Where C represents the total cost of planning and operation. Annual investment cost, Annual operating costs, Annual energy purchase cost, Annual environmental cost; A 0-1 variable representing whether to plan, where r is the inflation coefficient. Ω represents the unit capacity and power cost of the planned equipment χ, respectively. PEQ This refers to the set of equipment to be planned, namely WT, PV, ES, CCS, and P2G. For system maintenance costs, κ χ For unit power maintenance cost, κ χ,on κ χ,off These are the costs of starting and stopping the equipment, This serves as a start / stop indicator for the equipment. For the system's cost of power curtailment, For abandoned power, c CUR The penalty factor per unit of abandoned power; The penalty cost for unsupplied load in the system, and These represent the power output without electricity, the thermal power, and the fuel quantity, respectively. UN ξ(s) represents the penalty coefficient for unit energy not supplied; ξ(s) represents the probability of the scenario occurring. and These are the system's purchased electrical power, thermal power, and fuel quantity, respectively. These are the prices of purchased electricity, heat, and gas, respectively. Let be the carbon allowance price for year y. For total carbon emissions, κ seq For carbon capture costs, Total carbon capture, To cover the cost of purchasing carbon dioxide, This refers to the quantity of carbon dioxide purchased externally.

[0038] The constraints of the lower-level planning and operation model include planning constraints, operation constraints of combined heat and power (CHP), operation constraints of gas boiler (GB), operation constraints of electric boiler (EB), operation constraints of new energy unit (RG), operation constraints of electric energy storage (ES), operation constraints of thermal energy storage (HS), operation constraints of gas storage (GS), operation constraints of carbon storage (CS), user-side load, energy balance, and emission constraints.

[0039] In step 3 above, the constructed medium- to long-term uncertainty set includes,

[0040]

[0041] Where, α χ For uncertainty margin, These are the predicted values ​​for the corresponding costs and prices. These are the actual possible values.

[0042] In step 4 above, the multi-objective optimization model is determined as follows:

[0043] (UL)maxα

[0044]

[0045] Production line model, a lower-level planning and operation model considering short-term uncertainties, and a set of medium- and long-term uncertainties.

[0046] Among them, Ω PEQ Let C be the set of equipment to be planned, C be the total system cost, RB be the uncertainty margin, and C0 be the system planning cost without considering uncertainty.

[0047] The specific process of step 5 above is as follows:

[0048] Determine the input parameters and hyperparameters of the agent, where the input parameters include a memory buffer. Evaluation Network Initial parameters θ1, θ2, policy network Q φ Hyperparameters include soft update factor τ, reward discount factor ∈, and pruning factor γ;

[0049] First, initialize the network and agent parameters as follows: Clear

[0050] The value of parameter i is looped from 1 to MaxStep as follows:

[0051] In action a i =Q φ (s i When +noise, a reward of r is obtained. i and the next state s i+1 ,in Experience (s) i ,a i ,r i ,a i ,s i+1 ,done i Store to Among them, done i This indicates the end of step i;

[0052] from M state transitions (s) are randomly sampled in the middle. i ,a i ,r i ,a i ,s i+1 ,done i This forms a small batch;

[0053] The action strategy considering noise conditions is as follows.

[0054]

[0055] Update evaluation network parameters

[0056] If i cannot be divided evenly by i freq Then update according to the following strategy:

[0057]

[0058] Update network parameters proportionally:

[0059]

[0060] The specific process of step 6 above is as follows:

[0061] upper bound of the upper optimization variable α

[0062] Latin hypercube sampling was used in The training set is formed on the above. As input to the sample;

[0063] Using commercial solvers to solve different The inner layer problem of LL is about to be solved. Substitute the values ​​into the inner problem of LL to obtain the corresponding objective function value;

[0064] The samples are normalized to construct a training set, and an artificial neural network is trained to obtain a proxy model for InnerLL.

[0065] Replace the inner layer problem of LL with a trained artificial neural network and call the surrogate optimization solver to estimate the worst objective value in the set of uncertainties.

[0066] After adopting the above solution, the beneficial effects of the present invention are as follows:

[0067] To mitigate the uncertainties across multiple time scales faced by system operators, a two-layer robust optimization framework is adopted. This framework combines information gap decision theory (IGDT) and stochastic optimization (SO), which can effectively address the multiple uncertainties encountered during the low-carbon transition and ensure the robustness of the long-term planning results of the integrated energy system in the industrial park. Attached Figure Description

[0068] Figure 1 This is a schematic diagram of the integrated energy system structure for industrial areas used in this invention;

[0069] Figure 2 This is a schematic diagram of the energy-carbon coupling production process in this invention;

[0070] Figure 3 This is a schematic diagram of the unit cost of planning and the carbon emission factor of upstream energy;

[0071] Figure 4 This is a schematic diagram of the electrical load;

[0072] Figure 5 This is a schematic diagram of the heat load;

[0073] Figure 6 This is a schematic diagram of the gas load;

[0074] Figure 7 This is a schematic diagram of the unit output power of wind power;

[0075] Figure 8 This is a schematic diagram of the unit output power of a photovoltaic system;

[0076] Figure 9 This is a diagram illustrating electricity prices;

[0077] Figure 10 This is a flowchart illustrating the implementation of the present invention. Detailed Implementation

[0078] The technical solution and beneficial effects of the present invention will be described in detail below with reference to the accompanying drawings.

[0079] (1) System Description

[0080] Figure 1 The structure of an Industrial Area Integrated Energy System (IRIES) is presented. This system involves three typical energy flows: electricity, heat, and natural gas, as well as a material flow of carbon dioxide.

[0081] On the demand side, regional loads are divided into production loads and non-production loads, depending on whether the load is related to a specific manufacturing process. In this embodiment, it is assumed that production loads can be scheduled by adjusting the manufacturing system. Non-production loads simulate the living and auxiliary needs of other regions and are assumed to be unadjustable.

[0082] On the supply side, the following power generation and conversion equipment are considered: combined heat and power (CHP), gas boilers (GB), electric boilers (EB), wind turbines (WT), photovoltaic panels (PV), energy storage (ES), thermal storage (HS), and gas storage (GS). The system is also connected to external power and natural gas sources for energy exchange.

[0083] Furthermore, carbon capture, utilization, and storage (CCUS) is considered a key component of the regional low-carbon transition, encompassing carbon capture systems (CCS), power-to-gas (P2G) as part of carbon utilization, and carbon storage (CS). Emissions from CHP, GB, and manufacturing processes are collected and fed into the CCS to separate carbon dioxide. The captured carbon dioxide can be utilized or stored. If utilized, the captured carbon dioxide will be used as feedstock to produce synthetic natural gas via methanation, with the reaction: 4H₂ + CO₂ → CH₄ + 2H₂O.

[0084] (2) Industrial process modeling

[0085] This invention proposes an Energy-Emission Embedded Manufacturing Process (E3MP) model. The key concept is that for a single, general manufacturing process, its raw material requirements, overall energy requirements, and emissions within a specific scheduling cycle are highly correlated with the current process output. For example... Figure 2 As shown, the coupling relationship of the l-th process of the r-th production line at scheduling time t can be described as follows:

[0086]

[0087]

[0088] For simplicity, year and scene indices have been omitted where unnecessary. Equation (1) represents a series of equations defining the proportional conversion between raw materials and process output; equations (2) to (4) reflect the overall energy demand of the process; equation (5) describes the emissions generated by the process due to chemical reactions and fossil fuel combustion; and equation (6) limits the output of the process. TP r,l,t B represents the current throughput of this process. r,ι,t The output volume of products belonging to the upstream products of the production line. Φ is the conversion ratio between the two. P (l) represents the set of upstream links connected to link l; These are the active power demand, thermal power demand, and fuel demand for this stage, respectively. The corresponding electrical power, thermal power, and natural gas energy consumption coefficients are given. The corresponding standby power consumption is given, and OF is the oxidation rate of the fuel, which is generally taken as 0.99. and These are the relative molecular masses of carbon dioxide and methane, respectively. This represents the carbon emission factor for this stage; x r,l,t This is a 0-1 variable for starting / stopping the production line, where 0 represents stopping and 1 represents starting. TP r,l , These are the upper and lower limits of the throughput of the process.

[0089] The cache stores middleware from the output of the previous production process and prepares it for subsequent processes. Its model is as follows:

[0090] BC r,l,t =(TP) r,l,t -B r,l,t )ΔT,t=1 (7)

[0091] BC r,l,t =BC r,l,t-1 +(TP r,l,t -B r,l,t )ΔT,t≥2 (8)

[0092]

[0093] x γ,ι,t ≤B r,l,t ,(γ,ι)∈Φ S (r,l) (11)

[0094] Equations (7)-(8) define the storage level BC of the buffer. r,l,t and the throughput TP in this stage r,l,t and the amount of material transported to the next stage B r,l,t The relationship between them. Equation (9) specifies the upper limit of cache storage. Equation (10) indicates that the process (r,l) should stop when the buffer is full, and Equation (11) indicates that the subsequent process should also stop when the preceding process stops delivering products; ΔT is the scheduling interval, which is generally taken as 1 hour; Φ S (r,l) represents the subsequent steps of (r,l).

[0095] The production line's output target is:

[0096]

[0097] in, For the daily output target of production line r, Φ L (r) is the set of final stages of production line r.

[0098] Taking into account the seasonality of production, the system is allowed to adjust output according to production conditions, i.e.:

[0099]

[0100] in, To transfer production, As the upper limit of transferable output, Equation (15) stipulates that the sum of transferable output in all scenarios is 0, in order to avoid the occurrence of underproduction.

[0101] (3) IRIES Collaborative Programming Model

[0102] The main planning objects of the IRIES collaborative planning model proposed in this invention are photovoltaic (PV), wind power (WT), energy storage (ES), and CCUS, where CCUS includes CCS and P2G.

[0103] ① Lower-level optimization problem: Scenario-based stochastic programming

[0104] The lower layer (LL) focuses on planning under short-term uncertainties, such as renewable resources, energy prices, and load changes.

[0105] The objective function is:

[0106]

[0107]

[0108] Equation (16) defines the total cost C of system planning and operation, including annual investment costs. Annual operating costs Annual energy purchase cost and annual environmental costs Equation (17) defines the investment cost of the system. A 0-1 variable representing whether to plan, where r is the inflation coefficient. Ω represents the unit capacity and power cost of the planned equipment χ, respectively. PEQ The set of equipment to be planned is WT, PV, ES, CCS, and P2G; Equation (19) defines the system maintenance cost. Among them κ χ For unit power maintenance cost, κ χ,on κ χ,off These are the costs of starting and stopping the equipment, This serves as a flag for equipment start-up and shutdown; Equation (20) defines the system's power curtailment cost. For abandoned power, c CUR The penalty factor per unit of abandoned power; Equation (21) defines the penalty cost of unsupplied load in the system. and c represents unsupplied power, thermal power, and fuel quantity. UN Let ξ(s) be the penalty coefficient for unit energy not supplied; Equation (22) defines the annual operating cost of the system, and ξ(s) is the probability of the scenario occurring. and These are the system's purchased electrical power, thermal power, and fuel quantity, respectively. These are the prices for purchased electricity, heat, and gas, respectively.

[0109] The constraints include:

[0110] 1) Planning constraints

[0111]

[0112]

[0113] in, Let x be a 0-1 variable for planning and decision-making of equipment. Cap χ , To plan the upper and lower limits of equipment capacity, For equipment capacity planning, Ω PEQ This refers to the collection of equipment to be planned. Cap ES,pow , To plan the upper and lower limits of charging and discharging power for energy storage, The energy storage is planned for charging and discharging power; Equation (26) ensures that the equipment is planned only once within the planning period. Equations (27) and (28) ensure that the capacity after planning is not less than the capacity before planning, where M is a relatively large number, usually 1e6.

[0114] 2) CHP

[0115] The operating constraints of CHP are:

[0116]

[0117] Equations (29)-(31) define the relationship between the output power and the full power of the CHP. For the full power of the CHP unit, For unit efficiency, For the CHP unit to consume fuel, H G Given the low calorific value of the fuel, we'll use natural gas as the fuel, H. G It is 10.81 MWh / km 3 ; For the unit's output electrical power, η CHP,E This represents the conversion ratio between electrical power and total power. For the unit to output thermal power, η CHP,H This represents the conversion ratio between the unit's thermal power and total power; Equation (32) reflects the aging of the unit's efficiency. For the efficiency of the unit before the planning period, DE CHP The efficiency degradation coefficient of the CHP unit; Equation (33) defines the carbon emissions of the unit. These are the relative molecular masses of carbon dioxide and natural gas, respectively, and OF is the oxidation rate, typically 0.99; The unit's on / off state is a 0-1 variable. P CHP,all , Let be the minimum power and maximum power of the unit, respectively. Equation (34) defines the ramp power of the unit. This represents the upper limit of ramp power. Equation (36) defines the 0-1 variables of the unit's start / stop event flags. Equations (37) and (38) limit the frequency of start-stop. T CHP,on T CHP,off It is the minimum start-stop interval of the unit.

[0118] 3) GB

[0119]

[0120] The meanings of the constraints in GB operation are similar to those in CHP. Among them, The thermal power output of GB, Fuel consumed for GB. The energy conversion efficiency is GB. For GB, the unit efficiency before planning, For GB carbon emissions, For GB unit switch 0-1 variables,H GB , These are the lower and upper limits of GB output, respectively. This is the upper limit of climbing power. This is a 0-1 variable used as a start / stop event flag. T GB,on T GB,off This is the minimum start / stop interval for GB.

[0121] 4) EB

[0122]

[0123] For the electrical power of EB, The thermal power output by EB This is the maximum capacity of EB.

[0124] 5) Renewable Energy Units (RG)

[0125]

[0126] This is the upper limit of RG's output. Units that contribute to the development of new energy power units For the actual output of new energy units, This refers to the amount of abandoned electricity.

[0127] 6)ES

[0128]

[0129] The charge level of energy storage is defined by equation (51). With charging power Discharge power The relationship between η ES,ch η ES,dis These are the charging efficiency and discharging efficiency, respectively. ES The upper limit of energy storage planning capacity, To plan the upper limit of inverter power for energy storage, The energy storage charge / discharge state is a 0-1 variable. The total power of energy storage operation, Net power for energy storage operation.

[0130] 7) HS / GS / CS

[0131] The operational constraints of HS, GS, and CS are similar to those of ES. Here, we take GS as an example:

[0132]

[0133] For the generalized charge level of GS, For GS charge / discharge power, η GS,ch η GS,dis For GS charge / discharge efficiency, The electrical power consumed for energy storage operation, δ GS,E Cap is the power consumption factor. GS For GS capacity, For maximum charge / discharge power, In the charge / discharge state, F GS,L This refers to the net power of charging and discharging.

[0134] 8) Load

[0135] User-side load is divided into controllable load and uncontrollable load. The modeling of controllable load is as described above, and its representation is as follows:

[0136]

[0137] in, To reduce production line load, Φ represents the total load of the corresponding equipment. χ This refers to all components belonging to equipment χ. The planned capacity of equipment χ.

[0138] In addition, the uncontrollable load is modeled as follows:

[0139]

[0140] in, The actual dispatch power of uncontrollable electrical, gas, and heat loads. This represents the corresponding unsupplied quantity. Forecast demand for users, Δ E Δ H Δ G This represents the load growth rate.

[0141] 9) Energy balance

[0142]

[0143] 10) Emissions

[0144]

[0145] ② Upper-level optimization problem: Multi-objective robust optimization method based on information gaps

[0146] The upper limit (UL) focuses on mitigating the severe uncertainties introduced by long-term variables, which are often difficult to predict accurately. This invention establishes a multi-objective robust optimization model based on information gap theory to account for these severe uncertainties. Long-term uncertain variables are constructed based on an envelope boundary model:

[0147]

[0148] Where, α χ For uncertainty margin, These are the predicted values ​​for the corresponding costs and prices. These are the actual possible values.

[0149] Therefore, the proposed model can be expressed as:

[0150]

[0151] (4) Solution Strategy

[0152] The model proposed in equation (85) is a multi-objective, multi-stage optimization model containing mixed integers, which presents certain difficulties in solving. Therefore, this invention proposes a solution framework based on multi-objective reinforcement learning:

[0153] 1) Environmental Model

[0154] To extend the reinforcement learning framework to solve multi-objective optimization problems, our goal is to train an agent to search for the Pareto Front (PF) while satisfying a set of constraints. First, we need to transform the solution process into a Markov Decision Process (MDP), which includes a state space, an action space, and a reward function.

[0155] Define a state space so that the trained agent can observe the current value of the objective function and RB corresponding to its current position:

[0156]

[0157] From s i Transition to s i+1 In this step, the agent needs to determine the amount of change for each optimization variable. Naturally, the action space is defined as:

[0158]

[0159] In the context of reinforcement learning, the reward function exhibits two key features that conflict with the solution process: (1) the output of the reward function is usually a single scalar value, while the target is represented as a multidimensional vector; (2) the main goal of training the agent is to maximize the cumulative reward obtained from a series of actions, while the solution process focuses on the final result rather than the trajectory taken to achieve that result.

[0160] To resolve the aforementioned conflict, a reward function using Chebyshev decomposition is constructed, as follows. Specifically, for the reward function derived from s... i Transition to s i+1 The action taken at that time a i It defines the reward it receives as:

[0161]

[0162] s i+1 The update satisfies:

[0163]

[0164] Equation (89) forces the agent to perform the current action a i When a violation of the constraints in equation (85) occurs, the user will revert to the previous state. Furthermore, negative penalties will be imposed according to equation (88) to deter frequent violations.

[0165] 2) Pareto search strategy based on TD3

[0166] Considering that the Pareto front search process is a continuous and sequential process In terms of state and action space, traditional Deep Deterministic Policy Gradient (DDPG) agents are well-suited for training to solve such problems. However, their Q-value update mechanism is susceptible to overestimation bias and can lead to convergence instability during training, especially in high-dimensional action spaces.

[0167] To address the aforementioned drawbacks, a reinforcement learning framework based on the Twin-delayed DDPG (TD3) agent is adopted. Compared to DDPG, TD3 incorporates three key techniques. First, it integrates a truncated double Q-value learning framework into the actor-critic network to mitigate the overestimation problem. Second, it reduces the update frequency of the policy and target networks, thereby enhancing convergence performance by avoiding high-error states. Third, it introduces noise following a truncated normal distribution into the actions to smooth Q-value estimation and prevent overfitting. The training algorithm is given in Algorithm 1.

[0168] Algorithm 1: TD3 Algorithm

[0169]

[0170] 3) Proxy optimization for LL solvers

[0171] Equations (88)-(89) require frequent verification of the constraints in equation (85). This can essentially be interpreted as repeatedly calculating the objective function of LL within a set of uncertainties determined by a specific α. Given that LL (lower level) is a two-level mixed integer programming problem with a continuous set of uncertainties, obtaining its exact solution is both difficult and extremely time-consuming, making this approach impractical. For a given weight w p The maximum training time is defined as MaxEpisode × MaxStep × SolveTime, where SolveTime represents the time required to solve the LL problem, MaxStep represents the maximum number of steps the agent can take in a single round, and MaxEpisode represents the maximum number of rounds the agent can train. Therefore, the key is to reduce the average SolveTime during training. To this end, this invention proposes an alternative optimization method for estimating the LL objective value, as follows:

[0172] a. Input the upper bound of the upper-level optimization variable α

[0173] b. Latin Hypercube Sampling (LHS) was used in... The training set is formed on the above. As input to the sample;

[0174] c. Using commercial solvers to solve different... The inner LL problem (InnerLL) is about to be solved. Substitute into InnerLL to obtain the corresponding objective function value;

[0175] d. Normalize the samples, construct a training set, and train an artificial neural network (ANN) to obtain a proxy model for InnerLL;

[0176] f. Replace InnerLL with a trained artificial neural network (ANN) and call the surrogate optimization solver to estimate the worst objective value in the uncertainty set.

[0177] 4) Pareto Frontier Search Process Based on Decomposition Technique in Multi-Objective Reinforcement Learning

[0178] The Multi-objective reinforcement learning based on decomposition (MORL / D) algorithm aims to explore the Pareto front of the optimization model proposed in Equation (85), and the process of the algorithm is detailed in Algorithm 2. The weight vector w is randomly generated and used to decompose the original multi-objective problem into several subproblems. The agent based on the TD3 algorithm is trained in an environment constructed using the weight vector and RB. To enhance the randomness of the environment, we introduce uniform randomness at the starting point of the agent's exploration. Subsequently, the trained agent is simulated to interact with the environment to generate a solution, denoted as PF.

[0179] Algorithm 2: MORL / D Multi-Objective Pareto Search Algorithm

[0180]

[0181]

[0182] Using a steel industrial park as an application scenario, this invention validates the multi-layered robust planning model for low-carbon transformation of IRIES (Integrated Regional Energy Systems). The validation period is ten years, and the system parameters are shown in Tables 1-5. Figures 3-9 The planning results are shown in Table 6.

[0183] Table 1 Parameters for Industrial Processes

[0184]

[0185]

[0186] Table 2. Prices of Heat and Natural Gas

[0187] Scene S1 S2 S3 <![CDATA[gas(￥ / km 3 )]]> 5945.7 3245.6 3742.3 <![CDATA[heat(￥ / MW h h)]]> 12175.0 5295.0 3996.0

[0188] Table 3 Equipment Planning Parameters (MW, MWh)

[0189] equipment PV WT ES,cap ES,pow CCS P2G upper limit 1000 1000 1000 1000 1000 1000 Lower 1 1 1 1 1 1

[0190] Table 4 Carbon emission prices (¥ / tCO2)

[0191] Year 1 2 3 4 5 Carbon price 197 217 240 263 287 Year 6 7 8 9 10 Carbon price 312 339 369 401 435

[0192] Table 5 Equipment Parameters

[0193]

[0194]

[0195] Table 6 shows the planning results with weights of [0.159, 0.104, 0.186, 0.202, 0.149, 0.201].

[0196]

[0197] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0198] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0199] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0200] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0201] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0202] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A low-carbon planning method for an integrated energy system in an industrial park that considers the production process, characterized in that... Includes the following steps: Step 1: Obtain information on the production lines in the industrial park and build a production line model; Step 2: Determine the equipment and parameters to be planned, and construct a lower-level planning and operation model that takes into account short-term uncertainties; Step 3: Determine the medium- and long-term uncertainty parameters and construct the medium- and long-term uncertainty set; Step 4: Determine the multi-objective optimization model; Step 5: Use multi-objective reinforcement learning based on decomposition techniques to search for the Pareto frontier; Step 6: Select an appropriate Pareto front solution and substitute it into the lower-level planning and running model to obtain the corresponding planning results.

2. The low-carbon planning method for integrated energy systems in industrial parks considering the production process as described in claim 1, characterized in that: In step 1, the constructed production line model includes, The coupling relationship of the r-th production line in the l-th process at scheduling time t is described as follows: Among them, TP r,l,t B represents the current throughput of process l. r,ι,t This refers to the output volume of the buffer zone containing upstream products on the production line. Φ is the conversion ratio coefficient between upstream products and process throughput on the production line. P (l) represents the set of upstream processes connected to process l; These are the active power demand for electricity, the thermal power demand, and the fuel demand for process l, respectively. The corresponding electrical power, thermal power, and natural gas energy consumption coefficients are given. The corresponding standby power consumption is given, and OF represents the fuel oxidation rate. and These are the relative molecular masses of carbon dioxide and methane, respectively. x is the carbon emission factor for process l; r,l,t This is a 0-1 variable for starting / stopping the production line, where 0 represents stopping and 1 represents starting. TP r,l These represent the upper and lower limits of the throughput of production line r in process l; The cache stores middleware output from the previous production process, and its model is as follows: BC r,l,t =(TP r,l,t -B r,l,t )ΔT,t=1 BC r,l,t =BC r,l,t-1 +(TP r,l,t -B r,l,t )ΔT,t≥2 x γ,ι,t ≤B r,l,t ,(γ,ι)∈Φ S (r,l) Among them, BC r,l,t For the storage level of the cache, B r,l,t For the amount to be transported to the next process, Φ is the upper limit of cache storage, ΔT is the scheduling interval, and Φ is the upper limit of cache storage. S (r,l) represents the subsequent stage of stage (r,l); The production line's output target is: in, For the daily output target of production line r, Φ L (r) represents the set of final stages of production line r; Production volume is adjusted based on production conditions, described as follows: in, To transfer production, This represents the upper limit of transferable production.

3. The low-carbon planning method for integrated energy systems in industrial parks considering the production process as described in claim 1, characterized in that: In step 2, the constructed lower-level planning operation model that considers short-term uncertainties includes, The planning objects of the lower-level planning and operation model include photovoltaic (PV), wind power (WT), energy storage (ES), and carbon capture, utilization and storage (CCUS). Among them, CCUS includes carbon capture system (CCS) and power-to-gas (P2G). The objective function of the lower-level planning and operation model is: Where C represents the total cost of planning and operation. Annual investment cost, Annual operating costs, Annual energy purchase cost, Annual environmental cost; A 0-1 variable representing whether to plan, where r is the inflation coefficient. Ω represents the unit capacity and power cost of the planned equipment χ, respectively. PEQ This refers to the set of equipment to be planned, namely WT, PV, ES, CCS, and P2G. For system maintenance costs, κ χ For unit power maintenance cost, κ χ,on κ χ,off These are the costs of starting and stopping the equipment, Indicators for starting and stopping equipment; For the system's cost of power curtailment, For abandoned power, c CUR The penalty factor per unit of abandoned power; The penalty cost for unsupplied load in the system, and These represent the power output without electricity, the thermal power, and the fuel quantity, respectively. UN ξ(s) represents the penalty coefficient for unit energy not supplied; ξ(s) represents the probability of the scenario occurring. and These are the system's purchased electrical power, thermal power, and fuel quantity, respectively. They are respectively purchased electricity, The price of heat and gas; Let be the carbon allowance price for year y. For total carbon emissions, κ seq For carbon capture costs, Total carbon capture, To cover the cost of purchasing carbon dioxide, This refers to the quantity of carbon dioxide purchased externally. The constraints of the lower-level planning and operation model include planning constraints, operation constraints of combined heat and power (CHP), operation constraints of gas boiler (GB), operation constraints of electric boiler (EB), operation constraints of new energy unit (RG), operation constraints of electric energy storage (ES), operation constraints of thermal energy storage (HS), operation constraints of gas storage (GS), operation constraints of carbon storage (CS), user-side load, energy balance, and emission constraints.

4. The low-carbon planning method for integrated energy systems in industrial parks considering the production process as described in claim 1, characterized in that: In step 3, the constructed medium- to long-term uncertainty set includes, Where, α χ For uncertainty margin, These are the predicted values ​​for the corresponding costs and prices. These are the actual possible values.

5. The low-carbon planning method for integrated energy systems in industrial parks considering the production process as described in claim 1, characterized in that: In step 4, the multi-objective optimization model is determined as follows: (UL)maxα χ∈Ω PEQ ∪{IS} Production line model, considering short-term uncertainties. Lower-level planning and operation model, medium- and long-term uncertainty set Among them, Ω PEQ Let C be the set of equipment to be planned, C be the total system cost, RB be the uncertainty margin, and C0 be the system planning cost without considering uncertainty.

6. The low-carbon planning method for integrated energy systems in industrial parks considering the production process as described in claim 1, characterized in that: The specific process of step 5 is as follows: Determine the input parameters and hyperparameters of the agent, where the input parameters include a memory buffer. Evaluation Network Initial parameters θ1, θ2, policy network Q φ Hyperparameters include soft update factor τ, reward discount factor ∈, and pruning factor γ; First, initialize the network and agent parameters as follows: Clear The value of parameter i is looped from 1 to MaxStep as follows: In action a i =Q φ (s i When +noise, a reward of r is obtained. i and the next state s i+1 ,in Experience (s) i ,a i ,r i ,a i ,s i+1 ,done i Store to Among them, done i This indicates the end of step i; from M state transitions (s) are randomly sampled. i ,a i ,r i ,a i ,s i+1 ,done i This forms a small batch; The action strategy considering noise conditions is as follows. Update evaluation network parameters If i cannot be divided evenly by i freq Then update according to the following strategy: Update network parameters proportionally:

7. The low-carbon planning method for integrated energy systems in industrial parks considering the production process as described in claim 1, characterized in that: The specific process of step 6 is as follows: upper bound of the upper optimization variable α Latin hypercube sampling was used in The training set is formed on the above. As input to the sample; Using commercial solvers to solve different The inner layer problem of LL is about to be solved. Substitute the values ​​into the inner problem of LL to obtain the corresponding objective function value; The samples are normalized to construct a training set, and an artificial neural network is trained to obtain a proxy model for InnerLL. Replace the inner layer problem of LL with a trained artificial neural network and call the surrogate optimization solver to estimate the worst objective value in the set of uncertainties.

Images