Optimized scheduling method, system, equipment and medium for electro-hydrogen ammonia alcohol comprehensive energy system
By constructing a scheduling model for an integrated energy system of electricity, hydrogen, ammonia, and methanol that balances economic optimization with the safety constraints of multiple devices, and combining Lagrange multiplier optimization and the diffusion actor-commentator algorithm, the problem of traditional methods being unable to balance economy and safety is solved, and an efficient scheduling scheme is achieved that can adapt to complex operating scenarios.
Patent Information
- Application Number
- CN202511674571.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-14
- Publication Date
- 2026-03-03
AI Technical Summary
Traditional methods struggle to balance the economic optimization objective of an integrated energy system of electricity, hydrogen, ammonia, and methanol with the safety constraints of multiple devices, and are inefficient in solving complex scheduling problems.
We employ a diffusion actor-critic algorithm that takes information entropy into account, combined with a Gaussian mixture model, and integrate Lagrange multiplier optimization and constrained Markov decision process to construct a scheduling model that balances economic optimality and multi-device security constraints. By reconstructing the policy network through the diffusion model and training the policy network and value evaluation network, we can output the optimal scheduling scheme.
It effectively avoids the risk of equipment constraints and violations, improves the rationality and efficiency of strategy exploration and solution, reduces the total operating cost of the system, adapts to the volatility of new energy sources, promotes the integration of wind and solar power, and outputs the optimal scheduling scheme that takes into account economy, safety and real-time performance.
Smart Images
Figure CN121599347A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of integrated energy dispatching technology, and in particular to an optimized dispatching method, system, equipment and medium for an integrated energy system of electricity, hydrogen, ammonia and alcohol. Background Technology
[0002] With the large-scale integration of new energy sources into the power system, the power system is gradually evolving from a traditional fossil fuel-dominated system to a new development pattern with new energy as the mainstay and traditional energy as a support. This presents unprecedented challenges to the grid's regulation capacity and the capacity to absorb new energy. Hydrogen energy, with its advantages of zero emissions, high calorific value, abundant resources, and long storage time, is attracting increasing attention from various countries, and an industrial layout centered on green hydrogen is gradually becoming one of the hottest directions in energy development. Furthermore, although technologies for producing hydrogen from fossil fuels and industrial by-products are relatively mature, the high carbon emissions during hydrogen production mean that in the short term, they can only serve as a transitional solution for the complete clean transformation of hydrogen energy. Therefore, replacing electrolytic hydrogen production and ammonia synthesis technologies with clean energy is a crucial approach to reducing carbon consumption and is the inevitable path for the green hydrogen and ammonia industry in the present and long term.
[0003] However, the scheduling problem of integrated energy systems (electricity, hydrogen, ammonia, pure oxygen) has high time and space complexity. It is a large-scale, non-convex, nonlinear mixed-integer programming problem, belonging to the NP-hard category, and is difficult to solve using traditional methods. Deep reinforcement learning (DRL) is an important artificial intelligence algorithm that does not require model approximation and has comprehensive uncertainty awareness. It shifts the computational burden of the iterative optimization process to the offline training process, thereby enabling online decision-making and expansion of complex problems. DRL has already attracted attention in the field of integrated energy system scheduling. In most traditional RL algorithms, the policy is usually parameterized as a diagonal Gaussian distribution with learned mean and variance, which limits their ability to acquire complex policies. Summary of the Invention
[0004] In view of the aforementioned existing problems, the present invention is proposed.
[0005] Therefore, this invention provides an optimized scheduling method for an integrated energy system of electricity, hydrogen, ammonia, and alcohol to solve the problems of traditional methods, which struggle to balance economic optimization with the safety constraints of multiple devices and have low efficiency in solving complex scheduling problems.
[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides an optimized scheduling method for an integrated energy system of electricity, hydrogen, ammonia, and methanol, comprising: Obtain the parameters of each device in the integrated energy system of electricity, hydrogen, ammonia, and methanol, and establish an optimal scheduling model for the integrated energy system of electricity, hydrogen, ammonia, and methanol with the goal of economic optimization; The established integrated energy system optimization scheduling model of hydrogen, ammonia, and alcohol is reconstructed into a constrained Markov decision process, and the constrained Markov decision process is optimized using Lagrange multipliers. The diffusion actor-critic algorithm, which takes into account information entropy, is used in conjunction with a Gaussian mixture model to train and solve the optimal scheduling model of the integrated energy system of electricity, hydrogen, ammonia, and alcohol corresponding to the constrained Markov decision process, so as to obtain the optimal scheduling scheme.
[0007] As a preferred embodiment of the optimized scheduling method for the integrated energy system of hydrogen, ammonia, and methanol described in this invention, the method includes: establishing an optimized scheduling model for the integrated energy system of hydrogen, ammonia, and methanol with economic optimality as the objective, comprising: Based on constraints, an optimal scheduling model for the integrated energy system of electricity, hydrogen, ammonia, and methanol is constructed with the goal of minimizing economic costs. The economic costs include the power generation costs of thermal power units, the wind and solar curtailment penalty costs of the green hydrogen chemical virtual energy pool, the sales revenue of methanol and ammonia, the production and compression costs of hydrogen, the operation and maintenance costs of hydrogen storage tanks, and the power shortage penalty costs.
[0008] As a preferred embodiment of the optimized scheduling method for the integrated energy system of electricity, hydrogen, ammonia, and methanol described in this invention, the constraints include thermal power unit operation constraints, wind and solar power output constraints, hydrogen production and compression constraints, hydrogen storage constraints, ammonia synthesis constraints, methanol synthesis constraints, and power balance constraints.
[0009] As a preferred embodiment of the optimized scheduling method for the integrated energy system of hydrogen, ammonia, and methanol described in this invention, the optimized scheduling model of the integrated energy system of hydrogen, ammonia, and methanol is reconstructed into a constrained Markov decision process, including: The core components of the Markov decision-making process are clearly defined, and the key information of the optimization scheduling model is mapped to each component. Specifically, the operating state of the integrated energy system of electricity, hydrogen, ammonia and alcohol is used as the state space, the system scheduling decision variables are used as the action space, the system economic optimal goal is used as the reward, and the risk cost of violating the constraints is used as the constraint violation cost. At the same time, risk thresholds and discount factors are set.
[0010] As a preferred embodiment of the optimized scheduling method for the integrated energy system of hydrogen, ammonia, and methanol described in this invention, the method includes: optimizing the constrained Markov decision process using Lagrange multipliers, comprising: To achieve the dual objectives of maximizing cumulative reward and not exceeding the constraint violation risk threshold in the constrained Markov decision-making process, the Lagrange multiplier is introduced into the reward function of the constrained Markov decision-making process. The Lagrange multiplier is used to reward and punish behaviors that violate constraints in the integrated energy system of hydrogen, ammonia, and alcohol. When a scheduling decision triggers a constraint violation, the Lagrange multiplier reduces the corresponding reward value.
[0011] As a preferred embodiment of the optimized scheduling method for the integrated energy system of hydrogen, ammonia, and methanol described in this invention, the method employs a diffusion actor-commentator algorithm that considers information entropy, including: The diffusion actor-critic algorithm, which takes into account information entropy, is used as the core solution tool. An approximator for the policy network is constructed using the reverse process of the diffusion model, while a Gaussian mixture model is combined to process information entropy.
[0012] As a preferred embodiment of the optimized scheduling method for the integrated energy system of hydrogen, ammonia, and methanol described in this invention, the optimal scheduling scheme is obtained by training and solving the optimized scheduling model of the integrated energy system of hydrogen, ammonia, and methanol corresponding to the constrained Markov decision process using a Gaussian mixture model, including: By fitting the action distribution output by the policy network using a Gaussian mixture model, the entropy value of the diffusion policy is estimated, and the exploration degree of the policy is adjusted according to the entropy value. In the simulated operating environment, based on the state space, action space, and optimized reward of the constrained Markov decision process, a policy network and a value assessment network are trained. During the training process, the entropy value estimated by the Gaussian mixture model is used to dynamically adjust the policy to balance economic benefits and constraint compliance. After the training of the policy network and the value assessment network converges, the parameters of the policy network are extracted and substituted into the optimization scheduling model of the integrated energy system of electricity, hydrogen, ammonia, and alcohol. The optimal scheduling scheme that meets the economic optimization objective and complies with the equipment constraints is output.
[0013] Secondly, the present invention provides an optimized scheduling system for an integrated energy system of hydrogen, ammonia, and methanol, comprising: a model building module for acquiring the parameters of each device in the integrated energy system of hydrogen, ammonia, and methanol, and establishing an optimized scheduling model for the integrated energy system of hydrogen, ammonia, and methanol with the goal of economic optimization; The model optimization module is used to reconstruct the established integrated energy system optimization scheduling model of hydrogen, ammonia and alcohol into a constrained Markov decision process, and to optimize the constrained Markov decision process using Lagrange multipliers. The model solving module is used to train and solve the optimal scheduling model of the integrated energy system of electricity, hydrogen, ammonia and alcohol corresponding to the constrained Markov decision process by employing the diffusion actor-commentator algorithm that takes into account information entropy and combining it with a Gaussian mixture model, so as to obtain the optimal scheduling scheme.
[0014] Thirdly, the present invention provides an electronic device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the optimized scheduling method for the integrated energy system of hydrogen, ammonia, and methanol.
[0015] Fourthly, the present invention provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the optimized scheduling method for the electro-hydrogen-ammonia-ethanol integrated energy system.
[0016] Compared with existing technologies, the beneficial effects of this invention are as follows: By constructing a scheduling model that balances economic optimization and multi-device safety constraints, and combining constrained Markov decision process reconstruction and Lagrange multiplier constraint optimization, this invention can effectively avoid the risk of constraint violations by thermal power units, wind and solar power, hydrogen production and storage, and ammonia or methanol synthesis equipment. At the same time, by using the diffusion actor-commentator algorithm that takes into account information entropy and the Gaussian mixture model, the rationality and solution efficiency of strategy exploration are improved. This not only reduces the total operating cost of the system, but also efficiently copes with the volatility of new energy sources to promote the consumption of wind and solar power. The final output optimal scheduling scheme balances economy, safety and real-time performance, and is suitable for the complex operating scenarios of integrated energy systems of electricity, hydrogen, ammonia and methanol, providing technical support for the stable and efficient operation and maintenance of the system. Attached Figure Description
[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 This is a schematic diagram of the overall process of the optimized scheduling method for the integrated energy system of hydrogen, ammonia, and alcohol according to an embodiment of the present invention.
[0019] Figure 2 This is a schematic diagram of an integrated energy system for hydrogen, ammonia, and alcohol, illustrating the optimized scheduling method for such an system according to an embodiment of the present invention.
[0020] Figure 3 This is a schematic diagram of the green hydrogen chemical production operation of the optimized scheduling method of the integrated energy system of electro-hydrogen, ammonia, and alcohol as described in one embodiment of the present invention. Detailed Implementation
[0021] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.
[0022] Example 1, referring to Figure 1As an embodiment of the present invention, an optimized scheduling method for an integrated energy system of electricity, hydrogen, ammonia, and methanol is provided, comprising: S1: Obtain the parameters of each device in the integrated energy system of electricity, hydrogen, ammonia and alcohol, and establish an optimal scheduling model for the integrated energy system of electricity, hydrogen, ammonia and alcohol with the goal of economic optimization; Preferably, based on constraints, an optimal scheduling model for the integrated energy system of electricity, hydrogen, ammonia, and methanol is constructed with the goal of minimizing economic costs. The economic costs include the power generation costs of thermal power units, the wind and solar curtailment penalty costs of the green hydrogen chemical virtual energy pool, the sales revenue of methanol and ammonia, the production and compression costs of hydrogen, the operation and maintenance costs of hydrogen storage tanks, and the power shortage penalty costs.
[0023] Preferably, the constraints include thermal power unit operation constraints, wind and solar power output constraints, hydrogen production and compression constraints, hydrogen storage constraints, ammonia synthesis constraints, methanol synthesis constraints, and power balance constraints.
[0024] In this embodiment of the application, the objective function of the integrated energy system scheduling model for hydrogen, ammonia, and methanol is as follows: (1) In the formula, Cop represents the power generation cost of the thermal power unit; Cdis_ele represents the wind and solar curtailment penalty cost of the green hydrogen chemical virtual energy pool; Csale_MeOH and Csale_NH3 represent the sales revenue of methanol and ammonia, respectively; Cec represents the hydrogen production and compression cost; Chst represents the operation and maintenance cost of the hydrogen storage tank; and Clack represents the electricity gap penalty cost. The calculation formula is shown below: (2) (3) (4) (5) (6) (7) In the formula, Pk,t is the output of the k-th thermal power unit at time t; ak, bk, and ck are the power generation cost coefficients of thermal power unit k; cthermal, cw, and cs are the unit power generation costs of thermal power, wind power, and photovoltaic power respectively; csale_MeOH and csale_NH3 are the selling prices of methanol and ammonia respectively; cec is the unit operating cost of the electrolyzer; ccompress is the unit compression cost of hydrogen; chst is the unit operation and maintenance cost of the hydrogen storage tank; Pg,t EC is the electrical power input of hydrogen production station g at time t; Pg,t compress,H2 is the hydrogen compression power of electrolyzer g at time t; TD is the hourly dispatch cycle, which is 24 hours; cdis w and cdis s are the unit curtailment penalty costs of wind power and photovoltaic power respectively; Pdis w,t represents the curtailed wind power of wind farm w; Pdis s,t represents the curtailed solar power of photovoltaic power station s; ΩHGS represents the set of hydrogen storage tanks; Nk represents the number of thermal power units; NW represents the number of wind farms; NS represents the number of photovoltaic power stations; mg,t H2,HST+ represents the flow rate of hydrogen stored in hydrogen production station g at time t, mg,t; H2,P2X represents the flow rate of hydrogen used for chemical synthesis in hydrogen production station g at time t; mt NH3 represents the rate of ammonia synthesis at time t; mt MeOH represents the methanol yield at time t; clack represents the unit power deficit penalty; Pt lack represents the power deficit power.
[0025] The constraints for thermal power plants are as follows: (8) (9) In the formula, Pmin k and Pmax k are the lower and upper limits of the output of thermal power unit k, respectively; RUk and RDk represent the maximum values of the thermal power unit climbing upward and downward, respectively.
[0026] The power output constraints for wind and solar power are as follows: (10) In the formula, Pmax w,t is the upper limit of the output of wind turbine w at time t; Pmax s,t is the upper limit of the output of photovoltaic unit s at time t.
[0027] The hydrogen production, compression, and storage model is shown below: (11) (12) (13) (14) (15) In the formula, Pg,t au is the power consumed by the auxiliary equipment of hydrogen production station g at time t; αEC is the conversion coefficient between power consumption and hydrogen production flow rate, in MWh / Nm3, i.e., the power consumption to produce one standard cubic meter of hydrogen; mg,t H2 is the hydrogen production flow rate of hydrogen production station g at time t; PEC,min and PEC,max are the lower and upper limits of the input power of the electrolyzer, respectively; Eg,t buffer is the hydrogen storage capacity of the buffer tank in hydrogen production station g at time t; mg,t+1 H2,out is the hydrogen flow rate at the outlet of the buffer tank in hydrogen production station g at time t+1; Δt is the time interval of 1 hour; Ebuffer,min and Ebuffer,max represent the lower and upper limits of the buffer tank storage capacity, respectively; MH2 is the molar mass of hydrogen; R is the ideal gas constant; T is the compressor temperature; p1 and p2 are the pressures of hydrogen before and after compression, respectively; and ηcompress is the compressor efficiency.
[0028] The hydrogen storage model is shown below: (16) (17) In the formula, mg,t H2,out is the outlet hydrogen flow rate of the compressor in hydrogen production station g at time t; mg,t H2,HST+ is the flow rate of hydrogen stored in hydrogen production station g at time t; mg,t H2,P2X is the flow rate of hydrogen used for chemical synthesis in hydrogen production station g at time t; CHST is the capacity of the hydrogen storage tank; Eg,t HST is the energy stored in the hydrogen storage tank of hydrogen production station g at time t; mg,t H2,HST+ and mg,t H2,HST- represent the hydrogen storage and release flow rates of the hydrogen storage tank in hydrogen production station g at time t, respectively; ζmax HST is the maximum capacity coefficient of the hydrogen storage tank; mmax HST+ and mmax HST- represent the upper limits of the hydrogen storage and release flow rates of the hydrogen storage tank, respectively.
[0029] The constraints for power to ammonia (P2A) synthesis are as follows: The P2A process mainly includes air separation, compression buffering, and ammonia synthesis.
[0030] The nitrogen required for ammonia synthesis is obtained from the air separation section, which requires multi-stage compressors to pressurize the nitrogen step by step, consuming the following electrical energy: (18) In the formula, Pt compress,N2 is the power consumption of the compressor at time t; M N2 is the molar mass of nitrogen; p3 and p1 are the pressures of nitrogen before and after compression, respectively; and η compress,N2 is the efficiency of the compressor.
[0031] Since the load adjustment process in ammonia synthesis is relatively slow, and this change process approximates a unit step response process with a time constant of approximately 2 hours, and the time for the unit step response to reach stability is approximately four times the time constant, the scheduling cycle of the ammonia synthesis reactor is taken as ΔTAS = 8 hours. That is, the ammonia synthesis reactor has three adjustment cycles on the time scale, and no further adjustments are made until its state reaches stability again. The model is shown below: (19) The relationship between τh and t is as follows: (20) (twenty one) In the formula, h represents the adjustment period of the ammonia synthesis reactor, with values of 1, 2, and 3; mh NH3 is the initial ammonia production flow rate of the ammonia synthesis reactor in the h-th adjustment period; ∆mh NH3,adj is the adjustment amount in the h-th adjustment period; mτh NH3 is the ammonia production flow rate at time τh; and mmax NH3,adj represents the upper limit of the adjustment amount.
[0032] Because the ammonia synthesis section has significant inertia and time lag in process units such as the synthesis tower, circulation pipelines, and ammonia separation, it is necessary to limit the load ramp-up rate. (twenty two) In the formula, m NH3,min and m NH3,max are the lower and upper limits of ammonia yield, respectively.
[0033] Considering the material balance and conversion rate in the ammonia synthesis process, the relationship between ammonia yield and the injection flow rates of hydrogen and nitrogen in the ammonia synthesis process is as follows: (twenty three) (twenty four) In the formula, λP2A represents the conversion rate of the ammonia synthesis process.
[0034] The constraints for the synthesis of methanol (Power to Methanol, P2MeOH) are as follows: This paper utilizes CO2 captured by carbon capture technology (CCS) as the feedstock for methanol synthesis. CCS technology separates CO2 from the flue gas produced by coal-fired power plants and processes it to obtain high-purity CO2. Its operating model is shown below: (25) (26) In the formula, Pt CCS,P2MeOH is the energy consumption of the carbon capture device at time t, Qt thermal,P2MeOH is the carbon dioxide emitted by the coal-fired power plant used for methanol synthesis at time t, ηCCS,t is the carbon capture rate at time t, λCCS is the energy consumption of capturing one unit of CO2, and Qt CCS,P2MeOH is the amount of carbon captured at time t.
[0035] Where Qthermal,t represents the carbon emissions of the thermal power unit at time t, and its calculation formula is shown below: (27) In the formula, NM represents the number of thermal power units, γn represents the carbon emission factor of the nth thermal power unit, and the unit is kgCO2 / kWh. Pn,t P2MeOH represents the power generation of thermal power unit n used for methanol synthesis at time t.
[0036] The carbon dioxide obtained from the carbon capture section needs to pass through an intermediate condenser and undergo multi-stage compression before finally entering the mixer. The electrical energy consumed in the compression stage is: (28) In the formula, Pt compress,P2MeOH,CO2 is the electrical power consumed by the P2MeOH system to compress CO2 at time t, p4 and p1 are the pressures before and after CO2 compression, respectively, and η compress,CO2 is the efficiency of CO2 compression.
[0037] The yield during methanol synthesis needs to be maintained within a given range: (29) In the formula, mtMeOH represents the methanol yield at time t, and mMeOH,min and mMeOH,max represent the lower and upper limits of the methanol yield, respectively.
[0038] Considering the material balance and conversion rate in the methanol synthesis process, the relationship between methanol yield and the injection flow rates of hydrogen and nitrogen in the methanol synthesis process is as follows: (30) (31) In the formula, λP2MeOH represents the conversion rate in the methanol synthesis process.
[0039] The constraints of fuel cells are as follows: (32) (33) In the formula, Pt FC is the output power of the fuel cell at time t; Pt FC-HST is the input power of the fuel cell at time t; ηEC is the efficiency of the fuel cell; Pmin FC and Pmax FC are the minimum and maximum power of the fuel cell, respectively.
[0040] The power balance constraints are as follows: (34) The hydrogen energy balance constraints are shown below: (35) S2: The established integrated energy system scheduling model of electricity, hydrogen, ammonia and alcohol is reconstructed into a constrained Markov decision process, and the constrained Markov decision process is optimized using Lagrange multipliers; Preferably, the core components of the Markov decision-making process are clearly defined and the key information of the optimized scheduling model is mapped to each component. Specifically, this includes: using the operating state of the integrated energy system of electricity, hydrogen, ammonia and alcohol as the state space, using the system scheduling decision variables as the action space, using the system's economic optimal goal as the reward, and using the risk cost of violating the constraints as the constraint violation cost, while setting risk thresholds and discount factors.
[0041] Preferably, a dual indicator is set in the constrained Markov decision-making process to maximize the cumulative reward and not exceed the constraint violation risk threshold. The Lagrange multiplier is introduced into the reward function of the constrained Markov decision-making process. The Lagrange multiplier is used to apply rewards and penalties to the behavior of the integrated energy system of electricity, hydrogen, ammonia and alcohol that violates the constraints. When the scheduling decision triggers the constraint violation, the Lagrange multiplier reduces the corresponding reward value.
[0042] In this embodiment, the optimal scheduling model of the integrated energy system of hydrogen, ammonia, and methanol is reconstructed into a constrained Markov decision process (CMDP process), as follows: A constrained Markov decision process is defined as a tuple {S, A, P, r, c, d, γ}, where S represents the state space; A represents the action space; P is the state transition probability function; and r represents the state transition probability function. choose The reward received; c indicates that in choose The cost of violating the constraints obtained; d is the risk threshold; γ represents the discount factor.
[0043] In the operation of the integrated energy system of electricity, hydrogen, ammonia, and methanol, the agent formulates scheduling strategies based on the perceived operating status and action strategies. Then, Monte Carlo simulations are used to calculate rewards and constraint violation costs under environmental uncertainty. Subsequently, the value of the strategy is evaluated, and feedback is provided to improve the action strategy.
[0044] The key elements of CMDP formula are explained below: 1) State: The state definition in the constrained Markov decision process is shown in equation (36). It reflects the operating state of the system and provides the necessary input information for scheduling decisions, including the predicted wind and solar load values, the previous state of the hydrogen buffer tank, the previous state of the hydrogen storage tank, the previous ammonia production rate, and the initial production rates of ammonia at each production stage.
[0045] (36) 2) Actions: Actions involve decision variables for formulating scheduling strategies under the current state, as shown in Equation (37), including the output of wind, solar and thermal power units, power shortage, hydrogen production rate, and the adjustment amount of ammonia production rate at each stage.
[0046] (37) 3) Cost: Cost is related to constraint violations caused by the agent's random exploration in an uncertain environment. This cost forms the basis for the agent to meet the constraint violation risk threshold, ensuring the safe operation of the system.
[0047] The optimal scheduling model for the integrated energy system of hydrogen, ammonia, and methanol can be expressed in the following compact form: (38) The cost is defined as follows: (39) 4) Rewards: Rewards are a key factor driving the agent to learn the optimal objective of the integrated energy system scheduling model (electro-hydrogen-ammonia-ethanol). Rewards can be expressed as a combination of the integrated energy system's operating costs and the revenue from producing chemical products, as shown in the following formula: (40) CMDP-based security exploration technology achieves this by treating violation risk as an exploration cost and using a risk threshold as a cost objective. The learning strategy aims to maximize cumulative discounted rewards while satisfying violation cost constraints.
[0048] (41) in (42) (43) (44) The CMDP process for the integrated energy system of hydrogen, ammonia, and methanol will be optimized and scheduled as follows: To account for the Constrained Markov Decision Process (CMDP) process in deep reinforcement learning algorithms, Lagrange multipliers are added to the reward function to penalize constraint violations. Therefore, the Constrained Markov Decision Process problem is reformulated as: (45) S3: The diffusion actor-commenter algorithm that takes information entropy into account is used to train and solve the optimal scheduling model of the integrated energy system of electric hydrogen, ammonia and alcohol corresponding to the constrained Markov decision process, and the optimal scheduling scheme is obtained. Preferably, the diffusion actor-critic algorithm, which takes into account information entropy, is used as the core solution tool. The approximator of the policy network is constructed by the reverse process of the diffusion model, and the information entropy is processed by combining the Gaussian mixture model.
[0049] Preferably, the action distribution output by the policy network is fitted by a Gaussian mixture model to estimate the entropy value of the diffusion policy, and the exploration degree of the policy is adjusted according to the entropy value. In the simulated operation environment, the policy network and the value evaluation network are trained based on the state space, action space and optimized reward of the constrained Markov decision process. During the training process, the policy is dynamically adjusted in combination with the entropy value estimated by the Gaussian mixture model to balance economic benefits and constraint compliance. After the training of the policy network and the value evaluation network converges, the parameters of the policy network are extracted and substituted into the optimization scheduling model of the integrated energy system of electricity, hydrogen, ammonia and alcohol, and the optimal scheduling scheme that meets the economic optimization goal and the equipment constraint conditions is output.
[0050] In this embodiment, the solution method for the integrated energy system optimization scheduling model based on the diffusion actor-critic algorithm taking into account information entropy is as follows: Diffusion models are a class of efficient generative models. Their core idea is to progressively add noise to the data, transforming the original data distribution into a Gaussian noise distribution, and then reconstructing the data by progressively removing the noise through a reverse process. This process is typically described as a continuous Markov chain: the forward process progressively increases the noise level, while the reverse process trains a conditional generative model to predict the optimal transition process for each denoising step. Ultimately, the model can generate data samples by reversing the diffusion sequence from pure noise.
[0051] definition ,in Representation and data variables Implicit variables with the same dimension This represents the original data distribution. In the forward diffusion chain, noise is added to the data step-by-step over T steps. In this process, the process follows a pre-defined variance sequence. The mathematical expression is: (46) when When xT follows an isotropic Gaussian distribution.
[0052] The reverse diffusion process in the diffusion model can be represented as: (47) when hour, .
[0053] The design of the Diffusion Actor-Commentator (DACER) algorithm, which incorporates an entropy regulator, is as follows: First, the reverse process of the diffusion model is treated as a novel policy approximator, serving as the policy function in reinforcement learning. Second, the diffusion policy is directly optimized using gradient descent, with the objective function being to maximize the expected Q-value—a characteristic that allows it to be combined with mainstream reinforcement learning algorithms that do not require entropy. However, diffusion policies learned solely in this way produce overly deterministic actions, leading to poor performance. When attempting to combine it with a maximum entropy reinforcement learning framework, we find that the entropy of the diffusion policy is difficult to solve analytically. Therefore, this paper uses a Gaussian mixture model (GMM) to estimate the entropy of the diffusion policy, and then learns the parameter α based on this estimate to adjust the exploration level of the diffusion policy.
[0054] Using the inverse process of the conditional diffusion model as a parameterization strategy, the mathematical expression is: (48) in The final sample a0 of the reverse chain is the action used for reinforcement learning evaluation. Typically, It can be modeled as a Gaussian distribution. , This article will Set as a time-dependent constant And construct the mean through a noise prediction model. : (49) in , , This is a parameterized noise prediction model.
[0055] To obtain actions from a Gaussian diffusion model, sampling needs to be performed sequentially from T different Gaussian distributions. The sampling process can be rewritten as follows: (50) in, t is the reverse time step from T to 0, and .
[0056] The policy trained solely by the aforementioned diffusion policy learning method produces overly deterministic actions, resulting in poor performance of the final diffusion policy. Furthermore, the distribution of the diffusion policy lacks an analytical expression, making it impossible to directly calculate its entropy. However, under the same state, a series of actions can be obtained through multiple samplings, and by fitting these action points, the action distribution corresponding to that state can be estimated.
[0057] This invention employs a Gaussian Mixture Model (GMM) to fit the distribution. The GMM combines multiple Gaussian distributions to form a complex probability density function, the mathematical expression of which is: (51) Where K is the number of Gaussian distributions. The mixing weights for the k-th component (satisfying) and ), and are the mean and covariance matrix of the k-th Gaussian distribution, respectively.
[0058] For each state, we sample N actions using a diffusion strategy. Then, the parameters of the GMM are estimated using the Expectation-Maximization (EM) algorithm. In the expectation step, the posterior probability of each data point ai belonging to each component k is calculated, denoted as: (52) in, This represents the probability that the observed data ai comes from the k-th component under the current parameter estimate. In the maximization step, the result of the following equation will be used to update the parameters and mixing weights for each component: (53) Repeat the expected step and the maximization step as described above until the parameters converge.
[0059] According to formula (51), the entropy of the action distribution corresponding to this state can be estimated by the following formula: (54) Where d represents the action dimension. Subsequently, the mean of the action entropy corresponding to the selected batch states is used as the estimated entropy of the diffusion strategy. .
[0060] Similar to maximum entropy reinforcement learning, we update α based on the estimated entropy learning parameter α using the following formula: (55) in, The target entropy. Ultimately, during training... Adjust the entropy of the diffusion strategy (λ is a hyperparameter, a is the output action of the diffusion strategy); no noise is added during the evaluation phase.
[0061] The security diffusion reinforcement learning training process, taking information entropy into account, is as follows: ,
[0062] The commentator network loss function Cost network loss function And actor network loss function The distribution is shown below: (56) (57) (58) It should be noted that this invention constructs a scheduling model that balances economic optimization and multi-device safety constraints. By combining constrained Markov decision process reconstruction and Lagrange multiplier constraint optimization, it can effectively avoid the risk of constraint violations by thermal power units, wind and solar power, hydrogen production and storage, and ammonia or methanol synthesis equipment. At the same time, by using the diffusion actor-commentator algorithm that takes into account information entropy and the Gaussian mixture model, it improves the rationality and solution efficiency of strategy exploration. This not only reduces the total operating cost of the system, but also efficiently copes with the volatility of new energy sources to promote the consumption of wind and solar power. The final output optimal scheduling scheme balances economy, safety, and real-time performance, and is suitable for the complex operating scenarios of the integrated energy system of electricity, hydrogen, ammonia, and methanol, providing technical support for the stable and efficient operation and maintenance of the system.
[0063] Example 2, refer to Figures 2-3 Tables 1 and 2 are embodiments of the present invention. Based on the above embodiments, an optimized scheduling method for an integrated energy system of hydrogen, ammonia, and methanol is provided.
[0064] like Figure 2 As shown, this application embodiment uses the HRP-38 node test system for simulation analysis to verify the rationality and adaptability of the proposed method. The total installed capacity of thermal power is 261.6 GW, while the installed capacities of wind and solar power are 100 GW and 149.5 GW, respectively. The penetration rate of new energy is approximately 49%, and the peak electricity load is 5360 MW. Hydrogen energy storage is configured at 5% to 10% of the total installed capacity. Three wind farms and three photovoltaic power stations are added to the system, and green hydrogen to ammonia, green hydrogen to methanol, and green hydrogen to methane systems are respectively configured within the new energy stations. Each green hydrogen chemical system is equipped with a buffer tank with a capacity of 1200 kg, and 10 long-tube trailers are configured to realize hydrogen transportation between different hydrogen chemical industrial parks.
[0065] The operational results under different schemes in typical summer / winter scenarios and the production results of the Green Hydrogen Chemical Virtual Energy Pool are shown in Table 1 and Table 2, respectively.
[0066] Table 1. Simulation results of different schemes for typical summer / winter scenarios
[0067] Table 2 Results of renewable energy consumption in different calculation examples
[0068] As shown in Table 1, the operating costs under typical summer / winter scenarios are 40.95 billion yuan and 40.45 billion yuan, respectively. The operating costs have decreased by 470 million yuan and 280 million yuan, respectively. This is because the model proposed in this invention increases the utilization rate of hydrogen energy, significantly improves the yield of downstream hydrogen chemical products, and effectively reduces the amount of wind and solar power curtailed.
[0069] Table 2 shows that the curtailment rate is 660 GW under typical summer and winter scenarios. This indicates that a flexible storage and transportation model for hydrogen energy and the dynamic adjustment characteristics of green hydrogen chemical production can effectively reduce the curtailment rate of new energy sources and promote their consumption.
[0070] The chemical production and hydrogen energy utilization of the virtual energy pool for green hydrogen chemistry in a summer scenario are as follows: Figure 3 The figure shows the trends in load factor, electrolyzer hydrogen production flow rate, buffer tank outlet hydrogen flow rate, and buffer tank hydrogen storage capacity in chemical production. Due to limitations in chemical production and the volatility of new energy sources, ammonia production slowly increased and stabilized at 34 t / h, methanol production remained stable at 29 t / h for most of the time, while methane production fluctuated significantly, reaching a maximum of 25 t / h. The chemical production rates can be flexibly adjusted to adapt to wind and solar energy fluctuations, thereby increasing the electrolyzer hydrogen production flow rate and the buffer tank utilization rate. The above analysis indicates that considering the flexible adjustment capabilities of chemical production can effectively improve energy consumption levels and hydrogen energy utilization.
[0071] Example 3 illustrates an optimized scheduling method for an integrated energy system of electricity, hydrogen, ammonia, and methanol. It should be noted that the technical solution of this optimized scheduling system for the integrated energy system of electricity, hydrogen, ammonia, and methanol is based on the same concept as the optimized scheduling method for the integrated energy system of electricity, hydrogen, ammonia, and methanol described above. Details not described in detail in this example can be found in the description of the optimized scheduling method for the integrated energy system of electricity, hydrogen, ammonia, and methanol described above.
[0072] This embodiment also provides an optimized scheduling system for an integrated energy system of hydrogen, ammonia, and methanol, including: The model building module is used to obtain the parameters of each device in the integrated energy system of electricity, hydrogen, ammonia and alcohol, and to establish an optimal scheduling model for the integrated energy system of electricity, hydrogen, ammonia and alcohol with the goal of economic optimization. The model optimization module is used to reconstruct the established integrated energy system scheduling model of electricity, hydrogen, ammonia and alcohol into a constrained Markov decision process, and to optimize the constrained Markov decision process using Lagrange multipliers. The model solving module is used to train and solve the optimal scheduling model of the integrated energy system of electricity, hydrogen, ammonia and alcohol corresponding to the constrained Markov decision process by employing the diffusion actor-commentator algorithm that takes into account information entropy and combining it with a Gaussian mixture model, so as to obtain the optimal scheduling scheme.
[0073] This embodiment also provides an electronic device applicable to the optimized scheduling of an integrated energy system of hydrogen, ammonia, and methanol, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the optimized scheduling method for the integrated energy system of hydrogen, ammonia, and methanol as proposed in the above embodiment.
[0074] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the optimized scheduling method for the integrated energy system of hydrogen, ammonia, and methanol as proposed in the above embodiments.
[0075] The storage medium proposed in this embodiment and the optimized scheduling method for the integrated energy system of hydrogen, ammonia, and alcohol proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.
[0076] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.
[0077] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. An optimized scheduling method for an integrated energy system of electricity, hydrogen, ammonia, and methanol, characterized in that, include: Obtain the parameters of each device in the integrated energy system of electricity, hydrogen, ammonia, and methanol, and establish an optimal scheduling model for the integrated energy system of electricity, hydrogen, ammonia, and methanol with the goal of economic optimization; The established integrated energy system optimization scheduling model of hydrogen, ammonia, and alcohol is reconstructed into a constrained Markov decision process, and the constrained Markov decision process is optimized using Lagrange multipliers. The diffusion actor-critic algorithm, which takes into account information entropy, is used in conjunction with a Gaussian mixture model to train and solve the optimal scheduling model of the integrated energy system of electricity, hydrogen, ammonia, and alcohol corresponding to the constrained Markov decision process, so as to obtain the optimal scheduling scheme.
2. The optimized scheduling method for the integrated energy system of hydrogen, ammonia, and methanol as described in claim 1, characterized in that, An optimal scheduling model for the integrated energy system of electricity, hydrogen, ammonia, and methanol, with economic optimization as its objective, is established, including: Based on constraints, an optimal scheduling model for the integrated energy system of electricity, hydrogen, ammonia, and methanol is constructed with the goal of minimizing economic costs. The economic costs include the power generation costs of thermal power units, the wind and solar curtailment penalty costs of the green hydrogen chemical virtual energy pool, the sales revenue of methanol and ammonia, the production and compression costs of hydrogen, the operation and maintenance costs of hydrogen storage tanks, and the power shortage penalty costs.
3. The optimized scheduling method for the integrated energy system of hydrogen, ammonia, and methanol as described in claim 2, characterized in that, The constraints include thermal power unit operation constraints, wind and solar power output constraints, hydrogen production and compression constraints, hydrogen storage constraints, ammonia synthesis constraints, methanol synthesis constraints, and power balance constraints.
4. The optimized scheduling method for the integrated energy system of hydrogen, ammonia, and methanol as described in claim 3, characterized in that, The established optimal scheduling model for the integrated energy system of hydrogen, ammonia, and methanol will be reconstructed using a constrained Markov decision process, including: The core components of the Markov decision-making process are clearly defined, and the key information of the optimization scheduling model is mapped to each component. Specifically, the operating state of the integrated energy system of electricity, hydrogen, ammonia and alcohol is used as the state space, the system scheduling decision variables are used as the action space, the system economic optimal goal is used as the reward, and the risk cost of violating the constraints is used as the constraint violation cost. At the same time, risk thresholds and discount factors are set.
5. The optimized scheduling method for the integrated energy system of hydrogen, ammonia, and methanol as described in claim 4, characterized in that, Optimization of constrained Markov decision processes using Lagrange multipliers includes: To achieve the dual objectives of maximizing cumulative reward and not exceeding the constraint violation risk threshold in the constrained Markov decision-making process, the Lagrange multiplier is introduced into the reward function of the constrained Markov decision-making process. The Lagrange multiplier is used to reward and punish behaviors that violate constraints in the integrated energy system of hydrogen, ammonia, and alcohol. When a scheduling decision triggers a constraint violation, the Lagrange multiplier reduces the corresponding reward value.
6. The optimized scheduling method for the integrated energy system of hydrogen, ammonia, and methanol as described in claim 1, characterized in that, The diffusion actor-commentator algorithm, which takes into account information entropy, is employed, including: The diffusion actor-critic algorithm, which takes into account information entropy, is used as the core solution tool. An approximator for the policy network is constructed using the reverse process of the diffusion model, while a Gaussian mixture model is combined to process information entropy.
7. The optimized scheduling method for the integrated energy system of hydrogen, ammonia, and methanol as described in claim 6, characterized in that, By combining Gaussian mixture model training to solve the constrained Markov decision process corresponding to the optimal scheduling model of the integrated energy system of electricity, hydrogen, ammonia, and alcohol, the optimal scheduling scheme is obtained, including: By fitting the action distribution output by the policy network using a Gaussian mixture model, the entropy value of the diffusion policy is estimated, and the exploration degree of the policy is adjusted according to the entropy value. In the simulated operating environment, based on the state space, action space, and optimized reward of the constrained Markov decision process, a policy network and a value assessment network are trained. During the training process, the entropy value estimated by the Gaussian mixture model is used to dynamically adjust the policy to balance economic benefits and constraint compliance. After the training of the policy network and the value assessment network converges, the parameters of the policy network are extracted and substituted into the optimization scheduling model of the integrated energy system of electricity, hydrogen, ammonia, and alcohol. The optimal scheduling scheme that meets the economic optimization objective and complies with the equipment constraints is output.
8. An optimized scheduling system for an integrated energy system of electricity, hydrogen, ammonia, and methanol, comprising the method described in any one of claims 1-7, characterized in that, include: The model building module is used to obtain the parameters of each device in the integrated energy system of electricity, hydrogen, ammonia and alcohol, and to establish an optimal scheduling model for the integrated energy system of electricity, hydrogen, ammonia and alcohol with the goal of economic optimization. The model optimization module is used to reconstruct the established integrated energy system optimization scheduling model of hydrogen, ammonia and alcohol into a constrained Markov decision process, and to optimize the constrained Markov decision process using Lagrange multipliers. The model solving module is used to train and solve the optimal scheduling model of the integrated energy system of electricity, hydrogen, ammonia and alcohol corresponding to the constrained Markov decision process by employing the diffusion actor-commentator algorithm that takes into account information entropy and combining it with a Gaussian mixture model, so as to obtain the optimal scheduling scheme.
9. An electronic device, characterized in that, include: A memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the optimized scheduling method for the integrated energy system of hydrogen, ammonia, and methanol as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, It stores computer-executable instructions, which, when executed by a processor, implement the steps of the optimized scheduling method for the integrated energy system of hydrogen, ammonia, and methanol as described in any one of claims 1 to 7.