Energy optimization scheduling method and system for integrated energy system based on reinforcement learning

Abstract

The present application provides an energy optimization scheduling method and system for an integrated energy system based on reinforcement learning, which relates to the technical field of integrated energy systems. The method comprises: collecting operating data of each device in the integrated energy system; using a heuristic algorithm to solve the planned value of the output of the generator set of the integrated energy system based on the operating data, the energy conversion model of each operating device and the corresponding constraints, with the minimum system operating cost and carbon emissions as the optimization goal; establishing a Markov decision process in a finite space, determining the optimized value of the output of the generator set through a reinforcement learning model, and controlling the operation of the equipment according to the optimized value of the output of the generator set; wherein the operating data and the planned value of the output of the generator set are used as the input and output of the neural network respectively, the neural network is trained, and the trained neural network is used as the strategy network of the reinforcement learning model. In this way, real-time optimized operation of the integrated energy system can be achieved.

Description

Technical Field

[0001] The present application relates to the technical field of integrated energy systems, and in particular to an energy optimization scheduling method and system for integrated energy systems based on reinforcement learning. Background Art

[0002] The statements in this section merely provide background technical information related to the present application and do not necessarily constitute the prior art that has been known to those skilled in the art.

[0003] Integrated energy systems can adapt to local conditions and provide users with multiple energy sources such as electricity, heat, and cooling. They are one of the effective solutions for improving energy utilization, reducing CO2 emissions, increasing the proportion of renewable energy, and achieving the "dual carbon" goals. Optimal scheduling methods guarantee the safe, efficient, and economical operation of integrated energy systems. Traditional optimal scheduling methods primarily involve establishing a system equipment model; establishing a system optimization model; and solving the optimization model based on predicted data. This method is characterized by high solution efficiency for deterministic systems.

[0004] However, due to the high randomness of renewable energy and user loads, there may be deviations between predicted data and actual operation, resulting in inaccurate optimization results. Existing heuristic methods struggle to ensure the real-time optimal operation of integrated energy systems. Furthermore, heuristic algorithms are prone to falling into local optimal solutions when solving, making it difficult to find the global optimal solution. Deep reinforcement learning excels at solving continuous sequence decision-making problems. In the optimization of integrated energy systems, extensive training on historical data can achieve real-time optimal scheduling of the system and reduce the impact of source and load randomness on the system's optimal scheduling. However, reinforcement learning itself has problems such as low learning efficiency and convergence being significantly affected by hyperparameters. It requires long and extensive training to obtain an optimized scheduling solution. Therefore, how to obtain more accurate optimal scheduling results when the integrated energy system is operating in real time remains an unresolved issue. Summary of the Invention

[0005] In order to solve the above problems, the present application provides an energy optimization scheduling method and system for an integrated energy system based on reinforcement learning. The planned value of the output of the generator set of the integrated energy system is solved by a heuristic algorithm, and a neural network is trained according to the operating data and the planned value. The trained neural network is used as the strategy network of the reinforcement learning model. After a finite number of iterations, the optimized value of the output of the generator set is obtained to realize real-time optimized operation of the integrated energy system.

[0006] To achieve the above objectives, this application mainly includes the following aspects:

[0007] In a first aspect, an embodiment of the present application provides an energy optimization scheduling method for an integrated energy system based on reinforcement learning, the energy optimization scheduling method comprising:

[0008] Collect operating data of each device in the integrated energy system;

[0009] Based on the operating data, the energy conversion model of each operating device and the corresponding constraints, a heuristic algorithm is used to solve the planned output value of the generator set of the integrated energy system with the minimum system operating cost and carbon emissions as the optimization goal;

[0010] A Markov decision process in a finite space is established, the optimized value of the generator set output is determined through a reinforcement learning model, and the operation of the equipment is controlled according to the optimized value of the generator set output; wherein the operating data and the planned value of the generator set output are used as the input and output of the neural network respectively, the neural network is trained, and the trained neural network is used as the policy network of the reinforcement learning model.

[0011] In a possible implementation, the heuristic algorithm includes but is not limited to a genetic algorithm, an ant colony algorithm, a particle swarm optimization algorithm, and a simulated annealing algorithm.

[0012] In one possible implementation, collecting operating data of each device in the integrated energy system includes collecting photovoltaic power station power generation data, wind power station power generation data, electricity purchase price, electricity sales price, user electricity load data, and user cooling load data at a specific sampling frequency.

[0013] In one possible implementation, the Markov decision process includes the state, action, and reward function of the Markov decision process; defining the state s t {P pv (t),P wt (t),λ t ,μ t , P e (t),P c (t)}, action a t {P pgu,e (t)}, reward r t {-(f oc (t)+f es (t))};

[0014] Among them, t is the equipment operation period, s t is the state of the integrated energy system during the equipment operation period t, P pv (t) is the power generation data of the photovoltaic power station when the equipment is running during the t period, P wt (t) is the wind power station power generation data when the equipment is running at time t, λ t is the electricity purchase price when the equipment is running at time t, μ t is the electricity price when the equipment is running at time t, P e (t) is the user's electric load data when the equipment is running during the t period, Pc (t) is the user cooling load data when the equipment is running during the t period, a t is the action of the integrated energy system during the equipment operation period t, P pgu,e (t) is the generator output when the equipment is running at time t, r t is the reward of the integrated energy system during the equipment operation period t, f oc (t) is the system operation cost of the integrated energy system during the equipment operation period t, f es (t) is the system operation carbon emission of the integrated energy system during the equipment operation period t.

[0015] In one possible implementation, the training process of the reinforcement learning model includes:

[0016] Establish a virtual environment for a comprehensive energy system, which outputs actions a through reinforcement learning agents t , feedback reward r t and the state s at the next moment t+1 ;

[0017] Load the trained policy network and value network, and the action network is used to input the current state s t Output action a t , the value network is used to input state s t , a t , output action value q t ;

[0018] Initialize the target network of the above policy network and value network;

[0019] Initialize t=1 and get the current state s t ;

[0020] If t is less than the preset period T, according to the current state s t Select action a through the policy network t , and add appropriate motion noise;

[0021] According to the output action a t , get the next moment state s through virtual environment feedback t+1 and reward r t , update the target network until t is greater than the preset period T, and update the current state.

[0022] In one possible implementation, experience information {s is constructed at the end of each cycle during the training of the reinforcement learning model. t ,a t ,r t ,s t+1}, and store it in the experience pool.

[0023] In one possible implementation, it is determined whether the experience pool is full. If the experience pool is full, the value network parameters are updated using a gradient descent method, and the action network is updated with a delayed gradient ascent algorithm.

[0024] In a second aspect, an embodiment of the present application provides an energy optimization scheduling system for an integrated energy system based on reinforcement learning, comprising:

[0025] The acquisition module is used to collect the operating data of each device in the integrated energy system;

[0026] a solution module for solving the planned output value of the generator set of the integrated energy system using a heuristic algorithm based on the operating data, the energy conversion model of each operating device and the corresponding constraints, with the system operating cost and carbon emissions minimized as the optimization goal;

[0027] A control module is used to establish a Markov decision process in a finite space, determine the optimal value of the generator set output through a reinforcement learning model, and control the operation of the equipment according to the optimized value of the generator set output; wherein the operating data and the planned value of the generator set output are used as the input and output of the neural network respectively, the neural network is trained, and the trained neural network is used as the strategy network for reinforcement learning.

[0028] In a third aspect, an embodiment of the present application provides a computer device comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the energy optimization scheduling method described in the first aspect and any possible implementation scheme of the first aspect are performed.

[0029] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the energy optimization scheduling method described in the first aspect and any possible implementation scheme of the first aspect are executed.

[0030] The beneficial effects of this application are:

[0031] 1. A heuristic algorithm is used to solve the planned output value of the generator set of the integrated energy system, and a neural network is trained based on the operating data and the planned value. The trained neural network is used as the policy network of the reinforcement learning model. After a finite number of iterations, the optimized value of the generator set output is obtained. This method can overcome the shortcomings of both heuristic algorithms and deep reinforcement learning, and realize the real-time optimized operation of the integrated energy system.

[0032] 2. A method for optimizing the scheduling of integrated energy systems that integrates heuristic algorithms with deep reinforcement learning is proposed. By leveraging the "experience" of the heuristic algorithm and appropriately adjusting action noise during training, the policy network achieves rapid convergence and better explores the global optimal solution. Secondly, neural networks can map the correspondence between continuous system states and executed actions. Compared to heuristic algorithms, they can better address the problem of inaccurate optimization scheduling results caused by source-load tracking during the actual operation of integrated energy systems. Furthermore, they can cyclically update the optimization scheduling model based on real-time operating data, resulting in more accurate scheduling results. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The drawings in the specification, which constitute a part of this application, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0034] Figure 1 This is one of the flow charts of the energy optimization scheduling method provided in the embodiment of the present application;

[0035] Figure 2 is a schematic structural diagram of the integrated energy system provided by an embodiment of the present invention;

[0036] Figure 3 This is the second flow chart of the energy optimization scheduling method provided in the embodiment of the present application;

[0037] Figure 4 It is a structural diagram of the energy optimization scheduling system provided in an embodiment of the present application;

[0038] Figure 5 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The present application will be further described below with reference to the accompanying drawings and embodiments.

[0040] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0041] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0042] Example 1

[0043] See also Figure 1 , Figure 1 This is one of the flow charts of the energy optimization scheduling method provided in the embodiment of the present application, such as Figure 1 As shown in , the energy optimization scheduling method specifically includes the following steps:

[0044] S101: Collecting the operating data of each device in the integrated energy system;

[0045] In the specific implementation, the historical power generation data of photovoltaic power stations and wind power stations are collected and recorded as P pv (t),P wt (t); collect historical electricity purchase and sales prices, denoted as λ(t), μ(t); collect user electricity load and cooling load demand, denoted as P e (t),P c (t); where t represents 1, 2, 3, ..., 24, 24 hours of a day. Specifically, the operating data can be collected by sensing equipment such as a power meter.

[0046] S102: Based on the operating data, the energy conversion model of each operating device and the corresponding constraints, and with minimizing system operating costs and carbon emissions as optimization goals, a heuristic algorithm is used to solve the planned output value of the generator sets of the integrated energy system;

[0047] In a specific embodiment, Figure 2 As shown in the figure, the integrated energy system includes wind and solar generators, power grids, gas generators, electric chillers, heat recovery equipment, absorption chillers, electrical loads and cooling loads. Based on the energy conversion relationship of each operating device, an integrated energy system equipment model is established, specifically including:

[0048] i) Gas generator set model:

[0049] P pgu,gas (t) = P pgu,e (t) / COP pgu,e (t);

[0050] P pgu,heat (t)=(P pgu,gas (t)-P pgu,e (t))×COP pgu,heat ;

[0051] Among them, P pgu,gas (t) represents the gas power consumed by the gas generator, kW; P pgu,e (t) represents the generator power, kW; COP pgu,e (t) represents the real-time power generation efficiency of the generator; P pgu,heat(t) represents the waste heat power of the generator, kW; COP pgu,heat Represents the thermal efficiency of the generator.

[0052] ii) Absorption Refrigeration Machine Model:

[0053] P ec (t) = P pgu,heat (t)×COP ec ;

[0054] Among them, P ec (t) represents the cooling power output of the absorption chiller, kW; COP ec Indicates the efficiency of the absorption chiller.

[0055] iii) Electric Refrigerator Model:

[0056] P ac (t) = P grid,ac (t)×COP ac ;

[0057] Among them, P ac (t) represents the output power of the electric refrigerator, kW; P grid ,ac(t) represents the input power of the electric refrigerator, kW; COP ac Indicates the efficiency of the electric refrigerator.

[0058] Establishing an integrated energy system optimization model:

[0059] Electric power balance constraints:

[0060] P grid (t)+P pgu,e (t)+P pv (t)+P wt (t) = P e (t)+P grid,ac (t);

[0061] Cold power balance constraints:

[0062] P ec (t)+P ac (t) = P c (t)

[0063] Other constraints:

[0064] 0≤P pgu,e (t)≤P pgu,max

[0065] 0≤P ec (t)≤P ec,max

[0066] 0≤P ac(t)≤P ac,max

[0067] Among them, P grid (t) represents the power purchased / sold by the power grid, kW. A value greater than 0 indicates purchasing power from the power grid, and a value less than 0 indicates selling power to the power grid. P pgu,max 、P ec,max 、P ac,max They represent the maximum output power of gas generator, electric refrigerator and absorption refrigerator respectively.

[0068] Determine the objective function:

[0069] F(t)=min(f oc (t)+f es (t));

[0070] f oc (t) = λ t ×P grid (t)+C e ×P pgu,gas (t)P grid (t)>0;

[0071] f es (t) = μ t ×P grid (t)+C e ×P pgu,gas (t)P grid (t)<0;

[0072] Where, f oc represents the system operating cost; f es Indicates the carbon emissions of system operation; C e represents the natural gas cost coefficient; P grid (t)>0 means the system needs to purchase electricity from the grid, otherwise it means selling electricity to the grid.

[0073] As an optional embodiment, the generator power P pgu,e (t) is the decision variable, the system operation cost and carbon emissions are minimized as the optimization goal, and the heuristic algorithm is used to solve the planned value of the hourly generator output of the historical data, that is, to obtain the optimal decision variable P pgu,e,op (t). Here, the heuristic algorithm includes but is not limited to a genetic algorithm, an ant colony algorithm, a particle swarm optimization algorithm, a simulated annealing algorithm, and the like.

[0074] S103: Establish a Markov decision process in a finite space, determine the optimal value of the generator set output through a reinforcement learning model, and control the operation of the equipment according to the optimized value of the generator set output; wherein the operating data and the planned value of the generator set output are used as the input and output of the neural network respectively, the neural network is trained, and the trained neural network is used as the policy network of the reinforcement learning model.

[0075] In the specific implementation, the optimization scheduling problem of the integrated energy system is converted into a Markov decision process with a finite space, and the state s is defined. t {P pv (t),P wt (t),λ t ,μ t , P e (t),P c (t)}, action a t {P pgu,e (t)}, reward r t {-(f oc (t)+f es (t))}.

[0076] Build a strategic neural network, randomly initialize the parameter θ, and convert the P obtained in the heuristic algorithm pgu,e,op (t) as action a t And the corresponding status data s t , train the policy neural network and update the parameters θ.

[0077] Building a virtual environment for an integrated energy system and training an integrated energy system optimization scheduling model based on the TD3 (Twin Delayed Deep Deterministic Policy) algorithm includes the following steps:

[0078] (1) Establish a virtual environment for an integrated energy system, which outputs actions a through reinforcement learning agents. t , feedback reward r t and the state s at the next moment t+1 .

[0079] (2) Load the trained policy network with parameter θ, initialize the two value networks in the TD3 algorithm, and randomly initialize the parameters as φ1 and φ2. The action network is used to input s t Output action a t , the value network is used to input state s t , a t , output action value q t ; Initialize the experience pool with a capacity of M to store training experience.

[0080] (3) Initialize the target network of the above policy network and value network with the same parameters as above, denoted as θ targ 、φ targ,1 、φ targ,2 ,The target network is used to solve the overestimation problem that occurs during the training process.

[0081] (4) Initialize t=1 and obtain the current state s t .

[0082] (5) If t is less than the preset period T, according to the current state s t Select action a through the policy network t , and adding action noise appropriately can help reinforcement learning find the global optimal solution. Here, in practical applications, T represents 24 hours a day, that is, T = 24.

[0083] (6)According to the output action a t , get the next moment state s through virtual environment feedback t+1 and reward r t .

[0084] (7) Constructing Experience Information t ,a t ,r t ,s t+1}, and store it in the experience pool.

[0085] (8) Determine whether the experience pool is full. If it is full:

[0086] Update the value network parameters φ1 and φ2 by gradient descent:

[0087]

[0088]

[0089]

[0090] Among them, y(r t ,s t+1 ) represents the target value of the network; represents the gradient of the loss function; B represents the number of experience sequences extracted from the experience pool; α represents the learning rate of the value network.

[0091] Update the action network with delay through gradient ascent algorithm:

[0092]

[0093]

[0094] Where β represents the learning rate of the action network.

[0095] Since the policy network parameters have been updated to a better value using a heuristic algorithm before training, the learning rates α and β are set to a smaller value, which can be 0.0001, which is conducive to finding a better network parameter.

[0096] Update the target network parameters:

[0097] φ targ，i ←ρφ targ，i +(1-ρ)φ i for i=1,2

[0098] θ targ ←ρθ targ +(1-ρ)θ

[0099] (9) t=t+1, if t>T, return to step (4), otherwise, s t =s t+1 , return to step (5) to execute.

[0100] (10) Stop until the number of training times is reached.

[0101] After the training is completed, the real-time optimization scheduling method (strategy neural network) of the integrated energy system can be obtained. Its real-time nature is reflected in the current input state, and the optimal output action of the integrated energy system can be obtained in a very short time (seconds).

[0102] like Figure 3 As shown in , the embodiment of the present application first uses a heuristic algorithm to optimize the decision variables (generator output) of each hour of historical data, and uses the historical data and the optimized decision variables as the input and output (label) of the neural network, respectively, to train the neural network until convergence; Subsequently, the neural network is used as the policy network of reinforcement learning (Twin Delay Deep Deterministic Policy Gradient) to speed up the training speed of reinforcement learning. At the same time, in order to avoid the heuristic algorithm from falling into the local optimum and affecting the reinforcement learning to also fall into the local optimum, the noise of the reinforcement learning action (decision variable) is appropriately adjusted to explore the global optimization value. This method can effectively overcome the problem that the heuristic algorithm is difficult to handle the randomness of the system and cannot obtain the global optimal solution. At the same time, the method accelerates the convergence of reinforcement learning. When running in real time, while obtaining the optimized scheduling, the real-time data is recorded, fed back to the deep reinforcement learning and trained, and the online update optimization scheduling scheme model can be realized, so that the optimized scheduling result is more accurate.

[0103] Example 2

[0104] See also Figure 4 , Figure 4This is a schematic diagram of the structure of the energy optimization scheduling system provided in the embodiment of the present application. Figure 4 As shown in , the energy optimization scheduling system 400 provided in the embodiment of the present application includes:

[0105] The collection module 410 is used to collect the operating data of each device in the integrated energy system;

[0106] A solution module 420 is configured to solve a planned output value of a generator set of the integrated energy system based on the operating data, the energy conversion model of each operating device, and corresponding constraints, with the system operating cost and carbon emissions minimized as optimization objectives;

[0107] The control module 430 is used to establish a Markov decision process in a finite space, determine the optimized value of the generator set output through a reinforcement learning model, and control the operation of the equipment according to the optimized value of the generator set output; wherein the operating data and the planned value of the generator set output are used as the input and output of the neural network respectively, the neural network is trained, and the trained neural network is used as the strategy network for reinforcement learning.

[0108] Example 3

[0109] See also Figure 5 , Figure 5 Schematic diagram of a computer device according to an embodiment of the present application. Figure 5 As shown in FIG, the computer device 500 includes a processor 510 , a memory 520 and a bus 530 .

[0110] The memory 520 stores machine-readable instructions executable by the processor 510. When the computer device 500 is running, the processor 510 communicates with the memory 520 via the bus 530. When the machine-readable instructions are executed by the processor 510, the above-mentioned Figure 1 、 Figure 3 The specific implementation of the steps of the energy optimization scheduling method in the method embodiment shown can be found in the method embodiment and will not be repeated here.

[0111] Example 4

[0112] Based on the same inventive concept, an embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the energy optimization scheduling method described in the above method embodiment are executed.

[0113] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0114] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A method for optimizing energy scheduling of an integrated energy system based on reinforcement learning, characterized in that: The energy optimization scheduling method includes: Collect operating data of each device in the integrated energy system; Based on the operating data, the energy conversion model of each operating device and the corresponding constraints, a heuristic algorithm is used to solve the planned output value of the generator set of the integrated energy system with the minimum system operating cost and carbon emissions as the optimization goal; Establish a Markov decision process in a limited space, determine the optimal value of the generator set output through a reinforcement learning model, and control the operation of the equipment according to the optimized value of the generator set output; wherein the operating data and the planned value of the generator set output are used as the input and output of the neural network respectively, the neural network is trained, and the trained neural network is used as the strategy network of the reinforcement learning model The heuristic algorithms include but are not limited to genetic algorithms, ant colony algorithms, particle swarm optimization algorithms and simulated annealing algorithms; The Markov decision process includes the state, action and reward function of the Markov decision process; define the state s t {P pv (t),P wt (t),λ t ,μ t , P e (t),P c (t)}, action a t {P pgu,e (t)}, reward r t {-(f oc (t)+f es (t))}; Among them, t is the equipment operation period, s t is the state of the integrated energy system during the equipment operation period t, P pv (t) is the power generation data of the photovoltaic power station when the equipment is running during the t period, P wt (t) is the wind power station power generation data when the equipment is running at time t, λ t is the electricity purchase price when the equipment is running at time t, μ t is the electricity price when the equipment is running at time t, P e (t) is the user's electric load data when the equipment is running during the t period, P c (t) is the user cooling load data when the equipment is running during the t period, a t is the action of the integrated energy system during the equipment operation period t, P pgu,e (t) is the generator output when the equipment is running at time t, r t is the reward of the integrated energy system during the equipment operation period t, f oc (t) is the system operation cost of the integrated energy system during the equipment operation period t, f es (t) is the system operation carbon emission of the integrated energy system during the equipment operation period t.

2. The energy optimization scheduling method according to claim 1, characterized in that: The collection of operating data of each device in the integrated energy system includes: collecting photovoltaic power station power generation data, wind power station power generation data, electricity purchase price, electricity sales price, user electricity load data and user cooling load data at a specific sampling frequency.

3. The energy optimization scheduling method according to claim 1, characterized in that: The training process of the reinforcement learning model includes: Establish a virtual environment for a comprehensive energy system, which outputs actions a through reinforcement learning agents t , feedback reward r t and the state s at the next moment t+1 ; Load the trained policy network and value network, and the action network is used to input the current state s t Output action a t , the value network is used to input state s t , a t , output action value q t ; Initialize the target network of the above policy network and value network; Initialize t=1 and get the current state s t ; If t is less than the preset period T, according to the current state s t Select action a through the policy network t , and add appropriate motion noise; According to the output action a t , get the next moment state s through virtual environment feedback t+1 and reward r t , update the target network until t is greater than the preset period T, and update the current state.

4. The energy optimization scheduling method according to claim 3, characterized in that: Constructing experience information at the end of each cycle during the training of reinforcement learning models t ,a t ,r t ,s t+1 }, and store it in the experience pool.

5. The energy optimization scheduling method according to claim 4, characterized in that: Determine whether the experience pool is full. If the experience pool is full, update the value network parameters using the gradient descent method and delay the update of the action network using the gradient ascent algorithm.

6. An energy optimization scheduling system for an integrated energy system based on reinforcement learning, adopting the energy optimization scheduling method according to any one of claims 1 to 5, characterized in that: The energy optimization scheduling system includes: The acquisition module is used to collect the operating data of each device in the integrated energy system; a solution module for solving the planned output value of the generator set of the integrated energy system using a heuristic algorithm based on the operating data, the energy conversion model of each operating device and the corresponding constraints, with the system operating cost and carbon emissions minimized as the optimization goal; A control module is used to establish a Markov decision process in a finite space, determine the optimal value of the generator set output through a reinforcement learning model, and control the operation of the equipment according to the optimized value of the generator set output; wherein the operating data and the planned value of the generator set output are used as the input and output of the neural network respectively, the neural network is trained, and the trained neural network is used as the strategy network for reinforcement learning.

7. A computer device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the energy optimization scheduling method according to any one of claims 1 to 5 are performed.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the energy optimization scheduling method according to any one of claims 1 to 5 are executed.

Images