Multi-heat-source heating operation optimization method based on hierarchical optimization and multi-stage decision

Abstract

The application discloses a kind of based on layered optimization and multi-stage decision-making multi-heat source heating optimization operation method, comprising: constructing upper layer heat source dynamic scheduling model, to system multi-heat source joint scheduling operation benefit maximum, carbon emission minimum as objective function, set system operation constraint condition, optimization object is multi-heat source joint operation scheduling scheme;Upper layer heat source dynamic scheduling model is converted into the Markov process of multi-stage decision-making, in each decision-making stage, the process of cyclically carrying out heating system environmental state observation, making action decision, system state update and feedback reward and punishment information, and the Markov process of multi-stage decision-making is solved by reinforcement learning algorithm and obtains multi-heat source joint operation scheduling scheme;Build lower layer carbon emission evaluation model, obtain carbon emission heat source real-time carbon emission by establishing carbon emission evaluation model, and judge whether carbon emission reaches carbon emission reduction target, and correct multi-heat source joint operation scheduling scheme.

Description

Technical Field

[0001] This invention belongs to the field of smart heating technology, specifically relating to a multi-heat source heating optimization operation method based on hierarchical optimization and multi-stage decision-making. Background Technology

[0002] Multi-source interconnected heating systems refer to heating networks with two or more heat sources. Initially, only the most economical or efficient basic heat source operates. As outdoor temperatures decrease, peak-shaving heat sources are activated, enabling multi-source interconnected operation and thus improving system heating efficiency. Multi-source heating systems not only improve the reliability and safety of heating systems but also facilitate the combined short-term and long-term development of heating systems. Therefore, multi-source interconnected operation has become a major direction for the development of heating systems.

[0003] However, when optimizing multi-source networked heating systems, evolutionary algorithms such as particle swarm optimization and genetic algorithms are commonly used. These algorithms rely on establishing accurate physical models for iterative optimization, resulting in long computation times and a tendency to get trapped in local optima. Moreover, during the scheduling process of multi-source heating systems, for a given scheduling period, only the uncertainty information of that scheduling period and the periods before it is known; the uncertainty information of all subsequent periods is unknown. How to achieve interaction with the system environment, learn the system scheduling rules, adapt to the uncertainty of the system environment, and formulate reasonable operational decision-making schemes are urgent problems that need to be solved.

[0004] Based on the above technical problems, it is necessary to design a new multi-heat source heating optimization operation method based on hierarchical optimization and multi-stage decision-making. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a multi-heat source heating optimization operation method based on hierarchical optimization and multi-stage decision-making.

[0006] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0007] This invention provides a method for optimizing the operation of multi-heat source heating based on hierarchical optimization and multi-stage decision-making, which includes:

[0008] Step S1: Establish a digital twin model of the multi-heat-source networked heating system using mechanism modeling and data identification methods;

[0009] Step S2: Based on the digital twin model of the multi-heat source network heating system, construct the upper-level heat source dynamic scheduling model. With the objective function of maximizing the efficiency of the multi-heat source joint scheduling operation and minimizing carbon emissions, set system operation constraints and optimize the multi-heat source joint operation scheduling scheme.

[0010] Step S3: Transform the upper-level heat source dynamic scheduling model into a multi-stage decision-making Markov process. In each decision-making stage, the process of observing the environmental status of the heating system, making action decisions, updating the system status, and providing feedback reward and punishment information is carried out cyclically. The multi-heat source joint operation scheduling scheme is obtained by solving the multi-stage decision-making Markov process through reinforcement learning algorithm.

[0011] Step S4: Construct a lower-level carbon emission assessment model. By establishing the carbon emission assessment model, obtain the real-time carbon emission of the carbon emission heat source, determine whether the carbon emission has reached the carbon emission reduction target, and revise the multi-heat source joint operation scheduling scheme.

[0012] Furthermore, in step S1, a digital twin model of the multi-heat-source networked heating system is established using mechanism modeling and data identification methods, specifically including:

[0013] A digital twin model of a multi-heat-source networked heating system is established, comprising a physical entity, a virtual twin, a digital twin, and a digital space. The physical entity includes multiple heat sources, a heating network, heating stations, heat users, and corresponding heating equipment. The digital twin includes object twins, process twins, and performance twins. The digital space includes an information acquisition module, a data storage module, an expert knowledge base, a machine learning algorithm module, an evaluation module, and a decision-making module. Through the coupling and interaction of these modules, the digital twin drives the data mapping, information interaction, simulation, and optimized operation between the virtual and physical spaces of the multi-heat-source networked heating system.

[0014] The real-time operating data of the multi-heat source networked heating system equipment under multiple operating conditions are integrated into the established digital twin model of the multi-heat source networked heating system. The simulation results of the digital twin model of the multi-heat source networked heating system are adaptively identified and corrected using the reverse identification method to obtain the identified and corrected digital twin model of the multi-heat source networked heating system.

[0015] Furthermore, the multi-heat-source networked heating system operates with the basic heat source bearing the heat load during the initial and final stages of the heating season, and peak-shaving heat sources being put into operation during the severe cold period. It achieves joint heating through the coupling of hydraulic and thermal operating conditions with the basic heat source. The basic heat source includes at least coal-fired power plants, gas-fired power plants, coal-fired boiler rooms, gas-fired boiler rooms, and industrial waste heat and heat generated from waste incineration. The peak-shaving heat source includes at least gas-fired peak-shaving boiler rooms and coal-fired peak-shaving boiler rooms.

[0016] Furthermore, in step S2, the system operation constraints are set with the objective function of maximizing the benefits of the joint scheduling operation of multiple heat sources and minimizing carbon emissions. These constraints include:

[0017] The objective function, taking the maximization of the benefits of multi-heat-source joint scheduling and operation while minimizing carbon emissions, is expressed as:

[0018]

[0019] Where K is the number of heat sources participating in the joint scheduling of multiple heat sources; T is the scheduling period; C k P is the power generation capacity of heat source k; ek The grid connection price of heat source k; D k P represents the heating load of heat source k; hk For the heating revenue of heat source k; f k E represents the fuel consumption of heat source k. k The price of fuel used by heat source k; C RM,k C represents the carbon emissions from a coal-fired heat source k. RQ,k The carbon emissions of the gas-fired heat source k; C WG,k The total carbon emissions generated by purchased electricity and purchased heat from heat source k;

[0020] Set system operation constraints, including at least: heat load balance constraints, heat load variation range constraints, unit load increase / decrease rate constraints, and heating network distribution capacity constraints.

[0021] Furthermore, in step S3, the upper-level heat source dynamic scheduling model is transformed into a multi-stage decision-making Markov process. In each decision-making stage, the process of observing the environmental state of the heating system, making action decisions, updating the system state, and providing feedback reward and punishment information is repeated. The multi-heat source joint operation scheduling scheme is obtained by solving the multi-stage decision-making Markov process using a reinforcement learning algorithm, specifically including:

[0022] The dynamic scheduling model of the upper heat source within the scheduling cycle is divided into multiple decision-making cycles according to the decision-making steps of the Markov decision process.

[0023] In each decision-making stage, the multi-source heating system dispatch decision center observes the system environment status, arranges the planned output of heat sources, and applies the dispatch plan to the system environment. The system will then transition to the new state and provide feedback on reward and punishment information to the dispatch decision center to assist in optimizing the dispatch plan. This interactive process is repeated until the end of the entire dispatch cycle.

[0024] The variables in the decision-making process include:

[0025] State: The state s of the t-th decision step t It consists of the predicted load value, the predicted heat source output value, and the measured heat source output value at time t+1;

[0026] Action: The action a of the t-th decision step t Planned output values ​​for each heat source unit;

[0027] Reward function: The constraints in the upper-level heat source dynamic scheduling model are set in the reward function in the form of penalties, and the objective function is set in the reward function in the form of rewards. When the constraints are violated, the corresponding penalties are applied, and the objective function is achieved by maximizing the rewards.

[0028] The model of the Markov process with multi-stage decision-making is solved using a reinforcement learning algorithm to obtain a multi-heat source joint operation scheduling scheme.

[0029] Furthermore, the step of solving the multi-stage decision-making Markov process model using a reinforcement learning algorithm to obtain a multi-heat source joint operation scheduling scheme includes: training the agent of the multi-stage decision-making Markov process using a deep deterministic policy gradient algorithm to obtain a deep reinforcement learning agent; acquiring the state of the multi-heat source heating system, and enabling the deep reinforcement learning agent to obtain execution actions based on the state, and optimizing the scheduling of the multi-heat source heating system based on the execution actions.

[0030] Furthermore, in step S4, a lower-level carbon emission assessment model is constructed. This model is used to obtain real-time carbon emissions from carbon-emitting heat sources and determine whether the carbon emission reduction targets have been met. The multi-heat-source joint operation and scheduling scheme is then revised. Specifically, this includes:

[0031] Establish a carbon emission assessment model:

[0032]

[0033] Among them, C min (t) represents the minimum carbon emissions from the carbon-emitting heat source at time t when the system is operating stably; C RM (t), C RQ (t) represents the carbon emissions from coal-fired and gas-fired carbon emission heat sources at time t, respectively; C shimu (t) represents the real-time carbon emission target value of the carbon emission heat source at time t; C shi (t) represents the system's real-time carbon emissions at time t; C WG (t) represents the total carbon emissions generated by purchased electricity and heat at time t;

[0034] By calculating the real-time carbon emissions data of the multi-heat source joint operation and scheduling scheme and the carbon emissions of the scheme without using the scheduling scheme, it is determined whether the total carbon emissions up to the current moment have reached the daily carbon emission reduction target. If the emission reduction target has not been reached, the multi-heat source joint operation and scheduling scheme is modified; if the daily emission reduction target has been reached by a certain point in time, the efficiency of multi-heat source joint operation and scheduling is improved by adjusting the emission reduction constraints.

[0035] The model for determining whether the total carbon emissions up to the current moment have reached the daily carbon emission reduction target is expressed as follows: T sd C represents the total real-time scheduling time up to time t; rq (t) represents the carbon emissions at time t without the multi-heat source joint operation and scheduling scheme; C jpmb The target for reducing carbon emissions on that day.

[0036] Furthermore, the multi-heat source joint operation scheduling scheme is modified, and the output model of each heat source after modification is expressed as: P xz,k (t+1)=P wxz,k (t+1)+λΔP wxz,k (t); P xz,i (t+1) represents the output power of heat source k after correction at time t+1; P wxz,k (t+1) represents the uncorrected output of heat source k at time t+1; λ represents the state of the correction variable; ΔP wxz,k (t) represents the correction amount of the output of heat source k at the next moment.

[0037] Furthermore, the improvement in the efficiency of joint scheduling and operation of multiple heat sources by adjusting emission reduction constraints is expressed as: ωC min (t)≤C RM (t)+C RQ (t)≤ωC shimu (t); ω is the carbon emission adjustment coefficient.

[0038] Furthermore, the multi-heat source heating optimization operation method further includes, after step S4: verifying the optimal scheduling scheme for the joint operation of multiple heat sources based on the digital twin model of the multi-heat source network heating system; if the verification meets expectations, the scheme is issued and executed; otherwise, the scheme is optimized and adjusted.

[0039] The beneficial effects of this invention are:

[0040] This invention constructs an upper-level dynamic scheduling model for heat sources. Based on Markov decision process theory, it transforms dynamic scheduling into a multi-stage decision problem and employs a deep deterministic strategy gradient algorithm to solve the model. This results in a short online decision-making time, assisting dispatchers in making reasonable multi-heat source joint operation scheduling schemes. Furthermore, it constructs a lower-level carbon emission assessment model to obtain real-time carbon emissions from heat sources and determine whether carbon emissions meet emission reduction targets. This allows for the correction of multi-heat source joint operation scheduling schemes, considering changes in carbon emissions in conjunction with emission reduction targets to achieve low-carbon emission reduction goals. Finally, based on a digital twin model of a multi-heat source networked heating system, it constructs a two-layer model and makes scheduling decisions through simulation, online analysis, and scheduling decisions, improving the accuracy of scheduling schemes.

[0041] Other features and advantages will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained through the structures particularly pointed out in the description and the drawings.

[0042] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0043] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0044] Figure 1 This is a flowchart of a multi-heat source heating optimization operation method based on hierarchical optimization and multi-stage decision-making according to the present invention;

[0045] Figure 2 This is a schematic diagram of the multi-heat source networked heating system of the present invention;

[0046] Figure 3 This is a flowchart illustrating the Markov process implementation of the multi-stage decision-making method of this invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] like Figure 1-3 As shown, this embodiment provides a multi-heat source heating optimization operation method based on hierarchical optimization and multi-stage decision-making, which includes:

[0049] Step S1: Establish a digital twin model of the multi-heat-source networked heating system using mechanism modeling and data identification methods;

[0050] Step S2: Based on the digital twin model of the multi-heat source network heating system, construct the upper-level heat source dynamic scheduling model. With the objective function of maximizing the efficiency of the multi-heat source joint scheduling operation and minimizing carbon emissions, set system operation constraints and optimize the multi-heat source joint operation scheduling scheme.

[0051] Step S3: Transform the upper-level heat source dynamic scheduling model into a multi-stage decision-making Markov process. In each decision-making stage, the process of observing the environmental status of the heating system, making action decisions, updating the system status, and providing feedback reward and punishment information is carried out cyclically. The multi-heat source joint operation scheduling scheme is obtained by solving the multi-stage decision-making Markov process through reinforcement learning algorithm.

[0052] Step S4: Construct a lower-level carbon emission assessment model. By establishing the carbon emission assessment model, obtain the real-time carbon emission of the carbon emission heat source, determine whether the carbon emission has reached the carbon emission reduction target, and revise the multi-heat source joint operation scheduling scheme.

[0053] In this embodiment, step S1 involves establishing a digital twin model of a multi-heat-source networked heating system using mechanistic modeling and data identification methods, specifically including:

[0054] A digital twin model of a multi-heat-source networked heating system is established, comprising physical entities, virtual twins, digital twins, and a digital space. The physical entities include multiple heat sources, heating networks, heating stations, heat users, and corresponding heating equipment. The digital twins include object twins, process twins, and performance twins. The digital space includes information acquisition modules, data storage modules, an expert knowledge base, machine learning algorithm modules, evaluation modules, and decision-making modules. Through the coupling and interaction of these modules, the digital twin drives data mapping, information interaction, simulation, and optimized operation between the virtual and physical spaces of the multi-heat-source networked heating system.

[0055] The real-time operating data of the multi-heat source networked heating system equipment under multiple operating conditions are integrated into the established digital twin model of the multi-heat source networked heating system. The simulation results of the digital twin model of the multi-heat source networked heating system are adaptively identified and corrected using the reverse identification method to obtain the identified and corrected digital twin model of the multi-heat source networked heating system.

[0056] In this embodiment, the multi-heat-source networked heating system operates with the basic heat source bearing the heat load at the beginning and end of the heating season, and the peak-shaving heat source being put into operation during the severe cold period. The system provides joint heating through the coupling of hydraulic and thermal operating conditions with the basic heat source. The basic heat source includes at least coal-fired power plants, gas-fired power plants, coal-fired boiler rooms, gas-fired boiler rooms, and industrial waste heat and heat generated by waste incineration. The peak-shaving heat source includes at least gas-fired peak-shaving boiler rooms and coal-fired peak-shaving boiler rooms.

[0057] In this embodiment, in step S2, the system operation constraints are set with the objective function of maximizing the benefits of the joint scheduling operation of multiple heat sources and minimizing carbon emissions. These constraints include:

[0058] The objective function, taking the maximization of the benefits of multi-heat-source joint scheduling and operation while minimizing carbon emissions, is expressed as:

[0059]

[0060] Where K is the number of heat sources participating in the joint scheduling of multiple heat sources; T is the scheduling period; C k P is the power generation capacity of heat source k; ek The grid connection price of heat source k; D k P represents the heating load of heat source k; hk For the heating revenue of heat source k; f k E represents the fuel consumption of heat source k. k The price of fuel used by heat source k; C RM,k C represents the carbon emissions from a coal-fired heat source k. RQ,k The carbon emissions of the gas-fired heat source k; C WG,k The total carbon emissions generated by purchased electricity and purchased heat from heat source k;

[0061] Set system operation constraints, including at least: heat load balance constraints, heat load variation range constraints, unit load increase / decrease rate constraints, and heating network distribution capacity constraints.

[0062] In this embodiment, step S3 transforms the upper-level heat source dynamic scheduling model into a multi-stage decision-making Markov process. In each decision-making stage, the process of observing the heating system's environmental state, making action decisions, updating the system state, and providing feedback on rewards and penalties is repeated. A reinforcement learning algorithm is used to solve the multi-stage decision-making Markov process to obtain a multi-heat source joint operation scheduling scheme, specifically including:

[0063] The dynamic scheduling model of the upper heat source within the scheduling cycle is divided into multiple decision-making cycles according to the decision-making steps of the Markov decision process.

[0064] In each decision-making stage, the multi-source heating system dispatch decision center observes the system environment status, arranges the planned output of heat sources, and applies the dispatch plan to the system environment. The system will then transition to the new state and provide feedback on reward and punishment information to the dispatch decision center to assist in optimizing the dispatch plan. This interactive process is repeated until the end of the entire dispatch cycle.

[0065] The variables in the decision-making process include:

[0066] State: The state s of the t-th decision step t It consists of the predicted load value, the predicted heat source output value, and the measured heat source output value at time t+1;

[0067] Action: The action a of the t-th decision step t Planned output values ​​for each heat source unit;

[0068] Reward function: The constraints in the upper-level heat source dynamic scheduling model are set in the reward function in the form of penalties, and the objective function is set in the reward function in the form of rewards. When the constraints are violated, the corresponding penalties are applied, and the objective function is achieved by maximizing the rewards.

[0069] The model of the Markov process with multi-stage decision-making is solved using a reinforcement learning algorithm to obtain a multi-heat source joint operation scheduling scheme.

[0070] In this embodiment, a multi-stage decision-making Markov process model is solved using a reinforcement learning algorithm to obtain a multi-heat source joint operation scheduling scheme. This includes: training the agent of the multi-stage decision-making Markov process using a deep deterministic policy gradient algorithm to obtain a deep reinforcement learning agent; acquiring the state of the multi-heat source heating system and enabling the deep reinforcement learning agent to obtain execution actions based on the state, and optimizing the scheduling of the multi-heat source heating system based on the execution actions.

[0071] In this embodiment, step S4 involves constructing a lower-level carbon emission assessment model. This model is used to obtain real-time carbon emissions from carbon-emitting heat sources and determine whether the carbon emissions meet the carbon emission reduction target. The resulting adjustment of the multi-heat-source joint operation scheduling scheme includes:

[0072] Establish a carbon emission assessment model:

[0073]

[0074] Among them, C min (t) represents the minimum carbon emissions from the carbon-emitting heat source at time t when the system is operating stably; C RM (t), C RQ (t) represents the carbon emissions from coal-fired and gas-fired carbon emission heat sources at time t, respectively; C shimu (t) represents the real-time carbon emission target value of the carbon emission heat source at time t; C shi (t) represents the system's real-time carbon emissions at time t; C WG (t) represents the total carbon emissions generated by purchased electricity and heat at time t;

[0075] By calculating the real-time carbon emissions data of the multi-heat source joint operation and scheduling scheme and the carbon emissions of the scheme without using the scheduling scheme, it is determined whether the total carbon emissions up to the current moment have reached the daily carbon emission reduction target. If the emission reduction target has not been reached, the multi-heat source joint operation and scheduling scheme is modified; if the daily emission reduction target has been reached by a certain point in time, the efficiency of multi-heat source joint operation and scheduling is improved by adjusting the emission reduction constraints.

[0076] The model for determining whether the total carbon emissions up to that point have reached the daily carbon emission reduction target is expressed as follows: T sd C represents the total real-time scheduling time up to time t; rq (t) represents the carbon emissions at time t without the multi-heat source joint operation and scheduling scheme; C jpmb The target for reducing carbon emissions on that day.

[0077] In this embodiment, the multi-heat source joint operation scheduling scheme is modified, and the output model of each heat source after modification is expressed as: P xz,k (t+1)=P wxz,k (t+1)+λΔP wxz,k (t); P xz,i (t+1) represents the output power of heat source k after correction at time t+1; P wxz,k (t+1) represents the uncorrected output of heat source k at time t+1; λ represents the state of the correction variable; ΔP wxz,k (t) represents the correction amount of the output of heat source k at the next moment.

[0078] In this embodiment, adjusting emission reduction constraints to improve the efficiency of joint scheduling of multiple heat sources is expressed as: ωC min (t)≤C RM (t)+C RQ (t)≤ωC shimu (t); ω is the carbon emission adjustment coefficient.

[0079] It should be noted that a carbon emission assessment model was established using carbon emissions as an indicator. This model primarily assesses whether carbon emission reduction targets have been met and allows for timely adjustments to the scheduling scheme, enabling the system to pursue both minimal carbon emissions and economic efficiency. The adjustment plan aims for minimal adjustments by regulating the output of various heat sources to balance the system's supply and demand, thereby improving system safety and stability.

[0080] In this embodiment, the multi-heat source heating optimization operation method further includes, after step S4: verifying the optimal scheduling scheme for the joint operation of multiple heat sources based on the digital twin model of the multi-heat source network heating system; if the verification meets expectations, the scheme is issued and executed; otherwise, the scheme is optimized and adjusted.

[0081] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can also be implemented in other ways. The system embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0082] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0083] If the functionality is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0084] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A multi-heat source heating operation optimization method based on hierarchical optimization and multi-stage decision, characterized in that, It comprises: Step S1, a mechanism modeling and data identification method is used to establish a digital twin model of a multi-heat source networked heating system; Step S2, based on the digital twin model of the multi-heat source networked heating system, a top-level heat source dynamic scheduling model is constructed, taking the maximum operation benefit and the minimum carbon emission of the system as the objective function, setting the system operation constraint condition, and optimizing the object as the multi-heat source joint operation scheduling scheme; Step S3, the top-level heat source dynamic scheduling model is converted into a multi-stage decision Markov process, in each decision stage, the process of observing the environment state of the heating system, making action decisions, updating the system state and feeding back reward and punishment information is cycled, and the multi-stage decision Markov process is solved by a reinforcement learning algorithm to obtain the multi-heat source joint operation scheduling scheme; Step S4, a lower-level carbon emission evaluation model is constructed, the real-time carbon emission of the carbon emission heat source is obtained by establishing the carbon emission evaluation model, and it is judged whether the carbon emission reaches the carbon emission reduction target, and the multi-heat source joint operation scheduling scheme is corrected; In step S3, the top-level heat source dynamic scheduling model is converted into a multi-stage decision Markov process, in each decision stage, the process of observing the environment state of the heating system, making action decisions, updating the system state and feeding back reward and punishment information is cycled, and the multi-stage decision Markov process is solved by a reinforcement learning algorithm to obtain the multi-heat source joint operation scheduling scheme, specifically including: The top-level heat source dynamic scheduling model in the scheduling period is divided into a Markov decision process of multiple stage decision periods according to the decision steps; In each decision stage, the multi-heat source heating system scheduling decision center observes the system environment state, arranges the heat source plan output, and applies the scheduling scheme to the system environment, the system will transfer to a new state, and feedback the reward and punishment information to the scheduling decision center to assist the scheduling decision center to optimize the scheduling scheme, and the interaction process is cycled until the end of the whole scheduling period; Wherein the variables of the decision process include: State: state s of the tth decision step t composed of the load prediction value, the heat source output prediction value, and the heat source output measured value at the t+1th time Action: Action a of the tth decision step t Plan output action value for each heat source unit; Reward function: set each constraint condition in the top-level heat source dynamic scheduling model in the form of punishment in the reward function, set the objective function in the form of reward in the reward function, respond to the punishment when the constraint is violated, and realize the objective function by maximizing the reward; The model of the multi-stage decision Markov process is solved by a reinforcement learning algorithm to obtain the multi-heat source joint operation scheduling scheme; Said solving the model of the multi-stage decision Markov process by a reinforcement learning algorithm to obtain the multi-heat source joint operation scheduling scheme, including: A deep deterministic policy gradient algorithm is used to train the agent of the multi-stage decision Markov process to obtain a deep reinforcement learning agent; The state of the multi-heat source heating system is obtained, and the deep reinforcement learning agent is caused to execute an action according to the state, and the multi-heat source heating system is optimized and scheduled according to the execution action; In step S4, the lower-level carbon emission evaluation model is constructed, the real-time carbon emission of the carbon emission heat source is obtained by establishing the carbon emission evaluation model, and it is judged whether the carbon emission reaches the carbon emission reduction target, and the multi-heat source joint operation scheduling scheme is corrected, specifically including: A carbon emission evaluation model is established: Wherein, C min (t) is the minimum carbon emission of the carbon emission heat source when the system is stably running at time t; C RM (t), C RQ (t) are the carbon emissions of the coal-fired and gas-fired carbon emission heat sources at time t, respectively; C shimu (t) is the real-time carbon emission target value of the carbon emission heat source at time t; C shi (t) is the real-time carbon emission of the system at time t; C WG (t) is the total carbon emission generated by the purchased electricity and purchased heat at time t; By calculating the real-time carbon emission data of the multi-heat source joint operation scheduling scheme and the carbon emission without the scheduling scheme, it is determined whether the total carbon emission has reached the carbon emission reduction target of the day by the current time. If the reduction target has not been reached, the multi-heat source joint operation scheduling scheme is modified; if the reduction target has been reached by a certain time, the multi-heat source joint scheduling operation benefit is improved by adjusting the emission reduction constraint; The model for judging whether the total carbon emission amount has reached the carbon emission reduction target of the day as of the current time is represented as: T sd is the total real-time scheduling scheme time length as of t; C rq (t) is the carbon emission amount without the multi-heat-source joint operation scheduling scheme as of t; C jpmb is the carbon emission reduction target of the day. The modified output model of each heat source in the multi-heat source joint operation scheduling scheme is represented as: P xz,k (t+1) = P wxz,k (t+1) + λΔP wxz,k (t); P (t) = P (t) + ΔP (t) (1) xz,i P (t+1) is the output of the heat source k at time t+1 after correction; P wxz,k P (t+1) is the output of the heat source k at time t+1 without correction; λ is the correction variable state; ΔP wxz,k P (t) is the correction amount of the output of the heat source k at the next time.

2. The method of claim 1, wherein, In step S1, a digital twin model of the multi-heat source networking heating system is established by using mechanism modeling and data identification methods, specifically including: A digital twin model of the multi-heat source networking heating system is established, including the physical entity, virtual twin, digital twin, and digital space of the multi-heat source networking heating system; The physical entity includes multiple heat sources, heat networks, heat stations, heat users, and corresponding heating equipment; The digital twin includes object twin, process twin, and performance twin; The digital space includes an information acquisition module, a data storage module, an expert knowledge base, a machine learning algorithm module, an evaluation module, and a decision-making module. Through the coupling and interaction of each module, the digital twin realizes data mapping, information interaction, simulation, and optimization of the virtual space and physical space of the multi-heat source networking heating system. The multi-heat source networking heating system equipment multi-condition real-time operation data is connected to the established multi-heat source networking heating system digital twin model, and the simulation results of the multi-heat source networking heating system digital twin model are adaptively identified and corrected using a reverse identification method to obtain the identified and corrected multi-heat source networking heating system digital twin model.

3. The method of claim 1, wherein, The multi-heat source networking heating system operates in the initial and final stages of heating, with the basic heat source bearing the heat load, and the peak shaving heat source being put into operation in the severe cold period. The system provides joint heating through the coupling of hydraulic and thermal conditions and the basic heat source. The basic heat source includes at least coal-fired power plants, gas-fired power plants, coal-fired boiler houses, gas-fired boiler houses, and industrial waste heat and waste incineration heat. The peak shaving heat source includes at least gas-fired peak shaving boiler houses and coal-fired peak shaving boiler houses.

4. The method of claim 1, wherein, In step S2, the system multi-heat source joint scheduling operation benefit maximization and carbon emission minimization are set as the objective function, and the system operation constraint conditions are set, including: The system multi-heat source joint scheduling operation benefit maximization and carbon emission minimization are set as the objective function, represented as: Wherein, maxF1 is the maximum benefit, minF2 is the minimum carbon emission, K is the number of heat sources participating in the joint scheduling, T is the scheduling period, C k is the power generation of heat source k; P ek is the on-grid electricity price of heat source k; D k is the heating load of heat source k; P hk is the heating income of heat source k; f k is the fuel consumption of heat source k; E k is the fuel price used by heat source k; C RM,k is the carbon emission of coal-fired heat source k; C RQ,k is the carbon emission of gas-fired heat source k; C WG,k is the total carbon emission generated by the purchased electricity and purchased heat of heat source k; The system operation constraint conditions are set, including at least: heat load balance constraint, heat load variation range constraint, unit load rate constraint, and heat pipe network transmission and distribution capacity constraint.

5. The multi-heat source heating optimization operation method of claim 1, wherein The adjustment of the emission reduction constraint improves the multi-heat source joint scheduling operation benefit, represented as: ωC min (t)≤C RM (t)+C RQ (t)≤ωC shimu (t) Where ω is the carbon emission adjustment coefficient.

6. The method of claim 1, wherein, In the multi-heat source heating optimization operation method, after step S4, the multi-heat source joint operation optimal scheduling scheme is verified based on the multi-heat source networking heating system digital twin model. If the verification meets the expectation, the scheme is issued and executed; otherwise, the scheme is optimized and adjusted.

Images