On-demand scheduling control method for heat supply system integrating pipe network heat storage and release and energy storage device

Abstract

The invention discloses a heat supply system on-demand scheduling control method integrating pipe network heat storage and release and an energy storage device, which comprises the following steps of: predicting heat supply of a passive heat source and a heat load demand of a total heating station, and analyzing a typical operation condition type to which the heat supply system belongs by combining an output characteristic of an active heat source; aiming at each typical operation condition type, establishing a heat storage and release mechanism of a pipe network for absorbing a passive heat source and a heat storage device; taking the active heat source and the passive heat source at the source side as leaders, taking the pipe network and the heat storage device as followers, establishing a master-slave game model, and carrying out supply and demand collaborative optimization scheduling between the source side and the heating station; and taking the thermal station dispatching center as an intelligent agent, continuously receiving the state variable of each thermal substation at each moment, selecting a corresponding pump valve regulation control action of each thermal substation, calculating an action reward value after the action is executed, guiding the intelligent agent to update an action strategy, and obtaining an optimal pump valve regulation control strategy of each thermal substation.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent heating dispatch control, and particularly relates to a demand-based dispatch control method for a heating system integrating heat storage and release of a pipe network and an energy storage device. Background Art

[0002] In the development process of the intelligent heating industry, in order to cope with global climate change and energy crisis, the renewable energy field represented by wind and light in China has developed rapidly. Compared with traditional energy, energy such as wind power, photovoltaic power, and industrial waste heat depends on the natural environment and the output of the process production, and its volatility and uncertainty increase the regulation burden on the heating system and pose challenges to the heat supply and demand balance of the heating system.

[0003] Currently, heat storage devices are usually used to solve the volatility and uncertainty of the heating system, as well as the supply-demand imbalance problem between the source side and the user load side. However, with the access of resources such as wind power, photovoltaic power, industrial waste heat, and heat storage to the heating system, the operating conditions of the heating system become more complex. How to carry out collaborative optimization scheduling of system resources for different operating conditions, while ensuring demand-based regulation control of the heat load, improving the overall operating efficiency of the system and the heating quality, and at the same time reducing manual intervention, improving the automation level and intelligence degree of the system, is an urgent problem to be solved at present.

[0004] Based on the above technical problems, it is necessary to design a new demand-based dispatch control method for a heating system integrating heat storage and release of a pipe network and an energy storage device. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to overcome the deficiencies of the prior art and provide a demand-based dispatch control method for a heating system integrating heat storage and release of a pipe network and an energy storage device. By judging the type of operating conditions to which the heating system belongs, establishing a heat storage and release mechanism, and establishing a master-slave game model, heat supply strategies for active heat sources and passive heat sources on the source side under different typical operating conditions, as well as heat storage and release strategies for the pipe network itself and the heat storage device, are obtained, so as to realize collaborative optimization scheduling and resource allocation of supply and demand and improve the overall operating efficiency of the system; in addition, the optimal pump valve regulation control strategies for each heat sub-station are obtained through intelligent agents and reinforcement learning technology, so as to realize refined control, meet the heat load requirements of each heat sub-station, improve the heating quality and energy-saving effect, and at the same time reduce manual intervention, improve the automation level and intelligence degree of the system.

[0006] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0007] The present invention provides a demand-based dispatch control method for a heating system integrating heat storage and release of a pipe network and an energy storage device, including:

[0008] S1. Obtain the heat supply data sequences of the active heat sources and passive heat sources on the source side and the heat load demand data sequence on the total heat substation side during the historical time period, and conduct a clustering analysis on the typical operation conditions of the supply-demand between the heat sources and the total heat substation;

[0009] S2. Predict the heat supply of the passive heat sources and the heat load demand of the total heat substation, and combine with the output characteristics of the active heat sources to analyze the typical operation condition types to which the heating system belongs;

[0010] S3. For each type of typical operation condition, establish the heat storage and release mechanisms of the pipe network itself and the heat storage device for absorbing and consuming the passive heat sources;

[0011] S4. Considering the heat storage and release mechanisms of the pipe network itself and the heat storage device under different typical operation conditions, taking the active heat sources and passive heat sources on the source side as the leaders and the pipe network itself and the heat storage device as the followers, establish a master-slave game model. The leaders determine their heat supply strategies, and after observing the decisions of the leaders, the followers select the optimal heat storage and release strategies to conduct the collaborative optimal scheduling of the supply-demand between the source side and the heat substation;

[0012] S5. Take the heat substation dispatching center as an intelligent agent, continuously receive the state variables of each heat substation at each moment, select the corresponding pump-valve adjustment control actions of each heat substation, calculate the action reward value after executing the actions, guide the intelligent agent to update the action strategy, and obtain the optimal pump-valve adjustment control strategy of each heat substation.

[0013] Furthermore, the active heat sources on the source side include coal-fired heating units, cogeneration units, and hot water boilers; the passive heat sources include photovoltaic heating units, wind power heating units, and industrial waste heat heating units.

[0014] Furthermore, the S1 includes:

[0015] Obtain the heat supply data sequences of the active heat sources and passive heat sources on the source side during the historical time period, and the heat load demand data sequence on the total heat substation side; the heat load demand data sequence on the total heat substation side is obtained by accumulating the user heat load demand data of each subordinate heat substation;

[0016] Take the heat supply data sequence of the heat sources on the source side and the heat load demand data sequence on the total heat substation side during the historical time period as the input of the clustering algorithm, and set the number of clusters and the maximum number of iterations;

[0017] Extract the initial cluster centers from the data sequences, calculate the Euclidean distances between each data in the data sequences and the initial cluster centers, and classify the data and update the initial cluster centers;

[0018] Repeat the iteration until the convergence condition is met, that is, when the change error of the clustering center is less than the preset value, complete all clustering, and the K clustering centers after the final stop of the iteration are the obtained K typical operating conditions.

[0019] Further, S2 includes:

[0020] According to the type of passive heat source, obtain the data characteristics affecting the output of the passive heat source in the historical time period, use machine learning algorithms for training and learning, establish a heat supply prediction model for the passive heat source, and obtain the predicted heat supply values of the passive heat source at different times;

[0021] Analyze the data characteristics affecting the heat load demand of users in each heat sub-station, use machine learning algorithms to establish a heat load demand prediction model for users in the heat sub-station, and obtain the predicted heat load demand values of each heat sub-station at different times;

[0022] According to the predicted heat load demand values of each heat sub-station at different times, perform cumulative calculation to obtain the heat load demand values of the main heat station at different times;

[0023] Determine the maximum output, minimum output and regulation speed of the active heat source, and analyze the predicted output range of the active heat source at different times according to the equipment characteristics, operating parameters and fuel supply situation of the active heat source;

[0024] According to the predicted heat supply values of the passive heat source at different times, the heat load demand values of the main heat station at different times and the predicted output range of the active heat source at different times, analyze the typical operating condition types to which the heating system belongs based on the supply-demand balance degree, the dominant type of heat source and the heat source combination ratio.

[0025] Further, S3 includes:

[0026] For each type of typical operating condition, preferentially absorb the passive heat source, and then determine the proportion of heat supply of the active heat source at different times according to the supply-demand balance degree. When the supply is greater than the demand, store the excess heat in the pipe network itself first, and then in the heat storage device; when the supply is less than the demand, preferentially release the heat stored in the pipe network itself, then release the heat in the heat storage device, and finally, the active heat source makes up for the missing heat.

[0027] Further, in S4, establishing the master-slave game model includes:

[0028] Determine the participants in the master-slave game process, take the active heat source and passive heat source on the source side as the leaders, the pipe network itself and the heat storage device as the followers, and set the benefit functions of the leaders and followers as U L and U F 、game strategies as s L and sF , construct a leader-follower game model;

[0029] Among them, the benefit function of the leader is to maximize the revenue on the source side, expressed as:

[0030]

[0031] K is the category of typical operating conditions; T is the total number of time steps in the scheduling period; λ a,k,t , λ p,k,t are the heat selling prices of the active heat sources and passive heat sources on the source side at time t under the k-th type of typical operating conditions respectively; H a,k,t , H p,k,t are the supplied heat quantities of the active heat sources and passive heat sources on the source side at time t under the k-th type of typical operating conditions respectively; C a,k,t , C p,k,t are the operating costs of the active heat sources and passive heat sources on the source side at time t under the k-th type of typical operating conditions respectively; is the penalty cost of the passive heat source at time t under the k-th type of typical operating conditions;

[0032] The game strategy s of the leader L is the heat supply value of the active heat sources and passive heat sources on the source side at time t under the k-th type of typical operating conditions;

[0033] The benefit function of the follower is to minimize the charging and discharging operation cost, expressed as:

[0034]

[0035] b r,cha,k,t , b r,dis,k,t are the charging and discharging cost coefficients of the pipe network itself at time t under the k-th type of typical operating conditions respectively; P r,cha,k,t , P r,dis,k,t are the charging and discharging powers of the pipe network itself at time t under the k-th type of typical operating conditions respectively; X r,cha,k,t , X r,dis,k,t are the charging and discharging states of the pipe network itself at time t under the k-th type of typical operating conditions respectively, taking values of 0, 1; b c,cha,k,t , b c,dis,k,t are the charging and discharging cost coefficients of the heat storage device at time t under the k-th type of typical operating conditions respectively; P c,cha,k,t , P c,dis,k,t are the charging and discharging powers of the heat storage device at time t under the k-th type of typical operating conditions respectively; X c,cha,k,t , X c,dis,k,t are the charging and discharging states of the heat storage device at time t under the k-th type of typical operating conditions respectively, taking values of 0, 1;

[0036] The game strategy s of the followerF is the heat storage and release state and power at time t of the pipe network itself and the heat storage device under the k-th typical operating condition.

[0037] Under the given strategies s of the active heat sources and passive heat sources on the leader's source side L the follower obtains the optimal strategy s by solving its benefit function U F * F Then, the leader solves its benefit function U * F according to the follower's optimal strategy s L to obtain the optimal strategy s * L .

[0038] Furthermore, the establishment of the master-slave game model also includes setting operating constraint conditions, including heat load balance constraint, heat storage and release operation constraint, heat storage and release capacity constraint, and operating constraints of the active heat sources and passive heat sources on the source side.

[0039] Furthermore, the S5 includes:

[0040] Converting the pump-valve regulation and control problems of each heat sub-station into a Markov decision model;

[0041] Regarding the heat sub-station dispatching center as an agent, continuously receiving the state variables of each heat sub-station at each moment, selecting the pump-valve regulation and control action strategies of each heat sub-station, and applying the pump-valve regulation and control action strategies to the operating environment of each heat sub-station in the heating system. The system will transfer to a new state, and at the same time, feedback the action reward value to the heat sub-station dispatching center to guide the agent to update the action strategy and obtain the optimal pump-valve regulation and control strategy of each heat sub-station;

[0042] wherein, the state variables include the heat load demand of each heat sub-station, the total heat supply of the total heat sub-station, the output of the heat source on the source side, the heat storage and release state and power of the pipe network itself and the heat storage device;

[0043] The action strategies include the operating frequency of the water pumps of each heat sub-station and the regulating opening of the electric control valves;

[0044] The action reward value includes the maximum satisfaction level of the on-demand distribution of the heat load of each heat sub-station.

[0045] Furthermore, the Markov decision model is trained through a reinforcement learning algorithm to obtain the reinforcement learning agent of the heat sub-station dispatching center.

[0046] Furthermore, during the training of the agent, the safe operation constraints of each heat sub-station are set in the form of penalties. When the constraints are violated, corresponding penalties are imposed. If the safe operation constraints are met, the penalty value is 0. ​

[0047] The beneficial effects of the present invention are as follows:

[0048] (1) By obtaining the data of the active heat sources, passive heat sources and the heat load on the total heat supply station side in the historical time period, the present invention conducts a clustering analysis on the typical operation conditions of the supply-demand between the heat sources and the total heat supply station; it can comprehensively master the past operation conditions of the heating system, conduct a clustering analysis on the typical operation conditions of the supply-demand, divide the complex and diverse historical operation data into multiple types of typical conditions, and provide a basis for the dispatching control of different conditions in the future;

[0049] (2) By predicting the heat supply of the passive heat sources and the heat load demand of the total heat supply station, and combining with the output characteristics of the active heat sources, the present invention analyzes the typical operation condition types to which the heating system belongs; it can understand in advance the operation conditions faced by the heating system in the future, thereby specifically judging the condition types to which the heating system belongs, providing a forward-looking guidance for the optimized dispatching, and improving the adaptability and flexibility of the heating system;

[0050] (3) For each type of typical operation condition, the present invention establishes a heat storage and release mechanism for the pipe network itself and the heat storage device for consuming the passive heat sources; it can make full use of the heat of the passive heat sources, reduce heat waste, improve the energy utilization efficiency, and through the heat storage and release regulation, better balance the heat supply and the heat load demand under different conditions, relieve the supply-demand contradiction, and improve the stability and reliability of the heating system;

[0051] (4) By considering the heat storage and release mechanisms of the pipe network itself and the heat storage device under different typical operation conditions, taking the active heat sources and passive heat sources on the source side as the leaders and the pipe network itself and the heat storage device as the followers, the present invention establishes a master-slave game model. The leaders determine their heat supply strategies, and after observing the decisions of the leaders, the followers select the optimal heat storage and release strategies to conduct the supply-demand collaborative optimized dispatching between the source side and the heat supply station; it can enable all participating parties to achieve the supply-demand collaborative optimized dispatching through the game on the basis of considering their own interests, optimize the resource allocation of the heating system, and improve the overall operation efficiency of the system; in addition, the decision-making processes of the leaders and the followers influence and restrict each other, which can prompt all parties to more comprehensively consider various factors when making decisions, improve the scientificity and rationality of the decisions, and avoid the one-sidedness and limitations that may occur in a single decision-making subject;

[0052] (5) In the present invention, by taking the heat station dispatching center as an agent, continuously receiving the state variables of each heat sub-station at each moment, selecting the corresponding pump valve adjustment control actions of each heat sub-station, calculating the action reward value after executing the actions, guiding the agent to update the action strategy, and obtaining the optimal pump valve adjustment control strategy for each heat sub-station; it can adjust the pump valve operation parameters of the heat sub-station in real time according to the actual operation conditions, realize the refined control of the heating system, meet the heat load requirements of each heat sub-station, improve the heating quality and energy-saving effect, reduce manual intervention at the same time, and improve the automation level and intelligence degree of the heating system.

[0053] Other features and advantages will be described in the following specification, and, in part, will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention are achieved and obtained by the structures specifically pointed out in the specification and the drawings.

[0054] To make the above objectives, features and advantages of the present invention more obvious and understandable, the following specific preferred embodiments are given, and in conjunction with the accompanying drawings, the detailed description is as follows. Brief Description of the Drawings

[0055] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0056] Figure 1 It is a flowchart of a heating system on-demand dispatching control method integrating pipe network heat storage and release and energy storage devices of the present invention. Detailed Embodiments

[0057] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the drawings. Obviously, the described embodiments are some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0058] As Figure 1 shown, this embodiment provides a heating system on-demand dispatching control method integrating pipe network heat storage and release and energy storage devices, including:

[0059] S1. Obtain the heat source data of the active heat source, passive heat source and total heat station side in the historical time period, and perform cluster analysis on the typical operation conditions of supply and demand between the heat source and the total heat station;

[0060] S2. Predict the heat supply of passive heat sources and the heat load demand of the total heat supply station, and analyze the typical operating condition types of the heating system in combination with the output characteristics of active heat sources.

[0061] S3. For each typical operating condition type, establish the heat storage and release mechanisms of the pipe network itself and the heat storage device for accommodating passive heat sources.

[0062] S4. Considering the heat storage and release mechanisms of the pipe network itself and the heat storage device under different typical operating conditions, taking the active heat sources and passive heat sources on the source side as the leaders and the pipe network itself and the heat storage device as the followers, establish a master-slave game model. The leaders determine their heat supply strategies, and after observing the decisions of the leaders, the followers select the optimal heat storage and release strategies to carry out the coordinated optimization dispatch of the supply and demand between the source side and the heat supply station.

[0063] S5. Take the heat supply station dispatching center as an intelligent agent, continuously receive the state variables of each heat sub-station at each moment, select the corresponding pump and valve regulation control actions of each heat sub-station, calculate the action reward value after executing the actions, guide the intelligent agent to update the action strategy, and obtain the optimal pump and valve regulation control strategy of each heat sub-station.

[0064] In this embodiment, the active heat sources on the source side include coal-fired heating units, combined heat and power units, and hot water boilers; the passive heat sources include photovoltaic heating units, wind power heating units, and industrial waste heat heating units.

[0065] It should be noted that for coal-fired heating units: They use coal as fuel, generate heat energy by burning coal, heat water into high-temperature hot water or steam for central heating. Such units have a large heating capacity and can meet large-scale heating demands, but they cause certain pollution to the environment. Pollutants such as sulfur dioxide, nitrogen oxides, and particulate matter are emitted during the combustion process. At the same time, coal resources are non-renewable resources, and with the improvement of environmental protection requirements, their development is restricted to a certain extent.

[0066] Combined heat and power units: Units that produce electric energy and heat energy simultaneously. Generally, they use the extracted steam or exhaust steam of steam turbines to meet the heating demand. They utilize the heat energy that was originally wasted during the power generation process, improving the comprehensive utilization efficiency of energy and reducing energy waste. Compared with pure heating units, combined heat and power units have certain advantages in terms of economy and environmental protection, and can effectively reduce the unit heating cost and pollutant emissions.

[0067] Hot water boiler: By burning fuels (such as coal, natural gas, fuel oil, etc.) or using other energy sources (such as electricity), water is heated to a certain temperature to produce hot water for heating. The structure of a hot water boiler is relatively simple and it operates flexibly. It can adjust its output according to the actual heating demand and is suitable for heating systems of various scales, with wide applications in both civil and industrial heating.

[0068] Photovoltaic heating unit: It uses the electric energy generated by solar photovoltaic power generation to drive a heat pump or other heating equipment to upgrade low-grade heat energy into high-grade heat energy available for heating. Solar energy is a clean and renewable energy source, inexhaustible and renewable. Using a photovoltaic heating unit can reduce the dependence on traditional fossil energy, lower carbon emissions, and hardly produce pollutants during operation, being environmentally friendly. However, the energy density of solar energy is relatively low, and its heating capacity is greatly affected by weather and seasons, so a certain energy storage device needs to be equipped to ensure the stability of heating.

[0069] Wind power heating unit: It relies on the electric energy generated by wind power generation to achieve heating. Its principle is similar to that of a photovoltaic heating unit, which also drives heating equipment through electric energy. Wind energy is also a clean and renewable energy source. In areas rich in wind resources, wind power heating units have good application prospects. However, the instability of wind leads to large fluctuations in power generation, and the continuity and stability of heating need to be ensured through a reasonable energy storage system or combined operation with other heat sources.

[0070] Industrial waste heat heating unit: It recovers the waste heat generated during industrial production and converts it into available heat energy for heating. The sources of industrial waste heat are extensive. For example, in the production processes of industries such as iron and steel, chemical engineering, and building materials, a large amount of high-temperature waste gas, waste water, etc. are discharged. Through heat exchange equipment and heat pump technology, this waste heat can be extracted and the temperature can be raised to meet the heating demand. Utilizing industrial waste heat for heating can not only reduce the consumption of external energy and lower the heating cost, but also effectively reduce the thermal pollution of the environment caused by industrial waste heat emissions, with significant economic and environmental benefits.

[0071] The differences between active and passive heat sources are as follows:

[0072] Energy source: Active heat sources mainly rely on non-renewable fossil energy (such as coal-fired heating units, some hot water boilers) or achieve heating by consuming primary energy (such as cogeneration units); while passive heat sources mainly utilize renewable energy (such as photovoltaic heating units, wind power heating units) or waste heat in industrial production (industrial waste heat heating units), and the energy sources are cleaner and more sustainable.

[0073] Energy conversion methods: Active heat sources usually directly generate heat energy by burning fuels or simultaneously generate heat energy for heating during the power generation process; Passive heat sources first convert other forms of energy (such as electrical energy converted from solar energy and wind energy, and heat energy from industrial waste heat) into heat energy available for heating through specific equipment and technologies. Generally, equipment such as heat pumps and heat exchangers are required to enhance or convert the energy.

[0074] Operation autonomy: Active heat sources can actively adjust the fuel input and operating conditions according to the heating demand, and can more flexibly control the heating output; The output of passive heat sources largely depends on external natural conditions (such as sunlight and wind) or the generation of waste heat during industrial production processes. Their own autonomy is relatively weak, and they need to rely on energy storage equipment or cooperate with other heat sources to meet the requirements of stable heating.

[0075] Impact on the environment: Active heat sources will emit pollutants during the combustion process, causing certain pressure on the environment; Passive heat sources basically do not produce or produce very few pollutants during operation, are more environmentally friendly, and contribute to achieving the goals of energy conservation, emission reduction, and sustainable development.

[0076] In this embodiment, S1 includes:

[0077] Obtain the heat supply data sequences of the active heat sources and passive heat sources on the source side during the historical time period, as well as the heat load demand data sequence on the total heat station side; The heat load demand data sequence on the total heat station side is obtained by accumulating the user heat load demand data of each subordinate heat sub-station;

[0078] Use the heat supply data sequences of the heat sources on the source side and the heat load demand data sequence on the total heat station side during the historical time period as the input of the clustering algorithm, and set the number of clusters and the maximum number of iterations;

[0079] Extract the initial cluster centers from the data sequences, calculate the Euclidean distances between each data in the data sequences and the initial cluster centers, and classify the data and update the initial cluster centers;

[0080] Repeat the iteration until the convergence condition is met, that is, when the change error of the cluster centers is less than the preset value, complete all clustering. The K cluster centers after the final stop of the iteration are the obtained K typical operating conditions.

[0081] It should be noted that the selection and setting of the clustering algorithm: Select the clustering algorithm and set the number of clusters and the maximum number of iterations. The purpose of the clustering algorithm is to group similar data points into the same class, and there are significant differences between data points in different classes. The number of clusters determines how many typical operating condition categories the data will be divided into finally, while the maximum number of iterations limits the running time and computational amount of the algorithm to prevent the algorithm from looping infinitely;

[0082] Initial cluster center determination: Extract the initial cluster centers from the data sequence. The selection of the initial cluster centers will affect the quality of the clustering results and the convergence speed of the algorithm. Usually, data points can be randomly selected as the initial cluster centers, or some heuristic methods can be used to select them to ensure that the initial cluster centers can better represent the distribution of the data as much as possible;

[0083] Distance calculation and data classification: Calculate the Euclidean distance between each data in the data sequence and the initial cluster centers. The Euclidean distance is a commonly used distance metric method for measuring the similarity between two data points in space. According to the distance, each data point is classified into the category to which the nearest cluster center belongs. In this way, the data is initially divided into different categories;

[0084] Cluster center update: After completing the data classification, update the positions of the cluster centers according to the distribution of the data points in each category. Usually, statistical quantities such as the mean or median of all data points in each category are calculated as the new cluster centers. By continuously updating the cluster centers, the clustering results can more accurately reflect the internal structure of the data;

[0085] Iteration and convergence judgment: Repeat the above steps of distance calculation, data classification, and cluster center update until the convergence condition is met. The convergence condition is usually set that the change error of the cluster centers is less than a preset value. When the change of the cluster centers is very small, it means that the clustering results have been stabilized, and the algorithm has found a more reasonable clustering division. At this time, all clustering is completed, and the K cluster centers after the final stop of iteration are the obtained K typical operating conditions.

[0086] In this way, the data with similar characteristics in the historical operation data of the heating system can be grouped into one category, and each cluster center represents a typical operating condition. These typical operating conditions can help the operation and management personnel of the heating system better understand the operating characteristics of the system, predict the heating demand under different conditions, optimize the system scheduling and control, and improve the operating efficiency and reliability of the heating system.

[0087] In this embodiment, the S2 includes:

[0088] According to the type of passive heat sources, obtain the data characteristics affecting the output of passive heat sources during the historical time period, use machine learning algorithms for training and learning, establish a heat supply prediction model for passive heat sources, and obtain the heat supply prediction values of passive heat sources at different time periods;

[0089] Analyze the data characteristics affecting the heat load demand of users in each heat sub-station, use machine learning algorithms to establish a heat load demand prediction model for users in the heat sub-station, and obtain the heat load demand prediction values of each heat sub-station at different time periods;

[0090] Based on the predicted values of the heat load demands of each heat sub-station at different time periods, cumulative calculations are performed to obtain the heat load demand values of the main heat sub-station at different time periods;

[0091] Determine the maximum output, minimum output, and regulation speed of the active heat sources, and analyze the predicted output range of the active heat sources at different time periods according to the equipment characteristics, operating parameters, and fuel supply conditions of the active heat sources;

[0092] According to the predicted heat supply values of the passive heat sources at different time periods, the heat load demand values of the main heat sub-station at different time periods, and the predicted output range of the active heat sources at different time periods, analyze the typical operating condition types of the heating system based on the supply-demand balance degree, the dominant type of heat source, and the heat source combination ratio.

[0093] In actual applications, the operating conditions are divided according to the heat source combination method, including:

[0094] Pure active heat source heating condition: When the passive heat sources cannot operate normally due to weather conditions (such as photovoltaic heating units on cloudy days, wind heating units in windless conditions) or equipment failures, etc., the heating system relies only on active heat sources (such as coal-fired heating units, cogeneration units, hot water boilers) to provide heat to meet the heat load demand on the main heat sub-station side. In this condition, the load pressure on the active heat sources is relatively large, and more fuel resources need to be consumed;

[0095] Active and passive heat source combined heating condition: Under normal operating conditions, the active heat sources and passive heat sources operate together and cooperate with each other to provide heat for the heating system. According to different weather conditions, heat load demands, and the characteristics of various heat sources, reasonably adjust the output ratio of the active heat sources and passive heat sources to achieve the efficient and economic operation of the heating system. For example, on sunny and windy days, appropriately increase the output of photovoltaic heating units and wind heating units, reduce the fuel consumption of active heat sources, and lower the heating cost;

[0096] Passive heat source dominant heating condition: When the output of the passive heat sources can meet most of the heat load demand on the main heat sub-station side, the heating system is mainly heated by passive heat sources, and the active heat sources are used as backup or supplement. This condition usually occurs in areas where passive heat source resources are rich and stable, such as industrial parks with a large amount of industrial waste heat, or areas with sufficient solar and wind energy resources. At this time, the heating system can make full use of renewable energy and industrial waste heat, reduce the dependence on traditional fossil energy, and reduce environmental pollution.

[0097] The operating conditions are divided according to the supply-demand balance, including:

[0098] Supply-demand balance condition: When the supply-demand balance degree is within the allowable error range and the heat source combination ratio meets the normal operation requirements, it is determined as the supply-demand balance condition. At this time, the heating system can operate stably, meet the heating demand of users, and parameters such as heating temperature and pressure are maintained within the normal range;

[0099] Supply exceeding demand condition: If the supply-demand balance degree is greater than the upper limit of the allowable error, it indicates that the heating system is in the supply-exceeding-demand condition. At this time, it is necessary to consider appropriately reducing the heat source output or storing the excess heat in the pipe network itself or the heat storage device to avoid energy waste and damage to the system caused by excessive heating parameters;

[0100] Supply falling short of demand condition: When the supply-demand balance degree is less than the lower limit of the allowable error, it means that the heating system supplies less than the demand. At this time, first release the heat in the pipe network itself and the heat storage device, and then, according to the output characteristics of the active heat source, meet the heat load demand by increasing the output of the active heat source or adjusting the operation mode;

[0101] Special conditions: For example, under extreme weather conditions, the heat load demand of the total heat supply station increases significantly, which may cause the heating system to enter a special high-load operation condition; or when the passive heat source cannot operate normally due to equipment failure, weather conditions, etc., the heating system may enter an emergency operation condition mainly based on the active heat source;

[0102] The operation conditions of the heating system are divided by combining the supply-demand balance degree, the dominant type of heat source, and the heat source combination ratio.

[0103] In this embodiment, S3 includes:

[0104] For each typical operation condition type, preferentially absorb the passive heat source, and then determine the proportion of the active heat source for heat supply at different times according to the supply-demand balance degree. When the supply exceeds the demand, store the excess heat preferentially in the pipe network itself, and then in the heat storage device; when the supply is less than the demand, preferentially release the heat stored in the pipe network itself, and then release the heat in the heat storage device, and finally, the active heat source makes up for the missing heat.

[0105] In this embodiment, in S4, establishing the master-slave game model includes:

[0106] Determine the participants in the master-slave game process, take the active heat source and the passive heat source on the source side as the leaders, the pipe network itself and the heat storage device as the followers, and set the benefit functions of the leaders and the followers as U L and U F respectively, and the game strategies as s L and s F respectively, and construct the master-slave game model;

[0107] Among them, the benefit function of the leader is to maximize the revenue on the source side, expressed as:

[0108]

[0109] K is the category of typical operating conditions; T is the total number of time steps in the scheduling period; λ a,k,t 、λ p,k,t are the heat selling prices of the active heat sources and passive heat sources on the source side at time t under the k-th typical operating condition respectively; H a,k,t 、H p,k,t are the supplied heat quantities of the active heat sources and passive heat sources on the source side at time t under the k-th typical operating condition respectively; C a,k,t 、C p,k,t are the operating costs of the active heat sources and passive heat sources on the source side at time t under the k-th typical operating condition respectively; is the penalty cost of the passive heat source at time t under the k-th typical operating condition;

[0110] The game strategy s of the leader L is the heat supply values of the active heat sources and passive heat sources on the source side at time t under the k-th typical operating condition;

[0111] The benefit function of the follower is to minimize the charging and discharging operating costs, expressed as:

[0112]

[0113] b r,cha,k,t 、b r,dis,k,t are the charging and discharging cost coefficients of the pipe network itself at time t under the k-th typical operating condition respectively; P r,cha,k,t 、P r,dis,k,t are the charging and discharging powers of the pipe network itself at time t under the k-th typical operating condition respectively; X r,cha,k,t 、X r,dis,k,t are the charging and discharging states of the pipe network itself at time t under the k-th typical operating condition respectively, with values of 0, 1; b c,cha,k,t 、b c,dis,k,t are the charging and discharging cost coefficients of the heat storage device at time t under the k-th typical operating condition respectively; P c,cha,k,t 、P c,dis,k,t are the charging and discharging powers of the heat storage device at time t under the k-th typical operating condition respectively; X c,cha,k,t 、X c,dis,k,t are the charging and discharging states of the heat storage device at time t under the k-th typical operating condition respectively, with values of 0, 1;

[0114] The game strategy s of the follower F is the charging and discharging states and powers of the pipe network itself and the heat storage device at time t under the k-th typical operating condition;

[0115] Given the active heat source and passive heat source strategies \(s\) on the leader's source side L , the follower obtains the optimal strategy \(s\) by solving its benefit function \(U\) F ; then the leader solves its benefit function \(U\) according to the follower's optimal strategy \(s\) * F to obtain the optimal strategy \(s\) * F ; L ; * L .

[0116] It should be noted that in the master-slave game model, the relationship between the participants reflects the dominant and subordinate relationship, and there is an obvious priority order in the decision-making process of each participant: in the game process, the leader makes its own decision first, and then the follower can formulate corresponding strategies according to the existing information. Therefore, the leader has a priority position and a first-mover advantage in this game process. When the decisions made by all the participants in the master-slave game reach the optimal, the game is said to reach equilibrium.

[0117] The solution process of the master-slave game model includes:

[0118] Step 1) Initialize the parameters and variables of the heat supply system source side and the total heat substation, including the typical operating condition types, the output of the active heat source, the power of the passive heat source, the heat storage and release power, the heat load demand, etc.;

[0119] Step 2) At each time step, given the active heat source and passive heat source strategies \(s\) on the leader's source side L , solve the benefit function of the follower to obtain the optimal strategy \(s\) * F ;

[0120] Step 3) Substitute the optimal strategy \(s\) of the follower * F into the benefit function of the leader to solve the optimal strategy \(s\) of the leader * L ;

[0121] Step 4) Repeat steps 2 and 3 until the global optimal solution or the total number of time steps \(T\) of the scheduling period is reached;

[0122] Step 5) Output the optimal scheduling strategy of the heat supply system, including the optimal output of the active heat source and passive heat source on the source side, the heat storage and release states of the pipe network itself and the heat storage device, and the optimal heat storage and release power.

[0123] In this embodiment, the establishment of the master-slave game model further includes setting operation constraint conditions, including heat load balance constraint, heat storage and release operation constraint, heat storage and release capacity constraint, and operation constraints of active heat sources and passive heat sources on the source side.

[0124] In this embodiment, S5 includes:

[0125] Converting the pump-valve regulation control problem of each heat sub-station into a Markov decision model;

[0126] Regarding the heat sub-station scheduling center as an agent, continuously receiving the state variables of each heat sub-station at each moment, selecting the pump-valve regulation control action strategies of each heat sub-station, and applying the pump-valve regulation control action strategies to the operation environment of each heat sub-station in the heating system. The system will transfer to a new state, and at the same time, feedback the action reward value to the heat sub-station scheduling center to guide the agent to update the action strategy and obtain the optimal pump-valve regulation control strategy for each heat sub-station;

[0127] Among them, the state variables include the heat load demand of each heat sub-station, the total heat supply of the total heat sub-station, the heat output of the heat source on the source side, the heat storage and release states of the pipe network itself and the heat storage device, and the heat storage and release power;

[0128] The action strategies include the operating frequency of the water pumps of each heat sub-station and the adjustment opening of the electric control valves;

[0129] The action reward value includes the maximum satisfaction level of the on-demand distribution of the heat load of each heat sub-station.

[0130] In this embodiment, the Markov decision model is trained through a reinforcement learning algorithm to obtain a heat sub-station scheduling center reinforcement learning agent.

[0131] In actual applications, an evaluation network (Critic) is constructed to evaluate the pros and cons of the current action strategy. Using the current state variables as the input of the neural network, estimating the current state value function, outputting a reliable estimated value, and optimizing the strategy update process. At the same time, an action network (Actor) is constructed to evaluate the difference between the old and new strategies. Using the observed state variables as the input of the neural network, the output layer outputs an action, and then samples this action to obtain an action value, which is mapped to the operation environment of each heat sub-station in the simulated heating system, and the evaluation network evaluates the pros and cons of the current strategy.

[0132] The reinforcement learning agent can autonomously select the optimal control strategy according to the current system state without the need for complex rules to be set in advance manually. As the training progresses, the agent can continuously optimize its strategy to adapt to various changes in the operation of the heat sub-station and achieve intelligent optimization regulation control.

[0133] Handling complex constraint conditions: The regulation and control of heat stations need to consider various constraint conditions, such as the output limit of heat sources, the transmission capacity of heat networks, and the comfort requirements of users. The Markov decision model and reinforcement learning algorithm can incorporate these constraint conditions into the model. By designing appropriate reward functions and state spaces, the intelligent agent is guided to find the optimal regulation and control strategy while satisfying the constraints. This can effectively handle complex practical problems and improve the reliability and stability of heat station operation.

[0134] In this embodiment, during the training process of the intelligent agent, the safe operation constraints of each heat sub-station are set in the form of penalties. When the constraints are violated, corresponding penalties are imposed. If the safe operation constraints are met, the penalty value is 0.

[0135] In several embodiments provided in this application, it should be understood that the disclosed systems and methods can also be implemented in other ways. The system embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of systems, methods, and computer program products according to multiple embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code. A module, a program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0136] In addition, each functional module in various embodiments of the present invention may be integrated together to form an independent part, or each module may exist alone, or two or more modules may be integrated to form an independent part. If the function is implemented in the form of a software functional module and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories, random access memories, magnetic disks, or optical discs that can store program codes.

[0137] Taking the above ideal embodiments of the present invention as an inspiration, through the above description, relevant staff can completely make various changes and modifications without departing from the technical idea of this invention. The technical scope of this invention is not limited to the content in the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A method for on-demand dispatching and controlling a heating system integrating a heat storage and energy storage device in a pipe network, characterized in that: include: S1. Obtain the heat load data of active heat sources, passive heat sources and the total heat station side in the historical period, and perform cluster analysis on the typical operating conditions of supply and demand between the heat source and the total heat station; S2. Predict the heat supply of passive heat sources and the heat load demand of the total thermal power station, and analyze the typical operating conditions of the heating system in combination with the output characteristics of active heat sources; S3. For each typical operating condition type, establish a heat storage and release mechanism for the pipe network itself and the heat storage device for absorbing passive heat sources; S4. Consider the heat storage and release mechanism of the pipeline network itself and the heat storage device under different typical operating conditions, take the active heat source and passive heat source on the source side as the leader, and the pipeline network itself and the heat storage device as the follower, and establish a master-slave game model. The leader determines its heat supply strategy. After observing the leader's decision, the follower selects the optimal heat storage and release strategy to perform supply and demand coordinated optimization scheduling between the source side and the thermal power station; S5. Take the thermal power station dispatching center as an intelligent agent, continuously receive the state variables of each thermal substation at each moment, and select the corresponding pump and valve regulation control actions of each thermal substation, calculate the action reward value after executing the action, guide the intelligent agent to update the action strategy, and obtain the optimal pump and valve regulation control strategy for each thermal substation.

2. The heating system on-demand scheduling control method according to claim 1 is characterized in that: The active heat sources on the source side include coal-fired heating units, cogeneration units, and hot water boilers; the passive heat sources include photovoltaic heating units, wind power heating units, and industrial waste heat heating units.

3. The on-demand dispatching control method for a heating system according to claim 1, characterized in that: The S1 includes: Obtaining the heat supply data sequence of active heat sources and passive heat sources on the source side and the heat load demand data sequence on the total heat power station side within the historical time period; the heat load demand data sequence on the total heat power station side is accumulated from the heat load demand data of users of the heat power substations under its jurisdiction; The heat supply data sequence of the heat source on the source side and the heat load demand data sequence on the total thermal power station side in the historical time period are used as the input of the clustering algorithm, and the number of clusters and the maximum number of iterations are set; Extract the initial cluster center from the data sequence, calculate the Euclidean distance between each data in the data sequence and the initial cluster center, classify the data and update the initial cluster center; The iteration is repeated until the convergence condition is met, that is, when the cluster center change error is less than the preset value, all clustering is completed, and the K cluster centers after the iteration is finally stopped are the K typical operating conditions obtained.

4. The on-demand dispatching control method for a heating system according to claim 1, characterized in that: The S2 includes: According to the type of passive heat source, the data characteristics that affect the output of passive heat sources in the historical period are obtained, and the machine learning algorithm is used for training and learning to establish a heat supply prediction model for passive heat sources, and the heat supply prediction values ​​of passive heat sources in different periods are obtained; Analyze the data characteristics that affect the heat load demand of users in each thermal substation, establish a heat load demand prediction model for users in the thermal substation using machine learning algorithms, and obtain the predicted values ​​of heat load demand for each thermal substation in different periods of time; According to the predicted heat load demand values ​​of each thermal power substation at different time periods, cumulative calculation is performed to obtain the heat load demand values ​​of the total thermal power station at different time periods; Determine the maximum output, minimum output and adjustment speed of the active heat source, and analyze the expected output range of the active heat source at different times based on the equipment characteristics, operating parameters and fuel supply of the active heat source; According to the predicted heat supply values ​​of passive heat sources in different time periods, the heat load demand values ​​of the total thermal power station in different time periods and the expected output range of active heat sources in different time periods, the typical operating condition type of the heating system is analyzed according to the supply and demand balance, the dominant type of heat source and the combination ratio of heat sources.

5. The on-demand dispatching control method for a heating system according to claim 1, characterized in that: The S3 includes: For each typical operating condition type, passive heat sources are consumed first, and then the proportion of heat supply by active heat sources in different periods is determined according to the balance between supply and demand. When supply exceeds demand, excess heat is stored in the pipeline network itself first, and then in the heat storage device. When supply is less than demand, the heat stored in the pipeline network itself is released first, and then the heat in the heat storage device is released, and finally the missing heat is replenished by active heat sources.

6. The heating system on-demand scheduling control method according to claim 1, characterized in that: In S4, establishing a master-slave game model includes: Determine the participants in the master-slave game process, take the active heat source and passive heat source on the source side as the leader, the pipe network itself and the heat storage device as the follower, and set the benefit functions of the leader and follower as U L and U F , the game strategies are s L and F , construct a master-slave game model; Among them, the leader's benefit function is to maximize the source side benefit, which is expressed as: K is the category of typical operating conditions; T is the total time steps of the scheduling cycle; λ a,k,t , p,k,t are the heat selling prices of active heat source and passive heat source at the source side at time t under the kth typical operating condition; H a,k,t , H p,k,t are the heat supplied by the active heat source and the passive heat source at the source side at time t under the kth typical operating condition; C a,k,t , C p,k,t are the operating costs of the active heat source and the passive heat source on the source side at time t under the kth typical operating condition; is the penalty cost of the passive heat source at time t under typical operating conditions of type k; Leaders’ game strategies L is the heat supply value of the active heat source and passive heat source on the source side at time t under the kth typical operating condition; The follower's benefit function is to minimize the heat storage and release operation cost, which can be expressed as: b r,cha,k,t , b r,dis,k,t are the heat storage and heat release cost coefficients of the pipeline network itself at time t under typical operating conditions of type k; P r,cha,k,t , P r,dis,k,t are the heat storage and heat release powers of the network at time t under typical operating conditions of type k; X r,cha,k,t , X r,dis,k,t are the heat storage and heat release states of the pipeline network at time t under typical operating conditions of type k, with values ​​of 0 and 1; b c,cha,k,t , b c,dis,k,t are the heat storage and heat release cost coefficients of the heat storage device at time t under typical operating conditions of type k; P c,cha,k,t , P c,dis,k,t are the heat storage and heat release powers of the heat storage device at time t under typical operating conditions of type k; X c,cha,k,t , X c,dis,k,t are the heat storage and heat release states of the heat storage device at time t under typical operating conditions of type k, with values ​​of 0 and 1; Followers' game strategies F is the heat storage and release state and heat storage and release power of the pipe network itself and the heat storage device at time t under the kth typical operating condition; Active heat source and passive heat source strategies s on a given leader source side L In the case of F Get the optimal strategy * F , then the leader follows the follower's optimal strategy s * F Solve its benefit function U L Get the optimal strategy * L .

7. The heating system on-demand scheduling control method according to claim 6, characterized in that: The establishment of the master-slave game model also includes setting operating constraints, including heat load balance constraints, heat storage and release operating constraints, heat storage and release capacity constraints, and operating constraints of active heat sources and passive heat sources on the source side.

8. The heating system on-demand scheduling control method according to claim 1, characterized in that: The S5 includes: The pump and valve regulation control problem of each thermal substation is transformed into a Markov decision model; The thermal power station dispatch center is used as an intelligent agent to continuously receive the state variables of each thermal substation at each moment, select the pump and valve regulation control action strategy of each thermal substation, and apply the pump and valve regulation control action strategy to the operating environment of each thermal substation in the heating system. The system will transfer to a new state and at the same time feedback the action reward value to the thermal power station dispatch center to guide the intelligent agent to update the action strategy and obtain the optimal pump and valve regulation control strategy for each thermal substation. Among them, the state variables include the heat load demand of each thermal substation, the total heat supply of the main thermal station, the heat source output on the source side, the heat storage and release state of the pipe network itself and the heat storage device, and the heat storage and release power; The action strategy includes the operating frequency of the water pumps in each thermal substation and the regulating opening of the electric regulating valve; The action reward value includes the maximum satisfaction level of the on-demand distribution of heat load of each thermal substation.

9. The heating system on-demand scheduling control method according to claim 8, characterized in that: The Markov decision model is trained by a reinforcement learning algorithm to obtain a reinforcement learning agent of a thermal power station dispatching center.

10. The heating system on-demand scheduling control method according to claim 9, characterized in that: During the training of the intelligent agent, the safe operation constraints of each thermal substation are set in the form of penalties. When the constraints are violated, corresponding penalties are imposed. If the safe operation constraints are met, the penalty value is 0.

Images