Optimized scheduling method based on deep reinforcement learning and capable of improving flexibility of heat supply system

Abstract

The invention discloses an optimal scheduling method based on deep reinforcement learning and improving the flexibility of a heat supply system. The method comprises the following steps: carrying out collection and coding extraction on high-dimensional uncertainty characteristics related to a flexibility influence caused by operation scheduling of the heat supply system; the optimization scheduling model problem for improving the flexibility of the heat supply system is defined as follows: in order to achieve the minimum total cost of the heat supply system as the target, the output of a source side conventional heat supply unit and renewable energy sources, the structural layout of a pipe network and the flexible heat load adjusting amount under multiple time scales are controlled and adjusted, and the influence caused by a source network load side and disturbance random factors is dealt with; the optimization scheduling model problem for improving the flexibility of the heat supply system is converted into a Markov decision process, high-dimensional features extracted by embedding codes and a state set of an environment where a design agent observes, an action set for exerting influence on the environment and a profit set obtained after actions are executed on the environment. And training and optimizing the scheduling model through a deep reinforcement learning algorithm, and outputting an optimal scheduling decision.

Description

Technical Field

[0001] The present invention belongs to the technical field of heating systems, and specifically relates to an optimization scheduling method based on deep reinforcement learning and improving the flexibility of heating systems. Background Art

[0002] The rapid development of renewable energy is of great significance to alleviating the pressure on the current energy supply of the heating system and promoting the low-carbon operation of the power system. However, the output of renewable energy, especially wind and solar energy, is uncertain. If we want to ensure the safe and reliable operation of the heating system while introducing renewable energy, the heating system needs to be flexible enough.

[0003] In addition, the flexibility of the heating system can be obtained from multiple aspects such as the source side, the network side and the load side. Different flexibility resources have different characteristics and require qualitative and quantitative analysis. For example, the output of the heating unit on the source side and the load demand of the end user change over time, the heat transmitted by the pipeline network also changes according to the changes in heat supply and demand and the topological structure of the pipeline network, and the continuous increase in the scale of the heating system, all of which bring great uncertainty to the heating system. At the same time, it also greatly increases the difficulty of modeling the optimal scheduling model of the heating system, and puts forward higher requirements for scheduling. How to specifically analyze the multiple uncertainties in the heating system, reduce the uncertainty of the heating system scheduling problem and improve the flexibility and optimization scheduling of the heating system is an urgent problem to be solved. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide an optimization scheduling method based on deep reinforcement learning and improving the flexibility of the heating system.

[0005] In order to solve the above technical problems, the technical solution of the present invention is:

[0006] The present invention provides an optimization scheduling method based on deep reinforcement learning and improving the flexibility of a heating system, comprising:

[0007] S1. Analyze the first flexibility impact on system operation and scheduling when renewable energy on the source side is connected to the heating system, including the impact of the randomness of renewable energy output on the balance of supply and demand and the impact of renewable energy access on operating costs;

[0008] S2. Analyze the second flexibility impact on system operation and scheduling when users of load-side flexible heat loads are connected to the heating system, including the impact of flexible heat loads on supply and demand balance, the impact of flexible heat load access on heat load demand forecast, and the impact of the mismatch between source-load output and load-side demand on renewable energy utilization;

[0009] S3. Analyze the impact of random disturbance factors on the third flexibility of the operation and scheduling of the heating system, including the impact of uncontrollable external extreme weather, the impact of changes in the pipe network topology caused by heating network failures, and the impact of equipment operation status;

[0010] S4. Analyze the fourth flexibility impact on the operation and scheduling of the heating system caused by the expansion of the grid-side node scale, the increase in the access to heat sources on the source side, and the increase in the access to heat users on the load side;

[0011] S5. collecting and encoding and extracting high-dimensional uncertainty features involved in the first flexibility impact, the second flexibility impact, the third flexibility impact and the fourth flexibility impact caused by the operation and scheduling of the heating system;

[0012] S6. The optimization scheduling model problem for improving the flexibility of the heating system is defined as: considering the heat load supply and demand balance constraints of the heating system, the output constraints of the source-side heating units, the ramping constraints, the flexible heat load regulation capacity constraints, and the network operation constraints, with the goal of minimizing the total cost of the heating system, controlling and regulating the output of the source-side conventional heating units and renewable energy, the network structure layout and the flexible heat load regulation under multiple time scales, and responding to the impact of the source-grid load side and disturbance random factors;

[0013] S7. Convert the optimization scheduling model problem for improving the flexibility of the heating system into a Markov decision process, embed the high-dimensional features extracted by the encoding and design the set of states of the environment observed by the intelligent agent, the set of actions that affect the environment, and the set of benefits obtained after performing actions on the environment, and train and optimize the scheduling model through a deep reinforcement learning algorithm to output the optimal scheduling decision.

[0014] Further, the S1 specifically includes:

[0015] When the renewable energy heating units including solar heating units and wind heating units are connected to the heating system, during the system scheduling process, the source side scheduling object is changed from the original non-renewable energy heating units with controllable output to a mixed heating system of non-renewable energy heating units and renewable energy heating units with increased randomness and uncertainty on the source side;

[0016] Among them, the output of renewable energy heating units is uncertain due to the influence of weather, season and geographical location. When the output is insufficient, it will cause insufficient heat supply to users, while when the output is excessive, it will cause resource waste and lead to imbalance between supply and demand.

[0017] The cost of connecting renewable energy heating units to the heating system is higher than that of non-renewable energy. In addition, due to the distribution characteristics of renewable energy heating units, the heat transmission cost and equipment operation and maintenance cost are increased. In order to compensate for the uncertainty of renewable energy output, the supply and demand balancing cost and standby unit cost are increased while meeting the safe operation constraints.

[0018] Further, the S2 specifically includes:

[0019] Analysis: When flexible heat load users on the load side are connected to the heating system, they adjust their own heat load according to their own needs and heat load regulation capabilities; during peak hours, flexible heat load users reduce their own heat loads to reduce the peak load of the heating system, thus avoiding or reducing system crashes or performance degradation caused by excessive loads; reasonably dispatch flexible heat load users, and the heating system flexibly adjusts the supply and demand balance;

[0020] Obtain the heat usage data and behavior model of flexible heat load users, and optimize and adjust the heat load demand prediction model as the number of flexible heat load users increases;

[0021] Flexible heat load users are dispatched to reduce the waste of renewable energy caused by source-load output mismatch, and the flexible heat load acts as a buffer to adjust the operation of its own load balancing system when the renewable energy output fluctuates or the load-side demand changes, thereby improving the utilization efficiency of renewable energy.

[0022] Further, the S3 specifically includes:

[0023] Analyze the impact of sudden changes in external temperature and severe weather on the heat demand of load-side users and the heating design capacity of the source-side heating units, resulting in supply-demand balance and damage, rupture and leakage of heating pipelines;

[0024] Analysis of heat network pipeline damage, rupture, leakage failures, resulting in heat loss, heating interruption, changes in the heating network heating path and pipeline topology, affecting the equipment operating status and operating parameters of water pumps, valves, heating units, and thermal power stations.

[0025] Further, the S4 specifically includes:

[0026] Analysis: The expansion of the network-side node scale has increased the complexity of the network structure and topology of the heating system;

[0027] Analysis shows that the increase in access to heat sources on the source side and the increase in access to heat users on the load side has increased the flexibility and complexity of system operation scheduling, and increased the probability of mismatch between supply and demand.

[0028] Furthermore, the analysis of the first flexibility impact on the system operation scheduling caused by the connection of the source-side renewable energy to the heating system also includes establishing an output prediction model for the renewable energy heating unit. The specific process is as follows:

[0029] Obtain real historical renewable energy heating unit output data, including environmental parameters, unit operating status, and time series data, and initialize the parameters of the generator and discriminator in the generative adversarial network;

[0030] Fixed discriminator, training generator: The generator takes random noise as input to generate synthetic data close to real data, and the discriminator judges the authenticity score of the data; the goal of the generator is to deceive the discriminator so that it cannot accurately judge the authenticity of the data; the parameters of the generator are updated through the back propagation algorithm;

[0031] Fix the generator and train the discriminator: After the generator training is completed, fix the parameters of the generator and train the discriminator; the goal of the discriminator is to determine whether the input data is real or generated; the parameters of the discriminator are updated through the back propagation algorithm;

[0032] Through cyclic training, the generator can generate synthetic data close to real data, and the discriminator cannot judge the authenticity of the data;

[0033] After the training is completed, the generator generates virtual output data of new energy heating units, which is then fused with the real historical output data of renewable energy heating units to train the recurrent neural network RNN ​​and establish an output prediction model for renewable energy heating units.

[0034] Furthermore, the analysis of the second flexibility impact on the system operation scheduling caused by the load-side flexible heat load user connecting to the heating system also includes establishing a user heat load prediction model, and the specific process is as follows:

[0035] After obtaining the original load signal of the user including the flexible heat load, the EEMD algorithm is used to decompose it into high-frequency heat load component, low-frequency heat load component and residual component;

[0036] For the high-frequency heat load component, the XGBoost algorithm is used to extract features and perform nonlinear fitting processing to obtain the corresponding first heat load prediction sequence. For the low-frequency heat load component and the residual component, the SARIMA model is used to fit the heat load trend curve to obtain the corresponding second heat load prediction value.

[0037] Define the first heat load prediction sequence as a, the second heat load prediction sequence as b, and the corresponding actual heat load sequence as h. Assign weights w1 and w2 to the first heat load prediction sequence a and the second heat load prediction sequence b, respectively, w1+w2=1, and establish the error of the training heat load reconstruction prediction model as:

[0038] ε=h-w1a-w2b=h-w1a-(1-w1)b;

[0039] The weights are solved by minimizing the error of the reconstructed prediction model as the optimization goal, which is expressed as:

[0040]

[0041] The optimization objective function is set as:

[0042]

[0043] n is the number of heat load sampling points;

[0044] Set the constraints as:

[0045] 0≤w1≤1;

[0046] An intelligent optimization algorithm is used to optimize the weights, obtain the optimal weights w1 and w2, and establish a heat load reconstruction prediction model.

[0047] Further, the S5 specifically includes:

[0048] A machine learning algorithm is used to collect and encode high-dimensional uncertainty features involved in the first flexibility impact, the second flexibility impact, the third flexibility impact and the fourth flexibility impact caused by the operation and scheduling of the heating system, and an encoding result characterizing the high uncertainty features of the heating system is obtained;

[0049] Among them, high-dimensional uncertainty characteristics include at least: uncertainty characteristics of heat source output on the source side; uncertainty characteristics of changes in heat user demand; uncertainty characteristics of weather changes in temperature, wind speed, rainfall and snowfall; uncertainty characteristics of events and locations of damage, rupture, leakage failures in the heat network pipeline; and uncertainty characteristics of changes in network topology layout.

[0050] Further, in S7, the state set of the environment observed by the design agent includes: current heat load demand, renewable energy heating unit status, conventional heating unit status, pipe network operation status, pipe network topology structure and access volume scale;

[0051] The set of actions that the design exerts on the environment includes: adjusting the output of conventional heating units and renewable energy units, changing the layout of the pipe network structure, adjusting the flexible heat load and regulating the operating parameters of valves, water pumps, and heat exchanger units;

[0052] The benefit set obtained after the design performs actions on the environment includes: the total cost of the heating system composed of the original total cost and the subsequent optimization scheduling cost, and the supply-demand balance; the original total cost includes the system heating cost, the unit maintenance cost and the start-up and shutdown cost; the subsequent optimization scheduling cost includes the heat cost increased by the source-side optimization scheduling, the subsidy cost increased by the load-side flexible heat load optimization scheduling, and the cost of re-layout of the pipeline network structure; the supply-demand balance is the ratio of the total heat load demand on the load side to the total heat output of the source-side unit. If the ratio is within the preset threshold range, it indicates that the supply and demand are balanced, and a benefit is given; if the ratio is outside the preset threshold range, it indicates that the supply and demand are unbalanced, and a penalty is given;

[0053] The deep reinforcement learning algorithm is a TD3 algorithm based on a double replay buffer mechanism and a GRU neural network fusion; the double replay buffer mechanism stores experience data of different importance levels in categories according to the reward value, and during training, extracts some experience data of high importance from each replay buffer in order of probability from large to small, and extracts some experience data of low importance from each replay buffer in order of probability from small to large as samples; the reward value is positively correlated with the importance of the data; the Actor-Critic network in the TD3 algorithm adopts a fully connected network structure, and the state data processing process is integrated into the GRU network. The GRU network retains the meaningful historical state information of the previous moment and the current input state for calculation, and the retained state that needs to be remembered is passed to the last layer of the Actor-Critic network for processing.

[0054] Furthermore, the supply and demand flexibility between the source-side renewable energy heating unit and the load-side heat load is expressed as:

[0055]

[0056] are the upward and downward flexibility requirements of supply and demand in period t respectively; q u ,q d are the upward and downward adjustment flexibility coefficients caused by the output forecast error of the renewable energy heating unit; e u 、e d are the up and down adjustment flexibility coefficients caused by the heat load prediction error; P re,t+1 , P h,t+1 are the output forecast value and heat load forecast value of renewable energy heating units in period t+1 respectively; P h,t is the predicted value of heat load during period t+1; P re,max It is the maximum output of the renewable energy heating unit;

[0057] The optimization scheduling method further includes:

[0058] The flexibility of conventional thermal power heating units is analyzed and expressed as:

[0059]

[0060] They are respectively the upward and downward flexible supply capabilities of conventional thermal power heating units; are the upper and lower limits of the output of conventional thermal power heating unit m; u th,m,t is the operating status of the conventional thermal power heating unit m at time t. When its value is 0, it means it is shut down, and when it is 1, it means it is running; P th,m,t is the output of conventional thermal power heating unit m at time t; are the upward and downward climbing capabilities of the conventional thermal power heating unit m, respectively; Δt is the scheduling time interval;

[0061] The flexibility of the thermal storage device is analyzed and expressed as:

[0062]

[0063] They are the upward and downward flexible supply capabilities of the heat storage device; is the maximum heat storage power of the heat storage device o; P st,o,c,t is the heat storage power of heat storage device o at time t; is the maximum heat release power of the heat storage device o; P st,d is the heat release power of the heat storage device o at time t;

[0064] Calculate the operating cost of conventional thermal power heating units participating in peak load dispatching, expressed as:

[0065]

[0066] I th,m,t are the peak-shaving cost, start-stop cost, heat loss cost and peak-shaving compensation income of conventional thermal power heating unit m at time t; T is the total dispatching time; N th The number of conventional thermal power generating units;

[0067] Calculate the operating cost of the heat storage device participating in peak load scheduling, expressed as:

[0068]

[0069] C st,o,t I is the cost of the heat storage device participating in peak load regulation, including heat storage cost, heat release cost and loss cost; st,o,t Peak-shifting benefits for thermal storage devices;

[0070] Calculate the operating cost of the flexible heat load participating in peak load scheduling, expressed as:

[0071]

[0072] n is the number of times or time periods that the flexible heat load participates in peak regulation; C i,fh C is the cost of load reduction and transfer in the i-th peak load event; i,re is the cost incurred in response to the change in the behavior of the flexible heat load in the i-th peak load event; C i,de Cost of control equipment related to flexible heat load peak regulation; C i,ma Maintenance costs for peak load shaving equipment;

[0073] Furthermore, in the optimization scheduling model for improving the flexibility of the heating system, when the goal is to minimize the total cost of the heating system, in addition to considering the heat supply cost, unit maintenance cost, unit start-up and shutdown cost, external impact cost, node scale expansion cost, pipeline structure layout cost, and equipment failure recovery cost of the source-side renewable energy heating units and non-renewable energy heating units, it is also necessary to consider the operating cost of conventional thermal power heating units participating in peak-shaving scheduling, the operating cost of heat storage devices participating in peak-shaving scheduling, and the operating cost of flexible heat loads participating in peak-shaving scheduling.

[0074] The beneficial effects of the present invention are:

[0075] The present invention analyzes the impact of source-side renewable energy and flexible heat load on the flexibility of the source and load when they are connected to the heating system, as well as the impact of flexibility caused by external disturbance random factors and system scale, and formulates a high-dimensional feature encoding method for the heating system with flexibility impact. Taking into account the flexibility factor, the heating system optimization scheduling problem is defined as a reinforcement learning model, and the state-action space and benefit function of the intelligent agent are designed. This improves the accuracy of the flexible heating system scheduling and the training efficiency of the intelligent agent, thereby ensuring the safe and reliable operation of the heating system.

[0076] Other features and advantages will be described in the following description, and partly become apparent from the description, or understood by practicing the invention. The purpose and other advantages of the invention are realized and obtained by the structures particularly pointed out in the description and the drawings.

[0077] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0079] Figure 1 This is a flow chart of an optimization scheduling method based on deep reinforcement learning and improving the flexibility of a heating system according to the present invention. DETAILED DESCRIPTION

[0080] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0081] Example 1

[0082] like Figure 1 As shown, this embodiment 1 provides an optimization scheduling method based on deep reinforcement learning and improving the flexibility of the heating system, including:

[0083] S1. Analyze the first flexibility impact on system operation and scheduling when renewable energy on the source side is connected to the heating system, including the impact of the randomness of renewable energy output on the balance of supply and demand and the impact of renewable energy access on operating costs;

[0084] S2. Analyze the second flexibility impact on system operation and scheduling when users of load-side flexible heat loads are connected to the heating system, including the impact of flexible heat loads on supply and demand balance, the impact of flexible heat load access on heat load demand forecast, and the impact of the mismatch between source-load output and load-side demand on renewable energy utilization;

[0085] S3. Analyze the impact of random disturbance factors on the third flexibility of the operation and scheduling of the heating system, including the impact of uncontrollable external extreme weather, the impact of changes in the pipe network topology caused by heating network failures, and the impact of equipment operation status;

[0086] S4. Analyze the fourth flexibility impact on the operation and scheduling of the heating system caused by the expansion of the grid-side node scale, the increase in the access to heat sources on the source side, and the increase in the access to heat users on the load side;

[0087] S5. collecting and encoding and extracting high-dimensional uncertainty features involved in the first flexibility impact, the second flexibility impact, the third flexibility impact and the fourth flexibility impact caused by the operation and scheduling of the heating system;

[0088] S6. The optimization scheduling model problem for improving the flexibility of the heating system is defined as: considering the heat load supply and demand balance constraints of the heating system, the output constraints of the source-side heating units, the ramping constraints, the flexible heat load regulation capacity constraints, and the network operation constraints, with the goal of minimizing the total cost of the heating system, controlling and regulating the output of the source-side conventional heating units and renewable energy, the network structure layout and the flexible heat load regulation under multiple time scales, and responding to the impact of the source-grid load side and disturbance random factors;

[0089] S7. Convert the optimization scheduling model problem for improving the flexibility of the heating system into a Markov decision process, embed the high-dimensional features extracted by the encoding and design the set of states of the environment observed by the intelligent agent, the set of actions that affect the environment, and the set of benefits obtained after performing actions on the environment, and train and optimize the scheduling model through a deep reinforcement learning algorithm to output the optimal scheduling decision.

[0090] In this embodiment, S1 specifically includes:

[0091] When the renewable energy heating units including solar heating units and wind heating units are connected to the heating system, during the system scheduling process, the source side scheduling object is changed from the original non-renewable energy heating units with controllable output to a mixed heating system of non-renewable energy heating units and renewable energy heating units with increased randomness and uncertainty on the source side;

[0092] Among them, the output of renewable energy heating units is uncertain due to the influence of weather, season and geographical location. When the output is insufficient, it will cause insufficient heat supply to users, while when the output is excessive, it will cause resource waste and lead to imbalance between supply and demand.

[0093] The cost of connecting renewable energy heating units to the heating system is higher than that of non-renewable energy. In addition, due to the distribution characteristics of renewable energy heating units, the heat transmission cost and equipment operation and maintenance cost are increased. In order to compensate for the uncertainty of renewable energy output, the supply and demand balancing cost and standby unit cost are increased while meeting the safe operation constraints.

[0094] In this embodiment, S2 specifically includes:

[0095] Analysis: When flexible heat load users on the load side are connected to the heating system, they adjust their own heat load according to their own needs and heat load regulation capabilities; during peak hours, flexible heat load users reduce their own heat loads to reduce the peak load of the heating system, thus avoiding or reducing system crashes or performance degradation caused by excessive loads; reasonably dispatch flexible heat load users, and the heating system flexibly adjusts the supply and demand balance;

[0096] Obtain the heat usage data and behavior model of flexible heat load users, and optimize and adjust the heat load demand prediction model as the number of flexible heat load users increases;

[0097] Flexible heat load users are dispatched to reduce the waste of renewable energy caused by source-load output mismatch, and the flexible heat load acts as a buffer to adjust the operation of its own load balancing system when the renewable energy output fluctuates or the load-side demand changes, thereby improving the utilization efficiency of renewable energy.

[0098] In this embodiment, S3 specifically includes:

[0099] Analyze the impact of sudden changes in external temperature and severe weather on the heat demand of load-side users and the heating design capacity of the source-side heating units, resulting in supply-demand balance and damage, rupture and leakage of heating pipelines;

[0100] Analysis of heat network pipeline damage, rupture, leakage failures, resulting in heat loss, heating interruption, changes in the heating network heating path and pipeline topology, affecting the equipment operating status and operating parameters of water pumps, valves, heating units, and thermal power stations.

[0101] In this embodiment, S4 specifically includes:

[0102] Analysis: The expansion of the network-side node scale has increased the complexity of the network structure and topology of the heating system;

[0103] Analysis shows that the increase in access to heat sources on the source side and the increase in access to heat users on the load side has increased the flexibility and complexity of system operation scheduling, and increased the probability of mismatch between supply and demand.

[0104] In this embodiment, the analysis of the first flexibility impact on the system operation scheduling caused by the connection of the source-side renewable energy to the heating system also includes establishing an output prediction model for the renewable energy heating unit. The specific process is as follows:

[0105] Obtain real historical renewable energy heating unit output data, including environmental parameters, unit operating status, and time series data, and initialize the parameters of the generator and discriminator in the generative adversarial network;

[0106] Fixed discriminator, training generator: The generator takes random noise as input to generate synthetic data close to real data, and the discriminator judges the authenticity score of the data; the goal of the generator is to deceive the discriminator so that it cannot accurately judge the authenticity of the data; the parameters of the generator are updated through the back propagation algorithm;

[0107] Fix the generator and train the discriminator: After the generator training is completed, fix the parameters of the generator and train the discriminator; the goal of the discriminator is to determine whether the input data is real or generated; the parameters of the discriminator are updated through the back propagation algorithm;

[0108] Through cyclic training, the generator can generate synthetic data close to real data, and the discriminator cannot judge the authenticity of the data;

[0109] After the training is completed, the generator generates virtual output data of new energy heating units, which is then fused with the real historical output data of renewable energy heating units to train the recurrent neural network RNN ​​and establish an output prediction model for renewable energy heating units.

[0110] It should be noted that combining virtual data generated by the Generative Adversarial Network (GAN) with historical real data to train the output prediction model of renewable energy heating units can improve the accuracy of the prediction model by increasing the size and diversity of the data set. The following is the specific process:

[0111] 1) Data collection and preprocessing:

[0112] Collect historical real output data of new energy heating units, including time series data, environmental parameters (such as temperature, wind speed, etc.), unit operating status, etc.;

[0113] Preprocess the data, including data cleaning, normalization, missing value processing, etc., to ensure the quality and consistency of the data;

[0114] 2) Generate virtual data using GAN:

[0115] Using the generator and discriminator in GAN, a GAN model is trained based on historical real data;

[0116] The generator takes random noise as input and outputs simulated output data of renewable energy heating units;

[0117] The discriminator is responsible for distinguishing whether the input data is real or generated by the generator;

[0118] Through continuous iterative training, the generator can eventually generate virtual data that is close to the distribution of real data;

[0119] 3) Data Fusion:

[0120] Merge the generated virtual data with historical real data to form a larger and more diverse dataset;

[0121] When fusing data, you can consider mixing real data and virtual data according to a certain ratio or strategy to ensure the generalization ability of the model;

[0122] 4) Build a neural network prediction model:

[0123] Choose a neural network model suitable for time series forecasting, such as recurrent neural network (RNN), long short-term memory network (LSTM), or variational autoencoder (VAE);

[0124] Design the structure of the neural network, including the input layer, hidden layer, and output layer, according to the characteristics of the data and prediction requirements;

[0125] 5) Model training:

[0126] Use the fused data set (including real data and virtual data) to train the neural network prediction model;

[0127] Set appropriate loss functions and optimizers, as well as training iterations and hyperparameters;

[0128] The weights and biases of the neural network are continuously updated through the back-propagation algorithm to minimize the prediction error.

[0129] In this embodiment, the analysis of the second flexibility impact on the system operation scheduling caused by the load-side flexible heat load user connecting to the heating system also includes establishing a user heat load prediction model, and the specific process is as follows:

[0130] After obtaining the original load signal of the user including the flexible heat load, the EEMD algorithm is used to decompose it into high-frequency heat load component, low-frequency heat load component and residual component;

[0131] For the high-frequency heat load component, the XGBoost algorithm is used to extract features and perform nonlinear fitting processing to obtain the corresponding first heat load prediction sequence. For the low-frequency heat load component and the residual component, the SARIMA model is used to fit the heat load trend curve to obtain the corresponding second heat load prediction value.

[0132] Define the first heat load prediction sequence as a, the second heat load prediction sequence as b, and the corresponding actual heat load sequence as h. Assign weights w1 and w2 to the first heat load prediction sequence a and the second heat load prediction sequence b, respectively, w1+w2=1, and establish the error of the training heat load reconstruction prediction model as:

[0133] ε=h-w1a-w2b=h-w1a-(1-w1)b;

[0134] The weights are solved by minimizing the error of the reconstructed prediction model as the optimization goal, which is expressed as:

[0135]

[0136] The optimization objective function is set as:

[0137]

[0138] n is the number of heat load sampling points;

[0139] Set the constraints as:

[0140] 0≤w1≤1;

[0141] An intelligent optimization algorithm is used to optimize the weights, obtain the optimal weights w1 and w2, and establish a heat load reconstruction prediction model.

[0142] It should be noted that the specific implementation process of using the XGBoost algorithm to extract features and perform nonlinear fitting processing on the high-frequency heat load component to obtain the first heat load prediction sequence can be divided into the following steps:

[0143] 1) Feature Engineering:

[0144] Perform feature extraction to extract features that are helpful for predicting heat load from the original high-frequency heat load component data; this may include lagged values ​​of time series, rolling statistics (such as mean, standard deviation, etc.), periodic characteristics (such as daily, weekly, seasonal changes), and external influencing factors (such as weather, holidays, etc.);

[0145] Feature selection is also an important step, which can help reduce the dimension of features and improve the training efficiency and prediction performance of the model;

[0146] 2) Divide the data set:

[0147] The processed dataset is divided into a training set and a test set. The training set is used to train the XGBoost model, while the test set is used to evaluate the model's predictive performance;

[0148] Model training:

[0149] Use the XGBoost algorithm to train the model. During the training process, XGBoost automatically performs feature importance evaluation and nonlinear fitting. Adjust the model parameters, such as learning rate, maximum tree depth, subsample ratio, etc., to optimize the performance of the model;

[0150] 3) Nonlinear fitting:

[0151] XGBoost achieves nonlinear fitting by building multiple decision trees and combining them. Each decision tree tries to fit the residual of the previous tree, thereby gradually approaching the true heat load value;

[0152] By continuously adding new decision trees and adjusting their weights, XGBoost can handle complex nonlinear relationships and improve prediction accuracy;

[0153] 4) Prediction and evaluation:

[0154] Use the trained XGBoost model to predict the test set and obtain the corresponding first heat load prediction sequence;

[0155] To evaluate the predictive performance of the model, various evaluation indicators (such as mean square error, mean absolute error, R square value, etc.) can be used to quantify the predictive effect of the model.

[0156] It should be noted that for the low-frequency heat load component and residual component, the SARIMA (Seasonal Autoregressive Integrated Moving Average) model is a good choice because it can capture the seasonality, trend and randomness in time series data.

[0157] The process of fitting the heat load trend curve using the SARIMA model to obtain the second heat load prediction value is as follows:

[0158] 1) Model selection and fitting:

[0159] Select the SARIMA model: Select the appropriate SARIMA model based on the seasonality, trend and randomness characteristics of the data. The parameters of the SARIMA model include autoregressive term (AR), difference order (I), moving average term (MA), seasonal autoregressive term (SAR) and seasonal moving average term (SMA). Model fitting: Use the selected SARIMA model to fit the low-frequency heat load component and the residual component. This usually involves estimating the parameters of the model, such as autoregressive coefficients, moving average coefficients, etc.;

[0160] 2) Model Validation:

[0161] Model diagnosis: Check whether the residuals of the model meet the white noise assumption, that is, the residuals should be random, trendless and non-seasonal. Tools such as residual plots and autocorrelation plots can be used for diagnosis;

[0162] Cross-validation: Evaluate the predictive performance of the model using cross-validation methods such as rolling forecasts, K-fold cross-validation, etc. This helps avoid overfitting and provides a more accurate estimate of the model's generalization ability;

[0163] 3) Prediction and evaluation:

[0164] Prediction: Use the trained SARIMA model to predict the low-frequency heat load component and the residual component to obtain the corresponding second heat load prediction value;

[0165] Prediction evaluation: Use appropriate evaluation metrics (such as mean square error, mean absolute error, R-squared value, etc.) to quantify the accuracy of the prediction results;

[0166] It should be noted that the SARIMA model is very sensitive to the choice of parameters, so choosing appropriate parameters is crucial to the predictive performance of the model.

[0167] In this embodiment, S5 specifically includes:

[0168] A machine learning algorithm is used to collect and encode high-dimensional uncertainty features involved in the first flexibility impact, the second flexibility impact, the third flexibility impact and the fourth flexibility impact caused by the operation and scheduling of the heating system, and an encoding result characterizing the high uncertainty features of the heating system is obtained;

[0169] Among them, high-dimensional uncertainty characteristics include at least: uncertainty characteristics of heat source output on the source side; uncertainty characteristics of changes in heat user demand; uncertainty characteristics of weather changes in temperature, wind speed, rainfall and snowfall; uncertainty characteristics of events and locations of damage, rupture, leakage failures in the heat network pipeline; and uncertainty characteristics of changes in network topology layout.

[0170] It should be noted that the following are the collection and encoding extraction of high-dimensional uncertainty features involved in some influencing factors:

[0171] 1) Uncontrollable external extreme weather impact:

[0172] Uncertainty characteristics: uncertainty in weather changes, such as temperature, wind speed, snowfall, etc.;

[0173] Encoding extraction: Use time series data to model weather changes and extract key features such as temperature change rate, wind speed distribution, snowfall frequency, etc.

[0174] 2) Impact of changes in the network topology caused by heating network failure:

[0175] Uncertainty characteristics: uncertainty in the timing and location of events such as pipeline rupture and valve failure;

[0176] Coding extraction: record historical fault data, extract characteristics such as time, location, and frequency of fault occurrence, and encode fault modes using methods such as fault tree analysis;

[0177] 3) Impact of equipment operating status:

[0178] Uncertainty characteristics: uncertainty in equipment performance degradation and failure occurrence;

[0179] Code extraction: Collect equipment operation data, extract performance indicators, fault warning signals and other features, and use machine learning algorithms to predict and encode equipment status;

[0180] 4) Expansion of network-side node scale:

[0181] Uncertainty characteristics: the increase in network complexity caused by the increase in nodes and the uncertainty of traffic distribution;

[0182] Code extraction: Use network graph theory and other methods to model the relationship between nodes and edges, and extract network topology features, traffic distribution features, etc.

[0183] 5) Increase in access to heat sources on the source side:

[0184] Uncertainty characteristics: uncertainty in heat source type and output, fluctuations in energy prices;

[0185] Coding and extraction: Collect heat source data, extract characteristics such as heat source type, output range, energy price, etc., and use statistical methods or machine learning algorithms to predict and encode energy prices;

[0186] 6) Increase in the number of heat users on the load side:

[0187] Uncertainty characteristics: uncertainty in changes in user needs and diversity in user behavior patterns;

[0188] Coding extraction: Collect user usage data, extract characteristics such as user demand change trends and behavior patterns, and use data mining or machine learning technology to predict and classify user needs;

[0189] By collecting and encoding the high-dimensional uncertainty features of these influencing factors, we can establish a more accurate heating system operation scheduling model and improve the accuracy and reliability of scheduling decisions. At the same time, this also provides basic data support for the intelligent and automated scheduling of heating systems.

[0190] In this embodiment, in S7, the state set of the environment observed by the design agent includes: current heat load demand, renewable energy heating unit status, conventional heating unit status, pipe network operation status, pipe network topology structure and access volume scale;

[0191] The set of actions that the design exerts on the environment includes: adjusting the output of conventional heating units and renewable energy units, changing the layout of the pipe network structure, adjusting the flexible heat load and regulating the operating parameters of valves, water pumps, and heat exchanger units;

[0192] The benefit set obtained after the design performs actions on the environment includes: the total cost of the heating system composed of the original total cost and the subsequent optimization scheduling cost, and the supply-demand balance; the original total cost includes the system heating cost, the unit maintenance cost and the start-up and shutdown cost; the subsequent optimization scheduling cost includes the heat cost increased by the source-side optimization scheduling, the subsidy cost increased by the load-side flexible heat load optimization scheduling, and the cost of re-layout of the pipeline network structure; the supply-demand balance is the ratio of the total heat load demand on the load side to the total heat output of the source-side unit. If the ratio is within the preset threshold range, it indicates that the supply and demand are balanced, and a benefit is given; if the ratio is outside the preset threshold range, it indicates that the supply and demand are unbalanced, and a penalty is given;

[0193] The deep reinforcement learning algorithm is a TD3 algorithm based on a double replay buffer mechanism and a GRU neural network fusion; the double replay buffer mechanism stores experience data of different importance levels in categories according to the reward value, and during training, extracts some experience data of high importance from each replay buffer in order of probability from large to small, and extracts some experience data of low importance from each replay buffer in order of probability from small to large as samples; the reward value is positively correlated with the importance of the data; the Actor-Critic network in the TD3 algorithm adopts a fully connected network structure, and the state data processing process is integrated into the GRU network. The GRU network retains the meaningful historical state information of the previous moment and the current input state for calculation, and the retained state that needs to be remembered is passed to the last layer of the Actor-Critic network for processing.

[0194] It should be noted that converting the optimization scheduling problem of improving the flexibility of the heating system into a Markov decision process (MDP) is an effective method, which can use deep reinforcement learning algorithms to train and optimize the scheduling model. The following are the specific steps of the conversion process:

[0195] 1) Define the state set

[0196] State representation: The state set should contain all key information that may affect decision making, such as current heat load demand, renewable energy supply, status of conventional heating units, network operation status, etc.

[0197] High-dimensional feature embedding: High-dimensional features extracted using encoding techniques (such as autoencoders, convolutional neural networks, etc.) can be used as part of the state to capture the complex dynamics of the system;

[0198] 2) Define action set

[0199] Action definition: The action set should include all possible scheduling decisions, such as adjusting the output of conventional heating units and renewable energy units, changing the layout of the pipeline network, adjusting the flexible heat load, etc.

[0200] Action space: Each action should correspond to a specific scheduling instruction, forming a discrete or continuous action space;

[0201] 3) Define the income set

[0202] Revenue function design: The set of benefits should be defined based on the optimization objective (such as minimizing total cost). For example, a revenue function related to the cost reduction caused by the current scheduling decision can be defined;

[0203] Immediate and long-term benefits: The benefit function should be able to balance immediate and long-term benefits to encourage the agent to learn a scheduling strategy that meets current needs while taking into account future states.

[0204] 4) Constructing Markov decision process

[0205] State transition: Given the current state and action, define how the state transitions to the next state. This usually involves understanding and modeling the dynamics of the heating system;

[0206] Markov property: Assume that the future state is only related to the current state and has nothing to do with the past state. This is the core assumption of MDP;

[0207] 5) Designing intelligent agents

[0208] Observation state: The agent should be able to observe and understand the state of the current environment;

[0209] Select an action: Based on the observed state, the agent should be able to choose an action to perform according to a certain strategy (such as a greedy strategy);

[0210] Learning optimization: Through deep reinforcement learning algorithms (such as deep Q-networks, policy gradient methods, etc.), the agent should be able to learn a scheduling strategy that maximizes long-term benefits;

[0211] 6) Training optimization scheduling model

[0212] Environmental interaction: The agent interacts with the heating system environment, performs actions and observes the results, and collects experience data;

[0213] Model update: Use the collected experience data to update the agent's scheduling model and optimize its decision-making strategy;

[0214] Iterative optimization: Through multiple rounds of iterative training, the scheduling model is gradually optimized until convergence or reaching the preset training target;

[0215] 7) Output optimal scheduling decision

[0216] Final strategy: After training and optimization, the agent will learn an optimal scheduling strategy that can minimize the total cost of the heating system while satisfying various constraints;

[0217] Real-time decision-making: In actual operation, the agent will use the learned optimal strategy to output real-time scheduling decisions based on the real-time observed status;

[0218] In this way, we can use deep reinforcement learning algorithms to effectively solve the optimization scheduling problem of improving the flexibility of the heating system and output the optimal scheduling decision.

[0219] In this embodiment, the supply and demand flexibility between the source-side renewable energy heating unit and the load-side heat load is expressed as:

[0220]

[0221] are the upward and downward flexibility requirements of supply and demand in period t respectively; q u ,q d are the upward and downward adjustment flexibility coefficients caused by the output forecast error of the renewable energy heating unit; e u 、e d are the up and down adjustment flexibility coefficients caused by the heat load prediction error; P re,t+1 , P h,t+1 are the output forecast value and heat load forecast value of renewable energy heating units in period t+1 respectively; P h,t is the predicted value of heat load during period t+1; P re,max It is the maximum output of the renewable energy heating unit;

[0222] The optimization scheduling method further includes:

[0223] The flexibility of conventional thermal power heating units is analyzed and expressed as:

[0224]

[0225] They are respectively the upward and downward flexible supply capabilities of conventional thermal power heating units; are the upper and lower limits of the output of conventional thermal power heating unit m; u th,m,t is the operating status of the conventional thermal power heating unit m at time t. When its value is 0, it means it is shut down, and when it is 1, it means it is running; P th,m,t is the output of conventional thermal power heating unit m at time t; are the upward and downward climbing capabilities of the conventional thermal power heating unit m, respectively; Δt is the scheduling time interval;

[0226] The flexibility of the thermal storage device is analyzed and expressed as:

[0227]

[0228] They are the upward and downward flexible supply capabilities of the heat storage device; is the maximum heat storage power of the heat storage device o; P st,o,c,t is the heat storage power of heat storage device o at time t; is the maximum heat release power of the heat storage device o; P st,d is the heat release power of the heat storage device o at time t;

[0229] Calculate the operating cost of conventional thermal power heating units participating in peak load dispatching, expressed as:

[0230]

[0231] I th,m,t are the peak-shaving cost, start-stop cost, heat loss cost and peak-shaving compensation income of conventional thermal power heating unit m at time t; T is the total dispatching time; N th The number of conventional thermal power generating units;

[0232] Calculate the operating cost of the heat storage device participating in peak load scheduling, expressed as:

[0233]

[0234] C st,o,t I is the cost of the heat storage device participating in peak load regulation, including heat storage cost, heat release cost and loss cost; st,o,t Peak-shifting benefits for thermal storage devices;

[0235] Calculate the operating cost of the flexible heat load participating in peak load scheduling, expressed as:

[0236]

[0237] n is the number of times or time periods that the flexible heat load participates in peak regulation; C i,fh C is the cost of load reduction and transfer in the i-th peak load event; i,re is the cost incurred in response to the change in the behavior of the flexible heat load in the i-th peak load event; C i,de Cost of control equipment related to flexible heat load peak regulation; C i,ma Maintenance costs for peak load shaving equipment;

[0238] Furthermore, in the optimization scheduling model for improving the flexibility of the heating system, when the goal is to minimize the total cost of the heating system, in addition to considering the heat supply cost, unit maintenance cost, unit start-up and shutdown cost, external impact cost, node scale expansion cost, pipeline structure layout cost, and equipment failure recovery cost of the source-side renewable energy heating units and non-renewable energy heating units, it is also necessary to consider the operating cost of conventional thermal power heating units participating in peak-shaving scheduling, the operating cost of heat storage devices participating in peak-shaving scheduling, and the operating cost of flexible heat loads participating in peak-shaving scheduling.

[0239] In several embodiments provided in the present application, it should be understood that the disclosed system and method can also be implemented in other ways. The system embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and a part of the module, program segment or code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0240] In addition, the functional modules in each embodiment of the present invention can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part. If the function is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), disk or optical disk and other media that can store program codes.

[0241] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. An optimization scheduling method based on deep reinforcement learning and improving the flexibility of heating system, characterized in that: include: S1. Analyze the first flexibility impact on system operation and scheduling when renewable energy on the source side is connected to the heating system, including the impact of the randomness of renewable energy output on the balance of supply and demand and the impact of renewable energy access on operating costs; S2. Analyze the second flexibility impact on system operation and scheduling when users of load-side flexible heat loads are connected to the heating system, including the impact of flexible heat loads on supply and demand balance, the impact of flexible heat load access on heat load demand forecast, and the impact of the mismatch between source-load output and load-side demand on renewable energy utilization; S3. Analyze the impact of random disturbance factors on the third flexibility of the operation and scheduling of the heating system, including the impact of uncontrollable external extreme weather, the impact of changes in the pipe network topology caused by heating network failures, and the impact of equipment operation status; S4. Analyze the fourth flexibility impact on the operation and scheduling of the heating system caused by the expansion of the grid-side node scale, the increase in the access to heat sources on the source side, and the increase in the access to heat users on the load side; S5. collecting and encoding and extracting high-dimensional uncertainty features involved in the first flexibility impact, the second flexibility impact, the third flexibility impact and the fourth flexibility impact caused by the operation and scheduling of the heating system; S6. The optimization scheduling model problem for improving the flexibility of the heating system is defined as: considering the heat load supply and demand balance constraints of the heating system, the output constraints of the source-side heating units, the ramping constraints, the flexible heat load regulation capacity constraints, and the network operation constraints, with the goal of minimizing the total cost of the heating system, controlling and regulating the output of the source-side conventional heating units and renewable energy, the network structure layout and the flexible heat load regulation under multiple time scales, and responding to the impact of the source-grid load side and disturbance random factors; S7. Convert the optimization scheduling model problem for improving the flexibility of the heating system into a Markov decision process, embed the high-dimensional features extracted by the encoding and design the set of states of the environment observed by the intelligent agent, the set of actions that affect the environment, and the set of benefits obtained after performing actions on the environment, and train and optimize the scheduling model through a deep reinforcement learning algorithm to output the optimal scheduling decision.

2. The optimization scheduling method for improving the flexibility of a heating system according to claim 1, characterized in that: The S1 specifically includes: When the renewable energy heating units including solar heating units and wind heating units are connected to the heating system, during the system scheduling process, the source side scheduling object is changed from the original non-renewable energy heating units with controllable output to a mixed heating system of non-renewable energy heating units and renewable energy heating units with increased randomness and uncertainty on the source side; Among them, the output of renewable energy heating units is uncertain due to the influence of weather, season and geographical location. When the output is insufficient, it will cause insufficient heat supply to users, while when the output is excessive, it will cause resource waste and lead to imbalance between supply and demand. The cost of connecting renewable energy heating units to the heating system is higher than that of non-renewable energy. In addition, due to the distribution characteristics of renewable energy heating units, the heat transmission cost and equipment operation and maintenance cost are increased. In order to compensate for the uncertainty of renewable energy output, the supply and demand balancing cost and standby unit cost are increased while meeting the safe operation constraints.

3. The optimization scheduling method for improving the flexibility of a heating system according to claim 1, characterized in that: The S2 specifically includes: Analysis: When flexible heat load users on the load side are connected to the heating system, they adjust their own heat load according to their own needs and heat load regulation capabilities; during peak hours, flexible heat load users reduce their own heat loads to reduce the peak load of the heating system, thus avoiding or reducing system crashes or performance degradation caused by excessive loads; reasonably dispatch flexible heat load users, and the heating system flexibly adjusts the supply and demand balance; Obtain the heat usage data and behavior model of flexible heat load users, and optimize and adjust the heat load demand prediction model as the number of flexible heat load users increases; Flexible heat load users are dispatched to reduce the waste of renewable energy caused by source-load output mismatch, and the flexible heat load acts as a buffer to adjust the operation of its own load balancing system when the renewable energy output fluctuates or the load-side demand changes, thereby improving the utilization efficiency of renewable energy.

4. The optimization scheduling method for improving the flexibility of a heating system according to claim 1, characterized in that: The S3 specifically includes: Analyze the impact of sudden changes in external temperature and severe weather on the heat demand of load-side users and the heating design capacity of the source-side heating units, resulting in supply-demand balance and damage, rupture and leakage of heating pipelines; Analysis of heat network pipeline damage, rupture, leakage failures, resulting in heat loss, heating interruption, changes in the heating network heating path and pipeline topology, affecting the equipment operating status and operating parameters of water pumps, valves, heating units, and thermal power stations.

5. The optimization scheduling method for improving the flexibility of a heating system according to claim 1, characterized in that: The S4 specifically includes: Analysis: The expansion of the network-side node scale has increased the complexity of the network structure and topology of the heating system; Analysis shows that the increase in access to heat sources on the source side and the increase in access to heat users on the load side has increased the flexibility and complexity of system operation scheduling, and increased the probability of mismatch between supply and demand.

6. The optimization scheduling method for improving the flexibility of a heating system according to claim 1, characterized in that: The analysis of the first flexibility impact on the system operation and scheduling caused by the connection of the source-side renewable energy to the heating system also includes establishing an output prediction model for the renewable energy heating unit. The specific process is as follows: Obtain real historical renewable energy heating unit output data, including environmental parameters, unit operating status, and time series data, and initialize the parameters of the generator and discriminator in the generative adversarial network; Fixed discriminator, training generator: The generator takes random noise as input to generate synthetic data close to real data, and the discriminator judges the authenticity score of the data; the goal of the generator is to deceive the discriminator so that it cannot accurately judge the authenticity of the data; the parameters of the generator are updated through the back propagation algorithm; Fix the generator and train the discriminator: After the generator training is completed, fix the parameters of the generator and train the discriminator; the goal of the discriminator is to determine whether the input data is real or generated; the parameters of the discriminator are updated through the back propagation algorithm; Through cyclic training, the generator can generate synthetic data close to real data, and the discriminator cannot judge the authenticity of the data; After the training is completed, the generator generates virtual output data of new energy heating units, which is then fused with the real historical output data of renewable energy heating units to train the recurrent neural network RNN ​​and establish an output prediction model for renewable energy heating units.

7. The optimization scheduling method for improving the flexibility of a heating system according to claim 1, characterized in that: The analysis of the second flexibility impact on the system operation scheduling caused by the load-side flexible heat load user connecting to the heating system also includes establishing a user heat load prediction model, and the specific process is as follows: After obtaining the original load signal of the user including the flexible heat load, the EEMD algorithm is used to decompose it into high-frequency heat load component, low-frequency heat load component and residual component; For the high-frequency heat load component, the XGBoost algorithm is used to extract features and perform nonlinear fitting processing to obtain the corresponding first heat load prediction sequence. For the low-frequency heat load component and the residual component, the SARIMA model is used to fit the heat load trend curve to obtain the corresponding second heat load prediction value. Define the first heat load prediction sequence as a, the second heat load prediction sequence as b, and the corresponding actual heat load sequence as h. Assign weights w1 and w2 to the first heat load prediction sequence a and the second heat load prediction sequence b, respectively, w1+w2=1, and establish the error of the training heat load reconstruction prediction model as: ε=h-w1a-w2b=h-w1a-(1-w1)b; The weights are solved by minimizing the error of the reconstructed prediction model as the optimization goal, which is expressed as: The optimization objective function is set as: n is the number of heat load sampling points; Set the constraints as: 0≤w1≤1； An intelligent optimization algorithm is used to optimize the weights, obtain the optimal weights w1 and w2, and establish a heat load reconstruction prediction model.

8. The optimization scheduling method for improving the flexibility of a heating system according to claim 1, characterized in that: The S5 specifically includes: A machine learning algorithm is used to collect and encode high-dimensional uncertainty features involved in the first flexibility impact, the second flexibility impact, the third flexibility impact and the fourth flexibility impact caused by the operation and scheduling of the heating system, and an encoding result characterizing the high uncertainty features of the heating system is obtained; Among them, high-dimensional uncertainty characteristics include at least: uncertainty characteristics of heat source output on the source side; uncertainty characteristics of changes in heat user demand; uncertainty characteristics of weather changes in temperature, wind speed, rainfall and snowfall; uncertainty characteristics of events and locations of damage, rupture, leakage failures in the heat network pipeline; and uncertainty characteristics of changes in network topology layout.

9. The optimization scheduling method for improving the flexibility of a heating system according to claim 1, characterized in that: In S7, the state set of the environment observed by the design agent includes: current heat load demand, renewable energy heating unit status, conventional heating unit status, pipe network operation status, pipe network topology structure and access volume scale; The set of actions that the design exerts on the environment includes: adjusting the output of conventional heating units and renewable energy units, changing the layout of the pipe network structure, adjusting the flexible heat load and regulating the operating parameters of valves, water pumps, and heat exchanger units; The benefit set obtained after the design performs actions on the environment includes: the total cost of the heating system composed of the original total cost and the subsequent optimization scheduling cost, and the supply-demand balance; the original total cost includes the system heating cost, the unit maintenance cost and the start-up and shutdown cost; the subsequent optimization scheduling cost includes the heat cost increased by the source-side optimization scheduling, the subsidy cost increased by the load-side flexible heat load optimization scheduling, and the cost of re-layout of the pipeline network structure; the supply-demand balance is the ratio of the total heat load demand on the load side to the total heat output of the source-side unit. If the ratio is within the preset threshold range, it indicates that the supply and demand are balanced, and a benefit is given; if the ratio is outside the preset threshold range, it indicates that the supply and demand are unbalanced, and a penalty is given; The deep reinforcement learning algorithm is a TD3 algorithm based on a double replay buffer mechanism and a GRU neural network fusion; the double replay buffer mechanism stores experience data of different importance levels in categories according to the reward value, and during training, extracts some experience data of high importance from each replay buffer in order of probability from large to small, and extracts some experience data of low importance from each replay buffer in order of probability from small to large as samples; the reward value is positively correlated with the importance of the data; the Actor-Critic network in the TD3 algorithm adopts a fully connected network structure, and the state data processing process is integrated into the GRU network. The GRU network retains the meaningful historical state information of the previous moment and the current input state for calculation, and the retained state that needs to be remembered is passed to the last layer of the Actor-Critic network for processing.

10. The optimization scheduling method for improving the flexibility of a heating system according to claim 1, characterized in that: The supply and demand flexibility between the source-side renewable energy heating unit and the load-side heat load is expressed as: are the upward and downward flexibility requirements of supply and demand in period t respectively; q u ,q d are the upward and downward adjustment flexibility coefficients caused by the output forecast error of the renewable energy heating unit; e u 、e d are the up and down adjustment flexibility coefficients caused by the heat load prediction error; P re,t+1 , P h,t+1 are the output forecast value and heat load forecast value of renewable energy heating units in period t+1 respectively; P h,t is the predicted value of heat load during period t+1; P re,max It is the maximum output of the renewable energy heating unit; The optimization scheduling method further includes: The flexibility of conventional thermal power heating units is analyzed and expressed as: They are respectively the upward and downward flexible supply capabilities of conventional thermal power heating units; are the upper and lower limits of the output of conventional thermal power heating unit m; u th,m,t is the operating status of the conventional thermal power heating unit m at time t. When its value is 0, it means it is shut down, and when it is 1, it means it is running; P th,m,t is the output of conventional thermal power heating unit m at time t; are the upward and downward climbing capabilities of the conventional thermal power heating unit m, respectively; Δt is the scheduling time interval; The flexibility of the thermal storage device is analyzed and expressed as: They are the upward and downward flexible supply capabilities of the heat storage device; is the maximum heat storage power of the heat storage device o; P st,o,c,t is the heat storage power of heat storage device o at time t; is the maximum heat release power of the heat storage device o; P st,d is the heat release power of the heat storage device o at time t; Calculate the operating cost of conventional thermal power heating units participating in peak load dispatching, expressed as: I th,m,t are the peak-shaving cost, start-stop cost, heat loss cost and peak-shaving compensation income of conventional thermal power heating unit m at time t; T is the total dispatch time; N th The number of conventional thermal power generating units; Calculate the operating cost of the heat storage device participating in peak load scheduling, expressed as: C st,o,t I is the cost of the heat storage device participating in peak load regulation, including heat storage cost, heat release cost and loss cost; st,o,t Peak-shifting benefits for thermal storage devices; Calculate the operating cost of the flexible heat load participating in peak load scheduling, expressed as: n is the number of times or time periods that the flexible heat load participates in peak regulation; C i,fh C is the cost of load reduction and transfer in the i-th peak load event; i,re is the cost incurred in response to the change in the behavior of the flexible heat load in the i-th peak load event; C i,de Cost of control equipment related to flexible heat load peak regulation; C i,ma Maintenance costs for peak load shaving equipment; Furthermore, in the optimization scheduling model for improving the flexibility of the heating system, when the goal is to minimize the total cost of the heating system, in addition to considering the heat supply cost, unit maintenance cost, unit start-up and shutdown cost, external impact cost, node scale expansion cost, pipeline structure layout cost, and equipment failure recovery cost of the source-side renewable energy heating units and non-renewable energy heating units, it is also necessary to consider the operating cost of conventional thermal power heating units participating in peak-shaving scheduling, the operating cost of heat storage devices participating in peak-shaving scheduling, and the operating cost of flexible heat loads participating in peak-shaving scheduling.

Images