A heat station intelligent regulation and control method and system based on reinforcement learning

Abstract

The application discloses a kind of based on reinforcement learning's heat station intelligent regulation and control method and system, belong to the intelligent control field of heating system, to solve the existing heat station control method and system difficult to consider user comfort and system energy saving simultaneously, often appear heating shortage, excessive heating and the system fluctuation caused by frequent regulation and other problems, the method provided by the application comprises the following steps: building data acquisition module, building data preprocessing module, building feature engineering module, building dataset division module, building reinforcement learning modeling module, building reinforcement learning algorithm selection and network structure design module, building model training and deployment module, the system provided corresponds to the above method, this method and system implement intelligent regulation and control to heat station, realize user comfort and system energy efficiency dynamic balance, adapt to dynamic working condition, regulation and control fine smooth, guarantee system stable operation.

Description

Technical Field

[0001] This invention relates to the field of intelligent control of heating systems, and in particular to an intelligent control method and system for heating stations based on reinforcement learning. Background Technology

[0002] Existing control methods for urban centralized heating or district heating systems are based on adjusting the secondary water supply temperature at heating stations according to changes in outdoor temperature. Specifically, this can be done according to climate compensation curves or through manual adjustments by experienced personnel. However, due to differences in building thermal inertia, diverse types of heat dissipation terminals, random user heating behavior, and dynamic changes in meteorological conditions, this control method makes it difficult to simultaneously consider user comfort and system energy efficiency, often resulting in problems such as insufficient heating, excessive heating, and system fluctuations caused by frequent adjustments.

[0003] While some intelligent control methods or systems exist in the existing technology, such as the invention patent CN113110057A, which discloses an energy-saving control method and intelligent decision-making system for heating stations based on artificial intelligence, this patent emphasizes machine learning (ML) modeling to predict room temperature and reinforcement learning (RL) decision-making based on the predictive model, which belongs to the traditional approach of relying on supervised machine learning to build predictive models. Modeling through machine learning has the following drawbacks: no matter how accurate the model is, it cannot perfectly match the actual system; the dynamics and complexity of the real system cannot be fully captured, inevitably leading to model errors; model performance is entirely dependent on training data; missing data or imbalanced samples can lead to significant deviations in results, and the data itself may contain implicit biases, which the model amplifies; the model performs well within the distribution range of the training data, but its prediction accuracy drops significantly when faced with data outside this range.

[0004] The existing technology CN118519340B also proposes an intelligent heating regulation method based on deep reinforcement learning and adaptive control. Its state space consists of outdoor environmental data (temperature, humidity, wind speed, solar radiation) and heat load data (historical and real-time). It focuses on the historical patterns of heat load and outdoor weather, but does not consider indoor thermal comfort and energy consumption.

[0005] Therefore, existing technologies urgently need an adaptive heating station control method that can not only control the indoor temperature of users within a constrained range, but also reduce the energy consumption of the system, thereby improving the overall operating performance of the heating system. Summary of the Invention

[0006] To address the shortcomings and defects of existing technologies, this invention provides an intelligent control method for thermal power stations based on reinforcement learning, comprising the following steps: S1: Set up a data acquisition module. This module collects indoor temperature, outdoor meteorological parameters, heating system operation data and energy consumption data through detection devices set up at the heating station and user end. S2: Build a data preprocessing module, which performs missing value imputation and outlier processing on the data acquired by the data acquisition module; S3: Build a feature engineering module. This module normalizes and standardizes the data preprocessed by the data preprocessing module and constructs time-series features. The time-series features are used to characterize the thermal inertia of buildings and the hysteresis characteristics of heating networks. S4: Build a dataset partitioning module, which divides the historical data in the heating system operation data obtained by the data acquisition module into training set, validation set and test set; S5: Build a reinforcement learning modeling module. This module models the heating system regulation problem as a constrained Markov decision process. The state space includes current and trend quantities, the action space is the setpoint of the continuous secondary water supply temperature, and the reward function comprehensively considers user comfort and system energy efficiency. The current quantities in the state space in step S5 are: the average indoor temperature of the user, the secondary water supply temperature, the secondary return water temperature, the secondary water supply pressure, the secondary return water pressure, outdoor meteorological parameters, pump operating power, and time information. The trend quantity in the state space in step S5 is: data from the past M time points integrated based on the sliding window method.

[0007] The reward function in step S5 is:

[0008] in, Let be the reward value at time t; The average indoor temperature for users at time t; Let be the real-time operating power of the k-th pump at time t; This is the rated power of the pump; This represents the total number of pumps. and These are weighting coefficients, ranging from 0 to 1; and The weights for comfort constraints are between 0 and 1, where ; S6: Build a reinforcement learning algorithm selection and network structure design module. This module uses a policy optimization algorithm to train the policy network and the value network. The strategy network in step S6 takes the current and trend quantities in the state space as input and outputs the continuous secondary water supply temperature of the heating station. The input to the value network in step S6 is a state-action pair, which is used to estimate the state value or advantage function to guide the optimization of the policy network.

[0009] S7: Build a model training and deployment module. This module acquires data from the data acquisition module in real time and adjusts the secondary water supply temperature of the heating station in real time according to the trained strategy network to achieve a dynamic balance between user room temperature comfort and system energy efficiency. The specific steps of step S7 are as follows: The S71 first uses historical operating data to train the policy network and value network offline to obtain an initial policy. During the training process, the state-action samples are evaluated through a reward function to optimize system efficiency while maintaining user comfort. After the S72 is trained, the strategy is deployed to the real heating system to collect user indoor temperature, outdoor meteorological parameters, heating system operation data and energy consumption data in real time. Based on the strategy, the system outputs dynamic secondary water supply temperature of the heating station to achieve a dynamic balance between user comfort and system energy efficiency, and realizes online adaptive control.

[0010] Furthermore, the operating data of the heating system includes secondary water supply temperature, secondary return water temperature, secondary water supply pressure, secondary return water pressure, circulating pump power, and valve opening.

[0011] Furthermore, in step S2, the missing value filling of the data acquired by the data acquisition module is specifically performed by using a proportional interpolation method based on time sequence to fill in the missing values; the outlier handling is specifically based on the 3 sigma principle to identify outliers and replace them with the nearest neighbor valid observations.

[0012] Furthermore, in step S3, the data normalization and standardization specifically involve: normalizing the user's indoor temperature, outdoor meteorological parameters, heating system operation data, and energy consumption data to eliminate the influence of dimensional differences on the model; and constructing time-series features specifically involves: organizing the data in chronological order, extracting historical states using a sliding window, and characterizing the building's thermal inertia and the hysteresis characteristics of the pipeline network.

[0013] Furthermore, a constraint processing module is added to the output layer of the strategy network to impose boundary restrictions on the secondary water supply temperature action of the heating station, ensuring that the action does not exceed the allowable range of the equipment, while maintaining continuity and smoothness.

[0014] Furthermore, when the system detects that the user's indoor temperature exceeds the limit, the sensor is malfunctioning, or the strategy output is not executable, it automatically reverts to the traditional climate compensation curve or PID control strategy. After the strategy is corrected, it resumes reinforcement learning control.

[0015] This invention also provides an intelligent control system for heating stations based on reinforcement learning. The system includes a data acquisition module, a data preprocessing module, a feature engineering module, a dataset partitioning module, a reinforcement learning modeling module, a reinforcement learning algorithm selection and network structure design module, and a model training and deployment module. The system performs adaptive control of the secondary water supply temperature of the heating station according to any one of the above methods, thereby achieving a dynamic balance between user indoor temperature comfort and system energy efficiency.

[0016] The intelligent control method and system for thermal power stations based on reinforcement learning provided by this invention does not rely on traditional machine learning prediction models and is better able to adapt to dynamic and uncertain operating conditions. In terms of data processing, compared with the fuzzy data correspondence of existing technologies, a systematic scheme covering multi-source data acquisition, preprocessing, and time-series feature engineering is designed. The reward function design takes into account both user comfort and system energy efficiency, which is superior to the single room temperature difference reciprocal design in existing technologies and also differs from the orientation that focuses on heat supply and demand balance, achieving unified optimization of comfort and energy consumption. The action control adopts a continuous secondary water supply temperature setting within a range of ±3℃, which is more precise in control, smooth in operation, and practically feasible than the heat load distribution scheme of existing technologies. At the same time, the Actor-Critic network combined with a time-series network (LSTM / Transformer / TCN) is explicitly used to accurately model thermal inertia and pipeline hysteresis, further ensuring the adaptive capability and stability of the control, and ultimately achieving accurate adaptation of the secondary water supply temperature under dynamic operating conditions, efficiently balancing user indoor comfort and system energy efficiency. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the specific embodiments of the present invention, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below.

[0018] Figure 1 This is a schematic diagram of the method flow according to a first aspect embodiment of the present invention. Detailed Implementation

[0019] The structure and operation of this invention will be further described in detail below with reference to the accompanying drawings. Obviously, the drawings are provided only for a better understanding of this invention and should not be construed as limiting it. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0020] Example 1 Combination Figure 1 As shown in the figure, this embodiment discloses an intelligent control method for thermal power stations based on reinforcement learning, including the following steps: S1: Set up a data acquisition module. This module collects indoor temperature, outdoor meteorological parameters, heating system operation data and energy consumption data through detection devices set up at the heating station and user end. S2: Build a data preprocessing module, which performs missing value imputation and outlier processing on the data acquired by the data acquisition module; S3: Build a feature engineering module. This module normalizes and standardizes the data preprocessed by the data preprocessing module and constructs time-series features. The time-series features are used to characterize the thermal inertia of buildings and the hysteresis characteristics of heating networks. S4: Build a dataset partitioning module, which divides the historical data in the heating system operation data obtained by the data acquisition module into training set, validation set and test set; S5: Build a reinforcement learning modeling module. This module models the heating system regulation problem as a constrained Markov decision process. The state space includes current and trend quantities, the action space is the setpoint of the continuous secondary water supply temperature, and the reward function comprehensively considers user comfort and system energy efficiency. S6: Build a reinforcement learning algorithm selection and network structure design module. This module uses a policy optimization algorithm to train the policy network and the value network. S7: Build a model training and deployment module. This module acquires data from the data acquisition module in real time and adjusts the secondary water supply temperature of the heating station in real time according to the trained strategy network to achieve a dynamic balance between user room temperature comfort and system energy efficiency.

[0021] Specifically, the operating data of the heating system includes secondary water supply temperature, secondary return water temperature, secondary water supply pressure, secondary return water pressure, circulating pump power, and valve opening.

[0022] Specifically, in step S2, missing value imputation of the data acquired by the data acquisition module is performed by using a proportional interpolation method based on time sequence to fill in the missing values; the outlier handling is specifically based on the 3 sigma principle to identify outliers and replace them with the nearest neighbor valid observations.

[0023] Specifically, in step S3, the data normalization and standardization are performed as follows: the user's indoor temperature, outdoor meteorological parameters, heating system operation data and energy consumption data are normalized to eliminate the influence of dimensional differences on the model; the construction of time series features is performed as follows: the data is organized in chronological order, historical states are extracted using a sliding window, and the building's thermal inertia and pipeline lag characteristics are characterized.

[0024] Specifically, the current quantities in the state space in step S5 are: the average indoor temperature of the user, the secondary water supply temperature, the secondary return water temperature, the secondary water supply pressure, the secondary return water pressure, outdoor meteorological parameters, pump operating power, and time information. The trend quantity in the state space in step S5 is: data from the past M time points integrated based on the sliding window method.

[0025] Specifically, the reward function in step S5 is:

[0026] in, for The reward value at any given moment; for Average indoor temperature for users at any given time; for The first moment Real-time operating power of each pump; This is the rated power of the pump; This represents the total number of pumps. and These are weighting coefficients, ranging from 0 to 1; and The weights for comfort constraints are between 0 and 1, where .

[0027] Specifically, in step S6, the strategy network takes the current and trend quantities in the state space as input and outputs the continuous secondary water supply temperature of the heating station. The input to the value network in step S6 is a state-action pair, which is used to estimate the state value or advantage function to guide the optimization of the policy network.

[0028] Specifically, a constraint processing module is added to the output layer of the strategy network to impose boundary restrictions on the secondary water supply temperature action of the heating station, ensuring that the action does not exceed the allowable range of the equipment, while maintaining continuity and smoothness.

[0029] Specifically, the steps of step S7 are as follows: The S71 first uses historical operating data to train the policy network and value network offline to obtain an initial policy. During the training process, the state-action samples are evaluated through a reward function to optimize system efficiency while maintaining user comfort. After the S72 is trained, the strategy is deployed to the real heating system to collect user indoor temperature, outdoor meteorological parameters, heating system operation data and energy consumption data in real time. Based on the strategy, the system outputs dynamic secondary water supply temperature of the heating station to achieve a dynamic balance between user comfort and system energy efficiency, and realizes online adaptive control.

[0030] Specifically, when the indoor temperature exceeds the limit, the sensor malfunctions, or the strategy outputs an unexecutable action, it automatically reverts to the traditional climate compensation curve or PID control strategy, and resumes reinforcement learning control after the strategy is corrected.

[0031] Taking a municipal centralized heating system as an example, the system consists of a heat source, a primary pipe network, heating stations, and a secondary pipe network, providing stable heat to multiple residential communities. To obtain the system's operational characteristics, operational data was collected from multiple heating stations during the heating season. Collected parameters included indoor user temperature, outdoor meteorological data (air temperature, humidity, wind speed, etc.), supply and return water temperature and pressure, circulating pump power, and related energy consumption data. The data sampling period was set to 5 minutes, and data was collected continuously for one heating season to form a complete historical operational dataset.

[0032] The collected data is transmitted to the central control server via the data acquisition module. Due to data gaps and abnormal fluctuations during actual operation, the data preprocessing module fills in missing values ​​and corrects outliers. Subsequently, the feature engineering module normalizes and standardizes features such as room temperature, meteorological parameters, heating system operation volume, and pump power. Considering the building's thermal inertia and the hysteresis characteristics of the pipe network, this embodiment adopts a sliding window method with a window length set to M=12, integrating the historical states of the past hour to construct the time-series feature input.

[0033] In the dataset partitioning module, historical data is divided into training, validation, and test sets in a 7:2:1 ratio for model training, parameter tuning, and performance evaluation.

[0034] In the reinforcement learning modeling module, the temperature control of the secondary water supply at the heating station is modeled as a constrained Markov decision process (CMDP). Its state space includes: Current parameters: indoor average temperature, secondary supply and return water temperature and pressure, pump power, outdoor meteorological parameters and time information; Trend quantity: It consists of the state sequence of the previous M=12 time points and is used to characterize the dynamic trend.

[0035] The action space is a continuous variable, corresponding to the secondary water supply temperature setpoint. To ensure control smoothness and executability, the action range is limited to ±3℃ of the previous setpoint.

[0036] The reward function is designed as follows:

[0037] The parameters can be set as follows: α =0.4, β =0.6, w 1 = 0.6 w2 = 0.4. This design ensures that the user's room temperature is maintained within the range of 18~22℃, while suppressing excessive energy consumption of the circulation pump.

[0038] In the reinforcement learning algorithm selection and network design module, this embodiment employs the proximal policy optimization (PPO) algorithm based on policy gradients. The constructed policy network (Actor) is a deep neural network model, whose input is a state vector (including room temperature, supply and return water temperatures, pump power, outdoor meteorological parameters, time information, and historical time series features), and whose output is continuous secondary water supply temperature actions. The value network (Critic) takes state-action pairs as input and outputs the corresponding state value, which is used to guide the optimization of the policy network. To better characterize building thermal inertia and pipeline network dynamics, this embodiment uses a Transformer structure for encoding in the temporal feature extraction, making full use of its self-attention mechanism to capture long-term dependencies.

[0039] During the model training phase, offline training was conducted using historical data from the entire heating season, and hyperparameters were tuned using a validation set. Fine-tuning methods were combined to improve the model's generalization ability and adaptability.

[0040] During the deployment phase, the heating station collects indoor temperature, system operating data, and meteorological parameters in real time, and inputs this information into a trained strategy network to dynamically output secondary water supply temperature control commands. For example, when the outdoor temperature drops sharply by 5°C and the user's indoor temperature falls below 18°C, the strategy network automatically adjusts the secondary water supply temperature from 45°C to 47°C to quickly improve indoor comfort. When a high circulation pump load is detected, the system appropriately lowers the secondary water supply temperature setting to reduce pump power consumption. Through this method, the system can achieve a dynamic balance between user comfort and energy efficiency under different operating conditions.

[0041] Compared to some existing technologies, this invention achieves superior technical results. For example, invention patent CN113110057A discloses an energy-saving control method and intelligent decision-making system for heating stations based on artificial intelligence. This patent emphasizes machine learning (ML) modeling to predict room temperature + reinforcement learning (RL) decision-making based on the prediction model. That is, the prediction model is the environment, and the RL decision is the controllable quantity, which is a combination of "prediction model + decision model". It only mentions obtaining the correspondence between room temperature tn and the controllable quantity tc of the heating station, without giving specific data processing methods. The reward function is the reciprocal of the absolute value of the difference between room temperature and target temperature. The action space is the controllable quantity tc of the heating station (multiple physical quantities on the primary / secondary side, including temperature, pressure, flow rate, valve position, pump frequency, etc., or one or more of them), and the network structure is not limited. This invention directly models the heating system regulation problem as a Constrained Markov Decision Process (CMDP) in reinforcement learning. Unlike traditional methods that rely on supervised machine learning to build predictive models, this invention does not use traditional machine learning prediction models as the environment. Instead, it directly drives the reinforcement learning process through real-time collected operational data and the constructed state space, utilizing reward signals from environmental feedback for end-to-end intelligent control. This avoids problems associated with machine learning modeling, such as models not perfectly matching the actual system, dependence on and bias of modeling data, significantly reduced prediction accuracy for situations outside the training data, and insufficient robustness. In this way, the system can adaptively adjust the secondary water supply temperature under dynamic and uncertain operating conditions, achieving a unified optimization of user indoor comfort and system energy efficiency. This invention designs a data acquisition module (indoor temperature, outdoor weather, system operating data, energy consumption), a data preprocessing module (missing values, outliers), and a feature engineering module (normalization, temporal features, thermal inertia, and network hysteresis), thus making the solution more systematic. In particular, it introduces temporal features, which existing technologies do not utilize. The reward function of this invention is explicitly defined, integrating user comfort (penalty for room temperature deviation of 18–22℃, with α and β weights) and system energy efficiency (pump power normalized energy consumption penalty), and weighted synthesis. The action space is the continuous secondary water supply temperature, with a limit range of ±3℃, which is a refined single action control, emphasizing smoothness and executability. It proposes an Actor-Critic policy network and value network, and explicitly uses a time-series network (LSTM / Transformer / TCN) to model thermal inertia and hysteresis, which greatly improves the control accuracy.

[0042] Existing technology CN118519340B also proposes an intelligent heating regulation method based on deep reinforcement learning and adaptive control. Its state space consists of outdoor environmental data (temperature, humidity, wind speed, solar radiation) and heat load data (historical and real-time). It explicitly proposes generating state feature vectors according to time periods (data of the same time period in the previous n years). The state space of this patent includes current quantities (indoor temperature, secondary supply and return water temperature / pressure, pump power, outdoor meteorological parameters, time information) and trend quantities (historical data of sliding window).

[0043] Existing technologies use outdoor environmental data and historical heat load data, while this patent uses indoor temperature data, outdoor meteorological data, system operation data, and historical trend data. Notably, existing technologies do not use indoor temperature data. Therefore, this patent prioritizes indoor comfort, while existing technologies prioritize historical patterns of heat load and outdoor weather. The reward function of existing technologies is constructed based on the difference between the heat load supply and demand of all heating stations and the heat generation and heat loss of the power plant, emphasizing heat supply and demand balance and energy efficiency. This patent's reward function integrates indoor temperature and the energy consumption of heating stations, prioritizing user comfort and energy consumption optimization. The action set of existing technologies is not a direct continuous value, but a discrete set of actions generated by multiplying the historical heat load average EAi by an arithmetic sequence Q. Essentially, it calculates a set of candidate heat load allocation schemes for each time period, from which a reinforcement learning algorithm selects. In contrast, the actions of this patent are continuous secondary water supply temperature setpoints, limited to ±3℃, which improves the smoothness and executability of control.

[0044] Example 2 This embodiment discloses an intelligent control system for a heating station based on reinforcement learning. The system includes a data acquisition module, a data preprocessing module, a feature engineering module, a dataset partitioning module, a reinforcement learning modeling module, a reinforcement learning algorithm selection and network structure design module, and a model training and deployment module. The system performs adaptive control of the secondary water supply temperature of the heating station according to any one of the methods in Embodiment 1, so as to achieve a dynamic balance between the user's indoor temperature comfort and the system's energy efficiency.

[0045] The above embodiments are only used to illustrate the present invention patent. The structure, connection method and manufacturing process of each component can be varied. Any equivalent transformations and improvements made on the basis of this technical solution should not be excluded from the protection scope of the present invention patent.

Claims

1. A method for intelligent control of thermal power stations based on reinforcement learning, characterized in that: Includes the following steps: S1: Set up a data acquisition module. This module collects indoor temperature, outdoor meteorological parameters, heating system operation data and energy consumption data through detection devices set up at the heating station and user end. S2: Build a data preprocessing module, which performs missing value imputation and outlier processing on the data acquired by the data acquisition module; S3: Build a feature engineering module. This module normalizes and standardizes the data preprocessed by the data preprocessing module and constructs time-series features. The time-series features are used to characterize the thermal inertia of buildings and the hysteresis characteristics of heating networks. S4: Build a dataset partitioning module, which divides the historical data in the heating system operation data obtained by the data acquisition module into training set, validation set and test set; S5: Build a reinforcement learning modeling module. This module models the heating system regulation problem as a constrained Markov decision process. The state space includes current and trend quantities, the action space is the setpoint of the continuous secondary water supply temperature, and the reward function comprehensively considers user comfort and system energy efficiency. The current quantities in the state space in step S5 are: the average indoor temperature of the user, the secondary water supply temperature, the secondary return water temperature, the secondary water supply pressure, the secondary return water pressure, outdoor meteorological parameters, pump operating power, and time information. The trend quantity in the state space in step S5 is: data from the past M time points integrated based on the sliding window method; The reward function in step S5 is: in, Let be the reward value at time t; The average indoor temperature for users at time t; Let be the real-time operating power of the k-th pump at time t; This is the rated power of the pump; This represents the total number of pumps. and These are weighting coefficients, ranging from 0 to 1; and The weights for comfort constraints are between 0 and 1, where ; S6: Build a reinforcement learning algorithm selection and network structure design module. This module uses a policy optimization algorithm to train the policy network and the value network. The strategy network in step S6 takes the current and trend quantities in the state space as input and outputs the continuous secondary water supply temperature of the heating station. The input to the value network in step S6 is a state-action pair, which is used to estimate the state value or advantage function to guide the optimization of the policy network. S7: Build a model training and deployment module. This module acquires data from the data acquisition module in real time and adjusts the secondary water supply temperature of the heating station in real time according to the trained strategy network to achieve a dynamic balance between user room temperature comfort and system energy efficiency. The specific steps of step S7 are as follows: S71: First, the policy network and value network are trained offline using historical operating data to obtain an initial policy; during the training process, the state-action samples are evaluated through a reward function to optimize system efficiency while maintaining user comfort. S72: After training, the strategy is deployed to the real heating system to collect user indoor temperature, outdoor meteorological parameters, heating system operation data and energy consumption data in real time, and output dynamic secondary water supply temperature of the heating station according to the strategy to achieve a dynamic balance between user comfort and system energy efficiency, and realize online adaptive control.

2. The method according to claim 1, characterized in that: The operating data of the heating system includes secondary water supply temperature, secondary return water temperature, secondary water supply pressure, secondary return water pressure, circulating pump power, and valve opening.

3. The method according to claim 1, characterized in that: In step S2, missing value imputation of the data acquired by the data acquisition module is specifically performed by using a proportional interpolation method based on time sequence to fill in the missing values; the outlier handling is specifically based on the 3 sigma principle to identify outliers and replace them with the nearest neighbor valid observations.

4. The method according to claim 1, characterized in that: In step S3, the data normalization and standardization specifically involve: normalizing user indoor temperature, outdoor meteorological parameters, heating system operation data, and energy consumption data to eliminate the influence of dimensional differences on the model; and constructing time-series features specifically involves: organizing data in chronological order, extracting historical states using a sliding window, and characterizing building thermal inertia and pipeline lag characteristics.

5. The method according to claim 1, characterized in that: A constraint processing module is added to the output layer of the strategy network to impose boundary restrictions on the temperature action of the secondary water supply in the heating station, ensuring that the action does not exceed the allowable range of the equipment, while maintaining continuity and smoothness.

6. The method according to claim 1, characterized in that: When the system detects that the indoor temperature exceeds the limit, the sensor is malfunctioning, or the strategy output is not executable, it automatically reverts to the traditional climate compensation curve or PID control strategy. After the strategy is corrected, it resumes reinforcement learning control.

7. A thermal power station intelligent control system based on reinforcement learning, characterized in that: The system includes a data acquisition module, a data preprocessing module, a feature engineering module, a dataset partitioning module, a reinforcement learning modeling module, a reinforcement learning algorithm selection and network structure design module, and a model training and deployment module. The system performs adaptive regulation of the secondary water supply temperature of the heating station according to any one of claims 1 to 6, so as to achieve a dynamic balance between the user's indoor temperature comfort and the system's energy efficiency.

Images