A reinforcement learning variable load pressure control method based on enthalpy calculation

Abstract

The application relates to an enthalpy calculation-based reinforcement learning variable-load pressure control method and system, a computer device and a computer readable storage medium. Through the use of the self-adaptive capability of reinforcement learning, the control strategy is continuously learned and optimized in the operation of the unit, and a neural network model is introduced to predict the load and system pressure characteristic changes in advance, thereby solving the problems of insufficient control precision and large pressure fluctuation caused by poor adaptability and lack of foresight of the traditional fixed model under the variable-load working condition, realizing the real-time adaptation to dynamic factors such as coal quality changes, start-stop grinding disturbances, continuous variable-load conditions and equipment aging, improving the response speed of the unit to rapid variable load, reducing pressure fluctuation, reducing energy loss and equipment wear, and improving the overall operation efficiency and reliability of the unit.

Description

Technical Field

[0001] This application relates to the field of thermal power generation technology, and in particular to a reinforcement learning-based variable load pressure control method based on enthalpy calculation. Background Technology

[0002] Current variable load pressure control technology mainly relies on traditional control strategies, such as proportional-integral-derivative (PID) control based on a fixed model.

[0003] However, these methods have significant limitations. On the one hand, the operation of thermal power units is extremely complex, and their dynamic characteristics under varying load conditions change with various factors, such as coal quality changes, mill start-up and shutdown disturbances, and equipment aging. Traditional fixed models struggle to accurately adapt to these changes, resulting in insufficient control precision. For example, when coal quality changes, control parameters based on the original model cannot be adjusted in a timely manner, leading to significant fluctuations in main steam pressure. On the other hand, existing technologies have weak predictive capabilities for future load changes, relying primarily on current or recent operating data for control decisions, lacking foresight. When faced with rapid load changes, they cannot make effective adjustments in advance, thus affecting the unit's stability and response speed, and increasing energy consumption and equipment wear risks.

[0004] Currently, no effective solution has been proposed to address the problems of traditional fixed models in related technologies, which are unable to adapt to dynamic changes and lack forward-looking control capabilities, resulting in insufficient control accuracy, large pressure fluctuations, and poor stability. Summary of the Invention

[0005] This application provides a reinforcement learning-based variable load pressure control method, system, computer device, and computer-readable storage medium based on enthalpy calculation, to at least solve the problems of insufficient accuracy and poor stability in generator set control methods in related technologies.

[0006] In a first aspect, embodiments of this application provide a reinforcement learning-based variable load pressure control method based on enthalpy calculation, the method comprising:

[0007] Using the generator set load command, coal mill feed rate, water feed rate and main steam regulating valve opening as model inputs, and the main steam temperature, main steam pressure and actual unit load as model outputs, a neural network model is constructed to predict the changes in system pressure characteristics under different load segments of the generator set.

[0008] Historical operating data of the generator set is acquired. Under the reinforcement learning algorithm environment, the scheduling command for controlling the generator set is taken as the decision action of the agent. The prediction result obtained by the neural network model based on the decision action is taken as the observation value. The optimization objective is to maximize the score of the preset reward function. The policy network of the reinforcement learning agent is trained to obtain a reinforcement learning model for pressure control under variable load conditions.

[0009] Based on the dead zone data of the generator set, an expert system is configured to correct the bias. The reinforcement learning model is combined with the expert system for real-time control of coal and water supply under operating conditions.

[0010] In some embodiments, constructing a neural network model for predicting changes in system pressure characteristics under different load segments of the generator set includes:

[0011] Build a progressive network architecture that includes an input layer, a hidden layer, and an output layer;

[0012] The input layer is divided into multiple sub-layers based on the physical characteristics and interrelationships of the model input. The sub-layers include: an energy input sub-layer for receiving the coal feed from the coal mill, a water input sub-layer for receiving the water feed, and a steam regulation sub-layer for receiving the main steam valve opening data.

[0013] The hidden layer includes a shallow hidden layer and a deep hidden layer. The shallow hidden layer extracts features from the model input to obtain basic features, and the deep hidden layer integrates and abstracts the basic features to obtain deep fused features. Skip connections are introduced between multiple hidden layers to pass information from previous hidden layers to the deep hidden layer.

[0014] In some embodiments, the neural network model is trained using an adaptive dynamic training optimization method based on preprocessed historical running data, specifically including:

[0015] An adaptive learning rate strategy is adopted during training, and the learning rate is dynamically adjusted according to the performance of the neural network model on the validation set.

[0016] A dynamic sample weighting mechanism based on operating conditions is introduced, which assigns different weight coefficients to data samples under different operating conditions according to the importance and frequency of data samples in actual operation.

[0017] In some embodiments, the agent includes an architecture with hierarchical decision-making modules, wherein:

[0018] The lowest-level module processes and analyzes the direct observations output from the neural network model to obtain real-time status assessment results characterizing the current physical condition of the generator set. The direct observations include: main steam temperature, main steam pressure, and actual load of the unit.

[0019] Based on the real-time status assessment results, the intermediate layer module performs preliminary strategy planning by combining the current unit load command, coal mill feed rate, water feed rate and main steam valve opening, and outputs short-term control strategies to coordinate the various execution units.

[0020] The top-level module adjusts and optimizes the short-term control strategy based on the overall operating status and long-term goals of the generator set, and generates and outputs the decision-making actions.

[0021] In some embodiments, the method further includes:

[0022] The current main steam temperature, main steam pressure, and actual unit load are defined as the initial observation values ​​of the observed values.

[0023] The rate of change of the main steam pressure over a preset time period is defined as an observation parameter. The observation value is constructed by combining the initial observation value and the observation parameter.

[0024] In some embodiments, the preset reward function is configured based on the proximity of the main steam pressure to the steam setpoint, the stability of pressure changes, and the comprehensive influence parameters of the generator set operating indicators.

[0025] Specifically, when the main steam pressure approaches the set value in steady state during the load change process, the main steam temperature remains within the preset temperature range, and the actual load change of the generator set is within the preset compliance range, a corresponding reward value is configured.

[0026] If the main steam pressure fluctuation is within the preset risk fluctuation range, and the key parameters of generator set 1 exceed the preset safety range due to pressure control, a corresponding penalty value is configured.

[0027] In some embodiments, the method further includes, during the training of the policy network for the reinforcement learning agent:

[0028] An experience-based training strategy is adopted to store the experience generated by the agent's interaction with the reinforcement learning algorithm environment in an experience replay buffer.

[0029] The neural network model is trained by extracting target experience data from the experience replay buffer, wherein the experience data includes observations, decision actions, reward values ​​obtained for the decision actions, and observations at the next time step.

[0030] Among them, priority information is configured for each experience data based on the reward value obtained in each experience data session;

[0031] When extracting experience from the experience playback buffer, the experience data is extracted according to the priority information.

[0032] In some embodiments, data processing of the historical operational data includes:

[0033] The historical running data is smoothed using Savitzky-Golay filtering, wherein the Savitzky-Golay filtering is based on least squares fitting.

[0034] Furthermore, based on the mean and standard deviation of the historical operating data, the filtered historical operating data is subjected to Z-score normalization processing to map data of different magnitudes and distribution ranges to a unified interval, thereby obtaining preprocessed historical operating data.

[0035] In some embodiments, correcting the bias using an expert system includes:

[0036] Real-time monitoring of the outlet differential pressure of the coal mill and the oxygen content of the unit;

[0037] When the differential pressure at the outlet of the coal mill increases and the oxygen content decreases, the coal reduction and water addition actions are performed according to the preset mapping relationship. The preset mapping relationship is as follows: when the oxygen content decreases by 0.5%, 1%, 1.5%, and 2%, it corresponds to a coal reduction of 3t / h, 5t / h, 7t / h, and 9t / h, respectively, and a water increase of 15t / h, 25t / h, 35t / h, and 45t / h.

[0038] Secondly, embodiments of this application provide a reinforcement learning-based variable load pressure control system based on enthalpy calculation, the system comprising:

[0039] The module is used to construct a neural network model for predicting the changes in system pressure characteristics under different load segments of the generator set, with the generator set load command, coal mill feed rate, water feed rate and main steam valve opening as model inputs, and the main steam temperature, main steam pressure and actual unit load as model outputs.

[0040] The training module is used to acquire historical operating data of the generator set. Under the reinforcement learning algorithm environment, the scheduling command for controlling the generator set is used as the decision action of the agent, and the prediction result obtained by the neural network model based on the decision action is used as the observation value. The optimization objective is to maximize the score of the preset reward function. The policy network of the reinforcement learning agent is trained to obtain a reinforcement learning model for pressure control under variable load conditions.

[0041] The control module is used to configure an expert system for bias correction based on the dead zone data of the generator set, and to combine the reinforcement learning model with the expert system for real-time control of coal and water supply under operating conditions.

[0042] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0043] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect above.

[0044] Compared to related technologies, the present application provides a reinforcement learning-based variable load pressure control method, system, computer equipment, and computer-readable storage medium based on enthalpy calculation. By utilizing the adaptive capabilities of reinforcement learning, the control strategy is continuously learned and optimized during unit operation. A neural network model is introduced to predict changes in load and system pressure characteristics in advance, solving the problems of insufficient control accuracy and large pressure fluctuations caused by poor adaptability and lack of foresight in traditional fixed models under variable load conditions. This enables real-time adaptation to the effects of dynamic factors such as coal quality changes, mill start-up and shutdown disturbances, continuous load changes, and equipment aging. The technical effects include improving the unit's response speed to rapid load changes, reducing pressure fluctuations, reducing energy consumption and equipment wear, and improving the overall operating efficiency and reliability of the unit. Attached Figure Description

[0045] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0046] Figure 1 This is a flowchart of a reinforcement learning-based variable load pressure control method based on enthalpy calculation, according to an embodiment of the present invention.

[0047] Figure 2 This is another flowchart of reinforcement learning variable load pressure control based on enthalpy calculation according to an embodiment of this application;

[0048] Figure 3 This is a structural block diagram of a reinforcement learning variable load pressure control system based on enthalpy calculation according to an embodiment of this application;

[0049] Figure 4 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0051] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0052] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0053] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0054] Current variable load pressure control technology primarily relies on traditional PID control. However, the operation of thermal power units is extremely complex. The dynamic characteristics under variable load conditions change with factors such as coal quality variations, mill start-up and shutdown disturbances, and equipment aging. Traditional fixed models struggle to adapt accurately, leading to insufficient control precision. For example, changes in coal quality can cause significant fluctuations in main steam pressure. Furthermore, existing technologies have weak predictive capabilities for future load changes, lacking foresight and impacting unit stability and response speed, while increasing energy consumption and equipment wear risks.

[0055] In view of this, embodiments of this application provide a reinforcement learning-based variable load pressure control method based on enthalpy calculation. Figure 1 This is a flowchart of a reinforcement learning-based variable load pressure control method based on enthalpy calculation, according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0056] S101, Data reception and processing;

[0057] Collect historical operating data of the unit, including parameters such as unit load command, coal feed rate of each coal mill, water feed rate, main steam valve opening, actual unit load, main steam pressure, main steam temperature, water-cooled wall outlet temperature, and superheater air temperature.

[0058] The collected raw data undergoes data processing to remove industrial noise and outlier values. This step specifically includes the following sub-steps:

[0059] S1.1, Savitzky-Golay filtering

[0060] Savitzky-Golay filtering based on least squares fitting is employed. Its key feature is that it effectively preserves the characteristic shape and inflection point information of the data while smoothing it. For key parameter curves in thermal power unit operation data, such as temperature and pressure changes over time, Savitzky-Golay filtering can remove noise without losing important trends and feature points, thus providing a high-fidelity data foundation for subsequent model training.

[0061] It is understandable that using Savitzky-Golay filtering to smooth data can effectively remove noise while preserving the changing trends and characteristic inflection points of key parameter curves, thereby improving data fidelity.

[0062] S1.2, Z-score normalization processing

[0063] The filtered data undergoes Z-score normalization. Its key feature is mapping data of different magnitudes and distribution ranges to a unified interval, such as an interval with a mean of 0 and a standard deviation of 1. This helps accelerate the convergence speed of subsequent neural network models and prevents certain features from dominating model training due to excessively large or small values, thus affecting the model's accuracy and stability. Specifically, based on the data... Standard deviation Normalization is performed using the following formula: This yields preprocessed historical runtime data.

[0064] This sub-step uses Z-score normalization to map data of different magnitudes to a unified interval, avoiding feature-driven bias and helping to accelerate the convergence speed and stability of model training.

[0065] Step S101 involves data processing, including filtering and normalization, to address industrial noise interference and inconsistent unit measurements in the original data. This method improves the quality and consistency of the input data, laying a high-fidelity data foundation for the subsequent accurate representation of system characteristics by the neural network model, and ensuring the stability and accuracy of model training.

[0066] S102, Neural Network Model Establishment and Training;

[0067] The data processed in step S101 is categorized according to input and output. The model inputs are the generator unit's load command, coal feed rate of each coal mill, water feed rate, and main steam regulating valve opening; the model outputs are the main steam temperature, main steam pressure, and actual unit load. Based on this, a neural network model is constructed to predict the changes in system pressure characteristics under different load segments of the generator unit.

[0068] This step specifically includes the following sub-steps:

[0069] S2.1: Building a Layered and Progressive Network Architecture

[0070] A progressive network architecture comprising an input layer, hidden layers, and an output layer is constructed. Based on the physical characteristics and interrelationships of the input variables, the input layer is innovatively divided into multiple sub-layers. This design corresponds to the claims. For example, the coal feed rates of each coal mill related to energy input are grouped into an energy input sub-layer, the main steam regulating valve opening related to steam regulation is set as a steam regulation sub-layer, and a water input sub-layer is used to receive feedwater. This division helps the model capture the influence of different types of input variables on the system pressure characteristics in greater detail.

[0071] Next, multiple hidden layers are set up, including shallow hidden layers and deep hidden layers. The first few hidden layers (shallow hidden layers) focus on extracting the basic features of the input data. As the number of layers increases, the subsequent hidden layers (deep hidden layers) gradually integrate and abstract these basic features to obtain deep fused features, in order to deeply explore the complex nonlinear relationship between input and output under different load segments.

[0072] Meanwhile, skip connections are introduced between multiple hidden layers, enabling information from shallow hidden layers to be directly transmitted to deeper hidden layers. This avoids the loss of key information due to layer-by-layer abstraction and ensures that the model can comprehensively and accurately represent changes in system pressure characteristics under different load conditions.

[0073] This sub-step designs a hierarchical, progressive network architecture, which, through input sub-layers and skip connections, enables fine-grained capture of inputs with different physical characteristics and effective transmission of key information.

[0074] S2.2: Adaptive Dynamic Training and Optimization

[0075] Based on the preprocessed historical running data, an adaptive dynamic training optimization method is used to train the neural network model.

[0076] One aspect is the use of an adaptive learning rate strategy during training. In the initial phase, a relatively large learning rate is set. This allows the model to quickly perform a wide-ranging search within the parameter space, accelerating convergence. As training progresses, the learning rate is dynamically adjusted based on the model's performance on the validation set (e.g., the loss value L). If L fluctuates or no longer decreases significantly, the learning rate automatically decreases, for example... This prompts the model to make fine adjustments around the current parameters, thus avoiding missing the optimal solution.

[0077] On the other hand, a dynamic sample weighting machine based on operating conditions (or load segments) is introduced. Different weighting coefficients are assigned to data samples under different operating conditions (load segments) according to the importance and frequency of the data samples in actual operation. For example, higher weights are assigned to load segments commonly found in thermal power units, allowing the model to focus more on accurately representing the system pressure characteristics under these load segments; while for data samples from rare or extreme load segments, although the weights are relatively lower, their influence on the model is still retained to ensure the model's generalization ability across the entire load range.

[0078] Through this adaptive dynamic training and optimization process, a neural network model that can accurately characterize the changes in system pressure characteristics under each load segment is established. Step S102 adopts adaptive learning rate and dynamic sample weighting, which solves the problems of easily getting trapped in local optima and ignoring key operating conditions during training, and improves the model's generalization ability across the entire load range.

[0079] S103, Reinforcement learning model training;

[0080] Configure the neural network model established in S102 as a reinforcement learning algorithm environment. Acquire historical operating data of the generator set. In the reinforcement learning algorithm environment, take the dispatching command for controlling the generator set as the decision action A of the agent, and the prediction result obtained by the neural network model based on the decision action as the observation value S. Take maximizing the score of the preset reward function R as the optimization objective, train the policy network of the reinforcement learning agent, and obtain a reinforcement learning model for pressure control under variable load conditions.

[0081] This step specifically includes the following sub-steps:

[0082] S3.1: Construction of Reinforcement Learning Environment and Design of Intelligent Agent

[0083] First, the established neural network model is used as the environment. The agent's decisions (actions) will affect the state of the environment (i.e., the input of the neural network model), and the environment will feed back new states (the output of the neural network model) and rewards.

[0084] Furthermore, the intelligent agent structure is designed as an architecture with hierarchical decision-making modules, wherein:

[0085] The lowest level module: This module processes and analyzes direct observations from the output of the neural network model, including direct observations such as main steam temperature. Main steam pressure Actual load of the unit Thus, real-time status assessment results characterizing the current physical condition of the generator set were obtained 71.

[0086] Intermediate layer module: This module, based on real-time status assessment results and combined with current unit load commands, Coal feed rate of each coal mill Water supply Main steam valve opening The system performs preliminary strategy planning based on input information and outputs short-term control strategies to coordinate the various execution units.

[0087] Top-level module: This module adjusts and optimizes short-term control strategies based on the overall operating status and long-term goals of the generator set, and generates and outputs the final decision action A.

[0088] Furthermore, in terms of constructing the observation values, the current main steam temperature, main steam pressure, and actual unit load are defined as the initial observation values. Simultaneously, the rate of change of the main steam pressure over a preset time period, dP / dt, is defined as the observation parameter. Combining the initial observation values ​​and the observation parameter, a complete observation value S is constructed. Introducing the rate of change information enables the agent to more comprehensively perceive the dynamic changes of the system and make more forward-looking decisions.

[0089] Regarding the configuration of the reward function, a preset reward function R is configured based on the proximity of the main steam pressure to the steam setpoint, the stability of pressure changes, and the comprehensive influence parameters of the generator set's operating indicators. For example, the reward function can be set as follows: ,in , Set the pressure and temperature values. , For coefficients, These are the weighting coefficients, and .

[0090] When the main steam pressure P approaches the set value in steady state during the load change process When the main steam temperature T remains within the preset temperature range and the actual load L of the generator set changes smoothly (within the preset load range), a corresponding (higher) bonus value is configured.

[0091] If the main steam pressure fluctuation is within the preset risk fluctuation range, and if other key parameters of the generator set (such as main steam temperature) exceed the preset safety range due to pressure control, a corresponding (negative) penalty value will be configured.

[0092] This sub-step constructs a hierarchical agent and designs observations with rates of change, enabling the agent to fully perceive system dynamics and perform strategy planning at multiple levels, thereby improving the foresight of decision-making.

[0093] S3.2: Historical Data-Driven Model Training and Optimization

[0094] During training, an experience-based training strategy is adopted. The experience e=(s,a,r,s') generated by the interaction between the agent and the reinforcement learning algorithm environment (where s is the current observation, a is the decision action, r is the reward value, and s' is the observation at the next time step) is stored in the experience replay buffer.

[0095] A batch of target experience data is drawn (e.g., randomly selected) from the experience replay buffer to train the neural network model (or policy network). Random sampling breaks the sequential correlation of the data and improves the model's generalization ability.

[0096] To further accelerate learning, a priority-based experience replay mechanism is introduced. Priority information is assigned to each experience data point based on the reward value *r* obtained. For example, a larger reward corresponds to a higher priority. ( , ),in, It is a small positive number.

[0097] When extracting experience from the experience replay buffer, experience data is extracted according to priority information. Experiences with higher priority are more likely to be extracted, enabling the agent to learn valuable experiences more effectively.

[0098] Furthermore, the weight coefficients in the reward function R are dynamically adjusted based on the agent's performance under different load conditions. For example, in load sections where pressure control is more challenging, the reward weight related to pressure stability can be appropriately increased. This sub-step employs priority experience replay and dynamic reward weights, addressing the issues of low training data utilization and insufficient learning of key operating conditions, thus significantly accelerating the convergence of the optimal control strategy.

[0099] Step S103 addresses the lack of self-learning and optimization capabilities in traditional control by configuring the neural network environment, designing hierarchical agents, and implementing efficient training strategies. This approach enables the model to autonomously learn the optimal strategy for variable load pressure control based on historical data, achieving forward-looking and adaptive control.

[0100] S104, Expert System Corrected Bias Design

[0101] Based on the generator set's dead zone data (such as the turbine control valve dead zone) or specific fault conditions (such as mill blockage), an expert system is configured to correct the bias. This expert system is used to handle nonlinear effects or special cases that are difficult for the model to cover.

[0102] Taking the clogging of the mill as an example, the specific implementation of the expert system to correct the bias includes:

[0103] Real-time monitoring of the coal mill outlet differential pressure and oxygen content of the unit; when an increase in the coal mill outlet differential pressure and a rapid decrease in oxygen content are detected, the risk of mill blockage is predicted.

[0104] At this point, the expert system executes corrective actions (biases) to reduce coal consumption and add water based on preset mapping relationships (rule base), specifically:

[0105] Oxygen content decreased by 0.51-1.52%, unit: %

[0106] Coal quantity decreased by 3579 tons per hour.

[0107] Water volume increase: 15253545, unit: t / h

[0108] In an exemplary embodiment, the preset mapping relationship is as follows:

[0109] When the differential pressure at the coal mill outlet increases and the oxygen content decreases, the coal reduction and water addition actions are executed according to the preset mapping relationship. The preset mapping relationship is as follows: when the oxygen content decreases by 0.5%, 1%, 1.5%, and 2%, it corresponds to a coal reduction of 3t / h, 5t / h, 7t / h, and 9t / h, respectively, and a water increase of 15t / h, 25t / h, 35t / h, and 45t / h.

[0110] Step S104, the expert system's bias correction design, addresses the challenge of reinforcement learning models struggling to handle nonlinearity and sudden failures by establishing a rule base for special operating conditions such as mill blockage. This approach provides crucial corrective biases for the model, ensuring that the system can still perform safe and reliable coal-water control even under extreme conditions such as mill blockage or valve dead zones.

[0111] S105, control target achievement;

[0112] The reinforcement learning model (output control effect) ) and expert system (output correction bias) This combination is used for real-time control of coal and water supply under operating conditions.

[0113] Specifically, an adaptive weight control method is adopted to enhance the control effect of reinforcement learning. Correcting bias with expert system Perform adaptive weight stacking.

[0114] Output after superposition ,in As an adaptive weight, it can be dynamically adjusted according to real-time operating conditions (e.g., increasing the weight when a risk of clogging is detected). (weight).

[0115] The final superimposed control signal It is applied to the control of coal and water quantities to achieve the control objectives of reducing main steam pressure and temperature fluctuations, improving unit operating efficiency, and maintaining equipment operating safety.

[0116] Step S105 addresses the issue that a single control strategy cannot simultaneously achieve optimality and safety by adaptively weighting the reinforcement learning control effect with the expert system's corrected bias. This approach integrates intelligent control and rule correction, ultimately impacting coal-water control, effectively reducing main steam pressure and temperature fluctuations, and improving unit operating efficiency.

[0117] also, Figure 2 This is another flowchart of reinforcement learning variable load pressure control based on enthalpy calculation according to an embodiment of this application.

[0118] Through the above steps S101 to S105, the adaptive capability of reinforcement learning is utilized to continuously learn and optimize the control strategy during unit operation. A neural network model is introduced to predict changes in load and system pressure characteristics in advance, which solves the problems of insufficient control accuracy and large pressure fluctuations caused by poor adaptability and lack of foresight in traditional fixed models under variable load conditions. It realizes the technical effect of adapting in real time to the influence of dynamic factors such as coal quality changes, mill start-up and shutdown disturbances, continuous load changes, and equipment aging, improving the unit's response speed to rapid load changes, reducing pressure fluctuations, reducing energy loss and equipment wear, and improving the overall operating efficiency and reliability of the unit.

[0119] Secondly, this application also provides a reinforcement learning-based variable load pressure control system based on enthalpy calculation, which is used to implement the above-described method embodiments. Figure 3 This is a structural block diagram of a reinforcement learning-based variable load pressure control system based on enthalpy calculation, according to an embodiment of this application. Figure 3 As shown, the system includes:

[0120] Module 30 is used to construct a neural network model for predicting the changes in system pressure characteristics under different load segments of the generator set, with the load command of the generator set, coal feed rate of the coal mill, water feed rate and main steam regulating valve opening as model inputs, and the main steam temperature, main steam pressure and actual load of the unit as model outputs.

[0121] Training module 31 is used to acquire historical operating data of generator sets. In the reinforcement learning algorithm environment, the scheduling instructions for controlling generator sets are used as the decision actions of the agent, the prediction results obtained by the neural network model based on the decision actions are used as the observations, and the optimization objective is to maximize the score of the preset reward function. The policy network of the reinforcement learning agent is trained to obtain a reinforcement learning model for pressure control under variable load conditions.

[0122] The control module 32 is used to configure an expert system for bias correction based on the dead zone data of the generator set, and combines the reinforcement learning model with the expert system for real-time control of coal and water supply under operating conditions.

[0123] In some embodiments, when constructing a neural network model, the construction module is specifically used to: build a progressive network architecture including an input layer, a hidden layer, and an output layer; wherein, the input layer is divided into multiple sub-layers according to the physical characteristics and interrelationships of the model input, the sub-layers include: an energy input sub-layer for receiving coal feed from the coal mill, a water input sub-layer for receiving water feed, and a steam regulation sub-layer for receiving main steam valve opening data; the hidden layer includes a shallow hidden layer and a deep hidden layer, the shallow hidden layer extracts features from the model input to obtain basic features, and the deep hidden layer integrates and abstracts the basic features to obtain deep fusion features, wherein skip connections are introduced between multiple hidden layers to pass information from the preceding hidden layer to the deep hidden layer.

[0124] In some embodiments, when training a neural network model, the training module is further configured to: adopt an adaptive learning rate strategy during training, dynamically adjust the learning rate according to the performance of the neural network model on the validation set; and introduce a dynamic sample weighting mechanism based on operating conditions, assigning different weight coefficients to data samples under different operating conditions according to the importance and frequency of data samples in actual operation.

[0125] In some embodiments, the agent includes an architecture with hierarchical decision-making modules. The training module, when designing the agent, is specifically used to: process and analyze direct observations from the output of the neural network model through the lowest-level module to obtain a real-time state assessment result characterizing the current physical condition of the generator set. 128 The intermediate layer module, based on real-time status assessment results and combined with the current unit load command, coal mill feed rate, water feed rate and main steam valve opening, performs preliminary strategy planning and outputs short-term control strategies to coordinate various execution units. The top layer module adjusts and optimizes the short-term control strategies according to the overall operating status and long-term goals of the generator unit, and generates and outputs decision actions.

[0126] In some embodiments, the training module is further configured to: define the current main steam temperature, main steam pressure, and actual unit load as the initial observed values; define the rate of change of the main steam pressure over a preset time period as the observed parameter; and construct the observed values ​​by combining the initial observed values ​​and the observed parameter.

[0127] In some embodiments, the training module is further configured to: configure a preset reward function based on the proximity of the main steam pressure to the steam setpoint, the stability of pressure changes, and the comprehensive impact parameters of the generator set operating indicators; wherein, when the main steam pressure is close to the setpoint in steady state during load changes, the main steam temperature remains within a preset temperature range, and the actual load change of the generator set is within a preset acceptable range, a corresponding reward value is configured; if the main steam pressure fluctuation is within a preset risk fluctuation range, and the key parameters of the generator set exceed a preset safety range due to pressure control, a corresponding penalty value is configured.

[0128] In some embodiments, during the training of the policy network of the reinforcement learning agent, the training module is further configured to: adopt an experience replay-based training strategy to store the experience generated by the agent's interaction with the reinforcement learning algorithm environment in an experience replay buffer; extract target experience data from the experience replay buffer to train the neural network model; wherein, priority information is configured for the experience data according to the reward value obtained for each experience data; and when extracting experience from the experience replay buffer, experience data is extracted according to the priority information.

[0129] In some embodiments, before constructing the neural network model, the construction module is further configured to smooth the historical running data using Savitzky-Golay filtering; and to perform Z-score normalization on the filtered historical running data based on the mean and standard deviation of the historical running data to obtain the preprocessed historical running data.

[0130] In some embodiments, when the control module uses an expert system to correct the bias, it is specifically used to: monitor the outlet differential pressure of the coal mill and the oxygen content of the unit in real time; when it is detected that the outlet differential pressure of the coal mill increases and the oxygen content decreases, it performs the actions of reducing coal and adding water according to a preset mapping relationship.

[0131] This system utilizes the adaptive capabilities of reinforcement learning to continuously learn and optimize control strategies during unit operation. It also introduces a neural network model to predict changes in load and system pressure characteristics in advance. This solves the problems of insufficient control accuracy and large pressure fluctuations caused by the poor adaptability and lack of foresight of traditional fixed models under variable load conditions. It enables real-time adaptation to the effects of dynamic factors such as changes in coal quality, start-up and shutdown of mills, continuous load changes, and equipment aging. This improves the unit's response speed to rapid load changes, reduces pressure fluctuations, reduces energy consumption and equipment wear, and enhances the overall operating efficiency and reliability of the unit.

[0132] In one embodiment, Figure 4 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application, such as... Figure 4 As shown, an electronic device is provided, which can be a server, and its internal structure diagram can be as follows. Figure 4 As shown, the electronic device includes a processor, a network interface, internal memory, and non-volatile memory connected via an internal bus. The non-volatile memory stores the operating system, computer programs, and a database. The processor provides computing and control capabilities, the network interface communicates with external terminals via a network connection, the internal memory provides the environment for the operating system, the computer programs are executed by the processor to implement a reinforcement learning-based variable load pressure control method based on enthalpy calculation, and the database stores data.

[0133] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0134] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by hardware related to computer program instructions. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0135] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0136] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A reinforcement learning-based variable load pressure control method based on enthalpy calculation, characterized in that, The method includes: Using the generator set load command, coal mill feed rate, water feed rate and main steam regulating valve opening as model inputs, and the main steam temperature, main steam pressure and actual unit load as model outputs, a neural network model is constructed to predict the changes in system pressure characteristics under different load segments of the generator set. Historical operating data of the generator set is obtained. Under the reinforcement learning algorithm environment, the scheduling command for controlling the generator set is taken as the decision action of the agent. The prediction result obtained by the neural network model based on the decision action is taken as the observation value. The optimization objective is to maximize the score of the preset reward function. The policy network of the reinforcement learning agent is trained to obtain a reinforcement learning model for pressure control under variable load conditions. An expert system for bias correction is configured based on the dead zone data of the generator set. The reinforcement learning model is combined with the expert system for real-time control of coal and water supply under operating conditions. The neural network model for predicting system pressure characteristics changes under different load conditions of generator sets includes: Build a progressive network architecture that includes an input layer, a hidden layer, and an output layer; The input layer is divided into multiple sub-layers based on the physical characteristics and interrelationships of the model input. The sub-layers include: an energy input sub-layer for receiving the coal feed from the coal mill, a water input sub-layer for receiving the water feed, and a steam regulation sub-layer for receiving the main steam regulating valve opening data. The hidden layer includes a shallow hidden layer and a deep hidden layer. The shallow hidden layer extracts features from the model input to obtain basic features, and the deep hidden layer integrates and abstracts the basic features to obtain deep fusion features. Skip connections are introduced between multiple hidden layers to pass information from previous hidden layers to the deep hidden layer. Based on the preprocessed historical running data, the neural network model is trained using an adaptive dynamic training optimization method, specifically including: An adaptive learning rate strategy is adopted during training, and the learning rate is dynamically adjusted according to the performance of the neural network model on the validation set. A dynamic sample weighting mechanism based on operating conditions is introduced, which assigns different weight coefficients to data samples under different operating conditions according to the importance and frequency of data samples in actual operation. Correcting biases using expert systems includes: Real-time monitoring of the outlet differential pressure of the coal mill and the oxygen content of the unit; When the differential pressure at the outlet of the coal mill increases and the oxygen content decreases, the coal reduction and water addition actions are performed according to the preset mapping relationship. The preset mapping relationship is as follows: when the oxygen content decreases by 0.5%, 1%, 1.5%, and 2%, it corresponds to a coal reduction of 3t / h, 5t / h, 7t / h, and 9t / h, respectively, and a water increase of 15t / h, 25t / h, 35t / h, and 45t / h.

2. The method according to claim 1, characterized in that, The intelligent agent includes an architecture with hierarchical decision-making modules, wherein: The lowest-level module processes and analyzes the direct observations output by the neural network model to obtain real-time status assessment results that characterize the current physical condition of the generator set. The direct observations include: main steam temperature, main steam pressure, and actual load of the unit. Based on the real-time status assessment results, the intermediate layer module performs preliminary strategy planning by combining the current unit load command, coal mill feed rate, water feed rate and main steam valve opening, and outputs short-term control strategies to coordinate the various execution units. The top-level module adjusts and optimizes the short-term control strategy based on the overall operating status and long-term goals of the generator set, and generates and outputs the decision-making actions.

3. The method according to any one of claims 1 or 2, characterized in that, The method further includes: The current main steam temperature, main steam pressure, and actual unit load are defined as the initial observation values ​​of the observed values. The rate of change of the main steam pressure over a preset time period is defined as an observation parameter. The observation value is constructed by combining the initial observation value and the observation parameter.

4. The method according to claim 1, characterized in that: The preset reward function is configured based on the proximity of the main steam pressure to the steam setpoint, the stability of pressure changes, and the comprehensive influence parameters of the generator set's operating indicators. Specifically, when the main steam pressure approaches the set value in steady state during the load change process, the main steam temperature remains within the preset temperature range, and the actual load change of the generator set is within the preset compliance range, a corresponding reward value is configured. If the main steam pressure fluctuation is within the preset risk fluctuation range, and the key parameters of generator set 1 exceed the preset safety range due to pressure control, a corresponding penalty value is configured.

5. The method according to claim 1, characterized in that, The method further includes the following steps in training the policy network of the reinforcement learning agent: An experience-based training strategy is adopted to store the experience generated by the agent's interaction with the reinforcement learning algorithm environment in an experience replay buffer. The neural network model is trained by extracting target experience data from the experience replay buffer, wherein the experience data includes observations, decision actions, reward values ​​obtained for the decision actions, and observations at the next time step. Among them, priority information is configured for each experience data based on the reward value obtained in each experience data session; When extracting experience from the experience playback buffer, the experience data is extracted according to the priority information.

6. The method according to claim 1, characterized in that, Data processing of the historical operational data includes: The historical running data is smoothed using Savitzky-Golay filtering, wherein the Savitzky-Golay filtering is based on least squares fitting. Furthermore, based on the mean and standard deviation of the historical operating data, the filtered historical operating data is subjected to Z-score normalization processing to map data of different magnitudes and distribution ranges to a unified interval, thereby obtaining preprocessed historical operating data.

7. A reinforcement learning-based variable load pressure control system based on enthalpy calculation, characterized in that, The system includes: The module is used to construct a neural network model for predicting the changes in system pressure characteristics under different load segments of the generator set, with the generator set load command, coal mill feed rate, water feed rate and main steam valve opening as model inputs, and the main steam temperature, main steam pressure and actual unit load as model outputs. The training module is used to acquire historical operating data of the generator set. Under the reinforcement learning algorithm environment, the scheduling command for controlling the generator set is used as the decision action of the agent, and the prediction result obtained by the neural network model based on the decision action is used as the observation value. The optimization objective is to maximize the score of the preset reward function. The policy network of the reinforcement learning agent is trained to obtain a reinforcement learning model for pressure control under variable load conditions. The control module is used to configure an expert system for bias correction based on the dead zone data of the generator set, and to combine the reinforcement learning model with the expert system for real-time control of coal and water supply under operating conditions. The neural network model for predicting system pressure characteristics changes under different load conditions of generator sets includes: Build a progressive network architecture that includes an input layer, a hidden layer, and an output layer; The input layer is divided into multiple sub-layers based on the physical characteristics and interrelationships of the model inputs. These sub-layers include: an energy input sub-layer for receiving the coal feed from the coal mill; a water input sub-layer for receiving the water feed; and a sub-layer for receiving... Main steam valve opening data Steam conditioning sublayer; The hidden layer includes a shallow hidden layer and a deep hidden layer. The shallow hidden layer extracts features from the model input to obtain basic features, and the deep hidden layer integrates and abstracts the basic features to obtain deep fusion features. Skip connections are introduced between multiple hidden layers to pass information from previous hidden layers to the deep hidden layer. Based on the preprocessed historical running data, the neural network model is trained using an adaptive dynamic training optimization method, specifically including: An adaptive learning rate strategy is adopted during training, and the learning rate is dynamically adjusted according to the performance of the neural network model on the validation set. A dynamic sample weighting mechanism based on operating conditions is introduced, which assigns different weight coefficients to data samples under different operating conditions according to the importance and frequency of data samples in actual operation. Correcting biases using expert systems includes: Real-time monitoring of the outlet differential pressure of the coal mill and the oxygen content of the unit; When the differential pressure at the outlet of the coal mill increases and the oxygen content decreases, the coal reduction and water addition actions are performed according to the preset mapping relationship. The preset mapping relationship is as follows: when the oxygen content decreases by 0.5%, 1%, 1.5%, and 2%, it corresponds to a coal reduction of 3t / h, 5t / h, 7t / h, and 9t / h, respectively, and a water increase of 15t / h, 25t / h, 35t / h, and 45t / h.

Images