A Control Method for Pneumatic Ash Conveying Systems in Thermal Power Plants Based on Digital Twins and Reinforcement Learning
By employing a closed-loop self-optimization control method based on digital twins and reinforcement learning, the problems of high energy consumption, easy pipe blockage, and outdated control strategies in the pneumatic ash conveying system of thermal power plants were solved. This method achieved global optimization and collaborative optimization of the system, and improved equipment health management and predictive maintenance capabilities.
Patent Information
- Application Number
- CN202611207939.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-11
- Publication Date
- 2026-11-17
AI Technical Summary
The pneumatic ash conveying system in thermal power plants suffers from problems such as high energy consumption, easy pipe blockage, high wear, outdated control strategies, poor system coordination, and passive operation and maintenance mode. Existing technologies have failed to effectively solve core problems such as prediction accuracy, experience-based control strategies, and lack of system coordination.
A closed-loop self-optimizing control method based on digital twins and reinforcement learning is adopted. By constructing a high-fidelity digital twin model, the state space, action space and multi-objective reward function of reinforcement learning are defined. The optimal control strategy is trained using reinforcement learning algorithm and deployed to the edge computing gateway to realize real-time closed-loop control. Combined with the PLC system, safe redundancy switching is performed to achieve collaborative optimization between the system and upstream and downstream.
It achieves global optimization of system performance, reduces energy consumption, avoids pipe blockage and equipment damage, improves predictive maintenance capabilities, realizes the transformation from passive operation and maintenance to proactive predictive maintenance, and improves the system's adaptability and coordination.
Smart Images

Figure CN123113647A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of automatic control technology for thermal power plants, specifically relating to a closed-loop self-optimization control method, system, and computer-readable storage medium for a pneumatic ash conveying system in a thermal power plant based on digital twins and reinforcement learning. Background Technology
[0002] The pneumatic ash conveying system in thermal power plants is a key auxiliary system responsible for transporting fly ash collected by dust collectors to ash silos using compressed air. However, this system has long faced the following technical challenges: (1) High energy consumption and low efficiency: Traditional pneumatic ash conveying systems generally have high energy consumption, serious waste of compressed air, and the system design does not fully consider energy saving, resulting in high operating load of air compressor.
[0003] (2) Easy to clog and high wear: Due to changes in ash content, complex pipeline layout and improper setting of conveying parameters, the system is very prone to pipe blockage. The high flow rate (up to 20 m / s or more at the end) used to ensure conveying leads to severe wear of pipes and elbows, resulting in leakage and environmental pollution.
[0004] (3) Outdated control strategies: Existing control systems rely heavily on historical experience databases and manual adjustments, lacking real-time online self-learning capabilities. The system cannot automatically optimize operating parameters based on extreme conditions such as sudden changes in coal quality or load, and the prediction model has low accuracy in predicting peak ash content.
[0005] (4) Poor system coordination: The optimization of the ash conveying system is mostly carried out in isolation, and data sharing and collaborative optimization have not been achieved with the upstream boiler combustion system, the whole plant compressed air pipeline network and the downstream ash and slag comprehensive utilization link.
[0006] (5) Passive operation and maintenance mode: Equipment maintenance mainly relies on regular inspections and post-event handling, lacking equipment health management and digital twin support, and unable to achieve fault early warning and predictive maintenance.
[0007] Existing technologies, such as LSSVM-based ash volume prediction, pilot-operated ash conveying, and self-optimizing turbulent flow anti-blockage techniques, have achieved some success, but they still fail to fundamentally solve core problems such as prediction accuracy, empirical control strategies, and lack of system coordination. Therefore, there is an urgent need for a new solution that can integrate existing technologies and combine cutting-edge artificial intelligence and digital twins to achieve a comprehensive intelligent upgrade of the ash conveying system. Summary of the Invention
[0008] This invention provides a closed-loop self-optimizing control method, system, and computer-readable storage medium for a pneumatic ash conveying system in a thermal power plant based on digital twins and reinforcement learning, in order to solve the technical problems existing in the prior art.
[0009] To achieve the above objectives, this invention provides a closed-loop self-optimizing control method for an ash conveying system based on digital twins and reinforcement learning, comprising the following steps: Step S1: Construct a high-fidelity digital twin model of the ash conveying system; Step S2: In the high-fidelity digital twin model, define the state space, action space, and multi-objective reward function for reinforcement learning; the state space includes the pressure of the ash conveying main pipe, the ash hopper level, the pilot valve status, the ash type, and the elbow wear index; the action space includes the selection of the conveying unit, the conveying time, the inlet pressure, and the pilot valve opening threshold; the multi-objective reward function integrates energy consumption, conveying efficiency, operational stability, pipeline wear, and equipment health indicators. Step S3: In the digital twin environment, the optimal control policy is trained using a reinforcement learning algorithm, wherein the reinforcement learning algorithm is a proximal policy optimization or a soft actor-critic algorithm; Step S4: Deploy the trained strategy model to the edge computing gateway, connect it to the PLC control system through a security interface to achieve real-time closed-loop control, and retain the original PLC control logic as a safety redundancy switching mechanism.
[0010] In one embodiment of the present invention, optionally, in step S1, the high-fidelity digital twin model includes a geometric model, a gas-solid two-phase flow physical model based on computational particle fluid dynamics, and a data-driven model based on least squares support vector machine or long short-term memory network.
[0011] In one embodiment of the present invention, optionally, in step S1, the geometric model is established by three-dimensional modeling software, the physical model is based on the CPFD method to simulate the gas-solid two-phase flow, pressure drop, particle deposition and blockage mechanism in the pipeline, and the data-driven model is trained using historical operating data as a rapid replacement or correction module for the physical model.
[0012] In one embodiment of the present invention, optionally, in step S2, the multi-objective reward function specifically includes: Energy-saving rewards are given when the total power consumption of the system is negative. Efficiency rewards are based on the amount of ash transported per unit time. Stability rewards are given for the absence of pipe blockage events, while a preset negative penalty value is applied for pipe blockage events. Wear penalty, with a negative penalty equal to the square of the average flow velocity in the pipe; and Equipment health penalties are negatively imposed based on the number of times valves or other equipment operate or the duration of high-load operation. By adjusting the weighting coefficients of each reward item, the optimization direction can be flexibly adjusted according to the actual needs of the power plant.
[0013] In one embodiment of the present invention, optionally, the edge computing gateway in step S4 collects field sensor data in real time, runs a strategy model for rapid reasoning, and sends the optimal action command to the PLC control system to realize real-time closed-loop control of the physical ash conveying system; when the edge computing gateway fails, communication is interrupted, or an abnormal command is issued, the system automatically switches back to the original PLC control logic.
[0014] In one embodiment of the present invention, the method may optionally include step S5: using actual operating data, periodically perform incremental training on the digital twin model and reinforcement learning strategy to achieve continuous self-evolution of the system; at the same time, based on the digital twin model and real-time data, use an anomaly detection algorithm to monitor the equipment status, realize fault warning and remaining life prediction, and automatically generate predictive maintenance plans.
[0015] This invention also provides a closed-loop self-optimizing control system for an ash conveying system based on digital twins and reinforcement learning, comprising: The data acquisition module is used to collect real-time operating data of the ash conveying system; The digital twin model building module is used to build high-fidelity digital twin models as described above; A reinforcement learning training module is used to train optimal control strategies in a digital twin environment; Edge computing gateways are used to deploy trained policy models, perform real-time inference, and issue control commands. The PLC control system is used to receive instructions from the edge computing gateway and execute physical control, while retaining a safe redundancy switching mechanism. The collaborative optimization interface is used for data interaction and collaborative optimization with the boiler combustion system, the whole plant air compressor station, and the ash and slag sales system.
[0016] In one embodiment of the present invention, the data acquisition module may optionally include a pressure sensor and a position sensor deployed on a pilot valve, a weighing sensor or a level gauge deployed on a silo pump, an acoustic emission sensor or a wall thickness monitoring sensor deployed at a pipe bend, and a flow meter deployed on an air compressor.
[0017] In one embodiment of the present invention, optionally, the collaborative optimization interface includes: The interface with the boiler combustion system is used to acquire online coal quality analysis data and unit load commands; The interface with the plant's air compressor station is used to obtain the pressure, flow rate, and operating status of each air compressor in the compressed air pipeline network; The interface with the ash sales system is used to obtain downstream users' demand information for ash.
[0018] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.
[0019] The closed-loop self-optimization control method, system, and computer-readable storage medium for pneumatic ash conveying systems in thermal power plants based on digital twins and reinforcement learning provided by this invention have the following beneficial technical effects: (1) First deep integration: For the first time, digital twin technology and reinforcement learning algorithm are deeply integrated and applied to the pneumatic ash conveying system of thermal power plant, and a closed-loop self-optimizing control architecture combining virtual and real is constructed.
[0020] (2) Multi-objective collaborative optimization: A multi-objective reward function was designed to achieve global optimal system performance, avoiding the drawbacks of single-objective optimization.
[0021] (3) Safe and efficient training and deployment: The use of a digital twin environment for safe training of reinforcement learning strategies completely avoids the risks of pipe blockage and equipment damage caused by "trial and error" in the actual system. Low-latency real-time control is achieved through an edge computing gateway, while retaining security redundancy, thus balancing advanced technology and reliability.
[0022] (4) System-level collaboration: Through the collaborative optimization interface, the linkage between the ash conveying system and boiler combustion, the whole plant gas source and ash sales are realized, achieving global optimization from source to end.
[0023] (5) Continuous self-evolution capability: Through continuous model updates and equipment health management, the system can adapt to changes brought about by long-term operation and realize the transformation from passive operation and maintenance to proactive predictive maintenance. Attached Figure Description
[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0025] Figure 1 This is an overall flowchart of the closed-loop self-optimization control method for an ash conveying system provided in an embodiment of the present invention; Figure 2 This is a diagram illustrating the reinforcement learning training framework provided in an embodiment of the present invention. Figure 3 This is a schematic diagram of the system hardware architecture and collaborative optimization interface provided in the embodiments of the present invention; Figure 4This is a schematic diagram of the system hardware architecture and collaborative optimization interface provided in the embodiments of the present invention. Detailed Implementation
[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0027] Figure 1 This is an overall flowchart of the closed-loop self-optimization control method for an ash conveying system provided in an embodiment of the present invention. Figure 2 This is a diagram illustrating the reinforcement learning training framework provided in an embodiment of the present invention. Figure 3 This is a schematic diagram of the system hardware architecture and collaborative optimization interface provided in an embodiment of the present invention. Figure 4 This is a schematic diagram of the system hardware architecture and collaborative optimization interface provided in the embodiments of the present invention, such as... Figures 1-4 As shown, this invention provides a closed-loop self-optimizing control method for an ash conveying system based on digital twins and reinforcement learning, which includes the following steps: Step S1: Construct a high-fidelity digital twin model of the ash conveying system; In step S1, the high-fidelity digital twin model includes a geometric model, a gas-solid two-phase flow physical model based on the computational particle fluid dynamics (CPFD) method, and a data-driven model based on least squares support vector machine (LSSVM) or long short-term memory network (LSTM).
[0028] In step S1, the geometric model is established using 3D modeling software, such as SolidWorks, to create a precise 3D solid model of all equipment (silo pumps, valves, pipes, elbows, ash silos, etc.) of the ash conveying system at a 1:1 scale and with millimeter-level precision. This model not only includes the external shape but should also accurately depict the internal flow channel structure of valves and pipes.
[0029] The physical model simulates gas-solid two-phase flow, pressure drop, particle deposition, and blockage mechanisms within pipelines using the CPFD method. For example, the physical model is established in simulation software such as ANSYS Fluent or Barracuda Virtual Reactor based on the CPFD method. This model requires input parameters such as fly ash particle size distribution, particle density, and pipeline roughness to accurately simulate the pressure distribution, particle velocity field, pressure drop, and particle deposition and blockage formation processes within the pipeline. Simultaneously, an energy consumption model based on the air compressor performance curve and a pressure loss model based on valve flow characteristics are established.
[0030] The data-driven model is trained using historical operational data and serves as a rapid replacement or correction module for the physical model, used to predict ash volume, pipe blockage risk, etc. For example, the data-driven model collects at least six months of historical operational data from the ash conveying system (including pressure, material level, valve status, air compressor current, ash volume, etc.), cleans and normalizes it, and then uses this data to train an LSTM neural network. This LSTM model, as a rapid replacement for the physical model, can quickly predict changes in ash volume and the probability of pipe blockage risk over the next 10-30 minutes based on data from the previous hour, providing boundary conditions for the physical model or acting as a corrector.
[0031] Finally, the three models are integrated on digital twin platforms such as ANSYS TwinBuilder. Through model reduction techniques, the complex CPFD model is simplified into a reduced-order model that can complete simulations within seconds, and then co-simulated with the LSTM model and the geometric model to form a complete, real-time running digital twin.
[0032] Step S2: In the high-fidelity digital twin model, define the state space, action space, and multi-objective reward function for reinforcement learning; the state space defines key parameters that reflect the system's operating state, including the pressure of the ash conveying main pipe, the ash hopper level, the pilot valve status, the ash type, and the elbow wear index; the action space defines the control commands that the system can execute, including the selection of the conveying unit, the conveying time, the inlet pressure, and the pilot valve opening threshold; the multi-objective reward function integrates energy consumption, conveying efficiency, operational stability, pipeline wear, and equipment health indicators. The multi-objective reward function specifically includes: Energy-saving rewards are given as negative rewards based on the total power consumption of the system, in order to encourage low-energy operation. Efficiency rewards are given based on the amount of ash transported per unit time, in order to encourage efficient transport. Stability rewards are given for the absence of pipe blockage events, while a preset negative penalty value is applied for pipe blockage events. Wear penalty, calculated as a negative penalty based on the square of the average flow velocity within the pipe, suppresses wear caused by high flow velocities; and Equipment health penalties are negatively imposed based on the number of times valves or other equipment operate or the duration of high-load operation. By adjusting the weighting coefficients of each reward item, the optimization direction can be flexibly adjusted according to the actual needs of the power plant.
[0033] State Space: The state space is a multi-dimensional vector, represented as S = [t, P_mains, L_hoppers, V_valves, T_bends, Q_ash_pred, ...]. Specifically, it includes: t: Current time, encoded as a periodic time feature.
[0034] P_mains: Real-time pressure of the ash conveying main pipe.
[0035] L_hoppers: The material level of all critical hoppers.
[0036] V_valves: The open / closed status of all pilot valves.
[0037] T_bends: Feature values extracted by acoustic emission sensors at critical bends, reflecting wear or blockage trends.
[0038] Q_ash_pred: The future gray amount predicted by the LSTM model.
[0039] Other parameters include current total power, actual ash conveying flow rate, and ash type.
[0040] Action Space: The action space is also a multi-dimensional vector, represented as A = [unit_selected, t_deliver, P_inlet, threshold_on, t_wait]. Specifically, it includes: unit_selected: Select the next silo pump unit ID for delivery.
[0041] t_deliver: The duration of this delivery.
[0042] P_inlet: Intake pressure setting value.
[0043] threshold_on: The opening pressure threshold of the pilot valve.
[0044] t_wait: Waiting time between different delivery units.
[0045] Multi-objective reward function: The design of the reward function is crucial to the success of reinforcement learning. In this embodiment, the immediate reward function R is a weighted sum of various rewards or penalties: R = w1 * R_energy + w2 * R_efficiency + w3 * R_stability + w4 * R_wear + w5 * R_maintenance.
[0046] R_energy is the energy-saving reward, R_energy=-(P_current / P_max), where P_current is the current real-time total power.
[0047] R_efficiency is the efficiency bonus, R_efficiency=Q_transported / Q_max, where Q_transported is the amount of ash transported per unit time.
[0048] R_stability is the stability reward. If the time interval since the last blockage exceeds a preset threshold (e.g., 1 hour), a small positive reward is given; if a blockage event occurs, a large negative reward (e.g., -100) is given.
[0049] R_wear is the wear penalty, R_wear = -(V_flow_avg / V_max)^2, where V_flow_avg is the average flow velocity in the pipe. Since the wear is proportional to the cube of the flow velocity, the square term is used here as an approximate penalty.
[0050] R_maintenance is the equipment health penalty, R_maintenance=-(N_valve_actions / N_max), where N_valve_actions is the number of times critical valves such as pilot valves are activated within the statistical period.
[0051] The weighting coefficients w1 to w5 can be adjusted according to the specific operational objectives of the power plant (such as prioritizing extreme energy conservation or efficiency).
[0052] Step S3: In the digital twin environment, the optimal control policy is trained using a reinforcement learning algorithm, wherein the reinforcement learning algorithm is the proximal policy optimization (PPO) algorithm or the soft actor-critic (SAC) algorithm. The agent learns the optimal control policy by continuously interacting with the environment and making mistakes, updating the policy network parameters based on the state, actions, and rewards, until it learns the optimal control policy that maximizes long-term cumulative rewards.
[0053] This embodiment uses the PPO algorithm, which updates the amplitude through a pruning strategy, ensuring training stability and data efficiency. It is very suitable for tasks with continuous states and action spaces, such as ash conveying systems.
[0054] Training process: (1) Initialization: Under the RayRLlib distributed training framework, initialize an Actor neural network (policy network) and a Critic neural network (value network).
[0055] (2) Environment Interaction: At each time step, the Actor network outputs an action A_t based on the current state S_t. The digital twin model executes this action, simulates and calculates the next state S_{t+1}, and calculates the reward R_t based on the reward function.
[0056] (3) Policy update: The agent stores (S_t, A_t, R_t, S_{t+1}) into the experience replay buffer. The PPO algorithm randomly samples a batch of experiences from the buffer and updates the parameters of the Actor and Critic networks by calculating the advantage function and the pruning loss function.
[0057] (4) Iteration: Repeat steps 2 and 3 for millions or even tens of millions of iterations until the policy converges, that is, the cumulative reward value tends to be stable on the validation set and no longer increases significantly.
[0058] Strategy Validation: After training, the optimal strategy is validated offline in a digital twin environment using a batch of historical data that was never used in the training and includes various extreme operating conditions (such as sudden changes in coal quality and low-load operation). The strategy is compared with a baseline strategy (such as existing PLC logic) to evaluate its performance in terms of average energy consumption, pipe blockage rate, and average flow rate.
[0059] Step S4: Deploy the trained strategy model (neural network) to the edge computing gateway, connect it to the PLC control system through a security interface to achieve real-time closed-loop control, and retain the original PLC control logic as a safety redundancy switching mechanism. In step S4, the edge computing gateway collects real-time data from field sensors, runs a strategy model for rapid inference (<50ms), and sends the optimal action command to the PLC control system to achieve real-time closed-loop control of the physical ash conveying system. When the edge computing gateway fails, communication is interrupted, or an abnormal command is issued, the system automatically switches back to the original PLC control logic.
[0060] The optimal policy model (i.e., the Actor network) that has been trained and validated will be deployed to the actual system.
[0061] Model conversion and deployment: The PyTorch format model file was standardized using the ONNX tool and converted into C++ code. This code was then burned into the edge computing gateway deployed in the field control room.
[0062] Real-time closed-loop control: The edge computing gateway collects sensor data (pressure, level, valve status, etc.) from the field PLC in real time via the OPCUA protocol, processes it into a state vector S_t, and calls the model for forward inference to calculate the optimal action A_t. The total time for this process is less than 50 milliseconds. Subsequently, the gateway sends the action command to the main PLC through a secure interface, and the PLC executes the specific equipment control.
[0063] Safety redundancy mechanism: In the main PLC, the original threshold- and rule-based control logic is retained as an independent "watchdog" module. This module continuously monitors: The edge computing gateway's heartbeat signal. If no valid heartbeat is received within a preset time (e.g., 5 seconds), the gateway is deemed to have failed.
[0064] Does the action command issued by the gateway exceed the preset safety boundary (e.g., the pressure setting value exceeds 90% of the device's maximum withstand pressure)?
[0065] If any of the above conditions are triggered, the PLC will immediately cut off control from the edge gateway, automatically switch back to the original control logic, and send an alarm to the central control room to ensure the absolute safety of the system.
[0066] Step S5: Utilize actual operational data to periodically perform incremental training on the digital twin model and reinforcement learning strategy to achieve continuous self-evolution of the system; at the same time, based on the digital twin model and real-time data, use anomaly detection algorithms to monitor equipment status, realize fault early warning and remaining life prediction, and automatically generate predictive maintenance plans.
[0067] Continuous model updates: The system automatically collects real-world operational data from the past 7 days weekly. This data is used to incrementally train the digital twin model and reinforcement learning strategies deployed in the cloud or on servers (e.g., fine-tuning using experience replay techniques). After offline validation, the updated model is deployed to an edge computing gateway through a reliable software upgrade process, completing the system's self-evolution.
[0068] Equipment health management: An anomaly detection model based on the Isolation Forest algorithm runs in parallel within the digital twin model. This model analyzes the spectral data from the acoustic emission sensors at the elbow in real time. When an abnormal high-frequency signal (wear characteristic) is detected, the system issues a wear warning. Simultaneously, based on historical wear data and current operating conditions, a proportional risk model is used to predict the remaining service life of the elbow, and a predictive maintenance work order containing spare parts models and planned replacement times is automatically generated in a visual interface.
[0069] This invention also provides a closed-loop self-optimizing control system for an ash conveying system based on digital twins and reinforcement learning, comprising: The data acquisition module is used to collect real-time operating data of the ash conveying system; The data acquisition module includes a pressure sensor and a position sensor deployed on the pilot valve, a weighing sensor or level gauge deployed on the silo pump, an acoustic emission sensor or wall thickness monitoring sensor deployed at the pipe bend, and a flow meter deployed on the air compressor.
[0070] The digital twin model building module is used to build high-fidelity digital twin models as described above; A reinforcement learning training module is used to train optimal control strategies in a digital twin environment; Edge computing gateways are used to deploy trained policy models, perform real-time inference, and issue control commands. The PLC control system is used to receive instructions from the edge computing gateway and execute physical control, while retaining a safe redundancy switching mechanism. The collaborative optimization interface is used for data interaction and collaborative optimization with the boiler combustion system (to obtain coal quality and load data), the whole plant air compressor station (to obtain pipeline pressure and flow data), and the ash and slag sales system (to obtain ash demand data).
[0071] The collaborative optimization interface includes: The interface with the boiler combustion system is used to acquire online coal quality analysis data and unit load commands; The interface with the plant's air compressor station is used to obtain the pressure, flow rate, and operating status of each air compressor in the compressed air pipeline network; The interface with the ash sales system is used to obtain downstream users' demand information for ash.
[0072] The closed-loop self-optimizing control system for the ash conveying system based on digital twins and reinforcement learning in this embodiment is configured as follows: Data Acquisition Module: This module consists of a network of intelligent sensors deployed in the field, specifically including: pressure sensors and Hall position sensors mounted on the pilot valve; high-precision load cells mounted on the silo pump support legs; acoustic emission sensors attached to the outside of key bends; and a thermal mass flow meter mounted on the air compressor outlet manifold. All sensors are connected to the central data server via an industrial Ethernet ring network supporting the PROFINET protocol.
[0073] Digital Twin Model Building Module: This module runs on a high-performance computing server and is built on the ANSYS TwinBuilder platform. It integrates the aforementioned geometric model, CPFD physical model, and LSTM data-driven model, providing a digital twin environment for offline training and simulation.
[0074] Reinforcement Learning Training Module: This module also runs on the computing server and implements the PPO algorithm based on the RayRLlib framework. It calls upon a digital twin environment for large-scale parallel training to generate the optimal control policy model.
[0075] Edge computing gateway: This gateway is an embedded industrial control computer deployed in the field, internally loaded with a transformed and optimized policy model. It is responsible for real-time inference and acts as a hub for cloud-edge-device collaboration.
[0076] PLC control system: This system is a programmable logic controller that is either existing or newly deployed on-site. Its internal logic has been modified according to the requirements of this invention. It can receive and execute instructions from the edge gateway through the communication interface, and also retains complete traditional control logic as a safety redundancy.
[0077] Collaborative Optimization Interface: This interface is a set of software interfaces based on RESTful API and the OPCUA protocol. Specifically, it includes: Interface with the boiler combustion system: Obtain online coal quality analysis data (ash content, moisture, calorific value) and real-time load commands from the plant-level MES system.
[0078] Interface with the plant's air compressor station: Obtain the plant's compressed air main pressure, start / stop status of each air compressor, and real-time air production from the air compressor group control system.
[0079] Interface with the ash sales system: Obtain the order demand from downstream users for ash with different fineness and carbon content from the operation and management system.
[0080] The data acquired through these interfaces serves as external input to the digital twin model, driving the system to perform forward-looking and collaborative optimal control.
[0081] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.
[0082] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention.
[0083] Those skilled in the art will understand that the modules in the apparatus of the embodiments can be distributed in the apparatus of the embodiments as described in the embodiments, or they can be located in one or more devices different from this embodiment with corresponding changes. The modules of the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.
[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A closed-loop self-optimizing control method for an ash conveying system based on digital twins and reinforcement learning, characterized in that, Includes the following steps: Step S1: Construct a high-fidelity digital twin model of the ash conveying system; Step S2: In the high-fidelity digital twin model, define the state space, action space, and multi-objective reward function for reinforcement learning; the state space includes the pressure of the ash conveying main pipe, the ash hopper level, the pilot valve status, the ash type, and the elbow wear index; the action space includes the selection of the conveying unit, the conveying time, the inlet pressure, and the pilot valve opening threshold; the multi-objective reward function integrates energy consumption, conveying efficiency, operational stability, pipeline wear, and equipment health indicators. Step S3: In the digital twin environment, the optimal control policy is trained using a reinforcement learning algorithm, wherein the reinforcement learning algorithm is a proximal policy optimization or a soft actor-critic algorithm; Step S4: Deploy the trained strategy model to the edge computing gateway, connect it to the PLC control system through a security interface to achieve real-time closed-loop control, and retain the original PLC control logic as a safety redundancy switching mechanism.
2. The method according to claim 1, characterized in that, In step S1, the high-fidelity digital twin model includes a geometric model, a gas-solid two-phase flow physical model based on computational particle hydrodynamics, and a data-driven model based on least squares support vector machine or long short-term memory network.
3. The method according to claim 2, characterized in that, In step S1, the geometric model is established using 3D modeling software, the physical model is based on the CPFD method to simulate gas-solid two-phase flow, pressure drop, particle deposition and blockage mechanism in the pipeline, and the data-driven model is trained using historical operating data as a rapid replacement or correction module for the physical model.
4. The method according to claim 1, characterized in that, In step S2, the multi-objective reward function specifically includes: Energy-saving rewards are given when the total power consumption of the system is negative. Efficiency rewards are based on the amount of ash transported per unit time. Stability rewards are given for the absence of pipe blockage events, while a preset negative penalty value is applied for pipe blockage events. Wear penalty, with a negative penalty equal to the square of the average flow velocity in the pipe; and Equipment health penalties are negatively imposed based on the number of times valves or other equipment operate or the duration of high-load operation. By adjusting the weighting coefficients of each reward item, the optimization direction can be flexibly adjusted according to the actual needs of the power plant.
5. The method according to claim 1, characterized in that, In step S4, the edge computing gateway collects real-time data from field sensors, runs a strategy model for rapid reasoning, and sends the optimal action command to the PLC control system to achieve real-time closed-loop control of the physical ash conveying system. When the edge computing gateway fails, communication is interrupted, or an abnormal command is issued, the system automatically switches back to the original PLC control logic.
6. The method according to claim 1, characterized in that, It also includes step S5: using actual operating data, the digital twin model and reinforcement learning strategy are incrementally trained on a regular basis to achieve continuous self-evolution of the system; at the same time, based on the digital twin model and real-time data, anomaly detection algorithms are used to monitor the equipment status, realize fault warning and remaining life prediction, and automatically generate predictive maintenance plans.
7. A closed-loop self-optimizing control system for an ash conveying system based on digital twins and reinforcement learning, characterized in that, include: The data acquisition module is used to collect real-time operating data of the ash conveying system; A digital twin model building module for building a high-fidelity digital twin model as described in claim 1; A reinforcement learning training module is used to train optimal control strategies in a digital twin environment; Edge computing gateways are used to deploy trained policy models, perform real-time inference, and issue control commands. The PLC control system is used to receive instructions from the edge computing gateway and execute physical control, while retaining a safe redundancy switching mechanism. The collaborative optimization interface is used for data interaction and collaborative optimization with the boiler combustion system, the whole plant air compressor station, and the ash and slag sales system.
8. The system according to claim 7, characterized in that, The data acquisition module includes a pressure sensor and a position sensor deployed on the pilot valve, a weighing sensor or level gauge deployed on the silo pump, an acoustic emission sensor or wall thickness monitoring sensor deployed at the pipe bend, and a flow meter deployed on the air compressor.
9. The system according to claim 7, characterized in that, The collaborative optimization interface includes: The interface with the boiler combustion system is used to acquire online coal quality analysis data and unit load commands; The interface with the plant's air compressor station is used to obtain the pressure, flow rate, and operating status of each air compressor in the compressed air pipeline network; The interface with the ash sales system is used to obtain downstream users' demand information for ash.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.
