An adaptive control method and system for a negative pressure pneumatic conveying system for stones and coal

By combining offline pre-training and online fine-tuning of deep reinforcement learning agents with multi-objective reward functions and hard constraint safety checks, the adaptive capability problem of the negative pressure pneumatic conveying system for stone coal under coal quality fluctuations and operating condition changes was solved, and the system was able to operate efficiently and reliably.

CN123105285APending Publication Date: 2026-11-17HEBEI QIANLU ELECTRIC POWER EQUIPMENT CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611013372.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-08
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing control methods for negative pressure pneumatic conveying systems for stone and coal cannot effectively adapt to fluctuations in coal quality and changes in operating conditions. This results in poor controller adaptability, affecting the system's operational economy and reliability, and increasing the risk of pipe blockage accidents and equipment wear.

Method used

We employ deep reinforcement learning agents for offline pre-training, construct multi-objective reward functions, and combine shadow mode and progressive switching mechanisms to achieve online fine-tuning of the agents in real systems. Through hard constraint security checks and dynamic adjustment of multi-objective reward functions, we optimize delivery stability, energy consumption, and equipment wear.

Benefits of technology

It significantly reduces transmission energy consumption by approximately 5-15%, reduces equipment wear by approximately 10-20%, reduces the frequency of pipe blockage events by more than 50%, improves system adaptability and operational reliability, and meets the personalized operation needs of power plants.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123105285A_ABST
    Figure CN123105285A_ABST
Patent Text Reader

Abstract

This invention relates to the field of control technology, and more particularly to an adaptive control method and system for a negative pressure pneumatic conveying system for gravel and coal. The method first constructs a dynamic simulation environment for the system based on historical operating data, and performs offline pre-training of a deep reinforcement learning agent within the simulation environment to generate a pre-trained control strategy. A multi-objective reward function is constructed to guide the agent to consider multiple optimization objectives during the control process. The pre-trained control strategy is deployed to the real system, and the agent is fine-tuned online using real operating data collected in shadow mode. Control is smoothly transferred from the original controller to the agent. Hard constraint safety checks are performed on the control actions output by the agent, and the control priority mode of the agent is dynamically switched according to the weight coefficients of the real-time adjusted multi-objective reward function. This invention can significantly reduce conveying energy consumption, reduce equipment wear, and simultaneously achieve flexible trade-offs between multiple objectives and improve system operational safety.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of control technology, and in particular to an adaptive control method and system for a negative pressure pneumatic conveying system for gravel and coal. Background Technology

[0002] The negative pressure pneumatic conveying system for pulverized coal is a key auxiliary equipment in coal mill units of thermal power plants, widely used in the recovery and conveying of pulverized coal particles. Related technologies utilize the coordinated operation of Roots blowers, conveying pipelines, and valve assemblies to construct a gas-solid two-phase flow conveying system driven by negative pressure. Specifically, the system's control encompasses the entire process from blower speed regulation and pipeline negative pressure stabilization to material flow rate monitoring, including typical schemes such as fixed PID control, cascade PID control, and expert rule control, aiming to ensure the continuity and safety of the conveying task.

[0003] However, existing control methods directly adopt parameter tuning based on fixed models or human experience, without considering the impact of dynamic operating conditions such as coal quality fluctuations and changes in slag discharge on system characteristics. This may lead to poor controller adaptability, a sharp decline in performance or even instability when operating conditions deviate, or difficulty in balancing multiple optimization objectives such as conveying stability, energy efficiency and equipment lifespan, thereby affecting the system's operating economy and reliability, and increasing the risk of pipe blockage accidents and equipment wear. Summary of the Invention

[0004] The present invention aims to at least partially solve one of the technical problems in the related art.

[0005] Therefore, the first objective of this invention is to propose an adaptive control method for a negative pressure pneumatic conveying system for gravel and coal, comprising the following steps:

[0006] S1. Construct a dynamic simulation environment for the system based on historical operating data, and perform offline pre-training on the deep reinforcement learning agent in the simulation environment to generate a pre-trained control strategy. S2, Construct a multi-objective reward function, which includes at least a delivery stability reward term, an energy consumption penalty term, and a device wear penalty term, to guide the agent to take into account multiple optimization objectives during the control process; S3, the pre-trained control strategy is deployed to the real system, real running data is collected through shadow mode to fine-tune the agent online, and a gradual switching mechanism is used to smoothly transfer control from the original controller to the agent; S4. During the entire cycle of the intelligent agent's execution control, hard constraint safety verification is performed on the control actions output by the intelligent agent, and the control priority mode of the intelligent agent is dynamically switched according to the weight coefficient of the multi-objective reward function adjusted in real time.

[0007] In one embodiment of the present invention, S1 includes: A data-driven agent model is constructed using a long short-term memory network based on historical runtime sequence data, and Monte Carlo Dropout technology is introduced to quantify and evaluate the prediction uncertainty of the agent model. Using the agent model that has completed training and uncertainty assessment as an offline simulation environment, the agent is pre-trained offline using a deep deterministic policy gradient algorithm to generate the pre-trained control policy.

[0008] In one embodiment of the present invention, the method of constructing a data-driven agent model based on historical runtime sequence data using a long short-term memory network is as follows: Historical operating sequence data of the negative pressure conveying system for stone and coal in a power plant are collected and divided into input sequence and output sequence according to time series. The input sequence is the system state vector and control action vector at the current moment, and the output sequence is the system state vector at the next moment. A two-layer long short-term memory network architecture is adopted, with the input sequence as the model input, the output sequence as the supervision label, and the mean squared error as the loss function to supervise the training of the long short-term memory network.

[0009] In one embodiment of the present invention, the system state vector includes: negative pressure of the coal mill outlet pipeline, wind speed of the conveying pipeline, stone coal feed rate, carbon content of stone coal, operating current of the Roots blower, blower outlet temperature, valve opening degree, and material-to-gas ratio; The control action vector includes: the adjustment amount of the Roots blower speed and the adjustment amount of the conveying valve opening.

[0010] In one embodiment of the present invention, S4 includes: The agent policy network, which has been pre-trained offline, is deployed to the DCS control system of the power plant. In shadow mode, the agent collects the current operating status of the system and outputs suggested control actions in each control cycle. However, the actions are not directly sent to the actuators, but are only recorded in the operation log in sync with the actual execution actions of the original controller and the system status change data. Based on the real operating data accumulated in the shadow mode, the real working condition samples collected online are added to the experience replay buffer of the deep deterministic policy gradient algorithm, mixed with offline pre-training data for training, and the learning rate is reduced to 10% of the offline training to fine-tune the agent policy network online. A gradual switching mechanism is adopted to transfer control from the original controller to the intelligent agent in stages.

[0011] In one embodiment of the present invention, the total reward function of the multi-objective reward function in step S4 is defined as:

[0012] in, R The total reward value for a single control cycle. , , These are the weighting coefficients for the three optimization objectives: transport stability, energy efficiency, and equipment wear. R_transport is the reward for transport stability, R_energy consumption is the penalty for energy consumption, and R_wear is the penalty for equipment wear.

[0013] In one embodiment of the present invention, the conveying stability reward item R conveying is given a positive reward when the system conveys normally and no pipe blockage alarm is triggered within a single control cycle, and a large negative penalty is given when a pipe blockage alarm is triggered; The formula for calculating the energy consumption penalty term R is as follows:

[0014] in, This represents the current real-time operating power of the wind turbine. This refers to the rated power of the fan. This is the energy consumption penalty coefficient; The formula for calculating the equipment wear penalty term R wear is as follows:

[0015] in, This is an estimate of the particle impact velocity at the bend in the pipe. This represents the wear penalty coefficient.

[0016] In one embodiment of the present invention, the estimated particle impact velocity at the bend in the pipe is... v impact The calculation formula is based on the estimation of pipeline air velocity and material-to-air ratio:

[0017] in, This represents the real-time airflow velocity inside the duct. For solid phase material flow rate, This refers to the gas phase air flow rate. and These are coefficients based on the pipeline structure calibration.

[0018] To achieve the above objectives, a second aspect of the present invention provides an adaptive control system for a negative pressure pneumatic conveying system for gravel and coal, comprising: The offline pre-training module is used to construct a dynamic simulation environment for the system based on historical running data, and to perform offline pre-training on the deep reinforcement learning agent in the simulation environment to generate pre-trained control strategies. A multi-objective reward configuration module is used to construct a multi-objective reward function, which includes at least a delivery stability reward item, an energy consumption penalty item, and a device wear penalty item, so as to guide the agent to take into account multiple optimization objectives during the control process; The online smooth iterative control module is used to deploy the pre-trained control strategy to the real system, collect real running data through shadow mode to fine-tune the agent online, and use a progressive switching mechanism to smoothly transfer control from the original controller to the agent. The safety verification and dynamic control module is used to perform hard constraint safety verification on the control actions output by the agent throughout the entire cycle of the agent's execution control, and dynamically switch the control priority mode of the agent according to the weight coefficient of the multi-objective reward function adjusted in real time.

[0019] To achieve the above objectives, a third aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0020] The methods, systems, and storage media of this invention, and the beneficial effects of this application include: 1. Significantly reduce transmission energy consumption Through intelligent optimization control using deep reinforcement learning, the intelligent agent can dynamically adjust the fan speed and valve opening according to real-time operating conditions, avoiding excessive overshoot and steady-state error inherent in traditional PID control. Compared to fixed logic control, the DRL adaptive method can reduce transmission energy consumption by approximately 5% to 15%, demonstrating significant energy-saving effects.

[0021] 2. Effectively reduces equipment wear and tear. The wear penalty term Rwear in the multi-objective reward function guides the agent to appropriately reduce the particle impact velocity at the bend, while meeting the conveying conditions. Compared to traditional control strategies, the DRL adaptive method can reduce the bend wear rate by approximately 10% to 20%, extend equipment maintenance cycles, and reduce maintenance costs.

[0022] 3. Significantly reduce the frequency of pipe blockage events. By learning from historical data about early signs of pipe blockage, the intelligent agent can adjust control actions in advance to maintain pipeline pressure and material flow rate within safe ranges before blockage occurs. Compared to passive response control, the DRL adaptive method can reduce the frequency of blockage events by more than 50%, improving the reliability and continuity of system operation.

[0023] 4. Enhance the adaptive capability of the control system. Traditional PID control uses fixed parameters, making it unable to adapt to changes in coal quality and fluctuations in operating conditions. This invention utilizes deep reinforcement learning, enabling the agent to continuously learn and adapt to dynamic changes in the system, maintaining good control performance even under significant changes in coal quality.

[0024] 5. Achieving a flexible trade-off among multiple objectives By using sliding bars that prioritize energy saving, output, and maintenance, operators can flexibly adjust the weights of control objectives, and the intelligent agent can smoothly switch strategies within approximately one second. This design not only meets the personalized operational needs of power plants but also avoids the complex parameter tuning problems inherent in multi-objective optimization.

[0025] 6. Ensure the security of system operation. A three-layer safety mechanism, consisting of offline pre-training, shadow mode, and a hard constraint checker, ensures the safety of the agent's policy throughout the entire process from training to deployment. Monte Carlo Dropou uncertainty assessment enables the system to identify state regions where the model is not proficient and automatically revert to a conservative policy, avoiding dangerous exploratory behavior.

[0026] 7. Promote the intelligent upgrading of the stone and coal negative pressure conveying system. This invention directly serves the design optimization, equipment selection research, and engineering demonstration application tasks of the research and development project for coal powder particle recovery technology and equipment in coking coal, providing core technical support for the intelligent operation of the negative pressure pneumatic conveying system for coking coal. Intelligent control is an important technical foundation for achieving efficient separation of coking coal and recovery of coal powder particles, and is of great significance for improving resource utilization, reducing power plant operating costs, and achieving environmentally compliant emissions.

[0027] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0028] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of an adaptive control method for a negative pressure pneumatic conveying system for stones and coal according to an embodiment of the present invention; Figure 2 This is a structural diagram of an adaptive control system for a negative pressure pneumatic conveying system for stones and coal according to an embodiment of the present invention; Figure 3 This invention relates to an adaptive control method for a negative pressure pneumatic conveying system for gravel and coal based on deep reinforcement learning. Detailed Implementation

[0029] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] The following describes, with reference to the accompanying drawings, an adaptive control method and system for a negative pressure pneumatic conveying system for gravel and coal according to an embodiment of the present invention.

[0032] Example 1 Figure 1 This is a flowchart of an adaptive control method for a negative pressure pneumatic conveying system for gravel and coal according to an embodiment of the present invention.

[0033] like Figure 1 As shown, the adaptive control method for the negative pressure pneumatic conveying system for gravel and coal includes the following steps: S1. Construct a dynamic simulation environment for the system based on historical operating data, and perform offline pre-training on the deep reinforcement learning agent in the simulation environment to generate a pre-trained control strategy.

[0034] First, a dynamic simulation environment is constructed based on historical operational data. This environment simulates the dynamic response characteristics of the negative pressure pneumatic conveying system for stone and coal, providing a safe and repeatable offline training platform for the deep reinforcement learning agent. Specifically, using historical operational sequence data, a surrogate model reflecting the system's state transition patterns is established through a data-driven approach. This model can predict the system state change at the next moment based on the current system state and the applied control actions, thus approximating the dynamic behavior of the real system. Based on this, the predictive reliability of the surrogate model is evaluated, identifying its confidence level in different operating conditions. When the model's prediction uncertainty for the current operating condition exceeds a preset threshold, a safety protection mechanism is triggered to prevent dangerous explorations caused by model inaccuracies. Subsequently, using this surrogate model as the simulation environment, a deep reinforcement learning algorithm suitable for continuous action spaces is employed for offline pre-training of the agent. The agent continuously optimizes its policy network parameters through interactive loops of state perception, action decision-making, and reward feedback within the simulation environment, ultimately converging to the pre-trained control strategy that maximizes cumulative rewards.

[0035] As one implementation method, a long short-term memory network can be used to construct an agent model, and Monte Carlo Dropout technology can be introduced to quantify the uncertainty of prediction. At the same time, a deep deterministic policy gradient algorithm can be used to complete the pre-training of the agent in an offline simulation environment.

[0036] Through the above steps, a pre-trained strategy with basic control capabilities can be efficiently generated using historical data in a safe environment completely detached from the real system. This fundamentally avoids the safety risks and equipment damage caused by trial and error training directly on industrial systems, laying a safe and reliable foundation for subsequent online deployment and optimization.

[0037] S2, Construct a multi-objective reward function, which includes at least a delivery stability reward term, an energy consumption penalty term, and a device wear penalty term, to guide the agent to take into account multiple optimization objectives during the control process.

[0038] This method constructs a multi-objective reward function, configured to include at least a delivery stability reward, an energy consumption penalty, and an equipment wear penalty, to provide comprehensive optimization guidance for the deep reinforcement learning agent during its learning and decision-making process. Specifically, this multi-objective reward function integrates multiple performance indicators reflecting different dimensions of system operation into a unified scalar reward value through weighted summation or similar combinations. This transforms potentially conflicting control objectives, such as ensuring the continuity and stability of material delivery, reducing the energy consumed by the system, and slowing down the wear rate of critical equipment like pipelines, into a single objective that can be directly optimized by the agent. The system employs several reward mechanisms. The first, a transport stability reward, evaluates the quality and reliability of the system's transport tasks within a unit control cycle. Positive incentives are given when the system operates smoothly without any abnormal events, while negative penalties are imposed for unsatisfactory performance. The second, an energy consumption penalty, quantifies the real-time operating power or energy consumption level of the system's main energy-consuming equipment, guiding the agent to minimize unnecessary energy expenditure while meeting transport requirements. The third, an equipment wear penalty, estimates or monitors operating parameters related to equipment wear, such as the flow velocity or particle impact characteristics of the gas-solid two-phase flow within the pipeline, penalizing behaviors that may accelerate wear and guiding the agent to adopt smoother, more equipment-friendly control actions. In this way, the multi-objective reward function provides clear numerical guidance for the agent to autonomously weigh and optimize multiple mutually constraining performance indicators under complex dynamic conditions.

[0039] By constructing a reward function that includes multiple dimensions such as transmission stability, energy consumption, and equipment wear, this method can guide a deep reinforcement learning agent to automatically seek the optimal balance between multiple optimization objectives during the control process. This overcomes the limitations of traditional single-objective control strategies, effectively reduces operating energy consumption and delays the wear of key equipment while ensuring stable system operation, and significantly improves the overall operational economy and reliability of the system.

[0040] S3, the pre-trained control strategy is deployed to the real system, real operating data is collected in shadow mode to fine-tune the agent online, and a gradual switching mechanism is used to smoothly transfer control from the original controller to the agent.

[0041] After deploying the pre-trained control strategy to the real system, a secure interaction between the agent and the real system is first achieved through a shadow mode. In this mode, the agent collects the system's operating status in real time and generates suggested control actions based on the pre-trained strategy network. However, these actions are not directly issued to the actuators but are recorded synchronously with the actual execution actions of the original controller and system status change data. The original system continues to operate in the predetermined control mode, thereby accumulating operating data under real working conditions without affecting normal production. When the number of valid samples collected reaches a preset number, the agent's strategy network is fine-tuned online using this real operating data. By adding real samples to the experience replay buffer and mixing them with offline pre-trained data for training, while reducing the learning rate to avoid catastrophic forgetting of the pre-trained strategy, the strategy network can quickly adapt to the dynamic characteristics of the real system. After completing the online fine-tuning, a gradual switching mechanism is used to smoothly transition control from the original controller to the agent. This mechanism gradually increases the proportion of the agent's control cycle in stages. Only after each stage is running stably and the operating indicators meet the standards can the next stage be entered, until the agent achieves complete autonomous control, thereby ensuring system safety and operational stability during the control switching process.

[0042] As one implementation method, the shadow mode can run for several weeks to collect enough samples. When fine-tuning online, the learning rate can be reduced to one-tenth of that in offline training. The gradual switching can be set to multiple stages, for example, the agent can execute most of the control cycles while the original controller executes the remaining parts. After stabilization, the control ratio of the agent can be gradually increased to finally achieve fully autonomous control.

[0043] This step achieves secure online adaptation and data accumulation of pre-trained strategies through shadow mode, combined with online fine-tuning to enable the strategy network to quickly adapt to the dynamic characteristics of the real system, and adopts a gradual switching mechanism to ensure the smoothness and security of the control transition. This effectively solves the domain adaptation problem between the offline simulation environment and the real industrial system, avoids the operational risks brought about by direct deployment of strategies, and improves the reliability and practicality of the control system.

[0044] S4. During the entire cycle of the intelligent agent's execution control, hard constraint safety verification is performed on the control actions output by the intelligent agent, and the control priority mode of the intelligent agent is dynamically switched according to the weight coefficient of the multi-objective reward function adjusted in real time.

[0045] Throughout the entire control cycle executed by the intelligent agent, to ensure that the system operation always remains within the safety boundary, a hard constraint safety check is applied to each control action output by the intelligent agent. This hard constraint safety check is based on the system's rated design parameters or physical limits, pre-setting an insurmountable safety boundary. When the action parameters output by the intelligent agent exceed this safety boundary, the check mechanism automatically corrects the action to a safe value within the boundary range, thereby ensuring that the control commands issued to the actuators are always within the safe range and eliminating the risk of exceeding limits due to the intelligent agent's exploratory decisions or model biases. Simultaneously, to adapt to the dynamically changing operational needs of the power plant under all operating conditions, the weight coefficients of the multi-objective reward function support real-time adjustment. The intelligent agent dynamically switches its control priority mode according to the adjusted weight coefficients. Specifically, the weight coefficients corresponding to each optimization objective are modified in real time through a visual interactive interface or external commands. After normalization, the intelligent agent recalculates the reward signal based on the new weight allocation in subsequent control cycles, thereby smoothly adjusting its control strategy tendency and achieving seamless switching between different operating modes (such as prioritizing transmission stability, prioritizing energy consumption reduction, or prioritizing equipment wear control) without requiring retraining of the strategy network.

[0046] As one implementation method, hard constraint safety verification can set upper and lower limits for key parameters such as fan speed, pipeline negative pressure, valve opening and material flow rate. When the action output by the agent exceeds any constraint, a trimming correction is performed. The real-time adjustment of the weight coefficient can be achieved through the slider component. After adjustment, the system automatically performs normalization processing to ensure that the sum of weights is constant. The agent can complete the switching of strategy preference in the next control cycle.

[0047] This technical step, through a collaborative mechanism of hard constraint safety verification and dynamic weight adjustment, enables the control strategy to adapt to changes in field conditions in real time while ensuring the inherent safety of system operation. This significantly improves the flexibility and robustness of the control system, avoids safety accidents caused by model mismatch or sudden changes in operating conditions, and meets the flexible switching requirements of power plants for multi-objective operation.

[0048] Example 2 To further illustrate the method proposed in this invention, specific embodiments are described below.

[0049] like Figure 3As shown, this invention addresses the technical shortcomings of existing negative pressure pneumatic conveying systems for stone and coal, such as poor adaptability, difficulty in simultaneously addressing multiple objectives, uncontrollable risks of pipe blockage, and severe equipment wear. It proposes an adaptive control method for these systems based on deep reinforcement learning (DRL). The core of this method lies in using the Deep Deterministic Policy Gradient (DDPG) algorithm as the control core. This involves offline pre-training to construct a basic control strategy, online safety fine-tuning to implement the strategy, dynamic adjustment of multiple objectives to adapt to on-site operational needs, and a comprehensive safety protection mechanism. This enables adaptive closed-loop optimization control of the conveying system's fan speed and valve opening.

[0050] S1, Offline simulation environment setup and policy pre-training.

[0051] The core objective of this phase is to address the security risks associated with training deep reinforcement learning directly on real industrial systems. By constructing a high-precision dynamic simulation environment using historical operational data, the control strategy can be pre-trained offline, laying a secure foundation for online deployment.

[0052] First, a data-driven surrogate model for a negative-pressure pneumatic conveying system for coal and stone is constructed using a Long Short-Term Memory (LSTM) network. The gating structure of LSTM can accurately capture the long-term dependencies of time-series operational data, enabling it to accurately predict system state changes at the next moment based on the current system operating state and applied control actions. This invention collects historical operational sequence data of the negative-pressure pneumatic conveying system for coal and stone in a power plant, dividing it into input and output sequences according to the time series. The input sequence contains the system state vector and control action vector at the current moment, while the output sequence contains the system state vector at the next moment. This process completes the supervised training of the LSTM surrogate model, making it approximate the dynamic characteristics of the real conveying system.

[0053] Secondly, addressing the inherent model error between the surrogate model and the real system, as well as the inconsistency in model prediction accuracy across different operating conditions, this invention introduces Monte Carlo Dropout technology to quantitatively evaluate the prediction uncertainty of the LSTM surrogate model. Specifically, the Dropout mechanism is retained during the LSTM network inference stage, performing multiple forward propagations on the same set of input states and actions, and calculating the prediction uncertainty based on the variance of the multiple output results; the larger the variance, the lower the model's prediction confidence for that operating condition. This invention presets an uncertainty threshold; when the surrogate model's prediction uncertainty for the current operating condition exceeds the threshold, the control strategy automatically switches to a conservative PID control strategy, avoiding dangerous exploration caused by model inaccuracies and ensuring the inherent safety of offline training and subsequent online operation.

[0054] Finally, using the LSTM agent model that has completed training and uncertainty assessment as the offline simulation environment, the DDPG algorithm is employed to pre-train the deep reinforcement learning agent (hereinafter referred to as the agent) offline. The DDPG algorithm is adapted to the control requirements of continuous action spaces in industrial scenarios. Combining the environmental perception capabilities of deep learning with the continuous decision-making capabilities of the policy gradient method, the agent continuously optimizes the policy network parameters through ongoing state exploration, action execution, and reward feedback in the simulation environment, ultimately converging to the optimal pre-trained control policy that maximizes cumulative rewards, thus completing the offline training phase.

[0055] S2, Construction of multi-objective reward function.

[0056] The reward function is the optimization objective and learning guide of the deep reinforcement learning agent, which directly determines the convergence direction and final control effect of the control strategy. In this stage, in view of the operation requirements of the stone and coal negative pressure pneumatic conveying system, a multi-objective reward function that takes into account the conveying stability, energy efficiency and equipment life is constructed to solve the technical pain point of traditional control methods that it is difficult to take multiple objectives into account.

[0057] The total reward function of this invention is defined as follows:

[0058] In the formula, R is the total reward value for a single control cycle. , , These are the weighting coefficients for three optimization objectives: conveying stability, energy efficiency, and equipment wear. The weighting coefficients can be flexibly configured on-site. Conveying is a reward item for conveying stability, energy consumption is a penalty item for energy consumption, and wear is a penalty item for equipment wear.

[0059] Among them, conveying is used to quantify the completion quality of conveying tasks and the stability of system operation. When materials are conveyed within a set period without pipe blockage alarms, a positive reward is given, and when a pipe blockage alarm occurs, a large negative penalty is given. Energy consumption is used to quantify the energy consumption level of the system. It is calculated based on the real-time operating power of the Roots blower. The higher the power, the larger the penalty value, guiding the agent to reduce unnecessary energy consumption. Wear is used to quantify the wear rate of pipeline equipment. It estimates the particle impact speed at the bend based on the wind speed and material-to-air ratio in the pipeline. The higher the impact speed, the larger the penalty value, guiding the agent to reduce equipment wear while meeting conveying requirements.

[0060] Weighting coefficient , , Power plant operators can flexibly adjust the system according to their production needs, equipment status, and performance indicators. The system can achieve dynamic switching of control objectives without retraining the strategy network, adapting to the operational needs of different scenarios.

[0061] S3 offers online security fine-tuning and smooth switching of control.

[0062] The core objective of this phase is to address the domain adaptation issue between offline simulation environments and real industrial systems. Through a three-layer mechanism of shadow mode, progressive control switching, and hard constraint security protection, pre-trained strategies can be safely and reliably deployed from offline simulation to real systems, avoiding operational risks associated with direct strategy deployment.

[0063] First, a shadow mode is used to complete the online adaptation and data accumulation of the strategy. The agent strategy network, which has been pre-trained offline, is deployed to the power plant's DCS control system. During the shadow mode operation, the agent collects the current operating status of the system in real time during each control cycle, inputs the suggested control actions into the strategy network, but these actions are not directly issued to the actuators. Instead, they are synchronously recorded in the operation log along with the actual execution actions of the original controllers and system status change data. The original system continues to operate in the established PID control or manual operation mode, without affecting the normal production of the unit. The shadow mode is continuously run for 2-4 weeks, accumulating no less than 10,000 valid control samples to complete online data accumulation.

[0064] Secondly, based on real-world operational data accumulated using the shadow model, the agent policy network is fine-tuned online. Real-world operational samples collected online are added to the experience replay buffer of the DDPG algorithm and mixed with offline pre-training data for training. During fine-tuning, the learning rate is reduced to 10% of that used in offline training to prevent catastrophic forgetting of the pre-trained policy, enabling the policy network to quickly adapt to the dynamic characteristics of the real system and complete online optimization.

[0065] Finally, a gradual switching mechanism is adopted to achieve a smooth transition of control, simultaneously accompanied by a hard constraint safety check mechanism. The control switching is implemented in three phases: In the first phase, 70% of the control cycle is executed by the agent, and 30% of the control cycle is executed by the original controller. After one week of stable operation and meeting the operational indicators, the process proceeds to the next phase. In the second phase, 85% of the control cycle is executed by the agent, and 15% of the control cycle is executed by the original controller. After one week of stable operation and without any safety anomalies, the process proceeds to the final phase. In the third phase, 100% of the control cycle is autonomously controlled by the agent. Throughout the entire control cycle in which the agent directly executes the control, the output action of each control cycle must undergo safety verification by the hard constraint checker. The hard constraint checker sets an inviolable safety boundary based on the system's rated design parameters. When the agent's output action exceeds the safety boundary, the checker automatically trims the action back to within the constraint boundary range, ensuring that the executed action always remains within the safe range and preventing safety incidents.

[0066] S4, dynamic adjustment of multi-objective control strategy.

[0067] The core objective of this stage is to adapt to the dynamic operating requirements of power plants under all operating conditions, solve the problem that traditional control strategies cannot flexibly adjust and optimize objectives according to production needs, and achieve real-time and seamless switching of control strategies.

[0068] During power plant operation, the core operational requirements vary significantly under different conditions: during peak shaving, priority must be given to ensuring transmission stability to avoid pipe blockage affecting unit load response; during routine base load operation, priority must be given to reducing transmission energy consumption to meet power plant energy conservation assessment requirements; and before unit maintenance, priority must be given to controlling pipeline wear to extend equipment maintenance cycles. To address these dynamic requirements, this invention designs a visual weight adjustment interface, allowing operators to adjust weights in real time using a slider component within the interface. , , The relative magnitudes of the three weighting coefficients correspond to switching between different control priority modes: increasing... The weighted output priority mode prioritizes ensuring stable transportation and high task completion rates; it also increases... The weight corresponds to the energy-saving priority mode, prioritizing the reduction of wind turbine operating energy consumption; increasing The weight corresponds to the pipe protection priority mode, which prioritizes controlling the pipe wear rate.

[0069] After the weight coefficients are adjusted, the system automatically normalizes the weights to ensure the stability of the weight allocation logic. The normalization formula is as follows:

[0070] The normalized weights are passed to the reward function calculation module, and the agent can smoothly adjust the control strategy tendency within the next control cycle (≤1s) without retraining the policy network, thus achieving real-time and seamless switching of control objectives and greatly improving the flexibility and adaptability of the control system.

[0071] The specific implementation steps are as follows: S41, LSTM surrogate model The LSTM proxy model adopts a two-layer LSTM network architecture, with the following specific configuration: Input layer: The input vector is a concatenated vector of the system state vector and action vector at the current moment, with a dimension of 10. The state vector is 8-dimensional and specifically includes: negative pressure of the coal mill outlet pipeline, wind speed of the conveying pipeline, feed rate of gravel and coal, carbon content of gravel and coal, operating current of the Roots blower, blower outlet temperature, valve opening degree, and material-to-gas ratio; the action vector is 2-dimensional and specifically includes: adjustment amount of Roots blower speed and adjustment amount of conveying valve opening degree.

[0072] Network hidden layers: A two-layer LSTM structure is adopted, with 128 hidden units in each layer. The length of the input time series is set to 10, that is, the time series data of 10 consecutive control cycles is used as the model input. A Dropout layer is set between the two LSTM layers, with a Dropout rate of 0.2, which is used to improve the generalization ability of the model and provide a basis for Monte Carlo Dropout uncertainty assessment.

[0073] Output layer: A fully connected layer is used to output the 8-dimensional system state prediction value at the next time step, which corresponds to the dimension of the state vector of the input layer.

[0074] The model was trained using the Adam optimizer with an initial learning rate of 0.001, a training batch size of 64, and a maximum training epoch of 200. Mean squared error (MSE) was used as the loss function for model training, and the formula for the loss function is as follows:

[0075] After training is complete, save the model weight file for use in building an offline simulation environment.

[0076] S42, DDPG algorithm The Actor network structure consists of an 8-dimensional system state vector as input and a 2-dimensional continuous action vector as output. The network hierarchy is as follows: input layer → first fully connected layer (256 neurons, ReLU activation function) → second fully connected layer (128 neurons, ReLU activation function) → output layer (tanh activation function, output value normalized to the range of [-1,1]). Finally, through linear transformation, the normalized output is mapped to the actual action adjustment range of fan speed and valve opening.

[0077] The Critic network structure consists of an input vector (10 dimensions in total) concatenated from an 8-dimensional state vector and a 2-dimensional action vector, and an output vector (Q-value scalar) corresponding to the state-action pair. The network layers are: input layer → first fully connected layer (256 neurons, ReLU activation function) → second fully connected layer (128 neurons, ReLU activation function) → output layer (linear activation, output Q-value scalar).

[0078] Training support mechanisms: An experience replay buffer is set up with a capacity of 100,000 experience samples. In each training session, 64 samples are randomly selected from the buffer for gradient updates to break the temporal correlation between samples and improve training stability. A target network soft update mechanism is adopted. Both Actor and Critic are set with corresponding target networks. The soft update coefficient τ is set to 0.005. The target network parameters are updated once in each training step to avoid oscillations and divergence during the training process.

[0079] S43, Monte Carlo Dropout Uncertainty Assessment During the inference phase of the LSTM surrogate model, the Dropout layer is always kept on, and the random deactivation mechanism is not turned off. For the same set of input state and action vectors, perform T=30 forward propagations and collect the state prediction results of the model output 30 times. Calculate the mean and variance of the 30 prediction results. The formula for calculating the variance is as follows:

[0080] In the formula, To predict the variance of the results, Forward propagation count, For the first The prediction results of this time, for The mean of the prediction results.

[0081] The preset uncertainty threshold is 0.1. When the prediction variance exceeds 0.1, it is determined that the prediction confidence of the current operating condition area model is too low, and the system automatically switches to a conservative PID control strategy. The conservative PID strategy is a single-loop closed-loop control for pipeline negative pressure with the PID parameters tuned as follows: proportional coefficient Kp=2.0, integral time Ti=10s, and derivative time Td=0.5s, to ensure the stability and safety of the control process.

[0082] S44, Specific calculation formula for the multi-objective reward function Delivery stability reward item delivery: Delivery is carried out when the system delivers normally and no pipe blockage alarm is triggered within a single control cycle; when a pipe blockage alarm is triggered, delivery is carried out, and the agent is guided to avoid pipe blockage risk through a large penalty.

[0083] Energy consumption penalty: The calculation formula is as follows

[0084] in This represents the current real-time operating power of the wind turbine. This refers to the rated power of the fan. This is the energy consumption penalty coefficient, with a default value of 5.

[0085] Wear penalty item: The calculation formula is as follows

[0086] in The estimated particle impact velocity at the bend in the pipe is calculated using the pipe air velocity and the material-to-air ratio, using the following formula:

[0087] In the formula, This represents the real-time airflow velocity inside the duct. For solid phase material flow rate, is the gas-phase air flow rate, , is the coefficient calibrated based on the pipeline structure; is the wear penalty coefficient, with a default value set to 2.

[0088] S45, hard constraint checker and weight adjustment , the constraint conditions of the hard constraint checker: based on the rated design parameters of the system, the following non-breakable safety boundaries are set: the rotational speed range of the Roots blower is 40%-110% of the rated rotational speed; the negative pressure range of the conveying pipeline is -55kPa~-5kPa; the opening range of the conveying valve is 5%-95%; the material flow velocity in the pipeline does not exceed 90% of the designed maximum flow velocity. The trimming and correction logic is: when the action a output by the agent is greater than amax, the corrected action acorrected = amax; when a is less than amin, acorrected = amin; in other cases, the original action remains unchanged, and the corrected action is issued to the actuator for execution.

[0089] Implementation of dynamic weight adjustment: three sliders are set on the control interface, respectively corresponding to , , , the value range of the sliders is 0.0~1.0; after the values of the sliders are adjusted, the system automatically executes the normalization process described above to ensure that the sum of the weights is always 1; the normalized weights are transmitted to the reward function calculation module in real time, and the agent can complete the policy adjustment in the next control cycle without retraining.

[0090] Embodiment 3 As Figure 2 shown is an adaptive control system 10 for a stone coal negative-pressure pneumatic conveying system proposed by the present invention, comprising: An offline pre-training module 100, configured to construct a system dynamic simulation environment based on historical operation data, and perform offline pre-training on a deep reinforcement learning agent in the simulation environment to generate a pre-trained control policy; A multi-objective reward configuration module 200, configured to construct a multi-objective reward function, where the multi-objective reward function at least includes a conveying stability reward term, an energy consumption penalty term, and an equipment wear penalty term, so as to guide the agent to take multiple optimization objectives into account during the control process; An online smooth iterative control module 300, configured to deploy the pre-trained control policy to a real system, collect real operation data through a shadow mode to perform online fine-tuning on the agent, and adopt a progressive switching mechanism to smoothly transfer the control right from the original controller to the agent; The security verification dynamic control module 400 is used to perform hard constraint security verification on the control actions output by the intelligent agent throughout the entire cycle of the intelligent agent's execution control, and dynamically switch the control priority mode of the intelligent agent according to the weight coefficient of the multi-objective reward function adjusted in real time.

[0091] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the adaptive control method of the above-mentioned negative pressure pneumatic conveying system for stone and coal.

[0092] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0093] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. An adaptive control method for a negative pressure pneumatic conveying system for stones and coal, characterized in that, Includes the following steps: S1. Construct a dynamic simulation environment for the system based on historical operating data, and perform offline pre-training on the deep reinforcement learning agent in the simulation environment to generate a pre-trained control strategy. S2, construct a multi-objective reward function, which includes at least a delivery stability reward term, an energy consumption penalty term, and a device wear penalty term, to guide the agent to take into account multiple optimization objectives during the control process; S3, the pre-trained control strategy is deployed to the real system, real running data is collected through shadow mode to fine-tune the agent online, and a gradual switching mechanism is used to smoothly transfer control from the original controller to the agent; S4. During the entire cycle of the intelligent agent's execution control, hard constraint safety verification is performed on the control actions output by the intelligent agent, and the control priority mode of the intelligent agent is dynamically switched according to the weight coefficient of the multi-objective reward function adjusted in real time.

2. The method as described in claim 1, characterized in that, S1 includes: A data-driven agent model is constructed using a long short-term memory network based on historical runtime sequence data, and Monte Carlo Dropout technology is introduced to quantify and evaluate the prediction uncertainty of the agent model. Using the agent model that has completed training and uncertainty assessment as an offline simulation environment, the agent is pre-trained offline using a deep deterministic policy gradient algorithm to generate the pre-trained control policy.

3. The method as described in claim 2, characterized in that, The method of constructing a data-driven agent model based on historical runtime sequence data using a long short-term memory network is as follows: Historical operating sequence data of the negative pressure conveying system for stone and coal in a power plant are collected and divided into input sequence and output sequence according to time series. The input sequence is the system state vector and control action vector at the current moment, and the output sequence is the system state vector at the next moment. A two-layer long short-term memory network architecture is adopted, with the input sequence as the model input, the output sequence as the supervision label, and the mean squared error as the loss function to supervise the training of the long short-term memory network.

4. The method as described in claim 3, characterized in that, The system state vector includes: negative pressure at the coal mill outlet pipe, wind speed in the conveying pipe, feed rate of gravel and coal, carbon content of gravel and coal, operating current of the Roots blower, blower outlet temperature, valve opening degree, and material-to-air ratio. The control action vector includes: the adjustment amount of the Roots blower speed and the adjustment amount of the conveying valve opening.

5. The method as described in claim 1, characterized in that, S4 includes: The agent policy network, which has been pre-trained offline, is deployed to the DCS control system of the power plant. In shadow mode, the agent collects the current operating status of the system and outputs suggested control actions in each control cycle. However, the actions are not directly sent to the actuators, but are only recorded in the operation log in sync with the actual execution actions of the original controller and the system status change data. Based on the real operating data accumulated in the shadow mode, the real working condition samples collected online are added to the experience replay buffer of the deep deterministic policy gradient algorithm, mixed with offline pre-training data for training, and the learning rate is reduced to 10% of the offline training to fine-tune the agent policy network online. A gradual switching mechanism is adopted to transfer control from the original controller to the intelligent agent in stages.

6. The method as described in claim 1, characterized in that, The total reward function of the multi-objective reward function mentioned in step S4 is defined as follows: in, R The total reward value for a single control cycle. , , These are the weighting coefficients for the three optimization objectives: transport stability, energy efficiency, and equipment wear. R_transport is the reward for transport stability, R_energy consumption is the penalty for energy consumption, and R_wear is the penalty for equipment wear.

7. The method as described in claim 6, characterized in that, The aforementioned transport stability reward item R transport is given a positive reward when the system transports normally and no pipe blockage alarm is triggered within a single control cycle, and a large negative penalty is given when a pipe blockage alarm is triggered; The formula for calculating the energy consumption penalty term R is as follows: in, This represents the current real-time operating power of the wind turbine. This refers to the rated power of the fan. This is the energy consumption penalty coefficient; The formula for calculating the equipment wear penalty term R wear is as follows: in, This is an estimate of the particle impact velocity at the bend in the pipe. This represents the wear penalty coefficient.

8. The method as described in claim 7, characterized in that, The estimated particle impact velocity at the bend in the pipe v impact The calculation formula is based on the estimation of pipeline air velocity and material-to-air ratio: in, This represents the real-time airflow velocity inside the duct. For solid phase material flow rate, This refers to the gas phase air flow rate. and These are coefficients based on the pipeline structure calibration.

9. An adaptive control system for a negative pressure pneumatic conveying system for stones and coal, characterized in that, include: The offline pre-training module is used to construct a dynamic simulation environment for the system based on historical running data, and to perform offline pre-training on the deep reinforcement learning agent in the simulation environment to generate pre-trained control strategies. A multi-objective reward configuration module is used to construct a multi-objective reward function, which includes at least a delivery stability reward item, an energy consumption penalty item, and a device wear penalty item, so as to guide the agent to take into account multiple optimization objectives during the control process; The online smooth iterative control module is used to deploy the pre-trained control strategy to the real system, collect real running data through shadow mode to fine-tune the agent online, and use a progressive switching mechanism to smoothly transfer control from the original controller to the agent. The safety verification and dynamic control module is used to perform hard constraint safety verification on the control actions output by the agent throughout the entire cycle of the agent's execution control, and dynamically switch the control priority mode of the agent according to the weight coefficient of the multi-objective reward function adjusted in real time.

10. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method as claimed in any one of claims 1-8.