Pneumatic conveying control method and system based on fusion LLM and reinforcement learning

Abstract

The invention relates to a pneumatic conveying control method and system based on fusion LLM and reinforcement learning, and belongs to the technical field of control systems. Comprising the following steps that an operator inputs a natural language control instruction, and a structured instruction is generated through a preset LLM model; a historical working condition library is retrieved based on an RAG algorithm, physical constraint data is added for a structured instruction to form a multi-target control set, and a multi-target reward function and a strategy network are constructed by an LLM model according to the multi-target control set. Acquiring a system working condition, initializing a reinforcement learning agent, inputting a strategy network and the current system working condition to generate and execute an optimal control action, and skipping the state to the next moment; and calculating a multi-target reward value, constructing an experience tuple in combination with the current working condition, the execution action and the working condition at the next moment, storing the experience tuple into an experience playback pool, and updating the strategy network by using data in the pool. And the LLM model monitors system working condition parameters in real time, and if abnormity occurs or a new instruction is received, the reward function and the strategy network are adjusted.

Description

Technical Field

[0001] This invention relates to a pneumatic conveying control method and system based on the fusion of LLM and reinforcement learning, belonging to the field of control system technology. Background Technology

[0002] Pneumatic conveying is a key industrial technology that utilizes gas energy to transport powdery, granular, and other bulk materials within closed pipelines. It is widely used in chemical, building materials, food, and pharmaceutical industries. The process is highly nonlinear, time-varying, and uncertain. The core objective of the control system is to minimize energy consumption and equipment wear while ensuring safe, stable, and efficient conveying. However, optimizing the strategy network has long been a challenge in this field.

[0003] For example, Chinese invention patent application CN118778445A discloses an intelligent control system for pneumatic conveying equipment, including: an input unit with multiple sensor configurations to collect key data about equipment operation in real time, such as pressure, temperature, flow rate, and material density. The data processing unit includes an adaptive algorithm module and a hybrid neural network model. The adaptive algorithm module utilizes a reward-based reinforcement learning agent to dynamically adjust from the data acquired by the input unit to find the optimal operating parameters. The hybrid neural network model combines convolutional neural networks and recurrent neural networks to predict potential pressure fluctuations and pipe blockages during the conveying process, and adjusts the operating parameters accordingly to optimize system performance. The control execution unit adjusts the equipment operating settings based on the finally determined optimal parameters. While this patent is applied to the field of pneumatic conveying and employs RL technology, the reward function requires manual design and cannot dynamically adjust the weights of multiple objectives, resulting in high learning costs.

[0004] For example, Chinese invention patent application CN119861633A discloses a process control and optimization method based on a pneumatic conveying system. This method includes four core modules: data source integration, data acquisition and processing, model + algorithm, and closed-loop control. The data source integration module collects data from DCS / PLC systems and sensor networks; the data acquisition and processing module is responsible for data cleaning, format conversion, and outlier detection; the model and algorithm module combines material prediction, step sequence planning, material-to-air ratio calculation, and conveying strategy network to achieve deep learning and optimized control; the closed-loop control module adjusts the conveying process based on real-time feedback to ensure operational stability. This patent focuses on the field of pneumatic conveying, which is consistent with the application scenario of this application. However, it relies on pre-set models and static algorithms, cannot respond to natural language commands, and lacks explanation of the decision-making process, making it relatively backward in terms of intelligence and understandability.

[0005] Therefore, there is an urgent need for a pneumatic conveying control method and system that can solve the problems of multi-objective dynamic balance, low sample efficiency, and black box decision-making in pneumatic conveying; and can realize the dynamic mapping from natural language to control logic, replacing the manually designed fixed reward function. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention proposes a pneumatic conveying control method and system based on the fusion of LLM and reinforcement learning.

[0007] The technical solution of the present invention is as follows: On the one hand, this invention proposes a pneumatic conveying control method based on the fusion of LLM and reinforcement learning, comprising the following steps: The operator receives a multi-objective control task, formulates natural language control instructions through a preset human-computer interaction interface, and sends them to a preset LLM model to output structured control instructions; based on RAG technology, the knowledge base is retrieved, physical constraint data is added to the structured control instructions, and a multi-objective control set is generated; based on the multi-objective control set, the LLM model constructs a multi-objective reward function and policy network for the reinforcement learning agent; The system continuously monitors the pneumatic conveying system's operating status, inputs it into the reinforcement learning agent, generates the control action for the current moment, and simultaneously jumps to the next moment to obtain the next moment's pneumatic conveying system operating status. It then obtains multi-objective reward values ​​based on a multi-objective reward function. Based on the current control action, the current pneumatic conveying system operating status, the multi-objective reward values, and the next moment's pneumatic conveying system operating status, it constructs an experience tuple. This experience tuple is then input into an experience replay pool, and the policy network is updated using data from the experience replay pool. The LLM model monitors the operating condition of the pneumatic conveying system in real time. When an abnormal state occurs or a new operator control command is received, the multi-objective reward function and policy network are adjusted.

[0008] Preferably, the LLM model includes a target parsing module, a knowledge base management module, a reward function generation module, a policy network management module, and an inference and interpretation module, wherein: The target parsing module is used to understand the target control set and generate a reward function; The knowledge base management module is used to store and manage engineering principles, expert rules, and historical operating condition data. The reward function generation module is used to generate a multi-objective reward function based on a multi-objective control set; The policy network management module is used to generate a reasonable initial policy network or suggested actions by searching the knowledge base through RAG when a reinforcement learning task begins or encounters a new state, so as to guide the reinforcement learning module to conduct more efficient exploration. The reasoning and interpretation module is used to receive and process sensor data streams from the input unit, integrate the data streams into semantic descriptions that can be understood by humans, and provide alarms or assist in decision-making.

[0009] Preferably, the method outputs structured control instructions from the LLM model, specifically as follows: The LLM model parses natural language control commands into structured control commands that include material type, material source, conveying safety constraints, and core conveying objectives, based on preset command-parameter mapping rules.

[0010] Preferably, the LLM model is based on a multi-objective control set, constructing a multi-objective reward function to balance multiple sub-objectives. It combines the rewards of different objectives into a single overall objective reward value through weighted summation, expressed by the formula: ; In the formula, Indicates at time step Reinforce the state of the agent during learning. This indicates that the reinforcement learning agent is in a state. The action to choose from, Indicates multi-objective reward value, This indicates the weight of each sub-objective within the overall objective. Indicates at time step Reinforcement learning agents in state Executed Energy reward gained after an action Indicates at time step Reinforcement learning agents in state Executed The efficiency reward gained after an action. Indicates at time step Reinforcement learning agents in state Executed The stability bonus gained after an action. Indicates at time step Reinforcement learning agents in state Executed The blocking reward obtained after the action.

[0011] Preferably, the reinforcement learning agent specifically employs the SAC or PPO algorithm.

[0012] Preferably, the reinforcement learning agent employs the SAC algorithm to update the policy network based on data in the experience replay pool; wherein the design of the policy network includes the following features: The reinforcement learning agent aims to maximize the sum of entropy and cumulative reward. The input to the strategy network is the operating state vector of the pneumatic conveying system; The output of the strategy network is a continuous control action vector for the actuators of the pneumatic conveying system.

[0013] On the other hand, the present invention also proposes a pneumatic conveying control system based on the fusion of LLM and reinforcement learning, comprising the following modules: Data acquisition module: used to monitor the operating conditions of the pneumatic conveying system in real time and collect real-time operating data of the pneumatic conveying system; LLM parsing module: The operator receives a multi-objective control task, formulates natural language control instructions through a preset human-computer interaction interface, and sends them to the preset LLM model to output structured control instructions; based on RAG technology, the knowledge base is retrieved, physical constraint data is added to the structured control instructions, and a multi-objective control set is generated; based on the multi-objective control set, the LLM model constructs a multi-objective reward function and policy network for the reinforcement learning agent; The reinforcement learning module acquires the real-time operating conditions of the pneumatic conveying system and inputs them into the reinforcement learning agent to generate the control action at the current moment. Simultaneously, the operating conditions of the pneumatic conveying system are jumped to the next moment to obtain the operating conditions of the pneumatic conveying system at the next moment. It obtains multi-objective reward values ​​based on a multi-objective reward function. Based on the control action at the current moment, the operating conditions of the pneumatic conveying system at the current moment, the multi-objective reward values, and the operating conditions of the pneumatic conveying system at the next moment, it constructs an experience tuple. The experience tuple is input into an experience replay pool, and the policy network is updated using the data in the experience replay pool. Dynamic feedback module: The LLM model monitors the operating condition of the pneumatic conveying system in real time. When an abnormal state occurs or a new operator control command is received, the multi-objective reward function and strategy network are adjusted. Control execution module: Used to execute the optimal control action of the reinforcement learning module at the current moment, and control the pneumatic conveying pipe according to the optimal control action at the current moment.

[0014] The present invention has the following beneficial effects: (1) This invention is a pneumatic conveying control method and system based on the fusion of LLM and reinforcement learning. Through the technical design of combining LLM model with RAG technology, the operator can directly input natural language control commands without professional code or parameter settings, which greatly reduces the operation threshold. LLM can accurately parse natural language into structured commands containing material information, safety constraints, etc., and the RAG algorithm can retrieve historical working condition database to supplement physical constraint data to form a multi-objective control set, so that the reward function and policy network constructed subsequently are naturally adapted to the actual conveying scenario, effectively avoiding blind control that is out of touch with the working condition and improving the initial adaptability of control.

[0015] (2) This invention is a pneumatic conveying control method and system based on the fusion of LLM and reinforcement learning. By relying on the experience playback and policy network iteration mechanism of reinforcement learning agents, the data such as state, action and reward in the conveying process are used to construct experience tuples and store them in the playback pool to realize the full reuse and efficient utilization of historical experience. Combined with SAC or PPO algorithm, through discount cumulative reward modeling, gradient optimization and other means to reduce variance, the policy network can be continuously iterated and updated to continuously optimize the accuracy of control actions, significantly improve the energy utilization rate, conveying efficiency and operation stability of pneumatic conveying, and reduce the risk of pipeline blockage.

[0016] (3) This invention is a pneumatic conveying control method and system based on the fusion of LLM and reinforcement learning. It utilizes an LLM model to monitor the operating conditions of the pneumatic conveying system in real time and dynamically adjust the system based on feedback. When the system operating conditions become abnormal or a new control command is received, it can quickly respond and adjust the multi-objective reward function and the strategy network, overcoming the limitation of traditional control systems where fixed strategy networks are difficult to adapt to sudden situations or new demands. This dynamic adaptation capability can promptly correct deviations, adapt to new control objectives, ensure the continuity and safety of the conveying process, and enhance the robustness of the system under complex operating conditions. Attached Figure Description

[0017] Figure 1 This is a flowchart of the pneumatic conveying control method proposed in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the intelligent control equipment for pneumatic conveying of fly ash from a coal-fired power plant, as proposed in Embodiment 2 of the present invention.

[0018] The attached figures are labeled as follows: 11. Hopper; 12. Hopper gate valve; 13. Feed valve; 14. Balance valve; 15. Silo pump; 21. Air tank; 22. Air tank safety valve; 23. Air tank drain valve; 24. Air tank shut-off valve; 31. Conveying gas pipeline; 32. Conveying gas manual ball valve; 33. Conveying gas intelligent regulating valve; 34. Conveying gas check valve; 41. Fluidizing gas pipeline; 42. Fluidizing gas manual ball valve; 43. Fluidizing gas intelligent regulating valve; 44. Fluidizing gas check valve; 51. Auxiliary blowing gas pipeline; 52. Auxiliary blowing gas manual ball valve; 53. Auxiliary blowing gas intelligent regulating valve; 54. Auxiliary blowing gas check valve; 61. Discharge valve; 62. Material conveying pipeline; 63. Unblocking valve; 71. Pipeline pressure sensor; 72. Silo pump pressure sensor; 8. Control host; 9. Terminal unit. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] It should be understood that the step numbers used in the text are for ease of description only and are not intended to limit the order in which the steps are performed.

[0021] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0022] The terms “comprising” and “including” indicate the presence of the described feature, whole, step, operation, element and / or component, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0023] The term “and / or” refers to any combination of one or more of the associated listed items, as well as all possible combinations, and includes these combinations.

[0024] Example 1: See Figure 1 This embodiment proposes a pneumatic conveying control method based on the fusion of LLM and reinforcement learning, including the following steps: S100: The operator receives a multi-target control task, formulates natural language control instructions through a preset human-machine interface, sends them to a preset LLM model, and outputs structured control instructions. It should be noted that the pre-defined LLM model includes a target parsing module, a knowledge base management module, a reward function generation module, a policy network management module, and an inference and interpretation module, among which: The target parsing module is used to understand the target control set and generate a reward function; The knowledge base management module is used to store and manage engineering principles, expert rules, and historical operating condition data. The reward function generation module is used to generate a multi-objective reward function based on a multi-objective control set; The policy network management module is used to generate a reasonable initial policy network or suggested actions by searching the knowledge base through RAG when a reinforcement learning task begins or encounters a new state, so as to guide the reinforcement learning module to conduct more efficient exploration. The reasoning and interpretation module is used to receive and process sensor data streams from the input unit, and integrate the data streams into semantic descriptions that can be understood by humans for use in alarms or decision support.

[0025] Furthermore, the target parsing module of the LLM model is invoked to parse the structured control instructions and access the knowledge base content in the knowledge base management module. The structured control instructions are compared with the engineering principles, expert rules, and historical working condition data in the knowledge base. Physical constraint data is added to the structured control instructions to generate a multi-objective control set. The reward function generation module is then invoked to convert the multi-objective control set into a specific multi-objective reward function with weighted coefficients, expressed by the formula: ; In the formula, Indicates at time step Reinforce the state of the agent during learning. This indicates that the reinforcement learning agent is in a state. The action to choose from, Indicates at time step Reinforcement learning agents in state Executed The multi-target reward value obtained after the action This indicates the weight of each sub-objective within the overall objective. Indicates at time step Reinforcement learning agents in state Executed Energy reward gained after an action Indicates at time step Reinforcement learning agents in state Executed The efficiency reward gained after an action. Indicates at time step Reinforcement learning agents in state Executed The stability bonus gained after an action. Indicates at time step Reinforcement learning agents in state Executed The negative reward for pipe blockage obtained after the action; Furthermore, the policy network management module of the LLM model generates a policy network based on the multi-objective control tasks received by the operator. The multi-objective reward function is passed to the reinforcement learning agent; at the same time, the inference and interpretation module of the LLM model is invoked to generate logs of the multi-objective reward function and policy network in natural language form and send them to the operator.

[0026] S200. Obtain the current operating condition of the pneumatic conveying system and initialize the reinforcement learning agent; input the policy network and the current operating condition of the pneumatic conveying system into the reinforcement learning agent to generate and execute the optimal control action for the current moment, expressed by the formula: ; In the formula, Representing state Time-based policy network value, This indicates that the reinforcement learning agent is in a state. The action of making a selection; Optimal action Sending data to various actuators and those used to manage conveying process parameters for physical operation; It should be noted that the actuator types include: the opening and closing of valves such as feed valves and discharge valves; the opening degree of valves in compressed air supply air pipelines; the opening degree of valves for conveying gas; the opening degree of valves for fluidizing gas; and the opening degree of valves for purging gas. Process parameters include: waiting time and purging time during the conveying process; lower limit of conveying end pressure; and upper limit of conveying pipe blockage pressure.

[0027] At the same time, the pneumatic conveying system's operating condition jumps to the next moment, obtaining the pneumatic conveying system's operating condition at the next moment. The optimal control action is input into the multi-objective reward function to obtain the multi-objective reward value, which is expressed by the formula: ; In the formula, Indicates time step Multi-objective reward value; State Optimal action Time step Multi-objective reward value and state By combining these, we obtain the empirical tuple. The experience tuples are placed into the replay pool, and data is collected from the replay pool to update the strategy network parameters.

[0028] Each time, random sampling is used to extract multi-objective reward values ​​from the experience replay pool at multiple time steps; The cumulative discount reward is defined by the following formula: ; In the formula, Indicates time step Cumulative discount rewards Indicates time step Multi-objective reward value, Indicates the discount factor. Indicates the nth time after the current time. Time step; Based on the cumulative discount reward, its expected value is calculated, expressed by the formula: ; In the formula, This represents the objective function for the training iteration. Represents the policy network Calculation of the expected value of the generated interaction sequence. Indicates the policy network parameters, This indicates the cumulative reward based on the initial discount. The interaction between the reinforcement learning agent and the environment yields a complete sequence of interaction processes, expressed as a formula: ; In the formula, Represents the sequence of interaction processes. This represents the environmental state detected by the reinforcement learning agent at the initial moment. This indicates that the reinforcement learning agent is in a state. The action to choose from, Indicates time step The multi-objective reward value below, This represents the environmental state monitored by the reinforcement learning agent at the termination time. This indicates that the reinforcement learning agent is in a state. The action to choose from, This represents the multi-objective reward value at the time of termination.

[0029] In this embodiment, the SAC algorithm is used as an example for the policy network. Its state Soft state value function and soft Q function This can be expressed as a formula:

[0030]

[0031] In the formula, Representational Policy Network In state The soft-state value function under the following conditions Representational Policy Network In state The soft Q function below, It is a discount factor. It is a temperature parameter (entropy regularization coefficient) used to weigh the importance of reward and entropy. Representational Policy Network In state The entropy below, Indicates the state Next action Instant rewards at any time This indicates that for all policies in the network... Generated trajectory Take the expected value. Indicates the time step index; Two Q-networks are used to learn the soft Q-function, and the training is stabilized by a separate target Q-network, sampling a batch of data from the empirical replay pool. The loss function of Q-network Defined as the minimum mean square error, expressed by the formula:

[0032] Among them, the target value The calculation method is as follows:

[0033] In the formula, This represents the loss function of the Q-network. It is an experience replay pool. Indicates the current policy network. This indicates the network's next state based on the current policy. New sampling procedures This indicates that the two parameters are respectively and The minimum soft Q value output by the target Q-network. Representational Policy Network In state Select Action The logarithmic probability; By analyzing the loss function Calculate the gradient and perform gradient descent to update the parameters of the Q-network. .

[0034] Training the policy network, the policy network (parameters are...) The goal is to maximize the expected Q-value of its output action and the entropy of the policy network. The loss function of the policy network... Expressed as a formula:

[0035] In the formula, It is a state sampled from the experience replay pool. It is based on the current policy network In state The resampling action.

[0036] By taking the gradient of the loss function and performing gradient descent, it is equivalent to performing gradient ascent on the original target, thereby updating the parameters of the policy network. .

[0037] In summary, by using random samples from the empirical replay pool, minimizing the Bellman error, maximizing the soft Q value, and employing gradient ascent, the parameters of the policy network are updated. .

[0038] The S300 and LLM models monitor the current operating status in real time. If abnormal situations such as pipe blockage risk are detected, the reward function is proactively adjusted, and a temporary pipe blockage penalty term is added to guide the reinforcement learning agent to quickly collect avoidance actions. At the same time, if a new instruction is detected from the operator, the LLM model will also proactively update the reward function and policy network in real time.

[0039] Example 2: Based on Example 1, this example proposes a pneumatic conveying control method suitable for coal-fired power plants, including the following steps: A100, see also Figure 2 Deploy intelligent control equipment for pneumatic conveying of fly ash from coal-fired power plants, the equipment specifically including the following hardware: 11. Hopper 12. Hopper gate valve 13. Feed valve 14. Balance valve 15. Silo pump 16. Air tank 21. Air tank safety valve 22. Air tank drain valve 23. Air tank shut-off valve 24. Conveying gas pipeline 31. Conveying gas manual ball valve 32. Conveying gas intelligent regulating valve 33. Conveying gas check valve 34. Fluidizing gas pipeline 41. Fluidizing gas manual ball valve 42. Fluidizing gas intelligent regulating valve 43. Fluidizing gas check valve 44. Auxiliary blowing gas pipeline 51. Auxiliary blowing gas manual ball valve 52. Auxiliary blowing gas intelligent regulating valve 53. Auxiliary blowing gas check valve 54. Discharge valve 61. Material conveying pipeline 62. Unblocking valve 63. Pipeline pressure sensor 71. Silo pump pressure sensor 72. Control host 8. Terminal 9, wherein: The control host 8 is connected to the valves and sensors of the pneumatic conveying intelligent control equipment through a cable, and can receive sensor data and issue operation commands to each control actuator. The gas flow rate of the gas delivery pipeline 31, fluidizing gas pipeline 41, and blowing gas pipeline 51 is controllable, and they are equipped with intelligent regulating valves 33 for delivery gas, 43 for fluidizing gas, and 53 for blowing gas. Preferably, the opening degree and air mass flow rate of the intelligent regulating valve 33 for conveying gas are calibrated before use, and the relationship function curve between valve opening degree and flow rate is pre-constructed and stored in the knowledge base management module of the control host 8. The valve opening degree feedback signal is sent to the control host 8. The gas conveying pipe 31 is responsible for pushing the material in the silo pump 15 into the material conveying pipe 62, and allowing the material to move along the airflow direction, and finally enter the terminal silo 9; The main function of the fluidizing gas pipeline 41 is to introduce air into the bottom of the fluidizing chamber pump 15 to fluidize the material inside the chamber pump. When the system shows signs of blockage, the blowing gas pipeline 51 shuts off the delivery gas and fluidizing gas, using only the blowing gas to prevent blockage. If the system is already blocked, the blowing gas pipeline 51 works in conjunction with the unblocking valve 63 to clear the blockage.

[0040] A200: The operator issues control commands in natural language through the control room terminal: "Start conveying fly ash from power plant No. 1. Pay attention to pipeline blockage prevention and prioritize conveying efficiency." Furthermore, the target parsing module of the LLM model is invoked to parse the structured control instructions, interpreting the keywords: "fly ash" represents the material type, "power plant" (source), "pay attention to blockage prevention" (safety constraint), and "prioritize efficiency" (core control objective); the knowledge base content in the knowledge base management module is accessed to obtain fly ash characteristics and view historical operating data, expert rules, and engineering principles, etc., and physical constraint data is added to the structured control instructions to generate a multi-objective control set; The reward function generation module is invoked to convert the multi-objective control set into a specific multi-objective reward function with weighted coefficients, expressed by the following formula: ; In the formula, Indicates at time step Reinforce the state of the agent during learning. This indicates that the reinforcement learning agent is in a state. The action to choose from, Indicates at time step Reinforcement learning agents in state Executed The multi-target reward value obtained after the action This indicates the weight of each sub-objective within the overall objective. Indicates at time step Reinforcement learning agents in state Executed Energy reward gained after an action Indicates at time step Reinforcement learning agents in state Executed The efficiency reward gained after an action. Indicates at time step Reinforcement learning agents in state Executed The stability bonus gained after an action. Indicates at time step Reinforcement learning agents in state Executed The negative reward for pipe blockage obtained after the action; Furthermore, the policy network management module of the LLM model generates a policy network based on the multi-objective control tasks received by the operator. The multi-objective reward function is passed to the reinforcement learning agent along with the multi-objective reward function; at the same time, the inference interpretation module of the LLM model is called to generate logs of the multi-objective reward function and policy network in natural language form and send them to the operator. The reinforcement learning agent aims to maximize the sum of entropy and cumulative reward; the input to the policy network is the operating state vector of the pneumatic conveying system; and the output of the policy network is the continuous control action vector for the actuators of the pneumatic conveying system.

[0041] A300, feed valve 13 is opened until the level gauge triggers a full bin signal; the full bin signal is transmitted to the reinforcement learning agent, and the reinforcement learning agent obtains the environmental state of the system. After receiving the reward function from the LLM model, the reinforcement learning agent starts working; the reinforcement learning agent adjusts the opening of the fluidizing gas intelligent regulating valve 43 and the conveying gas intelligent regulating valve 33 according to the real-time pressure, level gauge and other status in the silo pump 15. In the shortest time and with the least amount of air consumption, the fly ash in the silo pump 15 is fully fluidized and stably pressurized to the discharge pressure set point, creating an optimal initial material-air mixing state for the conveying stage. It should be noted that the setpoint value of the discharge pressure can be obtained by the LLM module from the knowledge base in the knowledge base management module, or it can be set manually.

[0042] A400, when the pressure of the silo pump 15 reaches the preset value, the discharge valve 61 opens, and the conveying operation begins. Specifically: The reinforcement learning agent monitors real-time comprehensive data such as pressure along the main pipeline, pressure in gas storage tank 21, pressure drop rate of silo pump 15, and vibration of each pipeline; it dynamically and collaboratively adjusts the opening of fluidizing gas valve and delivery gas valve to maintain an optimal, dynamically changing gas volume.

[0043] The LLM module analyzes the data flow at a higher dimension. If it detects a sustained upward trend in pipeline pressure, which is significantly correlated with "dust accumulation" in historical data, even if the blockage alarm threshold hasn't been reached, the LLM will proactively adjust the reward function, temporarily and drastically increasing the penalty for "pressure fluctuations." This adjustment will instantly force the reinforcement learning agent to take the following actions: close the fluidizing gas intelligent regulating valve 43 and the delivery gas regulating valve 33, and increase the opening of the purging gas intelligent regulating valve 53, thus eliminating potential blockage.

[0044] Furthermore, if the operator sees gas volume fluctuations on the control room terminal screen, they can ask the terminal in natural language: "Why did the gas volume suddenly fluctuate just now?" The LLM module will reply in natural language: "A risk of pipe blockage was detected. The system automatically shut off the delivery gas and fluidizing gas and turned on the purging gas to prevent blockage. The risk has been eliminated, and the system has returned to efficient delivery mode." A500: When the pressure inside the silo pump drops to the set lower limit, the system determines that the current conveying cycle has ended. It should be noted that the lower pressure limit can be obtained by the LLM from the knowledge base or set by the operator.

[0045] Furthermore, the LLM module automatically generates a text report for this delivery task. The text report can be based on a preset template in the knowledge base or can be customized by the operator. The text report includes information such as timestamps of key steps in the delivery process, total delivery time, number of abnormal situations and their resolution. At the same time, all the experience data generated in this reinforcement learning agent cycle is used for the offline or online training of the reinforcement learning agent.

[0046] Example 3: This embodiment proposes a pneumatic conveying control system based on the fusion of LLM and reinforcement learning, including the following modules: Data acquisition module: used to monitor the operating conditions of the pneumatic conveying system in real time and collect real-time operating status data of the pneumatic conveying pipe; The LLM parsing module is used by operators to formulate natural language control commands, input them into a preset LLM model, and output structured control commands. Based on RAG technology, it retrieves historical operating condition databases, adds physical constraint data to the structured control commands, and generates a multi-objective control set. The LLM model is based on the multi-objective control set to construct a multi-objective reward function and policy network for the reinforcement learning agent. The reinforcement learning module is used to input the current operating condition of the pneumatic conveying system collected by the data acquisition module into the reinforcement learning agent, generate the optimal control action for the current moment, and simultaneously jump the operating condition of the pneumatic conveying system to the next moment; obtain the multi-objective reward value according to the multi-objective reward function; construct experience tuples based on the current control action, the current operating condition of the pneumatic conveying system, the multi-objective reward value, and the pneumatic conveying system condition for the next moment; input the experience tuples into the experience replay pool, and update the policy network using the data in the experience replay pool; Dynamic feedback module: used to adjust the multi-objective reward function and strategy network based on the operating conditions of the pneumatic conveying system collected by the data acquisition module, when an abnormal state occurs or a new operator control command is received; Control execution module: Used to execute the optimal control action of the reinforcement learning module at the current moment, and control the pneumatic conveying pipe according to the optimal control action at the current moment.

[0047] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, A and B simultaneously, or B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0048] Those skilled in the art will recognize that the units and algorithm steps described in the embodiments disclosed herein can be implemented using electronic hardware, computer software, or a combination of electronic hardware and software. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0049] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0050] In the several embodiments provided in this application, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0051] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A pneumatic conveying control method based on the fusion of LLM and reinforcement learning, characterized in that, Includes the following steps: The operator receives a multi-objective control task, formulates natural language control instructions through a preset human-machine interface, sends them to a preset LLM model, and outputs structured control instructions. The knowledge base is retrieved based on RAG technology, and physical constraint data is added to the structured control instructions to generate a multi-objective control set. The LLM model is based on the multi-objective control set to build a multi-objective reward function and policy network for the reinforcement learning agent. The operating status of the pneumatic conveying system is acquired in real time, input into the reinforcement learning agent, and the control action at the current moment is generated. At the same time, the operating status of the pneumatic conveying system jumps to the next moment to obtain the operating status of the pneumatic conveying system at the next moment. Obtain multi-objective reward values ​​based on the multi-objective reward function; construct experience tuples based on the current control action, the current pneumatic conveying system condition, the multi-objective reward values, and the next pneumatic conveying system condition. The experience tuples are input into the experience replay pool, and the policy network is updated using the data in the experience replay pool. The LLM model monitors the operating condition of the pneumatic conveying system in real time. When an abnormal state occurs or a new operator control command is received, the multi-objective reward function and policy network are adjusted.

2. The pneumatic conveying control method based on the fusion of LLM and reinforcement learning according to claim 1, characterized in that, The LLM model comprises an objective resolution module, a knowledge base management module, a reward function generation module, a policy network management module, and an inference and interpretation module, wherein: The target parsing module is used to understand the target control set and generate a reward function; The knowledge base management module is used to store and manage engineering principles, expert rules, and historical operating condition data. The reward function generation module is used to generate a multi-objective reward function based on a multi-objective control set; The policy network management module is used to generate a reasonable initial policy network or suggested actions by searching the knowledge base through RAG when a reinforcement learning task begins or encounters a new state, so as to guide the reinforcement learning module to conduct more efficient exploration. The reasoning and interpretation module is used to receive and process sensor data streams from the input unit, integrate the data streams into semantic descriptions that can be understood by humans, and provide alarms or assist in decision-making.

3. The pneumatic conveying control method based on the fusion of LLM and reinforcement learning according to claim 1, characterized in that, The method outputs structured control instructions from the LLM model, specifically: The LLM model parses natural language control commands into structured control commands that include material type, material source, conveying safety constraints, and core conveying objectives, based on preset command-parameter mapping rules.

4. The pneumatic conveying control method based on the fusion of LLM and reinforcement learning according to claim 1, characterized in that, The LLM model is based on a multi-objective control set and constructs a multi-objective reward function to balance multiple sub-objectives. It combines the rewards of different objectives into a single overall reward value through weighted summation, expressed by the formula: ； In the formula, Indicates at time step Reinforce the state of the agent during learning. This indicates that the reinforcement learning agent is in a state. The action to choose from, Indicates multi-objective reward value, This indicates the weight of each sub-objective within the overall objective. Indicates at time step Reinforcement learning agents in state Executed Energy reward gained after an action Indicates at time step Reinforcement learning agents in state Executed The efficiency reward gained after an action. Indicates at time step Reinforcement learning agents in state Executed The stability bonus gained after an action. Indicates at time step Reinforcement learning agents in state Executed The blocking reward obtained after the action.

5. The pneumatic conveying control method based on the fusion of LLM and reinforcement learning according to claim 1, characterized in that, The reinforcement learning agent specifically employs either the SAC or PPO algorithm.

6. The pneumatic conveying control method based on the fusion of LLM and reinforcement learning according to claim 5, characterized in that, The reinforcement learning agent employs the SAC algorithm, updating the policy network based on data from the experience replay pool; its policy network design includes the following features: The reinforcement learning agent aims to maximize the sum of entropy and cumulative reward. The input to the strategy network is the operating state vector of the pneumatic conveying system; The output of the strategy network is a continuous control action vector for the actuators of the pneumatic conveying system.

7. A pneumatic conveying control system based on the fusion of LLM and reinforcement learning, characterized in that, Includes the following modules: Data acquisition module: used to monitor the operating conditions of the pneumatic conveying system in real time and collect real-time operating data of the pneumatic conveying system; LLM parsing module: When the operator receives a multi-objective control task, he formulates natural language control instructions through a preset human-machine interface, sends them to the preset LLM model, and outputs structured control instructions. The knowledge base is retrieved based on RAG technology, and physical constraint data is added to the structured control instructions to generate a multi-objective control set. The LLM model is based on the multi-objective control set to build a multi-objective reward function and policy network for the reinforcement learning agent. Reinforcement learning module: Real-time acquisition of the pneumatic conveying system's operating condition, input into the reinforcement learning agent, generating the control action at the current moment, and simultaneously jumping the pneumatic conveying system's operating condition to the next moment to obtain the pneumatic conveying system's operating condition at the next moment; Obtain multi-objective reward values ​​based on the multi-objective reward function; construct experience tuples based on the current control action, the current pneumatic conveying system condition, the multi-objective reward values, and the next pneumatic conveying system condition. The experience tuples are input into the experience replay pool, and the policy network is updated using the data in the experience replay pool. Dynamic feedback module: The LLM model monitors the operating condition of the pneumatic conveying system in real time. When an abnormal state occurs or a new operator control command is received, the multi-objective reward function and strategy network are adjusted. Control execution module: Used to execute the optimal control action of the reinforcement learning module at the current moment, and control the pneumatic conveying pipe according to the optimal control action at the current moment.

Images