A reward method based on pipe transportation particle deposition rate and energy consumption constraints
By combining a deep reinforcement learning framework with a reward function that considers particle deposition rate and energy consumption constraints, the inlet velocity of the pipeline is dynamically adjusted, solving the balance problem between particle deposition and energy consumption optimization, and achieving a synergistic effect of particle deposition suppression and energy consumption optimization.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- UNIV OF SHANGHAI FOR SCI & TECH
- Filing Date
- 2026-04-22
- Publication Date
- 2026-07-14
AI Technical Summary
In existing pipeline transportation systems, particles tend to deposit in bends during gas-solid two-phase flow transport, leading to blockages. Increasing the inlet velocity to suppress deposition increases system energy consumption, making it difficult to achieve a balance between deposition suppression and energy optimization.
By employing a deep reinforcement learning framework that combines policy neural networks and value neural networks, and through a reward function constrained by particle deposition rate and energy consumption, the inlet fluid velocity of the pipeline is dynamically adjusted to achieve coordinated control of particle deposition suppression and system energy consumption optimization.
It effectively suppresses particle deposition and optimizes energy consumption under different flow conditions, improves pipeline transportation efficiency and stability, and reduces system energy consumption.
Smart Images
Figure CN122088302B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of flow control and optimization technology, specifically relating to a reward method based on the particle deposition rate and energy consumption constraints of pipeline transportation. Background Technology
[0002] Pipeline transportation systems, as crucial infrastructure connecting resource production sites and consumption terminals, are widely used in fields such as oil and gas extraction, chemical production, and urban water supply. During gas-solid two-phase flow transportation, particles are easily deposited and blocked in the bend area due to the complex local flow field, inertia, and wall interactions within the pipe. This results in a reduction in the effective flow area of the pipeline, a decrease in transportation efficiency, and in severe cases, even pipe bursts.
[0003] Most existing fluid control methods rely on empirical formulas or fixed design specifications, making it difficult to adaptively adjust the flow state under unsteady conditions. While simply increasing the inlet velocity can reduce particle deposition to some extent, it usually leads to increased system energy consumption, making it difficult to achieve a balance between deposition suppression and energy optimization. Summary of the Invention
[0004] To address the problem that existing methods for controlling particle deposition in pipelines struggle to simultaneously achieve both deposition suppression and system energy optimization, this invention provides a reward-based method based on pipeline transport particle deposition rate and energy consumption constraints, comprising the following steps:
[0005] A deep reinforcement learning framework for numerical simulation of gas-solid two-phase flow in a curved pipe and artificial deep neural network was established.
[0006] Artificial deep neural networks, acting as the intelligent agents within the framework, output corresponding changes based on the states provided by numerical simulations of gas-solid two-phase flow in a curved pipe. The action value of the gas velocity at the pipe inlet at a given time is used to control the flow process, and then it enters... At that moment, continue with the next round of state perception and action decision-making;
[0007] Based on the particle deposition rate quality, inlet velocity, and pressure drop within the pipeline, a reward function is defined that considers the particle deposition rate and energy consumption constraints. The value function obtained at different times is calculated and recorded. After tracing each trajectory, the cumulative reward at the current instant is obtained, forming a training database. The artificial deep neural network is then updated using data from the training database.
[0008] Preferably, the artificial deep neural network includes a policy neural network and a value neural network. The policy neural network outputs action distribution parameters based on the state values provided by the flow field, and the value neural network evaluates the value of the current state based on the flow field information and entropy value.
[0009] Preferably, the boundaries of the numerical simulation of the gas-solid two-phase flow in the curved pipe are the pipe inlet, the pipe wall, and the pipe outlet.
[0010] Preferably, the pipe inlet is configured as a velocity inlet boundary condition, with the velocity direction perpendicular to the inlet surface.
[0011] Preferably, the fluid velocity at the pipe inlet is adjusted by one or more of the following methods: constant value adjustment, graded adjustment, continuous adjustment, or periodic adjustment, in order to achieve active control.
[0012] Preferably, the flow conditions under basic operating conditions are obtained. When the flow develops to the point where stable deposits appear in the pipe, this moment is used as the initial state for deep reinforcement learning training. At this moment, the gas phase fraction obtained by the probe set in the numerical simulation flow field is used as the initial state value, which is then used to output the corresponding value used to change the flow pattern. The action value of the gas velocity at the pipe inlet at a given time.
[0013] Preferably, the action value is the value of the agent in the first... The action variable output by each control cycle represents the value of the fluid velocity at the pipe inlet within that control cycle, and the value is constrained by a preset range.
[0014] Preferably, the reward function formula is as follows:
[0015]
[0016] In the formula, This represents the instantaneous reward value at time t; This represents the particle accumulation rate at the current time step; This represents the particle packing rate under basic operating conditions. Indicates the magnitude of the sudden change in pressure drop; Indicates the upper limit of pressure reduction; Indicates the current entry speed; Indicates the maximum inlet velocity; The weights are the particle deposition rates. The weighting of the changes in pressure drop within the pipeline. This is a constraint on the magnitude of the inlet velocity.
[0017] This invention provides a reward method based on particle deposition rate and energy consumption constraints in pipeline transportation. The ratio of the particle deposition rate calculated from post-processed numerical simulation data to the deposition rate under basic operating conditions is used as the primary objective term of the reward function. The ratio of pressure drop to the upper limit of pressure drop, and the ratio of inlet velocity to the upper limit of inlet velocity, are used as penalty terms in the reward function. Through continuous interaction between a deep neural network and the numerical simulation environment of the flow field, the network parameters are dynamically updated to obtain the optimal control strategy. Based on this control strategy, the inlet fluid velocity is adjusted to change the flow state and particle transport behavior within the pipe, thereby achieving coordinated control of particle deposition suppression and system energy consumption optimization. Attached Figure Description
[0018] Figure 1 Here is a flowchart of a reward method based on pipeline transport particle deposition rate and energy consumption constraints.
[0019] Figure 2 The performance curve of the training process using the reward method of this invention;
[0020] Figure 3 This is a comparison between the training results using the reward method of this invention and the basic operating conditions. Detailed Implementation
[0021] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.
[0022] like Figure 1 As shown in the figure, an embodiment of the present invention provides a reward method based on pipeline transport particle deposition rate and energy consumption constraints, comprising the following steps:
[0023] 1) Construct a deep reinforcement learning framework for numerical simulation of gas-solid two-phase flow in a curved pipe and an artificial deep neural network; acquire the experimental data required for the numerical simulation of gas-solid two-phase flow in the pipe and use it as input values; the overall flow field serves as the environment within the reinforcement learning framework, and the artificial deep neural network serves as the agent within the framework, including a policy neural network. Value neural networks and policy neural networks. The action distribution parameters are output based on the state values provided by the flow field; the value neural network evaluates the value of the current state based on the flow field information and entropy value.
[0024] In step 1), the policy neural network and the value neural network each contain one and four sets of neural networks, respectively. Each set of neural network structures contains two fully connected layers with 256 neurons per layer, and the activation function is set to the Adam function.
[0025] In step 1), the numerical simulation is based on the open-source platform OpenFOAM. The numerical simulation boundaries for the gas-solid two-phase flow in the pipeline are the pipeline inlet, pipeline wall, and pipeline outlet. The pipeline inlet is set as a velocity inlet boundary condition, with the velocity direction perpendicular to the inlet surface. In the numerical simulation, the fluid velocity at the pipeline inlet is adjustable, and the overall flow field serves as the environment within a reinforcement learning framework. Active control methods include one or more of the following: constant value adjustment, graded adjustment, continuous adjustment, or periodic adjustment of the inlet fluid velocity.
[0026] 2) The numerical simulation environment for gas-solid two-phase flow in a pipeline provides the state required by the agent. First, the flow conditions under basic operating conditions are obtained. When the flow develops to the point where stable deposition occurs in the pipeline, this moment is used as the initial state for deep reinforcement learning training. At this moment, the gas phase fraction obtained by the probe set in the numerical simulation flow field is used as the initial state value. .
[0027] 3) The agent, based on the current moment... Environmental transport status values Output the corresponding action value This action value is used to change The system records the gas velocity at the pipe inlet at a given time and uses this velocity to control the flow process. After completing the control update at that time, the system enters... At that moment, continue with the next round of state perception and action decision-making.
[0028] Step 3) For deep reinforcement learning agents in the first The action variable output in each control cycle represents the value of the fluid velocity at the pipe inlet within that control cycle, and the value is constrained by a preset range.
[0029] 4) Establish a reward function based on the particle deposition rate and energy consumption constraints within the pipeline, incorporating the particle deposition rate, inlet velocity, and pressure drop within the pipeline into the reward function, as shown in the following formula:
[0030]
[0031] In the formula, express t The instantaneous reward value at any given moment; This represents the particle accumulation rate at the current time step; This represents the particle packing rate under basic operating conditions. Indicates the magnitude of the sudden change in pressure drop; Indicates the upper limit of pressure reduction; Indicates the current entry speed; Indicates the maximum inlet velocity; The weights are the particle deposition rates. The weighting of the changes in pressure drop within the pipeline. The constraints on the inlet velocity are determined by different flow conditions.
[0032] Calculate the value function obtained at different times and record it. After tracing each trajectory, obtain the cumulative reward at the current instant. This forms the training database.
[0033] In step 4), the particle deposition rate inside the pipeline, the pressure drop inside the pipeline, and energy consumption together constitute the evaluation index of the reward function. The particle deposition rate inside the pipeline is calculated from the change in the mass of particles deposited on the pipe wall between adjacent control cycles. The pressure drop inside the pipeline is calculated from the inlet and outlet monitoring sections. Energy consumption is positively correlated with the inlet velocity. It is obtained directly from the inlet fluid velocity value.
[0034] In step 4), the method is applicable to particle transport and deposition control processes under laminar and turbulent flow, two-dimensional or three-dimensional, straight or curved pipe conditions.
[0035] In step 4), the particle deposition index is calculated from the data output by the post-processing module. The particle deposition index includes one or more of the following: total deposition amount, deposition rate, and average deposition amount. , The weighting coefficients for pressure drop and energy consumption indicators are set according to the operating conditions and control objectives.
[0036] 5) Update the value neural network using data from the training database.
[0037] In this embodiment of the invention, a reward function is constructed by combining the particle deposition rate mass, inlet velocity, and pressure drop inside the pipeline. The neural network is trained based on this reward function to obtain a suitable control strategy π, thereby achieving active control of the neural network and aiming to obtain the best optimization effect of reducing deposition inside the pipeline.
[0038] This invention addresses the challenge of balancing deposition suppression and system energy optimization in existing pipeline particle deposition control methods by proposing a closed-loop active control method based on deep reinforcement learning. This method employs a Soft Actor-Critic (SAC) algorithm as the training framework, fusing particle deposition indices, energy consumption indices, and pressure drop indices as a reward function to iteratively update the policy network and value network, thereby obtaining the optimal inlet velocity control strategy. Compared to traditional methods that rely on experience-based adjustment or fixed operating parameters, this invention can adaptively adjust the inlet fluid velocity according to the flow field state, dynamically changing the flow state and particle transport behavior within the pipe to effectively suppress particle deposition. Simultaneously, it considers pressure drop and energy consumption constraints, which helps reduce system energy consumption and improve control stability and overall operational performance.
[0039] Application Example 1:
[0040] 1. This example uses a two-dimensional vertical pipe as the object to construct a numerical simulation environment. The diameter of the bend is... The region is normalized using the feature length, and the region is calculated. The flow field consists of the inlet section, the pipe wall, and the horizontal outlet section. The inlet is located at... =1.5, the exit location is at =6 locations. In the numerical simulation, the continuous phase is air, and the discrete phase is particles. Information such as particle retention, particle distribution, and inlet / outlet pressure are obtained by solving the gas-solid two-phase flow. A structured grid is used in the computational domain, with mesh refinement in bends and near-wall deposition-sensitive areas to improve the accuracy of calculating local flow characteristics and particle deposition behavior. A transient solution method is used for time progression, with the initial time set to 0 and the time step size... Set to 0.001 s In this example, the initial inlet velocity is 3. Numerical simulations show that, under basic operating conditions, the initial particle packing rate is 7.2. ;
[0041] 2. Define the deep reinforcement learning process:
[0042] 1) First, the agent obtains the initialization strategy and determines the duration of a single trajectory, i.e., the control period. Forty-one probes are positioned within the bend in the flow field. The gas phase fraction obtained by the probe at the last instant of a single trajectory is used as the state value in the training data, based on the state values transported by the environment. Output action value Controlled by updated jet velocity values. Enter At each moment, the reward value is calculated, and this is recorded as a trajectory, denoted as... The reward function proposed in this invention includes the particle deposition rate, pressure drop, and inlet velocity within the pipe.
[0043]
[0044] In the formula: As the particle deposition rate weight, The weighting of the changes in pressure drop within the pipeline. The weights for the inlet velocity constraints are set to 2, 0.5, and 0.1, respectively. This represents the particle packing rate under basic operating conditions, obtained through numerical simulation; in this example, it is taken as 7.2. . This indicates the upper limit of pressure drop, which is taken as 10000Pa in this example; This represents the maximum inlet velocity, which is 10 in this example. .
[0045] Repeat the above process, recording 100 trajectories, then reset the environment to its initial state, marking one round. All trajectory values obtained after multiple rounds form an experience pool, which serves as the training database needed to update the agent.
[0046] 2) Extract data from the experience pool and update the neural network using mean squared error. After updating the neural network, repeat step 1), iterating 400 times to obtain the optimal strategy. Use the optimal strategy for testing to obtain the cumulative reward value, particle deposition rate in the pipeline, pressure drop, and inlet velocity changes. Specific results are as follows: Figure 2 As shown, Figure 2 (a) shows the curve of the change of reward value with the total amount of particles retained in the pipeline over 400 training cycles, and (b) shows the curve of the change of reward value with the velocity of particles retained in the pipeline (i.e., the continuous phase velocity at the inlet) over 400 training cycles.
[0047] 3) To verify the control effect of the reward mechanism of this invention, the results obtained from training using the reward mechanism of this invention will be compared with the basic operating conditions. For example... Figure 3 As shown, Figure 3(a) is a comparison curve of the total particle retention in the pipeline under the basic operating conditions and the reward method for 100 training trajectories; (b) is a comparison curve of the rate of change of the total particle retention in the pipeline under the basic operating conditions and the reward method for 100 training trajectories. These are, respectively, a comparison of the particle retention in the pipeline for 100 training trajectories and a comparison of the particle retention rate. Compared to the basic operating conditions, the control strategy trained using the reward mechanism of this invention can reduce the particle retention in the pipeline more quickly, and its particle retention rate is significantly better than that of the basic operating conditions. This indicates that the control strategy can effectively improve the particle transport state in the pipe, enhance the particle discharge capacity, thereby reducing the accumulation and retention of particles in the pipeline and effectively suppressing particle deposition.
[0048] Application Example 2:
[0049] To verify the applicability of this invention under different initial inlet velocities, this embodiment is based on Application Example 1, only changing the initial inlet velocity; the remaining computational domain, boundary conditions, particle parameters, control period, reward function construction method, and deep reinforcement learning training process are the same as in Application Example 1. Specifically, the initial inlet velocity is set to 1. Numerical simulations and reinforcement learning control simulations based on the reward mechanism of this invention were carried out under this working condition.
[0050] At the end of each control cycle, the particle retention rate, inlet velocity, and inlet-outlet pressure difference output by the post-processing module are read and used as state input to the deep neural network. The network outputs the current inlet velocity control value and calculates the reward value according to the reward function. After training for a preset number of rounds, the optimal control strategy under the initial inlet velocity condition is obtained.
[0051] The results show that, compared with the basic working condition, the present invention can still effectively reduce the particle retention and accelerate the particle retention rate under different initial inlet velocity conditions, indicating that the present invention has good working condition adaptability and generalization ability.
Claims
1. A reward method based on pipeline transport particle deposition rate and energy consumption constraints, characterized in that, Includes the following steps: A deep reinforcement learning framework for numerical simulation of gas-solid two-phase flow in a curved pipe and artificial deep neural network was established. Artificial deep neural networks, acting as the intelligent agents within the framework, output corresponding changes based on the states provided by numerical simulations of gas-solid two-phase flow in a curved pipe. The action value of the gas velocity at the pipe inlet at a given time is used to control the flow process, and then it enters... At time 1, continue to the next round of state perception and action decision-making; the action value is the agent's state at time 1. The action variable output in each control cycle represents the value of the fluid velocity at the pipe inlet within that control cycle, and the value is constrained by a preset range. Based on the particle deposition rate quality, inlet velocity, and pressure drop within the pipeline, a reward function is defined that considers the particle deposition rate and energy consumption constraints. The value function obtained at different times is calculated and recorded. After trajectories are completed, the cumulative reward at the current instant is obtained, forming a training database. The artificial deep neural network is then updated using data from the training database. The reward function formula is as follows: In the formula, This represents the instantaneous reward value at time t; This represents the particle accumulation rate at the current time step; This represents the particle packing rate under basic operating conditions. Indicates the magnitude of the sudden change in pressure drop; Indicates the upper limit of pressure drop; Indicates the current entry speed; Indicates the maximum inlet velocity; The weights are the particle deposition rates. The weighting of the changes in pressure drop within the pipeline. This is a constraint on the magnitude of the inlet velocity.
2. The reward method based on pipeline transport particle deposition rate and energy consumption constraints as described in claim 1, characterized in that, The artificial deep neural network includes a policy neural network and a value neural network. The policy neural network outputs action distribution parameters based on the state values provided by the flow field, while the value neural network evaluates the value of the current state based on the flow field information and entropy value.
3. The reward method based on pipeline transport particle deposition rate and energy consumption constraints as described in claim 1, characterized in that, The boundaries of the numerical simulation of gas-solid two-phase flow in the curved pipe are the pipe inlet, the pipe wall, and the pipe outlet.
4. The reward method based on pipeline transport particle deposition rate and energy consumption constraints as described in claim 3, characterized in that, The pipe inlet is set as a velocity inlet boundary condition, with the velocity direction perpendicular to the inlet surface.
5. The reward method based on pipeline transport particle deposition rate and energy consumption constraints as described in claim 3, characterized in that, Active control can be achieved by adjusting the fluid velocity at the pipe inlet using one or more of the following methods: constant value adjustment, graded adjustment, continuous adjustment, or periodic adjustment.
6. The reward method based on pipeline transport particle deposition rate and energy consumption constraints as described in claim 1, characterized in that, The flow conditions under basic operating conditions are obtained. When the flow develops to the point where stable deposits appear in the pipe, this moment is used as the initial state for deep reinforcement learning training. At this moment, the gas phase fraction obtained by the probe set in the numerical simulation flow field is used as the initial state value, which is then used to output the corresponding value to change the flow pattern. The action value of the gas velocity at the pipe inlet at a given time.