Heavy low-concentration waste paper pulp deslagging control method and system

Abstract

The invention relates to a waste paper pulp heavy low-concentration deslagging control method and system, and the method comprises the following steps: S1, collecting real-time process data from a DCS system through an OPC UA protocol, and forming an original data flow; s2, preprocessing the original data to obtain preprocessed data; s3, derivative features are calculated based on the preprocessed data, feature vectors are formed, and state representation containing historical information is constructed; s4, constructing a grey box model as a reinforcement learning environment in combination with a deslagging process physical mechanism and a data driving method; s5, based on the grey box model, constructing a structured state space, an action space and a reward function of reinforcement learning; and S6, according to the structured state space, the action space and the reward function, adopting an improved SAC algorithm to obtain an optimal control scheme. The waste paper pulp heavy low-concentration deslagging efficiency and reliability are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent manufacturing, and in particular to a method and system for controlling the removal of heavy, low-concentration waste paper pulp. Background Technology

[0002] With the advancement of waste paper resource recycling and green manufacturing concepts, waste paper pulping technology is playing an increasingly important role in the paper industry. The heavy slag removal process, as a key link in the waste paper pulping system, is responsible for removing heavy impurities such as sand, metal, and plastic from the pulp, directly affecting pulp purity, the safe operation of subsequent process equipment, and the quality of the finished paper. However, waste paper pulp is characterized by low concentration, complex composition, and strong rheological properties, and its operating conditions fluctuate with raw material changes and frequent start-stop cycles, placing higher demands on the control of the slag removal process.

[0003] Traditional slag removal control methods are mostly based on manual experience or conventional automation methods, which have the following prominent shortcomings: (1) It is difficult to handle multivariable, strongly coupled, and nonlinear process characteristics, and the adjustment accuracy and control response are limited; (2) It cannot detect the fluctuation of working conditions and changes in equipment status in a timely manner, and it is difficult to adapt to equipment aging and external disturbances; (3) The objectives are often singular, and it is difficult to balance multiple objectives such as slag removal efficiency, fiber loss rate, pulp concentration and energy consumption, resulting in limited operating economy and easy to cause misoperation and damage risks. Summary of the Invention

[0004] To address the aforementioned problems, the present invention aims to provide a method and system for controlling the removal of slag from heavy, low-concentration waste paper pulp, thereby effectively improving the efficiency and reliability of slag removal from heavy, low-concentration waste paper pulp.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for controlling the slag removal of heavy, low-consistency waste paper pulp includes the following steps:

[0007] S1: Collect real-time process data from the DCS system via the OPC UA protocol to form a raw data stream;

[0008] S2: Preprocess the original data to obtain the preprocessed data;

[0009] S3: Calculate derived features based on the preprocessed data, form feature vectors, and construct a state representation containing historical information;

[0010] S4: Combining the physical mechanism of the slag removal process with data-driven methods, a gray box model is constructed as a reinforcement learning environment;

[0011] S5: Based on the gray box model, construct the structured state space, action space, and reward function for reinforcement learning;

[0012] S6: Based on the structured state space, action space, and reward function, the improved SAC algorithm is used to obtain the optimal control scheme.

[0013] Furthermore, the preprocessing includes anomaly detection and correction, using the 3σ criterion to identify outliers and employing a Kalman filter to eliminate measurement noise, as detailed below:

[0014] Based on the collected raw data stream Draw(t)={Q in (t),C in (t),ΔP(t),T(t),f(t),v i P(t), P(t)}, with timestamps uniformly in the second range;

[0015] For each variable x t Create a sliding window W t Calculate the mean μ t Standard deviation σ t Anomaly detection is performed using 3σ discrimination, and data identified as anomaly is deleted.

[0016] A first-order random walk model is used for each continuous process variable:

[0017] State transition: ;

[0018] Observation equation: ;

[0019] Where, x t-1 The variable from the previous time step; w t For process noise; v t For observation noise; y t These are the system's observations;

[0020] Kalman filters are used to eliminate measurement noise.

[0021] initialization: ;

[0022] predict: ;

[0023] Where q is the variance of the process noise; r is the variance of the observation noise; The initial state estimate is y0, which is the first observation value by default; P0 is the variance of the initial estimate error, which is equal to the variance of the initial observation sequence. ;

[0024] renew:

[0025] ;

[0026] ;

[0027] ;

[0028] in, These are prior estimates; K represents the prior error covariance, i.e., the prediction uncertainty; Q is the process noise covariance matrix; R is the observation noise covariance matrix; K t For Kalman gain, ; Pt is the posterior estimate; Pt is the posterior estimation error covariance, i.e., the uncertainty of the state estimate at the current time; I is the identity matrix.

[0029] For variables with spikes, a small window median filter is first used before entering the Kalman filter to avoid spike contamination.

[0030] Furthermore, derived features are calculated, including unit energy consumption, sliding window variance, and load allocation ratio, forming a feature vector F(t), and a state representation incorporating historical information is constructed. , where N is the historical step size, to ensure the capture of system lag characteristics.

[0031] Furthermore, the gray box model adopts a three-layer fusion architecture consisting of a physical mechanism layer, a data-driven layer, and an uncertainty quantification layer. The physical mechanism layer ensures the interpretability and generalization ability of the model, based on fluid mechanics, swirling separation theory, and the basic laws of heat and mass transfer. The data-driven layer learns complex nonlinear couplings and equipment aging characteristics through neural networks. The uncertainty quantification layer processes the confidence intervals of model parameters and the uncertainty range of predictions, providing a probabilistic assessment of safety constraints.

[0032] Furthermore, the physical mechanism layer forms the basic framework of the model. Based on the core physical laws of the slag removal process, the first is the fluid dynamics relationship, describing the basic correlation between flow rate and pressure difference. According to Bernoulli's equation and the throttling principle, the flow rate through the valve is expressed as:

[0033] ;

[0034] Among them, C d (v) is the flow coefficient related to the valve opening v, A(v) is the equivalent flow area, and ΔP loss (Q) represents the pipeline friction loss; ΔP is the pressure difference across the valve; ρ(T) is the density of the fluid at temperature T;

[0035] The effect of temperature on slurry viscosity is described using Andrade's empirical formula:

[0036] ;

[0037] Where μ(T) is the dynamic viscosity of the fluid at temperature T; μ0 is the reference dynamic viscosity constant; B is the empirical constant; T is the current temperature; and T0 is the empirical zero-point temperature.

[0038] The core mechanism of cyclone separation is based on the balance between centrifugal force and resistance, defining the critical particle size d. 50 :

[0039] ;

[0040] Where D is the characteristic diameter of the hydrocyclone, ρ p and ρ f α, β, γ, λ, and κ are the particle and slurry densities, respectively; Q is the system flow rate; D is the characteristic diameter of the hydrocyclone; λ is the concentration influence coefficient; C in This refers to the total concentration of the feed slurry;

[0041] Based on the critical particle size, the separation selection function S(d) is defined as:

[0042] ;

[0043] Where d is the particle size of any particle; m is the steepness index;

[0044] For a given particle size distribution p d (d) Overall separation efficiency for:

[0045] ;

[0046] Pump power consumption is based on hydraulic power and pump efficiency P:

[0047] ;

[0048] Where H(Q,f) is the head characteristic curve, η pump (Q,f) represents the efficiency curve.

[0049] Furthermore, the data-driven layer employs a structured neural network to conditionally correct key physical parameters, and the correction for the critical particle size uses a multiplicative factor:

[0050] ;

[0051] Where, δ d (F) represents a data-driven correction term; d 50 The critical particle size is the data-driven corrected value; F is the eigenvector.

[0052] Correction to pump characteristics:

[0053] ;

[0054] Where H(Q,f) is the actual pump head, that is, the head actually measured by the system under the current flow rate Q and frequency f; H phys (Q,f) represents the theoretical pump head; η phys (Q,f) represents the theoretical pump efficiency; δH(F) and δη(F) are their respective data-driven correction terms.

[0055] Correcting valve characteristics to address actuator hysteresis and wear:

[0056] ;

[0057] Among them, C d (v) is the actual discharge coefficient; C d,phys (v) is the theoretical release coefficient; δ Cd (F) is a data-driven correction item;

[0058] Fiber loss modeling combines physically inspired foundational terms with data-driven corrections:

[0059] ;

[0060] Among them, L fiber Fiber loss rate; For constant bias terms; To control the impact of traffic on churn and its index; Q ref The baseline flow rate; Q is the influence coefficient of dilution water flow rate; w To dilute the water flow rate; Q out This refers to the total discharge flow rate; The effect of critical particle size on loss rate and its index; d ref Reference particle size; δ fibe r(F) is a data-driven correction term.

[0061] Furthermore, based on the gray-box model, a structured state space, action space, and reward function for reinforcement learning are constructed, as follows:

[0062] Based on the gray-box environment built by S4, the overall reinforcement learning framework is defined as a quadruple (S, A, T, R), where S is the state space, A is the action space, T is the state transition probability, and R is the reward function.

[0063] The state space is composed of four types of sub-states:

[0064] ;

[0065] Among them, s meas,t , s perf,t , s hist,t , s cons,tThese are respectively real-time measurement status, performance status, historical status, and constraint status;

[0066] The motion space employs continuous incremental control, meeting the smoothness requirements of industrial control and the physical limitations of the actuator. The motion vector is defined as follows:

[0067] ;

[0068] Among them, a t Let Δf be the action vector at time t; t This is the incremental control command for the pump frequency in this step; Δv w,t This represents the increment of the dilution water valve opening in this step; Δv i,t This represents the valve opening increment of the i-th slag remover in this step; n is the number of controllable valves.

[0069] The actuator update mechanism uses incremental integration and includes rate limiting.

[0070] ;

[0071] in It is a saturation function; These are the maximum rate of change of frequency and valve position, respectively; f t f is the pump frequency at the current moment; t+1 The pump frequency at the next moment; Δt is the sampling period or step time; f min ,f max These are the minimum and maximum allowable values ​​for the pump frequency; clip(x,a,b) is the clipping function that restricts x to the interval [a,b]; v i,t v represents the opening degree of the i-th valve at the current moment; i,t+1 Let i be the valve opening degree at the next moment;

[0072] And a safe projection mechanism is used to ensure that actions meet system constraints:

[0073] ;

[0074] in, The safety action vector is constrained and modified. The original action output by the intelligent agent; G is the square of the Euclidean distance; G is the constraint coefficient matrix; H is the constraint coefficient matrix;

[0075] a is the optimization variable; b is the upper bound vector of the constraints; st represents the constraint that is satisfied; Gs t +Ha≤b is a linear inequality constraint that ensures that the safe action of the output is within the physical / process-permissible range.

[0076] The reward function integrates the business objectives of waste paper removal and adopts a weighted linear combination form:

[0077] ;

[0078] Where, r perf,t r qual,t r energy,t r smooth,t and r safe,t These are respectively: main performance reward, quality stability reward, energy efficiency reward, operational stability reward, and safety constraint reward.

[0079] Furthermore, the primary performance bonus directly optimizes separation efficiency and fiber retention:

[0080] ;

[0081] Where w1 is the weighting coefficient for slag removal efficiency; w1 is the estimated slag removal and separation efficiency at the current moment; w2 is the weighting coefficient of the fiber loss penalty term; This is an estimate of the fiber loss rate at the current moment;

[0082] The quality stability bonus ensures the precision and stability of pulp concentration control.

[0083] ;

[0084] Where w3 is the weighting coefficient for the penalty of pulp concentration deviation; C out,t C represents the current concentration of the discharged slurry. sp σ represents the target pulp concentration; w4 is the weighting coefficient for pulp concentration fluctuation penalty; σ (w) (Cout) t Let w be the standard deviation of the pulp concentration within a sliding window of width w;

[0085] Energy efficiency bonus minimizes energy consumption per unit of product:

[0086] ;

[0087] Where w5 is the weighting coefficient of the energy consumption penalty term; SEC t This represents the current energy consumption per unit of pulp output;

[0088] P t Q represents the energy consumption at the current moment. out,t This represents the current pulp flow rate.

[0089] Smooth operation rewards to suppress excessively frequent control actions:

[0090] ;

[0091] Where w6 is the weighting coefficient for the amplitude penalty; w7 is the weighting coefficient for the acceleration penalty.

[0092] m is the number of action variables; The second difference of the i-th action measures the smoothness of the action.

[0093] Security constraints and rewards ensure system security through strong penalties and soft guidance.

[0094] ;

[0095] Among them, κ hard w8 represents the hard constraint penalty strength; w8 represents the soft penalty weight when approaching the constraint boundary.

[0096] It is an exponential soft function; hard constraints include ΔP>ΔPmax and equipment interlocking triggers; the soft-guided term uses constraint distance d. j,t When approaching the boundary, a negative reward is given, with a w8 weight of 1.

[0097] Furthermore, the improved SAC algorithm includes a policy network, two value function networks, two objective networks, and an adaptive temperature parameter network;

[0098] The policy network employs a deep deterministic structure, taking a structured state vector defined by S5 as input and outputting action mean and variance parameters. It utilizes a hierarchical encoder structure, optimized for a four-layer structured design in the state space. The bottom encoders handle measurement, performance, historical, and constraint states respectively, using different network depths and activation functions: the measurement state uses a 2-layer MLP to capture immediate responses; the performance state uses a 3-layer network to model complex performance relationships; the historical state uses a 1D-CNN to extract temporal features; and the constraint state uses a shallow network to quickly respond to safety signals. The outputs of the four sub-encoders are fused through an attention mechanism to generate a unified feature representation, which is then input into the policy head.

[0099] The output layer of the action network uses a dual-head structure, outputting the action mean μ and the log standard deviation logσ respectively. The action distribution adopts a truncated normal distribution to satisfy the action boundary constraints.

[0100] The value network employs a dual-Q structure to mitigate the overestimation problem. Each Q network uses a 4-layer fully connected network with ReLU activation and LayerNorm normalization. The input is a concatenation of state-action pairs, and the output is a single scalar Q value.

[0101] The reinforcement learning-based heavy-duty low-consistency waste paper pulp deslagging control system includes a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, it specifically performs the steps in the waste paper pulp heavy-duty low-consistency deslagging control method described above.

[0102] The present invention has the following beneficial effects:

[0103] 1. This invention adopts a three-layer fusion architecture of white-box physical mechanism framework + black-box data-driven correction + gray-scale uncertainty quantification. The physical mechanism layer ensures the interpretability and generalization ability of the model, based on fluid mechanics, swirling separation theory and basic laws of heat and mass transfer; the data-driven layer learns complex nonlinear coupling and equipment aging characteristics through neural networks; the uncertainty quantification layer processes the confidence interval of model parameters and the uncertainty range of prediction, providing probabilistic assessment of safety constraints.

[0104] 2. This invention employs an improved SAC optimization algorithm, which can continuously train and fine-tune under multi-objective weight adaptation, complex operating conditions, and equipment changes to obtain the optimal control strategy that combines high slag removal efficiency, low fiber loss, and energy saving. Through phased training, online adaptation, and safety constraint mechanisms, the system has good robustness and self-learning ability. Even in the face of raw material fluctuations, equipment performance degradation, or unknown anomalies, it can maintain stable production and superior operating economy, significantly improving the automation and intelligence level of the factory. Attached Figure Description

[0105] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0106] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0107] refer to Figure 1 In this embodiment, a method for controlling the removal of slag from heavy, low-concentration waste paper pulp is provided, comprising the following steps:

[0108] S1: Collect real-time process data from the DCS system via the OPC UA protocol, including signals from devices such as flow meters, concentration meters, differential pressure transmitters, temperature sensors, and power meters, forming the raw data stream Draw(t)={Qin(t),Cin(t),T(t),f(t),vi(t),P(t)};

[0109] S2: Preprocess the original data to obtain the preprocessed data Dclean(t);

[0110] S3: Calculate derived features based on the preprocessed data Dclean(t), calculate unit energy consumption SEC(t) = P(t) / Qout(t), and sliding window variance. (Window length τ=300s), load distribution ratio Rload(t)=Qi(t) / ∑Qj(t), forming the feature vector F(t), and constructing a state representation containing historical information. , where N is the historical step size, to ensure the capture of system hysteresis characteristics;

[0111] S4: Combining the physical mechanism of the slag removal process with data-driven methods, a gray box model is constructed as a reinforcement learning environment;

[0112] S5: Based on the gray box model, construct the structured state space, action space, and reward function for reinforcement learning;

[0113] S6: Based on the structured state space, action space, and reward function, the improved SAC algorithm is used to obtain the optimal control scheme.

[0114] In this embodiment, preprocessing includes anomaly detection and correction. Outliers are identified using the 3σ criterion, and measurement noise is eliminated using a Kalman filter, as detailed below:

[0115] Based on the collected raw data stream Draw(t)={Q in (t),C in (t),ΔP(t),T(t),f(t),v i P(t), P(t)}, with timestamps uniformly in the second range;

[0116] For each variable x t Create a sliding window W t Calculate the mean μ t Standard deviation σ t Anomaly detection is performed using 3σ discrimination, and data identified as anomaly is deleted.

[0117] For each continuous process quantity (e.g., Q) in C in The model ΔP,T,P is a first-order random walk model.

[0118] State transition: ;

[0119] Observation equation: ;

[0120] Where, x t-1 The variable from the previous time step; w t For process noise; v t For observation noise; y t These are the system's observations;

[0121] Kalman filters are used to eliminate measurement noise.

[0122] initialization: ;

[0123] predict: ;

[0124] Where q is the variance (or covariance, if multivariable) of the process noise; r is the variance (or covariance, if multivariable) of the observation noise. The initial state estimate is y0, which is the first observation value by default; P0 is the variance of the initial estimate error, which is equal to the variance of the initial observation sequence. ;

[0125] renew:

[0126] ;

[0127] ;

[0128] ;

[0129] in, These are prior estimates; Let be the prior error covariance, i.e., the prediction uncertainty; Q is the process noise covariance matrix (scalar type is q); representing the dynamic noise intensity of the model; R is the observation noise covariance matrix (scalar type is r); representing the sensor noise intensity; K t Kt is the Kalman gain, which measures the relative weight of the current observation yt and the prior estimate in the state correction; the larger Kt is, the more trusted observations there are; the smaller Kt is, the more trusted model predictions there are. Pt is the posterior estimate, which is the optimal estimate of the state after combining prediction and observation; Pt is the posterior estimation error covariance, i.e. the uncertainty of the state estimate at the current time; I is the identity matrix.

[0130] For variables with spikes (such as ΔP), first use a small window median filter before entering the Kalman filter to avoid spike contamination.

[0131] In this embodiment, the derived characteristics are calculated, including the unit energy consumption SEC(t) = P(t) / Q. out (t), sliding window variance (Window length τ = 300s), load distribution ratio R load (t)=Q i (t) / ∑Q j (t), forming the feature vector F(t), and constructing a state representation containing historical information. , where N is the historical step size, to ensure the capture of system lag characteristics.

[0132] In this embodiment, the gray box model adopts a three-layer fusion architecture consisting of a physical mechanism layer, a data-driven layer, and an uncertainty quantification layer. The physical mechanism layer ensures the interpretability and generalization ability of the model, based on fluid mechanics, swirling separation theory, and the basic laws of heat and mass transfer. The data-driven layer learns complex nonlinear couplings and equipment aging characteristics through neural networks. The uncertainty quantification layer processes the confidence intervals of model parameters and the uncertainty range of predictions, providing a probabilistic assessment of safety constraints.

[0133] In this embodiment, the physical mechanism layer forms the basic framework of the model. Based on the core physical laws of the slag removal process, the first is the fluid dynamics relationship, describing the basic correlation between flow rate and pressure difference. According to Bernoulli's equation and the throttling principle, the flow rate through the valve is expressed as:

[0134] ;

[0135] Among them, C d (v) is the flow coefficient related to the valve opening v, A(v) is the equivalent flow area, and ΔP loss (Q) represents the pipeline friction loss; ΔP is the pressure difference across the valve; ρ(T) is the density of the fluid at temperature T;

[0136] The effect of temperature on slurry viscosity is described using Andrade's empirical formula:

[0137] ;

[0138] Where μ(T) is the dynamic viscosity of the fluid at temperature T; μ0 is the reference dynamic viscosity constant; B is the empirical constant; T is the current temperature; and T0 is the empirical zero-point temperature.

[0139] Viscosity variation directly affects the critical particle size and separation efficiency of cyclone separation, which is the key link between operating conditions and separation performance.

[0140] The core mechanism of cyclone separation is based on the balance between centrifugal force and resistance, defining the critical particle size d. 50 (Particle diameter corresponding to 50% separation efficiency):

[0141] ;

[0142] Where D is the characteristic diameter of the hydrocyclone, ρ p and ρ f α, β, γ, λ, and κ are the particle and slurry densities, respectively; Q is the system flow rate; D is the characteristic diameter of the hydrocyclone; λ is the concentration influence coefficient; C in This refers to the total concentration of the feed slurry;

[0143] Based on the critical particle size, the separation selection function S(d) is defined as:

[0144] ;

[0145] Where d is the particle size of any particle; m is the steepness index;

[0146] For a given particle size distribution p d (d) Overall separation efficiency for:

[0147] ;

[0148] Pump power consumption is based on hydraulic power and pump efficiency P:

[0149] ;

[0150] Where H(Q,f) is the head characteristic curve, η pump (Q,f) represents the efficiency curve.

[0151] In this embodiment, the data-driven layer uses a structured neural network to conditionally correct key physical parameters, maintaining interpretability while improving prediction accuracy. The correction for the critical particle size is performed using a multiplicative factor.

[0152] ;

[0153] Where, δ d (F) is a data-driven correction term, employing a multilayer perceptron (MLP). Its input is a feature vector F from S3, containing current operating conditions, historical statistics, and device status information; d 50 The critical particle size is determined by data-driven correction; F is the eigenvector; exponential form is used to ensure d 50 >0, and δ d The output is limited to a reasonable range (e.g., ±0.3) by tanh activation to avoid excessive deviation from the physical reference.

[0154] Correction to pump characteristics:

[0155] ;

[0156] Where H(Q,f) is the actual pump head, that is, the head actually measured by the system under the current flow rate Q and frequency f; H phys (Q,f) represents the theoretical pump head; η phys (Q,f) represents the theoretical pump efficiency; δH(F) and δη(F) are their respective data-driven correction terms.

[0157] Correcting valve characteristics to address actuator hysteresis and wear:

[0158] ;

[0159] Among them, C d (v) is the actual discharge coefficient; C d,phys (v) is the theoretical release coefficient; δ Cd (F) is a data-driven correction item;

[0160] Fiber loss modeling combines physically inspired foundational terms with data-driven corrections:

[0161] ;

[0162] Among them, L fiber Fiber loss rate; This is a constant bias term, reflecting the level of basic loss; To control the impact of traffic on churn and its index; Q ref The baseline flow rate; Q is the influence coefficient of dilution water flow rate; w To dilute the water flow rate; Q out This refers to the total discharge flow rate; The effect of critical particle size on loss rate and its index; d ref Reference particle size; δ fibe r(F) is a data-driven correction term that compensates for the physical model residuals to more accurately reflect the fiber loss phenomenon on site.

[0163] In this embodiment, based on the gray-box model, a structured state space, action space, and reward function for reinforcement learning are constructed, as follows:

[0164] Based on the gray-box environment built by S4, the task of S5 is to design a formal description of a Markov Decision Process (MDP) suitable for reinforcement learning algorithms. The design follows four core principles: completeness (the state contains sufficient decision information), observability (the state can be measured directly or indirectly by sensors), controllability (actions can effectively affect the system state), and safety (constraints are clearly modeled and handled).

[0165] Based on the gray-box environment built by S4, the overall reinforcement learning framework is defined as a quadruple (S,A,T,R), where S is the state space, A is the action space, T is the state transition probability (determined by the gray-box model), and R is the reward function.

[0166] The state space is composed of four types of sub-states:

[0167] ;

[0168] Among them, s meas,t , s perf,t , shist,t , s cons,t These are respectively the real-time measurement status, performance status (calculated online from the gray box model), historical status (multi-window mean / variance / slope, or short history stacking), and constraint status;

[0169] Measurement state s meas,t Includes direct sensor measurements and setpoints; performance status. perf,t Includes performance metrics calculated using the gray-box model; historical states. hist,t Includes statistical features across multiple time scales; constraint states s cons,t Quantify the distance from the system to various constraint boundaries;

[0170] The motion space employs continuous incremental control, meeting the smoothness requirements of industrial control and the physical limitations of the actuator. The motion vector is defined as follows:

[0171] ;

[0172] Among them, a t Let Δf be the action vector at time t; t This is the incremental control command for the pump frequency in this step; Δv w,t This represents the increment of the dilution water valve opening in this step; Δv i,t This represents the increment of the opening of the i-th (out of n) slag remover valve in this step; n is the number of controllable valves;

[0173] The actuator update mechanism uses incremental integration and includes rate limiting.

[0174] ;

[0175] in For a saturated function, restrict x to... Within the range; These are the maximum rate of change of frequency and valve position, respectively; f t f is the pump frequency at the current moment; t+1 The pump frequency at the next moment; Δt is the sampling period or step time; f min ,f max These are the minimum and maximum allowable values ​​for the pump frequency; clip(x,a,b) is the clipping function that restricts x to the interval [a,b]; v i,t v represents the opening degree of the i-th valve at the current moment; i,t+1 Let i be the valve opening degree at the next moment;

[0176] And a safe projection mechanism is used to ensure that actions meet system constraints:

[0177] ;

[0178] in, The safety action vector is constrained and modified. The original (unconstrained) actions output by the agent; G is the square of the Euclidean distance (distance metric); G is the constraint coefficient matrix (corresponding to the constraints in the state part); H is the constraint coefficient matrix (corresponding to the constraints in the action part).

[0179] a is the optimization variable (action vector); b is the upper bound vector of the constraints; st indicates that the following constraints are satisfied; Gs t +Ha≤b is a linear inequality constraint that ensures that the safe action of the output is within the physical / process-permissible range.

[0180] The reward function integrates the business objectives of waste paper removal and adopts a weighted linear combination form:

[0181] ;

[0182] Where, r perf,t r qual,t r energy,t r smooth,t and r safe,t These are respectively: main performance reward, quality stability reward, energy efficiency reward, operational stability reward, and safety constraint reward.

[0183] In this embodiment, the primary performance reward directly optimizes separation efficiency and fiber retention:

[0184] ;

[0185] Where w1 is the weighting coefficient for slag removal efficiency; w1 is the estimated slag removal and separation efficiency at the current moment; w2 is the weighting coefficient of the fiber loss penalty term; This is an estimate of the fiber loss rate at the current moment;

[0186] The quality stability bonus ensures the precision and stability of pulp concentration control.

[0187] ;

[0188] Where w3 is the weighting coefficient for the penalty of pulp concentration deviation; C out,t C represents the current concentration of the discharged slurry. sp w4 is the target pulp concentration (set value); w4 is the weighting coefficient for pulp concentration fluctuation penalty; σ (w) (Cout) t Let w be the standard deviation of the pulp concentration within a sliding window of width w;

[0189] Energy efficiency bonus minimizes energy consumption per unit of product:

[0190] ;

[0191] Where w5 is the weighting coefficient of the energy consumption penalty term; SEC t This represents the current energy consumption per unit of pulp output;

[0192] P t Q represents the energy consumption at the current moment. out,t This represents the current pulp flow rate.

[0193] Smooth operation rewards to suppress excessively frequent control actions:

[0194] ;

[0195] Where w6 is the weighting coefficient for the amplitude penalty; w7 is the weighting coefficient for the acceleration penalty.

[0196] m is the number of action variables; The second difference of the i-th action measures the smoothness of the action.

[0197] Security constraints and rewards ensure system security through strong penalties and soft guidance.

[0198] ;

[0199] Among them, κ hard w8 represents the hard constraint penalty strength; w8 represents the soft penalty weight when approaching the constraint boundary.

[0200] It is an exponential soft function; hard constraints include ΔP>ΔPmax, equipment interlocking triggering, and a penalty value κ. hard =100; Soft guide terms use constraint distance d j,t When approaching the boundary, a negative reward is given, with a w8 weight of 1.

[0201] In this embodiment, the improved SAC algorithm includes a policy network (Actor), two value function networks (Critic Q1 / Q2), two target networks (Target Q1 / Q2), and an adaptive temperature parameter network;

[0202] The policy network employs a deep deterministic structure, taking a structured state vector defined by S5 as input and outputting action mean and variance parameters. It utilizes a hierarchical encoder structure, optimized for a four-layer structured design in the state space. The bottom encoders handle measurement, performance, historical, and constraint states respectively, using different network depths and activation functions: the measurement state uses a 2-layer MLP to capture immediate responses; the performance state uses a 3-layer network to model complex performance relationships; the historical state uses a 1D-CNN to extract temporal features; and the constraint state uses a shallow network to quickly respond to safety signals. The outputs of the four sub-encoders are fused through an attention mechanism to generate a unified feature representation, which is then input into the policy head.

[0203] The output layer of the action network uses a dual-head structure, outputting the action mean μ and the log standard deviation logσ respectively. The action distribution adopts a truncated normal distribution to satisfy the action boundary constraints.

[0204] Specifically, the parameterization is as follows: μ is linearly scaled to the action range after activation by tanh; logσ is limited to a reasonable range (e.g., [-10, 2]) by the clip function to avoid numerical instability; the final action is sampled by reparameterization: a = μ + σ * ε, where ε ~ N(0,1);

[0205] The value network employs a dual-Q structure to mitigate the overestimation problem. Each Q network uses a 4-layer fully connected network with hidden layer dimensions of 256-256-128-64. It uses ReLU activation and LayerNorm normalization. The input is a concatenation of state-action pairs, and the output is a single scalar Q value.

[0206] The reinforcement learning-based heavy-consistency, low-concentration waste paper pulp deslagging control system includes a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, it specifically performs the steps in the waste paper pulp heavy-consistency, low-concentration deslagging control method described above.

[0207] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0208] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0209] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0210] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0211] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for controlling the removal of slag from heavy, low-concentration waste paper pulp, characterized in that, Includes the following steps: S1: Collect real-time process data from the DCS system via the OPC UA protocol to form a raw data stream; S2: Preprocess the original data to obtain the preprocessed data; S3: Calculate derived features based on the preprocessed data, form feature vectors, and construct a state representation containing historical information; S4: Combining the physical mechanism of the slag removal process with data-driven methods, a gray box model is constructed as a reinforcement learning environment; S5: Based on the gray box model, construct the structured state space, action space, and reward function for reinforcement learning; S6: Based on the structured state space, action space, and reward function, the improved SAC algorithm is used to obtain the optimal control scheme.

2. The method for controlling the slag removal of heavy, low-concentration waste paper pulp according to claim 1, characterized in that, The preprocessing includes anomaly detection and correction, using the 3σ criterion to identify outliers and employing a Kalman filter to eliminate measurement noise, as detailed below: Based on the collected raw data stream, the timestamps are standardized to the second level; for each variable x t Create a sliding window W t Calculate the mean μ t Standard deviation σ t Anomaly detection is performed using 3σ discrimination, and data identified as anomaly is deleted. A first-order random walk model is used for each continuous process variable: State transition: ; Observation equation: ; Where, x t-1 The variable from the previous time step; w t For process noise; v t For observation noise; y t These are the system's observations; Kalman filters are used to eliminate measurement noise. initialization: ; predict: ; Where q is the variance of the process noise; r is the variance of the observation noise; The initial state estimate is y0, which is the first observation value by default; P0 is the variance of the initial estimate error, which is equal to the variance of the initial observation sequence. ; renew: ； ； ； in, These are prior estimates; K represents the prior error covariance, i.e., the prediction uncertainty; Q is the process noise covariance matrix; R is the observation noise covariance matrix; K t For Kalman gain, ; Pt is the posterior estimate; Pt is the posterior estimation error covariance, i.e., the uncertainty of the state estimate at the current time; I is the identity matrix. For variables with spikes, a small window median filter is first used before entering the Kalman filter to avoid spike contamination.

3. The method for controlling the slag removal of heavy, low-concentration waste paper pulp according to claim 1, characterized in that, The computational derived features include calculating unit energy consumption, sliding window variance, and load allocation ratio, forming a feature vector F(t), and constructing a state representation containing historical information. , where N is the historical step size, to ensure the capture of system lag characteristics.

4. The method for controlling the slag removal of heavy, low-consistency waste paper pulp according to claim 1, characterized in that, The gray box model adopts a three-layer fusion architecture consisting of a physical mechanism layer, a data-driven layer, and an uncertainty quantification layer. The physical mechanism layer ensures the interpretability and generalization ability of the model, based on fluid mechanics, swirling separation theory, and the basic laws of heat and mass transfer. The data-driven layer learns complex nonlinear couplings and equipment aging characteristics through neural networks; the uncertainty quantification layer processes the confidence intervals of model parameters and the range of uncertainty in predictions, providing a probabilistic assessment of safety constraints.

5. The method for controlling the slag removal of heavy, low-concentration waste paper pulp according to claim 4, characterized in that, The physical mechanism layer forms the basic framework of the model. Based on the core physical laws of the slag removal process, it first describes the fluid dynamics relationship, describing the basic correlation between flow rate and pressure difference. According to Bernoulli's equation and the throttling principle, the flow rate through the valve is expressed as: ； Among them, C d (v) is the flow coefficient related to the valve opening v, A(v) is the equivalent flow area, and ΔP loss (Q) represents the pipeline friction loss; ΔP is the pressure difference across the valve; ρ(T) is the density of the fluid at temperature T; The effect of temperature on slurry viscosity is described using Andrade's empirical formula: ； Where μ(T) is the dynamic viscosity of the fluid at temperature T; μ0 is the reference dynamic viscosity constant; B is the empirical constant; T is the current temperature; and T0 is the empirical zero-point temperature. The core mechanism of cyclone separation is based on the balance between centrifugal force and resistance, defining the critical particle size d. 50 : ； Where D is the characteristic diameter of the hydrocyclone, ρ p and ρ f α, β, γ, λ, and κ are the particle and slurry densities, respectively; Q is the system flow rate; D is the characteristic diameter of the hydrocyclone; λ is the concentration influence coefficient; C in This refers to the total concentration of the feed slurry; Based on the critical particle size, the separation selection function S(d) is defined as: ； Where d is the particle size of any particle; m is the steepness index; For a given particle size distribution p d (d) Overall separation efficiency for: ； Pump power consumption is based on hydraulic power and pump efficiency P: ； Where H(Q,f) is the head characteristic curve, η pump (Q,f) represents the efficiency curve.

6. The method for controlling the slag removal of heavy, low-concentration waste paper pulp according to claim 5, characterized in that, The data-driven layer uses a structured neural network to conditionally correct key physical parameters, and the correction of the critical particle size is performed using a multiplicative factor. ； Where, δ d (F) represents a data-driven correction term; d 50 The critical particle size is the data-driven corrected value; F is the eigenvector. Correction to pump characteristics: ； Where H(Q,f) is the actual pump head, that is, the head actually measured by the system under the current flow rate Q and frequency f; H phys (Q,f) represents the theoretical pump head; η phys (Q,f) represents the theoretical pump efficiency; δH(F) and δη(F) are their respective data-driven correction terms. Correcting valve characteristics to address actuator hysteresis and wear: ； Among them, C d (v) is the actual discharge coefficient; C d,phys (v) is the theoretical release coefficient; δ Cd (F) is a data-driven correction item; Fiber loss modeling combines physically inspired foundational terms with data-driven corrections: ； Among them, L fiber Fiber loss rate; For constant bias terms; To control the impact of traffic on churn and its index; Q ref The baseline flow rate; Q is the influence coefficient of dilution water flow rate; w To dilute the water flow rate; Q out This refers to the total discharge flow rate; The effect of critical particle size on loss rate and its index; d ref Reference particle size; δ fibe r(F) is a data-driven correction term.

7. The method for controlling the slag removal of heavy, low-consistency waste paper pulp according to claim 6, characterized in that, The structured state space, action space, and reward function for reinforcement learning, based on the gray-box model, are constructed as follows: Based on the gray-box environment built by S4, the overall reinforcement learning framework is defined as a quadruple (S, A, T, R), where S is the state space, A is the action space, T is the state transition probability, and R is the reward function. The state space is composed of four types of sub-states: ； Among them, s meas,t , s perf,t , s hist,t , s cons,t These are respectively real-time measurement status, performance status, historical status, and constraint status; The motion space employs continuous incremental control, meeting the smoothness requirements of industrial control and the physical limitations of the actuator. The motion vector is defined as follows: ； Among them, a t Let Δf be the action vector at time t; t This is the incremental control command for the pump frequency in this step; Δv w,t This represents the increment of the dilution water valve opening in this step; Δv i,t This represents the valve opening increment of the i-th slag remover in this step; n is the number of controllable valves. The actuator update mechanism uses incremental integration and includes rate limiting. ； in It is a saturation function; These are the maximum rate of change of frequency and valve position, respectively; f t f is the pump frequency at the current moment; t+1 The pump frequency at the next moment; Δt is the sampling period or step time; f min ,f max These are the minimum and maximum allowable values ​​for the pump frequency; clip(x,a,b) is the clipping function that restricts x to the interval [a,b]; v i,t v represents the opening degree of the i-th valve at the current moment; i,t+1 Let i be the valve opening degree at the next moment; And a safe projection mechanism is used to ensure that actions meet system constraints: ； in, The safety action vector is constrained and modified. The original action output by the intelligent agent; G is the square of the Euclidean distance; G is the constraint coefficient matrix; H is the constraint coefficient matrix; a is the optimization variable; b is the upper bound vector of the constraints; st represents the constraint that is satisfied; Gs t +Ha≤b is a linear inequality constraint that ensures that the safe action of the output is within the physical / process-permissible range. The reward function integrates the business objectives of waste paper removal and adopts a weighted linear combination form: ； Where, r perf,t r qual,t r energy,t r smooth,t and r safe,t These are respectively: main performance reward, quality stability reward, energy efficiency reward, operational stability reward, and safety constraint reward.

8. The method for controlling the slag removal of heavy, low-consistency waste paper pulp according to claim 7, characterized in that, The primary performance bonus directly optimizes separation efficiency and fiber retention: ； Where w1 is the weighting coefficient for slag removal efficiency; w1 is the estimated slag removal and separation efficiency at the current moment; w2 is the weighting coefficient of the fiber loss penalty term; This is an estimate of the fiber loss rate at the current moment; The quality stability bonus ensures the precision and stability of pulp concentration control. ； Where w3 is the weighting coefficient for the penalty of pulp concentration deviation; C out,t C represents the current concentration of the discharged slurry. sp σ represents the target pulp concentration; w4 is the weighting coefficient for pulp concentration fluctuation penalty; σ (w) (Cout) t Let w be the standard deviation of the pulp concentration within a sliding window of width w; Energy efficiency bonus minimizes energy consumption per unit of product: ； Where w5 is the weighting coefficient of the energy consumption penalty term; SEC t This represents the current energy consumption per unit of pulp output; P t Q represents the energy consumption at the current moment. out,t This represents the current pulp flow rate. Smooth operation rewards to suppress excessively frequent control actions: ； Where w6 is the weighting coefficient for the amplitude penalty; w7 is the weighting coefficient for the acceleration penalty. m is the number of action variables; The second difference of the i-th action measures the smoothness of the action. Security constraints and rewards ensure system security through strong penalties and soft guidance. ； Among them, κ hard w8 represents the hard constraint penalty strength; w8 represents the soft penalty weight when approaching the constraint boundary. It is an exponential soft function; hard constraints include ΔP>ΔPmax and equipment interlocking triggers; the soft-guided term uses constraint distance d. j,t When approaching the boundary, a negative reward is given, with a w8 weight of 1.

9. The method for controlling the slag removal of heavy, low-consistency waste paper pulp according to claim 1, characterized in that, The improved SAC algorithm includes a policy network, two value function networks, two target networks, and an adaptive temperature parameter network. The policy network adopts a deep deterministic structure, with the input being a structured state vector defined by S5 and the output being the action mean and variance parameters. The policy network adopts a hierarchical encoder structure, which is optimized for a four-layer structured design in the state space. The bottom encoders handle the measurement state, performance state, historical state, and constraint state respectively, using different network depths and activation functions: the measurement state uses a 2-layer MLP to capture the immediate response. The performance status is modeled using a 3-layer network to depict complex performance correlations; the historical status is extracted using a 1D-CNN to extract temporal features. The constraint state uses a shallow network to quickly respond to safety signals; the outputs of the four sub-encoders are fused through an attention mechanism to generate a unified feature representation that is input into the policy head. The output layer of the action network uses a dual-head structure, outputting the action mean μ and the log standard deviation logσ respectively. The action distribution adopts a truncated normal distribution to satisfy the action boundary constraints. The value network employs a dual-Q structure to mitigate the overestimation problem. Each Q network uses a 4-layer fully connected network with ReLU activation and LayerNorm normalization. The input is a concatenation of state-action pairs, and the output is a single scalar Q value.

10. A reinforcement learning-based heavy-duty, low-concentration waste paper pulp deslagging control system, characterized in that, It includes a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, it specifically performs the steps in the method for controlling the heavy and low-concentration slag removal of waste paper pulp as described in any one of claims 1-9.

Images