Secondary heuristic planning multivariable sewage treatment process control method based on echo state network
By employing a multivariable online control strategy based on a leakage integral echo state network, the problem of inaccurate control of dissolved oxygen and nitrate nitrogen concentrations in traditional wastewater treatment was solved, achieving stable compliance of effluent quality and efficient system operation.
Patent Information
- Application Number
- CN202411050601.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-01
AI Technical Summary
Traditional wastewater treatment control methods are difficult to respond to environmental changes in real time, resulting in inaccurate control of dissolved oxygen and nitrate nitrogen concentrations, and the effluent quality is prone to exceed the standards, especially under conditions of significant disturbance.
A multivariable online control strategy based on a leaky integral echo state network is adopted, which combines strategy iteration and adaptive learning, uses gradient descent algorithm to update network parameters, and performs stability analysis through Lyapunov function to design the DHP-LESN controller.
It achieves precise control of dissolved oxygen and nitrate nitrogen concentrations, ensuring stable effluent quality and improving the operational stability and control accuracy of the wastewater treatment system.
Smart Images

Figure CN119002260A_ABST
Abstract
Description
Technical Field
[0001] The invention is applied to the process of urban sewage treatment, is based on a leakage integral echo state network, and considers adaptive dynamic programming to handle nonlinear system control problems, thereby realizing multivariable online control. Background Art
[0002] As global freshwater resources become increasingly scarce, effectively treating sewage has become a major challenge faced by many countries. As a key way to alleviate this pressure, the operating efficiency and treatment effect of sewage treatment are particularly important. The sewage treatment process involves complex biochemical reactions, which are characterized by high nonlinearity and uncertainty. Although traditional control methods can maintain the effluent quality to a certain extent, they rely on manual operations and are difficult to respond to environmental changes in real time, resulting in unstable treatment effects and effluent quality that easily exceeds standards. Therefore, it is an urgent problem to solve in the sewage treatment process to study appropriate intelligent control strategies for sewage treatment processes, achieve the stability of the control system, and ensure that the effluent quality is stable and meets standards. It is crucial to ensure that the effluent quality meets standards and achieve efficient and stable operation of sewage treatment.
[0003] In the sewage treatment control process, dissolved oxygen and nitrate nitrogen concentrations are the main control variables for the stable operation of the sewage treatment system. At present, most control methods can achieve stable output of effluent water quality under stable working conditions. However, under the condition of large interference, the dissolved oxygen and nitrate nitrogen concentrations affect each other, and are affected by uncontrollable factors such as fluctuations in influent flow, changes in pollutant concentrations, temperature and pH changes. Traditional control methods are difficult to achieve precise control of dissolved oxygen and nitrate nitrogen concentrations at the same time.
[0004] Adaptive dynamic programming provides a new way to solve nonlinear system control problems and can avoid the "curse of dimensionality" in dynamic programming. Therefore, the present invention proposes a quadratic heuristic programming multivariable online control strategy based on a leaky integral echo state network, which can achieve stable effluent water quality while ensuring that the control accuracy meets the ideal requirements. Summary of the invention
[0005] The present invention designs a quadratic heuristic programming multivariable controller (DHP-LESN) based on leaky integral echo state network to achieve tracking control of dissolved oxygen and nitrate nitrogen concentrations. Firstly, an online control framework is designed by combining strategy iteration and adaptive learning. Secondly, the gradient descent algorithm is used to update the network parameters in the control framework online. Then, the Lyapunov function is introduced for stability analysis, and the consistent ultimate boundedness of the system error and related parameters is proved. Finally, simulation experiments are carried out based on the BSM1 platform, and the experimental results verify the effectiveness and superiority of the proposed controller.
[0006] The specific technical solution of the present invention is:
[0007] Step 2.1: Analysis of optimal control problem of sewage treatment process:
[0008] The optimal control problem of the sewage treatment process can be expressed as:
[0009]
[0010] Where F(·,·) represents the unknown functional relationship, S O,5 (k) and S NO,2 (k) represents the dissolved oxygen concentration in the fifth zone and the nitrate nitrogen concentration in the second zone, respectively. L a5(k) represents the oxygen transfer coefficient, Q a (k) represents the internal recirculation rate, k is an arbitrary natural number, and N is a set of natural numbers. Therefore, the changes in dissolved oxygen concentration and nitrate nitrogen concentration x(k+1) can be regarded as the following formula:
[0011] x(k+1)=F(x(k),v(k)),k∈N (2)
[0012] Where x(k) represents the matrix [S O,5 (k), S NO,2 (k)], v(k) represents the matrix [K L a5(k), Q a (k)] changes.
[0013] Assumption 1: The state x(k) can be observed from the kth step of the control system.
[0014] Assumption 2: Assuming the control gain matrix is positive definite, then
[0015]
[0016] where (x(k),v(k))∈R + ×R + For any value in R + is a positive real number, and det[·] is an operation to find the determinant of a matrix.
[0017] The loss function J(k) in the policy iteration algorithm is defined as follows:
[0018]
[0019] Where U(i) is the utility function, and λ∈(0,1] is any value. The utility function is defined as:
[0020]
[0021] where x *(k) is the optimal value of x(k). According to Bellman's optimality principle and greedy algorithm, the optimal loss function J * (k) is defined as:
[0022] J * (k) = min (U (k) + λ J * (k+1)) (6)
[0023] Among them J * (k+1) is the optimal loss function at the next moment, then the optimal control vector v * (k) satisfies the following formula:
[0024]
[0025] Step 2.2: Determine the leaky integration echo state network:
[0026] Leaky Integral Echo State Network (LESN) has reservoir neurons with leaky integral properties and has been successfully used in feedback control systems. LESN mainly consists of three layers: input layer, reservoir and output layer, and the number of network nodes is set to K, N and M respectively. The input vector, reservoir state vector and output vector of LESN are i(k), z(k) and y(k) respectively. The input weight matrix is W in ∈W N×K For any value in, the state matrix of the reservoir is W∈W N×N The output weight matrix is W out ∈W( N+K ) ×M Any value in . Then the update formulas of the LESN's reserve pool state vector z(k) and output vector y(k) are:
[0027] z(k)=(1-α)z(k-1)+αf(W in i(k)+W(k-1)) (8)
[0028]
[0029] The network parameter leakage rate α∈(0,1] is an arbitrary value; W(k-1) is the reservoir matrix at the previous moment; f(·) represents the activation function of the reservoir, which uses the hyperbolic tangent function to enhance the network's ability to handle nonlinear environments. After LESN is initialized, only W out It will change as the learning process progresses.
[0030] Step 2.3: Online learning process of DHP-LESN controller:
[0031] (1) Use the LESN model to identify the nonlinear function F(x(k-1),v(k-1)), identify the current state, and obtain the system state estimate x m (k), m is the abbreviation of model, and the same applies to m in the following text. The output expression is as follows:
[0032]
[0033] where σ m (k)=[i(k),z(k)],i m (k)=[x(k-1),v(k-1)],W m (k) is the weight of the model LESN, z m (k) is the reservoir state vector of the model LESN.
[0034] Therefore, the error e given by (2) m (k) becomes:
[0035] e m (k) = x m (k)-x(k) (11)
[0036] Where x(k) represents the output state at time k. The cost function E m (k) is defined as:
[0037]
[0038] The increment ΔW of the model LESN output weight m (k) is:
[0039]
[0040] Among them l m is the learning rate.
[0041] (2) Use the judgment LESN to derive the loss function. The expression of the judgment LESN is defined as:
[0042]
[0043] Where W c (k) is the LESN weight of the judge, and c is the abbreviation of critic. The same applies to c appearing in the following text.
[0044] Taking the derivative of both sides of equation (4), we can obtain:
[0045]
[0046] According to the chain deduction rule, we have:
[0047]
[0048] According to (8)-(9), we have:
[0049]
[0050] where α m is the leakage rate of the model LESN, f' m is the derivative of the neuron activation function in the reserve pool, is the part of the input matrix that processes the input x(k-1), θ m (k) is the internal state of the model LESN, and its calculation formula is:
[0051]
[0052] in and represents the input matrix and reservoir matrix of the LESN model, i m (k) and z m (k) represents the input state and internal state of the model LESN, z m (k-1) is the internal state of the model LESN at the previous moment.
[0053] Therefore, the error e of judging LESN is c (k) can be defined as:
[0054]
[0055] The objective function E for judging LESN c (k) is:
[0056]
[0057] The update method of the same model LESN is used to judge the output weight ΔW of LESN c (k) is:
[0058]
[0059] Among them l c is the learning rate.
[0060] (3) In the DHP-LESN controller, the LESN is executed to provide the controller increment. The expression of the output Δv(k) can be expressed as:
[0061]
[0062] Where W a (k) is the execution LESN weight, and a is the abbreviation of actor. The same applies to a appearing in the following text.
[0063] Expected output Δv of executing LESN * (k) is:
[0064]
[0065] Where μ is the step size.
[0066] Since the output of the LESN is judged to be Combining (8)-(9), we have
[0067]
[0068] in is the part of the input matrix that processes the input v(k). From equations (25) and (26), we can see that the error e of judging LESN is a (k) is:
[0069]
[0070] Objective function E for judging LESN weight update a (k) is rewritten as follows:
[0071]
[0072] Same as the other two networks, perform the output weight update ΔW of LESN a (k) is rewritten as follows:
[0073]
[0074] Among them l a is the learning rate.
[0075] Step 2.4: System stability analysis
[0076] Lemma 1: Assumptions 1-2 hold. Given [x(k-1), v(k-1)] as the input of the model LESN, combined with the online gradient descent algorithm, the Lyapunov function L1(k) is defined as follows:
[0077]
[0078] Its first-order difference function ΔL1(k) is:
[0079]
[0080] in, And any value of γ1∈(0,∞],σ m (k) represents the internal state vector of the model LESN.
[0081] Proof: The first-order difference expression of L1(k) is:
[0082]
[0083] Known W m The update formula of (k) can be obtained:
[0084]
[0085] therefore, can be written as:
[0086]
[0087] Combining (35) and (36), we can infer that Lemma 1 holds.
[0088] Lemma 2: Assumptions 1-2 hold. It is known that the input of the LESN is x(k), so for the Lyapunov function L2(k):
[0089] The first-order difference function ΔL2(k) is:
[0090]
[0091] in And γ2∈(0,∞] is any value.
[0092] Proof: The first-order difference of L2(k) is:
[0093]
[0094] According to the update formula for judging LESN, it can be deduced that:
[0095]
[0096] For ease of writing, and Abbreviated as r(k-1) and P respectively x Therefore, through (34)-(35), we can know that:
[0097]
[0098]
[0099] (41)The third term on the right side can be simplified to:
[0100]
[0101] Using the Cauchy-Schwarz inequality, we can calculate:
[0102]
[0103] From (41) and (43), we can see that Lemma 2 holds.
[0104] Lemma 3: Assumptions 1-2 hold. The input to execute LESN is x(k), so for the Lyapunov function L3(k):
[0105] The first-order difference function ΔL3(k) is:
[0106]
[0107] in And γ3>0 is a constant factor.
[0108] Proof: It is known that the first-order difference of L3(k) is:
[0109]
[0110] By W a The update formula of (k) is:
[0111]
[0112] in Therefore, it can be calculated that:
[0113]
[0114] (48)The first term on the right side of the equal sign can be simplified to:
[0115]
[0116] Combining (48) and (49), we can obtain Lemma 3.
[0117] Step 2.5: Control performance evaluation
[0118] The evaluation indicators mainly include the integrated square error (ISE), the integrated absolute error (IAE) and the maximum absolute error (Dev max ).
[0119]
[0120] Dev max =max|R(k)-x(k)| (52)
[0121] Where R(k) and x(k) are the set point and system output. The evaluation index mainly evaluates the control accuracy of the controller and the fluctuation of the controlled quantity. BRIEF DESCRIPTION OF THE DRAWINGS
[0122] Figure 1 DHP-LESN controller schematic diagram
[0123] Figure 2 Changes in water flow rate under sunny weather
[0124] Figure 3 Control results of dissolved oxygen concentration under sunny weather
[0125] Figure 4 Control results of nitrate nitrogen concentration under sunny weather
[0126] Figure 5 Oxygen transfer coefficient change diagram
[0127] Figure 6 Internal reflow coefficient variation diagram DETAILED DESCRIPTION
[0128] The concentrations of dissolved oxygen and nitrate nitrogen in the wastewater treatment benchmark simulation model BSM1 are tracked and controlled under sunny weather conditions. The fixed set value of dissolved oxygen concentration is 2 mg / L, and the fixed set value of nitrate nitrogen concentration is 1 mg / L.
[0129] The parameters are set as follows:
[0130] 1) Constants γ1 = 1, γ2 = 1, γ3 = 1; constant λ = 0.9, step size μ = 0.2; network parameter leakage rate α = 0.9, leakage rate α m =0.9.
[0131] 2) Execute LESN: learning rate l a =0.05, number of input layer nodes: K=2, number of output layer nodes: M=2, number of reserve pool nodes: N=40;
[0132] 3) Judging LESN: learning rate l m =0.01, number of input layer nodes: K=2, number of output layer nodes: M=1, number of reserve pool nodes: N=40;
[0133] 4) Model LESN: learning rate l c =0.01, number of input layer nodes: K=4, number of output layer nodes: M=2, number of reserve pool nodes: N=30.
[0134] To evaluate the control performance of the DHP-LESN, it is compared with a proportional-derivative-integral (PID) controller. Figure 2 The water inlet flow rate change diagram under sunny weather was drawn. Figure 3 and Figure 4The control effects of DHP-LESN and PID controller on dissolved oxygen and nitrate nitrogen concentrations on sunny days are shown respectively. As can be seen from the figure, under the precise control of DHP-LESN, the dissolved oxygen concentration fluctuates between 1.94mg / L and 2.05mg / L, showing excellent control accuracy. At the same time, the fluctuation of nitrate nitrogen concentration is also significantly suppressed, maintaining between 0.95mg / L and 1.05mg / L. Figure 3 and Figure 4 It can be seen that the present invention can obtain more accurate control results. Figure 5 and Figure 6 The operation variables K are shown respectively L a5 and Q a From the changing trend diagram, we can clearly observe that the control law output by DHP-LESN is adjusted accordingly with the environmental changes. It is more sensitive than the PID controller and maintains stable control performance at the same time.
[0135] In addition, the performance comparison results of the two controllers under sunny weather are given:
[0136] For S O,5 :The IAE of DHP-LESN controller is 8.6×10 -4 , ISE is 2.6×10 -5 ,Dev max 8.8×10 -2 ; The IAE of the PID controller is 1.3×10 -3 , ISE is 4.9×10 -5 ,Dev max 9.7×10 -2 .
[0137] For S NO,2 :The IAE of DHP-LESN controller is 1.8×10 -3 , ISE is 6.2×10 -5 ,Dev max 7.9×10 -2 ; The IAE of the PID controller is 1.7×10 -3 , ISE is 7.4×10 -5 ,Dev max 9.8×10 -2 .
[0138] The experimental results show that: through verification, the model has the best accuracy under the experimental environment with the selected parameter values. maxThe index values are better than PID control. In summary, the designed DHP-LESN controller has higher control accuracy, making the sewage treatment system run more smoothly.
Claims
1. A multivariable sewage treatment process control method based on quadratic heuristic planning of echo state network, characterized in that the steps are as follows: 1.1 Analysis of optimal control problems in sewage treatment process: The optimal control problem of the sewage treatment process is expressed as: Where F(·,·) represents the unknown functional relationship, S O,5 (k) and S NO,2 (k) represents the dissolved oxygen concentration in the fifth zone and the nitrate nitrogen concentration in the second zone, respectively. L a5(k) represents the oxygen transfer coefficient, Q a (k) represents the internal recirculation rate, k is an arbitrary natural number, and N is a set of natural numbers; therefore, the changes in dissolved oxygen concentration and nitrate nitrogen concentration x(k+1) are expressed as follows: x(k+1)=F(x(k),v(k)),k∈N (2) Where x(k) represents the matrix [S O,5 (k), S NO,2 (k)], v(k) represents the matrix [K L a5(k), Q a (k)] changes; Assumption 1: The state x(k) is observed from the kth step of the control system; Assumption 2: Assuming the control gain matrix is positive definite, then where (x(k),v(k))∈R + ×R + For any value in R + is a positive real number, det[·] is the operation to find the determinant of the matrix; The loss function J(k) in the policy iteration algorithm is defined as follows: Where U(i) is the utility function, and λ∈(0,1] is any value; the utility function is defined as: where x * (k) is the optimal value of x(k); according to Bellman's optimality principle and greedy algorithm, the optimal loss function J * (k) is defined as: J * (k) = min (U (k) + λ J * (k+1)) (6) Among them J *(k+1) is the optimal loss function at the next moment, then the optimal control vector v * (k) satisfies the following formula: 1.2 Determine the leaky integral echo state network: Leaky Integral Echo State Network (LESN) has a reservoir neuron with leaky integral properties and has been successfully used in feedback control systems. LESN mainly consists of three layers: input layer, reservoir and output layer, and the number of network nodes is set to K, N and M respectively. The input vector, reservoir state vector and output vector of LESN are i(k), z(k) and y(k) respectively. The input weight matrix is W in ∈W N×K For any value in, the state matrix of the reservoir is W∈W N×N The output weight matrix is W out ∈W (N +K)×M Any value in; then the update formulas of the LESN's reserve pool state vector z(k) and output vector y(k) are: z(k)=(1-α)z(k-1)+αf(W in i(k)+W(k-1)) (8) The network parameter leakage rate α∈(0,1] is an arbitrary value; W(k-1) is the reservoir matrix at the previous moment; f(·) represents the activation function of the reservoir, which uses the hyperbolic tangent function to enhance the network's ability to handle nonlinear environments; after initialization, LESN only has W out will change as the learning process progresses; 1. Online learning process of 3DHP-LESN controller: 1) Use the LESN model to identify the nonlinear function F(x(k-1),v(k-1)), identify the current state, and obtain the system state estimate x m (k), m is the abbreviation of model, and the same applies to m appearing in the following text; the output expression is as follows: where σ m (k)=[i m (k),z m (k)],i m (k)=[x(k-1),v(k-1)],W m (k) is the weight of the model LESN, z m (k) is the reservoir state vector of the model LESN; Therefore, the error e given by (2)m (k) becomes: e m (k) = x m (k)-x(k) (11) Where x(k) represents the output state at time k; the cost function E m (k) is defined as: The increment ΔW of the model LESN output weight m (k) is: Among them l m is the learning rate; 2) Use the judgment LESN to derive the loss function; the expression of the judgment LESN is defined as: Where W c (k) is the LESN weight of the judge, c is the abbreviation of critic, and the same applies to the c that appears in the following text; Taking the derivative of both sides of equation (4), we get: According to the chain deduction rule, we have: According to (8)-(9), we have: where α m is the leakage rate of the model LESN, f' m is the derivative of the neuron activation function in the reserve pool, is the part of the input matrix that processes the input x(k-1), model θ m (k) is the internal state of LESN, and its calculation formula is: in and represents the input matrix and reservoir matrix of the LESN model, i m (k) and z m (k) represents the input state and internal state of the model LESN, z m (k-1) is the internal state of the model LESN at the previous moment; Therefore, the error e of judging LESN is c (k) is defined as: The objective function E for judging LESN c (k) is: The update method of the same model LESN is used to judge the output weight ΔW of LESN c (k) is: Among them l c is the learning rate; 3) In the DHP-LESN controller, LESN is executed to provide the controller increment; the expression of the output Δv(k) is expressed as: Where W a (k) is the execution LESN weight, a is the abbreviation of actor, and the a that appears in the following text is the same; Expected output Δv of executing LESN * (k) is: Where μ is the step size; Since the output of the LESN is judged to be Combining (8)-(9), we have in is the part of the input matrix that processes the input v(k); from equations (25) and (26), we can see that the error e of judging LESN is a (k) is: Objective function E for judging LESN weight update a (k) is rewritten as follows: Same as the other two networks, perform the output weight update ΔW of LESN a (k) is rewritten as follows: Among them l a is the learning rate; 1.4 System Stability Analysis Lemma 1: Assumptions 1-2 hold; it is known that [x(k-1), v(k-1)] is the input of the model LESN. Combined with the online gradient descent algorithm, the Lyapunov function L1(k) is defined as follows: Its first-order difference function ΔL1(k) is: in, And any value of γ1∈(0,∞],σ m (k) represents the internal state vector of the model LESN; Lemma 2: Assumptions 1-2 hold; the input to the LESN is x(k), so for the Lyapunov function L2(k): The first-order difference function ΔL2(k) is: in And any value of γ2∈(0,∞]; Lemma 3: Assumptions 1-2 hold; the input to execute LESN is x(k), so for the Lyapunov function L3(k): The first-order difference function ΔL3(k) is: in And any value of γ3∈(0,∞]; 1.5 Parameter settings are as follows: 1) Constants γ1 = 1, γ2 = 1, γ3 = 1; constant λ = 0.9, step size μ = 0.2; network parameter leakage rate α = 0.9, leakage rate α m =0.9; 2) Execute LESN: learning rate l a =0.05, number of input layer nodes: K=2, number of output layer nodes: M=2, number of reserve pool nodes: N=40; 3) Judging LESN: learning rate l m =0.01, number of input layer nodes: K=2, number of output layer nodes: M=1, number of reserve pool nodes: N=40; 4) Model LESN: learning rate l c =0.01, number of input layer nodes: K=4, number of output layer nodes: M=2, number of reserve pool nodes: N=30.
Citation Information
Patent Citations
Iterative-ADP-algorithm-based optimal control method for wastewater treatment
CN106873379A
Urban sewage treatment process optimization control method based on dynamic multi-objective particle swarm optimization algorithm
CN110161995A
Sewage treatment concentration control method utilizing iterative quadratic heuristic programming
CN112000004A
Sewage treatment process control method based on generalized value iteration
CN113741182A
Online ADHDP method for optimal tracking control of sewage treatment
CN114063452A
