Knowledge migration-fused sewage treatment process adaptive evaluation control method

Abstract

The invention provides a knowledge migration-fused sewage treatment process adaptive evaluation control method, which is used for realizing accurate tracking of a set value by dissolved oxygen concentration in a sewage treatment process. When optimization control of a sewage treatment system with an unknown model is achieved, a direct control strategy in a traditional adaptive control (ACC) algorithm is often sensitive to system disturbance, and the control performance can be suddenly reduced. According to the method, knowledge migration and incremental control are considered at the same time, and the anti-interference performance is effectively improved by optimizing a control strategy of a conventional ACC algorithm into an incremental type. And secondly, knowledge migration among different control methods is implemented by means of historical operation data of the system, so that the trial and error cost of early-stage training is successfully reduced. And thirdly, a novel utility function with a soft strategy constraint mechanism is designed, and the stability in the online optimization process is enhanced. Finally, the effectiveness of the method is verified through experiments.

Description

Technical Field

[0001] The invention relates to the field of sewage treatment. Background Art

[0002] With the acceleration of urbanization and the continuous increase in population density, the amount of urban sewage discharge has increased rapidly, posing a severe challenge to the urban water environment. As a key link in protecting the environment and promoting the recycling of water resources, the importance of sewage treatment is self-evident. Effective sewage treatment can not only protect the ecological balance, but also provide cities with a reliable way to reuse water resources, which has far-reaching significance for promoting sustainable development and building an ecological civilization society. Therefore, optimizing the sewage treatment process and improving treatment efficiency are important issues that need to be urgently addressed in the current environmental protection field.

[0003] Accurately regulating the dissolved oxygen concentration is crucial to ensure efficient and stable operation of the sewage treatment process. Some traditional control methods, such as proportional-integral-derivative (PID) control, switch control and feedforward control, can achieve the regulation of dissolved oxygen concentration to a certain extent. However, in the face of complex and changeable working conditions in the sewage treatment process, such as influent flow fluctuations, water temperature changes, sludge load differences, etc., these methods are difficult to dynamically adapt to the uncertainty changes in actual operation, resulting in unstable control effects and high energy consumption. In addition, traditional control methods do not have optimization capabilities and are difficult to meet the increasingly stringent effluent water quality standards and energy conservation and emission reduction requirements.

[0004] In view of this, exploring more intelligent and efficient control strategies has become a hot topic in the field of sewage treatment. Relevant researchers have begun to design controllers for sewage treatment processes based on intelligent control methods such as neural network control and model predictive control. However, neural network control only adjusts weights based on the tracking error at the current moment, resulting in limited optimization capabilities; model predictive control has good optimization performance due to its consideration of long-term tracking errors in the future, but it is highly dependent on accurate system models. The adaptive critic control (ACC) algorithm is widely used in control optimization problems of unknown nonlinear systems due to its powerful optimization and adaptive performance. The ACC algorithm combines the ideas of dynamic programming and reinforcement learning. It continuously optimizes the control strategy through iterative learning. It can automatically adjust parameters under conditions where the system model is unknown to adapt to changes in the system state, thereby achieving better control effects. The application of the ACC algorithm in the sewage treatment process is expected to improve sewage treatment efficiency and reduce operating costs, which is of great significance for promoting the intelligent upgrading of sewage treatment technology.

[0005] Although the ACC algorithm shows great potential in theory, it still faces a series of challenges when applied to the sewage treatment process in practice. First, the control strategy of the conventional ACC algorithm is only related to the current system state, and large fluctuations will occur when it is disturbed. Second, the training data of the online ACC algorithm needs to be obtained through gradual interaction with the system, and it often faces the dilemma of insufficient training data, which in turn leads to slow algorithm convergence speed and poor learning stability. To solve this problem, scholars usually introduce an experience replay mechanism to stabilize the learning process by storing historical experience data. However, this approach significantly increases the burden of data storage and calculation, which is not conducive to the wide application of the ACC algorithm in practical engineering. In summary, although the ACC algorithm provides a new idea for the dissolved oxygen concentration control in the urban sewage treatment process, there are still many challenges in its direct application. How to maintain the optimization and adaptive advantages of the ACC algorithm while overcoming its problems of being sensitive to disturbances, poor learning stability, and heavy storage and calculation burden has become the key to current research. Summary of the Invention

[0006] Compared with the classical ACC algorithm, the present invention improves its control strategy from direct type to incremental type, effectively improving the anti-interference performance of the algorithm. At the same time, the initial control strategy of the ACC algorithm is constructed by using the knowledge transfer technology and the expert experience in the traditional incremental PID control algorithm, overcoming the problem of slow convergence of the online ACC algorithm. To avoid the data storage pressure caused by the experience replay mechanism, the present invention innovatively designs a new utility function with a soft strategy constraint mechanism, which can improve the stability of the online optimization process of the algorithm while avoiding the increase of data storage burden. Finally, the simulation results show that the present invention has good control performance for the dissolved oxygen concentration in the sewage treatment process.

[0007] The structural diagram of the sewage treatment system applying the present invention is as Figure 1 shown, but not limited thereto. Among them, 1.1 represents the biochemical reaction tank, 1.2 represents the aerobic zone composed of unit one 1.4 and unit two 1.5 of the biochemical reaction tank, 1.3 represents the anaerobic zone composed of unit three 1.6, unit four 1.7 and unit five 1.8 of the biochemical reaction tank, 1.9 represents the secondary sedimentation tank, 2.1 represents the gas flowmeter, 2.2 represents the dissolved oxygen concentration sensor, 3.1 represents the programmable logic controller, 3.2 represents the frequency converter, and 4.1 represents the blower.

[0008] In the sewage treatment system (as Figure 1As shown in the figure, it mainly covers two key parts: the biochemical reaction tank 1.1 and the secondary sedimentation tank 1.9. The first two units of the biochemical reaction tank are the anaerobic zone 1.2, and the last three units are the aerobic zone 1.3. Municipal sewage flows through the anaerobic zone 1.2 and the aerobic zone 1.3 in sequence. During this period, through nitrification and denitrification reactions, the purification of organic pollutants, nitrogen elements, phosphorus elements, etc. in the sewage is realized. Subsequently, the sewage flows into the secondary sedimentation tank 1.9 for sedimentation operation to further separate the supernatant and sludge. In the whole biochemical reaction process, the dissolved oxygen concentration in unit five is one of the key factors affecting microbial activity, organic matter degradation efficiency and operation energy consumption. If the dissolved oxygen concentration is too low, the respiration of microorganisms will be restricted, resulting in a significant reduction in sewage treatment efficiency; on the contrary, too high a dissolved oxygen concentration will cause energy waste and may even promote the growth of harmful microorganisms. Therefore, accurately regulating the dissolved oxygen concentration is crucial for ensuring the efficient and stable operation of the sewage treatment process. In the sewage treatment system applying the present invention, the gas flow meter 2.1 and the dissolved oxygen concentration sensor 2.2 are used to collect data and transmit it to the programmable logic controller 3.1. The programmable logic controller 3.1 calculates and outputs a control signal to the frequency converter 3.2 through the present invention, and then the frequency converter 3.2 adjusts the motor speed of the blower 4.1, and finally changes the oxygen transfer coefficient to achieve the control of the dissolved oxygen concentration.

[0009] Figure 2 is the overall structure diagram of the present invention. The overall implementation method is as follows: execute step 1 to establish the tracking control optimization problem of the dissolved oxygen concentration in the sewage treatment system; execute step 2 to complete the initialization of the control strategy by using the knowledge transfer technology; repeat the execution of step 3 to achieve the precise control of the dissolved oxygen concentration in the sewage treatment system and the online optimization of the control strategy. Next, the present invention gives the specific implementation process of each step.

[0010] Step 1: Establish the tracking control optimization problem of the dissolved oxygen concentration in the sewage treatment process. The sewage treatment system can be expressed as a class of nonlinear systems as follows:

[0011] x k+1 =H(x k ,u k ),k=0,1,2,... (1)

[0012] Among them, the system state represents the dissolved oxygen concentration of unit five at time k, and the control input represents the oxygen transfer coefficient at time k. H(·,·) is an unknown system function, represents the set of all positive real numbers. The set value d of the dissolved oxygen concentration k is expressed as

[0013] d k =δ(k) (2)

[0014] Among them, δ(·) is a set value function. According to engineering experience, the set value of the dissolved oxygen concentration is usually selected as 2 mg / L. The tracking error e between the dissolved oxygen concentration and its set value k is defined as

[0015] e k = x k - d k (3)

[0016] In the conventional ACC algorithm, the calculated control strategy is a direct control strategy π(e k ), such that the control input u k = π(e k ). To enhance the anti-interference ability of the algorithm, the present invention designs an incremental control strategy η(e k ), such that the control input satisfies the following equation:

[0017]

[0018] In addition, to enhance the stability of the learning process, a utility function is proposed as follows:

[0019]

[0020] Among them, α 1 , α 2 and α 3 are constant weights greater than 0. The present invention takes α 1 = 0.1, α 2 = 0.01, α 3 = 0.005. For the utility function U(e k , Δu k ), is mainly used to reflect the cost caused by the tracking error and directly ensure the control accuracy; then focuses on reflecting the cost brought by the change of the control input, used to avoid excessive fluctuations of the control input and ensure the smoothness of the control process. In addition, S(e k , Δu k ) is a soft strategy constraint function innovatively designed by the present invention, which plays a key role in the adaptive soft strategy constraint in the process of optimizing the control strategy, and is expressed as

[0021]

[0022] Among them, θ(e k ) = -121e k + 102e k-1 - e k-2It is an incremental PID control strategy that can achieve stable control of the dissolved oxygen concentration. When the optimization effect is significant, The value is less than or equal to 1×10 -3 , S(e k ,Δu k ) this term will automatically relax the restrictions on the strategy optimization process, giving the control strategy greater adjustment freedom to fully exert its optimization performance; on the contrary, if the optimization effect is poor and the control accuracy is low, The value is greater than or equal to 1×10 -2 , this term will strengthen the constraint on the strategy optimization process, guiding the control strategy η(e k ) to approach the incremental PID control strategy θ(e k ), promoting the present invention to take into account stability while optimizing the control accuracy, thereby ensuring the reliable operation of the entire control system. Then, the value function V(e k ,Δu k ) is defined as

[0023]

[0024] where γ∈(0,1) is the discount factor, which is used to ensure the boundedness of the value function. Here, γ = 0.95 is selected. The optimal value function V * (e k ,Δu k ) and the optimal incremental control strategy η * (e k ) are respectively defined as

[0025]

[0026] and

[0027]

[0028] Since equation (8) belongs to the Hamilton-Jacobi-Bellman equation, it is difficult to directly obtain its analytical solution. Therefore, the present invention is based on an execution-evaluation online optimization mechanism and uses a neural network function approximation tool to continuously approximate the optimal value function V * (e k ,Δu k ) and the optimal incremental control strategy η * (e k ).

[0029] Step 2: Use knowledge transfer technology to complete the initialization of the control strategy. The present invention consists of an execution network and a evaluation network. The execution network is used to approximate the optimal incremental control strategy η * (e k ), and the evaluation network is used to approximate the optimal value function V* (e k , Δu k ). It should be noted that both the execution network and the evaluation network are backpropagation neural networks composed of an input layer, a hidden layer, and an output layer. Therefore, the initialization of the control strategy can be completed by pre-training the execution network.

[0030] In order to utilize the knowledge transfer technology to complete the initialization of the control strategy, the present invention first establishes a data set D = {(e′ j , Δu′ j )|j = 1, 2,..., n} by using the historical operation data of the sewage treatment system under the action of the incremental PID control algorithm, where e j ′ and Δu′ j respectively represent the j-th tracking error sample and the incremental control input sample under the action of the PID control strategy, and n is the total number of samples in the data set D. In implementation, n = 2687 is selected, but it is not limited thereto. Then, the pre-training of the execution network is completed based on the data set D. The output of the execution network during the pre-training process can be expressed as

[0031]

[0032] where, and are the weight vectors of the execution network. h a = 10 represents the number of neurons in the hidden layer of the execution network. In addition, the activation function φ(·) is set to the hyperbolic tangent function. The approximation error of the execution network during the pre-training process can be expressed as

[0033]

[0034] The performance index function of the execution network during the pre-training process can be expressed as

[0035]

[0036] where, e′ = [e 1 ′, e 2 ′,..., e n ′] is the input of the performance index function. According to the principle of gradient descent, the update rule of the weights of the execution network during the pre-training process can be expressed as

[0037]

[0038] where, := represents the assignment operation, and β a = 0.005 is the learning rate of the execution network during the pre-training process. Repeat (10)-(13) until Ψ(e′) < 10 -5At this time, the present invention has successfully completed the knowledge transfer of the PID control algorithm to the present invention based on the data set D, improving the control performance of the dissolved oxygen concentration in the initial stage.

[0039] Step 3: Online optimize the control strategy based on the execution-evaluation framework. To ensure the adaptive and optimization performance of the algorithm, the present invention continuously updates the value function and improves the strategy according to the real-time tracking error e k During the online training process, the output of the evaluation network can be expressed as

[0040]

[0041] where and

[0042] are the weight vectors of the evaluation network. h c = 12 represents the number of neurons in the hidden layer of the evaluation network. During the online training process, the approximation error of the evaluation network can be expressed as

[0043]

[0044] The performance index of the evaluation network can be expressed as

[0045]

[0046] According to the principle of gradient descent, the update rule of the weight vector w c2 can be expressed as

[0047]

[0048] where, l c = 0.01 is the learning rate during the online training process of the evaluation network. It should be noted that in order to reduce the computational burden of online training, the present invention only updates the outer-layer weight w c2 , and the inner-layer weight w c1 remains unchanged after random initialization. Repeat (14)-(17) until E c (e k ) < 10 -5 . At this time, the update of the evaluation network at time k is completed. Next, according to the evaluation result of the control strategy by the evaluation network, the strategy is improved. During the online training process, the output of the execution network can be expressed as

[0049]

[0050] The approximation error of the execution network during the online training process is defined as

[0051]

[0052] where, Ud is the ideal training target, and usually set U d = 0. Define the performance index of the execution network during online training as

[0053]

[0054] According to the principle of gradient descent, the update rule of the weight vector w a2 can be expressed as

[0055]

[0056] Repeat the execution of (18)-(21) until At this time, the optimization of the execution network at time k is completed. Finally, use the trained execution network to calculate the control input according to the current tracking error

[0057]

[0058] Apply the control input to the sewage treatment system to achieve the tracking of the dissolved oxygen concentration to the set value. Brief Description of the Drawings

[0059] Figure 1 Structural diagram of the sewage treatment system implemented by the present invention

[0060] Figure 2 Framework diagram of the present invention

[0061] Figure 3 Test results of the approximation error of the execution network after pre-training

[0062] Figure 4 Change process of the dissolved oxygen concentration under the action of different control algorithms

[0063] Figure 5 Change process of the oxygen transfer coefficient of the present invention Detailed Implementation Manner

[0064] The Benchmark simulation model no.1 (BSM1) was developed jointly by the EU's Science and Technology Cooperation Organization and the International Water Association. Based on the activated sludge model and the double-exponential sedimentation velocity function, it simulates the biochemical reactions occurring in the biochemical reaction tank and the clarification process in the secondary clarifier respectively. Given that BSM1 has a sufficient degree of reduction of the actual sewage treatment process, it has become a widely adopted benchmark test platform for researchers in the sewage treatment field, capable of fairly reflecting the performance of various control algorithms. Therefore, the present invention uses BSM1 to simulate a 14-day sewage treatment process to evaluate the control performance of the present invention on the dissolved oxygen concentration. Note that the following results are all obtained based on the influent data under sunny weather. According to the experience of algorithm debugging, the parameter values of the algorithm proposed in the present invention are selected as follows:

[0065] (1) The set value d k = 2, the constant weight α 1 = 0.1, α 2 = 0.01, α 3 = 0.005, the discount factor γ = 0.95;

[0066] (2) The total number of dataset samples n = 2687, the pre-training learning rate β a = 0.005, the pre-training stop criterion Ψ(e′) < 10 -5 , the number of neurons in the hidden layer h a = 10, h c = 12;

[0067] (3) The online training learning rate l c = 0.01, l a = 0.01, the stop criterion for online training E c (e k ) < 10 -5 ,

[0068]

[0069] After completing the pre-training stage of the execution network, 600 groups of data are used to test its approximation accuracy for the PID control strategy. The test results are as Figure 3 shown, and its approximation error is controlled within 0.025, showing good approximation accuracy. To visually demonstrate the superior performance of the present invention, it is comprehensively compared with the traditional PID control algorithm and the conventional ACC algorithm in terms of the control effect of the dissolved oxygen concentration. The experimental results are as Figure 4 shown. Compared with the other two control algorithms, the present invention has higher control accuracy and smaller error fluctuations for the dissolved oxygen concentration. In addition, the change process of the oxygen transfer coefficient, which is used as the control input in the present invention, is as Figure 5 shown.

[0070] In order to more accurately evaluate the control performance of the algorithm, the present invention further adopts two industry-recognized evaluation indicators, namely, the integral of squared error (ISE) and the maximal deviation from the set point (Dev max ) to quantitatively analyze the control performance of the algorithm. The definitions of the two evaluation indicators are as follows:

[0071]

[0072] where k 1 and k 2 respectively represent the start time and the end time of the tracking error between the statistical dissolved oxygen concentration and the set value. According to industry experience, usually the data of the latter 7 days in 14 days are statistically analyzed, that is, k 1 = 1345 represents the start time of the 8th day, and k 2 = 2687 is the end time of the 14th day. ISE focuses on reflecting the average control accuracy of the algorithm for the dissolved oxygen concentration on a long time scale. The smaller its value, the better the average control effect; Dev max focuses on reflecting the fluctuation degree of the algorithm's control of the dissolved oxygen concentration. The smaller the value, the higher the control stability. Under the action of the PID control algorithm, ISE = 5.64×10 -4 , Dev max = 0.1164; under the action of the conventional ACC control algorithm, ISE = 1.90×10 -5 , Dev max = 0.0223; under the action of the present invention, ISE = 3.19×10 -6 , Dev max = 0.0084. It can be seen from the comparison of these data that the present invention has obvious improvements in both the average control accuracy and the stable control performance, which strongly verifies its application value and innovation advantages in the field of dissolved oxygen concentration control in the sewage treatment process, and provides new ideas for the innovation of sewage treatment intelligent control technology.

[0073] The control strategy of the present invention is improved to be incremental, enhancing the anti-interference ability and enabling it to better cope with complex working conditions. The initial strategy is constructed by using knowledge transfer technology and traditional incremental PID control experience, effectively reducing the training cost. A new type of utility function is designed, which effectively guarantees the stability of online optimization without causing a data storage burden. After testing on BSM1, compared with traditional PID control algorithms and general ACC algorithms, the present invention has a relatively obvious improvement in control accuracy, can more accurately make the dissolved oxygen concentration track the set value, and at the same time effectively reduces the error fluctuation, ensuring the efficient and stable operation of the sewage treatment process.

Claims

1. An adaptive judgment control method for sewage treatment process integrating knowledge transfer, characterized in that The following steps are involved: Step 1: Establish the tracking and control optimization problem of dissolved oxygen concentration in the sewage treatment process; The sewage treatment system is represented as a nonlinear system as follows: x k+1 =H(x k ,u k ),k=0,1,2,... (1) Among them, the system status represents the dissolved oxygen concentration of unit 5 at time k, the control input represents the oxygen transfer coefficient at time k, H(·,·) is an unknown system function, represents the set of all positive real numbers; the set value of dissolved oxygen concentration d k Expressed as d k =δ(k) (2) Where δ(·) is the set value function; the set value of dissolved oxygen concentration is selected as 2 mg / L; the tracking error between dissolved oxygen concentration and its set value is e k Defined as e k =x k -d k (3) To design an incremental control strategy η(e k ), so that the control input satisfies the following formula: A utility function is proposed as follows: Among them, α1, α2 and α3 are constant weights greater than 0; among them: Among them, θ(e k )=-121e k +102e k-1 -e k-2 ; Then, the value function V(e k ,Δu k ) is defined as Among them, γ∈(0,1) is the discount factor; Optimal value function V * (e k ,Δu k ) and the optimal incremental control strategy η * (e k ) are defined as and Based on the execution-judgment online optimization mechanism and using the neural network function approximation tool, it continuously approaches the optimal value function V * (e k ,Δu k ) and the optimal incremental control strategy η * (e k ); Step 2: Use knowledge transfer technology to complete the initialization of the control strategy; The execution network is used to approximate the optimal incremental control strategy η * (e k ), the evaluation network is used to approximate the optimal value function V * (e k ,Δu k ); Both the execution network and the evaluation network are back-propagation neural networks consisting of an input layer, a hidden layer, and an output layer; The data set D = {(e′ j ,Δu′ j )|j=1,2,...,n}, where e′ j and Δu′ j They represent the jth tracking error sample and incremental control input sample under the PID control strategy, respectively, and n is the total number of samples in the data set D. Then, the pre-training of the execution network is completed based on the data set D. The output of the execution network during the pre-training process is expressed as in, and is the weight vector of the execution network; h a =10 represents the number of neurons in the hidden layer of the execution network; in addition, the activation function φ(·) is set to the hyperbolic tangent function; the approximation error of the execution network during pre-training is expressed as The performance index function of the network executed during pre-training is expressed as Where, e′=[e1′,e2′,...,e n ′] is the input of the performance indicator function; the update rule of the execution network weights in the pre-training process is expressed as Among them, := represents the assignment operation, β a = 0.005 is the learning rate of the execution network during the pre-training process; repeat (10)-(13) until Ψ(e′) < 10 -5 ; Step 3: Online optimization of control strategy based on the execution-criteria framework; The output of the judgment network during online training is expressed as in, and is the weight vector of the judgment network; h c =12 represents the number of hidden layer neurons in the evaluation network; the approximation error of the evaluation network during online training can be expressed as The performance index of the network can be expressed as According to the gradient descent principle, the weight vector w c2 The update rule can be expressed as Among them, l c = 0.01 is the learning rate during the online training of the network; only the outer weights w are updated c2 , the inner weight w c1 After random initialization, remain unchanged; repeat (14)-(17) until E c (e k )<10 -5 ; At this point, the update of the evaluation network at time k is completed; Next, the strategy is improved based on the evaluation results of the control strategy by the judgment network. The output of the execution network during the online training process is expressed as The approximation error of the execution network during online training is defined as Among them, U d is the ideal training target, setting U d =0; define the performance index of the execution network during online training as Weight vector w a2 The update rule is expressed as Repeat (18)-(21) until At this point, the optimization of the execution network at time k is completed; finally, the trained execution network is used to calculate the control input according to the current tracking error to achieve the tracking of the dissolved oxygen concentration to the set value;

Images