Sewage biological treatment model prediction control method

Abstract

The invention provides a sewage biological treatment model prediction control method. According to the method, firstly, parameters are sorted through global sensitivity analysis, and high-sensitivity parameters are emphatically processed; dynamically evaluating the local sensitivity of the parameters based on a multi-scale time window; the DDQN network based on sensitivity guidance is fused with data-driven learning and physical mechanism knowledge; and the model prediction control system realizes multi-objective optimization of stable and up-to-standard effluent quality and energy-saving operation of the system based on the calibrated ASM3 model. According to the method, a complete closed-loop system from sensitivity analysis to parameter correction to prediction control is constructed, and compared with a traditional off-line parameter correction and experience control method, the model prediction precision, the control response speed, the operation stability, the energy efficiency level and the like are remarkably improved; and a complete technical support is provided for intelligent operation of an activated sludge process.

Description

Technical Field

[0001] This invention relates to the field of intelligent control technology for wastewater treatment processes, and specifically to a predictive control method for a wastewater biological treatment model. Background Technology

[0002] Model predictive control (MPC) is an advanced control strategy based on process models for prediction and optimization. It determines the optimal control sequence by solving a finite-time optimization problem within each control cycle. MPC technology has significant advantages, including handling multivariate constraints, predicting future system behavior, and achieving rolling optimization control, making it particularly suitable for complex nonlinear industrial process control. In wastewater treatment, MPC can simultaneously consider effluent quality standards, equipment operating constraints, and economic indicators. By predicting changes in influent load and process response over a future period, it optimizes the control of key variables such as dissolved oxygen, internal recirculation ratio, and excess sludge discharge, thereby reducing treatment costs while meeting effluent standards. In recent years, with the improvement of computing power and optimization algorithms, the application of MPC technology in wastewater treatment process control has become increasingly widespread, becoming one of the key technologies for achieving intelligent operation of wastewater treatment plants.

[0003] The application of MPC technology in wastewater treatment has made significant progress, demonstrating superior control performance in numerous practical engineering cases. Actual operational data from the Lancaster Wastewater Treatment Plant in the UK shows that MPC achieved a 25% reduction in power consumption. The Modena Municipal Wastewater Treatment Plant in Italy, using a neurofuzzy MPC system, achieved significant energy savings while maintaining effluent quality superior to traditional control methods. After implementing Economic MPC (EMPC), operating costs decreased by 15-20%, and under heavy rainfall conditions, operating costs could be reduced by 20.02%, while effluent quality improved by 12.38%. However, the full realization of MPC control performance is highly dependent on the accuracy of the process model used. Studies on the impact of the process model on MPC show that a 20% parameter uncertainty typically leads to a 10-15% decrease in control performance. Therefore, achieving accurate, efficient, and adaptive correction of activated sludge model (ASM) parameters has become a key technical bottleneck in improving the effectiveness of MPC in wastewater treatment applications.

[0004] Existing ASM parameter correction methods mainly include intelligent optimization algorithms such as genetic algorithms and particle swarm optimization. Genetic algorithms, as the first intelligent optimization technique successfully applied to ASM parameter correction, effectively solve the local optima problem of traditional methods through evolutionary computation mechanisms. They can handle complex optimization problems involving nonlinearity and multimodality, and possess good global search capabilities. However, in practical applications, genetic algorithms suffer from drawbacks such as high computational workload and relatively slow convergence speed. Furthermore, parameter settings (such as crossover rate and mutation rate) significantly affect the results, requiring empirical adjustment, and their search efficiency is low in the absence of prior knowledge of parameter importance. Particle swarm optimization (PSO) algorithms exhibit unique technical characteristics in the treatment of activated sludge wastewater with alternating phases. This algorithm only requires an objective function, does not rely on gradient information, has fewer hyperparameters, is relatively simple to implement, and has a fast convergence speed. However, PSO algorithms are prone to getting trapped in local optima in high-dimensional problems, lack theoretical convergence guarantees, and also cannot handle the dynamic time-varying characteristics of parameters, limiting their application effectiveness in complex dynamic systems.

[0005] The development of deep reinforcement learning technology has opened up new technical paths for dynamic parameter correction. Chinese patent CN117111471A proposes an intelligent control method for wastewater treatment based on the DDPG algorithm combined with the ASM3 model. This method utilizes a combination of mechanistic models and intelligent algorithms to optimize and guide the actual production process, achieving intelligent control of oxidation ditches, effectively improving wastewater treatment efficiency and reducing operating energy consumption. The DQN real-time control system based on LSTM-ATT demonstrates technical capabilities in real-time control, achieving significant energy savings in the A^2 O (Anaerobic / Anoxic / Oxic) process with a single time step of only 7.26 seconds, meeting the time requirements of real-time control. However, current deep reinforcement learning applications mainly focus on control parameter optimization, and have not yet fully addressed the dynamic correction of model parameters. Furthermore, the exploration process may affect system stability, and the design of the reward function requires extensive domain expert knowledge. Multi-agent reinforcement learning technology shows technical potential in handling the interactive relationships between parameters in complex systems. Studies have shown that multi-agent systems using the G2ANet architecture can reduce COD modeling errors by 87% and TN modeling errors by 58%, providing a new solution for multi-parameter collaborative optimization. However, multi-agent systems have extremely high coordination complexity, difficulty in guaranteeing training stability, huge computational resource requirements, and lack of theoretical convergence guarantees. These factors limit their reliable application in practical engineering. Summary of the Invention

[0006] Purpose of the invention: The purpose of this invention is to provide a predictive control method for wastewater biological treatment models based on global sensitivity analysis and dynamic calibration of DDQN model parameters to improve the accuracy of ASM3 models.

[0007] Technical solution: A predictive control method for wastewater biological treatment models, comprising the following steps:

[0008] S1. Perform static global sensitivity analysis on the parameters of the ASM3 model to obtain the overall sensitivity index of the parameters;

[0009] S2. Calculate the local sensitivity index of the parameter at different time scales, construct the fused time-varying sensitivity index using the total sensitivity index and local sensitivity index of the parameter, and obtain the time-varying sensitivity state feature and real-time sensitivity weight of the parameter after encoding the fused time-varying sensitivity index.

[0010] S3. Based on the total sensitivity index of parameters, time-varying sensitivity state features, and real-time sensitivity weights, construct the model state space, model action space, and model reward function;

[0011] S4. Construct a dual-network training framework based on the DDQN model using the model state space, model action space, and model reward function. After training, an integrated sensitivity DDQN model is obtained.

[0012] S5. Collect the state information corresponding to the model state space from the sewage biological treatment system in real time, input it into the DDQN model with integrated sensitivity, output the adjustment range of each parameter, update the parameters of the ASM3 model using the parameter adjustment range, and obtain the parameter-corrected ASM3 model.

[0013] S6. Construct a model predictive control system based on the ASM3 model with parameter correction, and use the model predictive control system to solve and issue control commands to the wastewater biological treatment system.

[0014] Specifically, step S1 includes:

[0015] S11. Construct the parameter space of the ASM3 model, including the maximum growth rate of heterotrophic bacteria, the maximum growth rate of autotrophic bacteria, the yield coefficient of heterotrophic bacteria, the yield coefficient of autotrophic bacteria, the substrate half-saturation constant, the oxygen half-saturation constant of heterotrophic bacteria, the oxygen half-saturation constant of autotrophic bacteria, the nitrate half-saturation constant, the decay coefficient of heterotrophic bacteria, the decay coefficient of autotrophic bacteria, the anaerobic reducing factor, the anaerobic reducing factor, and the hydrolysis rate constant.

[0016] S12. Generate Sobol sequences based on the quasi-Monte Carlo method, perform stratified sampling in the parameter space of the ASM3 model, construct two independent parameter matrices, generate a mixed parameter combination matrix for sensitivity calculation through resampling technology, perform ASM3 model simulation on each parameter combination in the mixed parameter combination matrix, obtain the complete time series response of the output variables, and establish a parameter output response database. The output variables include chemical oxygen demand, ammonia nitrogen, nitrate nitrogen and total nitrogen.

[0017] S13. Calculate the total sensitivity index for each parameter based on the parameter-based output response database.

[0018] Specifically, step S2 includes:

[0019] S21. Preset multiple time windows of different lengths, calculate the local sensitivity index of the parameter at multiple time scales at each time step, and use the parameter output response database to approximate the partial derivatives by the finite difference method to obtain the corresponding dynamic local sensitivity database.

[0020] S22. Calculate the system change intensity index corresponding to multiple time windows of different lengths, use the system change intensity index to calculate the weights corresponding to multiple time windows of different lengths, and use the weights and the corresponding local sensitivity indices to introduce a weighted total sensitivity index to construct a fusion time-varying sensitivity index.

[0021] S23. Standardize the fused time-varying sensitivity index, combine it with the total sensitivity index of the parameters sorted by size, and perform feature dimensionality reduction through weighted principal component analysis to obtain the time-varying sensitivity state features and real-time sensitivity weights of the corresponding parameters.

[0022] Specifically, the formula for calculating the system change intensity index is as follows:

[0023]

[0024] In the formula: For time window The corresponding system change intensity index, For time steps, For the output variables of the ASM3 model, For time step Inside The mean.

[0025] Specifically, step S3 includes:

[0026] S31. Construct the model state space, which includes four dimensions: sensitivity dimension, corresponding to the time-varying sensitivity state characteristics of the parameters; process state dimension, including the influent component concentration and operating parameters of the ASM3 model; error state dimension, including the errors of chemical oxygen demand, ammonia nitrogen, and nitrate nitrogen; and historical state dimension, including the process state dimension of the current time step, the process state dimension of the previous time step, and the process state dimension of the next time step.

[0027] S32, Define coarse adjustment motion space :

[0028]

[0029] In the formula: The action to be selected for the parameter is determined by the weight of the parameter's total sensitivity index. Different adjustment intensity levels correspond to different amplitude movements;

[0030] Increase the space for fine-tuning movements :

[0031]

[0032] In the formula: To continuously adjust the increment;

[0033] By fusing the coarse-tuned action space and the fine-tuned action space, the model state space is obtained;

[0034] Action vectors The formula is:

[0035] ,

[0036] S33. Establish a parameter correction strategy based on sensitivity weights:

[0037]

[0038]

[0039] In the formula: For parameter correction, For sensitivity weighting coefficients, To integrate time-varying sensitivity indices, Based on the adjustment range, For indicator functions; For the updated parameters, These are the parameters from the previous time step; Represents the i-th parameter;

[0040] S34. Design the model reward function that incorporates sensitivity information, as shown in the following formula:

[0041]

[0042] In the formula: For the model reward function, For water quality compliance reward function, Adjust the smoothness reward function for the parameters. For sensitive consistency reward function; , , The weights are for the reward function.

[0043] Specifically, the formula for the water quality compliance reward function is as follows:

[0044]

[0045] In the formula: Chemical oxygen demand ammonia nitrogen content Nitrogen content The weight value, This is the error value. Chemical oxygen demand ammonia nitrogen content Nitrogen content The output value, for The maximum value, The overall sensitivity index of the parameter, for Relative to parameters The partial derivatives;

[0046] The formula for the parameter-adjusted smoothness reward function is:

[0047]

[0048] In the formula: For parameters The maximum value;

[0049] The formula for the sensitivity consistency reward function is:

[0050]

[0051] In the formula: For the sensitivity consistency reward function The gradient vector of parameters.

[0052] Specifically, in step S4, parameters with a total sensitivity index greater than a set threshold are designated as high-sensitivity parameters. In the DDQN model with integrated sensitivity, the sampling priority of empirical samples with errors greater than the set threshold and containing high-sensitivity parameters is increased. At the same time, a multi-step learning mechanism is introduced, using multi-step temporal difference objectives to reduce the variance of value estimation and accelerate learning convergence. Meanwhile, in the training of the DDQN model with integrated sensitivity, an exploration action selection combining time-varying sensitivity index and Q-value uncertainty is adopted.

[0053] Specifically, the formula for calculating sampling priority is:

[0054]

[0055] In the formula: The sampling priority for the i-th empirical sample. Let be the temporal difference error of the i-th empirical sample. For exploration rate, The total sensitivity index for the j-th parameter is... For indicator functions, For the i-th empirical sample, For the j-th parameter;

[0056] The expression for the exploration action selection, which combines the time-varying sensitivity index with Q-value uncertainty, is as follows:

[0057]

[0058] In the formula: Select a function for the action. Output the Q value. Sensitivity weighting factor For Q-value uncertainty, Let be the fusion time-varying sensitivity index of the j-th parameter.

[0059] Specifically, step S6 includes:

[0060] S61. Using the parameter-corrected ASM3 model, perform multi-step predictions within each control cycle:

[0061]

[0062] In the formula: For the first Time for the first Predicted values ​​of water quality at any given time. This represents the prediction process of the ASM3 model. For the predicted system state variables, To control variables, These are the corrected parameters. , For prediction in the time domain;

[0063] Establish state-space equations to describe the dynamic evolution process:

[0064]

[0065] In the formula: Represents the state function. For measurable disturbances;

[0066] S62. Constructing the control objective function The formula is as follows:

[0067]

[0068] In the formula: As a reference trajectory for effluent water quality, This is a reference value for the operation variable. , , These are the weight matrices for output tracking, input penalty, and control increment penalty, respectively. To control the time domain, To control the increment;

[0069] Introducing an energy consumption optimization term into the control objective function :

[0070]

[0071] In the formula: For aeration volume, To recover traffic, For stirring power, , , All are weighting coefficients;

[0072] Constructing a comprehensive objective function :

[0073]

[0074] In the formula: Energy consumption weighting factor;

[0075] S63. Apply range constraints to the effluent water quality, control variables, and control increments. Use a sequential quadratic programming algorithm to solve the constrained optimization problem. In each control cycle, obtain the optimal control sequence through iterative linearization and quadratic programming. Implement the first control action in the optimal control sequence. Repeat the optimization process in the next control cycle to complete the construction of the model predictive control system.

[0076] S64. Based on the first control action of the optimal control sequence, the corresponding control command is sent to the wastewater biological treatment system, and the wastewater biological treatment system adjusts the blower of the aeration system, controls the flow of the return pump, and adjusts the speed of the agitator.

[0077] Preferably, the above method further includes the following steps:

[0078] S7. Establish a closed-loop feedback control mechanism, monitor the actual data of parameters online, compare the measured data with the predicted data of the parameter-corrected ASM3 model, and when the feedback error exceeds the set threshold, perform parameter correction on the ASM3 model and adjust the control parameters of the model predictive control system.

[0079] Beneficial Effects: Compared with existing technologies, the significant advantages of this invention are: It prioritizes parameters through global sensitivity analysis, focusing on highly sensitive parameters to significantly improve parameter correction efficiency and accuracy; it dynamically evaluates the local sensitivity of parameters based on multi-scale time windows, achieving adaptive response of parameter correction to system changes at different time scales; it integrates data-driven learning and physical mechanism knowledge using a sensitivity-guided DDQN network, achieving intelligent real-time parameter correction while maintaining physical interpretability; the model predictive control system achieves multi-objective optimization of stable effluent quality and energy-saving system operation based on the calibrated ASM3 model; and it designs and constructs a complete closed-loop system from sensitivity analysis to parameter correction to predictive control, significantly improving model prediction accuracy, control response speed, operational stability, and energy efficiency compared to traditional offline parameter correction and empirical control methods, providing complete technical support for the intelligent operation of the activated sludge process. Attached Figure Description

[0080] Figure 1 This is a flowchart of the method in Embodiment 1 of the present invention.

[0081] Figure 2 This is a comparison chart of effluent water quality under PID control and MPC control in Embodiment 2 of the present invention.

[0082] Figure 3 This is a comparison chart of DO concentration and aeration pump controlled by PID control and MPC control in Embodiment 2 of the present invention. Detailed Implementation

[0083] A preferred embodiment of the present invention will be further described below with reference to the accompanying drawings.

[0084] Example 1

[0085] Please see Figure 1 As shown, this embodiment provides a predictive control method for wastewater biological treatment models, including the following steps:

[0086] S1. Perform static global sensitivity analysis on the parameters of the ASM3 model to obtain the overall sensitivity index of the parameters.

[0087] S11. Construct the parameter space of the ASM3 model:

[0088]

[0089] Includes the maximum growth rate of heterotrophic bacteria Maximum growth rate of autotrophic bacteria heterotrophic bacteria yield coefficient Autotrophic bacteria yield coefficient matrix half-saturation constant oxygen half-saturation constant of heterotrophic bacteria Oxygen half-saturation constant of autotrophic bacteria nitrate half-saturation constant Heterotrophic bacteria attenuation coefficient Autotrophic bacteria attenuation coefficient Anaerobic reducing factor Anaerobic reducing factor and hydrolysis rate constant These parameters constitute the complete set of parameters that need to be dynamically adjusted in subsequent deep reinforcement learning algorithms. The range of variation of the i-th parameter is defined as follows:

[0090]

[0091] In the formula: These are typical values ​​reported in the literature, laying the parameter foundation for the entire technical route.

[0092] S12. Generate Sobol sequences based on the quasi-Monte Carlo method, perform stratified sampling within the parameter space of the ASM3 model, and construct two mutually independent sequences. dimensional parameter matrix and ( The number of sampling points. (For parameter dimensions), a hybrid parameter combination matrix for sensitivity calculation is generated using resampling techniques. :

[0093]

[0094] Perform ASM3 model simulations on each parameter combination in the mixed parameter combination matrix to obtain the output variables. Complete time-series response, establish a parameter output response database. This provides sufficient sample data for sensitivity calculations.

[0095] S13. Calculate the total sensitivity index for each parameter based on the parameter-based output response database.

[0096] The total variance and conditional variance are calculated using variance decomposition techniques. The total variance of the output variable... Defined as:

[0097]

[0098] In the formula: This is the mean of the output variable.

[0099] parameter conditional variance of The calculation formula is:

[0100]

[0101] Calculate the overall sensitivity index for each parameter. :

[0102]

[0103] In the formula: Indicates the parameter All other parameters except those mentioned above.

[0104] Establish a parameter sensitivity ranking matrix ,Will (threshold) The parameters of ) are labeled as a set of highly sensitive parameters. .

[0105] Output a standardized parameter sensitivity index vector. and parameter importance weight matrix This serves as input data for subsequent steps. Highly sensitive parameters. This will allow for greater adjustment weights and finer control granularity in the action space of deep reinforcement learning, while low-sensitivity parameters can be fixed or have their optimization priority reduced, providing baseline sensitivity information and weighting basis for subsequent time-varying sensitivity analysis.

[0106] S2. Calculate the local sensitivity index of the parameter at different time scales. Construct a fused time-varying sensitivity index using the total sensitivity index and local sensitivity index of the parameter. Encode the fused time-varying sensitivity index to obtain the time-varying sensitivity state features and real-time sensitivity weights of the parameter.

[0107] S21. Preset multiple time windows of different lengths, calculate the local sensitivity index of the parameter at multiple time scales at each time step, and use the parameter output response database to approximate the partial derivatives through the finite difference method to obtain the corresponding dynamic local sensitivity database.

[0108] Define short window Mid-term window Long-term window At each time step Calculate parameters at three time scales The local sensitivity index, in short-term window Corresponding local sensitivity index For example calculations, the remaining windows are calculated in the same way:

[0109]

[0110] The parameter-output response relationship established using S1 The partial derivatives are approximated by the finite difference method. This extends the static global sensitivity of S1 into a dynamic local sensitivity time series.

[0111] S22. Calculate the system change intensity index corresponding to multiple time windows of different lengths, use the system change intensity index to calculate the weights corresponding to multiple time windows of different lengths, and use the weights and the corresponding local sensitivity indices to introduce a weighted total sensitivity index to construct a fused time-varying sensitivity index.

[0112] The formula for calculating the system change intensity index is:

[0113]

[0114] In the formula: For time window The corresponding system change intensity index, For time steps, For the output variables of the ASM3 model, For time step Inside The mean.

[0115] Calculate adaptive time weights:

[0116]

[0117] The fusion time-varying sensitivity index of the i-th parameter :

[0118]

[0119] In the formula: This is a static sensitivity retention factor that ensures that basic sensitivity information is preserved.

[0120] S23. Standardize the fused time-varying sensitivity index, combine it with the total sensitivity index of the parameters sorted by size, and perform feature dimensionality reduction through weighted principal component analysis to obtain the time-varying sensitivity state features and real-time sensitivity weights of the corresponding parameters.

[0121] First, the fusion time-varying sensitivity index is standardized:

[0122]

[0123] In the formula: For standardized processing of the fusion time-varying sensitivity index, and These are the historical mean and standard deviation vectors of the sensitivity index, respectively.

[0124] By combining the parameter importance weight matrix, feature dimensionality reduction is performed through weighted principal component analysis:

[0125]

[0126]

[0127] In the formula: For weighted fusion time-varying sensitivity index, To retain the cumulative variance contribution rate The principal component transformation matrix is ​​used, and the encoding process makes full use of the parameter importance information provided by S1.

[0128] Output the encoded time-varying sensitive state feature vector This vector will serve as the sensitivity dimension input to the model's state space. Simultaneously, it outputs a real-time sensitivity weight matrix. This provides important guidance for parameter tuning in model action space design and reward function construction, ensuring that reinforcement learning algorithms can make intelligent adjustment decisions based on the real-time sensitivity and historical importance of parameters.

[0129] S3. Based on the total sensitivity index, time-varying sensitivity state features, and real-time sensitivity weights of the parameters, construct the model state space, model action space, and model reward function.

[0130] S31. Construct a four-dimensional integrated state space that incorporates sensitive information. :

[0131] Sensitivity State Dimension: It directly receives the output of S2;

[0132] Process state dimension: ;

[0133] Error state dimension: The error is defined as:

[0134]

[0135]

[0136]

[0137] In the formula: For the target quantity, This refers to the actual quantity.

[0138] Historical state dimension: .

[0139] S32, Define coarse adjustment motion space :

[0140]

[0141] In the formula: Select an action for the parameters. The selection probability is the weight of the total sensitivity index of the parameter;

[0142] To adjust the amplitude of the movement, Different adjustment intensity levels correspond to different adjustment levels;

[0143] Increase the space for fine-tuning movements :

[0144]

[0145] In the formula: To continuously adjust the increment;

[0146] By fusing the coarse-tuned action space and the fine-tuned action space, the model state space is obtained;

[0147] Action vectors The formula is:

[0148] ,

[0149] S33. Establish a parameter correction strategy based on sensitivity weights:

[0150]

[0151]

[0152] In the formula: For parameter correction, For sensitivity weighting coefficients, To integrate time-varying sensitivity indices, Based on the adjustment range, For indicator functions; For the updated parameters, These are the parameters from the previous time step; Represents the i-th parameter; in this embodiment, , .

[0153] Safety constraints:

[0154]

[0155] In the formula: These are the final output parameters.

[0156] S34. Design the model reward function that incorporates sensitivity information, as shown in the following formula:

[0157]

[0158] In the formula: For the model reward function, For water quality compliance reward function, Adjust the smoothness reward function for the parameters. For sensitive consistency reward function; , , The weights are for the reward function.

[0159] Weights satisfy In this embodiment, , , .

[0160] The formula for the water quality compliance reward function is:

[0161]

[0162] In the formula: Chemical oxygen demand ammonia nitrogen content Nitrogen content The weight value, This is the error value. Chemical oxygen demand ammonia nitrogen content Nitrogen content The output value, for The maximum value, The overall sensitivity index of the parameter, for Relative to parameters The partial derivatives;

[0163] The formula for the parameter-adjusted smoothness reward function is:

[0164]

[0165] In the formula: For parameters The maximum value;

[0166] The formula for the sensitivity consistency reward function is:

[0167]

[0168] In the formula: For the sensitivity consistency reward function for the first The gradient vector of parameters.

[0169] Output complete state-action space definition and reward function design scheme This provides a specific framework for reinforcement learning problems in subsequent network training.

[0170] S4. Construct a dual-network training framework based on the DDQN model using the model state space, model action space, and model reward function. After training, an integrated sensitivity DDQN model is obtained.

[0171] The DDQN model consists of a main Q-network and a target Q-network, employing a deep multilayer perceptron architecture: the input layer receives a multidimensional state vector (including sensitivity features, process states, error states, and historical states); the first hidden layer contains 128 neurons using the ReLU activation function; the second hidden layer contains 64 neurons, also using the ReLU activation function; the output layer corresponds to the dimension of the action space, providing a Q-value estimate for each possible action. The main network is responsible for real-time decision output and parameter updates, while the target network provides a relatively stable training objective. The core of the DDQN model lies in the collaborative working mechanism of the two networks: the main network is used for action selection, and the target network is used for value evaluation, effectively reducing overestimation problems.

[0172] DDQN model objective value Calculation formula:

[0173]

[0174] In the formula: For the reward function, As a discount factor, For the next state, For the next action, Main network parameters, These are the target network parameters.

[0175] Main network loss function:

[0176]

[0177] In the formula: The expected value of the samples in the experience replay pool.

[0178] Target network soft update mechanism:

[0179]

[0180] In the formula: This is the soft update coefficient. The value estimation oscillation problem during training is reduced by periodically synchronizing the parameters of the main network.

[0181] The importance of each empirical sample in training is determined based on its temporal difference error and the sensitivity level of the parameters involved. The temporal difference error is then calculated. :

[0182]

[0183] In the formula: These are the target network parameters.

[0184] Calculate sampling priority by combining sensitivity weights:

[0185]

[0186] In the formula: The sampling priority for the i-th empirical sample. Let be the temporal difference error of the i-th empirical sample. For exploration rate, The total sensitivity index for the j-th parameter is... For indicator functions, For the i-th empirical sample, This is the j-th parameter.

[0187] We design a sensitivity-based adaptive exploration strategy that combines the traditional epsilon-greedy method with an uncertainty-based intelligent exploration mechanism.

[0188] In the early stages of training, a high exploration rate is used to allow for thorough environmental exploration. Attenuation occurs in the following manner:

[0189]

[0190] In the formula: The time constant for the rate of decay is the exploration rate.

[0191] As training progresses, the exploration rate is gradually reduced, shifting towards utilizing more of the learned knowledge. Simultaneously, sensitivity-based exploration guidance is introduced, using Monte Carlo Dropout to estimate Q-value uncertainty. :

[0192]

[0193] In the formula: The number of Monte Carlo Dropout samples. For the first Q-value estimate obtained from Dropout sampling for The average Q value of each sample.

[0194] The expression for the exploration action selection, which combines the time-varying sensitivity index with Q-value uncertainty, is as follows:

[0195]

[0196] In the formula: Select a function for the action. Output the Q value. For sensitivity weighting factors, For Q-value uncertainty, Let be the fusion time-varying sensitivity index of the j-th parameter.

[0197] More cautious exploration should be conducted on state-action pairs involving highly sensitive parameters to avoid systemic risks that may result from overexploration.

[0198] After training, an integrated sensitivity DDQN model is output, providing intelligent decision-making capabilities for subsequent real-time parameter correction.

[0199] S5. Collect the state information corresponding to the model state space from the wastewater biological treatment system in real time, input it into the DDQN model with integrated sensitivity, output the adjustment range of each parameter, update the parameters of the ASM3 model using the parameter adjustment range, and obtain the parameter-corrected ASM3 model.

[0200] A real-time state awareness module compatible with the data flow of the previous steps is established, which collects and organizes multi-dimensional state information in real time according to the state space structure of the model designed in S3. Sensitivity state updates calculate time-varying sensitivity features in real time by calling the sensitivity encoder established in S2; process state acquisition includes key operating condition information such as influent water quality indicators and reactor operating parameters; error state calculates the deviation of each water quality indicator from the target value and its rate of change; historical state organizes the state information of past time steps. The complete state vector integrates information from the four dimensions of sensitivity, process, error, and history, and performs standardized preprocessing to ensure consistency with the data in the training phase.

[0201] The DDQN model main network with integrated sensitivity is invoked for real-time decision reasoning. The preprocessed state vector is input into the network, and the final output layer generates Q-value estimates for each possible action.

[0202] Choose the action with the highest Q value as the optimal decision for the current moment:

[0203]

[0204] The network output action consists of three parts: parameter selection, adjustment range, and fine adjustment increment.

[0205] For the selected parameters that need to be adjusted, a parameter correction strategy based on sensitivity weights is used to correct the parameters. This design ensures that the parameter adjustment strategy can respond to the dynamic needs of the current system (reflected in real-time sensitivity) and also conform to the inherent importance characteristics of the parameters (reflected in static sensitivity), thus achieving intelligent parameter correction decision-making.

[0206] Update the parameters of the ASM3 model and perform a simplified performance evaluation. Run the ASM3 model with the adjusted parameters for short-term predictions and calculate the expected changes in the system response. Evaluate the consistency between the actual performance and the sensitivity forecasts.

[0207]

[0208] In the formula: For consistency scoring, To output the total number of elements in the variable, After parameter adjustment, the first The actual system response change values ​​of each output variable For the first expected value based on sensitivity weights The system response change values ​​of each output variable.

[0209] The system outputs parameter adjustment results, effect predictions, and consistency assessment information. Through this real-time parameter correction mechanism, the system can quickly respond to and adjust model parameters based on current operating conditions and sensitivity knowledge, thereby achieving intelligent control of the wastewater treatment process.

[0210] S6. Construct a model predictive control system based on the ASM3 model with parameter correction, and use the model predictive control system to solve and issue control commands to the wastewater biological treatment system.

[0211] S61. Define the prediction time domain and control time domain The parameter-corrected ASM3 model performs multi-step predictions within each control cycle:

[0212]

[0213] In the formula: For the first Time for the first Predicted values ​​of water quality at any given time. This represents the prediction process of the ASM3 model. For the predicted system state variables, To control variables, These are the corrected parameters. ;

[0214] Establish state-space equations to describe the dynamic evolution process:

[0215]

[0216] In the formula: Represents the state function. For measurable disturbances;

[0217] S62. Design a multi-objective control function that balances effluent quality compliance and energy consumption optimization, unifying water quality constraints, operational constraints, and energy consumption minimization into an optimization framework. First, construct the control objective function. The formula is as follows:

[0218]

[0219] In the formula: As a reference trajectory for effluent water quality, This is a reference value for the operation variable. , , These are the weight matrices for output tracking, input penalty, and control increment penalty, respectively. To control the increment;

[0220] Introducing an energy consumption optimization term into the control objective function :

[0221]

[0222] In the formula: For aeration volume, To recover traffic, For stirring power, , , These are all weighting coefficients, which can reflect the energy consumption characteristics of different devices;

[0223] Constructing a comprehensive objective function :

[0224]

[0225] In the formula: This is an energy consumption weighting factor; by adjusting this parameter, water quality control performance and energy-saving effects can be balanced.

[0226] S63. Apply range constraints to the effluent water quality, control variables, and control increments. Use the Sequential Quadratic Programming (SQP) algorithm to solve the constrained optimization problem. In each control cycle, obtain the optimal control sequence through iterative linearization and quadratic programming. Implement the first control action in the optimal control sequence. Repeat the optimization process in the next control cycle to complete the construction of the model predictive control system.

[0227] Effluent water quality constraints ensure that discharge meets standards:

[0228]

[0229] Control variable constraints ensure operational safety:

[0230]

[0231] Controlling incremental constraints to avoid drastic operational changes:

[0232]

[0233] S64. Based on the first control action of the optimal control sequence, the corresponding control command is sent to the wastewater biological treatment system, and the wastewater biological treatment system adjusts the blower of the aeration system, controls the flow of the return pump, and adjusts the speed of the agitator.

[0234] S7. Establish a closed-loop feedback control mechanism, monitor the actual data of parameters online, compare the measured data with the predicted data of the parameter-corrected ASM3 model, and when the feedback error exceeds the set threshold, perform parameter correction on the ASM3 model and adjust the control parameters of the model predictive control system.

[0235] Key indicators such as effluent quality, dissolved oxygen concentration, and sludge concentration are monitored in real time using online sensors. The measured data are then compared and analyzed with model predictions.

[0236]

[0237] In the formula: For feedback error, For measured values, This is the predicted value obtained based on the previous time step k-1.

[0238] When the feedback error exceeds a preset threshold, the model correction and control parameter adjustment mechanism is triggered. Simultaneously, the system's energy consumption indicators are monitored, and the comprehensive energy consumption per unit volume of water treated is calculated in real time. :

[0239]

[0240] In the formula: Total power consumption, The system represents the influent flow rate. Through rolling optimization and real-time feedback, the system can continuously optimize its operating strategy to achieve energy conservation and consumption reduction goals while ensuring stable compliance of the effluent water quality. It outputs optimized control commands, energy consumption statistics, and water quality prediction results, providing fundamental data for subsequent performance evaluation.

[0241] S8 receives the parameter adjustment results and effect evaluation information output from S5, establishes a comprehensive performance evaluation system integrating sensitivity verification, and comprehensively evaluates system performance by calculating prediction accuracy indicators, control effect indicators, and sensitivity consistency indicators. A performance degradation detection mechanism is established; when the comprehensive performance score falls below a preset threshold, model correction measures are automatically triggered, providing performance feedback information and improvement direction guidance for continuous model optimization. Based on the performance feedback from S6, a systematic optimization mechanism is implemented, including sensitivity model updates, incremental learning of the DDQN model, and expert knowledge fusion. When the sensitivity prediction accuracy decreases, the global sensitivity index is recalculated; when overall performance is insufficient, new operating data is used to fine-tune the neural network. Simultaneously, the experience and knowledge of operational experts are periodically integrated to update the control strategy through knowledge distillation technology, ensuring that the entire technical approach can adapt to long-term changes in the wastewater treatment system and maintain stable and efficient parameter correction capabilities and control performance.

[0242] Example 2

[0243] This embodiment illustrates the wastewater biological treatment model prediction and control method described in Example 1 in a specific scenario.

[0244] The wastewater biological treatment system used in this embodiment is a modified A² / O experimental reactor constructed by the Environmental Engineering Laboratory of Yangzhou University. It has a total volume of 70 L and a daily treatment capacity of 150 L, with the influent flow rate adjustable as needed. This device is primarily used to verify the effectiveness of deep reinforcement learning in parameter correction of the activated sludge model (ASM) and to simulate the biological treatment process under different influent water quality conditions. The entire process flow includes a pre-anoxic tank, an anaerobic tank, two anoxic tanks, two aerobic tanks, a post-anoxic tank, and a secondary sedimentation tank.

[0245] The pre-anoxic tank is located at the inlet end, reducing the dissolved oxygen concentration to provide an environment for denitrification; the anaerobic tank, under anaerobic conditions, allows microorganisms to release phosphorus and decompose organic matter; two anoxic tanks are used for denitrification, removing nitrate nitrogen returned from the aerobic zone. ) is converted into nitrogen gas ( Two aerobic tanks, through aeration, complete the degradation and nitrification of organic matter, reducing ammonia nitrogen ( ) is converted into nitrate nitrogen ( The anoxic tank further improves the denitrification effect; finally, the secondary sedimentation tank completes solid-liquid separation to ensure that the effluent meets the standards.

[0246] To ensure stable operation of the device and accuracy of the research, the system is equipped with online monitoring sensors manufactured by WTW, enabling real-time monitoring of the wastewater treatment process. The sensors include: SensoLyt... ® 700 IQ, used for monitoring pH and redox potential; ViSolid ®700 IQ, used for monitoring suspended solids; FDO ® 700 IQ, used for monitoring dissolved oxygen; VARiON ® The Plus 700IQ is used to monitor ammonia nitrogen and nitrate nitrogen. The data acquisition frequency is set to once every 30 seconds to capture rapid dynamic changes in the system.

[0247] Furthermore, the device has established a comprehensive data processing and transmission system. The acquired sensor data is transmitted in real-time to a PostgreSQL database via MQTT communication. The database is linked to a SCADA system, enabling real-time monitoring and providing support for experimental analysis and process control.

[0248] The wastewater biological treatment model predictive control method described in Example 1 is applied to the aforementioned A² / O experimental reactor. Using the integrated sensitivity DDQN model trained in step S4, a real-time parameter correction execution system is established in the reactor's SCADA system. The system triggers a parameter correction decision every 30 minutes, communicating with the PLC via the OPCUA protocol. In this embodiment, the prediction time domain is set in step S6. (10h), control time domain (2.5h). Under the same operating conditions, the wastewater biological treatment model predictive control method (MPC) and the traditional PID control method were respectively applied to the above-mentioned A² / O experimental reactor for a period of 15 days.

[0249] Please refer to Figure 2 As shown, Figure 2 This indicates that the effluent NH4 and NO3 concentrations under PID control exhibit significant fluctuations. Large fluctuations in the curves mean that in actual operation, the effluent quality will frequently deviate from the ideal values, leading to substandard water quality or unstable treatment effects. This fluctuation reflects the limited response speed and ability to suppress fluctuations of the PID controller when facing system disturbances. It can typically only adjust based on the current error and lacks the ability to predict future trends. In contrast, the effluent NH4 and NO3 concentrations under MPC simulation control demonstrate extremely high stability. The curves are very flat, almost remaining within the set target range. This stability is the core advantage of the MPC controller. By establishing a system model and predicting future system behavior, MPC can plan control actions in advance, effectively suppressing fluctuations and keeping the effluent quality within the target range for extended periods. Even small fluctuations are confined to a very small range.

[0250] Please refer to Figure 3 As shown, Figure 3The DO concentration under PID control fluctuated wildly within the range of 0.5-2.5 mg / L, and the aeration pump power consumption oscillated frequently and significantly between 0-15W. This instability reflects the limitation of the PID controller, which can only react to the current deviation and cannot predict the system's changing trends, resulting in poor water quality control and high energy consumption. In contrast, the DO concentration under MPC control remained stable within a reasonable range of 1.0-1.4 mg / L, and the aeration pump power consumption also showed a stable and significantly lower operating state. This significant improvement stems from the predictive control strategy of MPC, which can predict future changing trends based on the system model and formulate the optimal control scheme in advance. It minimizes energy consumption while ensuring water quality meets standards, demonstrating the significant advantages of MPC in wastewater treatment systems.

Claims

1. A method of model predictive control of a wastewater biological treatment process, characterized by, The method comprises the following steps: S1, static global sensitivity analysis is performed on parameters of the ASM3 model to obtain a total sensitivity index of the parameters; S2, local sensitivity indexes of the parameters are calculated respectively under different time scales, a fusion time-varying sensitivity index is constructed by using the total sensitivity index and the local sensitivity indexes, and a time-varying sensitivity state feature and a real-time sensitivity weight of the parameters are obtained after the fusion time-varying sensitivity index is coded; S3, a model state space, a model action space and a model reward function are constructed based on the total sensitivity index, the time-varying sensitivity state feature and the real-time sensitivity weight of the parameters; S4, a double-network training framework based on the DDQN model is constructed by using the model state space, the model action space and the model reward function, and an integrated sensitivity DDQN model is obtained after training; S5, state information corresponding to the model state space is collected in real time from the wastewater biological treatment system, and the integrated sensitivity DDQN model is input, and an adjustment amplitude of each parameter is output, the parameters of the ASM3 model are updated by using the adjustment amplitude, and a parameter-corrected ASM3 model is obtained; S6, a model predictive control system is constructed based on the parameter-corrected ASM3 model, and a control instruction is solved and issued to the wastewater biological treatment system by using the model predictive control system.

2. The wastewater biological treatment model predictive control method of claim 1, wherein, Step S1 comprises: S11, a parameter space of the ASM3 model is constructed, including maximum growth rate of heterotrophic bacteria, maximum growth rate of autotrophic bacteria, yield coefficient of heterotrophic bacteria, yield coefficient of autotrophic bacteria, substrate half-saturation constant, oxygen half-saturation constant of heterotrophic bacteria, oxygen half-saturation constant of autotrophic bacteria, nitrate half-saturation constant, decay coefficient of heterotrophic bacteria, decay coefficient of autotrophic bacteria, anaerobic reagent, anaerobic reagent and hydrolysis rate constant; S12, Sobol sequence is generated based on quasi-Monte Carlo method, stratified sampling is performed in the parameter space of the ASM3 model, two independent parameter matrices are constructed, a mixed parameter combination matrix used for sensitivity calculation is generated by resampling technology, ASM3 model simulation is performed on each parameter combination in the mixed parameter combination matrix, complete time series response of output variables is obtained, an output response database of the parameters is established, and the output variables include chemical oxygen demand, ammonia nitrogen content, nitrate nitrogen content and total nitrogen content; S13, the total sensitivity index of each parameter is calculated based on the output response database of the parameters.

3. The wastewater biological treatment model predictive control method of claim 2, wherein, Step S2 comprises: S21, a plurality of time windows of different lengths are preset, local sensitivity indexes of the parameters under a plurality of time scales are calculated respectively at each time step, partial derivatives are approximately calculated by using the output response database of the parameters through finite difference method, and a corresponding dynamic local sensitivity database is obtained; S22, system change intensity indexes corresponding to the plurality of time windows of different lengths are calculated, weights corresponding to the plurality of time windows of different lengths are calculated by using the system change intensity indexes, and a weighted total sensitivity index is constructed by using the weights and the corresponding local sensitivity indexes to construct a fusion time-varying sensitivity index; S23, the time-varying sensitivity index is standardized, combined with the total sensitivity index of the parameters sorted by size, and the feature dimension is reduced by weighted principal component analysis to obtain the time-varying sensitivity state feature and real-time sensitivity weight of the corresponding parameters.

4. The wastewater biological treatment model predictive control method of claim 3, wherein: The system change intensity index calculation formula is: wherein: is the time window corresponding system change intensity indicator, is the time step, is the output variable of the ASM3 model, is the time step is the mean value of .

5. The wastewater biological treatment model predictive control method of claim 3, wherein, Step S3 includes: S31, a model state space is constructed, which includes four dimensions: a sensitivity dimension corresponding to the time-varying sensitivity state feature of the parameters; a process state dimension including the influent component concentration and operating parameters of the ASM3 model; an error state dimension including the errors of chemical oxygen demand, ammonia nitrogen and nitrate nitrogen; and a historical state dimension including the process state dimension of the current time step, the process state dimension of the previous time step and the process state dimension of the next time step; S32, define coarse action space : In the formula: is a parameter selection action, and the probability is the weight of the total sensitivity index of the parameter; is an adjustment range action, corresponding to different adjustment intensity levels; Increasing the fine motion space : In the formula: is a continuous adjustment increment; The coarse adjustment action space and the fine adjustment action space are fused to obtain the model state space; Action vector The formula is: ， S33, a parameter correction strategy based on the sensitivity weight is established: wherein: is a parameter correction amount, is a sensitivity weight coefficient, is a fused time-varying sensitivity index, is a base adjustment amplitude, is an indicator function; is an updated parameter, is a parameter of a previous time step; represents the i-th parameter; S34, a model reward function integrating sensitivity information is designed, and the formula is as follows: In the formula, is a model reward function, is a water quality compliance reward function, is a parameter adjustment smoothness reward function, is a sensitive consistency reward function; , , is a reward function weight.

6. The wastewater biological treatment model predictive control method according to claim 5, characterized in that: The water quality standard reward function formula is: wherein: is the weight value of the chemical oxygen demand , the ammonia nitrogen amount , the nitrate nitrogen amount , is the error value, is the output value of the chemical oxygen demand , the ammonia nitrogen amount , the nitrate nitrogen amount , is the maximum value of , is the total sensitivity index of the parameters, is the partial derivative of with respect to the parameter ; The parameter adjustment smoothness reward function formula is: wherein: is a parameter is the maximum value; The sensitivity consistency reward function formula is: In the formula: For the sensitivity consistency reward function for the first The gradient vector of parameters.

7. The wastewater biological treatment model predictive control method of claim 1, wherein, In step S4, in the DDQN model integrating sensitivity, the parameters with a total sensitivity index greater than a set threshold are set as high-sensitivity parameters, the sampling priority of the experience samples with an error greater than a set threshold and containing high-sensitivity parameters is improved, a multi-step learning mechanism is introduced, a multi-step time difference target is used to reduce the variance of value estimation and accelerate learning convergence; meanwhile, an exploration action selection combining the time-varying sensitivity index and Q value uncertainty is adopted in the training of the DDQN model integrating sensitivity.

8. The wastewater biological treatment model predictive control method of claim 7, wherein: The sampling priority calculation formula is: wherein: is the sampling priority of the i-th empirical sample, is the timing differential error of the i-th empirical sample, is the exploration rate, is the total sensitivity index of the j-th parameter, is the indicator function, is the i-th empirical sample, is the j-th parameter; The exploration action selection expression combining the time-varying sensitivity index and Q value uncertainty is as follows: where: is the action selection function, is the Q-value output, is the sensitivity weight factor, is the Q-value uncertainty, is the fusion time-varying sensitivity index for the jth parameter.

9. The wastewater biological treatment model predictive control method of claim 1, wherein, Step S6 includes: S61, the ASM3 model with parameter correction is used for multi-step prediction in each control period: wherein: is the first is the predicted value of the water quality at the first is the predicted value of the water quality at the first represents the prediction process of the ASM3 model, is the predicted system state variable, is the control variable, is the corrected parameter, , is the prediction time horizon; A state space equation is established to describe the dynamic evolution process: wherein: represents a state function, is a measurable disturbance; S62, construct control objective function The formula is as follows: wherein: is a reference trajectory for the effluent water quality, is a reference value for the manipulated variable, , , are weight matrices for the output tracking, input penalty and control increment penalty, respectively, is a control time horizon, is a control increment; Introducing an energy consumption optimization term in the control objective function : In the formula: is the aeration amount, is the backflow amount, is the stirring power, , , are all weight coefficients; Constructing the comprehensive objective function : In the formula: is the energy consumption weight factor; S63, range constraints are applied to the effluent water quality, control variables and control increments, a sequential quadratic programming algorithm is used to solve the constrained optimization problem, in each control period, an optimal control sequence is obtained through iterative linearization and quadratic programming solution, the first control action in the optimal control sequence is implemented, and the optimization process is repeated in the next control period to complete the construction of the model predictive control system; S64, according to the first control action of the optimal control sequence, corresponding control instructions are issued to the wastewater biological treatment system, and the wastewater biological treatment system performs fan adjustment of the aeration system, flow control of the reflux pump and speed regulation of the stirrer.

10. The wastewater biological treatment model predictive control method of claim 9, wherein, Further comprising the following steps: S7, a closed-loop feedback control mechanism is established, the actual data of the parameters are monitored online, the measured data are compared with the predicted data of the ASM3 model with parameter correction, and when the feedback error exceeds a set threshold, the ASM3 model is corrected, and the control parameters of the model predictive control system are adjusted.

Images