Method and system for adjusting air volume based on sewage treatment aeration

Abstract

This application relates to a method and system for regulating airflow in wastewater treatment aeration. The method includes: constructing a multi-dimensional feature vector using historical DO concentration, airflow, influent load, and time data; using a Gaussian mixture model to cluster historical operating conditions into multiple operating modes and generate an operating mode classifier; for each operating mode, training an environment simulator based on a long short-term memory neural network using a subset of historical data to simulate state transitions and output multi-objective reward signals for comprehensive water quality compliance, energy consumption, and stability; constructing a deep reinforcement learning agent based on Actors and Critics for each operating mode, and offline training it in the corresponding environment simulator to obtain the optimal control strategy network; during online operation, calculating the membership probability of the current operating condition to each mode in real time using the operating mode classifier, and performing probability-weighted fusion of the candidate DO regulation airflow cycle and airflow regulation step size output by each strategy network to obtain the final control parameters.

Description

Technical Field

[0001] This application relates to the field of wastewater treatment technology, and in particular to a method and system for regulating air volume based on wastewater treatment aeration. Background Technology

[0002] In wastewater treatment processes, aeration control in the aerobic section of the biological treatment tank is a core element determining pollutant removal efficiency, system energy consumption, and operational stability. Aerobic microorganisms consume dissolved oxygen (DO) to metabolize and degrade organic pollutants and to complete nitrification. The DO concentration must be maintained within a specific process range by precisely controlling the blower's aeration airflow. Current aeration control methods focus on setting two key parameters: the DO adjustment airflow cycle (the time interval between two airflow adjustments) and the airflow adjustment step size (the magnitude of each airflow adjustment). Scientifically determining these two parameters based on the actual operating conditions of the biological treatment tank is the core technical challenge for achieving precise DO control and energy conservation.

[0003] Currently, the mainstream aeration control technology in wastewater treatment plants both domestically and internationally is mainly based on feedback-based fuzzy control. This technology uses online dissolved oxygen analyzers to collect the dissolved oxygen concentration in the biological treatment tank in real time and compare it with the process setpoint. When the dissolved oxygen concentration is lower than the setpoint, the control system increases the blower airflow by a preset fixed airflow adjustment step. When the dissolved oxygen concentration is higher than the setpoint, the blower airflow decreases by a preset fixed airflow adjustment step. The above airflow adjustment actions are executed periodically according to a preset fixed DO airflow adjustment cycle.

[0004] The aforementioned technology relies on manual experience to set the DO (Discharge) airflow regulation cycle, and the airflow adjustment step is fixed, which cannot adapt to dynamic changes in operating conditions and affects the wastewater treatment effect.

[0005] Based on this, this application provides a method and system for regulating air volume in wastewater treatment aeration. Summary of the Invention

[0006] To address the issue that the airflow regulation cycle of DO (discharge treatment) relies on manual experience and has a fixed adjustment step, which cannot adapt to dynamic changes in operating conditions and thus affects the wastewater treatment effect, this application provides an airflow regulation method and system based on wastewater treatment aeration.

[0007] Firstly, this application provides a method for regulating airflow in wastewater treatment aeration, employing the following technical solution: including: Receive historical DO concentration, air volume, influent load and time data, construct a multi-dimensional feature vector representing the operating conditions; use an unsupervised clustering algorithm to cluster the multi-dimensional feature vector, divide the historical operating conditions into several operating condition modes, and generate an operating condition classifier that can output the operating condition membership probability in real time. For each operating mode, a data-driven time series prediction model is trained using its corresponding historical data subset. The input of the time series prediction model is a state vector containing DO deviation, DO change rate and influent load, and a two-dimensional action vector consisting of DO regulation air volume cycle and air volume regulation step. The output is the next state prediction value and an instant reward value consisting of water quality compliance reward, energy consumption penalty and stability penalty. For each working condition mode, a deep reinforcement learning agent containing an Actor network and a Critic network is constructed. The agent is trained offline in the corresponding time series prediction model with the goal of maximizing the cumulative reward, so that the Actor network of the agent serves as the optimal control policy network for the corresponding working condition mode. Real-time data collection and construction of current multidimensional feature vectors; inputting the current multidimensional feature vectors into the working condition classifier; calculating the membership probability distribution of the current working condition belonging to several working condition modes. The current state vector is input into each of the optimal control strategy networks to obtain several sets of candidate DO airflow regulation cycles and airflow regulation steps. The membership probability distribution is used to weight and sum the several sets of candidate values ​​to obtain the DO airflow regulation cycle and airflow regulation step.

[0008] Preferably, the step of using an unsupervised clustering algorithm to cluster multidimensional feature vectors, dividing historical working conditions into several working condition modes, and generating a working condition classifier capable of outputting the working condition membership probability in real time includes: Preprocessing and feature extraction are performed on historical DO concentration, air volume, influent load and time data to construct a multi-dimensional feature vector corresponding to each historical moment. The multi-dimensional feature vector includes at least: the deviation between the current DO concentration and the set value, the rate of change of DO concentration within the past preset time window, water temperature and time context encoding features. The multidimensional feature vectors of all historical moments are input into the Gaussian mixture model for cluster analysis. The mean, covariance and weight parameters of each Gaussian component are estimated through the expectation-maximization algorithm until the model converges. The optimal number of clusters K is determined by using the Bayesian information criterion, and the historical operating conditions are divided into K operating condition modes, where K is an integer greater than or equal to 2; A working condition classifier is constructed based on the trained Gaussian mixture model. The input of the working condition classifier is the current multidimensional feature vector collected and constructed in real time, and the output is the posterior probability value of the current working condition belonging to each of the K working condition modes, forming the membership probability distribution. The posterior probability value is calculated by Bayes' formula from the probability density function of each Gaussian component in the Gaussian mixture model and the prior weights.

[0009] Preferably, the step of inputting the multidimensional feature vectors of all historical moments into a Gaussian mixture model for cluster analysis, and estimating the mean, covariance, and weight parameters of each Gaussian component through an iterative expectation-maximization algorithm until the model converges includes: Set an initial range for the number of clusters. The lower limit of the initial range for the number of clusters shall not be less than 2, and the upper limit of the initial range for the number of clusters shall be the preset maximum number of patterns. For each candidate cluster value within the range, the mean vector, covariance matrix, and mixture weights of the Gaussian components are randomly initialized in an equal number to the target candidate cluster value, and the sum of the mixture weights is equal to 1. The multidimensional feature vector set of all historical moments is input into the Gaussian mixture model, and the expectation step and the maximization step are executed iteratively: in the expectation step, the posterior probability value of each feature vector belonging to each Gaussian component is calculated based on the current model parameters; in the maximization step, the mean vector, covariance matrix and mixture weight of each Gaussian component are updated using the calculated posterior probability value. Repeat the expectation step and the maximization step until the log-likelihood function value of the model converges or the preset maximum number of iterations is reached, to obtain the Gaussian mixture model trained under the target candidate clustering numerical values; For each candidate cluster numerical value, the trained Gaussian mixture model is calculated, and the corresponding Bayesian information criterion value is positively correlated with the maximum log-likelihood value of the model. Compare the Bayesian information criterion values ​​corresponding to each candidate cluster value, select the candidate cluster value that maximizes the Bayesian information criterion value or makes the curve of the Bayesian information criterion value changing with the candidate cluster value reach an inflection point as the optimal number of clusters, and take the Gaussian mixture model corresponding to the optimal number of clusters as the final clustering model. Based on the maximum posterior probability of each feature vector belonging to each Gaussian component output by the final clustering model, the working condition modes are divided.

[0010] Preferably, the step of training a data-driven time series prediction model for each operating condition mode using its corresponding historical data subset includes: For each operating mode, extract the historical data subset corresponding to the target operating mode. The historical data subset contains the state vector sequence and action vector sequence of all historical moments belonging to the target operating mode label. The state vector sequence consists of at least DO concentration deviation, DO concentration change rate and water temperature, and the action vector sequence consists of DO regulating air volume cycle and air volume regulating step. A subset of historical data is constructed as a supervised learning sample set. The input features of each sample are a combination of the state vector and action vector at the current moment. The label of the sample is the state vector at the next moment. The time interval between the next moment and the current moment of the state vector is equal to the DO adjustment air volume cycle at the current moment. The long short-term memory neural network is used as the network structure of the time series prediction model. The input layer of the long short-term memory neural network receives the concatenated vector of the current state vector and action vector, extracts the temporal dependency features through at least one long short-term memory hidden layer, and outputs the predicted value of the state vector at the next time step through a fully connected output layer. With the goal of minimizing the mean square error between the predicted and true values ​​of the state vector, a gradient-based optimization algorithm is used to iteratively update the weight parameters of the long short-term memory neural network until the model converges or reaches the preset number of training rounds, thereby obtaining the time series prediction model trained under the target working condition. The trained time series prediction model is used as the environmental simulator corresponding to the target operating mode. At the same time, based on the preset multi-objective reward function, the water quality compliance reward is calculated according to the deviation between the predicted value of the state vector and the DO setting value, the energy consumption penalty is calculated according to the air volume adjustment step in the action vector, and the stability penalty is calculated according to the change amplitude between two adjacent action vectors. The three terms are weighted and summed to obtain the instantaneous reward value.

[0011] Preferably, the step of constructing a deep reinforcement learning agent containing an Actor network and a Critic network for each working condition mode, and performing offline training in the corresponding time series prediction model with the objective of maximizing cumulative reward, includes: For each operating mode, a deep reinforcement learning agent is constructed. The deep reinforcement learning agent contains an Actor network and a Critic network. The input of the Actor network is the current state vector, and the output is the mean and standard deviation of the two-dimensional action vector composed of the DO airflow adjustment cycle and airflow adjustment step. Based on the mean and standard deviation, the final action vector is generated through random sampling. The input of the Critic network is the concatenation vector of the current state vector and the action vector generated by the Actor network, and the output is the expected cumulative reward evaluation value of the state-action pair. An offline training loop is constructed. In each training round, a batch of initial state vectors is randomly sampled from the historical data subset corresponding to the target working condition mode. The initial state vectors are input into the Actor network to obtain action vectors. The initial state vectors and action vectors are input into the corresponding environment simulator after training to obtain the predicted value of the state vector at the next moment and the instant reward value. Each interaction generates a current state vector, action vector, instant reward value, next moment state vector, and interaction termination flag, which together form an experience sample and store it in the experience replay pool. The interaction termination flag is set to true when the DO concentration deviation continuously exceeds a preset threshold range or reaches a preset maximum number of interaction steps per round, thus terminating the interaction. Once the number of experience samples stored in the experience replay pool reaches the preset minimum sampling batch requirement, a batch of experience samples is randomly sampled from the experience replay pool, and the target cumulative reward value for each sample is calculated. The weight parameters of the Critic network are updated based on the mean square error between the cumulative target reward value and the evaluation value output by the Critic network. Based on the expected cumulative reward of the Actor network output action evaluated by the Critic network, the policy gradient method is used to update the weight parameters of the Actor network in order to maximize the expected cumulative reward of the Actor network output action. Repeat the training loop and weight adjustment steps until the preset training rounds are reached or the average cumulative reward value obtained by the agent in the environment simulator converges. Save the weight parameters of the trained Actor network. Use the saved Actor network as the optimal control strategy network under the target operating condition mode, and use it to directly output the optimal DO airflow adjustment cycle and airflow adjustment step based on the input state vector.

[0012] Preferably, the weight parameters of the Critic network are updated based on the mean squared error between the target cumulative reward value and the evaluation value output by the Critic network; the weight parameters of the Actor network are updated using a policy gradient method based on the expected cumulative reward of the Actor network's output action evaluated by the Critic network, including: For each empirical sample in the sampling batch, the predicted value of the state vector at the next time step is input into the Actor network to obtain the action vector at the next time step, and the state vector and action vector at the next time step are input into the Critic network to obtain the evaluation value of the state-action pair at the next time step. Based on the instant reward value, the evaluation value of the next state-action pair, and the preset discount factor, calculate the target cumulative reward value of the current state-action pair; The loss function is to minimize the mean squared error between the current state action pair evaluation value and the target cumulative reward value output by the Critic network. A gradient-based optimization algorithm is used to backpropagate and update the weight parameters of the Critic network. After updating the weight parameters of the Critic network, the current state vector is input into the Actor network to obtain the mean and standard deviation of the current action vector; The probability distribution of actions is constructed based on the mean and standard deviation. The current action vector is randomly sampled from the probability distribution. The current state vector and the current action vector are input into the Critic network to obtain the corresponding evaluation value. The expected value of the evaluation value corresponding to the sampled action under the probability distribution is calculated. The policy gradient method is used to backpropagate and update the weight parameters of the Actor network with the goal of maximizing the expected value. When the preset update frequency of the Critic network and Actor network is reached, the network weight parameters of the corresponding batch of empirical samples are updated.

[0013] Preferably, the real-time data acquisition and construction of the current multidimensional feature vector, inputting the current multidimensional feature vector into the working condition classifier, and calculating the membership probability distribution of the current working condition belonging to several working condition modes, includes: During online operation, the operation data of the aerobic section of the biological treatment tank of the sewage treatment plant is collected in real time according to the preset sampling cycle. The operation data includes at least the current DO concentration value, the current air volume value, and the current water temperature value. Construct a current multidimensional feature vector based on the operating data, input the current multidimensional feature vector into the working condition classifier, and calculate the posterior probability value of the current multidimensional feature vector belonging to each working condition mode through the working condition classifier. The calculated posterior probability values ​​are combined to form the membership probability distribution, in which the sum of all elements equals 1, and each element corresponds to the probability value of belonging to the corresponding working condition mode.

[0014] Secondly, this application discloses an airflow regulating device based on wastewater treatment aeration, which adopts the following technical solution, including: The operating condition classification module is used to receive historical DO concentration, air volume, influent load and time data, construct a multi-dimensional feature vector representing the operating condition, and use an unsupervised clustering algorithm to cluster the multi-dimensional feature vector, divide the historical operating conditions into several operating condition modes, and generate an operating condition classifier that can output the operating condition membership probability in real time. The time prediction module is used to train a data-driven time series prediction model for each operating mode using its corresponding historical data subset. The input of the time series prediction model is a state vector containing DO deviation, DO change rate and influent load and a two-dimensional action vector consisting of DO regulation air volume cycle and air volume regulation step. The output is the next state prediction value and an instant reward value consisting of water quality compliance reward, energy consumption penalty and stability penalty. The model training module is used to construct a deep reinforcement learning agent containing an Actor network and a Critic network for each working condition mode. The agent is trained offline in the corresponding time series prediction model with the goal of maximizing the cumulative reward, so that the Actor network of the agent can serve as the optimal control policy network for the corresponding working condition mode. The probability distribution module is used to collect data in real time and construct the current multidimensional feature vector. The current multidimensional feature vector is input into the working condition classifier to calculate the membership probability distribution of the current working condition belonging to several working condition modes. The air volume regulation module is used to input the current state vector into each of the optimal control strategy networks to obtain several sets of candidate DO regulation air volume cycles and air volume regulation steps. The membership probability distribution is used to weight and sum the several sets of candidate values ​​to obtain the DO regulation air volume cycle and air volume regulation step.

[0015] Thirdly, this application also provides a control device, the device comprising: It includes a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed, such as the above-described airflow regulation method based on wastewater treatment aeration.

[0016] Fourthly, this application also provides a computer-readable storage medium storing a computer program that can be loaded by a processor and executed as described above regarding the airflow regulation method based on wastewater treatment aeration.

[0017] In summary, this application systematically acquires historical operating data of the aerobic section of the biological treatment tank in a wastewater treatment plant. Through feature engineering, it constructs multi-dimensional feature vectors representing operating conditions. Unsupervised clustering algorithms, such as Gaussian mixture models, are used to classify historical operating conditions into several implicit operating condition modes, generating a condition classifier capable of outputting the probability distribution of operating condition membership in real time. For each operating condition mode, a time-series prediction model based on a long short-term memory neural network is trained using a subset of its corresponding historical data as an environmental simulator. This simulator takes as input a state vector containing DO deviation, DO change rate, and influent load, and a two-dimensional action vector consisting of the DO regulation airflow cycle and airflow regulation step size. It outputs the predicted state value for the next moment and an instantaneous reward value composed of water quality compliance reward, energy consumption penalty, and stability penalty. A deep reinforcement learning agent containing an Actor network and a Critic network is constructed for each operating condition mode. This agent is trained offline in the corresponding environmental simulator with the objective of maximizing the cumulative reward, making the Actor network the optimal control strategy network for that mode. During online operation, data is collected in real time and a current multi-dimensional feature vector is constructed. This vector is then input into a condition classifier to calculate the membership probability distribution of the current condition belonging to each mode. The current state vector is then input into the optimal control strategy network for each mode to obtain multiple sets of candidate DO (Displacement Oxide) regulation airflow cycles and airflow regulation steps. The candidate values ​​are weighted and summed using the membership probability distribution to output the final executed DO regulation airflow cycle and airflow regulation step. This adaptive identification of operating conditions reduces reliance on manual parameter setting and allows for dynamic adjustment of control parameters based on different operating conditions. This effectively solves the problems of control lag, DO oscillation, high energy consumption, and poor adaptability in traditional fixed-parameter control schemes. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of a method for regulating air volume in wastewater treatment aeration.

[0019] Figure 2 This is a structural block diagram of an airflow regulating device based on wastewater treatment aeration. Detailed Implementation

[0020] The following combination Figures 1-2 This application will be described in further detail.

[0021] This application addresses the technical shortcomings of existing aeration control methods in the aerobic section of biological treatment tanks for wastewater treatment. These methods involve fixed airflow adjustment steps and DO adjustment airflow cycles that rely on manual experience, leading to delayed dissolved oxygen regulation, aeration oscillations, and an inability to balance energy consumption and water quality. The application provides an airflow adjustment method based on wastewater treatment aeration. Through three core stages—data processing, algorithm modeling, and testing and verification—this method achieves adaptive optimization of core control parameters, resolving the disconnect between traditional fuzzy control and the actual operating conditions of the biological treatment tank and DO response characteristics, thus enabling intelligent operation of the aeration system.

[0022] The technical solution of this application is applied to the aeration control system of the aerobic section of the biological treatment tank in a wastewater treatment plant. The system includes at least an online DO monitoring instrument, a blower air volume monitoring module, an aeration valve opening monitoring module, a data acquisition and storage unit, and an intelligent control unit. The data acquisition and storage unit is used to collect and store historical DO concentration data, blower air volume data, aeration valve opening data, and running time data. The intelligent control unit is used to execute the data processing, algorithm modeling, and parameter output logic of this invention.

[0023] Reference Figure 1 The embodiments of this application include at least steps S10 to S50.

[0024] S10 receives historical DO concentration, air volume, influent load and time data, constructs a multi-dimensional feature vector representing the operating conditions, uses an unsupervised clustering algorithm to cluster the multi-dimensional feature vector, divides the historical operating conditions into several operating condition modes, and generates an operating condition classifier that can output the operating condition membership probability in real time.

[0025] S20: For each operating mode, a data-driven time series prediction model is trained using a subset of its corresponding historical data. The input of the time series prediction model is a state vector containing DO deviation, DO change rate and influent load, and a two-dimensional action vector consisting of DO regulation air volume cycle and air volume regulation step. The output is the next state prediction value and an instant reward value consisting of water quality compliance reward, energy consumption penalty and stability penalty.

[0026] S30 constructs a deep reinforcement learning agent containing an Actor network and a Critic network for each operating condition mode. It is then trained offline in the corresponding time series prediction model with the goal of maximizing the cumulative reward, so that the Actor network of the agent serves as the optimal control policy network for the corresponding operating condition mode.

[0027] S40 collects data in real time and constructs the current multidimensional feature vector. It then inputs the current multidimensional feature vector into the working condition classifier to calculate the membership probability distribution of the current working condition belonging to several working condition modes.

[0028] S50: Input the current state vector into each optimal control strategy network to obtain several sets of candidate DO regulating air volume cycles and air volume regulating steps. Use the membership probability distribution to weight and sum the several sets of candidate values ​​to obtain the DO regulating air volume cycle and air volume regulating steps.

[0029] Specifically, a multi-dimensional feature vector is constructed based on historical DO concentration, air volume, influent load, and time data. An unsupervised clustering algorithm is used to classify historical operating conditions into several typical modes and generate an operating condition classifier. For each operating condition mode, a data-driven time series prediction model is trained as a simulation environment to simulate state transitions and output a comprehensive reward signal including water quality, energy consumption, and stability factors. Based on this, a deep reinforcement learning agent is trained for each operating condition mode. The optimal control strategy for each mode is obtained through the collaborative optimization of the Actor network and the Critic network. During online operation, the operating condition classifier is used to calculate the membership probability of the current operating condition to each mode in real time. The candidate parameters output by each strategy network are probabilistically weighted and fused to obtain the DO regulation air volume cycle and air volume regulation step size adapted to the current operating condition. This achieves adaptive matching of control parameters with multi-mode operating conditions, effectively solving the problems of lag, poor adaptability, and difficulty in balancing energy consumption and water quality in traditional fixed parameter schemes.

[0030] In some embodiments, before constructing the multidimensional feature vector, the original historical data collected is first preprocessed. The corresponding processing steps are as follows: Curve smoothing: To address the issue that the original DO concentration data is subject to fluctuations and noise, making it impossible to directly represent the true trend of DO concentration changes, a rolling moving average function is used to smooth the original DO concentration time series curve. This eliminates the interference of instantaneous noise and random fluctuations on the curve trend, enabling the processed DO concentration curve to truly and continuously represent the overall trend of DO concentration changes over time and air volume.

[0031] Taking window size n as an example, the smooth calculation rule is as follows: For the i-th data point ( (where n is the preset window size), its smoothing value is calculated using the following formula: ; in, Let be the smoothed DO concentration value for the i-th data point. Let j be the j-th original DO concentration value, and n be the size of the sliding window. This smoothing process can eliminate the interference of transient noise and random fluctuations on the curve trend.

[0032] Data cleaning: Perform data cleaning operations on the smoothed historical operating data. Specific cleaning rules include: removing null and missing values ​​in the dataset to ensure the continuity of time series data; removing abnormal data that exceeds the normal operating boundary of the biochemical tank, including extreme values ​​of DO concentration, extreme values ​​of blower air volume, extreme values ​​of valve opening, and other abnormal samples that deviate from the normal operating range.

[0033] Offset Time Determination: A systematic analysis is performed on the response delay characteristics between changes in DO concentration and changes in blower airflow to determine the uniform offset time of DO concentration changes relative to airflow changes. The specific analysis logic is as follows: Locate the moment in historical data when blower airflow changes; synchronously match the smoothed DO concentration curve; identify the starting moment when the DO concentration begins to respond in the corresponding direction after the airflow change; calculate the time difference between the two moments as the single delay time; iterate through all valid airflow change samples, and summarize the uniform delay time by statistically averaging or using the median, providing a time benchmark for subsequent modeling and analysis.

[0034] The core objective of this systematic analysis of the response delay characteristics between changes in DO concentration and changes in blower airflow is to determine the uniform offset (delay) time of DO concentration changes relative to airflow changes over a complete historical operating period of the wastewater treatment plant. This time benchmark will provide crucial time information for subsequent algorithm modeling, enabling the prediction of DO concentration change trends based on airflow changes and precise control of the aeration system. This will ensure that control actions are synchronized with the DO concentration response rhythm, thereby improving control accuracy.

[0035] The specific analysis logic and execution process are as follows: First, based on all the valid air volume change point samples (including rising and falling change points) identified and screened in the subsequent steps, the specific time of each blower air volume change is accurately located in the historical time series data - that is, the timestamp corresponding to each valid change point. This time is used as the starting reference point of the air volume change. It is necessary to ensure that the timestamp is accurate and corresponds one-to-one with the air volume change point type (rising / falling) at the corresponding time to avoid the time delay calculation deviation caused by time series misalignment. At the same time, abnormal timestamp data caused by data acquisition failure is removed.

[0036] Subsequently, the DO concentration time-series curve, smoothed by the rolling function as described above, is retrieved synchronously. This curve effectively eliminates instantaneous noise and random fluctuations, accurately and continuously representing the trend of DO concentration changes over time. It can precisely capture the time node when the DO concentration exhibits a substantial response, avoiding misjudgment of the response time due to minor fluctuations. For each located airflow change moment, combined with the corresponding airflow change point type (increasing or decreasing), the time node when the DO concentration begins to respond in the corresponding direction is accurately identified in the smoothed DO concentration curve: If it is an increasing airflow change point, the focus is on observing the first time node after that moment when the DO concentration changes from a stable state to a continuous upward trend, strictly excluding meaningless minor fluctuations in DO concentration, and only recognizing the starting moment of a clear and continuous upward trend as a valid response node; if it is a decreasing airflow change point, the corresponding first time node after that moment when the DO concentration changes from a stable state to a continuous downward trend is identified, similarly eliminating irrelevant minor fluctuations, ensuring the authenticity and validity of the response node, and guaranteeing the accuracy of the single delay time calculation.

[0037] After determining the time of airflow change and the corresponding DO concentration response time, calculate the time difference between the two time points. This difference is the DO concentration response delay time corresponding to a single airflow change. During the calculation, the time unit must be standardized (it can be flexibly adjusted to the second level according to the water plant's data acquisition frequency) to ensure that the calculation standard for all single delay times is consistent and to avoid statistical deviations caused by inconsistent units. For example, if the airflow changes at 00:05:00 and the DO concentration begins to show a continuous increase at 00:08:00, then the delay time is 3 minutes.

[0038] After calculating the single delay time, all valid airflow change samples are iterated through, and the DO response delay time for each sample is calculated to form a complete single delay time dataset. Based on this, a systematic statistical and inductive analysis is performed on this dataset: First, abnormal delay values ​​(such as extremely large or small delay times caused by equipment failure, sudden water quality changes, or unexpected operating condition adjustments) are removed from the dataset to avoid outliers affecting the accuracy of the uniform delay time. Then, through scientific statistical analysis methods (such as calculating the average and median), combined with the actual operating conditions and process stability of the water plant's historical operation, a uniform delay time for DO concentration changes relative to airflow changes is determined within the complete historical operating period of the plant. If the delay time distribution in the statistical results is relatively concentrated without significant abnormal fluctuations, the average value can be used as the uniform delay time; if there are small fluctuations, the median or the delay time with the highest frequency can be selected as the uniform standard, taking into account operating condition stability, to ensure that the delay time can comprehensively and objectively reflect the general response law of the plant's DO concentration to airflow changes, providing a reliable time benchmark for subsequent algorithm modeling.

[0039] Valve and air volume data targeted filtering: Blower air volume data and aeration valve opening data are targeted according to preset operating condition rules. When the system has an aeration valve opening feedback signal, the operating data under six conditions are selected: valve opening and blower air volume increase simultaneously; valve opening and blower air volume decrease simultaneously; valve opening increases while blower air volume remains unchanged; valve opening decreases while blower air volume remains unchanged; blower air volume increases while valve opening remains unchanged; and blower air volume decreases while valve opening remains unchanged. When the system does not have an aeration valve opening feedback signal, only valid operating data where the blower air volume changes are selected.

[0040] Change point identification: The range of gas-water ratio variation is used as a threshold to identify valid gas volume change points in historical data. The absolute value of the difference between gas volume data at two adjacent moments in the time series is calculated. When this absolute value exceeds a preset gas-water ratio change threshold, that moment is considered a valid change point. Change points with increasing gas volume are marked as rising change points, and those with decreasing gas volume are marked as falling change points. The preferred gas-water ratio change threshold is 2.5%, but it can be adaptively adjusted according to different water plant operating conditions.

[0041] In some embodiments, step S10 specifically includes the following steps: preprocessing and feature extraction of historical DO concentration, air volume, influent load, and time data to construct a multidimensional feature vector corresponding to each historical moment. The multidimensional feature vector includes at least: the deviation between the current DO concentration and the set value, the rate of change of DO concentration within the past preset time window, water temperature, and time context encoding features; inputting the multidimensional feature vectors of all historical moments into a Gaussian mixture model for cluster analysis, estimating the mean, covariance, and weight parameters of each Gaussian component through expectation-maximization algorithm iteration until the model converges; determining the optimal number of clusters K using the Bayesian information criterion to divide the historical operating conditions into K operating condition modes, where K is an integer greater than or equal to 2; constructing an operating condition classifier based on the trained Gaussian mixture model. The input of the operating condition classifier is the current multidimensional feature vector collected and constructed in real time, and the output is the posterior probability value of the current operating condition belonging to each of the K operating condition modes, forming a membership probability distribution. The posterior probability value is calculated by Bayes' formula from the probability density function of each Gaussian component in the Gaussian mixture model and the prior weights.

[0042] In practice, the Gaussian mixture model assumes that all samples are generated by a mixture of K Gaussian components, and its probability density function is expressed as: ; in, Let F be the probability density value of the multidimensional eigenvector F, and K be the total number of Gaussian components. The mixing weights for the k-th Gaussian component. The mean is The covariance matrix is The probability density function of the k-th multivariate Gaussian distribution is obtained. The mean vector, covariance matrix, and mixing weights of each Gaussian component are estimated iteratively using the expectation-maximization algorithm until the log-likelihood function of the model converges.

[0043] The optimal number of clusters K is determined using the Bayesian information criterion, and the Bayesian information criterion value is... Calculate using the following formula: ; in, This represents the maximum log-likelihood value of the Gaussian mixture model for that cluster number. Let N be the number of free parameters in the model, and N be the total number of samples in the multidimensional feature vector. Select a model that... Maximum value or The K value corresponding to the inflection point of the curve as a function of K is taken as the optimal number of clusters. A condition classifier is constructed based on the trained Gaussian mixture model. For the current multidimensional feature vector input online... The posterior probability value of belonging to the kth working condition mode is calculated by the following formula: ; in, For the current multidimensional feature vector The posterior probability value belonging to the kth working condition mode. and These are the mixing weights for the k-th and j-th Gaussian components, respectively. This is the probability density function value of the current feature vector under the k-th Gaussian distribution, with the denominator being the sum of the terms corresponding to the K Gaussian components.

[0044] Specifically, historical DO concentration, air volume, influent load, and time data are preprocessed and feature-engineered to construct a multi-dimensional feature vector containing DO deviation, DO change rate, influent load, sludge concentration, water temperature, and time encoding. The feature vectors from all historical moments are input into a Gaussian mixture model. The expectation-maximization algorithm is used to iteratively estimate the mean, covariance, and mixing weights of each Gaussian component until convergence. The optimal number of clusters K is determined using the Bayesian information criterion, classifying historical operating conditions into K typical patterns. Based on this, an operating condition classifier is constructed. During online operation, the current multi-dimensional feature vector is input into the classifier. Based on the probability density function of each Gaussian component and the prior weights, the posterior probability distribution of the current operating condition belonging to each pattern is calculated using the Bayesian formula, providing a basis for subsequent multi-strategy weighted fusion. This avoids the problem of rigid cluster boundaries, making operating condition identification more consistent with the continuous and gradual changes in the wastewater treatment process.

[0045] Furthermore, to achieve automatic optimization of the number of clusters and division of working conditions in the Gaussian mixture model, a combination of the expectation-maximization algorithm and the Bayesian information criterion is chosen. The corresponding processing steps are as follows: Set an initial range for the number of clusters, with a lower limit of no less than 2 and an upper limit of a preset maximum number of patterns; for each candidate cluster value within the range, randomly initialize the mean vector, covariance matrix, and mixing weights of Gaussian components equal to the target candidate cluster value, with the sum of each mixing weight equal to 1; input the multidimensional feature vector set of all historical time points into the Gaussian mixture model, and iteratively execute the expectation step and the maximization step: in the expectation step, calculate the posterior probability value of each feature vector belonging to each Gaussian component based on the current model parameters; in the maximization step, update the mean vector, covariance matrix, and mixing weights of each Gaussian component using the calculated posterior probability values; repeat the expectation step and the maximization step until the log-likelihood function value of the model converges or reaches the preset maximum number of iterations, thus obtaining the Gaussian mixture model trained under the target candidate cluster values. For each candidate cluster value, the trained Gaussian mixture model is calculated, and the corresponding Bayesian information criterion value is positively correlated with the maximum log-likelihood value of the model. The Bayesian information criterion values ​​corresponding to each candidate cluster value are compared, and the candidate cluster value corresponding to the maximum Bayesian information criterion value or the inflection point of the curve of the Bayesian information criterion value changing with the candidate cluster value is selected as the optimal number of clusters. The Gaussian mixture model corresponding to the optimal number of clusters is taken as the final clustering model. Based on the maximum posterior probability of each feature vector output by the final clustering model belonging to each Gaussian component, the working condition mode is divided.

[0046] Specifically, within a preset range of candidate cluster numbers, a Gaussian mixture model with a corresponding number of Gaussian components is constructed for each candidate value. The mean, covariance, and mixture weights of each component are estimated through alternating expectation and maximization steps until the log-likelihood function converges, obtaining the optimal model parameters for that cluster number. Subsequently, the Bayesian information criterion value for each model is calculated. This criterion comprehensively balances model fitting accuracy and parameter complexity, selecting the cluster number corresponding to the maximum Bayesian information criterion value or the inflection point as the optimal cluster number, avoiding subjective bias from pre-setting. Finally, the attribution of the working condition mode at each historical moment is determined based on the maximum posterior probability output by the optimal model, ensuring that the classification results statistically possess both goodness of fit and generalization ability.

[0047] In some embodiments, step S20 specifically includes the following steps: For each operating mode, extract the historical data subset corresponding to the target operating mode. The historical data subset contains the state vector sequence and action vector sequence of all historical moments belonging to the target operating mode label. The state vector sequence consists of at least DO concentration deviation, DO concentration change rate and water temperature. The action vector sequence consists of DO regulating air volume cycle and air volume regulating step size. Construct the historical data subset into a supervised learning sample set. The input feature of each sample is the combination of the state vector and action vector at the current moment. The label of the sample is the state vector at the next moment. The time interval between the next moment and the current moment of the state vector is equal to the DO regulating air volume cycle at the current moment. A Long Short-Term Memory (LSTM) neural network is used as the network structure for the time series prediction model. The input layer of the LSTM neural network receives the concatenated vector of the current state vector and action vector. After passing through at least one LSTM hidden layer, temporal dependency features are extracted, and the predicted value of the state vector at the next time step is output through a fully connected output layer. The training objective is to minimize the mean squared error between the predicted and true values ​​of the state vector. A gradient-based optimization algorithm is used to iteratively update the weight parameters of the LSTM neural network until the model converges or reaches a preset number of training rounds, resulting in a time series prediction model trained under the target operating condition. The trained time series prediction model is used as an environmental simulator corresponding to the target operating condition. Simultaneously, based on a preset multi-objective reward function, a water quality compliance reward is calculated based on the deviation between the predicted value of the state vector and the DO setting value, an energy consumption penalty is calculated based on the airflow adjustment step size in the action vector, and a stability penalty is calculated based on the change amplitude between two adjacent action vectors. The three terms are weighted and summed to obtain the immediate reward value.

[0048] Specifically, the system extracts a subset of historical data corresponding to the pattern and constructs a supervised learning sample set with the current state vector and action vector as input and the next state vector as the label. A long short-term memory neural network is used to capture the temporal dependencies between variables such as DO concentration and influent load. The network weights are trained until convergence by minimizing the mean square error between the predicted and actual state values. After training, the network acts as an environmental simulator, receiving states and actions and outputting the next state prediction. Simultaneously, based on a preset multi-objective reward function, it calculates an instantaneous reward value by weighting three indicators: water quality compliance deviation, airflow step energy consumption, and action change stability. This provides a simulation interaction environment and optimization guidance for the subsequent offline training of the deep reinforcement learning agent.

[0049] In some embodiments, a deep reinforcement learning agent based on the Actor-Critic architecture is constructed for each operating mode. The optimal control strategy for that mode is obtained through offline interactive training with an environmental simulator. The corresponding processing steps are as follows: For each operating mode, a deep reinforcement learning agent is constructed. The deep reinforcement learning agent includes an Actor network and a Critic network. The input of the Actor network is the current state vector, and the output is the mean and standard deviation of the two-dimensional action vector composed of the DO airflow adjustment cycle and airflow adjustment step. The final action vector is generated by random sampling based on the mean and standard deviation. The input of the Critic network is the concatenation vector of the current state vector and the action vector generated by the Actor network, and the output is the expected cumulative reward evaluation value of the state-action pair. An offline training loop is constructed. In each training round, a batch of initial state vectors is randomly sampled from the historical data subset corresponding to the target working condition mode. The initial state vectors are input into the Actor network to obtain action vectors. The initial state vectors and action vectors are input into the corresponding environment simulator after training to obtain the predicted value of the state vector and the instant reward value of the next moment. The current state vector, action vector, instant reward value, next moment state vector and interaction termination flag generated by each interaction are combined into an experience sample and stored in the experience playback pool. The interaction termination flag is set to true when the DO concentration deviation continuously exceeds the preset threshold range or reaches the preset maximum number of interaction steps in a single round, and the interaction terminates. Once the number of experience samples stored in the experience replay pool reaches the preset minimum sampling batch requirement, a batch of experience samples is randomly sampled from the experience replay pool, and the target cumulative reward value for each sample is calculated. The weight parameters of the Critic network are updated based on the mean square error between the target cumulative reward value and the evaluation value output by the Critic network. Based on the expected cumulative reward of the Actor network output action evaluated by the Critic network, the weight parameters of the Actor network are updated using the policy gradient method to maximize the expected cumulative reward of the Actor network output action. The training loop and weight adjustment steps are repeated until the preset number of training rounds or the average cumulative reward value obtained by the agent in the environment simulator converges, and the weight parameters of the trained Actor network are saved. The saved Actor network is used as the optimal control policy network under the target operating condition mode, which is used to directly output the optimal DO regulating air volume cycle and air volume regulating step based on the input state vector.

[0050] Specifically, the Actor network in the agent receives the state vector and outputs the mean and standard deviation of the actions. It then generates an action vector composed of the DO adjustment period and stride through random sampling. The Critic network evaluates the expected cumulative reward of the state-action pair. During training, initial states are sampled from a subset of historical data, continuously interacting with the environment simulator and storing experience samples in a replay pool. Interaction terminates when the DO deviation exceeds the limit or the maximum number of steps is reached. Once the experience pool meets the sampling requirements, the Critic network is updated by minimizing the mean square error between the target cumulative reward and the Critic evaluation value. Simultaneously, the policy gradient method is used to maximize the expected reward of the Actor's output action to update the Actor network. This process is repeated until convergence, and the saved Actor network serves as the optimal control policy network for this mode.

[0051] Furthermore, the weight update of the deep reinforcement learning agent is achieved through alternating optimization of the Actor network and the Critic network. The corresponding processing steps are as follows: For each experience sample in the sampling batch, the predicted value of the state vector at the next time step is input into the Actor network to obtain the action vector at the next time step, and the state vector and the action vector at the next time step are input into the Critic network to obtain the evaluation value of the state-action pair at the next time step; Based on the immediate reward value, the evaluation value of the state-action pair at the next time step, and the preset discount factor, the target cumulative reward value of the current state-action pair is calculated; Using the mean square error between the evaluation value of the current state-action pair output by the Critic network and the target cumulative reward value as the loss function, a gradient-based optimization algorithm is used to backpropagate and update the weight parameters of the Critic network. After updating the weight parameters of the Critic network, the current state vector is input into the Actor network to obtain the mean and standard deviation of the current action vector. Based on the mean and standard deviation, a probability distribution of the action is constructed. The current action vector is randomly sampled from the probability distribution. The current state vector and the current action vector are input into the Critic network to obtain the corresponding evaluation value. The expected value of the evaluation value corresponding to the sampled action under the probability distribution is calculated. With maximizing this expected value as the optimization objective, the weight parameters of the Actor network are updated by backpropagation using the policy gradient method. When the preset update frequency of the Critic network and Actor network is reached, the network weight parameters under the corresponding batch of empirical samples are updated.

[0052] In practice, for each sample, the target cumulative reward value Calculate using the following formula: ; in, For instant reward value, Discount factor and , This is an interaction termination indicator. For the target Critic network, For the target Actor network, This is the predicted state vector value for the next time step output by the environment simulator.

[0053] The loss function of the Critic network is defined as the mean squared error between the cumulative reward value of the objective and the current evaluation value: ; in, Here, B represents the loss function value of the Critic network, and B represents the batch size. Let i be the target cumulative reward value for the i-th sample. Let be the evaluation value of the Critic network for the i-th sample.

[0054] The Actor network is updated using a policy gradient method, with the optimization objective being to maximize the expected cumulative reward of the Critic network evaluation. ; in, Let the objective function be the optimization objective function of the Actor network. This indicates that the state vector is sampled from the experience replay pool. This indicates that the action vector is sampled from the probability distribution defined by the Actor network.

[0055] Specifically, for each empirical sample in the sampling batch, the predicted state value for the next time step output by the environmental simulator is first input into the Actor network to obtain the next action vector. The Critic network is then used to calculate the evaluation value of the state-action pair for the next time step, and this evaluation is weighted with the immediate reward value and a discount factor to obtain the target cumulative reward value for the current state-action pair. Subsequently, the Critic weights are updated using the loss function of minimizing the mean square error between the Critic network's evaluation value and the target value. Based on this, the current state is input into the Actor network to obtain the action mean and standard deviation, and a probability distribution is constructed. After sampling, the Critic network evaluates the expected cumulative reward of the action, and the policy gradient method is used to maximize this expectation to update the Actor weights. This alternating update of the Critic and Actor networks is executed cyclically until a preset frequency is reached, enabling the agent to gradually learn the optimal control strategy that balances water quality compliance with energy conservation and emission reduction.

[0056] In some embodiments, as an alternative implementation, this application also provides a method for determining dynamic parameters based on statistical analysis of historical data. By constructing... Zone and DO change rate The historical mapping relationship database was used, and the control parameters were determined by a statistical method that selected the mode percentage and quantiles.

[0057] Specifically, this includes: invoking a pre-determined DO (Displacement Detector) to adjust the airflow cycle. Based on the moment of gas volume change, the calculation is performed at... Changes in DO concentration over a time interval All ΔDO' calculated from historical data are partitioned continuously according to their numerical values, and the historical DO change rate is calculated simultaneously. ,Establish Zone and DO change rate A historical mapping database; based on the actual required DO change rate r, match the closest historical value in the mapping database. Sample, retrieve the corresponding history in reverse Numerical values; using the mode percentage as the filtering criterion, filter out the values ​​corresponding to that percentage. The dataset is used to define the reasonable range of the target aeration change step size; the dynamic airflow adjustment step size is output through histogram statistics and the 80th percentile is used as the final selection criterion. This alternative solution does not require deep reinforcement learning training, is more lightweight in deployment, and is suitable for application scenarios with limited historical data or limited computing resources.

[0058] In some embodiments, step S40 specifically includes the following steps: During online operation, real-time data of the aerobic section of the biological treatment tank in the wastewater treatment plant is collected according to a preset sampling period. The data includes at least the current DO concentration, current air volume, and current water temperature. Based on the data, a current multidimensional feature vector is constructed. The current multidimensional feature vector is input into a condition classifier. The condition classifier calculates the posterior probability value of the current multidimensional feature vector belonging to each condition mode. The calculated posterior probability values ​​are combined to form a membership probability distribution. The sum of all elements in the membership probability distribution is equal to 1, and each element corresponds to the probability value of belonging to the corresponding condition mode.

[0059] Specifically, operational data such as DO concentration, air volume, and water temperature of the biological treatment tank are collected in real time according to a preset sampling period. A current multidimensional feature vector is constructed using the same feature engineering method as in the offline training phase. This multidimensional feature vector is then input into a pre-trained Gaussian mixture model classifier. The probability density function value of the feature vector under each Gaussian component of each operating condition is calculated. After normalization using Bayes' theorem with the prior weights of each component, the posterior probability value of the current operating condition belonging to each mode is output, forming a membership probability distribution vector whose sum is 1. This allows the operating condition identification results to smoothly reflect the continuous and gradual operational characteristics of the wastewater treatment process.

[0060] In some embodiments, step S50 specifically includes the following steps: obtaining the state vector at the current moment, the composition of which is completely consistent with that of the offline training phase, including at least: the deviation between the current DO concentration and the set value, the rate of change of DO concentration within the past preset time window, and the current water temperature value; inputting the state vector at the current moment into the K optimal control strategy networks that have been trained, each optimal control strategy network corresponding to a historical operating mode; after receiving the state vector, the kth optimal control strategy network calculates a set of candidate action vectors through the forward propagation of its Actor network, the candidate action vectors containing a candidate DO adjustment airflow cycle and a candidate airflow adjustment step, k=1,2,...,K; A total of K candidate DO regulating airflow cycles and K candidate airflow regulating steps were obtained, denoted as follows: and Obtain the working condition membership probability distribution output by the working condition classifier. ,in Let represent the probability that the current operating condition belongs to the k-th operating condition mode, satisfying the following formula: ; The K groups of candidate values ​​are weighted and fused using the operating condition membership probability distribution vector to calculate the final DO adjustment air volume cycle. and the final airflow adjustment step The weighted calculation formula is as follows: ; ; The final DO regulating air volume cycle obtained by weighted calculation and final air volume adjustment step The output is sent to the aeration control system's actuator, which then executes the output according to... The airflow is adjusted periodically at predetermined time intervals, with each adjustment involving a specific amount of airflow. .

[0061] In this embodiment, the aforementioned weighted fusion mechanism enables soft switching of control strategies under different operating conditions. Compared with the traditional hard switching method that directly switches a single strategy based on the operating condition identification result, the probability-weighted soft switching can effectively reduce abrupt changes in control parameters caused by misjudgment of operating conditions or abrupt changes in identification results at the operating condition boundary. This makes the output changes of the DO regulating air volume cycle and air volume regulating step more smooth and continuous, thereby reducing air volume fluctuations and DO concentration oscillations in the aeration system and improving the stability of system operation.

[0062] In practical implementation, reasonable working range constraints can be set for the weighted and fused parameters. For example, the lower limit for the calculated DO regulation airflow cycle can be set to be no less than 2 minutes, and the upper limit no more than 15 minutes, to prevent DO oscillation caused by an excessively short cycle or control lag caused by an excessively long cycle. Similarly, the lower limit for the airflow regulation step size can be set to be no less than the blower's minimum regulation resolution, and the upper limit no more than a preset ratio of the blower's rated flow rate, to prevent ineffective regulation due to an excessively small step size or equipment impact due to an excessively large step size. These constraints can be adapted and adjusted according to the specific equipment specifications and process requirements of the wastewater treatment plant.

[0063] In addition, during actual deployment, after the DO regulation air volume cycle and air volume regulation step obtained by weighted fusion calculation are output to the aeration control system, the system collects the current DO concentration value at regular intervals according to the cycle, compares it with the process set value, and executes the regulation action of increasing or decreasing air volume according to the comparison result and the step.

[0064] In some embodiments, considering the issues of system testing, verification, and iterative optimization, in order to deploy the above solution to the aeration control system of the biological treatment tank of the target wastewater treatment plant for on-site verification and continuous optimization, the corresponding processing steps are as follows: The trained operating condition classifier, environmental simulator, and optimal control strategy network are connected to the real-time data from the on-site DO monitoring instruments, airflow monitoring module, and valve opening monitoring module. Following the logic of this method, the DO regulation airflow cycle and airflow adjustment step are output in real time. Compared with traditional fixed-step and fixed-cycle control modes, the effectiveness of this method is verified by monitoring DO concentration stability indicators, aeration oscillation frequency, effluent ammonia nitrogen and total nitrogen compliance rates, and aeration system energy consumption indicators. Based on the control deviations, new operating scenarios, seasonal changes, or process modifications that occur during on-site verification, the data processing rules, clustering model parameters, reinforcement learning training strategy, and fusion weight calculation method are iteratively updated to continuously optimize the model's adaptability and accuracy, ensuring that the control parameters always match the real-time operating conditions of the biological treatment tank.

[0065] Through the above testing, verification, and iterative optimization steps, the system can adapt to the needs of process upgrades, environmental standard improvements, and dynamic changes in operating conditions during the long-term operation of wastewater treatment plants, maintaining the advanced nature and applicability of the technology.

[0066] The implementation principle of the airflow regulation method based on aeration in wastewater treatment according to an embodiment of this application is as follows: The system acquires historical operating data of the aerobic section of the biological treatment tank in a wastewater treatment plant, constructs a multi-dimensional feature vector representing the operating conditions through feature engineering, and uses unsupervised clustering algorithms such as Gaussian mixture models to divide the historical operating conditions into several implicit operating condition modes, generating an operating condition classifier that can output the probability distribution of operating condition membership in real time. For each operating condition mode, a time series prediction model based on a long short-term memory neural network is trained as an environmental simulator using its corresponding historical data subset. This simulator takes a state vector containing DO deviation, DO change rate, and influent load, and a two-dimensional action vector composed of the DO regulation airflow cycle and airflow regulation step as input, and outputs the predicted state value at the next moment and an instantaneous reward value composed of water quality compliance reward, energy consumption penalty, and stability penalty. A deep reinforcement learning agent containing an Actor network and a Critic network is constructed for each operating condition mode, and offline training is performed in the corresponding environmental simulator with the goal of maximizing the cumulative reward, so that the Actor network serves as the optimal control strategy network for that mode. During online operation, data is collected in real time and a current multi-dimensional feature vector is constructed. This vector is then input into a condition classifier to calculate the membership probability distribution of the current condition belonging to each mode. The current state vector is then input into the optimal control strategy network for each mode to obtain multiple sets of candidate DO (Displacement Oxide) regulation airflow cycles and airflow regulation steps. The candidate values ​​are weighted and summed using the membership probability distribution to output the final executed DO regulation airflow cycle and airflow regulation step. This adaptive identification of operating conditions reduces reliance on manual parameter setting and allows for dynamic adjustment of control parameters based on different operating conditions. This effectively solves the problems of control lag, DO oscillation, high energy consumption, and poor adaptability in traditional fixed-parameter control schemes.

[0067] Figure 1 This is a schematic flowchart of an airflow regulation method for wastewater treatment aeration in one embodiment. It should be understood that, although... Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows; unless explicitly stated otherwise, there is no strict order requirement for the execution of these steps, and they can be executed in other orders; and Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0068] Based on the same technical concept, referring to Figure 2This application also provides an airflow regulating device based on wastewater treatment aeration, which adopts the following technical solution: the device includes: The operating condition classification module is used to receive historical DO concentration, air volume, influent load and time data, construct a multi-dimensional feature vector representing the operating condition, and use an unsupervised clustering algorithm to cluster the multi-dimensional feature vector, divide the historical operating conditions into several operating condition modes, and generate an operating condition classifier that can output the operating condition membership probability in real time. The time prediction module is used to train a data-driven time series prediction model for each operating mode using its corresponding historical data subset. The input of the time series prediction model is a state vector containing DO deviation, DO change rate and influent load and a two-dimensional action vector consisting of DO regulation air volume cycle and air volume regulation step. The output is the next state prediction value and an instant reward value consisting of water quality compliance reward, energy consumption penalty and stability penalty. The model training module is used to build a deep reinforcement learning agent containing an Actor network and a Critic network for each working condition mode. It is trained offline in the corresponding time series prediction model with the goal of maximizing the cumulative reward, so that the Actor network of the agent can serve as the optimal control policy network for the corresponding working condition mode. The probability distribution module is used to collect data in real time and construct the current multidimensional feature vector. The current multidimensional feature vector is input into the working condition classifier to calculate the membership probability distribution of the current working condition belonging to several working condition modes. The air volume regulation module is used to input the current state vector into each optimal control strategy network to obtain several sets of candidate DO regulation air volume cycles and air volume regulation steps. The membership probability distribution is used to weight and sum the several sets of candidate values ​​to obtain the DO regulation air volume cycle and air volume regulation steps.

[0069] In some embodiments, the operating condition classification module is specifically used to preprocess and extract features from historical DO concentration, air volume, influent load and time data, and construct a multi-dimensional feature vector corresponding to each historical moment. The multi-dimensional feature vector includes at least: the deviation between the current DO concentration and the set value, the rate of change of DO concentration within the past preset time window, water temperature and time context encoding features. The multidimensional feature vectors of all historical moments are input into the Gaussian mixture model for cluster analysis. The mean, covariance and weight parameters of each Gaussian component are estimated through the expectation-maximization algorithm until the model converges. The optimal number of clusters K is determined by using the Bayesian information criterion, and the historical operating conditions are divided into K operating condition modes, where K is an integer greater than or equal to 2; A working condition classifier is constructed based on the trained Gaussian mixture model. The input of the working condition classifier is the current multidimensional feature vector collected and constructed in real time, and the output is the posterior probability value of the current working condition belonging to each of the K working condition modes, forming a membership probability distribution. The posterior probability value is calculated by Bayes' theorem using the probability density function of each Gaussian component in the Gaussian mixture model and the prior weights.

[0070] In some embodiments, the working condition classification module is specifically used to set the initial cluster number range, wherein the lower limit of the initial cluster number range is not less than 2, and the upper limit of the initial cluster number range is a preset maximum number of patterns. For each candidate cluster value within the range, the mean vector, covariance matrix, and mixture weights of the Gaussian components are randomly initialized in an equal number to the target candidate cluster value, and the sum of the mixture weights is equal to 1. The multidimensional feature vector set of all historical moments is input into the Gaussian mixture model, and the expectation step and the maximization step are executed iteratively: in the expectation step, the posterior probability value of each feature vector belonging to each Gaussian component is calculated based on the current model parameters; in the maximization step, the mean vector, covariance matrix and mixture weight of each Gaussian component are updated using the calculated posterior probability value. Repeat the expectation step and the maximization step until the log-likelihood function value of the model converges or the preset maximum number of iterations is reached, to obtain the Gaussian mixture model trained under the target candidate clustering numerical values; For each candidate cluster numerical value, the trained Gaussian mixture model is calculated, and the corresponding Bayesian information criterion value is positively correlated with the maximum log-likelihood value of the model. Compare the Bayesian information criterion values ​​corresponding to each candidate cluster value, select the candidate cluster value that maximizes the Bayesian information criterion value or makes the curve of the Bayesian information criterion value changing with the candidate cluster value reach an inflection point as the optimal number of clusters, and take the Gaussian mixture model corresponding to the optimal number of clusters as the final clustering model. The working condition modes are divided based on the maximum posterior probability of each feature vector belonging to each Gaussian component output by the final clustering model.

[0071] In some embodiments, the time prediction module is specifically used to extract a subset of historical data corresponding to the target operating mode for each operating mode. The subset of historical data includes a state vector sequence and an action vector sequence of all historical moments belonging to the target operating mode label. The state vector sequence consists of at least DO concentration deviation, DO concentration change rate and water temperature, and the action vector sequence consists of DO regulating air volume cycle and air volume regulating step. A subset of historical data is constructed as a supervised learning sample set. The input features of each sample are a combination of the state vector and action vector at the current moment. The label of the sample is the state vector at the next moment. The time interval between the next moment and the current moment of the state vector is equal to the DO adjustment air volume cycle at the current moment. The Long Short-Term Memory Neural Network is used as the network structure of the time series prediction model. The input layer of the Long Short-Term Memory Neural Network receives the concatenated vector of the current state vector and action vector. The temporal dependency features are extracted through at least one Long Short-Term Memory hidden layer, and the predicted value of the state vector at the next time step is output through a fully connected output layer. With the goal of minimizing the mean square error between the predicted and true values ​​of the state vector, a gradient-based optimization algorithm is used to iteratively update the weight parameters of the long short-term memory neural network until the model converges or reaches the preset number of training rounds, thus obtaining a time series prediction model trained under the target working condition. The trained time series prediction model is used as the environmental simulator corresponding to the target operating mode. At the same time, based on the preset multi-objective reward function, the water quality compliance reward is calculated according to the deviation between the predicted value of the state vector and the DO setting value, the energy consumption penalty is calculated according to the air volume adjustment step in the action vector, and the stability penalty is calculated according to the change amplitude between two adjacent action vectors. The three terms are weighted and summed to obtain the instantaneous reward value.

[0072] In some embodiments, the model training module is specifically used to construct a deep reinforcement learning agent for each working condition mode. The deep reinforcement learning agent includes an Actor network and a Critic network. The input of the Actor network is the current state vector, and the output is the mean and standard deviation of the two-dimensional action vector composed of the DO airflow adjustment cycle and airflow adjustment step. Based on the mean and standard deviation, the final action vector is generated by random sampling. The input of the Critic network is the concatenation vector of the current state vector and the action vector generated by the Actor network, and the output is the expected cumulative reward evaluation value of the state-action pair. An offline training loop is constructed. In each training round, a batch of initial state vectors is randomly sampled from the historical data subset corresponding to the target working condition mode. The initial state vectors are input into the Actor network to obtain action vectors. The initial state vectors and action vectors are input into the corresponding environment simulator after training to obtain the predicted value of the state vector and the instant reward value at the next moment. Each interaction generates a current state vector, action vector, instant reward value, next moment state vector, and interaction termination flag, which are combined into an experience sample and stored in the experience replay pool. The interaction termination flag is set to true when the DO concentration deviation continuously exceeds the preset threshold range or reaches the preset maximum number of interaction steps in a single round, and the interaction terminates. Once the number of experience samples stored in the experience replay pool reaches the preset minimum sampling batch requirement, a batch of experience samples is randomly sampled from the experience replay pool, and the target cumulative reward value for each sample is calculated. The weight parameters of the Critic network are updated based on the mean square error between the cumulative target reward value and the evaluation value output by the Critic network. Based on the expected cumulative reward of the Actor network output action evaluated by the Critic network, the policy gradient method is used to update the weight parameters of the Actor network in order to maximize the expected cumulative reward of the Actor network output action. Repeat the training loop and weight adjustment steps until the preset training rounds are reached or the average cumulative reward value obtained by the agent in the environment simulator converges. Save the weight parameters of the trained Actor network. Use the saved Actor network as the optimal control strategy network under the target operating condition mode, and use it to directly output the optimal DO airflow adjustment cycle and airflow adjustment step based on the input state vector.

[0073] In some embodiments, the model training module is specifically used to input the predicted value of the state vector at the next time step into the Actor network to obtain the action vector at the next time step for each empirical sample in the sampling batch, and input the state vector at the next time step and the action vector at the next time step into the Critic network to obtain the evaluation value of the state-action pair at the next time step. Calculate the target cumulative reward value of the current state-action pair based on the immediate reward value, the evaluation value of the state-action pair at the next moment, and the preset discount factor; The loss function is to minimize the mean squared error between the current state action pair evaluation value and the target cumulative reward value output by the Critic network. A gradient-based optimization algorithm is used to backpropagate and update the weight parameters of the Critic network. After updating the weight parameters of the Critic network, the current state vector is input into the Actor network to obtain the mean and standard deviation of the current action vector; The probability distribution of actions is constructed based on the mean and standard deviation. The current action vector is randomly sampled from the probability distribution. The current state vector and the current action vector are input into the Critic network to obtain the corresponding evaluation value. The expected value of the evaluation value corresponding to the sampled action under the probability distribution is calculated. The policy gradient method is used to backpropagate and update the weight parameters of the Actor network with the goal of maximizing the expected value. When the preset update frequency of the Critic network and Actor network is reached, the network weight parameters of the corresponding batch of empirical samples are updated.

[0074] In some embodiments, the probability distribution module is specifically used to collect operational data of the aerobic section of the biological treatment tank in a wastewater treatment plant in real time according to a preset sampling period during online operation. The operational data includes at least the current DO concentration value, the current air volume value, and the current water temperature value. Construct a current multidimensional feature vector based on the operating data, input the current multidimensional feature vector into the working condition classifier, and calculate the posterior probability value of the current multidimensional feature vector belonging to each working condition mode through the working condition classifier. The calculated posterior probability values ​​are combined to form a membership probability distribution. The sum of all elements in the membership probability distribution is equal to 1, and each element corresponds to the probability value of belonging to the corresponding working condition mode.

[0075] This application also discloses a control device.

[0076] Specifically, the control device includes a memory and a processor, the memory storing a computer program that can be loaded by the processor and executed by the aforementioned airflow regulation method based on wastewater treatment aeration.

[0077] This application also discloses a computer-readable storage medium.

[0078] Specifically, the computer-readable storage medium stores a computer program that can be loaded by a processor and executed, such as the above-described airflow regulation method based on wastewater treatment aeration. The computer-readable storage medium includes, for example, various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0079] The above are all preferred embodiments of this application and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A method for regulating airflow in wastewater treatment aeration, characterized in that, include: Receive historical DO concentration, air volume, influent load and time data, and construct a multi-dimensional feature vector characterizing the operating conditions; Unsupervised clustering algorithm is used to cluster multidimensional feature vectors, classify historical working conditions into several working condition modes, and generate a working condition classifier that can output the working condition membership probability in real time. For each operating mode, a data-driven time series prediction model is trained using its corresponding historical data subset. The input of the time series prediction model is a state vector containing DO deviation, DO change rate and influent load, and a two-dimensional action vector consisting of DO regulation air volume cycle and air volume regulation step. The output is the next state prediction value and an instant reward value consisting of water quality compliance reward, energy consumption penalty and stability penalty. For each working condition mode, a deep reinforcement learning agent containing an Actor network and a Critic network is constructed. The agent is trained offline in the corresponding time series prediction model with the goal of maximizing the cumulative reward, so that the Actor network of the agent serves as the optimal control policy network for the corresponding working condition mode. Real-time data collection and construction of current multidimensional feature vectors; inputting the current multidimensional feature vectors into the working condition classifier; calculating the membership probability distribution of the current working condition belonging to several working condition modes. The current state vector is input into each of the optimal control strategy networks to obtain several sets of candidate DO airflow regulation cycles and airflow regulation steps. The membership probability distribution is used to weight and sum the several sets of candidate values ​​to obtain the DO airflow regulation cycle and airflow regulation step.

2. The method for adjusting air volume based on aeration in wastewater treatment according to claim 1, characterized in that, The method of using unsupervised clustering algorithms to cluster multidimensional feature vectors, dividing historical working conditions into several working condition modes, and generating a working condition classifier capable of outputting the working condition membership probability in real time includes: Preprocessing and feature extraction are performed on historical DO concentration, air volume, influent load and time data to construct a multi-dimensional feature vector corresponding to each historical moment. The multi-dimensional feature vector includes at least: the deviation between the current DO concentration and the set value, the rate of change of DO concentration within the past preset time window, water temperature and time context encoding features. The multidimensional feature vectors of all historical moments are input into the Gaussian mixture model for cluster analysis. The mean, covariance and weight parameters of each Gaussian component are estimated through the expectation-maximization algorithm until the model converges. The optimal number of clusters K is determined by using the Bayesian information criterion, and the historical operating conditions are divided into K types of operating conditions, where K is an integer greater than or equal to 2. A working condition classifier is constructed based on the trained Gaussian mixture model. The input of the working condition classifier is the current multidimensional feature vector collected and constructed in real time, and the output is the posterior probability value of the current working condition belonging to each of the K working condition modes, forming the membership probability distribution. The posterior probability value is calculated by Bayes' formula from the probability density function of each Gaussian component in the Gaussian mixture model and the prior weights.

3. The method for adjusting air volume based on wastewater treatment aeration according to claim 2, characterized in that, The process involves inputting multidimensional feature vectors from all historical moments into a Gaussian mixture model for cluster analysis. Through iterative optimization using the expectation-maximization algorithm, the mean, covariance, and weight parameters of each Gaussian component are estimated until the model converges. This includes: Set an initial range for the number of clusters. The lower limit of the initial range for the number of clusters shall not be less than 2, and the upper limit of the initial range for the number of clusters shall be the preset maximum number of patterns. For each candidate cluster value within the range, the mean vector, covariance matrix, and mixture weights of the Gaussian components are randomly initialized in an equal number to the target candidate cluster value, and the sum of the mixture weights is equal to 1. The multidimensional feature vector set of all historical moments is input into the Gaussian mixture model, and the expectation step and the maximization step are executed iteratively: in the expectation step, the posterior probability value of each feature vector belonging to each Gaussian component is calculated based on the current model parameters; in the maximization step, the mean vector, covariance matrix and mixture weight of each Gaussian component are updated using the calculated posterior probability value. Repeat the expectation step and the maximization step until the log-likelihood function value of the model converges or the preset maximum number of iterations is reached, to obtain the Gaussian mixture model trained under the target candidate clustering numerical values; For each candidate cluster numerical value, the trained Gaussian mixture model is calculated, and the corresponding Bayesian information criterion value is positively correlated with the maximum log-likelihood value of the model. Compare the Bayesian information criterion values ​​corresponding to each candidate cluster value, select the candidate cluster value that maximizes the Bayesian information criterion value or makes the curve of the Bayesian information criterion value changing with the candidate cluster value reach an inflection point as the optimal number of clusters, and take the Gaussian mixture model corresponding to the optimal number of clusters as the final clustering model. Based on the maximum posterior probability of each feature vector belonging to each Gaussian component output by the final clustering model, the working condition modes are divided.

4. The method for adjusting air volume based on wastewater treatment aeration according to claim 2, characterized in that, The step of training a data-driven time series prediction model for each operating condition using its corresponding historical data subset includes: For each operating mode, extract the historical data subset corresponding to the target operating mode. The historical data subset contains the state vector sequence and action vector sequence of all historical moments belonging to the target operating mode label. The state vector sequence consists of at least DO concentration deviation, DO concentration change rate and water temperature, and the action vector sequence consists of DO regulating air volume cycle and air volume regulating step. A subset of historical data is constructed as a supervised learning sample set. The input features of each sample are a combination of the state vector and action vector at the current moment. The label of the sample is the state vector at the next moment. The time interval between the next moment and the current moment of the state vector is equal to the DO adjustment air volume cycle at the current moment. The long short-term memory neural network is used as the network structure of the time series prediction model. The input layer of the long short-term memory neural network receives the concatenated vector of the current state vector and action vector, extracts the temporal dependency features through at least one long short-term memory hidden layer, and outputs the predicted value of the state vector at the next time step through a fully connected output layer. With the goal of minimizing the mean square error between the predicted and true values ​​of the state vector, a gradient-based optimization algorithm is used to iteratively update the weight parameters of the long short-term memory neural network until the model converges or reaches the preset number of training rounds, thereby obtaining the time series prediction model trained under the target working condition. The trained time series prediction model is used as the environmental simulator corresponding to the target operating mode. At the same time, based on the preset multi-objective reward function, the water quality compliance reward is calculated according to the deviation between the predicted value of the state vector and the DO setting value, the energy consumption penalty is calculated according to the air volume adjustment step in the action vector, and the stability penalty is calculated according to the change amplitude between two adjacent action vectors. The three terms are weighted and summed to obtain the instantaneous reward value.

5. The method for adjusting air volume based on wastewater treatment aeration according to claim 4, characterized in that, The step of constructing a deep reinforcement learning agent containing an Actor network and a Critic network for each working condition mode, and training it offline in the corresponding time series prediction model with the goal of maximizing cumulative reward, includes: For each operating mode, a deep reinforcement learning agent is constructed. The deep reinforcement learning agent contains an Actor network and a Critic network. The input of the Actor network is the current state vector, and the output is the mean and standard deviation of the two-dimensional action vector composed of the DO airflow adjustment cycle and airflow adjustment step. Based on the mean and standard deviation, the final action vector is generated through random sampling. The input of the Critic network is the concatenation vector of the current state vector and the action vector generated by the Actor network, and the output is the expected cumulative reward evaluation value of the state-action pair. An offline training loop is constructed. In each training round, a batch of initial state vectors is randomly sampled from the historical data subset corresponding to the target working condition mode. The initial state vectors are input into the Actor network to obtain action vectors. The initial state vectors and action vectors are input into the corresponding environment simulator after training to obtain the predicted value of the state vector at the next moment and the instant reward value. Each interaction generates a current state vector, action vector, instant reward value, next moment state vector, and interaction termination flag, which together form an experience sample and store it in the experience replay pool. The interaction termination flag is set to true when the DO concentration deviation continuously exceeds a preset threshold range or reaches a preset maximum number of interaction steps per round, thus terminating the interaction. Once the number of experience samples stored in the experience replay pool reaches the preset minimum sampling batch requirement, a batch of experience samples is randomly sampled from the experience replay pool, and the target cumulative reward value for each sample is calculated. The weight parameters of the Critic network are updated based on the mean square error between the cumulative target reward value and the evaluation value output by the Critic network. Based on the expected cumulative reward of the Actor network output action evaluated by the Critic network, the policy gradient method is used to update the weight parameters of the Actor network in order to maximize the expected cumulative reward of the Actor network output action. Repeat the training loop and weight adjustment steps until the preset training rounds are reached or the average cumulative reward value obtained by the agent in the environment simulator converges. Save the weight parameters of the trained Actor network. Use the saved Actor network as the optimal control strategy network under the target operating condition mode, and use it to directly output the optimal DO airflow adjustment cycle and airflow adjustment step based on the input state vector.

6. The method for adjusting air volume based on wastewater treatment aeration according to claim 5, characterized in that, The weight parameters of the Critic network are updated based on the mean square error between the target cumulative reward value and the evaluation value output by the Critic network. Based on the expected cumulative reward of the Actor network output action evaluated by the Critic network, the weight parameters of the Actor network are updated using the policy gradient method, including: For each empirical sample in the sampling batch, the predicted value of the state vector at the next time step is input into the Actor network to obtain the action vector at the next time step, and the state vector and action vector at the next time step are input into the Critic network to obtain the evaluation value of the state-action pair at the next time step. Based on the instant reward value, the evaluation value of the next state-action pair, and the preset discount factor, calculate the target cumulative reward value of the current state-action pair; The loss function is to minimize the mean squared error between the current state action pair evaluation value and the target cumulative reward value output by the Critic network. A gradient-based optimization algorithm is used to backpropagate and update the weight parameters of the Critic network. After updating the weight parameters of the Critic network, the current state vector is input into the Actor network to obtain the mean and standard deviation of the current action vector; The probability distribution of actions is constructed based on the mean and standard deviation. The current action vector is randomly sampled from the probability distribution. The current state vector and the current action vector are input into the Critic network to obtain the corresponding evaluation value. The expected value of the evaluation value corresponding to the sampled action under the probability distribution is calculated. The policy gradient method is used to backpropagate and update the weight parameters of the Actor network with the goal of maximizing the expected value. When the preset update frequency of the Critic network and Actor network is reached, the network weight parameters of the corresponding batch of empirical samples are updated.

7. The method for adjusting air volume based on wastewater treatment aeration according to claim 2, characterized in that, The real-time data acquisition and construction of the current multidimensional feature vector, the input of the current multidimensional feature vector into the working condition classifier, and the calculation of the membership probability distribution of the current working condition belonging to several working condition modes include: During online operation, the operation data of the aerobic section of the biological treatment tank of the sewage treatment plant is collected in real time according to the preset sampling cycle. The operation data includes at least the current DO concentration value, the current air volume value, and the current water temperature value. Construct a current multidimensional feature vector based on the operating data, input the current multidimensional feature vector into the working condition classifier, and calculate the posterior probability value of the current multidimensional feature vector belonging to each working condition mode through the working condition classifier. The calculated posterior probability values ​​are combined to form the membership probability distribution, in which the sum of all elements equals 1, and each element corresponds to the probability value of belonging to the corresponding working condition mode.

8. A flow rate regulating device for wastewater treatment aeration, characterized in that, The device includes: The operating condition classification module is used to receive historical DO concentration, air volume, influent load and time data, construct a multi-dimensional feature vector representing the operating condition, and use an unsupervised clustering algorithm to cluster the multi-dimensional feature vector, divide the historical operating conditions into several operating condition modes, and generate an operating condition classifier that can output the operating condition membership probability in real time. The time prediction module is used to train a data-driven time series prediction model for each operating mode using its corresponding historical data subset. The input of the time series prediction model is a state vector containing DO deviation, DO change rate and influent load and a two-dimensional action vector consisting of DO regulation air volume cycle and air volume regulation step. The output is the next state prediction value and an instant reward value consisting of water quality compliance reward, energy consumption penalty and stability penalty. The model training module is used to construct a deep reinforcement learning agent containing an Actor network and a Critic network for each working condition mode. The agent is trained offline in the corresponding time series prediction model with the goal of maximizing the cumulative reward, so that the Actor network of the agent can serve as the optimal control policy network for the corresponding working condition mode. The probability distribution module is used to collect data in real time and construct the current multidimensional feature vector. The current multidimensional feature vector is input into the working condition classifier to calculate the membership probability distribution of the current working condition belonging to several working condition modes. The air volume regulation module is used to input the current state vector into each of the optimal control strategy networks to obtain several sets of candidate DO regulation air volume cycles and air volume regulation steps. The membership probability distribution is used to weight and sum the several sets of candidate values ​​to obtain the DO regulation air volume cycle and air volume regulation step.

9. A control device, characterized in that, The device includes: A memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer program is stored that can be loaded by a processor and executed as described in any one of claims 1 to 7.

Images