A sewage water quality prediction and dynamic treatment method and system

Abstract

This invention belongs to the technical field of wastewater prediction and treatment. To address the problems of lag and poor robustness in existing wastewater treatment methods, it proposes a wastewater quality prediction and dynamic treatment method and system. The method extracts the long-term and short-term dependency features of historical water quality data during the wastewater treatment process and combines them with real-time key water quality parameters to obtain future water quality prediction results. The future water quality prediction results and collected data are used as input sequences and fed into a deep reinforcement learning model to obtain a water quality treatment action sequence. Wastewater treatment operations are then executed according to this sequence. Each process unit corresponds to a policy network, and each policy network predicts the control actions of its corresponding process unit based on its local state. A centralized evaluation network calculates the expected cumulative reward of the actions based on the local states of all process units and the predicted control actions. This invention improves the dynamic performance and stability of the entire wastewater treatment process.

Description

Technical Field

[0001] This invention belongs to the technical field of wastewater prediction and treatment, and in particular relates to a method and system for predicting and dynamically treating wastewater quality. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] With the acceleration of urbanization and the increasing industrial emissions, wastewater treatment plants are facing increasingly severe challenges. Drastic fluctuations in influent flow, abrupt changes in pollutant concentrations, and increasingly stringent environmental emission standards place higher demands on the operation and management of wastewater treatment plants. Traditional wastewater treatment control strategies, such as PID control and empirical rule-based regulation methods, are ill-suited to the multivariable, highly coupled, and strongly nonlinear real-world operating conditions due to their fixed parameters and strong linear assumptions. When faced with sudden increases in influent load or sharp rises in pollutant concentrations, these methods often exhibit problems such as response lag, severe overshoot, and dead-zone effects, leading to fluctuations in effluent quality and persistently high energy and chemical consumption. Although some research has attempted to introduce advanced algorithms such as model predictive control (MPC), fuzzy control, or expert systems to optimize process scheduling, these methods are highly dependent on system models and easily affected by sensor noise and model mismatch. Furthermore, they mostly focus on optimizing a single subsystem, making it difficult to achieve coordinated scheduling across the entire plant and a unified balance between energy, chemical inputs, and effluent quality.

[0004] In recent years, deep reinforcement learning has brought new opportunities to the control of complex industrial processes due to its ability to automatically learn optimal strategies through trial and error in a continuous action space. Meanwhile, advanced time-series prediction models have also shown significant advantages in multi-step prediction of long-sequence data. However, most existing models are based on historical data for prediction, but wastewater treatment reactions have a lag, leading to large deviations between predictions and actual conditions. Furthermore, existing control strategies are static and cannot be adjusted according to real-time conditions, resulting in resource waste or decreased compliance rates.

[0005] In wastewater treatment scenarios, technical challenges remain, including ensuring real-time performance and robustness, and generating multi-step joint decisions. Therefore, there is an urgent need for an adaptive closed-loop optimization method that can sense changes in influent and operating conditions in real time to achieve high precision, low latency, low energy consumption, and low chemical dosage throughout the entire wastewater treatment process. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, the present invention provides a wastewater quality prediction and dynamic treatment method and system, which realizes adaptive dynamic optimization and control of the wastewater treatment process, improves treatment efficiency, reduces costs, and ensures stable compliance of effluent quality.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] In a first aspect, the present invention provides a method for predicting and dynamically treating wastewater quality, comprising:

[0009] By extracting the long-term and short-term dependence characteristics of historical water quality data during the wastewater treatment process and combining them with the acquired real-time key water quality parameters, future water quality prediction results can be obtained.

[0010] The future water quality prediction results and the collected data of each process unit in the wastewater treatment process are used as input sequences and input into a preset deep reinforcement learning model to obtain a water quality treatment action sequence. The deep reinforcement learning model adopts an Actor-Critic architecture, where each process unit corresponds to an Actor network. Each Actor network predicts the control action of the corresponding process unit based on the local state of each process unit. The centralized Critic network calculates the expected cumulative reward corresponding to the control action based on the local states of all process units and the corresponding predicted control actions.

[0011] Wastewater treatment operations are performed according to the sequence of water treatment actions.

[0012] Secondly, the present invention provides a wastewater quality prediction and dynamic treatment system, comprising:

[0013] The prediction module is configured to extract the long-term and short-term dependence features of historical water quality data during the wastewater treatment process, and combine them with the acquired real-time key water quality parameters to obtain future water quality prediction results.

[0014] The decision-making module is configured to: input the future water quality prediction results and the collected data of each process unit in the wastewater treatment process as input sequences into a preset deep reinforcement learning model to obtain a water quality treatment action sequence; wherein, the deep reinforcement learning model adopts an Actor-Critic architecture, with each process unit corresponding to an Actor network, and each Actor network predicts the control action of the corresponding process unit based on the local state of each process unit; the centralized Critic network calculates the expected cumulative reward corresponding to the control action based on the local states of all process units and the corresponding predicted control actions;

[0015] The execution module is configured to perform wastewater treatment operations according to the sequence of water treatment actions.

[0016] Thirdly, the present invention provides an electronic device including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.

[0017] Fourthly, the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in the first aspect.

[0018] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0019] The above one or more technical solutions have the following beneficial effects:

[0020] In this invention, an Actor-Critic architecture from deep reinforcement learning is adopted. Each key process unit corresponds to an Actor network, and each Actor network obtains the control action for the corresponding process unit based on the local state of each process unit. The centralized Critic network obtains the expected cumulative reward corresponding to the action based on the local states of all process units and the corresponding control actions. Each Actor network obtains its corresponding control action, and by sharing the knowledge of the centralized Critic network, the accuracy and stability of the control strategy are ensured, thereby improving the dynamic performance and stability of the entire wastewater treatment process.

[0021] In this invention, a sliding window incremental learning mechanism is adopted to receive water quality time-series data collected by online sensors in real time. The location encoding is fused through a multi-head self-attention mechanism to extract long-term dependency features of historical data. The decoder is based on mask self-attention and an encoder-decoder interaction layer to generate a water quality prediction sequence for the next M hours and a confidence index for each step. This effectively suppresses the accumulation and propagation of errors in the time-delay system, ensuring the accuracy and timeliness of the prediction data and providing a reliable basis for subsequent decision-making.

[0022] In this invention, the dynamic weight optimization strategy can be combined to adjust the weights of water quality compliance, energy consumption control, and cost optimization according to real-time operating conditions, so that the decision can not only cope with sudden pollution, but also avoid the problem of excessive energy consumption to maintain water quality or sacrificing compliance rate to reduce costs.

[0023] In this invention, a strategy buffer pool is constructed to store the initial multi-step control sequence; the multi-step fusion correction engine, based on a sliding window mechanism, precisely aligns the newly generated M-step action sequence with the old sequence in the buffer pool according to time steps; the confidence weight is calculated by the value evaluation score of the Critic network, and then the actions in the overlapping period are iteratively weighted and corrected. This process continues until the output sequence converges to a preset threshold. Finally, the optimized water quality treatment action sequence is transmitted to the execution unit. Even when faced with drastic fluctuations in influent water quality or sudden changes in operating parameters, the strategy can still maintain rapid adaptation and smooth transition, significantly improving the overall control accuracy and system stability.

[0024] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0025] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0026] Figure 1 This is a block diagram of the wastewater quality prediction and dynamic treatment method in Embodiment 1 of the present invention;

[0027] Figure 2 This is a simplified flowchart of the improved deep reinforcement learning algorithm in Embodiment 1 of the present invention;

[0028] Figure 3 This is a schematic diagram of the multi-step fusion correction module in Embodiment 1 of the present invention;

[0029] Figure 4 This is a flowchart of the real-time dynamic decision-making method for wastewater treatment plant operation using a deep reinforcement learning algorithm, as described in Embodiment 1 of the present invention. Detailed Implementation

[0030] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0031] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0032] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0033] Example 1

[0034] This embodiment discloses a method for predicting and dynamically treating wastewater quality, including:

[0035] By extracting the long-term and short-term dependence characteristics of historical water quality data during the wastewater treatment process and combining them with the acquired real-time key water quality parameters, future water quality prediction results can be obtained.

[0036] The future water quality prediction results and the collected data of each process unit in the wastewater treatment process are used as input sequences and input into a preset deep reinforcement learning model to obtain a water quality treatment action sequence. The deep reinforcement learning model adopts an Actor-Critic architecture, where each process unit corresponds to an Actor network. Each Actor network predicts the control action of the corresponding process unit based on the local state of each process unit. The centralized Critic network calculates the expected cumulative reward corresponding to the control action based on the local states of all process units and the corresponding predicted control actions.

[0037] Wastewater treatment operations are performed according to the sequence of water treatment actions.

[0038] This embodiment fundamentally improves the accuracy and timeliness of water quality prediction through head self-attention, position encoding, and real-time parameter fusion, providing a reliable basis for subsequent decision-making. The deep reinforcement learning model combined with the dynamic weight optimization strategy can adjust the weights of water quality compliance, energy consumption control, and cost optimization according to real-time operating conditions, so that the decision can not only cope with sudden pollution, but also avoid the problem of excessive energy consumption to maintain water quality or sacrificing compliance rate to reduce costs.

[0039] The following is a detailed description of a wastewater quality prediction and dynamic treatment method proposed in this embodiment:

[0040] Step 1: By fusing location codes through a multi-head self-attention mechanism, the long-term and short-term dependence features of historical water quality data during the wastewater treatment process are extracted. Combined with the obtained real-time key water quality parameters, the future water quality prediction results and confidence index are obtained.

[0041] This embodiment, based on a multi-layer encoder-decoder network architecture, can receive and process the latest water quality time-series data from online sensors in real time at each decision-making moment, covering key water quality parameters such as chemical oxygen demand (COD), total nitrogen (TN), and dissolved oxygen (DO). Utilizing a sliding window and streaming data processing mechanism, it achieves true real-time performance and multi-step prediction capabilities, providing forward-looking information for system decision-making. Streaming confidence time-series prediction enables high-precision real-time prediction of key water quality parameters in the wastewater treatment process.

[0042] Specifically, the encoder in this embodiment is based on the Transformer encoder. On the basis of the standard multi-head self-attention and position encoding of the Transformer encoder, a long-short-term feature extraction layer for historical data is added. This layer is implemented by LSTM to capture slowly varying parameters (such as sludge concentration) in wastewater treatment. The encoder can perform deep feature extraction on multi-dimensional time-series signals from the past few hours or even the most recent time, accurately capture the long-short-term dependencies in the data, and fully explore the inherent laws of water quality changes.

[0043] The decoder in this embodiment is based on the Transformer decoder, but it has been improved to suit the characteristics of wastewater treatment time-series data. Specifically, based on the standard Transformer decoder, a masked self-attention mechanism and an encoder-decoder interactive attention layer are introduced, and a confidence output header is added. This is used to gradually generate continuous prediction sequences of key water quality indicators for the next M hours. After the prediction results are formatted, they are transmitted to the dynamic decision-making module in real time, providing accurate forward-looking prior information for the formulation of subsequent control strategies. This drives the closed-loop prediction and decision-making process of the entire system, significantly improving the system's robustness and response speed to water quality fluctuations.

[0044] The multi-layer encoder-decoder network architecture of this embodiment will be described in detail below:

[0045] 1) Location coding:

[0046] By incorporating time and location information into the input, the network can recognize the sequential order of the sequences.

[0047]

[0048] in, This refers to the position index in the input sequence. This refers to the hidden layer dimension or the embedding dimension. This refers to positional encoding; This is a time delay compensation item.

[0049] 2) LSTM timing enhancement layer:

[0050] After location encoding, in order to more effectively capture the long-period dependencies of slowly varying parameters in the wastewater treatment process, the location-encoded sequence is input into a single-term long short-term memory network layer for feature enhancement:

[0051] in, It refers to the matrix vector of the position-encoded input matrix at time step t. These represent the hidden state and cell state of the LSTM at time step t, respectively. These are the weights of the LSTM network; These are the bias parameters of the LSTM network; ( ) is the Sigmoid activation function; It is the hyperbolic tangent activation function. These are the outputs of the forget gate, input gate, and output gate at time t, respectively. It represents the candidate cell state at time t. It represents the cell state at time t. It is the hidden state at time t.

[0052] 3) Linear mapping and generation of Query / Key / Value:

[0053] The feature matrix H output from the temporal enhancement layer is transformed into query, key, and value matrices through three independent linear transformations:

[0054]

[0055] Where, H It is the feature matrix composed of all the hidden states at all time steps obtained after the LSTM temporal enhancement layer is processed; The unified hidden layer dimension in the Transformer model determines the capacity of feature representation within the network, that is, mapping the input features (such as scalars like COD, TN, DO, etc.) at each time step to a single hidden layer. A high-dimensional vector space. Typical values ​​are 256 or 512. For each input time step, the water quality features, after location encoding and LSTM processing, are mapped to... 3D vector representation, Set the typical value to 512; These are the weight matrices for generating queries, keys, and values, respectively. This refers to the projection dimension used to calculate similarity within each attention head. In multi-head attention mechanisms, The dimensional features are segmented into h attention heads, each responsible for A dimensional feature subspace. Attention weights are computed independently for each head. , Set to a typical value of 8 (i.e., 8 attention heads), the dimension of each head is... =64.

[0056] 4) Encoder single-head self-attention mechanism:

[0057]

[0058] By scaling the dot product to calculate the correlation weights between time steps, global dependency information is incorporated into the representation at each time step; whereby... It refers to Matrix transpose, used to calculate queries s and keys Dot product similarity between them; It is a normalized exponential function

[0059] 5) Encoder multi-head attention mechanism:

[0060] ,

[0061]

[0062] in, This represents the operation of concatenating the outputs of multiple attention heads along the feature dimension. For the number of heads, The output projection weight matrix , , , The number of heads.

[0063] 6) Encoder feedforward network:

[0064] The output of the multi-head attention mechanism is passed through two fully connected layers and an activation function:

[0065]

[0066] in, The input vector of the feedforward neural network is the output of the multi-head attention mechanism. , These are the weight matrices for the first and second fully connected layers of the encoder, respectively. , These are the bias vectors for the first and second fully connected layers of the encoder, respectively.

[0067] 7) Decoder processing flow

[0068] The decoder in this implementation case incorporates key improvements over the standard Transformer decoder, including the introduction of a masked self-attention layer, an encoder-decoder interactive attention layer, and a confidence output header. The decoder receives historical features and embedded representations of future sequences from the encoder, progressively generating water quality prediction sequences for the next M time steps and their corresponding confidence levels. The specific processing flow is as follows:

[0069] a) Decoder input embedding and positional encoding

[0070] For the M future time steps that need to be predicted, the input matrix of the decoder is the target sequence that needs to be predicted. For the input matrix Adding position encoding follows the same process as the encoder:

[0071]

[0072] in It is the decoder input matrix with position encoding, and PE(D) is the position encoding matrix.

[0073] b) Masked Multi-Head Attention Layer

[0074] This layer is located in the first layer of the decoder. It ensures that when the decoder generates the output at position j, it can only focus on the information of the first j-1 positions, thus preventing information leakage.

[0075] First of all Linear projection for query ,key Sum :

[0076]

[0077] in, This is a learnable weight matrix.

[0078] Then calculate the mask self-attention:

[0079]

[0080] in, It is a lower triangular mask matrix, whose elements like ,otherwise This masking mechanism ensures that the current position can only focus on previous positions.

[0081] The multi-head attention mechanism is the same as that of the encoder, which involves concatenating h attention heads after parallel computation:

[0082]

[0083]

[0084] in, It is the final output feature matrix of the masked multi-head self-attention layer, and the output of the i-th masked attention head. ( ) is a matrix concatenation function.

[0085] c) Encoding-Decoding Interactive Attention Layer

[0086] This layer, located after the masked attention layer, is the crucial connection between the decoder and encoder. Its query... Output from the layer above the decoder , and key Sum The final output feature matrix from the encoder .

[0087]

[0088]

[0089] in, This is a learnable weight matrix.

[0090] Also employs a multi-head mechanism:

[0091]

[0092]

[0093] in It is the final output feature matrix of the decoding interactive attention layer. It is the output projection weight of the encoding-decoding interactive attention.

[0094] d) Decoder feedforward network

[0095] The deep feature representation of the decoder is then obtained through a feedforward network with the same structure as the encoder. :

[0096]

[0097] in It is the output feature matrix of the decoder feedforward network. , These are the weight matrices for the first and second fully connected layers of the decoder, respectively. , These are the bias vectors for the first and second fully connected layers of the decoder, respectively.

[0098] e) Output layer and confidence prediction

[0099] Finally, the output of the decoder feedforward network Water quality prediction sequences and corresponding confidence sequences are generated through two parallel linear projection layers:

[0100]

[0101]

[0102] in, It is a future water quality indicator prediction sequence. It is the high-dimensional feature matrix of the decoder feedforward network. It is the linear projective weight matrix for water quality prediction. It is the bias vector for water quality prediction; It is the confidence sequence of the next M steps. It is the linear projected weight matrix for confidence prediction. It is the bias vector for confidence prediction.

[0103] This embodiment uses a confidence feedback closed loop to correct the prediction results in real time, effectively suppressing the accumulation and propagation of errors in the time-delay system, ensuring the accuracy and timeliness of the prediction data, and providing a reliable basis for subsequent decision-making.

[0104] Specifically, the decoder output With real-time monitoring data Through weighted fusion;

[0105]

[0106] in, The final fused prediction output is transmitted in real time to the heterogeneous process real-time collaborative decision-making module; The confidence level output by the decoder can be used to fuse weights. The calculated weighting of predicted values ​​and real-time data in the final output is used to balance the proportions of these values. This refers to the latest real-time monitoring data of key water quality parameters obtained from online sensors.

[0107] This embodiment employs a sliding window incremental learning mechanism. During the inference phase of the improved Transformer model, the latest water quality data is continuously input within a fixed time window (the past 20 hours) to achieve dynamic data updates. During the training phase of the improved Transformer model, time-series segmented cross-validation is used to simulate the sliding window, ensuring that the Transformer model adapts to temporal changes.

[0108] In this embodiment, the confidence feedback closed loop's line of sight is at the end of the decoder, in addition to the original output... In addition to the linear projection layer for the indicator, a fully connected layer (confidence prediction head) is added in parallel to output the confidence score sequence for each prediction time step. The confidence score is dynamically calculated based on the error between the predicted and actual values, and the Transformer model parameters are adjusted through a feedback loop. A confidence-weighted term is added to the loss function. During the online learning phase, when the confidence score falls below a threshold of 0.8, the Transformer model is fine-tuned, and the weights are updated using the latest data.

[0109] A confidence-weighted term was added to the loss function of the improved Transformer model:

[0110]

[0111] in, This is the actual value; This is a predicted value; The confidence level of the model for each predicted value is represented by a scalar between (0,1). The closer the value is to 1, the more reliable the prediction is, and the closer it is to 0, the less uncertain the prediction is. and All of these are hyperparameters that balance the prediction error loss and the weight of the confidence penalty term, and are determined on the validation set through cross-validation, with values ​​ranging from [0.001, 0.1]. `-log(confidence)` is a regularization term, a scalar between (0,1), used to prevent the model from blindly outputting extremely high, unrealistic confidence scores to reduce prediction error (when confidence approaches 0, `-log(confidence)` becomes very large, causing the penalty term to increase sharply, thus preventing the model from outputting meaningless low confidence scores. When confidence approaches 1, `-log(confidence)` approaches 0, the penalty almost disappears, allowing the model to give high confidence scores when confident). The confidence score is directly output by the decoder's confidence prediction head (a separate, fully connected layer parallel to the main prediction output layer). This prediction head is trained simultaneously with the main prediction network, learning to automatically assess the uncertainty of the prediction based on the input features.

[0112] Through the above scheme, the system can quickly adjust the control strategy based on multi-step forward prediction information when the influent water quality changes suddenly or the operating conditions are uncertain, which significantly improves the overall response speed and system robustness.

[0113] Step 2: Control dynamic decision-making.

[0114] This embodiment adopts the Actor-Critic architecture in deep reinforcement learning, specifically the Deep Deterministic Policy Gradient Algorithm (DDPG), which is suitable for continuous action spaces (such as adjusting aeration rate, carbon source acceleration rate, etc.).

[0115] This embodiment employs a multi-agent reinforcement learning framework to achieve collaborative optimization control of the wastewater treatment process. Specifically, each key process unit corresponds to an independent Actor network (policy network), and they share a centralized Critic network (value network). Global coordination is achieved through a centralized training and distributed execution paradigm.

[0116] The following section details the aspects of agent design, state-action representation, training methods, and coordination mechanisms:

[0117] 1. Agent Architecture: Each process unit corresponds to one agent, and each agent contains an Actor network, which is used to predict the control actions of the corresponding process unit based on the local state of each process unit; all agents share a centralized Critic network, which is used to calculate the expected cumulative reward corresponding to the control action based on the local state of all process units and the corresponding predicted control actions.

[0118] 1) Agent allocation: The wastewater treatment process is decomposed into four main process units, each controlled by an independent agent:

[0119] Bioreactor (BRE) agent: responsible for adjusting the sludge return ratio.

[0120] Dissolved Oxygen Adapter (DOA) Smart Agent: Responsible for controlling the speed and airflow of the aerator.

[0121] Carbon Source Adaptive System (CAS) agent: responsible for optimizing the acceleration rate of added carbon sources (such as sodium acetate).

[0122] Flocculant Coordinator (FCS) agent: responsible for coordinating and adjusting the dosage of polyferric sulfate (PFS) and polyacrylamide (PAM) flocculants.

[0123] 2) Actor-Critic Architecture: Each agent contains an Actor network, i.e., a policy network, while all agents share a centralized Critic network, i.e., a value network.

[0124] Actor networks: Inputs are local states, and outputs are the control actions of the corresponding units. Each actor network makes independent decisions, but they are coordinated globally through a centralized critic network.

[0125] Critic network: The input is the global state, i.e., the state and actions of all units, and the output is the Q-value to evaluate the overall performance. The centralized Critic network is shared and used to guide the updates of all Actor networks.

[0126] This design avoids the complexity of a single agent handling a high-dimensional action space, while ensuring the coordination of actions among units through centralized Critic, thereby improving the system's robustness.

[0127] 2. State and Action Representation: The state and action space of each agent is specifically designed according to the technological unit it controls, ensuring the relevance and independence of information.

[0128] 1) State Representation:

[0129] Local state: Each agent receives sensor data and prediction information directly related to its unit.

[0130] Bioreactor actuator: Inputs include sludge concentration (MLSS), historical internal recirculation ratio, and influent COD / TN prediction sequence.

[0131] Dissolved oxygen adapter smart agent: Inputs include real-time dissolved oxygen (DO) values, aerator status, and effluent TN / TP prediction sequence.

[0132] Carbon source adaptive system agent: Inputs include influent C / N ratio, carbon source addition history, and real-time carbon source price.

[0133] Flocculation Coordinator Intelligent Agent: Inputs include turbidity, flocculant dosing history, and PFS / PAM price fluctuations.

[0134] 2) Global state: All agents share some global information, such as inflow rate, overall energy consumption, and water quality prediction confidence. This information is integrated through a centralized Critic network.

[0135] State vector format: Each state vector is multi-dimensional time series data, which is dynamically updated through a sliding window mechanism. The dimensions are [T, local], where T is the time step and local is the local feature dimension.

[0136] Action Representation:

[0137] Each agent outputs continuous action values, corresponding to the adjustment amount of its unit's control parameters:

[0138] Bio-reactive actuator intelligent agent: Action output is the internal reflux ratio adjustment ratio (range: 0-100%).

[0139] Dissolved oxygen adapter intelligent agent: The action output is the aerator speed setting value (range: 0-rated speed) and the air volume adjustment ratio.

[0140] Carbon source adaptive system agent: Action output is the carbon source injection acceleration rate (unit: mg / L·h).

[0141] Flocculation coordinator agent: Action output is the dosage of PFS and PAM (unit: mg / L).

[0142] The range of motion is scaled by the activation function (sigmoid function) of the output layer to ensure it remains within physically feasible limits.

[0143] This embodiment employs a core paradigm of centralized training and distributed execution to train multiple agents. The basic idea is that during training, global information is used to guide and optimize the strategies of all agents; after training, each agent can make independent and rapid decisions based solely on its own locally observed information during runtime.

[0144] The specific flow is divided into the following steps:

[0145] 1) Experience collection and storage:

[0146] The four intelligent agents—the bioreactor intelligent agent, the dissolved oxygen adapter intelligent agent, the carbon source adaptive system intelligent agent, and the flocculation coordinator intelligent agent—generate control actions in parallel according to their respective current strategies, forming a joint action and issuing it to the wastewater treatment system for execution.

[0147] The wastewater treatment plant reported a comprehensive overall benefit and new status resulting from the implementation of this joint action, encompassing water quality, energy consumption, and costs.

[0148] This complete interactive experience (old state, joint action, reward, new state) is stored as a data package in a shared experience replay pool.

[0149] 2) Sampling and Learning:

[0150] During training, a small batch of historical experience data is randomly sampled from the experience replay pool. This random sampling breaks the continuity between data, making the learning process more stable.

[0151] 3) Critic Network-based Intensive Learning and Evaluation:

[0152] A centralized Critic network acts as a coordinator during the training process. It has access to global information, including the states and joint actions of all agents.

[0153] The learning objective of a centralized Critic network is to accurately predict the long-term comprehensive reward, or Q-value, that can be obtained by performing a certain joint action in a given state.

[0154] By comparing predicted values ​​with actual returns, the centralized Critic network continuously adjusts its parameters to evaluate the merits of each agent's action combination with increasing accuracy.

[0155] 4) Optimization of distributed strategy for Actor network:

[0156] Each agent's own Actor network acts as the executor, making decisions based solely on its own local observations.

[0157] After completing its learning process, the Critic network generates a guidance signal called gradient, which indicates how to fine-tune the actions of each agent to maximize the global total reward.

[0158] Each actor network independently updates its own policy based on this guidance signal, enabling it to make better decisions and more collaborative decisions with other agents in the same situation next time.

[0159] 5) Target network soft update:

[0160] To maintain training stability, a replica, called the target network, is set up for each Actor and the centralized Critic network to provide a stable learning objective.

[0161] After each update of the main network, the target network will not synchronize immediately, but will instead move closer to the main network through soft updates to prevent policy abrupt changes and divergence.

[0162] Through this iterative process, the four agents, guided by the centralized Critic network, gradually transform into cooperative operators, ultimately learning a highly coordinated optimal control strategy. Once training is complete, each agent can be separated and operate efficiently and independently, relying solely on local information, thereby achieving fully automated intelligent control of complex wastewater treatment processes.

[0163] In this embodiment, the multi-agent collaborative control scenario has been expanded and optimized: In each control cycle, the centralized Critic network simultaneously receives a high-dimensional state vector, which includes the current real-time sensor readings, the forward-looking water quality and equipment status prediction information of the historical M steps (referring to the sequence of expected operating states of key equipment in the next M steps generated by the improved Transformer prediction module), and the operating constraints of each subsystem. It outputs a continuous control action sequence for the next M steps at once. This continuous control action sequence covers multiple dimensions of operations such as the internal reflux ratio adjustment ratio, aerator air volume and speed setting, external carbon source acceleration rate, and PFS and PAM chemical dosage.

[0164] The historical M-step prediction is generated using an improved Transformer: a Transformer encoder-decoder model is used, where the encoder processes historical data and the decoder progressively generates future sequences. The improved Transformer takes as input historical water quality data (e.g., COD, TN, DO) from the past 20 hours and real-time sensor readings; and outputs a water quality prediction sequence for the next M hours and a confidence index. The predicted data is used as part of the deep reinforcement learning state variables for forward-looking decision-making.

[0165] The Actor network uses a multi-layer fully connected neural network structure to generate specific control actions based on the input state (such as adjusting the aerator speed or adjusting the operating frequency of the additional carbon source dosing pump), while the centralized Critic network evaluates the expected cumulative reward corresponding to the action.

[0166] To ensure the stability of the training process, experience playback technology, target network soft update, and multi-objective reward function design are adopted. The reward function comprehensively reflects whether the water quality meets the standards, energy consumption, and chemical usage, thereby optimizing the final control strategy.

[0167] The Actor network outputs continuous actions. ( These correspond to four different Actor networks: dissolved oxygen concentration, carbon source addition, flocculant addition, and sludge return ratio.

[0168]

[0169] in, It is the policy function of the Actor network itself, which defines how to choose an action given a state. For Actor network parameters, This is the feature vector of the current state.

[0170] Critic network evaluates status-value of actions :

[0171]

[0172] in, It is a state-action value function, representing the state-action value function. Next execution action vector And has followed the strategy ever since. The expected cumulative discount reward that can be obtained It is the expectation operator, which calculates the average of all possible future trajectories (states, action sequences). It is the global state. =( , ) is the joint vector of the actions of the four agents. It is the global, immediate reward obtained after performing this joint action; As a discount factor, it determines the degree of importance attached to future rewards. For a multi-objective reward function, T represents the termination time step of the current training episode; In time step Instant reward function;

[0173] Reward function:

[0174] in,

[0175] Water quality compliance score:

[0176] Energy consumption index:

[0177]

[0178] Cost penalty items:

[0179] in, The water quality compliance index; Water quality compliance score: Calculates the degree to which the current effluent water quality meets the standard value. It is the theoretical maximum value of the water quality compliance score; Energy consumption for aeration; It is an energy consumption benchmark value, used to normalize actual energy consumption; This refers to the dosage of polyferric sulfate; It is polyacrylamide; As weight; It is the budgeted value of the drug cost, used to normalize and penalize the actual cost; It is the rated power of the equipment; In time step The actual rotational speed of the equipment; This is the maximum permissible speed of the equipment; It is the unit price of the carbon source; This refers to the dosage of polyferric sulfate; That is the unit price of polyferric sulfate; This refers to the dosage of polyacrylamide; That is the unit price of polyacrylamide.

[0180] Target network soft update:

[0181]

[0182] in, This is the update rate (usually 0.001).

[0183] This implementation employs a combined dynamic weighting optimization strategy, monitoring influent load fluctuations, chemical market price dynamics, and energy consumption data in real time. Based on this real-time information, the weights of sub-items in the multi-objective reward function are dynamically adjusted. For example, when the concentration of pollutants in the influent suddenly increases, the water quality weight is promptly increased to cope with sudden changes in water quality; conversely, when chemical prices rise, the cost weight is correspondingly increased to control operating costs. This dynamic adjustment mechanism enables the system to flexibly balance various objectives under different operating conditions, achieving efficient, economical, and stable operation of the wastewater treatment process, and improving the overall performance and adaptability of the system.

[0184] Continuous monitoring of influent water quality parameters, equipment operating status, and chemical market prices serves as the basis for weight adjustments. Based on a pre-set weight adjustment strategy, the weight coefficients of each sub-objective in the multi-objective reward function are updated in real time. This allows the system to flexibly balance the relationships between multiple objectives, such as water quality compliance, energy consumption control, and cost optimization, under different operating conditions. This effectively guides the continuous optimization of control strategies for various heterogeneous processes. While ensuring stable effluent quality compliance, it can effectively reduce energy consumption and rationally control chemical costs based on different water quality conditions, significantly improving the operational economy and environmental benefits of wastewater treatment plants.

[0185] Real-time monitoring of influent load fluctuation rate The average price of pharmaceutical products and real-time energy consumption Based on the fluctuation rate of influent load And changes in drug prices ,pass Function dynamically adjusts weights and To ensure a smooth transition in priorities during sudden changes in water quality or rising costs, the weights of sub-items in the multi-objective reward function are dynamically adjusted as follows:

[0186]

[0187] in, , This is the sensitivity coefficient. It is the change in drug price, that is, the difference between the purchase price of the drug and the general average price in the drug market.

[0188] For example, when the inflow load fluctuation rate When increased, water quality weight pass The function increases non-linearly to ensure the system responds quickly to contamination events.

[0189] Sigmoid response curve:

[0190]

[0191] The response curve is used to adjust the weights of the sub-items of the multi-objective reward function. The weights of the multi-objective reward function are calculated as follows: b=0.5 is the offset used to adjust the center of the response curve.

[0192] This embodiment also includes a multi-step fusion correction engine, which constructs a strategy buffer pool to store the initial multi-step control sequence. Based on a sliding window mechanism, the multi-step fusion correction engine precisely aligns the newly generated M-step action sequence with the old sequence in the buffer pool according to time steps. By combining the confidence index and the value evaluation score of the Critic network, a dual-weight fusion coefficient is calculated, and then the actions in overlapping periods are iteratively weighted and corrected. This process continues until the output sequence converges to a preset threshold. Finally, the optimized sequence is transmitted to the execution unit. Even when faced with drastic fluctuations in influent water quality or sudden changes in operating parameters, it can still maintain rapid adaptation and smooth transition of the strategy, significantly improving overall control accuracy and system stability.

[0193] The buffer pool stores and updates control sequences for iterative correction and optimization. Initial control sequences are generated based on the current state and prediction information. A sliding window mechanism is used to repeatedly execute the prediction and decision-making process, generating new control sequences, which are then compared and fused with the old sequences in the buffer pool. Fusion weights are dynamically allocated based on the evaluation network's value assessment and prediction confidence, and action items within overlapping time periods are iteratively corrected. The convergence of the control sequences is monitored to ensure stability within a preset convergence threshold. If convergence is not achieved, the iterative correction process is repeated. The corrected control sequences are then updated back to the buffer pool to replace the old ones, providing a more accurate reference for subsequent decisions.

[0194] Through the above steps, the multi-step fusion correction engine can ensure that the control strategy of the wastewater treatment process can quickly adapt and smoothly transition when faced with fluctuations in influent water quality and sudden changes in operating parameters, significantly improving the overall control accuracy, system robustness and stability in response to sudden operating conditions.

[0195] Specifically, it refers to the start time of the previous sliding window. Based on this, the length of the Actor generated is first set to... initial action sequence Write to the temporary pool:

[0196]

[0197] in This represents the number of actions in a multi-step decision-making process, for example, M; Indicates in Control actions issued in real time.

[0198] When the sliding time window moves one position to the right to the starting time At that time, the system generates a new multi-step action sequence again. :

[0199]

[0200] in, Indicates in Control actions are constantly regenerated based on the latest state.

[0201] Retrieve the old sequence from the temporary storage pool. With new sequence Action pairs at overlapping moments .

[0202] For each action pair, based on the evaluation function of the Critic network Calculate the confidence weights:

[0203]

[0204] in, Indicates time Environmental characteristics; This indicates that the Critic network represents state-action pairs. The expected cumulative reward estimate.

[0205] Subsequently, a weighted fusion correction is performed on the overlapping parts to obtain the corrected action. :

[0206]

[0207] in, This indicates that after iterative correction, in The final control action at any given moment; This refers to the blending weights of the old actions after action alignment.

[0208] Finally, the fused sequence Update back to the staging pool to replace the old entry:

[0209]

[0210] Each time window slides, the above prediction-decision-alignment-fusion process is repeated until the control actions at all times meet the preset convergence conditions in multiple iterations, forming an optimal multi-step control strategy that combines the advantages of feedforward prediction with iterative feedback adaptation, thereby significantly improving the system's response speed, robustness and control accuracy to fluctuations in influent water quality and sudden operating conditions.

[0211] Step 3: Process decoupling execution set, which is decomposed into quaternary heterogeneous units to achieve independent control of different process processes.

[0212] The system comprises several components: a Bioreactor (BRE) specifically responsible for adjusting the sludge return ratio; a Dissolved Oxygen Adapter (DOA) dynamically controlling the aerator's speed and airflow; a Carbon Source Adaptive System (CAS) optimizing the dosing rate of external carbon sources in real time; and a Flocculation Coordinator (FCS) coordinating the adjustment of PFS and PAM flocculant dosages. Each unit receives forward-looking water quality prediction data from the SCTE and correction instructions from the PFCE, generating precise decoupled control actions to achieve refined control of the wastewater treatment process. This ensures efficient and coordinated operation of all process stages, improving wastewater treatment efficiency and water quality compliance rates.

[0213] Each heterogeneous unit in the Process Decoupling Execution Set (PDAS) independently receives precise control commands from the dynamic decision module and executes corresponding operation commands based on its specific process task. The Bioreactor (BRE) focuses on the precise control of sludge return, optimizing the sludge return ratio to ensure the stability and activity of the microbial population in the bioreactor and improve pollutant removal efficiency. The Dissolved Oxygen Adapter (DOA) is responsible for the fine adjustment of aeration intensity, adjusting the aeration rate in real time based on dissolved oxygen prediction information to ensure that the dissolved oxygen concentration is maintained at a reliable level, balancing energy consumption and treatment efficiency. The Carbon Source Adaptive System (CAS) precisely controls the carbon source dosage based on carbon source demand prediction, improving carbon source utilization and reducing operating costs. The Flocculation Coordinator (FCS) intelligently controls the flocculant addition process, optimizing the flocculation effect and ensuring stable effluent quality. Through the independent operation and coordinated cooperation of each heterogeneous unit, this embodiment effectively avoids the problem of mutual interference between different process links under the traditional homogeneous control architecture, significantly improving the overall control accuracy and stability of the system.

[0214] This embodiment also includes real-time detection of process conflict conditions. Once a situation is detected where the carbon source dosage exceeds 250 mg / L and the flocculant dosage is below 50 mg / L, a circuit breaker mechanism is immediately triggered. Subsequently, a constrained optimization algorithm is invoked to generate a feasible solution, which covers the conflict action and is forcibly written into the temporary storage tank to ensure the stable operation of the system and avoid problems such as system instability or effluent quality exceeding standards caused by process conflicts, thus ensuring the continuity and reliability of the wastewater treatment process.

[0215] To further improve the system's security, real-time response capability, and seamless integration with existing industrial control systems, this embodiment introduces the following improvements: online model updates and adaptive adjustments. To ensure long-term stable operation of the system, an online experience playback and target network soft update mechanism is adopted. Specifically, this includes: continuously training and updating the network using historical operating data to ensure that the control strategy can adapt to dynamic changes in the external environment; in the multi-objective reward function, the weights of each item can be adjusted online based on real-time feedback data to ensure the best balance between water quality compliance, energy consumption reduction, and chemical conservation.

[0216] like Figure 1 , Figure 2 , Figure 3 , Figure 4 As shown, the various steps work together to form a complete closed-loop control system. The key update formula of the deep reinforcement algorithm provides a theoretical basis for the adaptive adjustment of the control strategy, while the neural network predicts future water quality changes, providing a reference for advance decision-making. Through experience playback, soft updating of the target network, and multi-level safety control, this embodiment demonstrates extremely high robustness and response speed in experiments.

[0217] This embodiment employs a sliding window incremental learning mechanism, combined with confidence feedback closed-loop real-time correction of prediction bias, significantly suppressing error propagation in time-delay systems; a multi-step fusion correction engine (PFCE) is based on a temporary storage pool-driven confidence-value dual-weight fusion to achieve iterative optimization of multi-step control sequences; a process decoupling execution set (PDAS) constructs a quaternary heterogeneous unit consisting of a bioreactor (BRE), a dissolved oxygen adapter (DOA), a carbon source adaptive system (CAS), and a flocculant coordinator (FCS), independently controlling sludge return, aeration intensity, carbon source, and flocculant dosage, eliminating coupling defects in homogeneous control architectures; a dynamic weighted optimizer (LCAW) adjusts the weight priorities of water quality compliance, energy consumption control, and cost optimization in real time based on influent load fluctuations and reagent cost changes; and a cross-unit fuse (CUCM) triggers a constrained optimization algorithm to generate feasible solutions when a process conflict is detected (e.g., carbon source dosage > 300 mg / L and flocculant dosage < 50 mg / L). Practical applications show that, while ensuring compliance with effluent quality standards, this method enables real-time dynamic decision-making on key indicators of wastewater treatment plants, significantly improving control accuracy and robustness, and effectively reducing energy consumption and chemical costs, thus demonstrating good energy-saving, environmental protection, and economic benefits.

[0218] Example 2

[0219] The purpose of this embodiment is to provide a wastewater quality prediction and dynamic treatment system, including:

[0220] The prediction module is configured to extract the long-term and short-term dependence features of historical water quality data during the wastewater treatment process, and combine them with the acquired real-time key water quality parameters to obtain future water quality prediction results.

[0221] The decision-making module is configured to: input the future water quality prediction results and the collected data of each process unit in the wastewater treatment process as input sequences into a preset deep reinforcement learning model to obtain a water quality treatment action sequence; wherein, the deep reinforcement learning model adopts an Actor-Critic architecture, where each Actor network obtains the control action of the corresponding process unit based on the local state of each process unit; and the centralized Critic network obtains the expected cumulative reward corresponding to the action based on the local states of all process units and the corresponding control actions;

[0222] The execution module is configured to perform wastewater treatment operations according to the sequence of water treatment actions.

[0223] In further embodiments, the following is also provided:

[0224] An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor. When executed by the processor, the computer instructions perform the method described in Embodiment 1. For brevity, further details are omitted here.

[0225] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0226] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.

[0227] A computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in Embodiment 1.

[0228] The method in Embodiment 1 can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.

[0229] A computer program product includes a computer program that, when executed by a processor, implements the method described in Embodiment 1.

[0230] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, which execute in a device on a target real or virtual processor to perform the processes / methods described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided among program modules as needed. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside in both local and remote storage media.

[0231] The computer program code used to implement the methods of the present invention may be written in one or more programming languages. This computer program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the computer or other programmable data processing device, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a computer, partially on a computer, as a stand-alone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0232] In the context of this invention, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like. Examples of signals may include electrical, optical, radio, sound, or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0233] Those skilled in the art will recognize that the units and algorithm steps described in conjunction with the embodiments herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0234] Those skilled in the art will recognize that the units and algorithm steps described in conjunction with the embodiments herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0235] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for predicting and dynamically treating wastewater quality, characterized in that, include: By extracting the long-term and short-term dependence characteristics of historical water quality data during the wastewater treatment process and combining them with the acquired real-time key water quality parameters, future water quality prediction results can be obtained. The future water quality prediction results and the collected data of each process unit in the wastewater treatment process are used as input sequences and input into a preset deep reinforcement learning model to obtain a water quality treatment action sequence. The deep reinforcement learning model adopts an Actor-Critic architecture, where each process unit corresponds to an Actor network. Each Actor network predicts the control action of the corresponding process unit based on the local state of each process unit. The centralized Critic network calculates the expected cumulative reward corresponding to the control action based on the local states of all process units and the corresponding predicted control actions. Perform wastewater treatment operations according to the water quality treatment action sequence; After obtaining the water treatment action sequence, the following is also included: Align the generated water treatment action sequence with the old action sequence of the previous sliding time window in the temporary storage tank by time step to determine the action pairs at overlapping times; For each action pair, the confidence weight of the action pair at overlapping times is determined based on the value assessment score corresponding to each time window. The overlapping actions are weighted and fused based on confidence weights to obtain a preliminary corrected sequence. The preliminary corrected sequence is placed in a temporary storage pool to replace the old strips until the water quality treatment actions at all times meet the preset convergence conditions in multiple iterations, and the optimized water quality treatment action sequence is obtained.

2. The wastewater quality prediction and dynamic treatment method as described in claim 1, characterized in that, Based on the improved Transformer prediction model, long-term and short-term dependence features of historical water quality data during wastewater treatment are extracted. Combined with the acquired real-time key water quality parameters, future water quality prediction results are obtained, specifically: Location coding is performed on historical water quality data during the wastewater treatment process. Long short-term memory network is used to enhance the time series of the location-coded historical water quality sequence to capture long- and short-term dependencies. The temporal enhancement features are encoded using a Transformer encoder to obtain historical water quality features; Based on historical water quality characteristics and the acquired real-time key water quality parameters, a Transformer decoder is used to make predictions and obtain future water quality prediction results.

3. The wastewater quality prediction and dynamic treatment method as described in claim 2, characterized in that, Based on historical water quality characteristics and acquired real-time key water quality parameters, a Transformer decoder is used for prediction to obtain future water quality prediction results and confidence indices, specifically: Position encoding is performed on the target sequence, and single-head mask self-attention value is calculated on the position-encoded target sequence. Multi-head mask self-attention values ​​are concatenated using a multi-head attention mechanism to obtain the output features of the multi-head attention layer. Using the output features of the multi-head attention layer as the query vector and the historical water quality features output by the encoder as the key vector and value vector, the water quality prediction results output by the decoder are obtained by using the multi-head attention mechanism. The water quality prediction results output by the decoder are weighted and fused with real-time key water quality parameters to obtain the final future water quality prediction results.

4. The wastewater quality prediction and dynamic treatment method as described in claim 1, characterized in that, In the reward function calculation, the weights for water quality compliance, energy consumption control, and cost optimization are adjusted in real time using a dynamic weight optimization strategy, specifically as follows: in, , This is the sensitivity coefficient; Fluctuations in influent load; For changes in drug prices; , , These are the weighting coefficients for water quality compliance, energy consumption control, and cost optimization, respectively. This represents the average price in the pharmaceutical market.

5. The wastewater quality prediction and dynamic treatment method as described in claim 3, characterized in that, A confidence prediction header is added to the end of the Transformer decoder to output the confidence score sequence for each prediction time step. The predicted confidence is added to the loss function to train the improved Transformer.

6. A wastewater quality prediction and dynamic treatment system, characterized in that, include: The prediction module is configured to: extract the long-term and short-term dependence features of historical water quality data during the wastewater treatment process, and combine them with the acquired real-time key water quality parameters to obtain future water quality prediction results; The decision-making module is configured to: input the future water quality prediction results and the collected data of each process unit in the wastewater treatment process as input sequences into a preset deep reinforcement learning model to obtain a water quality treatment action sequence; wherein, the deep reinforcement learning model adopts an Actor-Critic architecture, with each process unit corresponding to an Actor network, and each Actor network predicts the control action of the corresponding process unit based on the local state of each process unit; the centralized Critic network calculates the expected cumulative reward corresponding to the control action based on the local states of all process units and the corresponding predicted control actions; The execution module is configured to perform wastewater treatment operations according to a sequence of water treatment actions; after obtaining the sequence of water treatment actions, it also includes: Align the generated water treatment action sequence with the old action sequence of the previous sliding time window in the temporary storage tank by time step to determine the action pairs at overlapping times; For each action pair, the confidence weight of the action pair at overlapping times is determined based on the value assessment score corresponding to each time window. The overlapping actions are weighted and fused based on confidence weights to obtain a preliminary corrected sequence. The preliminary corrected sequence is placed in a temporary storage pool to replace the old strips until the water quality treatment actions at all times meet the preset convergence conditions in multiple iterations, and the optimized water quality treatment action sequence is obtained.

7. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, perform the method described in any one of claims 1-5.

9. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the method described in any one of claims 1-5.

Images