Municipal sewage pollution reduction and carbon reduction optimization control method based on double-layer multi-agent cooperation

Abstract

This invention provides an optimized control method for municipal wastewater pollution reduction and carbon reduction based on a two-layer multi-agent collaborative approach. By constructing a real-time computing environment that integrates data-driven and mechanism-based approaches, and utilizing a deep neural network surrogate model to achieve millisecond-level inference of carbon emissions, it significantly reduces computational latency and feedback lag. The invention employs a two-layer control architecture where the upper-layer agent makes decisions on setpoints, and the lower-layer PID controller performs adjustments. This leverages the global optimization capabilities of reinforcement learning while retaining the anti-interference and equipment protection characteristics of PID control. An LSTM-based influent load prediction and dynamic weight adjustment mechanism is introduced, enabling the system to proactively adapt to water quality shocks and load changes, achieving intelligent switching between shock resistance and deep emission reduction modes. A nonlinear flattening reward function is introduced, using the Sigmoid function to bring the reward gradient close to zero after water quality standards are met, effectively suppressing over-aeration and backflow at the algorithmic level, thus achieving source carbon reduction.

Description

Technical Field

[0001] This invention relates to the field of interdisciplinary technology of environmental engineering and artificial intelligence, and in particular to an optimized control method for municipal wastewater pollution reduction and carbon reduction based on two-layer multi-agent collaboration. Background Technology

[0002] With the deepening implementation of the "dual carbon" goals, municipal wastewater treatment plants, as high-energy-consuming and high-carbon-emission units in urban infrastructure, are facing an urgent need to transform from traditional "single pollution reduction" to "synergistic efficiency improvement in pollution reduction and carbon reduction." In the biochemical treatment process of wastewater, key operations such as aeration, recirculation, and carbon source addition not only consume a large amount of electricity but may also induce the direct release of high-temperature-potential greenhouse gases such as nitrous oxide due to improper dissolved oxygen control or an imbalance in the carbon-nitrogen ratio, exacerbating carbon emissions. Therefore, developing intelligent control methods that can ensure stable compliance with effluent quality standards while achieving energy conservation, emission reduction, and greenhouse gas reduction has become a research hotspot in the interdisciplinary field of environmental engineering and artificial intelligence.

[0003] Currently, the mainstream control technology for municipal wastewater treatment processes still mainly relies on a combination of traditional mechanistic models and PID controllers. PID controllers maintain process parameters such as dissolved oxygen and reflux ratio by fixing setpoints, and are simple in structure, highly stable, and widely used in practical engineering. However, faced with drastic fluctuations in influent load and water quality composition, fixed setpoints often lead to excessive effluent quality during peak periods or excessive aeration during off-peak periods, failing to achieve coordinated optimization of energy consumption, water quality, and carbon emissions at the global level. Furthermore, traditional mechanistic models are computationally complex and have slow response times, making it difficult to support real-time optimization decisions. In recent years, with the development of artificial intelligence technology, end-to-end reinforcement learning-based control methods have been proposed, attempting to directly output control commands such as pump speed and valve opening through algorithms. While these methods possess strong optimization capabilities, the extremely high safety and stability requirements of municipal wastewater systems mean that purely data-driven black-box models are prone to large fluctuations or aggressive behaviors, posing safety hazards such as equipment wear and process failure, making their implementation in practical engineering difficult.

[0004] Therefore, how to introduce the global optimization capability of reinforcement learning while retaining the stability and reliability advantages of PID controllers, and build an intelligent control method that can respond to changes in operating conditions in real time, ensure the safe and stable operation of the system, and achieve the triple goals of pollution reduction, energy saving and carbon reduction, has become a technical bottleneck that urgently needs to be solved in this field. Summary of the Invention

[0005] The purpose of this invention is to provide an optimized control method for municipal wastewater pollution reduction and carbon reduction based on two-layer multi-agent collaboration, so as to solve the problems existing in the prior art.

[0006] To achieve the above objectives, the present invention provides the following solution: This invention provides an optimized control method for municipal wastewater pollution reduction and carbon reduction based on a two-layer multi-agent collaboration, comprising the following steps: S1. Establish a real-time computing environment that integrates data-driven and mechanism-based approaches. Within this environment, define a multi-dimensional optimization target space and integrate the following computational logic: Real-time calculation of chemical consumption and sludge production: Based on biochemical reaction kinetics, the conversion efficiency of carbon source addition and the yield of excess sludge are calculated in real time to quantify the environmental footprint; Construct a data-driven real-time greenhouse gas feedback mechanism: collect historical data to train a deep neural network greenhouse gas proxy model, and construct a millisecond-level computational link from biochemical state to carbon emission indicators to replace the computationally intensive mechanism model. Dynamic environmental constraint boundary: Set flexible constraint boundaries for influent flow rate, pH and temperature that vary with season and runoff characteristics to limit the exploration space of the agent; Proactive environmental perception and dynamic weight generation: Collect time-series data of influent water quality, use LSTM model to predict future influent load trends, and establish a dynamic mapping relationship from influent load characteristics to reward function weight space; S2. Construct a state set, action set, and reward function based on the municipal wastewater treatment process; S3. Algorithm initialization and pre-training: Initialize the parameters of the multi-agent deep deterministic policy gradient algorithm, and use historical PID running data to perform supervised learning pre-training on the policy network; S4. Multi-agent collaborative training and policy iteration: A centralized training-decentralized execution architecture is adopted. The central review network evaluates the value of joint actions based on the global state and guides each agent to update its policy. S5. Policy Validation: Validate the robustness of the trained agent policy in a simulation environment containing different climatic conditions; S6. Deployment and Application: Deploy the trained agent model to the municipal wastewater treatment control system; S7. Online Adaptive Optimization: Fine-tuning the strategy network using real-time operational data to adapt to long-term changes in influent characteristics.

[0007] Preferably, the multi-agent system includes: Concentration setting agent: responsible for outputting the optimal biochemical reaction concentration setting value; Dissolved oxygen setting agent: responsible for determining the optimal spatiotemporal distribution of dissolved oxygen in the aerobic zone; Carbon source precision intelligent agent: responsible for responding to fluctuations in the influent C / N ratio and precisely controlling the amount of carbon source added; SRT control agent: responsible for regulating the discharge of excess sludge to maintain the optimal sludge age.

[0008] Preferably, in step S1, the greenhouse gas surrogate model is trained using a hybrid approach of mechanistic data and measured data, and is periodically calibrated using offline high-precision simulation data from the BSM2G mechanistic model.

[0009] Preferably, in step S2, the action set adopts a two-layer control logic, with the upper layer multi-agent outputting process parameter set values ​​and the lower layer PID controller receiving the set values ​​and adjusting the physical actuator.

[0010] Preferably, the lower-level PID controller includes: First PI controller: Receives the set value of nitrite or nitrate concentration and adjusts the speed of the internal reflux pump; The second PI controller group receives the dissolved oxygen concentration setpoint and adjusts the blower frequency and aeration valve opening.

[0011] Preferably, in step S2, the reward function adopts a non-linear flattened form, and its mathematical expression is: ; in, Key pollutants exist Real-time emission values ​​at any given moment. These are the critical values ​​for the emission standards of the corresponding pollutants. For morphological factors, To normalize the system energy consumption, Normalized greenhouse gas emission forecasts As a reward for sludge age stability, Penalties for exceeding limits , , These are weighting coefficients that are dynamically adjusted based on the inflow prediction results.

[0012] Preferably, the adjustment mechanism of the weight coefficients is based on the influent water quality prediction results of the LSTM model: when a high load shock is predicted, the water quality weight is increased; when a low load is predicted, the energy consumption and carbon emission optimization weights are increased.

[0013] Preferably, in step S3, the pre-training uses historical PID running data to supervise the learning of the Actor network, so as to shorten the online convergence time and avoid control strategy divergence.

[0014] Preferably, in step S3, the central review network of the multi-agent deep deterministic policy gradient algorithm receives observations and actions from all agents during the training phase to address the problem of environmental non-stationarity in the multi-agent environment; during the actual deployment phase, each agent makes independent decisions based solely on local observations.

[0015] Preferably, in step S7, the online adaptive optimization includes periodically fine-tuning the strategy network using real-time operational data to adapt to long-term changes in influent water quality, flow rate, and environmental conditions.

[0016] The present invention achieves the following beneficial technical effects compared to the prior art: This invention provides a municipal wastewater pollution reduction and carbon reduction optimization control method based on a two-layer multi-agent collaboration, characterized by stable control, rapid response, and strong collaborative optimization capabilities. Specifically, by constructing a real-time computing environment that integrates data-driven and mechanism-based approaches, a deep neural network surrogate model is used to achieve millisecond-level inference of carbon emissions, significantly reducing computational latency and feedback lag. A two-layer control architecture is adopted, with upper-layer agents making decisions on setpoints and lower-layer PID controllers performing adjustments. This leverages the global optimization capabilities of reinforcement learning while retaining the anti-interference and equipment protection characteristics of PID control, ensuring the robustness and safety of the system in actual operation. An LSTM-based influent load prediction and dynamic weight adjustment mechanism is introduced, enabling the system to proactively adapt to water quality shocks and load changes, achieving intelligent switching between "shock resistance" and "deep emission reduction" modes. A nonlinear flattened reward function is introduced, using the Sigmoid function to bring the reward gradient close to zero after water quality standards are met, effectively suppressing over-aeration and backflow at the algorithmic level, achieving source carbon reduction. Overall, this invention can automatically find the optimal operating conditions with the lowest energy consumption and carbon emissions while ensuring that the effluent quality meets the standards, thus promoting the transformation of municipal sewage treatment processes towards intelligence and low carbon emissions. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 The flowchart of the municipal wastewater pollution reduction and carbon reduction optimization control method based on two-layer multi-agent collaboration provided by the present invention is shown. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] The purpose of this invention is to provide an optimized control method for municipal wastewater pollution reduction and carbon reduction based on two-layer multi-agent collaboration, so as to solve the problems existing in the prior art.

[0021] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] Example 1: This invention aims to provide an intelligent control method that combines optimization capabilities with engineering stability, in order to solve the problem of the difficulty in synergistically achieving pollution reduction, carbon reduction, and energy conservation in municipal wastewater treatment. For example... Figure 1 As shown, in order to make the objectives, technical solutions and advantages of the present invention clearer, the following is a detailed description of the implementation of the present invention using an example of a municipal wastewater treatment plant with a typical A / A / O process and a design capacity of 10,000 cubic meters per day.

[0023] I. Constructing a Hybrid-Driven Simulation Environment for Multi-Objective Collaborative Optimization To achieve rapid simulation of complex biochemical processes and real-time assessment of carbon emissions, it is first necessary to construct a simulation environment that integrates data and mechanisms. This embodiment uses the internationally recognized BSM2 (Benchmark Simulation Model No. 2) model as the basic simulation platform, while also integrating the ASM1 and ASM2 kinetic modules to accurately describe the nitrification, denitrification, and anaerobic digestion processes of sludge, especially the kinetics of nitrite accumulation, thereby providing a reliable "digital twin" testbed for subsequent refined control.

[0024] However, traditional mechanistic models such as BSM2 have high computational loads, making it difficult to achieve real-time feedback of carbon emission indicators. To address this, this invention trains and embeds a deep neural network greenhouse gas proxy model. Specifically, it utilizes massive amounts of data (e.g., 1 million state-emission data pairs) generated by BSM2 under various operating conditions, including seasonal variations, rainy / sunny days, and different influent loads, to train a 5-layer fully connected DNN containing an input layer, three hidden layers, and an output layer. The model's input consists of key state variables of the wastewater treatment process (such as dissolved oxygen, ammonia nitrogen, nitrate nitrogen, nitrite concentration, temperature, COD, and TN concentration before influent), and its output is the combined emission equivalent of greenhouse gases such as nitrous oxide, carbon dioxide, and methane per unit time. In this way, carbon emission indicators, which previously required minutes or even longer for mechanistic calculations, are transformed into millisecond-level fast queries, providing low-latency feedback signals for the reinforcement learning agent.

[0025] To proactively perceive environmental changes and dynamically adjust optimization targets, this invention also constructs an influent prediction module. This module employs a two-layer long short-term memory (LSTM) neural network. Using the influent flow rate, chemical oxygen demand (COD), and ammonia nitrogen concentration sequences over the past month with a time step of one hour as input, this LSTM network can predict the influent load trend for the next 24-72 hours. This prediction result is crucial, as it will be used to dynamically adjust the weight coefficients of the reward function in subsequent steps, giving the control strategy "predictability." This allows for enhanced water quality protection before influent surges and proactive energy conservation and carbon reduction during low-load periods.

[0026] II. Establishing a two-layer spatiotemporally decoupled control architecture This invention abandons the traditional approach of directly controlling actuators (such as valves and pump speeds) using reinforcement learning, and innovatively adopts a two-layer architecture of "upper-layer intelligent agent decision-setting value - lower-layer PID controller execution". This architecture decouples optimization decision-making from stable execution in space and uses different control cycles in time. It utilizes the global optimization capability of reinforcement learning while inheriting the strong anti-interference capability and engineering reliability of PID control, which can protect equipment from violent actions.

[0027] The upper-level intelligent agent system consists of four clearly defined agents that, on an hourly basis, collaboratively output optimized setpoints for key process parameters based on observed system states. Concentration setting agent: Observes the nitrate nitrogen and nitrite concentrations in the anoxic zone, calculates and outputs the optimal nitrite concentration setting value for the key cell (e.g., cell 2) in the anoxic zone. This value is limited to the range of [0.5-5.0]. This indirectly determines the optimization target of the internal recirculation ratio. The output action range of the PID controller corresponds to the internal circulation flow rate and is limited to [0-309720]. Within the reasonable range of the project.

[0028] The dissolved oxygen setting agent observes the state concentrations of ammonia nitrogen and total suspended solids in the aerobic zone and, in conjunction with influent load prediction, establishes differentiated spatiotemporal distribution optimization targets for dissolved oxygen in different compartments (e.g., compartments 3, 4, and 5) within the aerobic zone. Its output dissolved oxygen setpoint is limited to [1.5-2.5]. The range corresponds to the PID controller's adjustment of the oxygen transfer coefficient, and is limited to the range [0-240]. .

[0029] Carbon source precision intelligent agent: Directly observes the carbon-to-nitrogen ratio in the anoxic zone and outputs flow rate commands for the external carbon source dosing pump directly through a strategy network to accurately respond to fluctuations in the influent carbon-to-nitrogen ratio, with the range limited to [0-5]. .

[0030] SRT control agent: Observes the sludge concentration of the system and directly outputs the opening command of the excess sludge discharge valve to maintain the optimal sludge age, limited to [0-1844.6]. .

[0031] The lower-level PID controller operates at a high-frequency cycle of seconds (e.g., 1 second), receiving setpoints from the upper-level intelligent agent and comparing them with real-time readings from field sensors. It calculates the deviation to generate a smooth and stable control signal to drive the physical equipment. The first PI controller receives the set value of nitrite (or nitrate nitrogen) concentration from the upper layer, and changes the internal reflux ratio by adjusting the frequency of the internal reflux pump, thereby stabilizing the reaction environment in the anoxic zone.

[0032] The second PI controller group receives the dissolved oxygen concentration setpoints for each aerobic cell from the upper layer and achieves precise and stable control of dissolved oxygen by coordinating the frequency of the blower and the opening of the regulating valve on the corresponding air pipeline.

[0033] III. Design a nonlinear flattened reward function with the ability to suppress overprocessing. To achieve a multi-objective balance of meeting water quality standards, minimizing energy consumption, and minimizing carbon emissions, and to fundamentally suppress "overtreatment," this embodiment designs a nonlinear flattened reward function of the following form: ; in, Key pollutants exist Real-time emission values ​​at any given moment. These are the critical values ​​for the emission standards of the corresponding pollutants. For morphological factors, To normalize the system energy consumption, Normalized greenhouse gas emission forecasts As a reward for sludge age stability, Penalties for exceeding limits , , These are weighting coefficients that are dynamically adjusted based on the inflow prediction results.

[0034] The core mechanism of this reward function lies in the "flattening" design of the first term: When real-time removal rate When the removal rate is below 95%, the sigmoid function is in a steeply rising region. At this point, the agent will receive a significant increase in reward if it improves the removal rate, which drives the system to prioritize and quickly adjust its actions to ensure that the effluent meets the standards.

[0035] When real-time removal rate After reaching 95%, the function enters a flat saturation region. At this point, the increase in reward value with increasing removal rate becomes negligible. Simultaneously, the energy consumption penalty in the second term of the formula... and the third carbon emission penalty It begins to dominate. This means that if the agent continues to increase aeration or recirculation to pursue a higher removal rate (i.e., "over-treatment"), it will actually lead to a decrease in the total reward. Therefore, the function mathematically guides the agent to stop consuming energy further near the "just enough" boundary, achieving energy conservation and carbon reduction at the source.

[0036] The dynamic weighting coefficients in the function are not fixed, but are dynamically adjusted by the aforementioned LSTM influent prediction module. For example, when a high ammonia nitrogen and high flow rate shock load is predicted for the next hour, the system automatically adjusts the weighting coefficients. Increase to twice the benchmark value, while appropriately reducing and This allows the control strategy to enter a "shock-resistant and compliance-enhancing" mode. Conversely, when a prolonged low load is predicted, the control level is increased. and This guides the system into a "deep energy-saving and carbon-reducing" mode.

[0037] IV. Multi-agent Cooperative Training and Deployment Based on MADDPG Algorithm To enable collaborative learning among multiple agents in a shared environment, this invention employs the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm and follows a centralized training and decentralized execution architecture.

[0038] Network Structure: Each agent has an independent Actor network (policy network) with a 4-layer fully connected structure (hidden layer nodes are 128, 64, and 32 respectively), responsible for generating deterministic actions (i.e., setpoints) based on local observations. Simultaneously, a Centralized Critic network is set up. This network can acquire the observation states and actions of all agents during the training phase, thereby accurately evaluating the global value of the current joint action and effectively solving the problem of environmental non-stationarity caused by simultaneous policy changes in multi-agent environments.

[0039] Training process: Pre-training: To accelerate convergence and avoid process collapse caused by completely random exploration by agents in the simulation environment, supervised learning pre-training is first performed on the Actor network of all agents using the historical operating data (state-setup pairs) of the wastewater treatment plant over the past year, so that they can obtain a preliminary and safe policy starting point.

[0040] Online collaborative training: Pre-trained agents are deployed to a constructed BSM2 simulation environment. At each training step, each agent explores a policy based on its Actor network and with the upper confidence bound, outputting a setpoint. After the lower-level PID controller executes, the environment transitions to a new state, and the aforementioned reward is calculated. The central review network calculates the TD error based on the global state and the actions of all agents, and uses this gradient to simultaneously update its own network parameters and guide the policy updates of each Actor network. This process iterates repeatedly until the policy converges.

[0041] Deployment and Application: After training, the trained strategy is thoroughly validated in simulation environments encompassing various typical and extreme weather and water quality scenarios to ensure its robustness. Following successful validation, the trained Actor network models are deployed to the actual distributed control system (DCS) of a wastewater treatment plant. In actual operation, the agents rely solely on their local observations (decentralized execution) to provide setpoints hourly, with the existing underlying PLC / PID control system performing second-level closed-loop adjustments, thus achieving a seamless transition from simulation to reality. The system also includes an online adaptive optimization module, which periodically (e.g., monthly) fine-tunes the strategy network using data from recent actual operations to adapt to long-term drift in influent characteristics.

[0042] In summary, this invention, through constructing a hybrid simulation environment, innovating a two-layer control architecture, designing a flattened reward function, and applying an advanced multi-agent collaborative training algorithm, forms a complete intelligent control solution. This method not only significantly improves the intelligence level and multi-objective collaborative optimization capability of the control, but also ensures the safe and reliable implementation of the technology in practical engineering by inheriting the stability of PID control, providing an effective technical path for achieving synergistic efficiency improvement in pollution reduction and carbon reduction in municipal wastewater treatment plants.

[0043] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0044] It should be noted that the components mentioned in the above embodiments are all general standard parts or components known to those skilled in the art. Their structure and principle can be learned by those skilled in the art through technical manuals or through conventional experimental methods.

[0045] This invention has illustrated its principles and implementation methods using specific examples. The descriptions of these embodiments are merely illustrative of the method and its core ideas; furthermore, those skilled in the art will recognize that modifications may be made to the specific implementation methods and application scope based on the principles of this invention. Therefore, the content of this specification should not be construed as limiting the invention.

Claims

1. A municipal wastewater pollution reduction and carbon reduction optimization control method based on two-layer multi-agent collaboration, characterized in that, Includes the following steps: S1. Establish a real-time computing environment that integrates data-driven and mechanism-based approaches. Within this environment, define a multi-dimensional optimization target space and integrate the following computational logic: Real-time calculation of chemical consumption and sludge production: Based on biochemical reaction kinetics, the conversion efficiency of carbon source addition and the yield of excess sludge are calculated in real time to quantify the environmental footprint; Construct a data-driven real-time greenhouse gas feedback mechanism: collect historical data to train a deep neural network greenhouse gas proxy model, and construct a millisecond-level computational link from biochemical state to carbon emission indicators to replace the computationally intensive mechanism model. Dynamic environmental constraint boundary: Set flexible constraint boundaries for influent flow rate, pH and temperature that vary with season and runoff characteristics to limit the exploration space of the agent; Proactive environmental perception and dynamic weight generation: Collect time-series data of influent water quality, use LSTM model to predict future influent load trends, and establish a dynamic mapping relationship from influent load characteristics to reward function weight space; S2. Construct a state set, action set, and reward function based on the municipal wastewater treatment process; S3. Algorithm initialization and pre-training: Initialize the parameters of the multi-agent deep deterministic policy gradient algorithm, and use historical PID running data to perform supervised learning pre-training on the policy network; S4. Multi-agent collaborative training and policy iteration: A centralized training-decentralized execution architecture is adopted. The central review network evaluates the value of joint actions based on the global state and guides each agent to update its policy. S5. Policy Validation: Validate the robustness of the trained agent policy in a simulation environment containing different climatic conditions; S6. Deployment and Application: Deploy the trained agent model to the municipal wastewater treatment control system; S7. Online Adaptive Optimization: Fine-tuning the strategy network using real-time operational data to adapt to long-term changes in influent characteristics.

2. The municipal wastewater pollution reduction and carbon reduction optimization control method based on two-layer multi-agent collaboration as described in claim 1, characterized in that: Multi-agent systems include: Concentration setting agent: responsible for outputting the optimal biochemical reaction concentration setting value; Dissolved oxygen setting agent: responsible for determining the optimal spatiotemporal distribution of dissolved oxygen in the aerobic zone; Carbon source precision intelligent agent: responsible for responding to fluctuations in the influent C / N ratio and precisely controlling the amount of carbon source added; SRT control agent: responsible for regulating the discharge of excess sludge to maintain the optimal sludge age.

3. The municipal wastewater pollution reduction and carbon reduction optimization control method based on two-layer multi-agent collaboration as described in claim 1, characterized in that: In step S1, the greenhouse gas surrogate model is trained using a hybrid approach of mechanistic data and measured data, and is periodically calibrated using offline high-precision simulation data from the BSM2G mechanistic model.

4. The municipal wastewater pollution reduction and carbon reduction optimization control method based on two-layer multi-agent collaboration as described in claim 1, characterized in that: In step S2, the action set adopts a two-layer control logic. The upper layer multi-agent outputs the process parameter set value, and the lower layer PID controller receives the set value and adjusts the physical actuator.

5. The municipal wastewater pollution reduction and carbon reduction optimization control method based on two-layer multi-agent collaboration as described in claim 4, characterized in that: The lower-level PID controller includes: First PI controller: Receives the set value of nitrite or nitrate concentration and adjusts the speed of the internal reflux pump; The second PI controller group receives the dissolved oxygen concentration setpoint and adjusts the blower frequency and aeration valve opening.

6. The municipal wastewater pollution reduction and carbon reduction optimization control method based on two-layer multi-agent collaboration as described in claim 1, characterized in that: In step S2, the reward function adopts a non-linear flattened form, and its mathematical expression is: ； in, Key pollutants exist Real-time emission values ​​at any given moment. These are the critical values ​​for the emission standards of the corresponding pollutants. For morphological factors, To normalize the system energy consumption, Normalized greenhouse gas emission forecasts As a reward for sludge age stability, Penalties for exceeding limits , , These are weighting coefficients that are dynamically adjusted based on the inflow prediction results.

7. The municipal wastewater pollution reduction and carbon reduction optimization control method based on two-layer multi-agent collaboration as described in claim 6, characterized in that: The adjustment mechanism of the weight coefficients is based on the influent water quality prediction results of the LSTM model: when a high load shock is predicted, the water quality weight is increased; when a low load is predicted, the energy consumption and carbon emission optimization weights are increased.

8. The municipal wastewater pollution reduction and carbon reduction optimization control method based on two-layer multi-agent collaboration as described in claim 1, characterized in that: In step S3, the pre-training uses historical PID running data to supervise the learning of the Actor network, so as to shorten the online convergence time and avoid the divergence of the control strategy.

9. The municipal wastewater pollution reduction and carbon reduction optimization control method based on two-layer multi-agent collaboration as described in claim 1, characterized in that: In step S3, the central review network of the multi-agent deep deterministic policy gradient algorithm receives observations and actions from all agents during the training phase to address the problem of environmental non-stationarity in the multi-agent environment; during the actual deployment phase, each agent makes independent decisions based solely on local observations.

10. The municipal wastewater pollution reduction and carbon reduction optimization control method based on two-layer multi-agent collaboration as described in claim 1, characterized in that: In step S7, the online adaptive optimization includes periodically fine-tuning the strategy network using real-time operational data to adapt to long-term changes in influent water quality, flow rate, and environmental conditions.

Images