Simulation model training methods and devices, wastewater treatment methods, equipment and media

Abstract

This invention discloses a simulation model training method and apparatus, a wastewater treatment method, equipment, and medium, applicable to the field of environmental engineering. It constructs state nodes based on state parameters, action nodes based on action parameters, and intermediate nodes based on exogenous variables and constants. A causal graph is constructed based on the state nodes, action nodes, and intermediate nodes. Decision nodes and inference nodes are determined based on the node relationships within the causal graph. A generator is constructed based on the inference model of the inference nodes and the decision model of the decision nodes. A value model that scores the generator's output is constructed as a discriminator. Adversarial learning is performed based on the generator and discriminator, incorporating causal decoupling and contrastive learning strategies. This invention proposes a wastewater treatment simulation scheme based on contrastive-adversarial imitation learning with causal feature decoupling, constructing a virtual environment with high realism and generalization ability, providing a reliable foundation for intelligent control of the entire wastewater treatment process.

Description

Technical Field

[0001] This invention relates to the field of environmental engineering, and in particular to a simulation model training method and device, a wastewater treatment method, an electronic device, and a computer-readable storage medium. Background Technology

[0002] The construction of wastewater treatment process simulation systems is highly complex, nonlinear, time-varying, and uncertain. Traditional simulation methods have many limitations in terms of accuracy, generalization ability, and practicality. Specifically, while first-principles mechanistic models have a clear physicochemical and biological basis, they rely on a large number of kinetic parameters that are difficult to measure accurately and many idealized assumptions, leading to difficulties in model calibration, high computational complexity, and difficulty in adapting to the dynamic changes of actual working conditions. On the other hand, in data-driven methods, traditional machine learning algorithms often lack time-series modeling capabilities, have poor interpretability, and severely inadequate extrapolation performance; while deep learning models, although able to capture time dependencies, are still limited by their black-box nature and dependence on large amounts of labeled data, and cannot distinguish between correlation and causality between variables, resulting in significant biases in intervention predictions. Summary of the Invention

[0003] The purpose of this invention is to provide a simulation model training method and apparatus, a wastewater treatment method, equipment and medium, which can be applied to the field of environmental engineering. This method constructs a virtual environment with high realism and generalization ability through a wastewater treatment simulation scheme based on contrastive-adversarial imitation learning with causal feature decoupling, providing a reliable foundation for intelligent control of the entire wastewater treatment process.

[0004] To address the aforementioned technical problems, this invention provides a simulation model training method, comprising:

[0005] Acquire state parameters and action parameters during the wastewater treatment process; construct state nodes based on the state parameters, construct action nodes based on the action parameters, and construct intermediate nodes based on exogenous variables and constants.

[0006] A causal graph is constructed based on the state nodes, action nodes, and intermediate nodes. Decision nodes and inference nodes are determined based on the node relationships in the causal graph. A decision model for the decision nodes and an inference model for the inference nodes are constructed.

[0007] A generator is constructed based on the inference model and the decision model. A value model is constructed to score the output of the generator as a discriminator. The reward rule of the generator is set, and the value loss is determined with the goal of maximizing the reward rule.

[0008] Based on causal feature decoupling, determine the causal and non-causal state nodes of the action node and the intermediate node from the state nodes, and determine the causal decoupling loss based on the causal and non-causal state nodes; construct positive sample pairs and negative sample pairs of the state nodes, and determine the contrastive learning loss based on the positive sample pairs and negative sample pairs.

[0009] The target loss is determined based on the value loss, the causal decoupling loss, and the contrastive learning loss. The model parameters of the generator and the discriminator are updated based on the target loss. A simulation model is constructed based on the trained generator and the discriminator.

[0010] Optionally, determining decision nodes and inference nodes based on the node relationships in the causal graph includes:

[0011] When the parent nodes of a node are determined to be the state node and / or the intermediate node based on the node relationships in the cause-effect graph, the node is determined as the decision node.

[0012] When it is determined based on the node relationships in the causal graph that the parent node of the node contains the action node, the node is identified as the inference node.

[0013] Accordingly, the decision model is used to predict the action data for the current step based on the state data of the current step;

[0014] The reasoning model is used to predict the predicted state data for the next step based on the state data of the current step and the predicted action data.

[0015] The value model is used to output a score based on the state data of the current step, the predicted action data of the current step, and the predicted state data of the next step.

[0016] Optionally, the generator's reward rules are set to determine the value loss with the goal of maximizing the reward rules, including:

[0017] The generator is configured with a reward rule, and the PPO loss is determined based on the PPO algorithm with the goal of maximizing the reward rule. The reward rule includes: adversarial reward rule, absolute error reward rule, domain knowledge reward rule, contrastive learning reward rule, causal feature decoupling reward rule, and contrastive consistency reward rule.

[0018] Optionally, determining the causal decoupling loss based on the causal state nodes and the non-causal state nodes includes:

[0019] Construct counterfactual actions, and determine the counterfactual state data of the causal state nodes and the counterfactual state data of the non-causal state nodes based on the counterfactual actions;

[0020] The loss due to changes in causal characteristics is determined based on the counterfactual state data and predicted state data of the causal state nodes.

[0021] The non-causal feature invariant loss is determined based on the counterfactual state data and the predicted state data of the non-causal state nodes.

[0022] Based on the predicted state data of the causal state nodes, the prediction value loss is determined, and based on the prediction value loss and the value loss, the causal feature prediction value loss is determined.

[0023] The causal decoupling loss is determined based on the loss due to changes in causal features, the loss due to invariance of non-causal features, and the loss due to the predictive value of causal features.

[0024] Optionally, constructing positive and negative sample pairs for the state nodes, and determining the contrastive learning loss based on the positive and negative sample pairs, includes:

[0025] The positive sample pairs are constructed based on the true state data of the state nodes, and the negative sample pairs are constructed based on the counterfactual state data of the state nodes.

[0026] The contrastive learning loss is determined based on the positive sample pairs and the negative sample pairs.

[0027] Optionally, determining the causal and non-causal state nodes of the action node and the intermediate node from the state nodes based on causal feature decoupling includes:

[0028] Based on the action node and the intermediate node, determine the predicted state data and counterfactual state data of the state node;

[0029] Determine the characteristic variance of the predicted state data and the counterfactual state data, and sort the state nodes in descending order based on the characteristic variance;

[0030] The state nodes that account for a predetermined percentage are determined as causal state nodes, and the remaining state nodes are determined as non-causal state nodes.

[0031] To solve the above-mentioned technical problems, the present invention provides a wastewater treatment method, comprising:

[0032] The state data of the current step in the wastewater treatment process is acquired, and the state data of the current step is input into the generator of the simulation model to predict the predicted action data of the current step; wastewater treatment control is performed based on the predicted action data.

[0033] The current state data and the predicted action data are input into the generator of the simulation model to obtain the predicted state data for the next step.

[0034] The output data of the generator is input into the discriminator of the simulation model to obtain a score value;

[0035] The simulation model is a model trained according to the simulation model training method described above.

[0036] To solve the above-mentioned technical problems, the present invention provides a simulation model training device, comprising:

[0037] The first module is used to acquire state parameters and action parameters in the wastewater treatment process; construct state nodes based on the state parameters; construct action nodes based on the action parameters; and construct intermediate nodes based on exogenous variables and constants.

[0038] The second module is used to construct a causal graph based on the state nodes, the action nodes, and the intermediate nodes; determine decision nodes and inference nodes based on the node relationships in the causal graph; construct a decision model for the decision nodes; and construct an inference model for the inference nodes.

[0039] The third module is used to construct a generator based on the reasoning model and the decision model, construct a value model that scores the output of the generator as a discriminator, and set the reward rules of the generator to determine the value loss with the goal of maximizing the reward rules.

[0040] The fourth module is used to determine the causal and non-causal state nodes of the action node and the intermediate node from the state nodes based on causal feature decoupling, determine the causal decoupling loss based on the causal and non-causal state nodes, construct positive sample pairs and negative sample pairs of the state nodes, and determine the contrastive learning loss based on the positive sample pairs and the negative sample pairs.

[0041] The fifth module is used to determine the target loss based on the value loss, the causal decoupling loss, and the contrastive learning loss, update the model parameters of the generator and the discriminator based on the target loss, and construct a simulation model based on the trained generator and the discriminator.

[0042] To solve the above-mentioned technical problems, the present invention provides an electronic device, comprising:

[0043] Memory, used to store computer programs;

[0044] A processor is used to implement the simulation model training method or the wastewater treatment method described above when executing the computer program.

[0045] To address the aforementioned technical problems, the present invention provides a computer-readable storage medium storing computer-executable instructions. When these computer-executable instructions are executed by a processor, they implement the aforementioned simulation model training method or the aforementioned wastewater treatment method.

[0046] As can be seen, this invention acquires state and action parameters during the wastewater treatment process; constructs state nodes based on state parameters, action nodes based on action parameters, and intermediate nodes based on exogenous variables and constants; constructs a causal graph based on state nodes, action nodes, and intermediate nodes, and determines decision nodes and inference nodes based on the node relationships in the causal graph; constructs decision models for decision nodes and inference models for inference nodes; constructs a generator based on the inference model and decision model, and constructs a value model that scores the generator's output as a discriminator; sets reward rules for the generator, and determines the value loss with the goal of maximizing the reward rules; determines causal and non-causal state nodes of action nodes and intermediate nodes from the state nodes based on causal feature decoupling, and determines the causal decoupling loss based on the causal and non-causal state nodes; constructs positive and negative sample pairs for the state nodes, and determines the contrastive learning loss based on the positive and negative sample pairs; determines the target loss based on the value loss, causal decoupling loss, and contrastive learning loss; updates the model parameters of the generator and discriminator based on the target loss; and constructs a simulation model based on the trained generator and discriminator.

[0047] This invention proposes a wastewater treatment simulation scheme based on contrastive-adversarial imitation learning with causal feature decoupling, constructing a virtual environment with high realism and generalization ability, providing a reliable foundation for intelligent control of the entire wastewater treatment process. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0049] Figure 1 A flowchart illustrating a simulation model training method provided in an embodiment of the present invention;

[0050] Figure 2 A flowchart of a wastewater treatment method provided in an embodiment of the present invention;

[0051] Figure 3 This is a structural block diagram of a simulation model training device provided in an embodiment of the present invention;

[0052] Figure 4 This is a structural block diagram of a wastewater treatment device provided in an embodiment of the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] Monitoring the water environment is a crucial means of preventing and controlling water pollution. Continuous and effective monitoring enables water environment protection departments to stay informed about regional water quality, facilitating scientific prevention and control of water pollution and timely early warning of water environment deterioration. The construction of simulation systems for wastewater treatment processes has long faced significant challenges, primarily due to the high complexity, nonlinearity, time-varying nature, and uncertainty of these industrial processes. These characteristics limit the accuracy, generalization ability, and practicality of traditional simulation methods.

[0055] Specifically, while first-principles mechanistic models (such as activated sludge models) have a clear physicochemical and biological basis, they rely on a large number of kinetic parameters that are difficult to measure precisely and many idealized assumptions, leading to difficulties in model calibration, high computational complexity, and difficulty in adapting to the dynamic changes of actual working conditions. On the other hand, in data-driven methods, traditional machine learning algorithms (such as support vector machines and random forests) often lack time-series modeling capabilities, have poor interpretability, and severely inadequate extrapolation performance; while deep learning models (such as long short-term memory networks), although able to capture time dependencies, are still limited by their black-box nature, dependence on a large amount of labeled data, and inability to distinguish between correlation and causation between variables, resulting in significant biases in intervention predictions.

[0056] To address the aforementioned issues, this paper proposes a novel simulation framework based on Causal Feature Disentanglement based Contrastive-Adversarial Imitation Learning, aiming to achieve high-fidelity, interpretable simulation of wastewater treatment processes with causal reasoning capabilities.

[0057] This method involves three synergistic mechanisms:

[0058] (1) The causal feature decoupling module decomposes high-dimensional observation data into a set of low-dimensional, sparse, and physically meaningful latent factors (such as influent load characteristics, biological reactivity, hydraulic state, etc.) through an encoder-decoder structure, thereby explicitly constructing a structured causal graph model between process variables. This mechanism not only significantly improves the interpretability of the model, but also enables it to distinguish between confounding variables and true causal effects, thus supporting reliable extrapolation and counterfactual inference for unexperienced operating conditions.

[0059] (2) A contrastive learning strategy is introduced, which utilizes data augmentation to construct positive and negative sample pairs. By maximizing the similarity of representations among samples of the same type and reducing the mutual information between samples of different types, robust and discriminative feature representations are learned in unsupervised or weakly supervised environments. This approach significantly reduces the dependence on a large amount of precisely labeled data and improves the applicability of the model in data-scarce scenarios.

[0060] (3) Finally, an adversarial imitation learning architecture is adopted, in which the generator (simulation model) iteratively generates simulated time series data, while the discriminator evaluates the consistency of its distribution with the real historical data. Through the minimax game between the two, the model can realistically imitate the dynamic evolution of the real process. In addition, by embedding physical constraints (such as the law of conservation of matter and nonnegativity constraints), it can be further ensured that the generated sequence conforms to basic scientific principles and avoid the occurrence of non-physical prediction results.

[0061] This invention is not a simple improvement on traditional methods, but a paradigm shift. It aims to build a "digital twin" that can not only predict but also understand the internal causal mechanisms of a process. Its comprehensive advantages are reflected in multiple key dimensions such as interpretability, causal reasoning ability, extrapolation generalization, data efficiency, and physical consistency, thus providing a simulation foundation that is far more reliable and powerful than existing technologies for the intelligent operation, advanced control, and real-time optimization of wastewater treatment plants.

[0062] The following combination Figure 1 , Figure 1 A flowchart of a simulation model training method provided in an embodiment of the present invention, the method may include:

[0063] S101: Obtain the state parameters and action parameters in the wastewater treatment process; construct state nodes based on state parameters, construct action nodes based on action parameters, and construct intermediate nodes based on exogenous variables and constants.

[0064] This embodiment can obtain state parameters, action parameters, exogenous variables and constants in the wastewater treatment process.

[0065] This embodiment does not limit the specific type of state parameters, but can generally include indicators that reflect key operating parameters of the wastewater treatment process, such as influent flow rate, influent chemical oxygen demand, influent ammonia nitrogen, dissolved oxygen concentration in the biochemical reaction tank, pH value (acidity and alkalinity index), mixed liquor suspended solids concentration, nitrate nitrogen concentration, and effluent water quality indicators.

[0066] This embodiment does not limit the specific type of action parameters. Generally, it can record control commands executed by the operator or the underlying control system in history or simulation, including: internal reflux ratio setting, external reflux ratio setting, aeration fan frequency setting, carbon source dosage setting, etc.

[0067] In this embodiment, the exogenous variables and constants are parameters in the wastewater treatment system that are not affected by the action. That is, the values ​​of exogenous variables and constants do not change due to changes in action data. For example, exogenous variables can be external parameters of the wastewater treatment system, such as meteorological parameters such as ambient temperature and ambient light. Constants can be fixed parameters inside the wastewater treatment system, such as peak influent flow rate and peak effluent flow rate.

[0068] This embodiment can simultaneously collect real data of various parameters for simulation model training. To ensure the diversity and comprehensiveness of the data, this embodiment can collect real state data and real action data of state parameters and action parameters under multiple operating conditions. For example, the collected data can cover typical operating conditions such as different seasons, weather conditions, and water inflow loads (low load, high load, shock load). The data is stored in time series form, strictly preserving the temporal causal relationship between each parameter, ensuring that the state and action of one time step can affect the state of the next time step.

[0069] This embodiment can perform data cleaning operations on the collected data, including invalid and outlier removal, data smoothing, and timestamp-based alignment and synchronization.

[0070] Because the data for each variable has a different scale, the cleaned data is standardized. This embodiment does not limit the specific method of standardization; generally, Z-score standardization can be used, and the mean and standard deviation of the data are recorded during the standardization process.

[0071] This embodiment can use a graph structure to formally describe the complex causal relationships between variables in the wastewater treatment process. Therefore, this embodiment can construct state nodes based on state parameters, action nodes based on action parameters, and intermediate nodes based on exogenous variables and constants.

[0072] S102: Construct a cause-effect graph based on state nodes, action nodes, and intermediate nodes; determine decision nodes and inference nodes based on the node relationships in the cause-effect graph; construct decision models for decision nodes and inference models for inference nodes.

[0073] This embodiment can construct a cause-effect graph, i.e. a directed graph, based on state nodes, action nodes, and intermediate nodes to represent the entire wastewater treatment system.

[0074] Connecting nodes that influence each other with graph edges represents the dependencies and causal relationships between variables. For example, the current "aeration rate" action data and the current "dissolved oxygen concentration in the biochemical reactor" status data together determine the "dissolved oxygen concentration in the biochemical reactor" status data at the next moment.

[0075] This embodiment can determine decision nodes and inference nodes based on the node relationships in the causal graph. Specifically, when the parent nodes of a node determined based on the node relationships in the causal graph are all state nodes and / or intermediate nodes, the node is determined as a decision node; when the parent nodes of a node determined based on the node relationships in the causal graph include action nodes, the node is determined as an inference node.

[0076] In this embodiment, an independent neural network is allocated to each decision node and inference node to learn the complex mapping relationship (i.e., transfer function) from its parent node to that node. This involves constructing the decision model for the decision node and the inference model for the inference node. Specifically, the decision model is used to predict the predicted action data for the current step based on the current step's state data; the inference model is used to predict the predicted state data for the next step based on the current step's state data and the predicted action data.

[0077] For the decision node: its neural network π(a) t |s t This constitutes the policy part of the agent. The network uses the state data s of all its parent nodes at the current step t. t As input, output the motion data a for the current step t. t The probability distribution (e.g., Gaussian distribution, output mean and variance) can be sampled during training and deployment to generate actions.

[0078] For the inference node: its neural network f(s) t+1 |s t ,a t It is used to simulate the dynamics of a system. It learns the state data s at a given current step t. t and motion data a t In this case, predict the next value s of the state node. t+1 It is part of the environment simulation and is used to generate the rollout trajectory.

[0079] This embodiment does not limit the specific type of neural network for decision nodes and inference nodes. It can be set according to actual application. Generally, it can be a multilayer perceptron (MLP).

[0080] The entire agent system is composed of neural network models of all nodes in the causal graph. These networks work together during training, generating a complete trajectory in the graph through forward propagation and jointly optimizing through backpropagation.

[0081] S103: Construct a generator based on the reasoning model and decision model, and construct a value model that scores the generator's output as a discriminator; set the generator's reward rules and determine the value loss with the goal of maximizing the reward rules.

[0082] This embodiment can build a generator based on inference and decision models. The generator can output the predicted action data of the current step and the predicted state data of the next step based on the state data of the current step.

[0083] Furthermore, a value model is constructed to score the generator's output as a discriminator. In this embodiment, the value model is used to output a score based on the current step's state data, predicted action data, and the predicted state data for the next step. In this embodiment, the value model can also employ a multilayer perceptron architecture.

[0084] The value model can be used as a discriminator to determine the similarity between the generator's predicted data and the real data. The value model takes a triplet of the current step's state data, the current step's predicted action data, and the next step's predicted state data as input, and outputs a score value d[0,1]. This score value represents the similarity between the predicted data and the real data (i.e., the collected real state data and real action data). The closer the score value is to 1, the more similar the predicted data is to the real data; the closer the score value is to 0, the less similar the triplet is to the real data.

[0085] This implementation can perform adversarial learning training based on the generator and discriminator, with the training process alternating and iterating between the generator and discriminator.

[0086] When training the discriminator: keep the generator fixed and use real data (positive samples) and the predicted data (negative samples) output by the generator to train the discriminator so that it can better distinguish between the two.

[0087] When updating the generator: fix the discriminator and use the reward rule signal it provides, along with a series of auxiliary reward rules, to update the generator.

[0088] This embodiment can set the reward rules for the generator and determine the value loss with the goal of maximizing the reward rules. This value loss can be used to update the model parameters of the generator and the discriminator.

[0089] To improve the model's performance, this embodiment can introduce contrastive learning strategies and causal feature structure strategies within the framework of adversarial learning.

[0090] S104: Based on causal feature decoupling, determine the causal and non-causal state nodes of action nodes and intermediate nodes from the state nodes, and determine the causal decoupling loss based on the causal and non-causal state nodes; construct positive sample pairs and negative sample pairs of state nodes, and determine the contrastive learning loss based on the positive sample pairs and negative sample pairs.

[0091] This embodiment can determine the causal and non-causal state nodes of action nodes and intermediate nodes from the state nodes based on causal feature decoupling. That is, it can determine the causal state parameters that have a causal relationship with action parameters, exogenous variables and constants, and the non-causal state parameters that do not have a causal relationship or have a weak causal relationship from the state parameters.

[0092] This embodiment does not limit the specific method of causal feature decoupling; generally, causal feature decoupling can be performed based on feature variance. Specifically, the predicted state data and counterfactual state data of the state nodes are determined based on the action nodes and intermediate nodes; the feature variance of the predicted state data and counterfactual state data is determined, and the state nodes are sorted in descending order based on the feature variance; the state nodes with the first preset percentage (e.g., 50%) are determined as causal state nodes, and the remaining state nodes are determined as non-causal state nodes.

[0093] In this embodiment, the counterfactual state data can be obtained from the counterfactual action data. That is, for the inference model, the input predicted action data is perturbed to obtain the counterfactual action data, and the current state data and the counterfactual action data are used as inputs to obtain the output counterfactual state data.

[0094] In this embodiment, the variance of each feature can be calculated and sorted using a statistical causal mask. The top 50% of the feature variables are selected as causal states with significant accompanying effects of the action and their masks are set to 1. The remaining variables are set to masks of 0.

[0095] Suppose that state node s can be decomposed into causal state node c and non-causal state node n: s = c ⊕ n;

[0096] The goal is to learn a causal mask M (based on feature variance) such that: causal state nodes c=M⊙s are sensitive to actions, and non-causal state nodes n=(1-M)⊙s are invariant to actions.

[0097] This embodiment uses feature variance as the criterion for causal features, which can achieve feature decoupling without complex neural network structures. It decomposes state variables into causal and non-causal variables, improves the model's influence on causal variables, maintains the model's stability on non-causal variables, and has high computational efficiency and is easy to interpret.

[0098] This embodiment can determine the causal decoupling loss based on causal state nodes and non-causal state nodes. Specifically, it constructs counterfactual actions and determines the counterfactual state data of causal state nodes and non-causal state nodes based on the counterfactual actions.

[0099] The loss for causal feature changes is determined based on the counterfactual state data and predicted state data of causal state nodes, encouraging the state data of causal state nodes to change with actions, as shown in the following equation:

[0100] ;

[0101] In the formula, L causal-var The loss is defined as the causal feature change loss, where c represents the causal state node, and a represents the predicted action data. cf c(a) represents the counterfactual action data, and c(a) represents the predicted state data of the causal state nodes. cf ) represents the counterfactual state data of the causal state nodes.

[0102] The non-causal feature invariant loss is determined based on the counterfactual state data and predicted state data of non-causal state nodes, encouraging the state data of non-causal state nodes to remain unchanged with actions, as shown in the following equation:

[0103] ;

[0104] In the formula, L noncausal-inv The loss is defined as the causal feature change loss, where n represents a non-causal state node, n(a) represents the predicted state data of the non-causal state node, and n(a) represents the predicted state data of the non-causal state node. cf ) represents the counterfactual state data of non-causal state nodes.

[0105] The prediction value loss is determined based on the prediction state data of causal state nodes. The prediction value loss and the value loss are then used to determine the causal feature prediction value loss, which is used to characterize the accuracy of the causal decoupling result, as shown in the following formula:

[0106] ;

[0107] In the formula, L value-pred Let R be the value loss for causal feature prediction, and V(c(a)) be the value loss determined based on the predicted state data of causal state nodes during adversarial learning training.

[0108] The causal decoupling loss is determined based on the loss due to changes in causal features, the loss due to invariance of non-causal features, and the loss due to the predictive value of causal features, as shown in the following formula:

[0109] ;

[0110] In the formula, L causal Loss due to causal decoupling.

[0111] This embodiment can construct positive and negative sample pairs for state nodes, and determine the contrastive learning loss based on the positive and negative sample pairs. Specifically, positive sample pairs can be constructed based on the real state data of state nodes, and negative sample pairs can be constructed based on the counterfactual state data of causal state nodes. The contrastive learning loss function can then be expressed as follows:

[0112] ;

[0113] In the formula, L contrast To compare learning loss, E is the mathematical expectation, and D... Generator s represents the distribution of the data output by the generator. t (a) is the predicted state data of the predicted action data a at the current time step t, s t (a * (a) represents the actual motion data at the current time step t. * Real state data, s t (a cf The counterfactual action data a at the current time step t cf Counterfactual state data, N t Let g be all counterfactual state data at the current time step t, g be the projection head, sim be the cosine similarity function, and τ be the temperature parameter used to control the sharpness of the distribution.

[0114] Furthermore, considering the possibility that the true state data at the current time step may be missing, this embodiment can determine the true state data from other time steps that is highly similar to the predicted state data at the current time step as the positive sample for the current time step. Therefore, the expression for the contrastive learning loss function can be:

[0115] ;

[0116] In the formula, t is the current time step, t' is the time step different from time step t, and s t (a) shows the predicted state data at time step t, s t' (a * () represents the actual state data at time step t', and m represents the distribution function, such as mean and variance. The indicator function is 1 when the distribution of the predicted state data at time step t is highly similar to that of the actual state data at time step t'; otherwise, it is 0.

[0117] This embodiment does not limit the specific method for determining the distribution high similarity. Generally, a similarity threshold can be set. Within a preset time window, when there is real state data with a distribution similarity greater than the similarity threshold, the real state data with the largest distribution similarity is taken for cosine similarity calculation.

[0118] This embodiment not only considers enhancing the similarity between positive sample pairs and model prediction data, i.e. enhancing the agent's perception of the inheritance of temporal causal states, but also considers weakening the similarity between negative samples and model prediction data, encouraging the model to learn causal representations that are invariant to time changes, thereby enhancing the model's feature discrimination ability.

[0119] S105: Determine the target loss based on value loss, causal decoupling loss, and contrastive learning loss; update the model parameters of the generator and discriminator based on the target loss; and construct a simulation model based on the trained generator and discriminator.

[0120] In the value loss calculation process, this embodiment can set the generator's reward rules to determine the value loss with the goal of maximizing the reward rules. This embodiment does not limit the calculation method of value loss; generally, the generator's reward rules can be set, and the PPO loss can be determined based on the PPO (Proximal Policy Optimization) algorithm with the goal of maximizing the reward rules.

[0121] This embodiment does not limit the setting method of reward rules. Generally, reward rules may include: adversarial reward rules, absolute error reward rules, domain knowledge reward rules, contrastive learning reward rules, causal feature decoupling reward rules, and contrast consistency reward rules.

[0122] Adversarial reward rule: The higher the score of the discriminator, the greater the adversarial reward;

[0123] Absolute error reward rule: The smaller the absolute error between the generator's predicted state data and the actual state data, and / or the smaller the absolute error between the generator's predicted action data and the actual action data, the greater the absolute error reward.

[0124] Domain knowledge reward rule: Inject domain knowledge; the more the generator's prediction data matches the domain knowledge, the greater the domain knowledge reward.

[0125] Contrastive learning reward rule: The higher the similarity between the generator's predicted state data and the real state data, and the greater the difference between the generator's predicted state data and the counterfactual state data, the greater the contrastive learning reward; generally, the negative of the contrastive learning loss function can be used as the reward function for contrastive learning.

[0126] The reward rule for causal feature decoupling is as follows: the greater the influence of action data changes on the state data of the causal state nodes obtained by causal feature decoupling, and the smaller the influence of action data changes on the state data of the causal state nodes, the greater the reward for causal feature decoupling. Generally, the negative of the causal feature decoupling loss function can be used as the reward function for causal feature decoupling.

[0127] The consistency reward rule is as follows: For predicted action data, actual action data, and counterfactual action data, the greater the difference between the corresponding predicted state data and the counterfactual state data, and the greater the difference between the corresponding predicted state data and the actual state data, the greater the consistency reward. By ensuring logical consistency in counterfactual predictions, the model's causal reasoning ability is improved, making the learned strategy more reasonable and stable.

[0128] This embodiment can construct a reward function based on the above reward rules, and calculate the value loss of the value function based on the reward function. In this embodiment, the value loss can be the PPO loss L. ppo .

[0129] This embodiment can determine the target loss based on value loss, causal decoupling loss, and contrastive learning loss, as shown in the following formula:

[0130] ;

[0131] In the formula, L total For the target loss, L causal For causal decoupling loss, L contrast To compare learning loss, L ppo For PPO loss, λ contrast To compare the learning loss weights, λ causal The loss weight is used for causal decoupling.

[0132] This embodiment can update the model parameters of the generator and discriminator based on the target loss, and build a simulation model based on the trained generator and discriminator. Specifically, the gradients of the generator model parameters and the discriminator model parameters can be calculated based on the target loss, and the model parameters can be updated through the backpropagation algorithm.

[0133] The above process is followed to iterate and train the model until the parameters converge, resulting in a trained generator and discriminator. A simulation model is then built based on the trained generator and discriminator.

[0134] Based on the above embodiments, this invention proposes a wastewater treatment simulation scheme based on contrastive-adversarial imitation learning with causal feature decoupling, constructing a virtual environment with high realism and generalization ability, providing a reliable foundation for intelligent control of the entire wastewater treatment process.

[0135] This invention employs a multi-paradigm fusion framework design, integrating three advanced paradigms from different subfields of artificial intelligence: causal reasoning, contrastive learning, and adversarial imitation learning, to construct a unified and collaborative learning framework. This fusion is not a simple stacking of paradigms, but rather aims to solve specific problems that cannot be addressed by a single paradigm.

[0136] This invention focuses on the construction of simulation models for "intervention": the core objective of the model is elevated from traditional "correlation prediction" to "causal intervention prediction," that is, it can answer the core question in process optimization: "How will the system change if a certain operation (such as increasing aeration) is performed?" This transforms the model from a passive prediction tool into an active decision support tool.

[0137] This invention is based on decoupled causal discovery and representation learning: a novel loss function structure is designed that can automatically identify and decouple potential causal factors from time series data under weak or unsupervised conditions, while ensuring the independence and interpretability of these factors.

[0138] The adversarial training mechanism of physical constraint embedding in this invention introduces physical rules based on domain prior knowledge as hard constraints or soft regularization terms in the adversarial training between the generator and the discriminator, ensuring that the generated simulation data is not only statistically realistic, but also physically reasonable.

[0139] This invention presents a contrastive learning strategy for time-series data: a method specifically designed for constructing contrastive learning sample pairs for non-stationary time-series data. For example, true-state data is used as positive sample pairs, and counterfactual state data is used as negative sample pairs, thereby better learning time-invariant causal representations.

[0140] This invention addresses the problem of few-shot learning: by leveraging the self-supervised nature of contrastive learning and the stability of causal representation, the model can learn from limited labeled data, effectively alleviating the dependence of deep learning on large amounts of high-quality labeled data, and better reflecting the actual situation of scarce wastewater treatment data.

[0141] The invention has excellent generalization and extrapolation capabilities: because the model learns stable, transferable causal mechanisms rather than superficial statistical correlations, it can still maintain high prediction reliability when the system operating conditions change or the influent water quality exceeds the historical range (distribution extrageneralization).

[0142] This invention provides a core engine for advanced control and real-time optimization: the model can serve as a high-precision "digital twin" for testing, evaluating, and training advanced control algorithms (such as model predictive control), or for safe offline operational optimization simulation, thereby reducing the risks and costs of actual trial and error.

[0143] This invention provides a new paradigm for the simulation of complex industrial processes: This research transcends the traditional dichotomy of "mechanism-driven" or "purely data-driven" and pioneers a new paradigm of "cause-guided data-driven", providing a blueprint for simulation modeling in other similar fields (such as chemical engineering, pharmaceuticals, and energy).

[0144] This invention realizes a closed loop of "perception-understanding-reasoning": the model can not only "perceive" data (such as sensor readings), but also "understand" the causal mechanism behind the data, and on this basis, "reason" (counterfactual, intervention query), taking a key step towards having cognitive intelligence for industrial processes.

[0145] The following combination Figure 2 , Figure 2 A flowchart of a wastewater treatment method provided in an embodiment of the present invention may include:

[0146] S201: Obtain the state data of the current step in the wastewater treatment process, input the state data of the current step into the generator of the simulation model to predict the predicted action data of the current step; and perform wastewater treatment control based on the predicted action data.

[0147] S202: Input the current state data and predicted action data into the generator of the simulation model to obtain the predicted state data for the next step;

[0148] S203: Input the generator's output data into the discriminator of the simulation model to obtain a score;

[0149] The simulation model is a model trained according to the simulation model training method.

[0150] Existing wastewater treatment process simulation methods are mostly based on mechanistic models or purely data-driven black-box models, which have the following problems:

[0151] (1) Insufficient simulation fidelity: Traditional mechanistic models rely on simplified assumptions and are difficult to accurately characterize complex biochemical reactions and microbial behavior; pure data-driven models lack causal explanatory power and have poor generalization ability when operating conditions change abruptly.

[0152] (2) Strong multivariate coupling: There are many variables in the process of sewage treatment, and there are complex relationships such as nonlinearity, time-varying and coupling. Traditional methods are difficult to effectively decouple the causal relationship between these variables.

[0153] (3) High risk of strategy training: If the control strategy (such as reinforcement learning) is trained directly in the real system, the immature strategy may lead to operational risks such as excessive water output and equipment damage, and there is a lack of safe and efficient training environment.

[0154] (4) Weak ability to cope with extreme working conditions: Existing simulation models have limited ability to simulate extreme working conditions such as water inrush, toxic substances, and equipment failure, resulting in insufficient ability of intelligent systems to cope with abnormal situations in practical applications.

[0155] To this end, we propose an environmental simulation method that integrates causal inference, contrastive learning, and adversarial imitation learning. The aim is to construct a virtual environment that can accurately reflect the inherent causal relationships of the process and has high realism and generalization ability, providing a reliable foundation for the training and optimization of intelligent control systems for the entire wastewater treatment process.

[0156] In this embodiment, the simulation model can be a model trained by a simulation model training method based on causal feature decoupling. After the model is trained, it can be deployed for wastewater treatment control.

[0157] In this embodiment, the data input to the model can be normalized based on the normalization parameters during the training process, and the data output by the model can be denormalized based on the normalization parameters to restore the data.

[0158] This embodiment can acquire the state data of the current step in the wastewater treatment process, input the state data of the current step into the generator of the simulation model to predict the predicted action data of the current step, and perform wastewater treatment control based on the predicted action data.

[0159] In this embodiment, the current state data and predicted action data are input into the generator of the simulation model to obtain the predicted state data for the next step; the generator's output data is then input into the discriminator of the simulation model to obtain a score. The score obtained based on the value model can be used to evaluate the reliability of the predicted action data.

[0160] Based on the above embodiments, this invention proposes a wastewater treatment simulation scheme based on contrastive-adversarial imitation learning with causal feature decoupling, constructing a virtual environment with high realism and generalization ability, providing a reliable foundation for intelligent control of the entire wastewater treatment process.

[0161] Figure 3 This is a structural block diagram of a simulation model training device provided in an embodiment of the present invention. The device may include:

[0162] The first module 100 is used to acquire state parameters and action parameters in the wastewater treatment process; construct state nodes based on state parameters, action nodes based on action parameters, and intermediate nodes based on exogenous variables and constants.

[0163] The second module 200 is used to construct a cause-effect graph based on state nodes, action nodes, and intermediate nodes; determine decision nodes and inference nodes based on the node relationships in the cause-effect graph; construct decision models for decision nodes; and construct inference models for inference nodes.

[0164] The third module 300 is used to construct a generator based on a reasoning model and a decision model, construct a value model that scores the output of the generator as a discriminator, and set the reward rules of the generator to determine the value loss with the goal of maximizing the reward rules.

[0165] The fourth module 400 is used to determine the causal and non-causal state nodes of action nodes and intermediate nodes from the state nodes based on causal feature decoupling, and to determine the causal decoupling loss based on the causal and non-causal state nodes; to construct positive and negative sample pairs of state nodes, and to determine the contrastive learning loss based on the positive and negative sample pairs.

[0166] The fifth module, 500, is used to determine the target loss based on value loss, causal decoupling loss, and contrastive learning loss, update the model parameters of the generator and discriminator based on the target loss, and build a simulation model based on the trained generator and discriminator.

[0167] Based on the above embodiments, this invention proposes a wastewater treatment simulation scheme based on contrastive-adversarial imitation learning with causal feature decoupling, constructing a virtual environment with high realism and generalization ability, providing a reliable foundation for intelligent control of the entire wastewater treatment process.

[0168] Based on the above embodiments, the second module 200 may include:

[0169] The first unit is used to determine a node as a decision node when the parent nodes of a node are all state nodes and / or intermediate nodes, based on the node relationships in the cause-effect graph.

[0170] The second unit is used to determine a node as a reasoning node when the parent node of a node contains an action node, based on the node relationships in the causal graph.

[0171] Correspondingly, the decision model is used to predict the action data for the current step based on the state data of the current step;

[0172] The inference model is used to predict the state data for the next step based on the state data of the current step and the predicted action data.

[0173] The value model is used to output a score based on the state data of the current step, the predicted action data of the current step, and the predicted state data of the next step.

[0174] Based on the above embodiments, the third module 300 may include:

[0175] The third unit is used to set the reward rules for the generator, and to determine the PPO loss based on the PPO algorithm with the goal of maximizing the reward rules. The reward rules include: adversarial reward rules, absolute error reward rules, domain knowledge reward rules, contrastive learning reward rules, causal feature decoupling reward rules, and contrastive consistency reward rules.

[0176] Based on the above embodiments, the fourth module 400 may include:

[0177] The fourth unit is used to construct counterfactual actions, and to determine the counterfactual state data of causal state nodes and the counterfactual state data of non-causal state nodes based on the counterfactual actions.

[0178] The fifth unit is used to determine the loss of causal feature changes based on counterfactual state data and predicted state data of causal state nodes;

[0179] The sixth unit is used to determine the non-causal feature invariant loss based on counterfactual state data and predicted state data of non-causal state nodes.

[0180] The seventh unit is used to determine the prediction value loss based on the prediction state data of causal state nodes, and to determine the prediction value loss of causal features based on the prediction value loss and the value loss.

[0181] Unit 8 is used to determine the causal decoupling loss based on the loss due to changes in causal features, the loss due to invariance of non-causal features, and the loss due to the predictive value of causal features.

[0182] Based on the above embodiments, the fourth module 400 may include:

[0183] Unit 9 is used to construct positive sample pairs based on the real state data of state nodes and negative sample pairs based on the counterfactual state data of state nodes.

[0184] Unit 10 is used to determine the contrastive learning loss based on positive and negative sample pairs.

[0185] Based on the above embodiments, the fourth module 400 may include:

[0186] Unit 11 is used to determine the predicted state data and counterfactual state data of state nodes based on action nodes and intermediate nodes.

[0187] The twelfth unit is used to determine the characteristic variance of the predicted state data and the counterfactual state data, and to sort the state nodes in descending order based on the characteristic variance.

[0188] The thirteenth unit is used to determine the state nodes with the pre-set percentage as causal state nodes and the remaining state nodes as non-causal state nodes.

[0189] Figure 4 This is a structural block diagram of a wastewater treatment device provided in an embodiment of the present invention. The device may include:

[0190] The sixth module 600 is used to acquire the status data of the current step in the wastewater treatment process, input the status data of the current step into the generator of the simulation model to predict the predicted action data of the current step, and perform wastewater treatment control based on the predicted action data.

[0191] The seventh module 700 is used to input the current step's state data and predicted action data into the generator of the simulation model to obtain the predicted state data for the next step.

[0192] The eighth module 800 is used to input the generator's output data into the discriminator of the simulation model to obtain a score value;

[0193] The simulation model is the model trained by the simulation model training device.

[0194] Based on the above embodiments, the present invention also provides an electronic device, which may include a memory and a processor. The memory stores a computer program, and when the processor calls the computer program in the memory, it can implement the steps provided in the above embodiments. Of course, the device may also include various necessary network interfaces, a power supply, and other components.

[0195] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by an execution terminal or processor, can implement the method provided in the embodiments of the present invention; the storage medium may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0196] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A simulation model training method, characterized in that, include: Acquire state and action parameters during the wastewater treatment process; State nodes are constructed based on the state parameters, action nodes are constructed based on the action parameters, and intermediate nodes are constructed based on exogenous variables and constants. A causal graph is constructed based on the state nodes, action nodes, and intermediate nodes. Decision nodes and inference nodes are determined based on the node relationships in the causal graph. A decision model for the decision nodes and an inference model for the inference nodes are constructed. A generator is constructed based on the inference model and the decision model, and a value model that scores the output of the generator is constructed as a discriminator. Set the reward rules for the generator to determine the value loss with the goal of maximizing the reward rules; Based on causal feature decoupling, determine the causal and non-causal state nodes of the action node and the intermediate node from the state nodes, and determine the causal decoupling loss based on the causal and non-causal state nodes; Constructing positive and negative sample pairs for the state nodes, and determining the contrastive learning loss based on the positive and negative sample pairs; including: constructing the positive sample pairs based on the true state data of the state nodes, constructing the negative sample pairs based on the counterfactual state data of the state nodes; and determining the contrastive learning loss based on the positive and negative sample pairs. The target loss is determined based on the value loss, the causal decoupling loss, and the contrastive learning loss. The model parameters of the generator and the discriminator are updated based on the target loss. A simulation model is constructed based on the trained generator and the discriminator. The expression for the contrastive learning loss function is: ； In the formula, L contrast Let E be the contrastive learning loss, and D be the mathematical expectation. Generator s represents the distribution of the data output by the generator. t (a) is the predicted state data of the predicted action data a at the current time step t, s t (a * (a) represents the actual motion data at the current time step t. * Real state data, s t (a cf The counterfactual action data a at the current time step t cf Counterfactual state data, N t For the current time step t, there are all counterfactual state data, g is the projection head, sim is the cosine similarity function, and τ is the temperature parameter used to control the sharpness of the distribution. The target loss is shown in the following formula: ； In the formula, L total For the target loss, L causal For the causal decoupling loss, L contrast For the contrastive learning loss, L ppo For PPO loss, λ contrast To compare the learning loss weights, λ causal The loss weight is used for causal decoupling.

2. The simulation model training method according to claim 1, characterized in that, Determining decision nodes and inference nodes based on the node relationships in the causal graph includes: When the parent nodes of a node are determined to be the state node and / or the intermediate node based on the node relationships in the cause-effect graph, the node is determined as the decision node. When it is determined based on the node relationships in the causal graph that the parent node of the node contains the action node, the node is identified as the inference node. Accordingly, the decision model is used to predict the action data for the current step based on the state data of the current step; The reasoning model is used to predict the predicted state data for the next step based on the state data of the current step and the predicted action data. The value model is used to output a score based on the state data of the current step, the predicted action data of the current step, and the predicted state data of the next step.

3. The simulation model training method according to claim 1, characterized in that, Setting the reward rules for the generator, and determining the value loss with the goal of maximizing the reward rules, including: The generator is configured with a reward rule, and the PPO loss is determined based on the PPO algorithm with the goal of maximizing the reward rule. The reward rule includes: adversarial reward rule, absolute error reward rule, domain knowledge reward rule, contrastive learning reward rule, causal feature decoupling reward rule, and contrastive consistency reward rule.

4. The simulation model training method according to claim 1, characterized in that, Determining the causal decoupling loss based on the causal state nodes and the non-causal state nodes includes: Construct counterfactual actions, and determine the counterfactual state data of the causal state nodes and the counterfactual state data of the non-causal state nodes based on the counterfactual actions; The loss due to changes in causal characteristics is determined based on the counterfactual state data and predicted state data of the causal state nodes. The non-causal feature invariant loss is determined based on the counterfactual state data and the predicted state data of the non-causal state nodes. Based on the predicted state data of the causal state nodes, the prediction value loss is determined, and based on the prediction value loss and the value loss, the causal feature prediction value loss is determined. The causal decoupling loss is determined based on the loss due to changes in causal features, the loss due to invariance of non-causal features, and the loss due to the predictive value of causal features.

5. The simulation model training method according to claim 1, characterized in that, Based on causal feature decoupling, the causal and non-causal state nodes of the action node and the intermediate node are determined from the state nodes, including: Based on the action node and the intermediate node, determine the predicted state data and counterfactual state data of the state node; Determine the characteristic variance of the predicted state data and the counterfactual state data, and sort the state nodes in descending order based on the characteristic variance; The state nodes that account for a predetermined percentage are determined as causal state nodes, and the remaining state nodes are determined as non-causal state nodes.

6. A wastewater treatment method, characterized in that, include: The state data of the current step in the wastewater treatment process is acquired, and the state data of the current step is input into the generator of the simulation model to predict the predicted action data of the current step; wastewater treatment control is performed based on the predicted action data. The current state data and the predicted action data are input into the generator of the simulation model to obtain the predicted state data for the next step. The output data of the generator is input into the discriminator of the simulation model to obtain a score value; The simulation model is a model trained according to the simulation model training method according to any one of claims 1 to 5.

7. A simulation model training device, characterized in that, include: The first module is used to acquire state parameters and action parameters during the wastewater treatment process; State nodes are constructed based on the state parameters, action nodes are constructed based on the action parameters, and intermediate nodes are constructed based on exogenous variables and constants. The second module is used to construct a causal graph based on the state nodes, the action nodes, and the intermediate nodes; determine decision nodes and inference nodes based on the node relationships in the causal graph; construct a decision model for the decision nodes; and construct an inference model for the inference nodes. The third module is used to construct a generator based on the reasoning model and the decision model, and to construct a value model that scores the output of the generator as a discriminator. Set the reward rules for the generator to determine the value loss with the goal of maximizing the reward rules; The fourth module is used to determine the causal and non-causal state nodes of the action node and the intermediate node from the state nodes based on causal feature decoupling, and to determine the causal decoupling loss based on the causal state nodes and the non-causal state nodes. Constructing positive and negative sample pairs for the state nodes, and determining the contrastive learning loss based on the positive and negative sample pairs; including: constructing the positive sample pairs based on the true state data of the state nodes, constructing the negative sample pairs based on the counterfactual state data of the state nodes; and determining the contrastive learning loss based on the positive and negative sample pairs. The fifth module is used to determine the target loss based on the value loss, the causal decoupling loss and the contrastive learning loss, update the model parameters of the generator and the discriminator based on the target loss, and construct a simulation model based on the trained generator and discriminator. The expression for the contrastive learning loss function is: ； In the formula, L contrast Let E be the contrastive learning loss, and D be the mathematical expectation. Generator s represents the distribution of the data output by the generator. t (a) is the predicted state data of the predicted action data a at the current time step t, s t (a * (a) represents the actual motion data at the current time step t. * Real state data, s t (a cf The counterfactual action data a at the current time step t cf Counterfactual state data, N t For the current time step t, there are all counterfactual state data, g is the projection head, sim is the cosine similarity function, and τ is the temperature parameter used to control the sharpness of the distribution. The target loss is shown in the following formula: ； In the formula, L total For the target loss, L causal For the causal decoupling loss, L contrast For the contrastive learning loss, L ppo For PPO loss, λ contrast To compare the learning loss weights, λ causal The loss weight is used for causal decoupling.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the simulation model training method as described in any one of claims 1 to 5, or the wastewater treatment method as described in claim 6, when executing the computer program.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the simulation model training method as described in any one of claims 1 to 5, or the wastewater treatment method as described in claim 6.

Images