Wastewater treatment environment modeling and control method and device based on joint embedding prediction, equipment and medium

Abstract

This application discloses a method, apparatus, equipment, and medium for wastewater treatment environment modeling and control based on joint embedding prediction, relating to the field of artificial intelligence technology. It generates an environment model based on a joint embedding prediction architecture, updates the environment model using a first total loss function to obtain the model's hyperparameters, defines a state space, action space, reward function, and termination condition, inputs the trajectory action into a virtual simulation environment model, and outputs the state embedding prediction result for the next moment, the virtual simulation environment state, and the reward value. If the current state meets the preset termination condition, the trajectory data is determined. A second total loss function is constructed based on the trajectory data, and the parameters of the initial policy network and initial value network are updated using the second total loss function to obtain the policy network structure and parameters. The corresponding target policy network is deployed to the wastewater treatment plant control system to achieve wastewater treatment control, solving the problems of unstable model training and reliance on complex heuristic techniques.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, equipment and medium for wastewater treatment environment modeling and control based on joint embedding prediction. Background Technology

[0002] Long-standing water pollution problems have severely disrupted the balance of aquatic ecosystems. Upgrading wastewater treatment plants to intelligent systems and implementing automated control are crucial for improving effluent quality, reducing energy consumption, and minimizing human intervention. However, directly training and optimizing intelligent control strategies in actual wastewater treatment plants carries significant risks: 1. Existing simulation models are unstable during training and rely heavily on heuristics: Current advanced environmental simulation methods are very fragile during training. To prevent models from learning "shortcuts" or experiencing "representation collapse" (i.e., the model maps all inputs to almost the same output, thus failing to distinguish between different states), they have to rely on many heuristics lacking theoretical guarantees. This makes the model training process extremely unstable, and hyperparameters (such as the weights of each loss term) are extremely sensitive, requiring a significant amount of time and computing power for grid search. When applied to complex and variable industrial scenarios such as wastewater treatment, the fragility of these heuristics is further amplified, leading to poor convergence or generalization ability of the model. 2. Existing models lack interpretability and causal reasoning ability: Many purely data-driven "black box" models, while capable of capturing statistical patterns in time series, have internal decision-making processes that are difficult to understand. More importantly, these models can only learn the correlations between variables but cannot distinguish causality. When control strategies change, or when operational methods differ from historical data distributions, this statistical correlation collapses, leading to severe biases in intervention predictions and a sharp decline in the model's predictive reliability.

[0003] 3. Existing end-to-end JEPA (Joint Embedding Predictive Architecture) methods lack stability guarantees and are difficult to implement in industrial scenarios: JEPA itself is an ideal blueprint for building environmental models because it directly predicts the "embedded representation" of future states, rather than reconstructing the original high-dimensional data, thus automatically ignoring details irrelevant to the prediction task (such as background noise, sensor errors, etc.). However, existing end-to-end JEPA methods still face the representation collapse problem during training, i.e., the model finds that "mapping all states to the same point" can perfectly minimize the prediction error, thus abandoning the learning of any meaningful state distinctions. To prevent collapse, PLDM introduces a large number of additional regularization terms, making the training objective extremely complex, and still cannot completely guarantee that collapse will not occur. These techniques do not theoretically solve the root cause of collapse, resulting in complex model design, unstable training, and hyperparameter sensitivity, making it difficult to implement in industrial scenarios with extremely high reliability requirements, such as wastewater treatment.

[0004] As can be seen from the above, how to solve the problems of unstable model training and reliance on complex heuristic techniques, improve the reliability and accuracy of model predictions, and reduce algorithm complexity are problems that need to be solved in this field. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method, apparatus, equipment, and medium for wastewater treatment environmental modeling and control based on joint embedding prediction, which can solve the problems of unstable model training and reliance on complex heuristic techniques, improve the reliability and accuracy of model prediction, and reduce algorithm complexity. The specific solution is as follows: In a first aspect, this application discloses a wastewater treatment environmental modeling and control method based on joint embedding prediction, including: Continuous time series data from wastewater treatment plants are preprocessed, and training datasets are constructed using the preprocessed data. Based on the joint embedding prediction architecture, an environment model is generated. Randomly sampled data from the training dataset is input into the environment model, and the current state embedding vector, the actual state embedding vector and the predicted state embedding vector are output. Calculate the embedding vector loss value between the actual state embedding vector and the predicted state embedding vector at the next time step, project the current state embedding vector onto the unit direction vector to obtain a one-dimensional projection sequence, use the one-dimensional projection sequence to determine the isotropic Gaussian regularization loss value, and determine the first total loss function based on the embedding vector loss value and the isotropic Gaussian regularization loss value. The model parameters in the environment model are updated using the first total loss function. The process of inputting random sampled data into the environment model is repeated until the first total loss function of the environment model converges on the validation set, and the hyperparameters of the model configuration are obtained. Based on the hyperparameters, a virtual simulation environment model is constructed, defining the state space of the virtual simulation environment, the action space of the agent in the virtual simulation environment, the reward function for evaluating the agent's behavior, and the termination condition for the operation of the virtual simulation environment. An initial policy network and an initial value network are constructed for the defined virtual simulation environment model. Trajectory actions are sampled using the initial policy network and input into the defined virtual simulation environment model. The state embedding prediction result, virtual simulation environment state, and reward value of the next time step are output. If the current time step meets the preset termination condition, the trajectory data is determined based on the state embedding prediction result, virtual simulation environment state, and reward value of the next time step. A second total loss function is constructed based on trajectory data. The parameters of the initial policy network and the initial value network are updated using the second total loss function. The process of sampling to obtain trajectory actions is repeated until the second total loss function converges, and the policy network structure and parameters are obtained. The target strategy network corresponding to the strategy network structure and parameters is deployed to the wastewater treatment plant control system so that the target strategy network can be used to generate and issue control actions to the actuators to achieve control of wastewater treatment.

[0006] Optionally, the preprocessing of continuous time series data from the wastewater treatment plant, and the construction of a training dataset using the preprocessed data, includes: Collect continuous time series data from the historical database or data acquisition and monitoring control system of the wastewater treatment plant; The continuous time series data undergoes outlier removal, missing value imputation, timestamp alignment, data slicing, and data standardization to obtain preprocessed data. The preprocessed data is integrated into triplet samples according to a preset format to construct a training dataset.

[0007] Optionally, the joint embedding prediction architecture consists of an encoder and a predictor; the encoder and predictor employ a multilayer perceptron and / or Transformer architecture. Accordingly, the step of inputting randomly sampled data from the training dataset into the environment model and outputting the current state embedding vector, the actual state embedding vector at the next time step, and the predicted state embedding vector includes: Random sampling is performed on the training dataset to obtain random sample data; The random sampled data is input into the encoder in the environment model to output the current state embedding vector and the actual state embedding vector at the next moment; The current state embedding vector and the action data from the randomly sampled data are concatenated, and the concatenated data is input into the predictor in the environment model to output the predicted state embedding vector for the next time step.

[0008] Optionally, the step of calculating the embedding vector loss value between the actual state embedding vector and the predicted state embedding vector at the next time step, projecting the current state embedding vector onto a unit direction vector to obtain a one-dimensional projection sequence, and using the one-dimensional projection sequence to determine the isotropic Gaussian regularization loss value includes: Calculate the mean square error between the actual state embedding vector and the predicted state embedding vector at the next time step, and use the mean square error as the embedding vector loss value; The unit direction vector is obtained by randomly sampling the unit hypersphere, which includes the direction vector. Project the current state embedding vector onto the unit direction vector to obtain a one-dimensional projection sequence; Calculate the test statistic between the one-dimensional projected sequence and the standard normal distribution; The isotropic Gaussian regularization loss value is calculated using the test statistic.

[0009] Optionally, the process of updating the model parameters in the environment model using the first total loss function, and repeatedly inputting random sampled data into the environment model, continues until the first total loss function of the environment model converges on the validation set, thus obtaining the hyperparameters of the model configuration, including: The backpropagation algorithm is used to update the model parameters in the environment model based on the optimizer and the first total loss function; the model parameters include encoder parameters and predictor parameters. Repeat the process of inputting random sampled data into the environment model until the first total loss function of the environment model converges on the validation set, and obtain the hyperparameters of the model configuration; The hyperparameters configured for the model are saved according to a preset format; wherein, the hyperparameters include the network parameters and network structure of the encoder, the network parameters and network structure of the predictor, the data normalization parameters, the embedding dimension, the number of projections, and the regularization coefficient.

[0010] Optionally, the step of sampling the trajectory action using the initial policy network, inputting the trajectory action into the defined virtual simulation environment model, and outputting the state embedding prediction result for the next time step, as well as the virtual simulation environment state and reward value, includes: Randomly sample the training dataset to obtain the initial state of the virtual simulation environment, and use the initial state as the current state of the first time step; The current state of each time step is determined based on the current state of the first time step; Based on the current state corresponding to each time step, the trajectory action is obtained by sampling using the initial policy network; The trajectory action is input into the defined virtual simulation environment model, and the state embedding prediction result for the next moment is output. The state embedding prediction result for the next time step is mapped to the original state to obtain the state of the virtual simulation environment for the next time step. Calculate the corresponding reward value using the reward function that evaluates the agent's behavior.

[0011] Optionally, the step of deploying the target strategy network corresponding to the strategy network structure and parameters to the wastewater treatment plant control system, so as to generate and issue control actions to the actuators using the target strategy network to achieve control of wastewater treatment, includes: Based on industrial communication protocols, a connection is established between the target strategy network and the wastewater treatment plant control system; The target strategy network corresponding to the strategy network structure and parameters is deployed to the control server or edge computing device of the wastewater treatment plant control system, so as to utilize the target strategy network in the wastewater treatment plant control system and generate control action distribution based on real-time sensor data. The control action distribution is denormalized to obtain the control actions; Through the aforementioned connection, the control actions are sent to the actuators to achieve control over wastewater treatment.

[0012] Secondly, this application discloses a wastewater treatment environment modeling and control device based on joint embedding prediction, comprising: The preprocessing module is used to preprocess continuous time series data from wastewater treatment plants and to construct a training dataset using the preprocessed data. The input module is used to generate an environment model based on the joint embedding prediction architecture. It inputs random sampled data from the training dataset into the environment model and outputs the current state embedding vector, the actual state embedding vector at the next time step, and the predicted state embedding vector. The first total loss function determination module is used to calculate the embedding vector loss value between the actual state embedding vector and the predicted state embedding vector at the next time step, project the current state embedding vector onto the unit direction vector to obtain a one-dimensional projection sequence, use the one-dimensional projection sequence to determine the isotropic Gaussian regularization loss value, and determine the first total loss function based on the embedding vector loss value and the isotropic Gaussian regularization loss value. The model parameter update module is used to update the model parameters in the environment model using the first total loss function. It repeatedly executes the process of inputting random sampled data into the environment model until the first total loss function of the environment model converges on the validation set, thus obtaining the hyperparameters configured by the model. The definition module is used to construct a virtual simulation environment model based on the hyperparameters, and to define the state space of the virtual simulation environment state, the action space of the agent in the virtual simulation environment, the reward function for evaluating the agent's behavior, and the termination condition for the operation of the virtual simulation environment. The trajectory data determination module is used to construct an initial policy network and an initial value network for the defined virtual simulation environment model, sample the trajectory actions using the initial policy network, input the trajectory actions into the defined virtual simulation environment model, and output the state embedding prediction result of the next moment, the virtual simulation environment state, and the reward value. If the current condition is met by a preset termination condition, the trajectory data is determined based on the state embedding prediction result of the next moment, the virtual simulation environment state, and the reward value. The network parameter update module is used to construct a second total loss function based on trajectory data, update the parameters of the initial policy network and the initial value network using the second total loss function, and repeatedly execute the process of sampling to obtain trajectory actions until the second total loss function converges, thereby obtaining the policy network structure and parameters. The deployment module is used to deploy the target strategy network corresponding to the strategy network structure and parameters to the wastewater treatment plant control system, so as to generate and issue control actions to the actuators using the target strategy network, thereby realizing the control of wastewater treatment.

[0013] Thirdly, this application discloses an electronic device, including: Memory, used to store computer programs; A processor is used to execute the computer program to implement the aforementioned wastewater treatment environment modeling and control method based on joint embedding prediction.

[0014] Fourthly, this application discloses a computer storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the aforementioned disclosed wastewater treatment environment modeling and control method based on joint embedding prediction.

[0015] Therefore, this application provides a wastewater treatment environment modeling and control method based on joint embedding prediction. This method includes preprocessing continuous time-series data from wastewater treatment plants, constructing a training dataset using the preprocessed data to ensure data diversity and comprehensiveness, eliminate outliers and dimensional differences, and provide a reliable and consistent data foundation for subsequent environmental model training. An environmental model is generated based on the joint embedding prediction architecture. Randomly sampled data from the training dataset is input into the environmental model, which outputs the current state embedding vector, the actual state embedding vector for the next time step, and the predicted state embedding vector. This prevents "representation collapse" in the model, achieving representation of the current state and prediction of the next state, thus providing a basis for subsequent loss calculation and... Model optimization provides a feature foundation; it calculates the embedding vector loss between the actual state embedding vector and the predicted state embedding vector at the next time step, projects the current state embedding vector onto the unit direction vector to obtain a one-dimensional projection sequence, and uses this one-dimensional projection sequence to determine the isotropic Gaussian regularization loss value. Based on the embedding vector loss value and the isotropic Gaussian regularization loss value, it determines the first total loss function, solving the problems of unstable model training and reliance on complex heuristic techniques, reducing the complexity of algorithm implementation and the difficulty of hyperparameter tuning; it uses the first total loss function to update the model parameters in the environment model, repeatedly executing the process of inputting randomly sampled data into the environment model until the environment model achieves its first result on the validation set. The convergence of the total loss function yields the hyperparameters for model configuration, reducing deployment difficulty and ensuring the prediction accuracy and generalization ability of the environmental model. Based on these hyperparameters, a virtual simulation environment model is constructed, defining the state space of the virtual simulation environment, the action space for agents to perform actions within the virtual simulation environment, the reward function for evaluating agent behavior, and the termination conditions for the virtual simulation environment. This forms a complete virtual interactive environment that can be used for reinforcement learning policy training, eliminating the need for interaction with a real wastewater treatment plant and mitigating safety risks such as excessive effluent discharge and equipment damage. An initial policy network and an initial value network are constructed for the defined virtual simulation environment model. Trajectory actions are sampled using the initial policy network and then processed... The input is given to the defined virtual simulation environment model, and the output is the state embedding prediction result of the next time step, the virtual simulation environment state, and the reward value. If the current condition is met, the trajectory data is determined based on the state embedding prediction result of the next time step, the virtual simulation environment state, and the reward value, providing detailed data for subsequent reinforcement learning algorithm optimization of the policy network. A second total loss function is constructed based on the trajectory data. The parameters of the initial policy network and the initial value network are updated using the second total loss function. The process of sampling to obtain trajectory actions is repeated until the second total loss function converges, thus obtaining the policy network structure and parameters. A control policy that can be directly deployed can be obtained without trial and error on real devices, avoiding getting trapped in local optima.The target strategy network, corresponding to the strategy network structure and parameters, is deployed to the wastewater treatment plant control system. This allows the target strategy network to generate and issue control actions to the actuators, achieving control of wastewater treatment. This enables end-to-end closed-loop automatic control from real-time sensor data to control commands, ensuring safe operation and making it suitable for industrial fields with continuous dynamic processes. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0017] Figure 1 This is a flowchart of a wastewater treatment environmental modeling and control method based on joint embedding prediction disclosed in this application; Figure 2 This is a schematic diagram of a wastewater treatment environmental modeling and control device based on joint embedding prediction disclosed in this application; Figure 3 This application provides a structural diagram of an electronic device. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Long-standing water pollution problems have severely disrupted the balance of aquatic ecosystems. Upgrading wastewater treatment plants to intelligent systems and implementing automated control are crucial for improving effluent quality, reducing energy consumption, and minimizing human intervention. However, directly training and optimizing intelligent control strategies in actual wastewater treatment plants carries significant risks: 1. Existing simulation models are unstable during training and rely heavily on heuristics: Current advanced environmental simulation methods are very fragile during training. To prevent models from learning "shortcuts" or experiencing "representation collapse" (i.e., the model maps all inputs to almost the same output, thus failing to distinguish between different states), they have to rely on many heuristics lacking theoretical guarantees. This makes the model training process extremely unstable, and hyperparameters (such as the weights of each loss term) are extremely sensitive, requiring a significant amount of time and computing power for grid search. When applied to complex and variable industrial scenarios such as wastewater treatment, the fragility of these heuristics is further amplified, leading to poor convergence or generalization ability of the model. 2. Existing models lack interpretability and causal reasoning ability: Many purely data-driven "black box" models, while capable of capturing statistical patterns in time series, have internal decision-making processes that are difficult to understand. More importantly, these models can only learn the correlations between variables but cannot distinguish causality. When control strategies change, or when operational methods differ from historical data distributions, this statistical correlation collapses, leading to severe biases in intervention predictions and a sharp decline in the model's predictive reliability.

[0020] 3. Existing end-to-end JEPA (Joint Embedding Predictive Architecture) methods lack stability guarantees and are difficult to implement in industrial scenarios: JEPA itself is an ideal blueprint for building environmental models because it directly predicts the "embedded representation" of future states, rather than reconstructing the original high-dimensional data, thus automatically ignoring details irrelevant to the prediction task (such as background noise, sensor errors, etc.). However, existing end-to-end JEPA methods still face the representation collapse problem during training, i.e., the model finds that "mapping all states to the same point" can perfectly minimize the prediction error, thus abandoning the learning of any meaningful state distinctions. To prevent collapse, PLDM introduces a large number of additional regularization terms, making the training objective extremely complex, and still cannot completely guarantee that collapse will not occur. These techniques do not theoretically solve the root cause of collapse, resulting in complex model design, unstable training, and hyperparameter sensitivity, making it difficult to implement in industrial scenarios with extremely high reliability requirements, such as wastewater treatment.

[0021] As can be seen from the above, how to solve the problems of unstable model training and reliance on complex heuristic techniques, improve the reliability and accuracy of model predictions, and reduce algorithm complexity are problems that need to be solved in this field.

[0022] See Figure 1 As shown in the figure, this invention discloses a wastewater treatment environment modeling and control method based on joint embedding prediction, which may specifically include: Step S11: Preprocess the continuous time series data of the wastewater treatment plant and use the preprocessed data to construct a training dataset.

[0023] In this embodiment, continuous time series data is collected from the historical database or data acquisition and monitoring control system of the sewage treatment plant; outlier removal, missing value filling, timestamp alignment, data slicing, and data standardization are performed on the continuous time series data to obtain preprocessed data; the preprocessed data is integrated into triplet samples according to a preset format to construct a training dataset.

[0024] In this step, continuous time series data is collected from the wastewater treatment plant's historical database or SCADA (Supervisory Control and Data Acquisition) system. The continuous time series data should cover different seasons, weather conditions, and typical operating conditions such as influent load (low load, normal load, shock load) to ensure the diversity and comprehensiveness of the data.

[0025] Continuous time series data includes State and Action.

[0026] Among them, State (state data) is time-series data that reflects key operating parameters of the wastewater treatment process.

[0027] Specifically, including but not limited to: influent indicators: (Inlet flow rate) (Influent Chemical Oxygen Demand) (Influent ammonia nitrogen) (Total phosphorus in influent), pH value of influent; Biological tank indicators: DO (Dissolved Oxygen), MLSS (Mixed Liquid Suspended Solids). (Nitrate nitrogen concentration), ORP (Oxidation-Reduction Potential), and temperature; Water output indicators: (Chemical Oxygen Demand in Effluent) (Ammonia nitrogen in effluent) (Total phosphorus in effluent) (Suspended solids in the effluent); Equipment status: blower operating frequency, return pump operating status, etc. (optional, can be indirectly reflected through action data).

[0028] Action (Motion Data): Records control commands executed by operators or the underlying control system in history. Specifically, it includes, but is not limited to: Aeration system: Blower frequency setting (or aeration volume); Reflux system: internal reflux ratio setpoint, external reflux ratio setpoint; Dosing system: dosage of carbon source (such as sodium acetate), dosage of phosphorus removal agent; Sludge discharge system: residual sludge discharge volume or sludge pump operating time; Data is stored in time series format, and the sampling interval can be set according to the dynamic characteristics of the process, typically ranging from 1 minute to 5 minutes. The temporal causal relationships between parameters are strictly preserved, ensuring that the state and actions of one time step can affect the state of the next time step.

[0029] The preprocessing process is as follows: Outlier removal: Remove outliers from continuous time series data that are significantly outside the physical range, such as negative COD concentrations; Missing value imputation: Fill missing values ​​with linear interpolation or by using valid values ​​from the previous time step; Timestamp alignment: Align different data sources based on timestamps to ensure that states and actions strictly correspond in time; Data slicing: Dividing aligned continuous time-series data into fixed-length, overlapping trajectory segments, for example, T =20 time steps, each trajectory segment contains continuous Triples, in which t =1, 2, ..., T -1. When slicing, the number of overlap steps between adjacent segments can be set to... T / 2, to increase the number of training samples; Data standardization: For each state variable and action variable, calculate its mean and standard deviation (based on the entire training dataset), and then perform Z-score (standard deviation) standardization. ; in, x The original variable value (state or action). The mean of this variable. The standard deviation of this variable. This is the standardized value.

[0030] The standardized state is denoted as Actions are recorded as Standardized parameters This data will be saved so that the same standardized processing can be applied to real-time data when the model is deployed.

[0031] The standardized trajectory segments are organized into a training dataset, with each training sample being a triplet. The training dataset is provided in batches for training the environment model. The batch size can be set according to the GPU (Graphics Processing Unit) memory, with typical values ​​being 128 or 256.

[0032] Step S12: Generate an environment model based on the joint embedding prediction architecture. Input random sampled data from the training dataset into the environment model and output the current state embedding vector, the actual state embedding vector at the next time step, and the predicted state embedding vector.

[0033] In this embodiment, an environment model is generated based on a joint embedding prediction architecture, which consists of an encoder and a predictor. This architecture is trained using Sketched Isotropic Gaussian Regularization (SIGReg) to prevent representation collapse. The entire training process requires no heuristics (such as exponential moving averages, stopping gradients, asymmetric networks, etc.) and only requires tuning one hyperparameter. The encoder and predictor employ a multilayer perceptron and / or Transformer architecture.

[0034] The Encoder uses either an MLP (Multilayer Perceptron) or Transformer (Neural Network) architecture.

[0035] Input: The standardized state is denoted as (dimension is) That is, the number of state variables.

[0036] Output: Low-dimensional state embedding vector , ,in D The embedding dimension (typically 128 or 256, much smaller) ).

[0037] The role of an encoder is to extract essential, abstract features for prediction from high-dimensional, noisy raw data.

[0038] Optionally, a projection head, or additional linear layer or MLP, can be added after the output of the last layer of the encoder to map the embeddings to another space of the same dimension to better accommodate SIGReg regularization. The parameters of the projection head are also used in training.

[0039] The predictor uses an MLP or Transformer architecture.

[0040] Input: The state embedding vector at the current time step and standardized actions (dimension is) (That is, the number of action variables). Inputs are usually combined using concatenation.

[0041] Output: Predicted value of the state embedding at the next time step. , .

[0042] The predictor is responsible for learning the inherent dynamics of the environment. When using a Transformer architecture, multi-step historical input can be supported to improve prediction accuracy (e.g., input...). ).

[0043] In this embodiment, after generating the environment model, the training dataset is randomly sampled to obtain random sampled data; the random sampled data is input into the encoder in the environment model to output the current state embedding vector and the actual state embedding vector at the next moment; the current state embedding vector and the action data in the random sampled data are concatenated, and the concatenated data is input into the predictor in the environment model to output the predicted state embedding vector at the next moment.

[0044] In this step, a batch of data is randomly sampled from the training dataset, containing... B There are 10 samples, each sample is 100 samples. , B Batch size; Then, perform a forward propagation to sample the data. and Input encoders respectively To obtain the state embedding vector at the current time. and the actual state embedding vector at the next time step : ; ; Note: Encoder Shared parameters at two time points (i.e., the same network); Then, embed the current state into the vector. and motion data The concatenated data is then input into the predictor. The predicted state embedding vector for the next time step is obtained: .

[0045] Step S13: Calculate the embedding vector loss value between the actual state embedding vector and the predicted state embedding vector at the next time step, project the current state embedding vector onto the unit direction vector to obtain a one-dimensional projection sequence, use the one-dimensional projection sequence to determine the isotropic Gaussian regularization loss value, and determine the first total loss function based on the embedding vector loss value and the isotropic Gaussian regularization loss value.

[0046] In this embodiment, the mean square error between the actual state embedding vector and the predicted state embedding vector at the next time step is calculated, and the mean square error is used as the embedding vector loss value; a unit hypersphere including the direction vector is randomly sampled to obtain a unit direction vector; the current state embedding vector is projected onto the unit direction vector to obtain a one-dimensional projection sequence; the test statistic between the one-dimensional projection sequence and the standard normal distribution is calculated; and the isotropic Gaussian regularization loss value is calculated using the test statistic.

[0047] In this embodiment, the first total loss function is a weighted sum of the embedding vector loss value and the isotropic Gaussian regularization loss value: ; in, This is the regularization weight coefficient, the only hyperparameter that needs to be adjusted, with a typical value of 0.1.

[0048] For the embedding vector loss value : Calculate the Mean Square Error (MSE) between the actual state embedding vector and the predicted state embedding vector at the next time step: ; This loss drives the encoder and predictor to learn a predictable state representation.

[0049] For isotropic Gaussian regularization loss value : SIGReg is used to force all state embeddings. z The set follows an isotropic Gaussian distribution. N (0, I This fundamentally prevents representation collapse. The calculation process is as follows: (1) Random sampling projection direction: From the unit hypersphere Up-random sampling M Unit direction vector , , . M A typical value is 1024. Sampling can be achieved by sampling from a standard normal distribution. M indivualD A dimensional vector is generated and then normalized along the vector direction.

[0050] (2) Projection: Embed the current state into the vector (The rows of Z are each) Projected into each direction The above yields a one-dimensional projection sequence: ; in, The i The elements are .

[0051] (3) Calculate the Epps-Pulley test statistic: For each one-dimensional projection sequence Calculate its Epps-Pulley test statistic with respect to the standard normal distribution. This statistic measures the difference in distributions by comparing empirical characteristic functions with theoretical characteristic functions: ; in: It is an empirical characteristic function.

[0052] It is the characteristic function of the standard normal distribution.

[0053] It is a Gaussian weighting function used to ensure that the integral converges and has an analytical form.

[0054] In practical calculations, the above integral is obtained through numerical integration (e.g., the trapezoidal rule) over a finite interval. The above approximation, The typical value is 5, and the typical number of integration points is 17. Due to the symmetry of the integrand, the computational efficiency is relatively high.

[0055] Isotropic Gaussian regularization loss value That is, the average of the statistics across all projection directions: .

[0056] Key advantages of SIGReg: Theoretically complete: According to the Cramér-Wold theorem, matching the distribution of all one-dimensional projections is equivalent to matching the joint distribution of high dimensions.

[0057] Bounded gradient: The Epps-Pulley statistic is bounded with respect to the gradient of the embedding, which avoids gradient explosion and ensures stable training.

[0058] Linear complexity: The computational complexity is O(n log n). It exhibits a linear relationship with batch size and embedding dimension, and has good scalability.

[0059] No heuristics required: It completely replaces fragile mechanisms such as exponential moving averages and stopping gradients.

[0060] Step S14: Update the model parameters in the environment model using the first total loss function, and repeat the process of inputting random sampled data into the environment model until the first total loss function of the environment model converges on the validation set, thus obtaining the hyperparameters configured for the model.

[0061] In this embodiment, the backpropagation algorithm is used, and the model parameters in the environment model are updated based on the optimizer and the first total loss function. The model parameters include encoder parameters and predictor parameters. The process of inputting random sampled data into the environment model is repeated until the first total loss function of the environment model converges on the validation set, thereby obtaining the hyperparameters configured for the model. The hyperparameters configured for the model are saved according to a preset format. The hyperparameters include the network parameters and network structure of the encoder, the network parameters and network structure of the predictor, data normalization parameters, embedding dimension, number of projections, and regularization coefficient.

[0062] In this step, an optimizer (such as AdamW) is used to jointly update the encoder parameters via backpropagation. and predictor parameters To minimize the first total loss function The learning rate can be scheduled using a cosine annealing strategy, with a typical initial learning rate of 5e-4 and a typical weight decay of 1e-2.

[0063] The process of repeatedly inputting randomly sampled data into the environment model continues until the environment model's first total loss function converges on the validation set. This typically requires tens of epochs, with each epoch representing a complete traversal of all training data. During training, the isotropic Gaussian regularized loss value is monitored. and embedding vector loss value The changing trend should be monitored to ensure that both decline steadily.

[0064] Finally, after training is complete, save the following content: encoder Network parameters and network structure.

[0065] Predictor Network parameters and network structure.

[0066] Data standardization parameters .

[0067] Embedded Dimension DNumber of projections M Regularization coefficient Hyperparameter configuration, etc.

[0068] The model can be saved in .pt or .pth format (PyTorch standard format). The hyperparameters of this saved model configuration will serve as a "virtual sandbox" for subsequent reinforcement learning policy training.

[0069] Step S15: Construct a virtual simulation environment model based on the hyperparameters, and define the state space of the virtual simulation environment state, the action space of the agent in the virtual simulation environment, the reward function for evaluating the agent's behavior, and the termination condition for the operation of the virtual simulation environment.

[0070] This invention uses a virtual simulation environment and employs a reinforcement learning (RL) algorithm to train a wastewater treatment control strategy. The entire training process takes place in a virtual environment, eliminating the need for interaction with a real wastewater treatment plant and thus completely avoiding the risk of trial and error. The trained policy network can be directly deployed to a real wastewater treatment plant for real-time control.

[0071] In this step, hyperparameters are loaded and the encoder is restored on the reinforcement learning training server. and predictor The network structure and parameters are defined, and standardized parameters are loaded. This model will serve as the dynamic model of the RL environment: given the current state embedding and action, predict the next state embedding.

[0072] Then, the reinforcement learning problem is defined as follows: (1) State space: The state observed by the RL agent is the standardized original state. This refers to the real-time readings of various sensors in the wastewater treatment plant. In actual training, the embedded data from the encoder output can also be used directly. As a state, but for the sake of simplifying implementation, this invention directly uses the original state space.

[0073] (2) Action space: The actions output by the agent are standardized control instructions. This includes aeration rate, reflux ratio, and dosage. The action space is continuous.

[0074] (3) Reward Function: The reward function is used to guide the agent to learn the desired control behavior. This invention designs a multi-objective reward function, which can be flexibly combined according to specific optimization objectives: ; in: Incentives for effluent quality: Encourage compliance with effluent standards. For example, a positive reward is given when the effluent COD is below the discharge standard, and a negative penalty is given when it exceeds it. ; Economic incentives: Encourage energy conservation and emission reduction. For example, reduce aeration energy consumption and chemical consumption. ; Stability-based rewards: Encourage smooth changes in movement and avoid drastic fluctuations. ; Weighting coefficient , , It can be adjusted according to process priority; a typical value is... , , .

[0075] (4) Termination condition: When the length of the simulated trajectory reaches the maximum number of steps. (Typical value is 50-100 steps), or if a serious violation occurs (such as DO concentration below the safety threshold), the current episode is terminated.

[0076] Step S16: Construct an initial policy network and an initial value network for the defined virtual simulation environment model. Use the initial policy network to sample the trajectory actions. Input the trajectory actions into the defined virtual simulation environment model. Output the state embedding prediction result, virtual simulation environment state, and reward value for the next time step. If the current condition is met, determine the trajectory data based on the state embedding prediction result, virtual simulation environment state, and reward value for the next time step.

[0077] In this embodiment, an initial policy network and an initial value network are constructed for the defined virtual simulation environment model. The training dataset is randomly sampled to obtain the initial state of the virtual simulation environment, and this initial state is used as the current state of the first time step. Based on the current state of the first time step, the current state of each time step is determined. Based on the current state corresponding to each time step, the trajectory action is obtained by sampling using the initial policy network. The trajectory action is input into the defined virtual simulation environment model, and the state embedding prediction result for the next time step is output. The state embedding prediction result for the next time step is mapped to the original state to obtain the virtual simulation environment state for the next time step. The corresponding reward value is calculated using the reward function that evaluates the agent's behavior.

[0078] In this step, the Actor (policy network) It adopts an MLP architecture, with standardized states as input. The output is the average of the actions. and logarithmic standard deviation (Used to construct a Gaussian distribution). During training, actions are obtained through sampling; during deployment, the mean is directly taken as the deterministic action. The network typically has 2-3 layers, with 256 neurons per layer, and the activation function is ReLU. Critic (Value Network) It adopts an MLP architecture, with standardized states as input. The output is a value estimate (scalar) for that state. The network structure is similar to the policy network, with a 1-dimensional output layer.

[0079] The specific process for determining trajectory data is as follows: (1) Reset the environment: Randomly sample an initial state from the training dataset. As the starting point of the round.

[0080] (2) Cyclic sampling: For each time step t = 1, 2, ..., : Based on the current state Sampling actions from the policy network: .

[0081] Will The input is fed into the virtual simulation environment model to obtain the state embedding prediction result for the next time step. However, reinforcement learning training requires the original state space, therefore the embeddings need to be mapped back to the original state: A lightweight decoder is trained simultaneously with the environment model, embedding the next-time state into the prediction result. Map back to the original state The decoder can approximate the value of . This decoder can minimize the approximation of . Training is performed without participating in the JEPA main loss. The virtual simulation environment state for the next time step is obtained. Then, the reward value is calculated based on the reward function. Determine if the preset termination condition is met (e.g., water quality exceeds standards). If met, then set the tuple to... The trajectory data is stored in the experience playback buffer.

[0082] Parallel sampling: To improve efficiency, multiple virtual environment instances can be used for parallel sampling.

[0083] Step S17: Construct a second total loss function based on the trajectory data, use the second total loss function to update the parameters of the initial policy network and the initial value network, and repeat the process of sampling to obtain trajectory actions until the second total loss function converges, thereby obtaining the policy network structure and parameters.

[0084] This invention employs the Proximal Policy Optimization (PPO) algorithm for reinforcement learning training. PPO is a stable and efficient policy gradient method suitable for continuous control tasks.

[0085] The process for updating the initial policy network and the initial value network is as follows: (1) Calculate the dominance function: Calculate the dominance estimate using GAE (Generalized Dominance Estimation). .

[0086] Simultaneously calculate the target value (Cumulative discount rewards).

[0087] (2) Policy network update: Calculate the probability ratio between the old and new strategies: .

[0088] The objective function for PPO shearing is: ; in, This is the shearing parameter, with a typical value of 0.2.

[0089] Strategy loss is .

[0090] (3) Value network update: Value loss is the mean squared error between the predicted value and the target value. ; (4) Entropy regularization: To encourage exploration, negative values ​​of policy entropy are added as regularization terms: ; in, The typical value is 0.01.

[0091] (5) Total loss and backpropagation: The total loss is: , , .

[0092] Update policy network parameters using the Adam optimizer. and value network parameters .

[0093] (6) Iterative training: Repeat steps (4) and (5), and perform multiple rounds (e.g., 10 rounds) of PPO updates after collecting a fixed number of steps (e.g., 2048 steps) of data.

[0094] Training continues for hundreds of rounds until the policy performance converges (i.e., the average cumulative reward tends to stabilize).

[0095] Finally, save the policy network. The parameters and network structure, as well as the data standardization parameters. The value network is no longer needed during deployment.

[0096] Step S18: Deploy the target strategy network corresponding to the strategy network structure and parameters to the wastewater treatment plant control system so as to generate and issue control actions to the actuators using the target strategy network, thereby realizing the control of wastewater treatment.

[0097] In this embodiment, a connection is established between the target strategy network and the wastewater treatment plant control system based on an industrial communication protocol. The target strategy network, corresponding to the structure and parameters of the strategy network, is deployed to the control server or edge computing device of the wastewater treatment plant control system. This allows the target strategy network in the wastewater treatment plant control system to be used to generate a control action distribution based on real-time sensor data. The control action distribution is then de-normalized to obtain the control actions. Through the connection, the control actions are sent to the actuators to achieve control of the wastewater treatment process.

[0098] In this step, the target strategy network is connected to the SCADA platform of the control system via industrial communication protocols such as OPC UA (Open Platform UA) or Modbus TCP (an Ethernet communication protocol based on Modbus). The strategy network runs as a software module, receiving real-time sensor data and outputting control commands. The target strategy corresponding to the strategy network structure and parameters is then... Control servers or edge computing devices deployed at the wastewater treatment plant site. Load the policy network and standardize the parameters.

[0099] Read the real-time current status from the PLC (Programmable Logic Controller). .

[0100] Standardize the state: .

[0101] Will Input the policy network to obtain the action distribution, and take the mean as the control action distribution: .

[0102] Denormalize the distribution of control actions: .

[0103] Will Write the corresponding control register of the PLC and send it to the actuator (frequency converter, valve, etc.).

[0104] During deployment, a safety protection layer can be added: limiting the actions of the policy output (such as the aeration volume not exceeding the maximum capacity of the equipment), and rule-based safety verification (such as forcibly increasing aeration when the DO concentration is lower than the safety threshold).

[0105] After deployment, the system runs continuously. New operational data can be collected periodically for online fine-tuning of the policy network to adapt to changing operating conditions. Fine-tuning can be performed through simulation training in a virtual environment model before deploying the updated policy, avoiding direct online trial and error.

[0106] The actual operation of wastewater treatment plants is extremely complex, involving multiple stages such as biochemical reactions and physical sedimentation. Various process parameters (such as dissolved oxygen concentration, sludge concentration, and influent water quality) are coupled and dynamically change. Training an intelligent control system to automatically adjust operations such as aeration rate, return ratio, and reagent dosage in an actual wastewater treatment plant would pose significant environmental risks and economic losses through trial and error—incorrect operation could lead to excessive effluent quality, equipment damage, or even serious environmental accidents.

[0107] The core problem this invention aims to solve is how to utilize existing historical operational data from wastewater treatment plants to automatically learn and establish a virtual simulation model that can highly replicate the dynamics of the actual process. Within this virtual model, various control strategies can be safely and repeatedly tested, and predictions such as "how the effluent COD will change in the next 5 minutes if aeration is increased now" can be observed. This allows for the training of the optimal intelligent control strategy without direct contact with the actual equipment.

[0108] This invention abandons the complex multi-objective loss functions and various heuristic techniques (such as exponential moving average, stopping gradient, asymmetric twin networks, etc.) of traditional methods, redefining the wastewater treatment environmental simulation problem as a concise "joint embedding prediction" problem. Specifically, this application constructs an environmental model consisting of an encoder and a predictor: Encoder: Compresses and maps high-dimensional, noisy raw process state data (such as sensor readings) into a low-dimensional embedding space to extract essential, dynamic evolution-related abstract features.

[0109] Predictor: Based on the current state embedding and the actions performed, predicts the state embedding at the next moment, thereby learning the inherent dynamic laws of the environment.

[0110] The entire model training objective consists of only two parts: prediction loss (enabling the predictor to accurately predict the next state) and regularization loss (preventing representation collapse). Through end-to-end joint optimization, the model automatically learns a high-fidelity virtual wastewater treatment environment.

[0111] The innovation of this application lies in introducing the Joint Embedded Prediction Architecture (JEPA) and Sketch Isotropic Gaussian Regularization (SIGReg) into the field of wastewater treatment environment simulation, and combining it with reinforcement learning to achieve safe and efficient intelligent control. Specifically: This invention successfully applies the LeJEPA / LeWorldModel theory to industrial process simulation, solving the pain points of unstable training of traditional environmental models and reliance on complex heuristic techniques; To address the intractable "representation collapse" problem in existing technologies (i.e., the model maps all inputs to the same or similar embeddings, thus losing its discriminative ability), this invention introduces isotropic Gaussian regularization of representation. The core idea of ​​this novel approach is to force the overall distribution of all state embedding vectors to conform to a standard Gaussian distribution (i.e., an isotropic Gaussian distribution). The theoretical basis for this idea is that when the embeddings conform to an isotropic Gaussian distribution, the model's generalization ability to unknown downstream tasks (such as various different control objectives) is theoretically optimal.

[0112] The specific implementation method of this new approach is as follows: Projection dimensionality reduction: Randomly sample a large number (e.g., 1024) of direction vectors from the unit hypersphere and project the high-dimensional embedding vectors onto these one-dimensional directions.

[0113] Distribution matching: For each one-dimensional projection, the Epps-Pulley test statistic is used to compare the distribution of that projection with the standard normal distribution. This statistic measures the difference in distributions by comparing empirical characteristic functions with theoretical characteristic functions.

[0114] Comprehensive regularization: The average of the test statistics in all directions is used as the final regularization loss term.

[0115] The relationship between this new approach and solving existing problems: Completely solves the representation collapse problem: SIGReg theoretically ensures that embeddings must be uniformly and dispersedly distributed throughout the entire space, and cannot collapse to a single point or a low-dimensional subspace. This completely replaces the fragile and theoretically unguaranteed heuristics in existing technologies (exponential moving average, stopping gradient, multi-objective loss balancing, etc.), making the training process extremely stable and preventing catastrophic failures.

[0116] Significantly reduces the difficulty of hyperparameter tuning: Compared to methods like PLDM which require tuning 6-7 sensitive hyperparameters, SIGReg introduces only one regularization weight coefficient λ, and the model is insensitive to this coefficient over a wide range (e.g., 0.01-0.2). This allows engineers to quickly find the optimal value through a simple binary search, rather than performing an exponential grid search.

[0117] Laying the foundation for causal reasoning: SIGReg forces embeddings to follow a Gaussian distribution, giving the embedding space a good geometric structure (isotropy). In this space, the embedding changes caused by different actions have a clear directionality, which provides a favorable mathematical basis for decoupling causal features through contrastive learning or counterfactual reasoning.

[0118] Completely abandoning heuristic techniques, it achieves stable training "out of the box": SIGReg regularization forces the embeddings to follow an isotropic Gaussian distribution, theoretically preventing representation collapse. It eliminates the need for cumbersome and fragile techniques such as EMA (Exponential Moving Average), stop-gradient, asymmetric Siamese networks, and multi-objective loss balancing, significantly reducing the complexity of algorithm implementation and the difficulty of hyperparameter tuning.

[0119] Strategy optimization: Security planning within the "imaginary space": After obtaining the aforementioned high-fidelity environmental model, this invention further applies it to the optimization of control strategies. Specifically, it employs the MPC (Model Predictive Control) framework, using the CEM (Cross-Entropy Method) as the solver: Given the current real state and the desired target state, the model performs multiple "imaginary" scrolling steps in a virtual environment to predict the future state trajectory corresponding to different action sequences.

[0120] Through iterative optimization, the optimal sequence of actions that makes the final state closest to the target state is found.

[0121] Only the first action of the sequence is executed, and then the observation and planning are re-observed and re-planned at the next moment to achieve rolling optimization control.

[0122] This process takes place entirely in a virtual environment, eliminating the need for trial and error in a real wastewater treatment plant, thus completely avoiding security risks during strategy training.

[0123] The comparison results between the technical means of this application and existing problems are shown in Table 1: Table 1. Comparison of the technical means of this application with existing problems.

[0124] This application allows the target strategy network to be directly deployed to wastewater treatment plants, reading sensor data in real time and outputting control commands (such as aeration rate, return ratio, and chemical dosage) to achieve fully automated closed-loop control from data to decision. Because the model learns the dynamic laws of the embedded space rather than the surface statistical characteristics of the original data, it exhibits stronger robustness to noise, sensor failures, and changes in operating conditions, providing a solid technical foundation for the "digital twin" of wastewater treatment plants. This application significantly lowers the engineering implementation threshold: it eliminates the need for expensive labeled data, complex heuristic parameter tuning, and multi-GPU distributed training (a single GPU is sufficient), enabling small and medium-sized wastewater treatment plants to afford and deploy intelligent control systems. The method of this invention does not depend on the specific mechanism of wastewater treatment and can be extended to other industrial fields with continuous dynamic processes, such as chemical, pharmaceutical, energy, and metallurgical industries, demonstrating good versatility and transfer value.

[0125] This application provides a wastewater treatment environment modeling and control method based on joint embedding prediction. The method includes preprocessing continuous time-series data from a wastewater treatment plant, constructing a training dataset using the preprocessed data to ensure data diversity and comprehensiveness, eliminate outliers and dimensional differences, and provide a reliable and consistent data foundation for subsequent environmental model training. An environmental model is generated based on the joint embedding prediction architecture. Randomly sampled data from the training dataset is input into the environmental model, which outputs the current state embedding vector, the actual state embedding vector for the next time step, and the predicted state embedding vector. This prevents "representation collapse" in the model, enabling the representation of the current state and the prediction of the next state, thus facilitating subsequent loss calculation and model development. The optimization provides a feature foundation; it calculates the embedding vector loss between the actual state embedding vector and the predicted state embedding vector at the next time step, projects the current state embedding vector onto the unit direction vector to obtain a one-dimensional projection sequence, and uses this one-dimensional projection sequence to determine the isotropic Gaussian regularization loss value. Based on the embedding vector loss value and the isotropic Gaussian regularization loss value, it determines the first total loss function, solving the problems of unstable model training and reliance on complex heuristic techniques, reducing the complexity of algorithm implementation and the difficulty of hyperparameter tuning; it uses the first total loss function to update the model parameters in the environment model, repeatedly executing the process of inputting randomly sampled data into the environment model until the environment model achieves its first total loss on the validation set. The loss function converges, yielding the hyperparameters for model configuration, reducing deployment difficulty and ensuring the prediction accuracy and generalization ability of the environmental model. Based on these hyperparameters, a virtual simulation environment model is constructed, defining the state space of the virtual simulation environment, the action space for agents to perform actions within the virtual simulation environment, the reward function for evaluating agent behavior, and the termination conditions for the virtual simulation environment. This forms a complete virtual interactive environment that can be used for reinforcement learning policy training, eliminating the need for interaction with a real wastewater treatment plant and avoiding safety risks such as excessive effluent discharge and equipment damage. An initial policy network and an initial value network are constructed for the defined virtual simulation environment model. Trajectory actions are sampled using the initial policy network and then processed... The input is given to the defined virtual simulation environment model, and the output is the state embedding prediction result of the next time step, the virtual simulation environment state, and the reward value. If the current condition is met, the trajectory data is determined based on the state embedding prediction result of the next time step, the virtual simulation environment state, and the reward value, providing detailed data for subsequent reinforcement learning algorithm optimization of the policy network. A second total loss function is constructed based on the trajectory data. The parameters of the initial policy network and the initial value network are updated using the second total loss function. The process of sampling to obtain trajectory actions is repeated until the second total loss function converges, thus obtaining the policy network structure and parameters. A control policy that can be directly deployed can be obtained without trial and error on real devices, avoiding getting trapped in local optima.The target strategy network, corresponding to the strategy network structure and parameters, is deployed to the wastewater treatment plant control system. This allows the target strategy network to generate and issue control actions to the actuators, achieving control of wastewater treatment. This enables end-to-end closed-loop automatic control from real-time sensor data to control commands, ensuring safe operation and making it suitable for industrial fields with continuous dynamic processes.

[0126] See Figure 2 As shown in the figure, this invention discloses a wastewater treatment environment modeling and control device based on joint embedding prediction, which may specifically include: Preprocessing module 11 is used to preprocess continuous time series data from the wastewater treatment plant and to construct a training dataset using the preprocessed data. Input module 12 is used to generate an environment model based on the joint embedding prediction architecture, input random sampled data from the training dataset into the environment model, and output the current state embedding vector, the actual state embedding vector and the predicted state embedding vector at the next time step. The first total loss function determination module 13 is used to calculate the embedding vector loss value between the actual state embedding vector and the predicted state embedding vector at the next time step, project the current state embedding vector onto the unit direction vector to obtain a one-dimensional projection sequence, use the one-dimensional projection sequence to determine the isotropic Gaussian regularization loss value, and determine the first total loss function based on the embedding vector loss value and the isotropic Gaussian regularization loss value. The model parameter update module 14 is used to update the model parameters in the environment model using the first total loss function. The process of inputting random sampled data into the environment model is repeated until the first total loss function of the environment model on the validation set converges, and the hyperparameters of the model configuration are obtained. The definition module 15 is used to construct a virtual simulation environment model based on the hyperparameters, and to define the state space of the virtual simulation environment state, the action space of the agent in the virtual simulation environment, the reward function for evaluating the agent's behavior, and the termination condition for the operation of the virtual simulation environment. The trajectory data determination module 16 is used to construct an initial policy network and an initial value network for the defined virtual simulation environment model, sample the trajectory action using the initial policy network, input the trajectory action into the defined virtual simulation environment model, and output the state embedding prediction result of the next moment, the virtual simulation environment state, and the reward value. If the current preset termination condition is met, the trajectory data is determined based on the state embedding prediction result of the next moment, the virtual simulation environment state, and the reward value. The network parameter update module 17 is used to construct a second total loss function based on trajectory data, update the parameters of the initial policy network and the initial value network using the second total loss function, and repeatedly execute the process of sampling to obtain trajectory actions until the second total loss function converges, thereby obtaining the policy network structure and parameters. The deployment module 18 is used to deploy the target strategy network corresponding to the strategy network structure and parameters to the wastewater treatment plant control system, so as to generate and issue control actions to the actuators using the target strategy network to realize the control of wastewater treatment.

[0127] In some specific embodiments, the preprocessing module 11 may specifically include: Collect continuous time series data from the historical database or data acquisition and monitoring control system of the wastewater treatment plant; The continuous time series data undergoes outlier removal, missing value imputation, timestamp alignment, data slicing, and data standardization to obtain preprocessed data. The preprocessed data is integrated into triplet samples according to a preset format to construct a training dataset.

[0128] In some specific embodiments, the joint embedding prediction architecture consists of an encoder and a predictor; the encoder and predictor employ a multilayer perceptron and / or Transformer architecture.

[0129] In some specific embodiments, the input module 12 may specifically include: Random sampling is performed on the training dataset to obtain random sample data; The random sampled data is input into the encoder in the environment model to output the current state embedding vector and the actual state embedding vector at the next moment; The current state embedding vector and the action data from the randomly sampled data are concatenated, and the concatenated data is input into the predictor in the environment model to output the predicted state embedding vector for the next time step.

[0130] In some specific embodiments, the first total loss function determination module 13 may specifically include: Calculate the mean square error between the actual state embedding vector and the predicted state embedding vector at the next time step, and use the mean square error as the embedding vector loss value; The unit direction vector is obtained by randomly sampling the unit hypersphere, which includes the direction vector. Project the current state embedding vector onto the unit direction vector to obtain a one-dimensional projection sequence; Calculate the test statistic between the one-dimensional projected sequence and the standard normal distribution; The isotropic Gaussian regularization loss value is calculated using the test statistic.

[0131] In some specific embodiments, the model parameter update module 14 may specifically include: The backpropagation algorithm is used to update the model parameters in the environment model based on the optimizer and the first total loss function; the model parameters include encoder parameters and predictor parameters. Repeat the process of inputting random sampled data into the environment model until the first total loss function of the environment model converges on the validation set, and obtain the hyperparameters of the model configuration; The hyperparameters configured for the model are saved according to a preset format; wherein, the hyperparameters include the network parameters and network structure of the encoder, the network parameters and network structure of the predictor, the data normalization parameters, the embedding dimension, the number of projections, and the regularization coefficient.

[0132] In some specific embodiments, the trajectory data determination module 16 may specifically include: Randomly sample the training dataset to obtain the initial state of the virtual simulation environment, and use the initial state as the current state of the first time step; The current state of each time step is determined based on the current state of the first time step; Based on the current state corresponding to each time step, the trajectory action is obtained by sampling using the initial policy network; The trajectory action is input into the defined virtual simulation environment model, and the state embedding prediction result for the next moment is output. The state embedding prediction result for the next time step is mapped to the original state to obtain the state of the virtual simulation environment for the next time step. Calculate the corresponding reward value using the reward function that evaluates the agent's behavior.

[0133] In some specific embodiments, the deployment module 18 may specifically include: Based on industrial communication protocols, a connection is established between the target strategy network and the wastewater treatment plant control system; The target strategy network corresponding to the strategy network structure and parameters is deployed to the control server or edge computing device of the wastewater treatment plant control system, so as to utilize the target strategy network in the wastewater treatment plant control system and generate control action distribution based on real-time sensor data. The control action distribution is denormalized to obtain the control actions; Through the aforementioned connection, the control actions are sent to the actuators to achieve control over wastewater treatment.

[0134] Figure 3This is a schematic diagram of an electronic device provided in an embodiment of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the wastewater treatment environment modeling and control method based on joint embedding prediction disclosed in any of the foregoing embodiments.

[0135] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0136] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored on it include operating system 221, computer program 222 and data 223, etc., and the storage method can be temporary storage or permanent storage.

[0137] The operating system 221 manages and controls the various hardware devices on the electronic device 20 and the computer program 222 to enable the processor 21 to perform operations and processing on the data 223 in the memory 22. The operating system 221 can be Windows, Unix, Linux, etc. The computer program 222, in addition to including a computer program capable of performing the wastewater treatment environment modeling and control method based on joint embedding prediction disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 223 may include data received by the wastewater treatment environment modeling and control device based on joint embedding prediction from external devices, and may also include data collected by its own input / output interface 25.

[0138] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0139] Furthermore, embodiments of this application also disclose a computer-readable storage medium storing a computer program. When the computer program is loaded and executed by a processor, it implements the steps of the wastewater treatment environment modeling and control method based on joint embedding prediction disclosed in any of the foregoing embodiments.

[0140] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0141] The above provides a detailed description of the wastewater treatment environment modeling and control method, apparatus, equipment, and medium based on joint embedding prediction provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for wastewater treatment environment modeling and control based on joint embedding prediction, characterized in that, include: Continuous time series data from wastewater treatment plants are preprocessed, and training datasets are constructed using the preprocessed data. The continuous time series data includes state data and action data; the state data is time-series data reflecting key operating parameters of the wastewater treatment process; the action data is used to record control commands executed by operators or the underlying control system in history. An environment model is generated based on a joint embedding prediction architecture. Randomly sampled data from the training dataset is input into the environment model, and the current state embedding vector, the actual state embedding vector at the next time step, and the predicted state embedding vector are output. The joint embedding prediction architecture consists of an encoder and a predictor. The encoder and predictor adopt a multilayer perceptron and / or Transformer architecture. Calculate the embedding vector loss value between the actual state embedding vector and the predicted state embedding vector at the next time step, project the current state embedding vector onto the unit direction vector to obtain a one-dimensional projection sequence, use the one-dimensional projection sequence to determine the isotropic Gaussian regularization loss value, and determine the first total loss function based on the embedding vector loss value and the isotropic Gaussian regularization loss value. The model parameters in the environment model are updated using the first total loss function. The process of inputting random sampled data into the environment model is repeated until the first total loss function of the environment model converges on the validation set, and the hyperparameters of the model configuration are obtained. The hyperparameters include the network parameters and network structure of the encoder, the network parameters and network structure of the predictor, the data normalization parameters, the embedding dimension, the number of projections, and the regularization coefficient. Based on the hyperparameters, a virtual simulation environment model is constructed, defining the state space of the virtual simulation environment, the action space of the agent in the virtual simulation environment, the reward function for evaluating the agent's behavior, and the termination condition for the operation of the virtual simulation environment. An initial policy network and an initial value network are constructed for the defined virtual simulation environment model. Trajectory actions are sampled using the initial policy network and input into the defined virtual simulation environment model. The state embedding prediction result, virtual simulation environment state, and reward value of the next time step are output. If the current time step meets the preset termination condition, the trajectory data is determined based on the state embedding prediction result, virtual simulation environment state, and reward value of the next time step. A second total loss function is constructed based on trajectory data. The parameters of the initial policy network and the initial value network are updated using the second total loss function. The process of sampling to obtain trajectory actions is repeated until the second total loss function converges, and the policy network structure and parameters are obtained. The target strategy network corresponding to the strategy network structure and parameters is deployed to the wastewater treatment plant control system so that the target strategy network can be used to generate and issue control actions to the actuators to achieve control of wastewater treatment; The random sampled data from the training dataset is input into the environment model, and the current state embedding vector, the actual state embedding vector and the predicted state embedding vector are output. This includes: randomly sampling the training dataset to obtain random sampled data. The random sampled data is input into the encoder in the environment model to output the current state embedding vector and the actual state embedding vector at the next moment. The current state embedding vector and the action data in the random sampled data are concatenated, and the concatenated data is input into the predictor in the environment model to output the predicted state embedding vector at the next moment.

2. The joint embedding prediction based modeling and control method of wastewater treatment environment according to claim 1, characterized in that, The process of preprocessing continuous time-series data from wastewater treatment plants and constructing a training dataset using the preprocessed data includes: Collect continuous time series data from the historical database or data acquisition and monitoring control system of the wastewater treatment plant; The continuous time series data is subjected to outlier removal, missing value imputation, timestamp alignment, data slicing, and data standardization to obtain preprocessed data. The preprocessed data is integrated into triplet samples according to a preset format to construct a training dataset.

3. The wastewater treatment environment modeling and control method based on joint embedding prediction according to claim 1, characterized in that, The process of calculating the embedding vector loss between the actual state embedding vector and the predicted state embedding vector at the next time step, projecting the current state embedding vector onto a unit direction vector to obtain a one-dimensional projection sequence, and using the one-dimensional projection sequence to determine the isotropic Gaussian regularization loss value includes: Calculate the mean square error between the actual state embedding vector and the predicted state embedding vector at the next time step, and use the mean square error as the embedding vector loss value; The unit direction vector is obtained by randomly sampling the unit hypersphere, which includes the direction vector. Project the current state embedding vector onto the unit direction vector to obtain a one-dimensional projection sequence; Calculate the test statistic between the one-dimensional projected sequence and the standard normal distribution; The isotropic Gaussian regularization loss value is calculated using the test statistic.

4. The wastewater treatment environment modeling and control method based on joint embedding prediction according to claim 1, characterized in that, The process of updating the model parameters in the environment model using the first total loss function, and repeatedly inputting randomly sampled data into the environment model, continues until the first total loss function of the environment model converges on the validation set, yielding the hyperparameters configured for the model, including: The backpropagation algorithm is used to update the model parameters in the environment model based on the optimizer and the first total loss function; the model parameters include encoder parameters and predictor parameters. Repeat the process of inputting random sampled data into the environment model until the first total loss function of the environment model converges on the validation set, and obtain the hyperparameters of the model configuration; Save the hyperparameters configured for the model according to the preset format.

5. The wastewater treatment environment modeling and control method based on joint embedding prediction according to claim 1, characterized in that, The process involves sampling the trajectory action using an initial policy network, inputting the trajectory action into a defined virtual simulation environment model, and outputting the state embedding prediction result for the next time step, the virtual simulation environment state, and the reward value, including: Randomly sample the training dataset to obtain the initial state of the virtual simulation environment, and use the initial state as the current state of the first time step; The current state of each time step is determined based on the current state of the first time step; Based on the current state corresponding to each time step, the trajectory action is obtained by sampling using the initial policy network; The trajectory action is input into the defined virtual simulation environment model, and the state embedding prediction result for the next moment is output. The state embedding prediction result for the next time step is mapped to the original state to obtain the state of the virtual simulation environment for the next time step. Calculate the corresponding reward value using the reward function that evaluates the agent's behavior.

6. The wastewater treatment environment modeling and control method based on joint embedding prediction according to any one of claims 1 to 5, characterized in that, The step of deploying the target strategy network corresponding to the strategy network structure and parameters to the wastewater treatment plant control system, so as to generate and issue control actions to the actuators using the target strategy network, thereby realizing the control of wastewater treatment, includes: Based on industrial communication protocols, a connection is established between the target strategy network and the wastewater treatment plant control system; The target strategy network corresponding to the strategy network structure and parameters is deployed to the control server or edge computing device of the wastewater treatment plant control system, so as to utilize the target strategy network in the wastewater treatment plant control system and generate control action distribution based on real-time sensor data. The control action distribution is denormalized to obtain the control actions; Through the aforementioned connection, the control actions are sent to the actuators to achieve control over wastewater treatment.

7. A wastewater treatment environment modeling and control device based on joint embedding prediction, used to implement the method described in claim 1, characterized in that, include: The preprocessing module is used to preprocess continuous time series data from wastewater treatment plants and to construct a training dataset using the preprocessed data. The input module is used to generate an environment model based on the joint embedding prediction architecture. It inputs random sampled data from the training dataset into the environment model and outputs the current state embedding vector, the actual state embedding vector at the next time step, and the predicted state embedding vector. The first total loss function determination module is used to calculate the embedding vector loss value between the actual state embedding vector and the predicted state embedding vector at the next time step, project the current state embedding vector onto the unit direction vector to obtain a one-dimensional projection sequence, use the one-dimensional projection sequence to determine the isotropic Gaussian regularization loss value, and determine the first total loss function based on the embedding vector loss value and the isotropic Gaussian regularization loss value. The model parameter update module is used to update the model parameters in the environment model using the first total loss function. It repeatedly executes the process of inputting random sampled data into the environment model until the first total loss function of the environment model converges on the validation set, thus obtaining the hyperparameters configured by the model. The definition module is used to construct a virtual simulation environment model based on the hyperparameters, and to define the state space of the virtual simulation environment state, the action space of the agent in the virtual simulation environment, the reward function for evaluating the agent's behavior, and the termination condition for the operation of the virtual simulation environment. The trajectory data determination module is used to construct an initial policy network and an initial value network for the defined virtual simulation environment model, sample the trajectory actions using the initial policy network, input the trajectory actions into the defined virtual simulation environment model, and output the state embedding prediction result of the next moment, the virtual simulation environment state, and the reward value. If the current condition is met by a preset termination condition, the trajectory data is determined based on the state embedding prediction result of the next moment, the virtual simulation environment state, and the reward value. The network parameter update module is used to construct a second total loss function based on trajectory data, update the parameters of the initial policy network and the initial value network using the second total loss function, and repeatedly execute the process of sampling to obtain trajectory actions until the second total loss function converges, thereby obtaining the policy network structure and parameters. The deployment module is used to deploy the target strategy network corresponding to the strategy network structure and parameters to the wastewater treatment plant control system, so as to generate and issue control actions to the actuators using the target strategy network, thereby realizing the control of wastewater treatment.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the wastewater treatment environment modeling and control method based on joint embedding prediction as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the wastewater treatment environment modeling and control method based on joint embedding prediction as described in any one of claims 1 to 6.

Images