Sewage treatment control system for digital twinning and AI analysis

Abstract

The invention discloses a digital twinning and AI analysis sewage treatment control system, and relates to the technical field of sewage equipment control, and the system comprises the steps: deploying a sensor to obtain a sewage treatment acoustic signal to form an acoustic energy matrix, and synchronously obtaining the operation data of a treatment plant to construct an operation data matrix; fusing the acoustic energy matrix and the operation data matrix to form a high-dimensional data matrix; mapping the high-dimensional data matrix into a low-dimensional point cloud, constructing a point cloud adjacent matrix, recognizing topological features according to the adjacent matrix, analyzing abnormal topological features by combining persistent coherence with topological entropy, and calculating an abnormal confidence coefficient; and establishing a hierarchical agent based on the abnormal confidence coefficient, defining a reward function through life cycle evaluation, adjusting a hierarchical agent optimization strategy by adopting DDPG reinforcement learning, predicting the performance of the optimization strategy in combination with a multi-step prediction loss function, and outputting the optimization strategy. According to the invention, the control strategy of the sewage treatment equipment is continuously optimized, and the strategy implementation efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of sewage equipment control, and in particular to a sewage treatment control system based on digital twinning and AI analysis. BACKGROUND

[0002] In recent years, with the acceleration of urbanization and the improvement of environmental protection requirements, sewage treatment technology has attracted widespread attention. Traditional sewage treatment methods, such as activated sludge method and oxidation ditch method, have been widely used in various sewage treatment plants. However, traditional methods usually rely on manual or experience-based control strategies, which limits the treatment efficiency and makes it difficult to respond to changes in various environmental factors in real time during the treatment process. With the rapid development of advanced technologies such as sensor technology, Internet of Things (IoT), and artificial intelligence (AI), modern sewage treatment systems are developing towards automation, intelligence, and refinement. In particular, the application of digital twinning technology and AI analysis in the field of sewage treatment provides new possibilities. Digital twinning technology maps the real system through a virtual model, making real-time monitoring and optimization possible, while AI analysis improves the system's adaptive and predictive capabilities through processing and mining of large-scale data, becoming the core of the development of intelligent sewage treatment systems.

[0003] However, existing technologies still face some technical bottlenecks. Traditional data collection, processing, and analysis are often used in existing technologies, ignoring the deep relationships between data in complex systems, and failing to fully reflect the complex changes in the sewage treatment process, resulting in a lag in system response to abnormal situations. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides a sewage treatment control system based on digital twinning and AI analysis, which solves the problem of ignoring the deep relationships between data in complex systems, failing to fully reflect the complex changes in the sewage treatment process, and causing a lag in system response to abnormal situations.

[0006] To solve the above technical problems, the present application provides the following technical solutions: In a first aspect, the present application provides a sewage treatment control system based on digital twinning and AI analysis, which includes, a data acquisition module for deploying sensors to obtain sewage treatment acoustic signals to form an acoustic energy matrix, and simultaneously obtaining treatment plant operation data to construct an operation data matrix, and fusing the acoustic energy matrix and the operation data matrix to form a high-dimensional data matrix; a topology analysis module, configured to map the high-dimensional data matrix into a low-dimensional point cloud and construct a point cloud adjacency matrix, identify topology features according to the adjacency matrix, analyze abnormal topology features by combining persistent homology with topology entropy, and calculate an abnormal confidence; a hierarchical reinforcement learning module, configured to establish a hierarchical agent based on the abnormal confidence, define a reward function through a life cycle evaluation, adjust the optimization strategy of the hierarchical agent by using DDPG reinforcement learning, predict the performance of the optimization strategy by combining a multi-step prediction loss function, and output the optimization strategy; a dynamic updating module, configured to collect feedback data and record the implemented optimization strategy, and extract core strategy parameters by using knowledge distillation; a display storage module, configured to display the formed optimization strategy, and record the core strategy parameters of the optimization strategy in a database.

[0007] As a preferred scheme of the digital twin and AI analysis sewage treatment control system, the deployment sensor acquires a sewage treatment acoustic signal to form an acoustic energy matrix, and simultaneously acquires treatment plant operation data to construct an operation data matrix, and the acoustic energy matrix and the operation data matrix are fused to form a high-dimensional data matrix. Acoustic sensors are arranged at equipment nodes of the sewage treatment plant to collect acoustic signals, and the acoustic energy of the acoustic signals is calculated. All acoustic energies are summarized to form an acoustic energy matrix E. Treatment plant operation data (such as dissolved oxygen, sludge concentration, flow, etc.) are acquired through a sewage treatment plant SCADA system, standardized, and formed into an operation data matrix S. The acoustic energy matrix E and the operation data matrix S are spliced and fused to form a high-dimensional data matrix D. The acoustic energy matrix E and the operation data matrix S are aligned according to timestamps, and a high-dimensional data stream is generated .

[0008] As a preferred scheme of the digital twin and AI analysis sewage treatment control system, the high-dimensional data matrix is mapped into a low-dimensional point cloud and a point cloud adjacency matrix is constructed, which means that a high-dimensional data stream is acquired , and data denoising and standardization processing are performed, and an initial mapping function is defined . The Euclidean distance of all data points i and j in the high-dimensional data stream is calculated , and low-dimensional point cloud data is acquired, and the Euclidean distance of all data points i and j in the low-dimensional space is calculated , and a multidimensional scaling algorithm loss function is defined . The loss function is minimized by gradient descent. When the loss function converges, the optimized mapping function is obtained. The optimized mapping function is used to map the high-dimensional data stream into low-dimensional point cloud data. The K-nearest neighbor algorithm is used to select the K nearest neighbors for each point in the low-dimensional point cloud data, forming connecting edges between adjacent point pairs, and recording the adjacency values. If the value is 1, then the value is 0. Construct a point cloud adjacency matrix based on the connecting edges and adjacency values.

[0009] As a preferred embodiment of the wastewater treatment control system based on digital twin and AI analysis described in this invention, the following steps are performed: First, topological features are identified based on the adjacency matrix. Then, persistent homology combined with topological entropy analysis is used to analyze abnormal topological features and calculate the anomaly confidence level. After obtaining the point cloud adjacency matrix, depth-first search is used to find the connectivity of the point cloud data and detect cycles. Finally, the point cloud is clustered using the DBSCAN clustering algorithm to identify clusters in the point cloud data. The connectivity, cycles, and clusters of the point cloud data are extracted as topological features, and the birth time b and death time d of each topological feature are recorded. The persistence of each topological feature is computed using persistence homology. It also outputs a persistent graph P of topological features; An abnormal topology identification model is constructed based on the SVM algorithm. The model input is topological features and persistent graph, and the output is abnormal topology features. The abnormal topology identification model is trained by loss function and Adam optimizer, and training stops after the loss function converges. Based on the trained abnormal topology identification model, abnormal topology features in point cloud data are identified, and the point cloud data in abnormal topology features are traced to high-dimensional data stream to obtain the corresponding sewage treatment equipment. The total persistence is obtained by summing the persistence of all topological features in the persistence graph. Calculate the entropy of the persistence graph. ; The entropy of the persistence graph The reciprocal of the absolute value is used as the anomaly confidence level C.

[0010] As a preferred embodiment of the wastewater treatment control system based on digital twin and AI analysis described in this invention, wherein: the establishment of a hierarchical intelligent agent based on anomaly confidence refers to establishing a tactical layer intelligent agent and an execution layer intelligent agent according to the wastewater treatment equipment corresponding to the anomaly topological characteristics; Each tactical layer agent corresponds to a single wastewater treatment device. A convolutional neural network is used to build a device risk assessment model for each tactical layer agent and train it based on device data. The trained device risk assessment model is then used to obtain the device risk for each tactical layer agent. The optimization objective function of the tactical layer agent is defined as follows: The initial values ​​of the weights in the objective function are obtained by calculating the anomaly confidence level. The tactical layer agent outputs control instructions corresponding to the sewage treatment equipment; The execution layer agent is constructed by the tactical layer agent, and the execution layer agent receives the output instructions of the tactical layer agent, outputs single device adjustment instructions corresponding to a single specific device in a single sewage treatment equipment, takes each execution layer agent as an initial genetic individual, and forms an initial genetic population, optimizes the genetic population through a genetic algorithm, sets optimization constraints according to the output instructions of the tactical layer agent, and obtains optimized execution layer agent output instructions.

[0011] As a preferred scheme of the sewage treatment control system of the digital twin and AI analysis, wherein: the reward function is defined by the life cycle assessment, the DDPG reinforcement learning is adopted to adjust the hierarchical agent optimization strategy, the multi-step prediction loss function is combined to predict the performance of the optimization strategy, and the optimized strategy is output from the sewage treatment plant operation data , the output instructions of the tactical layer agent and the output instructions of the execution layer agent are defined as actions , the strategy space is defined based on the actions , the energy consumption and greenhouse gas emission data are weighted and summed as the reward function value ; The prediction model is established based on the LSTM network, the input is the state and the action , and the output is the state and the action of the next moment; The multi-step prediction loss function is established according to the prediction model ; The prediction model parameters are updated and trained by the gradient update method , and the trained prediction model is used to output the state and the action of the next moment, and the reward function value of the next moment is calculated based on the state and the action of the next moment ; The Q value of the current state and the action is initialized , and the value network Critic is used to update the Q value of the next moment ; The target Q value is calculated by the Bellman equation ; The discount factor is adjusted by the gradient descent method to minimize the Q value of the next moment and the target Q value The difference between the two is calculated, and the adjusted Q value gradient is input into the policy network Actor to extract the adjustment parameters of the action in the policy space The adjustment parameters of the action in the policy space are updated through the policy network Actor. The policy parameters are updated through the gradient ascent method Until the next moment The updated policy parameters are output to form the optimized policy output.

[0012] As a preferred scheme of the digital twin and AI analysis sewage treatment control system, the feedback data is collected in real time through the sensor after the optimized policy output is implemented.

[0013] As a preferred scheme of the digital twin and AI analysis sewage treatment control system, the optimized policy is recorded, and the core policy parameters are extracted from the list through weighted averaging. The core policy parameters of all the policy parameters are extracted from the list to output the core policy parameters. As a preferred scheme of the digital twin and AI analysis sewage treatment control system, the optimized policy is recorded, and the core policy parameters are extracted from the list through weighted averaging.

[0014] As a preferred scheme of the digital twin and AI analysis sewage treatment control system, the optimized policy is recorded, and the core policy parameters are extracted from the list through weighted averaging.

[0015] As a preferred scheme of the digital twin and AI analysis sewage treatment control system, the optimized policy is recorded, and the core policy parameters are extracted from the list through weighted averaging.

[0016] In a second aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and wherein the computer program is executed by the processor to implement any step of the digital twin and AI analysis sewage treatment control system according to the first aspect of the present application.

[0017] In a third aspect, the present application provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement any step of the digital twin and AI analysis sewage treatment control system according to the first aspect of the present application.

[0018] The present application has the beneficial effects that: the present application generates a high-dimensional data matrix by collecting acoustic signals and operation data, introduces persistent homology and topological entropy analysis methods, accurately identifies abnormal changes in the sewage treatment process, and simultaneously uses hierarchical reinforcement learning combined with life cycle assessment and a multi-step prediction loss function to realize continuous optimization of the sewage treatment equipment control strategy and improve the strategy implementation efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0020] Figure 1 Structure diagram of the sewage treatment control system of the digital twin and AI analysis in embodiment 1.

[0021] Figure 2 Flowchart of the sewage treatment control system of the digital twin and AI analysis in embodiment 1. DETAILED DESCRIPTION

[0022] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings of the specification.

[0023] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the present application, therefore the present application is not limited to the specific embodiments disclosed below.

[0024] Secondly, the "one embodiment" or "embodiment" referred to herein means that the specific features, structures or characteristics can be included in at least one implementation of the present application. "In one embodiment" appearing in different places in the specification does not mean the same embodiment, nor is it an independent or alternative embodiment that excludes other embodiments.

[0025] Embodiment 1, reference Figure 1 and Figure 2 , the first embodiment of the present application provides a digital twin and AI analysis sewage treatment control system, comprising the following steps: S1, a data acquisition module for deploying sensors to obtain sewage treatment acoustic signals to form an acoustic energy matrix, simultaneously obtaining treatment plant operation data to construct an operation data matrix, and fusing the acoustic energy matrix and the operation data matrix to form a high-dimensional data matrix; Specifically, the deployment of sensors acquires sewage treatment acoustic signals to form an acoustic energy matrix, and simultaneously acquires treatment plant operation data to construct an operation data matrix, and the acoustic energy matrix and the operation data matrix are fused to form a high-dimensional data matrix. The acoustic sensor array is deployed at the equipment nodes (such as the biochemical reaction tank, the secondary sedimentation tank, the reflux pump station, and the key nodes of the aeration pipeline) of the sewage treatment plant to collect acoustic signals, including equipment mechanical vibration spectrum, water flow turbulent soundprint, and aeration bubble breaking sound pulse, and the acoustic energy of the acoustic signals is calculated: , wherein is the energy of the i-th type of acoustic signal collected by the sensor array at time t, is the signal amplitude value of the sensor at frequency f and time t, is the frequency range corresponding to the i-th type of acoustic signal, for example, the frequency range of the equipment mechanical vibration spectrum is 20Hz-20kHz, the water flow turbulent soundprint is 0.1kHz-5kHz, and the aeration bubble breaking sound pulse is 1kHz-50kHz; All acoustic energies are summarized to form an acoustic energy matrix E: , wherein n is the number of acoustic signals; The treatment plant operation data (such as dissolved oxygen, sludge concentration, flow, etc.) are obtained through the SCADA system of the sewage treatment plant, standardized, and formed into an operation data matrix S, and the acoustic energy matrix E and the operation data matrix S are spliced and fused to form a high-dimensional data matrix D: , In order to ensure the consistency of the data time sequence, the acoustic energy matrix E and the operation data matrix S are aligned according to the time stamp, and a high-dimensional data stream D is generated .

[0026] The acoustic signal can directly reflect the physical phenomena in the sewage treatment process, such as the running state of the equipment, the flow condition of the fluid and the aeration effect and the like. These information can identify potential equipment failure, pipe blockage or bubble rupture and the like in an early stage, and these problems cannot be found in real time by traditional water quality sensors. The calculation of acoustic energy can help to accurately evaluate the equipment performance, operation efficiency and overall health status of the system by analyzing the spectrum of equipment mechanical vibration, flow turbulence and bubble rupture and the like. The standardized and matrix processing of the operation data can provide clear and standardized data structure for subsequent data fusion and analysis. Through real-time monitoring of the dissolved oxygen, sludge concentration, flow and the like, the necessary basic data can be provided for subsequent high-dimensional data analysis. By fusing the acoustic signal and the operation data, the information can be obtained from multiple dimensions of the equipment, fluid and environment to form a high-dimensional data matrix containing multi-level information such as equipment performance, environmental impact and fluid state. Such high-dimensional data matrix provides a more comprehensive basis for subsequent topology analysis and anomaly detection, which helps to improve the fine management level of the system. By aligning the acoustic signal and the operation data matrix, the time synchronization is ensured, so that the subsequent topology analysis, anomaly detection and optimization adjustment are more accurate. The time sequence consistency not only improves the quality of data fusion, but also lays a solid foundation for subsequent real-time analysis and feedback mechanism based on data flow.

[0027] S2, a topology analysis module, configured to map the high-dimensional data matrix into a low-dimensional point cloud and construct a point cloud adjacency matrix, identify topology features according to the adjacency matrix, analyze abnormal topology features by combining persistent homology with topology entropy and calculate an abnormal confidence degree; Specifically, the high-dimensional data matrix is mapped into a low-dimensional point cloud and a point cloud adjacency matrix is constructed, which means that the high-dimensional data stream is obtained After data denoising and standardization processing, an initial mapping function is defined (PCA or t-SNE can be used for initialization) the standardized high-dimensional data stream is mapped into low-dimensional point cloud data by using multidimensional scaling algorithm : , The Euclidean distance of all data points i, j in the high-dimensional data stream is calculated , and the low-dimensional point cloud data is obtained, and the Euclidean distance of all data points i, j in the low-dimensional space is calculated , and a multidimensional scaling algorithm loss function is defined , The loss function is minimized by gradient descent method, and the optimized mapping function is obtained when the loss function converges. The high-dimensional data stream is mapped into low-dimensional point cloud data by using the optimized mapping function. The K-Nearest Neighbor algorithm is used to select the nearest K points as adjacent points for each point in the low-dimensional point cloud data, form a connection edge between the adjacent point pairs, and record the adjacency value The point cloud adjacency matrix is constructed according to the connection edge and the adjacency value.

[0028] Standardization and denoising are the basis for dimensionality reduction, solving the problem of redundant fluctuations and inconsistent scales in the original data, ensuring the numerical stability and model generalization ability of the subsequent mapping process, improving the geometric fidelity of low-dimensional embedding, enhancing the local consistency and stability of subsequent point cloud construction, minimizing the loss function of the distance difference between points in high-dimensional and low-dimensional spaces to realize geometric preservation of spatial structure, constructing a clear and differentiable loss function and using gradient descent to realize optimization, with clear convergence criteria and repeatability, and easy to combine with other deep learning modules for application, reconstructing the structural relationship between data points in the low-dimensional point cloud space after dimensionality reduction, forming a graph structure to support subsequent clustering or fault identification, limiting local connections through K-Nearest Neighbor method to avoid overfitting problems caused by full connection; The construction of the adjacency matrix also provides structural input support for subsequent graph neural networks, which has a significant advantage in processing non-Euclidean structure data (such as operating condition fault patterns), in addition, the point cloud construction process is naturally sensitive to local topological changes, improving the model's ability to identify different operating state switches.

[0029] Further, according to the adjacency matrix, identify the topological features, use persistent homology to analyze abnormal topological features and calculate abnormal confidence, after obtaining the point cloud adjacency matrix, find the point cloud data connectivity through depth-first search (check the connectivity of the point cloud through the depth-first search (DFS) algorithm, if all other nodes can be accessed from a certain node, the point cloud is considered to be connected) and detect the ring, which means that in the point cloud, there is a path from a certain point to several points, and eventually returns to the original point, if a point is accessed twice and there is no backtracking path, it can be confirmed that there is a ring, and through the DBSCAN clustering algorithm, the point cloud is clustered, and the clusters in the point cloud data are identified, the connectivity, ring and cluster of the point cloud data are extracted as topological features, and the birth time b and death time d of each topological feature are recorded. Use persistent homology to calculate the persistence of each topological feature And output the persistence diagram P of the topological features: , An abnormal topology identification model is constructed based on the SVM algorithm. The model input is topological features and persistent graph, and the output is abnormal topology features. The abnormal topology identification model is trained by loss function and Adam optimizer, and training stops after the loss function converges. Based on the trained abnormal topology identification model, abnormal topology features in point cloud data are identified, and the point cloud data in abnormal topology features are traced to high-dimensional data stream to obtain the corresponding sewage treatment equipment. The total persistence is obtained by summing the persistence of all topological features in the persistence graph. Calculate the entropy of the persistence graph. : , The entropy of the persistence graph The reciprocal of the absolute value is used as the anomaly confidence level C.

[0030] In complex point cloud environments, isolated points or anomalous connections often cause traditional algorithms to fail. Utilizing graph connectivity and cycle detection can effectively identify irregular structures, especially closed paths introduced by errors, providing an initial screening of anomalous data. DBSCAN does not require a preset number of clusters and is suitable for extracting irregular clusters in point clouds. It can preserve structural boundary features and use the results as topological features in persistence analysis, further improving the recognizability of information in persistent graphs and enhancing the semantic foundation of anomalous models. Persistent homology has the ability to reduce noise and ignore local perturbations, allowing SVM training to focus more on structural anomalies rather than noise errors. It quantifies the structural complexity of persistent graphs into entropy values ​​and uses its reciprocal to represent anomaly confidence, avoiding subjective scoring or threshold setting. Entropy, as an information content measure, has good mathematical interpretability, making the final confidence score C objective and generalizable, and applicable to anomaly detection of point cloud data from different sources.

[0031] S3, Hierarchical Reinforcement Learning Module, is used to build a hierarchical agent based on anomaly confidence, define a reward function through lifecycle evaluation, use DDPG reinforcement learning to adjust the optimization strategy of the hierarchical agent, combine a multi-step prediction loss function to predict the performance of the optimization strategy, and output the optimization strategy. Specifically, establishing a hierarchical intelligent agent based on anomaly confidence refers to establishing tactical layer intelligent agents and execution layer intelligent agents based on the sewage treatment equipment corresponding to the anomaly topological characteristics; The tactical layer agents correspond to individual wastewater treatment equipment, such as aeration groups and sedimentation tanks. A convolutional neural network is used to build an equipment risk assessment model for each tactical layer agent and train it based on equipment data. The trained equipment risk assessment model is then used to obtain the equipment risk for each tactical layer agent. The objective function for optimizing the tactical layer agent is defined as follows: , in Dissolved oxygen in the wastewater treatment equipment corresponding to the i-th tactical layer agent, Wastewater flow in the wastewater treatment equipment corresponding to the i-th tactical layer agent, Objective function value of the i-th tactical layer agent, , And Corresponding weight; The initial value of the weight in the optimization objective function is calculated by the abnormal confidence degree: , , , Wherein α, β and γ are constant coefficients; The output of the tactical layer agent is the control instruction of the corresponding wastewater treatment equipment, such as aeration quantity and sludge return flow; The execution layer agent is constructed by the tactical layer agent, corresponding to a single specific device in a single wastewater treatment equipment, such as an aerator and a pump. The execution layer agent receives the output instruction of the tactical layer agent and outputs the adjustment instruction of the single device. Each execution layer agent is taken as an initial genetic individual and forms an initial genetic population. The genetic population is optimized by genetic algorithm. The optimization constraint is set according to the output instruction of the tactical layer agent, such as the sum of the adjustment instructions of all execution layer agents being the same as the output instruction of the tactical layer agent, so as to obtain the optimized output instruction of the execution layer agent.

[0032] Energy consumption and emission are the core indicators for measuring the environmental and economic performance of wastewater treatment process. Standardized processing not only benefits the numerical stability in the training process, but also avoids the deviation caused by the dimension difference of different indicators. The life cycle assessment indicators are introduced into the reinforcement learning as the reward function. Unlike the traditional control strategy which only takes water production or effluent quality as the target, the environmental friendliness and energy efficiency goal is effectively embedded, and the environmental and economic dual optimization framework is established. Through the life cycle reward function, the control strategy tends to be sustainable operation path, solving the conflict between "short-term performance priority" and "long-term cost minimization". By introducing the multi-step prediction loss function and using gradient update, the prediction model has the ability of continuous optimization, thereby enhancing the accuracy and reliability of future state estimation, providing a solid foundation for policy optimization.

[0033] Further, the reward function is defined by the life cycle assessment, the DDPG reinforcement learning is used to adjust the hierarchical agent optimization strategy, the multi-step prediction loss function is combined to predict the performance of the optimization strategy, and the output optimization strategy is extracted from the wastewater treatment plant operation data. Energy consumption and greenhouse gas emission data are standardized as state The output instruction of the tactical layer agent and the output instruction of the execution layer agent are defined as actions Based on the action-defined policy space The reward function value is obtained by weighting and summing energy consumption and greenhouse gas emission data. ; A prediction model is built based on an LSTM network, with the state as the input. and actions The output is the state at the next time step. and actions ; Establish a multi-step prediction loss function based on the prediction model. : , in To predict the number of time steps, To predict the types of data, The actual value of the m-th predicted data point at time t. The predicted value for the m-th predicted data at time t; Loss function predicted in multiple steps Gradient updates are used to train and update the parameters of the prediction model, and the trained prediction model is used to output the state of the next time step. and actions Based on the state at the next moment and actions The reward function value at the next time step is calculated. ; Initialize the current state and actions Q value And use the value network Critic to calculate the Q-value for the next time step. renew: , Where E is the expectation and λ is the discount factor; Calculate the target Q value using the Bellman equation. : , Minimize the Q value at the next time step by adjusting the discount factor using the gradient descent method. With the target Q value The difference between them is used to calculate the adjusted Q-value gradient and input it into the policy network Actor to extract the policy space. The adjustment parameters of the actions are used as policy parameters, and the policy parameters are updated through the policy network (Actor). , in is a policy function, which is a probability distribution or a deterministic action (e.g., in continuous action space, usually a mean value) output by the policy network, is a policy parameter, is a parameter update gradient, represents the Q value under the value network parameter . The policy parameter is updated by the gradient ascent method until the Q value at the next moment reaches the maximum and stops updating, and the updated policy parameter is output to form the optimized policy output.

[0034] The life cycle assessment index is introduced into the reward function as the objective function of reinforcement learning, so that the optimization direction is more comprehensive in the environment. The state and action sequence are processed by LSTM, multi-step prediction is performed, and a weighted error loss function is introduced to improve the prediction accuracy. LSTM has a natural advantage in processing time series data and can capture long-term dependencies. The multi-step prediction loss function introduces data feedback at multiple time steps, improving the prediction model's ability to judge future operating trend changes. The DDPG algorithm is used, the Critic network is used to evaluate the Q value, and the Actor network is used for policy update. Through the cooperation of the double networks, the policy optimization is realized. DDPG effectively solves the problem of a large number of continuous control parameters (such as aeration rate and pumping frequency) in the sewage treatment system by modeling the continuous action space. The value network evaluates the long-term return of the state-action pair through the Bellman equation, enhancing the credibility of the policy convergence direction. The policy network realizes fine-grained action adjustment, making the optimization process more sensitive and accurate. The Q value gradient output by the Critic network is combined with the gradient descent method and the ascent method to cooperatively adjust the policy network parameters, strengthening the goal-oriented policy update and avoiding resource waste caused by blind search.

[0035] S4, a dynamic updating module, is configured to collect feedback data and record the implemented optimization policy, and adopt knowledge distillation to extract core policy parameters; Specifically, the feedback data is collected after the output of the optimization policy is implemented, and the feedback data is collected in real time by the sensor.

[0036] Further, the implemented optimization policy is recorded, and the feedback data after the real-time optimization policy is obtained is used to calculate the success degree Z of each implementation of the optimization policy by adopting knowledge distillation to extract core policy parameters. , wherein is the weight of the mth policy parameter, and are the maximum and minimum values of the mth policy parameter, which are obtained from historical data. The optimization strategies with success degrees greater than the threshold value are formed into a list, and the core strategy parameters are extracted from the list by weighted average: , wherein is a parameter in the i-th optimization strategy in the list, including all strategy parameters, and M is the number of optimization strategies in the list, is the weight of the i-th optimization strategy. The core strategy parameters of all strategy parameters are extracted from the list (each strategy parameter is extracted separately).

[0037] By introducing feedback data to calculate the success degree of the strategy, and combining the normalization method and the weight mechanism, the comprehensive evaluation of the strategy effect is realized. The success degree evaluation mechanism not only can depict the actual contribution of each strategy implementation, but also can sort the relative importance between strategy parameters, which is helpful to identify effective strategy features and promote the evolution of the strategy to the direction of high success degree. By setting a success degree threshold, only the strategies with excellent performance are selected into the optimization candidate set, and the strategies with poor performance are excluded to improve the credibility of core strategy extraction, and the robustness and robustness of the core parameters are effectively improved. The idea of knowledge distillation is used to weight and fuse the high-success-degree strategies to extract the most representative and stable core parameters, reduce the parameter space, and improve the execution efficiency.

[0038] S5, a display storage module is configured to display the formed optimization strategies and record the core strategy parameters of the optimization strategies in a database; Specifically, displaying the formed optimization strategies means outputting the formed optimization strategies and displaying them to the staff of the sewage treatment plant.

[0039] Further, recording the core strategy parameters of the optimization strategies in the database means synchronously storing the core strategy parameters of the optimization strategies and the optimization strategies in the database and forming optimization records.

[0040] The embodiment also provides a computer device suitable for the digital twin and AI analysis sewage treatment control system, which comprises a memory and a processor; the memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions to realize the digital twin and AI analysis sewage treatment control system proposed in the above embodiment.

[0041] The computer device can be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected by a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be achieved through WIFI, an operator network, NFC (Near Field Communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0042] The embodiment also provides a storage medium having a computer program stored thereon, the program being executed by a processor to implement the sewage treatment control system for realizing digital twinning and AI analysis as proposed in the above embodiment; and the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk or an optical disk.

[0043] To sum up, the present application generates a high-dimensional data matrix by collecting acoustic signals and operation data, introduces persistent homology and topological entropy analysis methods, accurately identifies abnormal changes in the sewage treatment process, and simultaneously adopts hierarchical reinforcement learning combined with life cycle assessment and a multi-step prediction loss function to realize continuous optimization of the control strategy of the sewage treatment equipment and improve the implementation efficiency of the strategy.

[0044] It should be noted that the above examples are only used to illustrate the technical solutions of the present application but not limit the present application. Although the present application is described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalently replaced, without departing from the spirit and scope of the technical solutions of the present application, which should be covered in the scope of the claims of the present application.

Claims

1. A digital twin and AI analytics-based wastewater treatment control system, characterized by: The application relates to a sewage treatment optimization method based on multi-layer reinforcement learning, which comprises the following steps: A data acquisition module is used to deploy sensors to acquire sewage treatment acoustic signals to form an acoustic energy matrix, to synchronously acquire treatment plant operation data to construct an operation data matrix, and to fuse the acoustic energy matrix and the operation data matrix to form a high-dimensional data matrix; A topology analysis module is used to map the high-dimensional data matrix into a low-dimensional point cloud and construct a point cloud adjacency matrix, to identify topology features according to the adjacency matrix, to analyze abnormal topology features by combining persistent homology with topology entropy, and to calculate an abnormal confidence degree; A hierarchical reinforcement learning module is used to establish a hierarchical agent based on the abnormal confidence degree, to define a reward function by life cycle evaluation, to adjust the hierarchical agent optimization strategy by DDPG reinforcement learning, to predict the optimization strategy performance by combining a multi-step prediction loss function, and to output the optimization strategy; A dynamic updating module is used to collect feedback data and record the implemented optimization strategy, and to extract core strategy parameters by using knowledge distillation; A display storage module is used to display the formed optimization strategy, and to record the core strategy parameters of the optimization strategy in a database.

2. The digital twin and Al analytics-based wastewater treatment control system of claim 1, wherein: The acoustic energy matrix E is formed by collecting acoustic signals through an acoustic sensor array arranged at equipment nodes in a sewage treatment plant; The acoustic energy matrix E is formed by collecting acoustic signals through an acoustic sensor array arranged at equipment nodes in a sewage treatment plant; The acoustic energy matrix E is formed by collecting acoustic signals through an acoustic sensor array arranged at equipment nodes in a sewage treatment plant; aligning the acoustic energy matrix E and the operational data matrix S by time stamp to generate a high-dimensional data stream .

3. The digital twin and Al analytics-based wastewater treatment control system of claim 2, wherein: The high-dimensional data matrix is mapped to a low-dimensional point cloud and a point cloud adjacency matrix is constructed Data denoising and normalization are performed, and an initial mapping function is defined The normalized high-dimensional data stream is mapped to low-dimensional point cloud data using a multidimensional scaling algorithm ; Compute the Euclidean distance of all data points i, j in the high-dimensional data stream , and obtain low-dimensional point cloud data, compute the Euclidean distance of all data points i, j in the low-dimensional space , define the loss function of the multi-dimensional scaling algorithm ; The high-dimensional data matrix D is formed by splicing and fusing the acoustic energy matrix E and the operation data matrix S; K nearest neighbor algorithm is adopted to select K nearest points of each point in low-dimensional point cloud data as adjacent points, and a connection edge is formed between the adjacent point pairs, and a connection value is recorded 1, otherwise 0, and a point cloud adjacency matrix is constructed according to the connection edge and the connection value.

4. The digital twin and Al analytics-based wastewater treatment control system of claim 3, wherein: The optimized mapping function is obtained when the loss function converges by minimizing the loss function through the gradient descent method; calculating the persistence of each topological feature using persistent homology and outputting a persistence plot P of the topological features; The abnormal topology features are identified based on the trained abnormal topology identification model, the point cloud data of the abnormal topology features is traced back to the corresponding sewage treatment equipment according to the abnormal topology features, and the abnormal topology features are analyzed and the abnormal confidence degree is calculated. summing the persistence of all topological features in the persistence diagram to obtain a total persistence , calculating an entropy of the persistence diagram ; The entropy of the persistence diagram The inverse of the absolute value as anomaly confidence C.

5. The digital twin and Al analytics-based wastewater treatment control system of claim 4, wherein: The initial value of the weight in the optimization objective function is calculated through the abnormal confidence degree; The tactical layer agent corresponds to a single sewage treatment device, a convolutional neural network is used to establish a device risk assessment model for each tactical layer agent and train the device risk assessment model according to device data, and a trained device risk assessment model is used to obtain the device risk of each tactical layer agent , and the optimization objective function of the tactical layer agent is defined as The tactical layer agent outputs the control instruction of the corresponding sewage treatment equipment; The initial value of the weight in the optimization objective function is calculated through the abnormal confidence degree; The execution layer agent is constructed by the tactical layer agent, corresponds to a single specific device in a single sewage treatment equipment, receives the output instruction of the tactical layer agent, outputs the adjustment instruction of the single device, takes each execution layer agent as an initial genetic individual, forms an initial genetic population, optimizes the genetic population through a genetic algorithm, sets optimization constraints according to the output instruction of the tactical layer agent, and obtains the optimized execution layer agent output instruction.

6. The digital twin and Al analytics-based wastewater treatment control system of claim 5, wherein: The reward function is defined by the life cycle assessment, the hierarchical agent optimization strategy is adjusted by DDPG reinforcement learning, the performance of the optimization strategy is predicted by combining the multi-step prediction loss function, and the output optimization strategy is obtained by extracting the energy consumption and greenhouse gas emission data from the operation data of the sewage treatment plant and standardizing them as the state , the output instructions of the tactical layer agent and the output instructions of the execution layer agent are defined as actions , the strategy space is defined based on the actions , the energy consumption and greenhouse gas emission data are weighted and summed as the reward function value ; A prediction model is established based on an LSTM network, the input is state and action , and the output is the state and action of the next moment; Establishing a multi-step prediction loss function according to a prediction model ; By a multi-step prediction loss function The prediction model parameters are updated and trained by gradient update, and the trained prediction model is used to output the state at the next time And action Based on the state at the next time And action The reward function value at the next time is calculated ; Initialize current state and action Q-value and use the value network Critic to update the Q-value for the next time step; Computing target Q values by Bellman equation ; The discount factor is adjusted by a gradient descent method to minimize the Q value at the next time The difference between the target Q value The adjusted Q value is calculated between the target Q value The adjustment parameters of the actions in the policy space are extracted as policy parameters in the policy network Actor, and the policy parameters are updated by the policy network Actor. updating the policy parameters by a gradient ascent method q value until next time stop updating after reaching maximum, output updated policy parameters form optimized policy output 7. The digital twin and Al analytics-based wastewater treatment control system of claim 6, wherein: The feedback data is collected after the optimized strategy is output and implemented, and the feedback data is collected in real time through a sensor.

8. The digital twin and Al analytics-based wastewater treatment control system of claim 7, wherein: The optimized strategy is recorded, and the core strategy parameter is extracted by using knowledge distillation. The success degree Z of each implementation of the optimized strategy is calculated according to the feedback data after the real-time optimized strategy is obtained. The optimized strategies with the success degree greater than a threshold value are screened to form a list, and the core strategy parameter is extracted from the list by weighted average.

9. The digital twin and Al analytics-based wastewater treatment control system of claim 8, wherein: The core strategy parameter of all strategy parameters is output from the list.

10. The digital twin and Al analytics-based wastewater treatment control system of claim 9, wherein: The formed optimized strategy is displayed. The core strategy parameter of the optimized strategy is recorded in the database. The core strategy parameter of the optimized strategy is stored in the database synchronously with the optimized strategy and an optimization record is formed.

Images