Drainage control method, device and server based on cloud edge collaboration and digital twinning

Abstract

The application provides a drainage control method and device based on cloud edge cooperation and digital twinning and a server, and relates to the technical field of intelligent control, which comprises the following steps: acquiring static data and real-time monitoring data of a drainage system of a target urban area, and constructing a digital twinning cooperative simulation platform based on the static data and the real-time monitoring data; performing centralized training processing of a central hybrid network deployed in the cloud based on the digital twinning cooperative simulation platform, obtaining a trained hybrid network model, and packaging network parameters in the hybrid network model into an edge inference package and issuing the edge inference package to each edge intelligent agent deployed locally on each drainage equipment; collecting state data and decision data of each edge intelligent agent in the actual operation process, and uploading the state data and the decision data to the cloud to perform incremental update processing on a Q network, and obtaining an updated target network model. The application can significantly improve the accuracy of overflow pollution control and enable the drainage system to operate stably.

Description

Technical Field

[0001] This invention relates to the field of intelligent control technology, and in particular to a drainage control method, device and server based on cloud-edge collaboration and digital twin. Background Technology

[0002] Currently, related technologies suggest that smart drainage systems can be used to solve the problem of overflow pollution in urban drainage systems. By utilizing technologies such as the Internet of Things, big data, and artificial intelligence, real-time monitoring and intelligent scheduling of facilities such as drainage pipe networks, pumping stations, and sewage treatment plants can be achieved to realize unified management of the drainage system. However, the management of overflow pollution in urban drainage systems involves a large number of devices. When managing a large number of devices simultaneously, existing smart drainage systems suffer from poor system coordination capabilities and insufficient operational stability, resulting in poor system stability and low accuracy in controlling overflow pollution. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a drainage control method, device and server based on cloud-edge collaboration and digital twin, which can significantly improve the accuracy of overflow pollution control and enable the drainage system to operate stably.

[0004] In a first aspect, embodiments of the present invention provide a drainage control method based on cloud-edge collaboration and digital twins. The method includes: acquiring static data and real-time monitoring data of the drainage system in a target urban area, and constructing a digital twin collaborative simulation platform based on the static data and real-time monitoring data. The digital twin collaborative simulation platform includes: a drainage network hydraulic and water quality coupling module, a sewage treatment plant biochemical reaction module, and a receiving water body water quality response module. Based on the digital twin collaborative simulation platform, a multi-agent centralized training process is performed on a central hybrid network deployed in the cloud to obtain a trained hybrid network model. The network parameters in the hybrid network model are packaged into edge inference packages and distributed to various edge agents deployed locally on each drainage device. This allows each edge agent to independently execute distributed real-time decision processing using the locally deployed Q network based on local real-time observation data and edge inference packages, and output local control commands for each drainage device. The state data and decision data of each edge agent during actual operation are collected and uploaded to the cloud. The Q network is incrementally updated to obtain an updated target network model, which is then used to execute a new round of drainage control.

[0005] In one implementation, the steps of constructing a digital twin collaborative simulation platform based on static data and real-time monitoring data include: constructing a drainage network hydraulic-water quality coupling module based on the one-dimensional Saint-Venant equations and convection-diffusion-reaction equations to simulate unsteady flow processes and water quality migration and transformation processes within the pipe network; constructing a wastewater treatment plant biochemical reaction module based on an activated sludge model to simulate carbon oxidation, nitrification, denitrification, biological phosphorus removal, and chemical precipitation processes; and constructing a receiving water body water quality response module based on a one-dimensional river water quality model to simulate the convection-diffusion and attenuation processes of pollutants in the receiving water body.

[0006] In one implementation, after the step of constructing the digital twin co-simulation platform, the method includes: receiving real-time monitoring data of the physical drainage system through a two-way communication interface, wherein the real-time monitoring data includes: rainfall radar data, water level monitoring data, flow monitoring data, and online water quality analysis data; and performing spatiotemporal unification processing on the real-time monitoring data to unify real-time monitoring data of different frequencies and spatial locations to the spatiotemporal grid nodes corresponding to various simulation models in the digital twin co-simulation platform, so as to use the real-time monitoring data to calibrate the digital twin co-simulation platform and obtain the target digital twin co-simulation platform.

[0007] In one embodiment, the step of calibrating a digital twin co-simulation platform using real-time monitoring data to obtain a target digital twin co-simulation platform includes: using an event-driven rapid calibration mechanism and a periodic deep calibration mechanism, based on real-time monitoring data, to perform a two-level hybrid calibration process on the model parameters of the digital twin co-simulation platform to obtain the target digital twin co-simulation platform. The event-driven rapid calibration mechanism is triggered when the rainfall intensity exceeds a preset threshold, and the periodic deep calibration mechanism is automatically executed quarterly or annually.

[0008] In one implementation, the steps of performing centralized multi-agent training on a central hybrid network deployed in the cloud based on a digital twin collaborative simulation platform to obtain a trained hybrid network model include: fusing the local Q-values ​​of each edge agent through the central hybrid network to obtain a Q-value estimate of the global joint action; and updating the parameters of the central hybrid network based on the difference between the Q-value estimate of the global joint action and the target Q-value until convergence, thereby obtaining the trained hybrid network model.

[0009] In one implementation, the step of packaging network parameters in the hybrid network model into an edge inference package and distributing it to various edge agents deployed locally on each drainage device includes: packaging network parameters, observation standardization parameters, security constraint rule set and model version metadata in the hybrid network model into an edge inference package, and transmitting the parameter changes in the edge inference package to each edge agent through differential update technology.

[0010] In one implementation, the step of enabling each edge agent to independently execute distributed real-time decision processing based on local real-time observation data and edge inference packets using a locally deployed Q-network, and outputting local control commands for each drainage device, includes: obtaining observation vectors based on local real-time observation data, and updating the parameters of the locally deployed Q-network based on edge inference packets to obtain an updated Q-network, wherein the observation vectors include: local real-time observation vectors, global state summaries, and historical action memories; inputting the observation vectors into the updated Q-network to obtain Q-value estimates corresponding to each selectable control action, and determining the local control commands for each drainage device at the current moment based on the Q-value estimates using an adaptive exploration strategy.

[0011] Secondly, embodiments of the present invention also provide a drainage control device based on cloud-edge collaboration and digital twins. The device includes: a platform construction module, which acquires static data and real-time monitoring data of the drainage system in a target urban area, and constructs a digital twin collaborative simulation platform based on the static data and real-time monitoring data. The digital twin collaborative simulation platform includes: a drainage network hydraulic-water quality coupling module, a sewage treatment plant biochemical reaction module, and a receiving water body water quality response module; and a cloud training module, which performs centralized multi-agent training on a central hybrid network deployed in the cloud based on the digital twin collaborative simulation platform to obtain a trained hybrid network model. The model is designed to package network parameters from a hybrid network model into edge inference packages, which are then distributed to edge agents deployed locally on each drainage device. This allows each edge agent to independently execute distributed real-time decision processing using the locally deployed Q network based on local real-time observation data and edge inference packages, and output local control commands for each drainage device. The model update module collects state and decision data from each edge agent during actual operation, uploads the state and decision data to the cloud, and incrementally updates the Q network to obtain the updated target network model, which is then used to execute a new round of drainage control.

[0012] Thirdly, embodiments of the present invention also provide a server, including a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement any of the methods provided in the first aspect.

[0013] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer-executable instructions, which, when invoked and executed by a processor, cause the processor to implement any of the methods provided in the first aspect.

[0014] The embodiments of the present invention bring the following beneficial effects: This invention provides a drainage control method, device, and server based on cloud-edge collaboration and digital twins. After acquiring static and real-time monitoring data of the drainage system in a target urban area, the method constructs a digital twin collaborative simulation platform based on these data. Then, based on this platform, it performs centralized multi-agent training on a central hybrid network deployed in the cloud, obtaining a trained hybrid network model. The network parameters in this model are packaged into edge inference packets and distributed to the edge agents deployed locally on each drainage device. Each edge agent then independently executes distributed real-time decision processing using its locally deployed Q-network based on local real-time observation data and the edge inference packets, outputting local control commands for each drainage device. Finally, it collects the state and decision data of each edge agent during actual operation and uploads them to the cloud. The Q-network is then incrementally updated to obtain an updated target network model, which is used to execute a new round of drainage control. This invention significantly improves the accuracy of overflow pollution control and ensures stable operation of the drainage system.

[0015] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained in accordance with the structures particularly pointed out in the description, claims and drawings.

[0016] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0017] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a drainage control method based on cloud-edge collaboration and digital twin provided in an embodiment of the present invention; Figure 2 A schematic diagram illustrating the specific process of a drainage control method based on cloud-edge collaboration and digital twin provided in an embodiment of the present invention; Figure 3 A schematic diagram of a drainage control device based on cloud-edge collaboration and digital twin provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a server provided in an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Currently, related technologies suggest that smart drainage systems can be used to solve the problem of overflow pollution in urban drainage systems. Utilizing technologies such as the Internet of Things (IoT), big data, and artificial intelligence (AI), these systems achieve unified management of the drainage system through real-time monitoring and intelligent scheduling of facilities such as drainage networks, pumping stations, and sewage treatment plants. However, the management of overflow pollution in urban drainage systems involves a large number of devices, and existing smart drainage systems suffer from the following problems when managing numerous devices simultaneously: First, multi-agent control schemes often involve simple aggregation of independent decisions, failing to form effective collaboration between local decision-making and global optimization, easily leading to system imbalance due to optimization of a single device; second, integrated models and intelligent algorithms often rely on unidirectional data input relationships, lacking closed-loop coupling mechanisms, resulting in insufficient robustness under complex conditions such as extreme rainfall; third, equipment control often employs start-stop regulation, which can easily lead to increased equipment wear and system operational shocks, affecting the long-term stable operation of the drainage system. Based on this, the drainage control method, device, and server based on cloud-edge collaboration and digital twins provided by this invention can significantly improve the accuracy of overflow pollution control, ensuring stable operation of the drainage system.

[0021] See Figure 1 The diagram shows a flow chart of a drainage control method based on cloud-edge collaboration and digital twins. The method mainly includes the following steps S102 to S106: Step S102: Obtain static data and real-time monitoring data of the drainage system in the target urban area, and construct a digital twin collaborative simulation platform based on the static data and real-time monitoring data. The digital twin collaborative simulation platform includes: a drainage network hydraulic and water quality coupling module, a sewage treatment plant biochemical reaction module, and a receiving water body water quality response module.

[0022] In one implementation, firstly, static and real-time monitoring data of the drainage system in the target urban area are acquired. Static data refers to basic information about the drainage system that does not change or changes slowly over time, specifically including: the topology of the drainage network, i.e., the connection relationships and flow directions between pipe sections; pipe attributes, including pipe diameter, length, slope, and roughness coefficient; design parameters of pumping stations and sewage treatment plants, such as pump rated power, start-up and shutdown water levels, and treatment process parameters; and topographic data, such as ground elevation and catchment area delineation. Real-time monitoring data refers to dynamic data collected in real time by various sensors deployed in the physical drainage system, specifically including: rainfall radar data, used to obtain the spatiotemporal distribution of rainfall intensity; water level monitoring data, used to obtain the water level height at network nodes and pumping station forebays; flow monitoring data, used to obtain the instantaneous flow rate in pipes or channels; and online water quality analysis data, used to obtain the concentrations of pollutants such as chemical oxygen demand (COD), ammonia nitrogen, total phosphorus, and total nitrogen.

[0023] Based on the aforementioned static and real-time monitoring data, a digital twin collaborative simulation platform was constructed. This platform is a virtual mapping system corresponding to the physical drainage system, capable of simulating and predicting the operational status of the physical system in digital space. Specifically, the platform comprises three core modules: a drainage network hydraulic-water quality coupling module, a wastewater treatment plant biochemical reaction module, and a receiving water body water quality response module. The drainage network hydraulic-water quality coupling module is constructed based on the one-dimensional Saint-Venant equations and convection-diffusion reaction equations, used to simulate the flow process of runoff generated by rainfall in the pipeline, the water level change process, and the migration and transformation process of pollutants during transportation. The wastewater treatment plant biochemical reaction module is constructed based on an activated sludge model, used to simulate the degradation process of organic matter by microorganisms, nitrification and denitrification processes, biological phosphorus removal processes, and chemical agent addition reactions after wastewater enters the treatment plant. The receiving water body water quality response module is constructed based on a one-dimensional river water quality model, used to simulate the dilution, diffusion, and attenuation processes of pollutants in the water body after the treated effluent is discharged into the river. Through the coupled operation of the above three modules, the digital twin collaborative simulation platform can fully reproduce the entire chain process of rainfall, runoff, pipeline transportation, sewage treatment and receiving water body response, providing a high-fidelity simulation environment for subsequent intelligent decision-making.

[0024] Step S104: Based on the digital twin collaborative simulation platform, perform centralized training processing of multiple agents on the central hybrid network deployed in the cloud to obtain the trained hybrid network model. Then, package the network parameters in the hybrid network model into edge inference packages and send them to the edge agents deployed locally on each drainage device. This enables each edge agent to independently perform distributed real-time decision processing based on local real-time observation data and edge inference packages, using the locally deployed Q network, and output local control commands for each drainage device.

[0025] In one implementation, based on the digital twin collaborative simulation platform constructed above, centralized training of multiple agents is performed on a central hybrid network deployed in the cloud. The central hybrid network refers to a neural network structure deployed in a cloud computing center. Its function is to receive the local Q-values ​​output by all edge agents during training and output a Q-value estimate of the global joint action through fusion processing. Edge agents are embedded decision-making units deployed locally on each drainage device; each edge agent corresponds to a specific drainage control device, such as a pumping station, a gate, or a dosing pump. Centralized training means that the Q-networks of all edge agents and the central hybrid network are trained uniformly in a cloud-based digital twin simulation environment. During training, each agent can acquire global information, thereby learning the optimal strategy for collaborative cooperation.

[0026] After training, a trained hybrid network model is obtained. This model includes the parameters of the central hybrid network and the parameters of each edge agent's Q-network. Subsequently, the network parameters in the hybrid network model are packaged into an edge inference package. The edge inference package is a data package for deployment, which includes: the weights and bias parameters of the edge agent's Q-network, the mean and standard deviation required for observation standardization, a set of safety constraint rules, and model version metadata. The set of safety constraint rules includes physical constraints such as the minimum start-up and shutdown time of the device and the maximum rate of change, which are used to ensure that the decisions of the edge agents do not violate the safe operating boundaries of the device.

[0027] Then, differential update technology is used to distribute the edge inference package to the various edge agents deployed locally on each drainage device. Differential update refers to transmitting only the parameter differences between the current version of the model and the previous version, rather than transmitting all parameters, thus significantly reducing the communication overhead between the cloud and the edge. After receiving the edge inference package, the edge agent updates its locally deployed Q-network according to the network parameters contained therein. Afterward, each edge agent can independently execute distributed real-time decision processing based on local real-time observation data. That is, each edge agent only relies on data collected by local sensors and model parameters distributed from the cloud, without needing real-time communication with other edge agents, to output local control commands for various drainage devices, such as pump start / stop, valve opening / closing, and chemical dosing pump acceleration rate. This centralized training and distributed execution architecture ensures both the cooperative optimality of the global strategy and achieves millisecond-level local response speed.

[0028] Step S106: Collect the state data and decision data of each edge agent during actual operation, upload the state data and decision data to the cloud, perform incremental update processing on the Q network to obtain the updated target network model, and use the target network model to execute a new round of drainage control.

[0029] In one implementation, during actual system operation, each edge agent continuously collects its state and decision data. Specifically, the state data includes information such as local water level, rainfall intensity, inflow and outflow rates, influent water quality indicators, and equipment status observed by the edge agents. This data reflects the actual operating status of the drainage system at the current moment. The decision data includes the control action selected by the edge agent based on the Q-value estimated by the Q-network output, and the reward value obtained after executing the action. The reward value is a feedback signal calculated based on the control effect; for example, positive effects such as reduced overflow and reduced energy consumption will generate positive rewards. The edge agents combine the above state data, decision data, and the new state after executing the action and upload them to the cloud.

[0030] After receiving the aforementioned data, the cloud performs incremental updates on the Q-network. Incremental updates refer to adjusting the model parameters locally based on the existing model using newly collected data, rather than retraining the entire model from scratch. This allows the model to gradually adapt to changes in the environment. Incremental updates involve two levels: The first level is at the edge agent side, where the local Q-network is fine-tuned using online gradient descent based on locally collected new data. Specifically, the gradient is calculated and parameters are updated for each new sample received, enabling the edge agent to quickly adapt to slow changes in local equipment, such as efficiency decline due to pump aging. To prevent significant differences between the fine-tuned and original policies, leading to abrupt behavioral changes, KL divergence constraints are used to limit the degree of difference, ensuring the new policy's behavior doesn't deviate too far from the deployed policy. The second level is at the cloud, where a centralized retraining process is triggered quarterly or semi-annually. This process mixes real data accumulated during online operation with offline training data and uses elastic weight consolidation to update the parameters of the central hybrid network. This prevents the model from forgetting learned skills when learning new knowledge, i.e., preventing catastrophic forgetting.

[0031] Through the incremental update process described above, an updated target network model is obtained. This model is used to execute a new round of drainage control, thus forming a complete closed loop of simulation training, cloud deployment, edge execution, data feedback, and model update. This closed-loop mechanism enables the system to continuously learn from actual operation, gradually adapting to long-term, slow changes such as pipeline aging, urban development, and climate change, and maintaining long-term stable control performance.

[0032] The drainage control method based on cloud-edge collaboration and digital twin provided in this embodiment of the invention can significantly improve the accuracy of overflow pollution control and enable the drainage system to operate stably.

[0033] See Figure 2The diagram shows a specific flow chart of a drainage control method based on cloud-edge collaboration and digital twins. This embodiment of the invention also provides an implementation method for coordinated control of urban rainy season overflow pollution, as detailed in (1) to (3) below: (1) Construct a digital twin collaborative simulation platform for the urban drainage system. First, collect static information such as the drainage network topology, pipeline attributes, design parameters of pumping stations and sewage treatment plants, and topographic data of the target urban area. Second, integrate hydrological and hydrodynamic and water quality transformation mechanism models to construct a comprehensive system that integrates mechanism models, real-time data-driven systems, and online parameter calibration to ensure that the model maintains excellent prediction accuracy under all weather and operating conditions. See (1-1) to (1-3) below for details: (1-1) Deep coupling and integration of mechanism models. In one implementation, a hydraulic and water quality coupling module for drainage pipe networks can be constructed based on the one-dimensional Saint-Venant equations and convection-diffusion reaction equations to simulate the unsteady flow process and water quality migration and transformation process in the pipe network; a biochemical reaction module for sewage treatment plants can be constructed based on the activated sludge model to simulate the carbon oxidation process, nitrification process, denitrification process, biological phosphorus removal process and chemical precipitation process; and a receiving water body water quality response module can be constructed based on the one-dimensional river water quality model to simulate the convection-diffusion and attenuation process of pollutants in the receiving water body.

[0034] (1-1-1) Drainage network hydraulic and water quality coupling module, mainly including water dynamics model and water quality migration and transformation mechanism model.

[0035] Hydraulic dynamics model: The unsteady flow in the pipe channel is described using the one-dimensional Saint-Venant equations, and the governing equations are as follows: Continuity equation:

[0036] Momentum equation:

[0037] Where A is the cross-sectional area of ​​the water passage (m²), Q is the flow rate (m³ / s), t is the time (s), and x is the friction distance (m). q l denoted as lateral inflow (flow rate per unit length, m² / s), h as water depth (m), and g as gravitational acceleration (m / s²). S f The friction slope is usually calculated using the Manning formula. , where n is the Manning roughness coefficient and R is the hydraulic radius (m).

[0038] Water quality migration and transformation mechanism model: considering the movement of suspended solids (SS) and dissolved pollutants. For each pollutant component... C i(e.g., COD, NH3-N), described using convection-diffusion reaction equations:

[0039] Where E is the longitudinal diffusion coefficient (m² / s). S r,i Source and sink terms (g / m³ / s) for biochemical reactions or physical processes (such as sedimentation / resuspension) within the pipeline. S ext,i This is the external input source term. The sedimentation-scour model uses the critical shear force criterion for dynamic simulation.

[0040] (1-1-2) Biochemical reaction module of wastewater treatment plant.

[0041] Based on Activated Sludge Model 2 (ASM2d), a kinetic model incorporating carbon oxidation, nitrification, denitrification, biological phosphorus removal, and chemical precipitation was constructed. The core of the model is a set of parameters describing each biological component. X j (such as heterotrophic bacteria, autotrophic bacteria, polyphosphate-accumulating bacteria) and soluble components S i A system of ordinary differential equations for the conservation of mass of substances such as ammonia nitrogen, nitrate, orthophosphate, and COD:

[0042]

[0043] in, Q in This indicates the influent flow rate of the wastewater treatment plant (m3 / d). Q r Indicates the internal / external reflux flow rate (m3 / d); Q w V represents the discharge flow rate of excess sludge (m3 / d); V is the effective volume of the reactor. X in,j This indicates the concentration (g / m3) of the j-th type of biological component in the influent. X r,j This indicates the concentration (g / m3) of the j-th type of biological component in the returned sludge. X w,j Concentration (g / m3) of the j-th type of biological component in the residual sludge.

[0044] S in,i Concentration (g / m3) of the i-th type of dissolved component in the influent. S r,i Concentration (g / m3) of the i-th type of soluble component in the returned sludge; ρ j,k and ρ i,kIt is the net conversion rate (g / m³d) of component j or i by the kth biological / chemical process (such as microbial growth, death, hydrolysis, oxidation, precipitation). X j Concentration (g / m3) of the jth type of biological component (particulate form); S i Concentration (g / m3) of the i-th type of soluble component.

[0045] (1-1-3) Receiving water body water quality response module.

[0046] A one-dimensional river water quality model is used, with the governing equations being the convection-diffusion-reaction equations:

[0047] Where C is the pollutant concentration (g / m³), t is time (s); u is the cross-sectional average flow velocity (m / s), D is the longitudinal dispersion coefficient (m² / s), k is the comprehensive attenuation coefficient (1 / d), and W(x,t) is the emission rate of point / area source pollution load along the path (g / s). A r The water-carrying area of ​​the river channel is (m²).

[0048] (1-2) Real-time data perception and interface standardization: A two-way communication interface with the Supervisory Control and Data Acquisition Network (SCADA) is established. This interface can not only acquire data in real time, but also inject virtual control commands into the digital twin model. Specifically, real-time monitoring data of the physical drainage system can be received through the two-way communication interface. Then, by performing spatiotemporal unification processing on the real-time monitoring data, real-time monitoring data of different frequencies and spatial locations are unified to the spatiotemporal grid nodes corresponding to various simulation models in the digital twin co-simulation platform. The digital twin co-simulation platform can be calibrated using the real-time monitoring data to obtain the target digital twin co-simulation platform. The real-time monitoring data includes: rainfall radar data, water level monitoring data, flow monitoring data, and online water quality analysis data.

[0049] In one implementation, real-time data from rainfall radar, online water level / flow meters, and water quality analyzers (monitoring COD, NH3-N, TN, TP, etc.) can be accessed. Data cleaning is performed using sliding window mean filtering and outlier detection based on physical constraints (e.g., water level should not exceed the top of the pipe). Then, monitoring data from different frequencies (e.g., 5-minute water level, 1-hour water quality) and spatial locations are unified onto the spatiotemporal grid nodes required by the simulation model using spatiotemporal kriging interpolation or linear interpolation methods.

[0050] (1-3) Dynamic Calibration Engine: A two-level hybrid calibration strategy is introduced to solve the problem of traditional models undergoing one-time calibration and long-term drift. Specifically, an event-driven rapid calibration mechanism and a periodic deep calibration mechanism can be used to perform two-level hybrid calibration processing on the model parameters of the digital twin co-simulation platform based on real-time monitoring data to obtain the target digital twin co-simulation platform. The event-driven rapid calibration mechanism is triggered when the rainfall intensity exceeds a preset threshold, and the periodic deep calibration mechanism is executed automatically on a quarterly or annual basis.

[0051] (1-3-1) Event-driven rapid calibration (short timescale, minute level): When the real-time rainfall intensity exceeds the threshold Ithreshold or the water level change rate at the monitoring node exceeds Δh / Δt threshold At that time, the calibration engine will start automatically.

[0052] Firstly, for the rainfall-runoff-pipeline response process, a set of highly sensitive and time-varying parameters, such as the surface runoff coefficient C, are rapidly optimized. runoff The initial thickness of the sediment in the pipeline, dsed,0, and the local correction factor αn for the Manning coefficient n are used. Then, the objective function is defined as the simulated water level at key monitoring points within a short future time window Tcal (e.g., 30 minutes). h sim Compared with the actual water level h obs Minimize the normalized root mean square error (NRMSE):

[0053] In the formula, θ={C runoff Let ,αn,...} be the vector of parameters to be optimized. This represents the average observed water level (m). Let t represent the actual water level (m) measured on site at time t. h sim (t,θ) At time t, the simulated water level (m); N is the total number of data entries within the calibration time window (e.g., 1 data entry per minute within 30 minutes, N=30); t represents the time step.

[0054] Then, the existing particle swarm optimization (PSO) algorithm is used to optimize the function until it converges, and the results are immediately applied to subsequent simulations of this event.

[0055] (1-3-2) Periodic depth calibration (long timescale, quarterly / annual): Automatically performed once every quarter or half a year, using comprehensive weather data accumulated in the past period.

[0056] First, a variance-based global sensitivity analysis method is used to screen all adjustable parameters of the model. Then, the parameter θ is calculated.i The order sensitivity index S i and total sensitivity index S Ti :

[0057]

[0058] Where Y is the model output (such as the terminal water flow rate). θ ~i Indicates except θ i All parameters other than those mentioned above; Expressing conditional expectation; V represents the expected variance; ar (Y) represents the total variance of the model output Y; θ i This represents the adjustable parameters of the i-th model to be analyzed.

[0059] Filter out S Ti A subset of global sensitive parameters Θ that are greater than a threshold (e.g., 0.05). sensitive Existing multi-objective genetic algorithms are used to analyze Θ. sensitive Optimization was performed by selecting a set of optimal compromise solutions as calibrated parameters through Pareto front analysis and updating the model database.

[0060] (2) Deployment of an edge-cloud collaborative multi-agent decision-making and learning architecture. By constructing a hybrid intelligent decision-making system with centralized training, distributed execution, cloud collaboration, and edge autonomy, the aim is to solve the fundamental contradictions of high latency and poor robustness in traditional centralized control, and the difficulty in achieving global optimization in fully distributed decision-making. This architecture innovatively separates the training and execution environments, achieving a combination of millisecond-level local response and day-level global policy optimization, as detailed in (2-1) to (2-3) below: (2-1) Specific design and implementation of edge agents. Each edge agent is an embedded decision-making unit deployed on the local controller or regional edge computing node of physical facilities (pump stations, valves, etc.) and is responsible for real-time control with ultra-low latency.

[0061] (2-1-1) Construction and standardization of observation vectors. The observation vector of the i-th edge agent at time t. It consists of three parts:

[0062] In the formula: Local real-time observations (local water level, rainfall, flow, etc.) have been normalized (mapped to the 0-1 range); This represents the transformation of historical rainfall sequences (rainfall data within the past Δ time period) into high-latitude feature vectors through encoding and mapping. The status at the previous moment (historical data of water level / system status in the previous step).

[0063] (a) Local real-time observation The calculation formula is as follows:

[0064] in, Local water level (e.g., forebay water level, in meters); Local rainfall intensity (mm / h); , : Inflow / outflow rate (m³ / s); Key water quality indicators for influent (COD, NH3-N, mg / L); : Device status vector (running / stopping, fault flag, cumulative running time, etc.).

[0065] (b) Global State Summary (Source: Cloud, communication delay Δ applies):

[0066] System average fill Downstream key section water quality risk index R risk,t Electricity price period code E price,t (Peak, Flat, Valley); Compressed coding vector for future k-step rainfall forecast Encoding is implemented using a lightweight autoencoder; Δ: cloud communication latency, with an update broadcast every Tg time (e.g., 15 minutes).

[0067] (c) Historical action memory The calculation formula is as follows:

[0068] In the formula: This indicates the control action (pump opening status) at time tm. This represents the local reward value at time tm; m represents the memory step size.

[0069] Continuous variables are normalized using an online sliding window:

[0070] Where, μ win(t) and σ win(t) It represents the mean and standard deviation of the variable over the past W time windows.

[0071] (2-1-2) Action Space Discretization Strategy. Different discretization schemes are adopted according to the equipment type: Pump station (discrete equipment): Action space A pump ={0,1,2,...,N pump} indicates the number of pumps that are turned on.

[0072] Variable frequency pump / regulating weir (continuous equipment): Adjusts the range [u] min ,u max Discretize into L levels. During discretization, for the Lth level (l=0,1,2,3….L-1), the corresponding control command value is... The relevant calculation expression is: empty movement A var It is all The finite set formed: .

[0073] Dosing pump: Assume its allowable dosing rate range is [0, R max Discretize the range into L dose If there are several levels, then the action space is:

[0074] in, R max This indicates the maximum dosing rate of the dosing pump. L dose Indicates the number of discretization bits.

[0075] (2-1-3) Lightweight Neural Network Architecture. The Q-Network (Q-Network represents the action-value network) of the edge agent adopts a hybrid structure of deep residual network and gating mechanism to reduce computational cost while ensuring accuracy:

[0076] This represents the observation vector embedding, which integrates the observation vectors of the edge agent. (Local + Global + Historical) is mapped to a low-dimensional feature vector.

[0077]

[0078] Representation layer normalization standardizes the features of the previous layer, accelerating training convergence and improving numerical stability.

[0079]

[0080] This represents the main path of the residual block, using a fully connected layer with ReLU activation to extract nonlinear features.

[0081]

[0082] This indicates a skip connection (residual connection), where the input is directly superimposed on the main path output, mitigating gradient vanishing and improving the training effect of deep networks.

[0083]

[0084] This represents a gated loop unit. Sequence feature modeling preserves temporal dependencies (such as historical water levels and historical actions).

[0085]

[0086] This represents the value output. The final features are mapped to Q-values ​​(action values) to determine the optimal control strategy.

[0087] in, It is the Q-value estimate for each optional action, network parameters Less than 100KB, it can complete forward computation within 10ms on typical edge devices such as ARM Cortex-A72.

[0088] (2-1-4) Constrained Exploration-Exploitation Strategy. An adaptive ε-greedy strategy (i.e., an adaptive exploration strategy) is adopted, where the ε value is dynamically adjusted according to the device's operating status:

[0089] in, Let represent the exploration rate of the i-th agent at time t; base Base exploration rate; t (current time); The time since the device last switched states; : Target time for stable equipment operation; β: Decay coefficient, which temporarily increases the exploration probability after the equipment switches states, encouraging the strategy to conduct sufficient exploration during the critical period after the state change in order to adapt to the current working conditions as soon as possible; as the stable operation time of the equipment increases, the exploration rate automatically decays, reducing the exploration probability, reducing unnecessary action changes, protecting the equipment, and reducing mechanical wear.

[0090] (2-2) Cloud-based hybrid network and centralized training mechanism. A digital twin training environment and a central hybrid network are deployed in the cloud to handle offline and online training. In one implementation, the local Q-values ​​of each edge agent are fused through the central hybrid network to obtain a Q-value estimate of the global joint action. Based on the difference between the Q-value estimate of the global joint action and the target Q-value, the parameters of the central hybrid network are updated until convergence, resulting in the trained hybrid network model.

[0091] (2-2-1) Structure and Monotonicity Guarantee of Hybrid Networks. Hybrid networks receive the local Q-value vectors Qi of all edge agents. t =[Q t 1 ,...,Q t N and global state s t Output the Q-value estimate of the global joint action. .

[0092] A monotonic hybrid network is used, and its structure satisfies:

[0093] This is achieved through a hypernetwork: a. Global feature vector extraction:

[0094] MLP global This represents a multilayer perceptron, responsible for processing complex global states s. t Processed into feature vector f t .

[0095] b. Weight generation:

[0096] w t Represents the action value weights used to generate each edge agent; Linear w : Represents a fully connected layer, responsible for extracting features f t Maps the original values ​​of the weights; Softplus represents the activation function, ensuring all weights are positive; This means that an N-dimensional vector is generated, with each element corresponding to the weight of an agent.

[0097]

[0098] This indicates the generation of a bias term, used to fine-tune the network output.

[0099] c. Monotonic Mixing:

[0100] Where τ is the action history sequence of all agents; a t Represents the current joint action vector. The set of actions of all agents at time t; w t,i This represents the dynamic weight of the i-th agent at time t; b represents the local Q-value of the i-th agent; t Global bias term.

[0101] (2-2-2) Details of the intensive training algorithm. Training employs a multi-objective, phased Q-mapping method. MIX algorithm: Phase 1: Pre-training and Imitation Learning.

[0102] Objective: To quickly obtain a secure baseline strategy.

[0103] Method: Using historical expert operation data, pre-trained edge agents are generated through behavioral cloning.

[0104]

[0105] in, This indicates that for the expert dataset D exp Calculate the expected value (average) of the samples in the sample; a represents the observation state of the i-th agent; i This indicates the expert's actions in this state (such as starting or stopping the pump). This indicates that the agent is in state o. i Next, select action a. i The probability of π θi The strategy is obtained by using the Q-value through softmax.

[0106] Phase Two: Multi-objective Collaborative Optimization.

[0107] Training cycle: Experience collection: In a digital twin environment, the agent interacts with the environment according to the current policy and stores experience tuples (o t ,a t ,R t ,o t+1 ) to shared playback buffer B.

[0108] ② Prioritized Experience Replay: The importance weight of each experience is: pj=∣δj∣+

[0109] Where, δ j It represents the TD error and the sampling probability.

[0110]

[0111] ③ Dual-Q learning and target network: using two Q networks (target network θ) And online networks (θ) reduce overestimation.

[0112] It should be noted that, within the double-Q learning framework, the weights w of the monotonic hybrid network... t and bias b t All are generated from the hypernetworks corresponding to the online networks. In calculating the target value y...j In this case, the action selection is completed by the online network, while the weights of the hybrid network used to evaluate the action are generated by the supernetwork of the target network. This ensures the consistency of the monotonicity constraint of the global Q-value decomposition under the dual-network structure and avoids overestimation or underestimation of Q-value due to asynchronous parameters of the hybrid network.

[0113]

[0114] In the formula: R j Indicates the first j Instant rewards for each step (such as overflow penalties or energy rewards); γ represents the discount factor; τ j+1 The sequence represents the global action history at step j+1; a′ represents the candidate action vector; θ represents the online network parameters and θ' represents the action vector. Target network parameters; arg max a′ Q tot (…;θ - The optimal action is selected using an online network.

[0115] ④ Loss function:

[0116] in, Given the sampling probability P of priority-based empirical replay, calculate the expected value of Q using empirical evidence. tot (τ j ,a j ;θ) The Q-value of the online network's prediction of current experience; λ reg Regularization coefficient; R(θ) regularization term.

[0117] Mean squared error (MSE).

[0118] Phase 3: Robust confrontation training.

[0119] Objective: To improve the robustness of the strategy to uncertainty.

[0120] Method: Inject noise (sensor noise, rainfall forecast error) into the training environment and use adversarial perturbations:

[0121] in, This represents constrained maximization; Indicates the perturbation norm constraint; Indicates the upper limit of the disturbance intensity; θ represents the model loss function; θ represents the model parameters; o+δ represents the observation with perturbation.

[0122] (2-3) Cloud-edge collaboration mechanism and continuous learning process.

[0123] (2-3-1) Model Distribution and Version Management. In one implementation, the network parameters, observation normalization parameters, security constraint rule set, and model version metadata of the hybrid network model can be packaged into an edge inference package. The parameter changes in the edge inference package are transmitted to each edge agent through differential update technology. Then, observation vectors can be obtained based on local real-time observation data, and the parameters of the locally deployed Q network can be updated according to the edge inference package to obtain the updated Q network. The observation vectors include: local real-time observation vectors, global state summary, and historical action memory. The observation vectors are then input into the updated Q network to obtain Q-value estimates corresponding to each optional control action. Based on an adaptive exploration strategy, the local control commands for each drainage device at the current moment are determined according to the Q-value estimates.

[0124] After training, the cloud packages the model into an edge inference package, which includes: ① Edge agent network parameters ; ② Observational standardized parameters (sliding window statistics); ③ Safety constraint rule set; ④ Model version metadata.

[0125] By using differential update technology, only the parameter changes are transmitted, reducing communication overhead:

[0126] In the formula, Δθ i This represents the change in model parameters for the i-th edge agent; After cloud training, the first i New model parameters for each agent; Currently on the edge side, the first i The old model parameters of the agent; if ||Δθ i If ||2<τ, then skip this update.

[0127] (2-3-2) Online Decision Making and Security Monitoring. When the edge agent runs online, in addition to selecting actions based on Q-values, it also integrates three layers of security monitoring: Physical constraint checker:

[0128] Among them, Projected( () indicates the projection action, which corrects the action to within the feasible region; This indicates a safety precaution, based on the current water level. Triggered emergency response strategy; The original action output by the reinforcement learning model; The control action ultimately executed by the i-th edge agent; It is a set of actionable actions based on the current water level.

[0129] Action smoothing filter:

[0130] in, The final action executed after smoothing; The original action output by the RL model at the current moment; The action actually performed in the previous moment; α smoothing coefficient (weight), with a value range of [0,1].

[0131] Emergency braking mechanism: When the water level is detected to exceed the safety threshold, the RL action is immediately overridden, and the preset emergency plan is executed.

[0132] (2-3-3) Data collection and incremental learning. The edge device periodically (e.g., hourly) uploads summary data to the cloud to construct a quantity vector.

[0133]

[0134] Among them, state-action pairs Locally calculated reward value Number of violations of the constraint Equipment operation statistics .

[0135] The cloud-based continuous learning framework combines federated learning with centralized training.

[0136] Federal average (per 24 hours):

[0137] Where N is the total number of agents; n i It is equipment i Number of samples collected; n total Total number of equipment; This represents the local model parameters of the i-th edge node; The aggregate weight of the i-th node.

[0138] Intensive retraining (weekly): Update replay buffer B with all new data; conduct a new round of training in the digital twin environment; use Elastic Weight Consolidation (EWC) to prevent catastrophic forgetting.

[0139] in, This represents the total loss due to the elastic weighting consolidation. F represents the training loss on new data; λ is the regularization strength coefficient; j It is the parameter θ j Fisher's information on the old task; This represents the old model (previous round of intensive training) in its first iteration. j Optimal parameters.

[0140] By minimizing this loss, we obtain the updated global model: .

[0141] (2-3-4) Performance Evaluation and A / B Testing. Before deploying the new strategy, run it in shadow mode: execute the old and new strategies in parallel, but only execute the actions of the old strategy, and compare and evaluate the predictive performance of the new strategy. The following metrics will be used to determine whether to update: The formula for calculating the predicted average value difference is as follows:

[0142] Where T represents the total test duration; Rnew(t) is the predicted reward of the new policy at time t; R old (t) The actual reward of the old strategy at time t. If ΔV>0: the new strategy has higher predictive value (the system is better); if ΔV<0: the new strategy performs worse than the old strategy, and is rejected.

[0143] Changes in constraint violation rate.

[0144] Strategy divergence: The proportion of different actions chosen by the old and new strategies.

[0145] A full update is only performed when the new strategy is significantly better than the old strategy (e.g., ΔV > 5% and the constraint violation rate does not increase).

[0146] (2-4) Communication protocols and system reliability.

[0147] (2-4-1) Lightweight Communication Protocol. Define the message format for cloud-edge communication: Downlink (cloud to edge): Model update, global summary, control parameters.

[0148] Uplink (edge ​​to cloud): running data summary, heartbeat signal, and abnormal alarm.

[0149] It adopts the MQTToverTLS protocol and supports reconnection after disconnection and data compression.

[0150] (2-4-2) Fault tolerance and degradation mechanism.

[0151] Network interruption: The edge device continues to run using the last available model, and cached data will be uploaded after the network is restored.

[0152] Cloud-based failure: The edge device switches to conservative mode and uses a pre-built rule base for control.

[0153] Model anomaly: When an abnormal Q value is detected (such as NaN or excessive variance), the system falls back to the safety controller.

[0154] (3) Closed-loop management of full-cycle learning, robust deployment and autonomous evolution.

[0155] This step defines the complete lifecycle management process of the intelligent control system of this invention, from algorithm training and engineering deployment to continuous evolution. Its core innovation lies in constructing a closed loop of "simulation training - shadow deployment - online learning - secure evolution," ensuring that the system strategy can not only learn from historical data but also securely iterate and optimize continuously from real-world operation, adapting to long-term changes such as pipeline aging, urban expansion, and climate change. See (3-1) to (3-3) below for details: (3-1) Offline training stage: large-scale pre-training based on synthetic and adversarial scenarios.

[0156] The phase is completed in a high-fidelity digital twin training environment in the cloud, with the goal of producing a basic version of the intelligent control strategy that combines high performance and high robustness.

[0157] (3-1-1) Systematic generation of multimodal training datasets.

[0158] The diversity and representativeness of training data are the cornerstone of policy generalization ability. We construct a hybrid rainfall dataset D containing both real and synthetic events. train .

[0159] Real Historical Events Database: Collects historical rainfall, flow, and water quality monitoring data from the past 5-10 years, extracting hundreds of independent rainfall events covering different seasons and magnitudes.

[0160] Synthetic precipitation event generation: By combining a statistical synthesis method based on stochastic processes with a physics-based weather model simulation, anomalous events far exceeding historical records are generated.

[0161] (3-1-2) Layered and progressive course-based training process.

[0162] To avoid convergence difficulties or instability caused by training directly in the most complex scenarios, a course-based learning strategy is adopted, which is carried out in three stages: Phase A: Single-facility basic control training.

[0163] Objective: To enable each edge agent to initially grasp the basic control logic of its corresponding facility.

[0164] Method: In simplified environments (e.g., ignoring strong coupling with other facilities), individual agents are allowed to learn to maximize their local rewards (e.g., a pumping station learns to drain water efficiently and avoid frequent start-stop cycles). A proximal policy optimization (PPO) algorithm is employed to ensure training stability.

[0165] Reward function: Use basic local rewards, such as:

[0166] Among them, h i Let Pi be the water level, and Pi be the real-time power consumption of the i-th device; The local reward of the i-th agent at time t; Equipment start / stop indication function; weight coefficients α, β, γ, ranging from [0,1] and the sum of the three is 1.

[0167] Phase B: Multi-agent collaborative training.

[0168] Objective: To train all agents to work collaboratively to achieve global optimum in a complete digital twin environment.

[0169] Methods: A hybrid algorithmic framework combining Multi-Agent Proximal Policy Optimization (MAPPO) and QMIX is employed. MAPPO is used to stably optimize the policy network of each agent, while QMIX is used to learn an efficient joint value function.

[0170] Centralized Training and Distributed Execution (CTDE): During training, each agent's policy network can learn from global state information; however, during execution, it only relies on local observations.

[0171] Phase C: Robustness and Adversarial Training.

[0172] Objective: To improve the robustness of the strategy to uncertainties (sensor noise, forecast error, model parameter drift).

[0173] method: Dynamic model parameter perturbation: In each training round, a set of digital twin model parameters (such as pipeline Manning coefficient ±20%) are randomly sampled from a preset distribution, forcing the policy to adapt to the uncertainty of the model.

[0174] Adversarial state perturbation: An adversarial perturbation network is added to the agent's observation input. This network learns to generate observation noise that most easily leads to policy failure. The agent's policy then becomes more robust in adversarial dealing with this "worst-case" noise. This constitutes a minimax game:

[0175] In the formula, The minimization operator represents the optimization of the agent's policy parameters θ. The maximization operator represents the optimization against the perturbation δ, constraining the perturbation norm to not exceed... ; Describe the state distribution and calculate the expectation; Indicates the immediate reward function; s t This represents the true state of the system at time t; Represents an agent policy network; o t This represents the agent's observed input at time t; o t +δ represents the observation input with counter-perturbation.

[0176] (3-1-3) Training monitoring, validation and model selection.

[0177] Training monitoring: Real-time tracking of key metrics, including average round reward, reward for each sub-objective, exploration rate, policy entropy, and value function estimation error.

[0178] Independent validation set: A set of historical and synthetic events that were not involved in training is used as the validation set, and testing is performed after a certain number of training steps.

[0179] Model selection criteria: Instead of simply selecting the model with the highest reward, we use the Pareto optimality principle to screen strategy models from the validation set that show a balanced performance across the three dimensions of pollution control, resource consumption, and equipment stability without any serious weaknesses, and save them as the candidate model pool Mcandidate.

[0180] (3-2) Online Deployment Phase: Security-First Shadow Deployment and Gradual Scale-Up. The safe and smooth deployment of the trained model to the physical world is crucial to the success of the project. We adopt a gradual deployment process to minimize risk.

[0181] (3-2-1) Model compilation and edge adaptation.

[0182] Model lightweighting and compilation: The cloud-trained neural network model (usually in PyTorch / TensorFlow format) is compressed through model pruning, quantization and knowledge distillation, and then compiled into an efficient inference engine suitable for edge hardware (such as ARM CPU, NVIDIA Jetson) using a dedicated compiler (such as TensorRT, ONNXRuntime).

[0183] Generate deployment package: In addition to the model files, the deployment package also includes: Observe standardized parameters (mean and standard deviation).

[0184] Action mapping table.

[0185] Security monitoring policy configuration file.

[0186] Version metadata and rollback instructions.

[0187] (3-2-2) Four-stage secure deployment process.

[0188] Phase 1: Shadow mode operation.

[0189] Method: The new strategy model runs in parallel on edge devices, receiving real-time sensor data and generating control commands, but these commands are intercepted and not actually sent to the actuators. The system is still controlled by the old strategy (or basic rules).

[0190] Objective: To compare the differences in instructions between old and new strategies in a completely safe environment, and to use a digital twin model to predict the results if the new strategy is executed. The key evaluation metric is the difference in predictive value (PVD).

[0191] in, The reward is predicted based on the new strategy instructions and the system status; It is the reward predicted based on the old strategy instructions and system state; T represents the total number of time points.

[0192] Phase 2: Small-scale, low-risk environment A / B testing.

[0193] Method: Select 1-2 subsystems that are geographically independent and whose failure consequences are controllable (such as a single pumping station and its small-scale pipeline network), and switch control to the new strategy.

[0194] Monitoring: The subsystem is monitored intensively, and the actual operating indicators (overflow, energy consumption, number of equipment actions) are compared with the predicted values ​​in the shadow mode to verify the model's performance in the real physical world.

[0195] Phase 3: Gradual release of supply in different regions.

[0196] Method: Divide the entire urban drainage network into multiple logical zones. Using zones as units, gradually increase the number of controllers using the new strategy from 10% to 100%. After each increase, observe the overall system stability indicators.

[0197] Decision-making logic for scaling up: Based on statistical tests (such as t-tests), determine whether the new strategy is significantly better than the old strategy in the already scaled-up area, and whether the number of key safety alarms (such as over-limit alarms) has not increased significantly.

[0198] Phase 4: Full network rollout and hot standby of the old system.

[0199] Method: The new strategy will take over full control of the system.

[0200] Hot standby mechanism: The old policy system continues to run in "shadow mode" as a hot backup. Once the new policy triggers a high-level security alarm (such as outputting abnormal values ​​for three consecutive control cycles), the system will automatically switch back to the old policy within 100 milliseconds and report the fault.

[0201] (3-3) Online Learning and Continuous Evolution Stage: Data-Driven Autonomous Performance Improvement. After the system is officially launched, it enters a perpetual "run-learn-optimize" cycle, enabling the control strategy to adapt to slow system changes.

[0202] (3-3-1) Real-time data acquisition, desensitization and quality assessment.

[0203] Data Stream: Edge devices transmit anonymized "state (i.e., state data) - action (i.e., decision data) - reward - new state" tuples at high frequency (e.g., every 5 minutes). t ,a t ,r t ,s t+1 Uploaded to the cloud data lake.

[0204] Data anonymization: Remove all potentially privacy-sensitive geolocation information and generalize the data.

[0205] Data quality assessment: Automatically detect and label outlier data (such as constant values ​​caused by sensor failure or physically impossible values), and construct a clean, high-quality empirical dataset D. online .

[0206] (3-3-2) Incremental learning and periodic retraining.

[0207] Two complementary learning models are employed: Mode A: Online fine-tuning on the edge side.

[0208] Objective: To quickly adapt to local, slowly changing environments (such as a slight decrease in the efficiency of a pumping station).

[0209] Method: On edge devices, based on newly collected local data, the agent's Q-network or policy network is fine-tuned using an online gradient descent method. Strict constraints are employed to prevent catastrophic forgetting and overfitting.

[0210] In the formula, This indicates the losses of the new strategy on new data; This represents the KL divergence between the new and old strategies.

[0211] That is, the new strategy π is required. θ′ With deployed strategy π θ The KL divergence does not exceed the threshold. This ensures that the behavior does not change abruptly.

[0212] Mode B: Periodic centralized retraining in the cloud.

[0213] Objective: To achieve a phased leap in the overall strategy and absorb the operational experience of the entire system.

[0214] Cycle: Triggered once per quarter or every six months.

[0215] method: Data aggregation and course construction: integrating online datasets D online Compared with the original offline dataset D train mix.

[0216] Elastic weighted reinforcement training: To prevent forgetting old skills, constraints on important old parameters are added to the loss function. The total loss function is calculated as follows:

[0217] Where, θ i It is the i-th parameter of the old model. It is the loss function under the new data model; θ i It is the i-th parameter of the current model; F i It is the parameter θ i The Fisher information diagonal matrix is ​​estimated to measure its importance.

[0218] (3-3-3) Performance degradation detection and automatic rollback.

[0219] Monitoring metrics: Continuously track the moving average of core performance indicators (KPIs), such as average single-event overflow rate and energy consumption per unit flow.

[0220] Degradation detection: Statistical process control (SPC) methods are used, such as setting upper control limits (UCL) and lower control limits (LCL) for KPIs. If the indicator consistently exceeds the control limits, a performance degradation alarm is triggered.

[0221] Automatic rollback: Once performance degradation is confirmed and strongly correlated with the new policy version, the system automatically initiates a rollback process, returning control to the previous stable version.

[0222] In summary, the present invention can: I. Enhancing System Collaboration Capabilities. This invention performs centralized multi-agent training on a central hybrid network deployed in the cloud, enabling each edge agent to complete global collaborative optimization in the cloud before independently executing local decisions. This solves the problem of local optimization and overall imbalance caused by simple aggregation after independent decision-making in existing technologies.

[0223] II. Enhancing Closed-Loop Coupling and Robustness. This invention collects state and decision data of edge agents during actual operation and uploads them to the cloud. It then performs incremental updates on the Q-network, forming a closed-loop mechanism of simulation training, cloud deployment, edge execution, data feedback, and model updates. This solves the problem of insufficient robustness caused by unidirectional data input in existing technologies.

[0224] Third, protect equipment and extend its service life. In this invention, the edge agent selects control actions based on an adaptive exploration strategy. The exploration probability of this strategy is dynamically adjusted according to the continuous running time of the equipment. That is, after the equipment has just completed a state switch, the exploration probability is reduced and the action changes are reduced, thereby avoiding frequent start-ups and shutdowns of the equipment, effectively reducing mechanical wear and ensuring long-term stable operation of the system.

[0225] Regarding the drainage control method based on cloud-edge collaboration and digital twins provided in the foregoing embodiments, this invention provides a drainage control device based on cloud-edge collaboration and digital twins. (See attached image.) Figure 3 The diagram shows a structural schematic of a drainage control device based on cloud-edge collaboration and digital twins. The device includes the following parts: Platform construction module 302 acquires static data and real-time monitoring data of the drainage system in the target urban area, and constructs a digital twin collaborative simulation platform based on the static data and real-time monitoring data. The digital twin collaborative simulation platform includes: a drainage network hydraulic and water quality coupling module, a sewage treatment plant biochemical reaction module, and a receiving water body water quality response module. The cloud training module 304, based on the digital twin collaborative simulation platform, performs centralized training processing of multiple agents on the central hybrid network deployed in the cloud to obtain the trained hybrid network model. The network parameters in the hybrid network model are packaged into edge inference packages and distributed to the edge agents deployed locally on each drainage device. This enables each edge agent to independently perform distributed real-time decision processing based on local real-time observation data and edge inference packages, using the locally deployed Q network, and output local control commands for each drainage device. The model update module 306 collects the state data and decision data of each edge agent during actual operation, uploads the state data and decision data to the cloud, performs incremental update processing on the Q network, and obtains the updated target network model, so as to use the target network model to execute a new round of drainage control.

[0226] The drainage control device based on cloud-edge collaboration and digital twin provided in this application embodiment can significantly improve the accuracy of overflow pollution control and enable the drainage system to operate stably.

[0227] In one embodiment, during the step of constructing a digital twin collaborative simulation platform based on static data and real-time monitoring data, the platform construction module 302 is further used to: construct a drainage network hydraulic-water quality coupling module based on the one-dimensional Saint-Venant equations and convection-diffusion reaction equations to simulate the unsteady flow process and water quality migration and transformation process within the pipeline; construct a wastewater treatment plant biochemical reaction module based on an activated sludge model to simulate the carbon oxidation process, nitrification process, denitrification process, biological phosphorus removal process, and chemical precipitation process; and construct a receiving water body water quality response module based on a one-dimensional river water quality model to simulate the convection-diffusion and attenuation process of pollutants in the receiving water body.

[0228] In one embodiment, after the step of constructing the digital twin collaborative simulation platform, the platform construction module 302 is further configured to: receive real-time monitoring data of the physical drainage system through a two-way communication interface, wherein the real-time monitoring data includes: rainfall radar data, water level monitoring data, flow monitoring data, and online water quality analysis data; and perform spatiotemporal unification processing on the real-time monitoring data to unify real-time monitoring data of different frequencies and spatial locations to the spatiotemporal grid nodes corresponding to various simulation models in the digital twin collaborative simulation platform, so as to use the real-time monitoring data to calibrate the digital twin collaborative simulation platform and obtain the target digital twin collaborative simulation platform.

[0229] In one embodiment, when performing the step of calibrating the digital twin co-simulation platform using real-time monitoring data to obtain the target digital twin co-simulation platform, the platform construction module 302 is further configured to: utilize an event-driven rapid calibration mechanism and a periodic deep calibration mechanism to perform a two-level hybrid calibration process on the model parameters of the digital twin co-simulation platform based on real-time monitoring data to obtain the target digital twin co-simulation platform. The event-driven rapid calibration mechanism is triggered when the rainfall intensity exceeds a preset threshold, and the periodic deep calibration mechanism is automatically executed quarterly or annually.

[0230] In one embodiment, when performing centralized training of multiple agents on a central hybrid network deployed in the cloud based on a digital twin collaborative simulation platform to obtain a trained hybrid network model, the cloud training module 304 is further configured to: fuse the local Q-values ​​of each edge agent through the central hybrid network to obtain a Q-value estimate of the global joint action, and update the parameters of the central hybrid network according to the difference between the Q-value estimate of the global joint action and the target Q-value, until convergence, to obtain the trained hybrid network model.

[0231] In one embodiment, when performing the step of packaging the network parameters in the hybrid network model into an edge inference package and distributing it to the various edge agents deployed locally on each drainage device, the cloud training module 304 is further configured to: package the network parameters, observation standardization parameters, security constraint rule set and model version metadata in the hybrid network model into an edge inference package, and transmit the parameter changes in the edge inference package to each edge agent through differential update technology.

[0232] In one embodiment, when performing the step of enabling each edge agent to independently execute distributed real-time decision processing based on local real-time observation data and edge inference packets using a locally deployed Q-network to output local control commands for each drainage device, the cloud training module 304 is further configured to: obtain observation vectors based on local real-time observation data, and update the parameters of the locally deployed Q-network based on edge inference packets to obtain an updated Q-network, wherein the observation vectors include: local real-time observation vectors, global state summaries, and historical action memories; input the observation vectors into the updated Q-network to obtain Q-value estimates corresponding to each selectable control action, and determine the local control commands for each drainage device at the current moment based on the Q-value estimates using an adaptive exploration strategy.

[0233] The device provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0234] This invention provides a server, specifically, the server includes a processor and a storage device; the storage device stores a computer program, which, when run by the processor, executes the method described in any of the above embodiments.

[0235] Figure 4 This is a schematic diagram of the structure of a server provided in an embodiment of the present invention. The server 100 includes: a processor 40, a memory 41, a bus 42 and a communication interface 43. The processor 40, the communication interface 43 and the memory 41 are connected through the bus 42. The processor 40 is used to execute executable modules, such as computer programs, stored in the memory 41.

[0236] The memory 41 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 43 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0237] Bus 42 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0238] The memory 41 is used to store programs. After receiving an execution instruction, the processor 40 executes the program. The method executed by the device for defining the flow process disclosed in any of the foregoing embodiments of the present invention can be applied to the processor 40 or implemented by the processor 40.

[0239] Processor 40 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 40 or by instructions in software form. Processor 40 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 41. The processor 40 reads the information in memory 41 and, in conjunction with its hardware, completes the steps of the above method.

[0240] The computer program product of the readable storage medium provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the foregoing method embodiments. For specific implementation, please refer to the foregoing method embodiments, which will not be repeated here.

[0241] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0242] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A drainage control method based on cloud-edge collaboration and digital twin, characterized in that, The method includes: Acquire static and real-time monitoring data of the drainage system in the target urban area, and construct a digital twin collaborative simulation platform based on the static and real-time monitoring data. The digital twin collaborative simulation platform includes: a drainage network hydraulic and water quality coupling module, a sewage treatment plant biochemical reaction module, and a receiving water body water quality response module. Based on the aforementioned digital twin collaborative simulation platform, a multi-agent centralized training process is performed on the central hybrid network deployed in the cloud to obtain a trained hybrid network model. The network parameters in the hybrid network model are packaged into an edge inference package and sent to each edge agent deployed locally on each drainage device. This enables each edge agent to independently perform distributed real-time decision processing using the locally deployed Q network based on local real-time observation data and the edge inference package, and output local control commands for each drainage device. The state data and decision data of each edge agent during actual operation are collected and uploaded to the cloud. The Q network is incrementally updated to obtain the updated target network model, which is then used to execute a new round of drainage control.

2. The drainage control method based on cloud-edge collaboration and digital twin as described in claim 1, characterized in that, The steps for constructing a digital twin collaborative simulation platform based on the static data and the real-time monitoring data include: Based on the one-dimensional Saint-Venant equations and the convection-diffusion reaction equations, a hydraulic-water quality coupling module for the drainage network is constructed to simulate the unsteady flow process and water quality migration and transformation process within the pipe network. Based on the activated sludge model, a biochemical reaction module for the wastewater treatment plant was constructed to simulate the carbon oxidation process, nitrification process, denitrification process, biological phosphorus removal process, and chemical precipitation process. Based on a one-dimensional river water quality model, a water quality response module for the receiving water body is constructed to simulate the convection diffusion and attenuation process of pollutants in the receiving water body.

3. The drainage control method based on cloud-edge collaboration and digital twin as described in claim 1, characterized in that, Following the step of constructing the digital twin co-simulation platform, the following is included: Real-time monitoring data of the physical drainage system is received through a two-way communication interface. The real-time monitoring data includes: rainfall radar data, water level monitoring data, flow monitoring data, and online water quality analysis data. By performing spatiotemporal unification processing on the real-time monitoring data, the real-time monitoring data of different frequencies and spatial locations are unified to the spatiotemporal grid nodes corresponding to various simulation models in the digital twin collaborative simulation platform, so as to use the real-time monitoring data to calibrate the digital twin collaborative simulation platform and obtain the target digital twin collaborative simulation platform.

4. The drainage control method based on cloud-edge collaboration and digital twin as described in claim 3, characterized in that, The step of calibrating the digital twin co-simulation platform using the real-time monitoring data to obtain the target digital twin co-simulation platform includes: Using an event-driven rapid calibration mechanism and a periodic deep calibration mechanism, the model parameters of the digital twin co-simulation platform are subjected to a two-level hybrid calibration process based on the real-time monitoring data to obtain the target digital twin co-simulation platform. The event-driven rapid calibration mechanism is triggered when the rainfall intensity exceeds a preset threshold, and the periodic deep calibration mechanism is executed automatically on a quarterly or annual basis.

5. The drainage control method based on cloud-edge collaboration and digital twin as described in claim 1, characterized in that, The step of performing centralized multi-agent training on a central hybrid network deployed in the cloud based on the digital twin collaborative simulation platform to obtain a trained hybrid network model includes: The local Q-values ​​of each edge agent are fused through the central hybrid network to obtain the Q-value estimate of the global joint action. Based on the difference between the Q-value estimate of the global joint action and the target Q-value, the parameters of the central hybrid network are updated until convergence, resulting in the trained hybrid network model.

6. The drainage control method based on cloud-edge collaboration and digital twin as described in claim 1, characterized in that, The step of packaging the network parameters in the hybrid network model into edge inference packages and distributing them to the edge agents deployed locally on each drainage device includes: The network parameters, observation normalization parameters, security constraint rule set, and model version metadata in the hybrid network model are packaged into the edge inference package, and the parameter changes in the edge inference package are transmitted to each edge agent through differential update technology.

7. The drainage control method based on cloud-edge collaboration and digital twin as described in claim 1, characterized in that, The step of enabling each edge agent to independently execute distributed real-time decision processing based on local real-time observation data and the edge inference package using a locally deployed Q network, and outputting local control commands for each drainage device, includes: The observation vector is obtained based on the local real-time observation data, and the parameters of the locally deployed Q network are updated based on the edge inference package to obtain the updated Q network. The observation vector includes: local real-time observation vector, global state summary and historical action memory. The observation vector is input into the updated Q network to obtain the Q value estimate corresponding to each optional control action. Based on the adaptive exploration strategy, the local control command of each drainage device at the current time is determined according to the Q value estimate.

8. A drainage control device based on cloud-edge collaboration and digital twin, characterized in that, The device includes: The platform construction module acquires static data and real-time monitoring data of the drainage system in the target urban area, and constructs a digital twin collaborative simulation platform based on the static data and the real-time monitoring data. The digital twin collaborative simulation platform includes: a drainage network hydraulic and water quality coupling module, a sewage treatment plant biochemical reaction module, and a receiving water body water quality response module. The cloud training module, based on the digital twin collaborative simulation platform, performs centralized training processing of multiple agents on the central hybrid network deployed in the cloud to obtain the trained hybrid network model. The network parameters in the hybrid network model are packaged into edge inference packages and sent to the edge agents deployed locally on each drainage device. This enables each edge agent to independently perform distributed real-time decision processing based on local real-time observation data and the edge inference packages, using the locally deployed Q network, and output local control commands for each drainage device. The model update module collects the state data and decision data of each edge agent during actual operation, uploads the state data and decision data to the cloud, performs incremental update processing on the Q network, and obtains the updated target network model, so as to use the target network model to execute a new round of drainage control.

9. A server, characterized in that, The method includes a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions that, when invoked and executed by a processor, cause the processor to perform the method according to any one of claims 1 to 7.

Images