Efficient prediction method and system for pipe network operation and maintenance data

Abstract

The invention relates to an efficient prediction method and system for pipe network operation and maintenance data, which combines protocol access and automatic format conversion technologies, realizes unified acquisition and hierarchical transmission of multi-source heterogeneous data, and solves the problems of low efficiency and inconsistent formats of traditional pipe network data acquisition. And secondly, the real-time verification and completion mechanism adopts Kalman filtering to perform anomaly detection and correction, and completes missing data through an interpolation algorithm, so that the data quality and timeliness are improved. In a data integration stage, structured cleaning and archiving are performed based on a preset data model, an efficient data basis is provided for a subsequent prediction model, finally time sequence features and spatial correlation analysis are fused, a joint feature space is constructed through event feature marking, a collaborative modeling module is utilized to train the prediction model, a potential fault point is accurately identified, and the prediction efficiency is improved. And an operation and maintenance optimization plan is intelligently generated, real-time sharing and interactive display of prediction results are realized through a decision pushing and visualization module, and closed-loop management from data acquisition to prediction decision is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of sewage treatment, in particular to a high-efficiency prediction method and system for pipe network operation and maintenance data. BACKGROUND

[0002] As an important part of urban infrastructure, the stable, safe and efficient operation of the sewage pipe network is directly related to the environmental protection, public health and the quality of life of residents. Under the current background of smart city construction, the collection, analysis and application of sewage pipe network operation and maintenance data are particularly important. However, the current sewage pipe network still has the following problems.

[0003] First, the key nodes of the existing sewage pipe network lack sufficient deployment of Internet of Things devices, which makes it impossible to comprehensively monitor the operation status of the entire system. Moreover, due to the use of different communication protocols by different devices and sensors, data integration is difficult, and it is difficult to achieve unified management and analysis of all device data.

[0004] Second, there may be noise or errors in the original data, which affects the subsequent data analysis results. In addition, due to device failure or communication interruption, etc., some data may be missing, which affects the accurate assessment of the operation status of the pipe network. There is a lack of effective real-time data verification and correction mechanism, so that incorrect data cannot be discovered and corrected in time.

[0005] Third, the analysis and prediction capabilities are limited, lacking in-depth time series feature extraction and spatial correlation analysis, making it difficult to capture potential patterns and abnormal situations in the operation of the pipe network. In addition, the existing prediction model is not accurate enough to effectively predict potential failure points and optimize operation and maintenance plans. SUMMARY

[0006] In order to solve the above problems, the present application adopts the following technical solutions.

[0007] A high-efficiency prediction method for pipe network operation and maintenance data, comprising the following steps: S1: Deploy Internet of Things devices at key nodes of the entire pipe network, and obtain pipe network operation data through a protocol access module, then automatically convert the operation data to a unified format through an embedded protocol, and transmit the operation data to a data warehouse using a hierarchical transmission module; S2: Real-time verify and correct the operation data in the data warehouse through a verification and completion module, and complete the missing data of the corrected data through an interpolation algorithm; S3: Transmit the completed operation data to an integration and cleaning module, integrate according to a preset data model, and clean and archive the integrated data; S4: Based on the archived data, time series feature extraction and spatial correlation analysis are performed, then joint features are formed through event feature labeling, and then model training and optimization are performed through a collaborative modeling module, which can predict potential failure points and optimize operation and maintenance plans; S5: The prediction results and real-time data are pushed to relevant personnel through a decision pushing module, and real-time synchronization of data is achieved through a message queue to ensure consistency of data among modules; S6: The prediction results are visualized through a visualization module, and a self-defined report function is provided to support users in generating data reports according to their needs and realizing interactive functions.

[0008] Preferably, the Internet of Things devices include but are not limited to pressure transmitters, static pressure liquid level meters, electromagnetic flowmeters, and hydrogen sulfide gas sensors, the operation data specifically include key data and regular data, the embedded protocol supports automatic conversion of Modbus / RS485 / MQTT protocols through a parser, and then hierarchical transmission is performed, and during hierarchical transmission, edge computing is further required for regular data through an edge computing module to preliminarily filter noise and abnormal values, so as to realize collection and hierarchical transmission of collaborative data of multiple source heterogeneous devices.

[0009] Preferably, the real-time verification and correction specifically include the following steps: S200: Grouping and initialization, grouping the sensors corresponding to the obtained operation data, including a high-precision sensor group and a low-precision sensor group, and individually initializing each sensor in the high-precision sensor group and the low-precision sensor group; S201: Filtering, after initialization, locally filtering and pre-filtering are respectively performed on each high-precision sensor and low-precision sensor, the local filtering adopts a Kalman filtering algorithm, the pre-filtering adopts a Bernoulli equation constraint, and then local optimal estimation values are respectively calculated; S202: Global fusion, the local optimal estimation values and estimation confidence weights are globally fused through a dynamic fusion module to calculate a global optimal estimation value, the estimation confidence weights are dynamically calculated from a historical error rate of each sensor, and then residual error analysis is performed based on the global optimal estimation value, the estimation confidence weight is automatically adjusted when the residual error is greater than 3σ, and a correction value is output when the residual error is normal.

[0010] Preferably, before residual error analysis, a new sequence is constructed in real time based on the global optimal estimation value to dynamically adjust noise, and the specific steps are as follows: first, the new sequence is calculated according to the new sequence, then it is judged whether the new sequence variance is continuously greater than a threshold for 10 times, otherwise the Q / R matrix is adaptively adjusted through an adaptive adjustment algorithm, the Q / R matrix is updated, and filtering processing is performed again, and the final estimation value data stream is output.

[0011] Preferably, the specific steps of the interpolation algorithm to complete the missing data include: S210: identification and classification, locate the missing points of the final estimated value data stream through global fusion by the missing detection module, and identify the missing type, the judgment logic of the missing type is: scan the data matrix (node x time), count the length of continuous missing according to the node, divide into single-point missing, continuous missing and regional missing according to the length of missing, and then output the missing type label; S211: constructing a spatio-temporal neighborhood, according to the missing type label, and then outputting the neighborhood data set Ω through a multi-modal missing identification engine; the multi-modal missing identification engine also uses different constraint types for conditional constraints, including spatial constraints, time constraints, hydraulic constraints and flow direction constraints; S212: algorithm selection, triggering different interpolation algorithms according to the missing type; the interpolation algorithms include spatio-temporal kriging, LSTM migration and topological constraints, which respectively use semi-variogram dynamic modeling, transfer learning fine-tuning and hydraulic equation solving; S213: uncertainty quantification, different interpolation algorithms need to match different uncertainty quantification methods, and the results generated by different interpolation methods are evaluated for reliability, and then the interpolation results with confidence labels are output; S214: interpolation calculation, executing calculation through the interpolation module according to different interpolation algorithms, and then outputting the completed operation data.

[0012] Preferably, the specific steps of integrating and cleaning and archiving according to the preset data model include: S300: spatio-temporal topology modeling, the operation data completed in step S2 is mapped into a structured graph model through a topology module, specifically including node mapping, edge relationship construction, attribute embedding, elevation compensation and topology verification, and outputting the constructed topology graph; S301: multi-modal data fusion, based on the topology graph, aligning the pressure data, flow data and gas data in space and time, and then outputting after tensor construction, time lag compensation and physical verification; S302: data cleaning, cleaning the fused data through a cleaning module in three levels, including L1 point anomaly cleaning, L2 context anomaly cleaning and L3 physical rule cleaning, wherein the L1 point anomaly cleaning uses the isolation forest algorithm for cleaning, the L2 context anomaly cleaning uses the GNN neighbor verification, and the L3 physical rule cleaning uses the Bernoulli equation constraint, and then respectively for correction and marking; S303: hierarchical archiving and storage, storing the corrected and marked data in a time series database, a graph database and a data warehouse respectively.

[0013] Preferably, based on the archived data, the specific steps of time series feature extraction, spatial correlation analysis, event feature labeling, model training and optimization include: S400: Time series feature extraction: the S3 cleaning and archived data are subjected to time series extraction through an extraction module, sliding segmentation, statistical quantity calculation, wavelet transform and feature fusion are adopted; S401: Spatial correlation analysis: a neural network (GNN) is used to construct a pipe network topology spatial relationship graph, the conduction effect weight between nodes is dynamically calculated through a hydraulic model, an abnormal conduction path is identified, and a spatial correlation feature matrix is output; S402: Event feature labeling: a pipe network event feature library is defined, a deep learning model is used to learn the spatio-temporal pattern of historical events, an event probability is output, and a spatio-temporal label is labeled for a suspected event; S403: Model training and optimization, modeling is performed based on the time series features, spatial features and event labels, a prediction result is output, and the model parameters are optimized in a reverse direction through a co-evolution mechanism, and an operation and maintenance plan is dynamically optimized.

[0014] Preferably, the present application also provides an efficient prediction system for pipe network operation and maintenance data, which comprises a collection and transmission module, a verification and completion module, an integration and cleaning module, an extraction and modeling module, a decision pushing module and a visualization module, the collection and transmission module is used to realize standardized collection and hierarchical transmission of multi-source heterogeneous data, the verification and completion module is used to ensure high precision and integrity of data, the integration and cleaning module is used to construct a physically consistent pipe network digital twin, the extraction and modeling module is used to realize accurate fault prediction and model self-evolution, the decision pushing module is used to realize multi-role collaborative decision support, and the visualization module provides an immersive decision support environment.

[0015] Preferably, the collection and transmission module comprises a protocol access module, an edge computing module and a hierarchical transmission module; the verification and completion module comprises a dynamic fusion module, a missing detection module and an interpolation module; the integration and cleaning module comprises a topology module, a multi-source fusion module and a cleaning module; the extraction and modeling module comprises an extraction module, an analysis module, a labeling module and a collaborative modeling module; the decision pushing module comprises a message routing module, a permission module and a terminal adaptation module; and the visualization module comprises a cockpit module, a report generation module and an interaction module.

[0016] The present application has the following advantages: Firstly, the method adopted by the application can automatically convert, solve the communication problem of different heterogeneous devices, and perform hierarchical transmission, secondly, the running data is constrained by Kalman filtering and Bernoulli equation, which can suppress the influence of noise, dynamic weight fusion + residual error > 3 sigma adaptive mode can correct sensor deviation, can identify missing type, and perform different interpolation calculation according to the missing type, which can ensure the accuracy of subsequent running data.

[0017] Secondly, the application can construct a real pipe network model through node mapping + elevation compensation, and through data alignment (tensor construction + time lag compensation), it can spatio-temporally fuse pressure / flow / gas data, and adopt three-level cleaning to reduce false positive rate, so that the abnormal data recognition rate is improved by 30%.

[0018] Thirdly, based on the archived data, time series feature extraction and spatial correlation analysis are performed, then joint features are formed through event feature marking, and then model training and optimization are performed through a collaborative modeling module, which can predict potential fault points and optimize operation and maintenance plans, through the above technical solutions, the prediction model can adapt to changes of different seasons and working conditions, and the fault prediction accuracy is improved by about 15% compared with the traditional method of 80%, and the operation and maintenance cost is reduced by 30%. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is the flow chart of the method of the application; Figure 2 is the detailed flow chart of S1 of the application; Figure 3 is the detailed flow chart of S2 of the application; Figure 4 is the logic flow chart of the adaptive adjustment algorithm of the application; Figure 5 is the detailed flow chart of S3 of the application; Figure 6 is the flow chart of the three-level quality control strategy of the application; Figure 7 is the local flow chart of the model integration cleaning of the application; Figure 8 is the detailed flow chart of S4 of the application; Figure 9 is the composition flow chart of the system of the application. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical scheme and advantages of the application more clear and explicit, the application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the application, and are not used to limit the application.

[0021] In the description of the present application, it should be understood that the terms "length", "width", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise explicitly specified.

[0022] Embodiment one Please refer to the drawings Figures 1-9 An efficient prediction method for pipe network operation and maintenance data, comprising the following steps: S1: Deploying Internet of Things devices on key nodes of the entire pipe network, and acquiring pipe network operation data through a protocol access module, then automatically converting the operation data to a unified format through an embedded protocol, and transmitting the operation data to a data warehouse through a hierarchical transmission module; when deploying Internet of Things devices on key nodes, the main steps include key node site selection, device installation, protocol configuration, local networking and central access, wherein the key node site selection includes main pipe intersection, old pipe section, pump station outlet, pipe network terminal and valve control point, the installed devices include pressure transmitter, electromagnetic valve flowmeter, static pressure liquid level meter and hydrogen sulfide gas sensor, etc., then local networking is performed according to the communication topology to solve the communication problem of different heterogeneous devices, then the data is output in a unified format and divided into key data and regular data, and hierarchical transmission is performed; wherein the key data is high-value data directly affecting safety warning, fault diagnosis and real-time decision-making, which requires millisecond-level response, and specifically includes pressure mutation value, H2S concentration overrun, flow abnormal oscillation and device offline signal, etc., the regular data includes stable temperature value, normal flow value, pipe wall corrosion rate and environmental temperature and humidity, etc., which generally performs edge computing, transmits after preliminary filtering of noise, through the above technical solution, the problem of non-uniform data format caused by multiple pipe network device protocols can be solved, and hierarchical processing and transmission of different hierarchical data can be performed; Example: 1. Deploy sensors at key nodes of the pipe network (such as pump station outlet, main pipe intersection): pressure transmitter (monitoring pressure mutation), static pressure liquid level meter (monitoring liquid level), electromagnetic flowmeter (monitoring flow), hydrogen sulfide gas sensor (monitoring H2S concentration).

[0023] 2. Collect data through a protocol access module: the sensor uses Modbus / RS485 protocol, the gateway supports MQTT protocol, and the embedded protocol parser automatically converts the data to JSON format.

[0024] 3. Data classification, key data: pressure mutation value (such as change rate >10kPa / s), H2S concentration over limit (>10ppm), abnormal flow oscillation (amplitude >20%), equipment offline signal. These data are directly transmitted to the central data warehouse. Regular data: stable temperature value (25±2℃), normal flow value (fluctuation <5%), pipe wall corrosion rate (<0.1mm / year). These data are first filtered by the edge computing node (deployed in the gateway), using sliding window mean filtering to remove noise, and then transmitted.

[0025] S2: The running data in the data warehouse is checked and corrected in real time by the verification completion module, mainly to solve the problems of uneven data reliability caused by sensor accuracy difference, environmental noise interference and poor adaptability of traditional filtering algorithm under dynamic working conditions. Through Kalman filtering (local filtering), the influence of noise can be suppressed, Bernoulli equation constraint (pre-filtering) can ensure the rationality of fluid mechanics, and based on the historical error rate, the confidence weight is dynamically allocated. Dynamic weight fusion and residual >3σ adaptive mode can correct sensor deviation, data correction error rate <2%, dynamic working condition adaptability is improved by 50%, and the missing data is completed by interpolation algorithm. Specifically, it can identify the missing type and execute different interpolation algorithms according to the missing type, including spatiotemporal Kriging, LSTM migration and topological constraint, so that the repair accuracy of missing data is greater than 97%, which can ensure the accuracy of subsequent running data; S3: The completed running data is transmitted to the integration and cleaning module, integrated according to the preset data model, and the integrated data is cleaned and archived. This technical solution mainly solves the problems of multi-source data time and space asynchronization, abnormal data pollution model training and discrete data difficult to express pipe network operation and maintenance topology. Since the running data is composed of multi-source data (pressure / flow / gas), due to the time and space asynchronization, this scheme can construct a real pipe network model through node mapping and elevation compensation, and can fuse pressure / flow / gas data in time and space through data alignment (tensor construction and time lag compensation), and can reduce the false positive rate by adopting three-level cleaning, so that the abnormal data recognition rate is improved by 30%; S4: Based on the archived data, time series feature extraction and spatial correlation analysis are performed, then joint features are formed through event feature labeling, and then model training and optimization are performed through the collaborative modeling module, which can predict potential failure points and optimize operation and maintenance plans. Through the above technical solution, when extracting time series features, time series decomposition (such as STL) is used to extract trend, seasonality and residual terms, combined with statistical features (mean, variance, autocorrelation, etc.) and time domain features (such as zero-crossing points, peak values, etc.), so as to extract effective time series and spatial features from massive data; Spatial correlation analysis uses neural networks (GNN) to model the topology of the pipe network, capturing the mutual influence between nodes, event feature labeling, based on rules (such as thresholds) or models (such as clustering) to label abnormal events and form feature vectors; Collaborative modeling: Multi-task learning or ensemble learning is used to simultaneously predict multiple fault types (such as pipe bursts, blockages, corrosion, etc.), and physical constraints (such as Bernoulli's equation) are introduced to enhance model interpretability and accuracy. Operation and maintenance optimization: Combined with the reinforcement learning PPO algorithm, dynamically adjust the operation and maintenance strategy, consider cost, risk and benefit, through the above technical solution, the prediction model can adapt to changes in different seasons and working conditions, and the fault prediction accuracy is improved by about 15% compared with the traditional method of 80%, and the operation and maintenance cost is reduced by 30%; S5: The prediction results and real-time data are pushed to relevant personnel through the decision pushing module, real-time synchronization of data is realized through the message queue, ensuring that the data of each module is consistent, the message queue (such as Kafka) is used to realize the publish / subscribe mode, ensuring real-time data distribution, which can make the data synchronization delay less than 1 second, and secondly, the combination of access control RBAC and attribute-based access control ABAC realizes fine-grained permission management; S6: The prediction results are visualized through the visualization module, and a self-defined report function is provided to support users to generate data reports according to their needs, realize interactive functions, specifically use three-dimensional visualization, use WebGL technology to realize three-dimensional display of the pipe network, use key indicators (KPI) for real-time monitoring in the data cockpit, automatically locate abnormal points, and provide a drag-and-drop report designer for custom reports, supporting SQL-free queries.

[0026] Referring to the accompanying Figures 1-2 , the Internet of Things device includes but is not limited to a pressure transmitter, a static pressure liquid level gauge, an electromagnetic flowmeter and a hydrogen sulfide gas sensor, the operation data specifically includes key data and regular data, the key data is directly transmitted in real time, and the regular data needs to be edge calculated, preliminarily filtered noise and abnormal values, the embedded protocol automatically converts through the parser to support Modbus / RS485 / MQTT protocol, the embedded protocol parser automatically converts multi-protocol data, which can eliminate compatibility problems, and then performs hierarchical transmission, so as to realize collaborative data acquisition and hierarchical transmission of multi-source heterogeneous devices.

[0027] Reference to the accompanying drawings Figure 3 The real-time verification and correction specific steps include: S200: Grouping and initialization, grouping the sensors corresponding to the obtained operation data, including a high-precision sensor group and a low-precision sensor group, due to the working condition interference of the sewage pipe network, including sludge deposition and gas corrosion leading to sensor baseline drift, so grouping is needed, the high-precision sensor group is generally arranged at a key risk point (pump station / easy blocking point), and the low-precision group is used for a conventional pipe section to reduce the cost, and each sensor in the high-precision sensor group and the low-precision sensor group is individually initialized, the zero point of the calibrated sensor is initialized, and drift is suppressed. For example, a high-precision pressure sensor (precision ±0.5%) at the outlet of a pump station is divided into a high-precision group, and a low-precision liquid level meter (precision ±5%) in a conventional pipe section is divided into a low-precision group. The zero point is calibrated during initialization (for example, the liquid level meter is zeroed when the pipe is empty); S201: Filtering, after initialization, local filtering and pre-filtering are respectively performed on each high-precision sensor and low-precision sensor, the local filtering adopts a Kalman filtering algorithm, the pre-filtering adopts Bernoulli equation constraint, the sewage fluid mechanics rationality in the pipe network can be ensured, and then local optimal estimation values are respectively calculated; S202: Global fusion, the local optimal estimation values and estimation confidence weights are globally fused through a dynamic fusion module to calculate a global optimal estimation value, the estimation confidence weights are dynamically calculated from the historical error rate of each sensor, and then residual error analysis is performed based on the global optimal estimation value, when the residual error is greater than 3σ, the estimation confidence weight is automatically adjusted, and when the residual error is normal, a correction value is output.

[0028] An example of data after filtering of three pressure sensors in a certain pipe section is taken as an example: High-precision pressure sensor A: 302.5 kPa (weight 0.7, historical error 2%); Low-precision pressure sensor B: 315.0 kPa (weight 0.2, historical error 8%); Low-precision pressure sensor C: 290.0 kPa (weight 0.1, historical error 15%); The local optimal estimation value of the high-precision pressure sensor A is 302.5*0.7; The local optimal estimation value of the low-precision pressure sensor B is 315*0.2; The local optimal estimation value of the low-precision pressure sensor C is 290*0.1; The global optimal estimation value = 302.5*0.7+315*0.2+290*0.1=304.75 kPa.

[0029] Residual error analysis: the residual error of the low-precision pressure meter C is >3σ, wherein σ≈3 kPa.

[0030] At this time, it is explained that the detection data of the low-precision pressure sensor C has a detection problem, in order to avoid affecting fusion, the weight thereof is automatically reduced from 0.1 to 0.08, so as to avoid polluting the global data.

[0031] Referring to the accompanying Figure 4 Before residual analysis, a new sequence is constructed in real time based on the global optimal estimation value, the new sequence is the difference between the observation value and the predicted observation value in the filter cake process, and the noise is dynamically adjusted, and the specific steps are as follows: first, the new sequence is calculated according to the new sequence, and the new sequence construction formula is:

[0032] Among them: is the new sequence at time step k; is the actual observation value at time step k; H is an observation matrix; is the prediction of the state vector based on the information up to time k-1 at time step k, then it is judged whether the new sequence variance is continuously 10 times beyond the threshold value, otherwise the Q / R matrix is adaptively adjusted through an adaptive adjustment algorithm, and the Q / R matrix is updated, and the filtering process is re-performed; yes, the final estimation value data stream is output.

[0033] An exemplary pump station rain season working condition is taken as an example: when the new sequence variance is continuously 10 times beyond the threshold value, it is explained that the noise characteristics of the system have changed in nature (for example, the flow state of the pipeline changes from laminar flow to turbulent flow after rainwater flows into). At this time, the adaptive algorithm will increase the process noise Q (admitting that the prediction error of the model is increased), and at the same time, the observation noise R is reduced (more trust in the original data of the sensor). This dynamic adjustment is faster than the response of manual parameter reset, and can avoid the problem of rising false alarm rate in the rain season. The calculation formula of the new sequence variance is:

[0034] Among them: is the new sequence variance, which represents the dispersion degree of the new sequence, and the greater the value, the greater the observation noise or model error; N is the size of the sliding window, which is used to calculate the sample number of the new sequence variance (for example, taking the last 50 data points); is the new sequence vector at time step i; k is the current time index; is the transpose of the new sequence vector at time step i, which is used to calculate the inner product of the vector (i.e. represents the square norm of the vector ). is the new sequence vector at time step i; k is the current time index;

[0035] For example, the pressure and liquid level (observation value dimension m=2) of a certain key node section, the sliding window N=5 (taking 5 time points of data), the new sequence (unit: pressure kPa, liquid level m). of The square norm is: = ; = ; = ; = = ; = = ; .

[0036] If 10 times in a row >λ×benchmark (λ is a coefficient of 2, and the benchmark value is 0.03, such as 2) If the noise level is 0.0.3, then adaptive noise adjustment will be triggered. The main purpose of this technical solution is to address sudden strong interference in the sewage pipe network, such as sudden changes in flow caused by heavy rain; and gradual equipment degradation, such as sensor probes being slowly covered by sludge. Traditional Kalman filters with fixed noise parameters cannot handle this type of time-varying noise, which is the core pain point that this solution aims to solve.

[0037] See appendix Figures 5-6 The specific steps of the interpolation algorithm to complete missing data include: S210: Identification and classification. The final estimated value data stream after global fusion is used to locate missing points through the missing detection module and identify the missing type. The judgment logic of the missing type is as follows: scan the data matrix (node ​​× time), count the length of continuous missing data by node, classify the missing data into single missing data, continuous missing data, and regional missing data according to the missing length, and then output the missing type label. S211: Construct a spatiotemporal neighborhood, and based on the missing type label, output the neighborhood dataset Ω through a multimodal missing data recognition engine; the multimodal missing data recognition engine also employs different constraint types for condition constraints, including spatial constraints, temporal constraints, hydraulic constraints, and flow direction constraints; its multimodal missing data recognition engine execution algorithm is as follows: #def build_neighborhood(missing_type, node, time_range): # Input: Missing type, node ID, missing time range #Spatial Constraints: Physically adjacent nodes (pipe spacing < 300 meters) spatial_nodes = find_spatial_neighbors(node, max_distance=300) # Time constraint: 30 minutes of data before and after missing period time_window = [time_range[0]-30, time_range[1]+30] units: minutes # Hydraulic constraint: same pipe diameter (±10%) and material nodes hydraulic_nodes = find_similar_pipes(node, diameter_tol=0.1, material="HDPE") # Flow direction constraint: prefer upstream nodes (based on network topology) upstream_nodes = get_upstream_nodes(node) # Combine constraints to generate Ω for candidate in set(spatial_nodes + hydraulic_nodes): # Flow direction weight: upstream nodes weight=1.0, downstream=0.7 weight = 1.0 if candidate in upstream_nodes else 0.7 # Extract valid data for candidate node within time window data = load_data(candidate, time_window) if data.is_valid(): Ω.append((candidate, data, weight)) return Ω# Output: neighborhood dataset Ω = [ (candidate node, data, weight) ].

[0038] Example: Node B has consecutive missing data Spatial neighbors: Nodes A1, A2 (within 200 meters of B) Hydraulically similar: Node C (same pipe diameter 300mm HDPE pipe) Flow direction preference: Node U (direct upstream of B) Time window: [t-30min, t+30min] Then the output neighborhood dataset Ω = { (A1, data_A1, 0.7), (A2, data_A2, 0.7), (C, data_C, 1.0), (U, data_U, 1.0)}.

[0039] Through the technical solution of space+time+hydraulic+flow direction four-dimensional correlation, compared with the traditional neighborhood only considering the spatial distance and ignoring the pipe network hydraulic characteristics, the neighborhood data can conform to the fluid motion law.

[0040] S212: Algorithm selection, different interpolation algorithms are triggered according to the missing type; the interpolation algorithms include spatiotemporal kriging, LSTM migration and topological constraint, and the spatiotemporal kriging, LSTM migration and topological constraint respectively adopt semi-variogram dynamic modeling, migration learning fine-tuning and hydraulic equation solving; S213: Uncertainty quantification, different interpolation algorithms need to match different uncertainty quantification methods, the results of different interpolation methods, namely kriging variance, LSTM confidence interval and hydraulic variance, are subjected to reliability evaluation, the evaluation structure adopts a three-level quality control strategy including point-level quality control (dynamic threshold), regional quality control (through an abnormal area detection algorithm, different strategies are adopted, such as: 5-10 nodes, increasing the LSTM historical window by 50%; 11-50 nodes, starting hydraulic model simulation; >50 nodes, manual inspection) and system quality control (combining point-level and regional quality control, the system quality control strategy is used for control), ensuring data quality, for high confidence data, interpolation calculation is directly performed, and for low confidence data, the algorithm needs to be reselected, and then the interpolation result with confidence mark is outputted; S214: Interpolation calculation, through the interpolation module, calculation is performed according to different interpolation algorithms, and then the completed operation data is outputted.

[0041] Through the above technical solution, through the three-in-one of multi-modal neighborhood construction+physical constraint algorithm selection+uncertainty quantification, the difficult problem of sewage pipe network complex missing scene repair is solved, and a high-integrity and high-credibility data basis is provided for the prediction model.

[0042] Referring to the accompanying Figure 7 , the specific steps of integration, cleaning and archiving according to the preset data model include: S300: Spatiotemporal topology modeling. The operational data completed in step S214 is used to map discrete sensor data into a structured graph model via a topology module. This includes node mapping, edge relationship construction, attribute embedding, elevation compensation, and topology verification, outputting the constructed topology graph. The core of this solution is to address the problem that discrete point data cannot represent the hydraulic connections within a pipe network. For example, in a sewage pipe network, the topological relationships between pumping stations, manholes, and treatment plants require first establishing a node-edge model, supplementing the Z-axis coordinates (such as manhole cover elevation), and then using Dijkstra's algorithm to verify connectivity. The algorithm for establishing the node-edge model is as follows: #def build_topology(sensors): graph = nx.Graph() # Node Mapping (Sensor → Pipeline Node) For sensor in sensors: graph.add_node(sensor.id, type=sensor.type, pos=(sensor.lon, sensor.lat)) #Edge Relationship Construction (Pipeline Connection) for pipe in pipeline_db: graph.add_edge(pipe.start_node, pipe.end_node, length = pipe.length, diameter = pipe.diameter) #Elevation compensation (DEM data correction) dem = load_digital_elevation_map() for node in graph.nodes: z = dem.query(graph.nodes[node]['pos']) graph.nodes[node]['elevation']= z #Add elevation attribute # Topology verification (Dijkstra's algorithm for verifying connectivity) if not nx.is_connected(graph): isolate_nodes = list(nx.isolates(graph)) auto_repair_connections(isolate_nodes) # Automatically repair isolated nodes Return graph. S301: Multimodal data fusion, based on a topology graph, aligns pressure, flow, and gas data spatiotemporally, then performs tensor construction, time delay compensation, and physical verification before fusion output; its fusion algorithm is as follows: #def multimodal_fusion(graph, pressure_data, flow_data, gas_data): # Spatiotemporal alignment (linear interpolation + time delay compensation) aligned_flow = align_data(flow_data, target_freq="1Hz" delay=calculate_hydraulic_delay(graph)) # Hydraulic delay calculation # Tensor construction [node × time × feature] tensor = np.zeros((len(graph.nodes), len(pressure_data), 3)) for i, node in enumerate(graph.nodes): tensor[i, :, 0] = pressure_data[node] # Pressure dimension tensor[i, :, 1] = aligned_flow[node] #flow dimension tensor[i, :, 2] = gas_data[node] # Hydrogen sulfide dimension #Physical verification (Bernoulli equation constraints) for t in range(tensor.shape[1]): For edge in graph.edges: node1, node2 = edge P1, Q1 = tensor[node1, t, 0], tensor[node1, t, 1] P2, Q2 = tensor[node2, t, 0], tensor[node2, t, 1] # Mass conservation check Q1 should ≈ Q2 (no leakage in pipe segment) if abs(Q1 - Q2)>0.1 * max(Q1, Q2): tensor[node1, t, 1] = (Q1 + Q2) / 2 # Correct flow value Bernoulli equation check ΔP = f(Δv, Δh) delta_P = calculate_pressure_drop(graph, node1, node2, Q1) if abs(P1 - P2 - delta_P)>5:#5kPa tolerance mark_physical_anomaly(node1, node2, t)#Mark physical anomaly return tensor Through this scheme, the pressure / flow / gas data time is kept synchronized, such as the change of hydrogen sulfide concentration lags behind the change of flow. When implementing, the transmission time lag between sensors needs to be calculated (such as 0.3 meters per second flow speed corresponds to 333 milliseconds per hundred meters), and the tensor is aligned with the time stamp. Physical check depends on Bernoulli equation.

[0043] S302: Data cleaning, the fused data is cleaned by a cleaning module through three levels of cleaning, including L1 point anomaly cleaning, L2 context anomaly cleaning and L3 physical rule cleaning, wherein the L1 point anomaly cleaning is cleaned by the isolation forest algorithm, the L2 context anomaly cleaning is cleaned by the GNN neighbor check, and the L3 physical rule cleaning is cleaned by the Bernoulli equation constraint. The three-level cleaning mechanism is aimed at different anomalies: L1 isolation forest handles transient pulses caused by pump start-stop (such as pressure gauge interference by bubbles); L2 GNN check can find "pressure anomaly but adjacent node normal" false failure (60% reduction in false alarm); L3 Bernoulli equation can capture the physical contradiction of "liquid level rising pressure falling" (timely find pipeline collapse), and then respectively perform correction and marking; S303: Hierarchical archival storage, the corrected and marked data is stored in time series database, graph database and data warehouse respectively according to time sequence, the time series database stores pressure fluctuation per second for real-time early warning, the graph database stores topological relationship for traceability analysis, and the data warehouse stores cleaned data to support report generation.

[0044] Through the above steps, the sewage pipe network data is transformed from "fragmentation" to "knowledge", providing a high-quality data basis for the prediction model.

[0045] Referring to the drawings Figure 8 Based on the archived data, the specific steps of time sequence feature extraction, spatial correlation analysis, event feature labeling, model training and optimization include: S400: Time sequence feature extraction: The S303 cleaning and archived data are subjected to time sequence extraction by an extraction module, using methods such as sliding segmentation, statistical quantity calculation, wavelet transformation and feature fusion; S401: Spatial correlation analysis: Based on a neural network (GNN), a pipe network topology spatial relationship graph is constructed, the conduction effect weight between nodes is dynamically calculated through a hydraulic model, abnormal conduction paths are identified, and a spatial correlation feature matrix is output; S402: Event feature labeling: A pipe network event feature library is defined, a deep learning model is used to learn the spatiotemporal pattern of historical events, an event probability is output, and a suspected event is labeled with a spatiotemporal tag; S403: Model training and optimization, based on time sequence features, spatial features and event labels, modeling is performed, a prediction result is output, and model parameters are optimized in a reverse direction through a co-evolution mechanism, while an operation and maintenance plan is dynamically optimized.

[0046] Referring to the drawings Figure 9 The present application also provides an efficient prediction system for pipe network operation and maintenance data, which comprises a collection and transmission module, a verification and completion module, an integration and cleaning module, an extraction and modeling module, a decision pushing module and a visualization module. The collection and transmission module is used to realize standardized collection and hierarchical transmission of multi-source heterogeneous data. The verification and completion module is used to ensure high precision and integrity of data. The integration and cleaning module is used to construct a physically consistent pipe network digital twin. The extraction and modeling module is used to realize accurate fault prediction and model self-evolution. The decision pushing module is used to realize multi-role collaborative decision support. The visualization module provides an immersive decision support environment.

[0047] The collection transmission module includes a protocol access module, an edge computing module and a hierarchical transmission module, the protocol access module is used for unified access and protocol conversion of multi-source heterogeneous devices, the edge computing module performs real-time preprocessing on conventional data, filters noise and abnormal values, and the hierarchical transmission module differentiates transmission according to data priority and optimizes bandwidth resources; the verification and completion module includes a dynamic fusion module, a missing detection module and an interpolation module, the dynamic fusion module is used for multi-sensor data weighted fusion and adaptive adjustment of filtering parameters, the missing detection module accurately identifies missing types (single point / continuous / region), and the interpolation module selects the optimal interpolation algorithm according to the missing type and quantifies the reliability; the integration and cleaning module includes a topology module, a multi-source fusion module and a cleaning module, the topology module is used for constructing a digital twin of the pipe network, the multi-source fusion module realizes multi-source data space-time alignment and physical fusion, and the cleaning module is used for performing three-level anomaly detection and correction; the extraction and modeling module includes an extraction module, an analysis module, a labeling module and a collaborative modeling module, the extraction module extracts time sequence and spatial features from cleaned data, the analysis module is used for quantifying spatial correlation between nodes, identifying fault conduction paths, the labeling module defines and identifies pipe network event features, and the collaborative modeling module is used for training a prediction model and realizing parameter self-evolution; the decision pushing module includes a message routing, a permission module and a terminal adaptation module, the message routing distributes early warning information according to event types, the permission module controls data access granularity, and the terminal adaptation module is used for multi-terminal data format conversion and offline support; the visualization module includes a cockpit module, a report generation module and an interaction module, the cockpit module is used for three-dimensional dynamic display, the report generation module can quickly generate customized analysis reports, and the interaction module is used for supporting SQL-free queries.

[0048] The above is only a preferred embodiment of the present application, and does not limit the present application in other forms. Any skilled person in the art can modify or change the above disclosed technical content to equivalent embodiments applied to other fields, but any simple modification, equivalent change and modification made according to the technical essence of the present application to the above embodiments still belongs to the protection scope of the present application technical solution.

Claims

1. An efficient prediction method for pipe network operation and maintenance data, characterized in that, Comprise the following steps: S1: Deploying Internet of Things devices on key nodes of the entire pipe network, obtaining pipe network operation data through a protocol access module, then automatically converting the operation data into a unified format through an embedded protocol, and transmitting the operation data to a data warehouse using a hierarchical transmission module; S2: Real-time checking and correcting the operation data in the data warehouse through a checking and completing module, and completing missing data through an interpolation algorithm based on the corrected data; S3: Transmitting the completed operation data to an integration and cleaning module, integrating the data according to a preset data model, and cleaning and archiving the integrated data; S4: Based on the archived data, extracting time sequence features and spatial correlation analysis, then forming joint features through event feature marking, and then training and optimizing the model through a collaborative modeling module, which can predict potential fault points and optimize operation and maintenance plans; S5: Pushing the prediction results and real-time data to relevant personnel through a decision pushing module, realizing real-time synchronization of data through a message queue to ensure consistency of data in each module; S6: Visualizing the prediction results through a visualization module, and providing a self-defined report function to support users in generating data reports according to their needs, and realizing interactive functions.

2. The method of claim 1, wherein, The Internet of Things devices include but are not limited to pressure transmitters, static pressure liquid level gauges, electromagnetic flowmeters, and hydrogen sulfide gas sensors, the operation data specifically include key data and regular data, the embedded protocol supports Modbus / RS485 / MQTT protocol automatic conversion through a parser, and then hierarchical transmission, and edge computing is further needed for regular data through an edge computing module during hierarchical transmission to preliminarily filter noise and abnormal values, so as to realize multi-source heterogeneous device collaborative data collection and hierarchical transmission.

3. The method of claim 1, wherein, The specific steps of real-time checking and correcting include: S200: Grouping and initialization, grouping the sensors corresponding to the obtained operation data, including a high-precision sensor group and a low-precision sensor group, and individually initializing each sensor in the high-precision sensor group and the low-precision sensor group; S201: Filtering, after initialization, locally filtering and pre-filtering each high-precision sensor and low-precision sensor, the local filtering adopts Kalman filtering algorithm, and the pre-filtering adopts Bernoulli equation constraint, and then calculating local optimal estimation value; S202: Global fusion, globally fusing the local optimal estimation value and estimation confidence weight through a dynamic fusion module to calculate a global optimal estimation value, the estimation confidence weight is dynamically calculated from the historical error rate of each sensor, and then residual error analysis is performed based on the global optimal estimation value, when the residual error is greater than 3σ, the estimation confidence weight is automatically adjusted, and when the residual error is normal, the correction value is output.

4. The method of claim 3, wherein, Before residual error analysis, a new information sequence is constructed in real time based on the global optimal estimation value, and noise is dynamically adjusted, and the specific steps are as follows: first, calculate the new information variance according to the constructed new information sequence, then judge whether the new information variance exceeds the threshold for 10 consecutive times, if not, adjust the Q / R matrix through an adaptive adjustment algorithm, update the Q / R matrix, and re-filter, and if yes, output the final estimation value data stream.

5. The method of claim 4, wherein, The new sequence construction formula is: where: is the innovation at time step k; is the actual observation at time step k; H is the observation matrix; is the prediction of the state vector at time step k based on information up to time k - 1; the innovation variance is calculated as: wherein: is the innovation variance; N is the size of the sliding window; is the innovation vector at the i-th time instant; k is the current time index; is the innovation vector at the i-th time instant; k is the current time index; is the transpose of the innovation vector at the i-th time instant.

6. The method of claim 1, wherein, The specific steps of the interpolation algorithm for completing the missing data include: S210: identification and classification, the final estimated value data stream after global fusion is located through a missing detection module, and the missing type is identified, the judgment logic of the missing type is: scanning the data matrix, counting the length of continuous missing points according to nodes, and dividing into single-point missing, continuous missing and regional missing according to the length of missing points, and then outputting the missing type label; S211: constructing a spatio-temporal neighborhood, according to the missing type label, and then outputting a neighborhood data set Omega through a multi-modal missing identification engine; the multi-modal missing identification engine also uses different constraint types for conditional constraints, including spatial constraints, time constraints, hydraulic constraints and flow direction constraints; S212: algorithm selection, different interpolation algorithms are triggered according to the missing type; the interpolation algorithms include spatio-temporal Kriging, LSTM migration and topological constraints, and the spatio-temporal Kriging, LSTM migration and topological constraints respectively use semi-variogram dynamic modeling, migration learning fine-tuning and hydraulic equation solving; S213: uncertainty quantification, different interpolation algorithms need to match different uncertainty quantification methods, the results generated by different interpolation methods are evaluated for reliability, and then the interpolation results with confidence labels are output; S214: interpolation calculation, the interpolation module performs calculation according to different interpolation algorithms, and then outputs the completed operation data.

7. The method of claim 1, wherein, According to the preset data model, the specific steps of integration, cleaning and archiving include: S300: spatio-temporal topology modeling, the operation data completed in step S2 is mapped into a structured graph model through a topology module, specifically including node mapping, edge relationship construction, attribute embedding, elevation compensation and topology verification, and the constructed topological graph is output; S301: multi-modal data fusion, based on the topological graph, the pressure data, flow data and gas data are spatio-temporally aligned, and then fused after tensor construction, time lag compensation and physical verification; S302: data cleaning, the fused data is cleaned through a cleaning module in three levels, including L1 point anomaly cleaning, L2 context anomaly cleaning and L3 physical rule cleaning, wherein the L1 point anomaly cleaning is cleaned by an isolation forest algorithm, the L2 context anomaly cleaning is cleaned by a GNN neighbor check, and the L3 physical rule cleaning is cleaned by a Bernoulli equation constraint, and then modified and labeled respectively; S303: hierarchical archiving and storage, the modified and labeled data is stored safely in a time series database, a graph database and a data warehouse respectively.

8. The method of claim 1, wherein, Based on the archived data, the specific steps of time series feature extraction, spatial correlation analysis, event feature labeling, model training and optimization include: S400: time series feature extraction: the cleaned and archived data in S3 is extracted by an extraction module in time series, using sliding segmentation, statistical quantity calculation, wavelet transform and feature fusion; S401: spatial correlation analysis: a neural network (GNN) is used to construct a pipe network topology spatial relationship graph, the conduction effect weight between nodes is dynamically calculated through a hydraulic model, an abnormal conduction path is identified, and a spatial correlation feature matrix is output; S402: Event feature labeling: define the pipe network event feature library, learn the historical event spatio-temporal pattern using a deep learning model, output the event probability, and label the suspected event with spatio-temporal tags; S403: Model training and optimization, modeling based on time sequence features, spatial features and event labels, outputting prediction results, and optimizing model parameters through a co-evolution mechanism, while dynamically optimizing operation and maintenance plans.

9. An efficient pipe network operation data prediction system for implementing the efficient pipe network operation data prediction method according to any one of claims 1-8, characterized in that, The system comprises a collection and transmission module, a verification and completion module, an integration and cleaning module, an extraction and modeling module, a decision pushing module and a visualization module, the collection and transmission module is used for realizing standardized collection and hierarchical transmission of multi-source heterogeneous data, the verification and completion module is used for guaranteeing high precision and integrity of data, the integration and cleaning module is used for constructing a physically consistent pipe network digital twin, the extraction and modeling module is used for realizing accurate fault prediction and model self-evolution, the decision pushing module is used for realizing multi-role collaborative decision support, and the visualization module provides an immersive decision support environment.

10. The system for efficient prediction of pipe network operation and maintenance data according to claim 9, characterized in that, The collection and transmission module further comprises a protocol access module, an edge computing module and a hierarchical transmission module; the verification and completion module comprises a dynamic fusion module, a missing detection module and an interpolation module; the integration and cleaning module comprises a topology module, a multi-source fusion module and a cleaning module; the extraction and modeling module comprises an extraction module, an analysis module, a labeling module and a collaborative modeling module; the decision pushing module comprises a message routing module, a permission module and a terminal adaptation module; and the visualization module comprises a cockpit module, a report generation module and an interaction module.

Images