Efficient prediction method and system for pipe network operation and maintenance data

Abstract

A kind of efficient prediction method and system of pipe network operation and maintenance data, combine protocol access and automatic format conversion technology, realize the unified collection and hierarchical transmission of multi-source heterogeneous data, solve the problem of low efficiency of traditional pipe network data collection, inconsistent format;Second, real-time verification and completion mechanism uses Kalman filter for anomaly detection and correction, and through interpolation algorithm to complete missing data, improve data quality and timeliness.In the data integration stage, based on the preset data model for structured cleaning and archiving, to provide efficient data basis for subsequent prediction model, finally fusion time sequence characteristics and spatial correlation analysis, through event feature marking to construct joint feature space, and use collaborative modeling module to train prediction model, accurately identify potential fault point, intelligent generation operation and maintenance optimization plan, through decision push and visualization module, realize the real-time sharing and interactive display of prediction result, realize the closed-loop management from data collection to prediction decision.

Description

Technical Field

[0001] This invention relates to the field of wastewater treatment, and in particular to an efficient method and system for predicting pipeline network operation and maintenance data. Background Technology

[0002] As a crucial component of urban infrastructure, the stable, safe, and efficient operation of sewage pipe networks directly impacts urban environmental protection, public health, and residents' quality of life. In the context of current smart city development, the collection, analysis, and application of sewage pipe network operation and maintenance data have become particularly important. However, current sewage pipe networks still face the following challenges.

[0003] First, the existing sewage pipe network lacks sufficient IoT devices at key nodes, making it impossible to fully monitor the operation status of the entire system. Furthermore, due to the different communication protocols used by different devices and sensors, data integration is difficult, making it hard to achieve unified management and analysis of data from all devices.

[0004] Secondly, the raw data may contain noise or errors, affecting the results of subsequent data analysis. Furthermore, due to equipment failure or communication interruption, some data may be missing, affecting the accurate assessment of the pipeline network's operating status. The lack of an effective real-time data verification and correction mechanism makes it impossible to detect and correct erroneous data in a timely manner.

[0005] Third, the analysis and prediction capabilities are limited, lacking in-depth time series feature extraction and spatial correlation analysis, making it difficult to capture potential patterns and anomalies in pipeline operation. Furthermore, existing prediction models are not accurate enough to effectively predict potential fault points and optimize operation and maintenance plans. Summary of the Invention

[0006] To address the above problems, the present invention adopts the following technical solution.

[0007] An efficient method for predicting pipeline network operation and maintenance data includes the following steps:

[0008] S1: Deploy IoT devices at key nodes of the entire pipeline network, acquire pipeline operation data through the protocol access module, and then automatically convert it into a unified format through the embedded protocol. Use the hierarchical transmission module to transmit the operation data to the data warehouse. S2: The verification and completion module performs real-time verification and correction on the operational data in the data warehouse, and uses an interpolation algorithm to complete the missing data on the corrected data; S3: The completed running data is transmitted to the integration and cleaning module, which integrates the data according to the preset data model, and cleans and archives the integrated data. S4: Based on archived data, it performs time-series feature extraction and spatial correlation analysis, then forms joint features through event feature labeling, and then trains and optimizes the model through the collaborative modeling module, which can predict potential fault points and optimize operation and maintenance plans; S5: Push the prediction results and real-time data to relevant personnel through the decision push module, and realize real-time data synchronization through the message queue to ensure data consistency among modules; S6: The prediction results are visualized through the visualization module, and a custom report function is provided to support users in generating data reports according to their needs and realizing interactive functions.

[0009] Preferably, the IoT devices include, but are not limited to, pressure transmitters, hydrostatic level gauges, electromagnetic flow meters, and hydrogen sulfide gas sensors. The operating data specifically includes key data and routine data. The embedded protocol supports automatic conversion between Modbus / RS485 / MQTT protocols through a parser, and then performs hierarchical transmission. During hierarchical transmission, routine data also needs to be processed by an edge computing module to initially filter noise and outliers, thereby realizing the collection and hierarchical transmission of collaborative data from multi-source heterogeneous devices.

[0010] Preferably, the specific steps of the real-time verification and correction include: S200: Grouping and initialization. The sensors corresponding to the acquired operation data are grouped into high-precision sensor groups and low-precision sensor groups, and each sensor in the high-precision sensor group and low-precision sensor group is initialized individually. S201: Filtering. After initialization, local filtering and pre-filtering are performed on each high-precision sensor and low-precision sensor respectively. The local filtering adopts the Kalman filtering algorithm, and the pre-filtering adopts Bernoulli equation constraints. Then, the local optimal estimate is calculated respectively. S202: Global fusion. The local optimal estimate and the estimated confidence weight are globally fused through the dynamic fusion module to calculate the global optimal estimate. The estimated confidence weight is dynamically calculated from the historical error rate of each sensor. Then, residual analysis is performed based on the global optimal estimate. When the residual is >3σ, the estimated confidence weight is automatically adjusted. When the residual is normal, the correction value is output.

[0011] Preferably, before residual analysis, a new information sequence should be constructed in real time based on the global optimal estimate, and noise should be dynamically adjusted. The specific steps are as follows: First, calculate the new information variance based on the constructed new information sequence, and then determine whether the new information variance exceeds the threshold for 10 consecutive times. If yes, the Q / R matrix is ​​adaptively adjusted by the adaptive adjustment algorithm, and the Q / R matrix is ​​updated and filtered again; otherwise, the final estimate data stream is output.

[0012] Preferably, the specific steps of the interpolation algorithm to complete missing data include: S210: Identification and classification. The final estimated value data stream after global fusion is used to locate missing points through the missing detection module and identify the missing type. The judgment logic of the missing type is as follows: scan the data matrix (node ​​× time), count the length of continuous missing data by node, classify the missing data into single missing data, continuous missing data, and regional missing data according to the missing length, and then output the missing type label. S211: Construct a spatiotemporal neighborhood, and output the neighborhood dataset Ω based on the missing type label through a multimodal missing data recognition engine; the multimodal missing data recognition engine also uses different constraint types for condition constraints, including spatial constraints, temporal constraints, hydraulic constraints, and flow direction constraints. S212: Algorithm selection, triggering different interpolation algorithms based on the missing type; the interpolation algorithms include spatiotemporal kriging, LSTM transfer learning, and topological constraints, wherein the spatiotemporal kriging, LSTM transfer learning, and topological constraints are respectively modeled by semi-variogram dynamic modeling, fine-tuned by transfer learning, and solved by hydraulic equations; S213: Uncertainty quantification. Different interpolation algorithms require different uncertainty quantification methods. The reliability of the results generated by different interpolation methods is evaluated, and then the interpolation results with confidence labels are output. S214: Interpolation calculation. The interpolation module performs calculations according to different interpolation algorithms and then outputs the completed running data.

[0013] Preferably, the specific steps for integrating, cleaning, and archiving data according to a preset data model include: S300: Spatiotemporal topology modeling, which maps the discrete sensor data into a structured graph model through the topology module, using the completed running data from step S2. Specifically, it includes node mapping, edge relationship construction, attribute embedding, elevation compensation, and topology verification, and outputs the constructed topology graph. S301: Multimodal data fusion, based on topology graph, aligns pressure data, flow data and gas data in time and space, and then fuses and outputs the data after tensor construction, time delay compensation and physical verification; S302: Data cleaning. The fused data is cleaned in three levels through the cleaning module, including L1 point anomaly cleaning, L2 context anomaly cleaning and L3 physical rule cleaning. L1 point anomaly cleaning uses the isolated forest algorithm, L2 context anomaly cleaning uses GNN neighbor verification, and L3 physical rule cleaning uses Bernoulli equation constraints. Then, they are corrected and labeled respectively. S303: Tiered archive storage securely stores corrected and tagged data according to time-series databases, graph databases, and data warehouses.

[0014] Preferably, the specific steps for performing time-series feature extraction, spatial correlation analysis, event feature labeling, model training and optimization based on archived data include: S400: Temporal Feature Extraction: The S3 cleaned and archived data is extracted in time series using the extraction module, employing sliding segmentation, statistical calculation, wavelet transform, and feature fusion. S401: Spatial correlation analysis: Based on a neural network (GNN), a spatial relationship diagram of the pipeline network topology is constructed. The weight of the transmission effect between nodes is dynamically calculated through a hydraulic model, abnormal transmission paths are identified, and a spatial correlation feature matrix is ​​output. S402: Event Feature Labeling: Define a pipeline event feature library, use a deep learning model to learn the spatiotemporal patterns of historical events, output event probabilities, and label suspected events with spatiotemporal tags; S403: Model training and optimization. Modeling is performed based on temporal features, spatial features, and event labels. Predictive results are output, and model parameters are optimized in reverse through a co-evolutionary mechanism. At the same time, the operation and maintenance plan is dynamically optimized.

[0015] Preferably, the present invention also provides an efficient prediction system for pipeline network operation and maintenance data. The system includes: a data acquisition and transmission module, a verification and completion module, an integration and cleaning module, an extraction and modeling module, a decision push module, and a visualization module. The data acquisition and transmission module is used to realize standardized acquisition and hierarchical transmission of multi-source heterogeneous data. The verification and completion module is used to ensure high accuracy and integrity of data. The integration and cleaning module is used to construct a physically consistent digital twin of the pipeline network. The extraction and modeling module is used to realize accurate fault prediction and model self-evolution. The decision push module is used to realize multi-role collaborative decision support. The visualization module provides an immersive decision support environment.

[0016] Preferably, the acquisition and transmission module includes a protocol access module, an edge computing module, and a hierarchical transmission module; the verification and completion module includes a dynamic fusion module, a missing detection module, and an interpolation module; the integration and cleaning module includes a topology module, a multi-source fusion module, and a cleaning module; the extraction and modeling module includes an extraction module, an analysis module, a labeling module, and a collaborative modeling module; the decision push module includes a message routing module, a permission module, and a terminal adaptation module; and the visualization module includes a cockpit module, a report generation module, and an interaction module.

[0017] The beneficial effects of this invention are as follows: First, the method used in this invention can automatically convert and solve the communication problem between different heterogeneous devices, and perform hierarchical transmission. Second, the operation data is constrained by Kalman filtering and Bernoulli equation, which can suppress the influence of noise. The dynamic weight fusion + residual > 3σ adaptive method can correct sensor deviation, identify missing types, and perform different interpolation calculations according to the missing type, thus ensuring the accuracy of subsequent operation data.

[0018] Secondly, this invention can construct a realistic pipeline network model through node mapping and elevation compensation, and can spatiotemporally fuse pressure / flow / gas data through data alignment (tensor construction + time delay compensation), and reduce the false alarm rate by adopting three-level cleaning, thereby improving the abnormal data identification rate by 30%.

[0019] Third, based on archived data, time-series feature extraction and spatial correlation analysis are performed. Then, joint features are formed through event feature labeling. The model is then trained and optimized through the collaborative modeling module, which can predict potential fault points and optimize operation and maintenance plans. Through the above technical solutions, the prediction model can adapt to changes in different seasons and operating conditions. The accuracy of fault prediction is improved by about 15% compared with the traditional method of 80%, and the operation and maintenance cost is reduced by 30%. Attached Figure Description

[0020] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a detailed flowchart of S1 of the present invention; Figure 3 This is a detailed flowchart of S2 of the present invention; Figure 4 This is a flowchart illustrating the adaptive adjustment algorithm of the present invention. Figure 5 This is a detailed flowchart of S3 of the present invention; Figure 6 This is a flowchart of the three-level quality control strategy of the present invention; Figure 7 This is a partial flowchart of the model integration and cleaning process of the present invention; Figure 8 This is a detailed flowchart of S4 of the present invention; Figure 9 This is a flowchart illustrating the composition of the invention system. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0022] In the description of this invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0023] Example 1 Please see the appendix Figure 1-9 An efficient prediction method for pipeline network operation and maintenance data includes the following steps: S1: Deploy IoT devices at key nodes throughout the pipeline network, acquire pipeline operation data through a protocol access module, and then automatically convert it to a unified format using an embedded protocol. The operation data is then transmitted to the data warehouse using a hierarchical transmission module. The main steps for deploying IoT devices at key nodes include key node site selection, device installation, protocol configuration, local networking, and central access. Key node site selection includes main pipeline junctions, old pipe sections, pump station outlets, pipeline ends, and valve control points. The installed equipment includes pressure transmitters, solenoid valves, flow meters, static pressure level gauges, and hydrogen sulfide gas sensors. Then, local networking is performed according to the communication topology to solve... The problem of communication between different heterogeneous devices is addressed by outputting data in a unified format, dividing it into critical data and routine data, and transmitting them in a hierarchical manner. Critical data, which is high-value data that directly affects safety warnings, fault diagnosis, and real-time decision-making, requires millisecond-level response and includes data such as pressure surges, H2S concentration exceeding limits, abnormal flow oscillations, and equipment offline signals. Routine data includes stable temperature values, normal flow values, pipe wall corrosion rates, and ambient temperature and humidity, and is generally transmitted after edge computing and preliminary noise filtering. This technical solution can solve the problem of inconsistent data formats caused by the diversity of pipeline network equipment protocols and can process and transmit data of different levels in a hierarchical manner. Example: 1. Deploy sensors at key nodes in the pipeline network (such as pump station outlets and main pipeline junctions): pressure transmitters (to monitor pressure changes), hydrostatic level gauges (to monitor liquid level), electromagnetic flow meters (to monitor flow rate), and hydrogen sulfide gas sensors (to monitor H2S concentration).

[0024] 2. Data acquisition via protocol access module: The sensor uses the Modbus / RS485 protocol, the gateway supports the MQTT protocol, and the embedded protocol parser automatically converts the data into JSON format.

[0025] 3. Data Classification: Key data includes pressure spikes (e.g., rate of change > 10 kPa / s), H2S concentration exceeding limits (> 10 ppm), abnormal flow oscillations (amplitude > 20%), and equipment offline signals. This data is directly transmitted to the central data warehouse. Routine data includes stable temperature values ​​(25 ± 2℃), normal flow rates (fluctuation < 5%), and pipe wall corrosion rates (< 0.1 mm / year). This data is first filtered by edge computing nodes (deployed on gateways), using a sliding window mean filter to remove noise before transmission.

[0026] S2: The verification and completion module performs real-time verification and correction of the operational data in the data warehouse. It mainly addresses the problems of uneven data reliability caused by differences in sensor accuracy, environmental noise interference, and poor adaptability of traditional filtering algorithms under dynamic conditions. Kalman filtering (local filtering) can suppress the influence of noise, and Bernoulli equation constraints (pre-filtering) can ensure the rationality of fluid dynamics. Based on the historical error rate, the reliability weight is dynamically allocated. Dynamic weight fusion and residual >3σ adaptive methods can correct sensor deviations. The data correction error rate is <2%, and the adaptability to dynamic conditions is improved by 50%. The corrected data is then filled with missing data through interpolation algorithms. Specifically, it can identify the missing data type and execute different interpolation algorithms according to the missing data type, including spatiotemporal kriging, LSTM transfer, and topological constraints, so that the accuracy of missing data repair is greater than 97%, which can ensure the accuracy of subsequent operational data. S3: The completed operational data is transmitted to the integration and cleaning module, integrated according to the preset data model, and then cleaned and archived. This technical solution mainly addresses the problems of spatiotemporal asynchrony of multi-source data, abnormal data contaminating model training, and the difficulty of expressing pipeline network operation and maintenance topology relationships with discrete data. Since the operational data consists of multi-source data (pressure / flow / gas) and is spatiotemporally asynchronous, this solution can construct a realistic pipeline network model through node mapping and elevation compensation. Through data alignment (tensor construction and time delay compensation), pressure / flow / gas data can be spatiotemporally fused. The three-level cleaning can reduce the false alarm rate and improve the abnormal data identification rate by 30%. S4: Based on archived data, time-series feature extraction and spatial correlation analysis are performed. Then, joint features are formed through event feature labeling. The collaborative modeling module is then used for model training and optimization, which can predict potential fault points and optimize operation and maintenance plans. Through the above technical solutions, time-series feature extraction uses time series decomposition (such as STL) to extract trends, seasonality, and residuals, combined with statistical features (mean, variance, autocorrelation, etc.) and time-domain features (such as zero crossings, peak values, etc.), thereby realizing the extraction of effective time-series and spatial features from massive data. Spatial correlation analysis uses a neural network (GNN) to model the pipeline network topology, capture the mutual influence between nodes, and label event features. Abnormal events are labeled based on rules (such as thresholds) or models (such as clustering) to form feature vectors. Collaborative modeling: multi-task learning or ensemble learning is used to predict multiple fault types (such as pipe bursts, blockages, corrosion, etc.) at the same time, and physical constraints (such as Bernoulli's equation) are introduced to enhance the interpretability and accuracy of the model. Operation and maintenance optimization: By combining reinforcement learning PPO algorithm, the operation and maintenance strategy is dynamically adjusted, taking into account cost, risk and benefit. Through the above technical solution, the prediction model can adapt to changes in different seasons and operating conditions. The accuracy of fault prediction is improved by about 15% compared with the traditional method of 80%, and the operation and maintenance cost is reduced by 30%. S5: The decision push module pushes the prediction results and real-time data to relevant personnel. The message queue realizes real-time data synchronization to ensure data consistency among modules. The message queue (such as Kafka) is used to implement the publish / subscribe mode to ensure real-time data distribution. The data synchronization delay can be less than 1 second. Secondly, based on the combination of access control RBAC and attribute-based access control ABAC, fine-grained permission management is realized. S6: The prediction results are visualized through the visualization module, and a custom report function is provided. Users can generate data reports according to their needs and realize interactive functions. Specifically, 3D visualization is adopted, and WebGL technology is used to realize the 3D display of the pipeline network. The data dashboard uses key indicators (KPIs) for real-time monitoring, and anomalies are automatically located. Custom reports provide a drag-and-drop report designer and support SQL-free queries.

[0027] See appendix Figure 1-2 The IoT devices include, but are not limited to, pressure transmitters, hydrostatic level gauges, electromagnetic flow meters, and hydrogen sulfide gas sensors. The operational data specifically includes critical data and routine data. Critical data is transmitted directly in real time, while routine data requires edge computing to initially filter noise and outliers. The embedded protocol supports automatic conversion between Modbus / RS485 / MQTT protocols through a parser. The embedded protocol parser automatically converts multi-protocol data, which can eliminate compatibility issues. Then, it performs hierarchical transmission to achieve collaborative data acquisition and hierarchical transmission from multiple heterogeneous devices.

[0028] See appendix Figure 3 The specific steps of the real-time verification and correction include: S200: Grouping and Initialization. Sensors acquiring operational data are grouped into high-precision and low-precision sensor groups. Due to interference from sewage network conditions, including sludge deposition and gas corrosion causing sensor baseline drift, grouping is necessary. High-precision sensor groups are typically placed at critical risk points (pump stations / prone areas), while low-precision groups are used for regular pipe sections to reduce costs. Each sensor within both high-precision and low-precision groups is initialized individually, calibrating the sensor zero point to suppress drift. For example, high-precision pressure sensors (accuracy ±0.5%) at pump station outlets are grouped into a high-precision group, and low-precision level gauges (accuracy ±5%) in regular pipe sections are grouped into a low-precision group. Zero point calibration is performed during initialization (e.g., level gauges are zeroed when the pipe is empty). S201: Filtering. After initialization, local filtering and pre-filtering are performed on each high-precision sensor and low-precision sensor. The local filtering uses the Kalman filtering algorithm, and the pre-filtering uses Bernoulli equation constraints to ensure the rationality of sewage fluid dynamics in the pipe network. Then, the local optimal estimate is calculated respectively. S202: Global fusion. The local optimal estimate and the estimated confidence weight are globally fused through the dynamic fusion module to calculate the global optimal estimate. The estimated confidence weight is dynamically calculated from the historical error rate of each sensor. Then, residual analysis is performed based on the global optimal estimate. When the residual is >3σ, the estimated confidence weight is automatically adjusted. When the residual is normal, the correction value is output.

[0029] For example, consider the filtered data from three pressure sensors on a certain pipe section: High-precision pressure sensor A: 302.5 kPa (weight 0.7, historical error 2%); Low-precision pressure sensor B: 315.0 kPa (weight 0.2, historical error 8%); Low-precision pressure sensor C: 290.0 kPa (weight 0.1, historical error 15%); The local optimal estimate of the high-precision pressure sensor A is: 302.5 × 0.7; The local optimal estimate of the low-precision pressure sensor B is: 315 × 0.2; The local optimal estimate of the low-precision pressure sensor C is: 290 × 0.1; Its global optimal estimate = 302.5 × 0.7 + 315 × 0.2 + 290 × 0.1 = 304.75 kPa.

[0030] Residual Analysis: Low-precision pressure gauge C residual = >3σ, where σ≈3kPa.

[0031] This indicates that there is a detection problem with the detection data of the low-precision pressure sensor C. In order to avoid affecting the fusion, its weight is automatically reduced from 0.1 to 0.08 to avoid polluting the global data.

[0032] See appendix Figure 4 Before residual analysis, a new information sequence needs to be constructed in real time based on the globally optimal estimate. The new information is the difference between the observed and predicted observed values ​​during the filter cake process, dynamically adjusting the noise. The specific steps are as follows: First, calculate the new information variance based on the constructed new information sequence, where the formula for constructing the new information sequence is:

[0033] in: For the new information of time step k; Here, H represents the actual observed value at time step k; H is the observation matrix. To predict the state vector before time step k based on information up to time k-1, we then determine whether the innovation variance exceeds the threshold for 10 consecutive times. If yes, we adaptively adjust the Q / R matrix using an adaptive adjustment algorithm, update the Q / R matrix, and re-process the filtering; otherwise, we output the final estimated value data stream.

[0034] Taking the pumping station's operation during the rainy season as an example: when the innovation variance exceeds the limit 10 times consecutively, it indicates a fundamental change in the system's noise characteristics (e.g., the flow state in the pipe changes from laminar to turbulent after rainwater inflows). At this point, the adaptive algorithm will increase the process noise Q (acknowledging increased model prediction error) while decreasing the observation noise R (relying more on raw sensor data). This dynamic adjustment responds faster than manually resetting parameters and avoids the problem of a surge in false alarm rates during the rainy season. The formula for calculating the innovation variance is:

[0035] in: The innovation variance represents the dispersion of the innovation sequence; a larger value indicates greater observation noise or model error. N is the sliding window size, used to calculate the number of samples for the innovation variance (e.g., the most recent 50 data points). For the first Time-information vector; k is the current time index; The transpose of the innovation vector at time i is used to calculate the inner product of the vectors (i.e., ...). Representing vectors (square norm), and then sum the information at N time points within the window.

[0036] For example, pressure and liquid level at a key node (observation dimension m=2), sliding window N=5 (data from 5 time points), and innovation sequence (units: pressure kPa, liquid level m). Vector of The square norm is: = ; = ; = ; = = ; = = ; .

[0037] If 10 times in a row >λ×benchmark (λ is a coefficient of 2, and the benchmark value is 0.03, such as 2) If the noise level is 0.0.3, then adaptive noise adjustment will be triggered. The main purpose of this technical solution is to address sudden strong interference in the sewage pipe network, such as sudden changes in flow caused by heavy rain; and gradual equipment degradation, such as sensor probes being slowly covered by sludge. Traditional Kalman filters with fixed noise parameters cannot handle this type of time-varying noise, which is the core pain point that this solution aims to solve.

[0038] See appendix Figure 5-6 The specific steps of the interpolation algorithm to complete missing data include: S210: Identification and classification. The final estimated value data stream after global fusion is used to locate missing points through the missing detection module and identify the missing type. The judgment logic of the missing type is as follows: scan the data matrix (node ​​× time), count the length of continuous missing data by node, classify the missing data into single missing data, continuous missing data, and regional missing data according to the missing length, and then output the missing type label. S211: Construct a spatiotemporal neighborhood, and based on the missing type label, output the neighborhood dataset Ω through a multimodal missing data recognition engine; the multimodal missing data recognition engine also employs different constraint types for condition constraints, including spatial constraints, temporal constraints, hydraulic constraints, and flow direction constraints; its multimodal missing data recognition engine execution algorithm is as follows: #def build_neighborhood(missing_type, node, time_range): # Input: Missing type, node ID, missing time range #Spatial Constraints: Physically adjacent nodes (pipe spacing < 300 meters) spatial_nodes = find_spatial_neighbors(node, max_distance=300) #Time Constraint: Data within 30 minutes before and after the missing time period time_window = [time_range[0]-30, time_range[1]+30] (Unit: minutes) # Hydraulic constraints: Same pipe diameter (±10%), same material joints hydraulic_nodes = find_similar_pipes(node, diameter_tol=0.1, material="HDPE") #Flow direction constraint: Prioritize upstream nodes (based on the pipeline topology diagram) upstream_nodes = get_upstream_nodes(node) # Generate Ω from combined constraints for candidate in set(spatial_nodes + hydraulic_nodes): #Flow weight: Upstream node weight = 1.0, downstream = 0.7 weight = 1.0 if candidate in upstream_nodes else 0.7 # Extract valid data of candidate nodes within the time window data = load_data(candidate, time_window) if data.is_valid(): Ω.append((candidate, data, weight)) return Ω # Output: Neighborhood dataset Ω = [ (candidate nodes, data, weights) ].

[0039] An example is the consecutive missing nodes B: Spatially adjacent: Nodes A1 and A2 (distance from B < 200 meters) Hydraulically similar: Node C (same diameter 300mm HDPE pipe) Flow priority: Node U (direct upstream of B) Time window: [t-30min, t+30min] The output neighborhood dataset Ω = { (A1, data_A1, 0.7), (A2, data_A2, 0.7), (C, data_C, 1.0), (U, data_U, 1.0)}.

[0040] By using a four-dimensional correlation technology that combines space, time, hydraulics, and flow direction, compared to the traditional neighborhood data which only considers spatial distance and ignores the hydraulic characteristics of the pipeline network, this approach ensures that the neighborhood data conforms to the laws of fluid motion.

[0041] S212: Algorithm selection, triggering different interpolation algorithms based on the missing type; the interpolation algorithms include spatiotemporal kriging, LSTM transfer learning, and topological constraints, wherein the spatiotemporal kriging, LSTM transfer learning, and topological constraints are respectively modeled by semi-variogram dynamic modeling, fine-tuned by transfer learning, and solved by hydraulic equations; S213: Uncertainty Quantification. Different interpolation algorithms require different uncertainty quantification methods. The reliability of the results generated by different interpolation methods, namely Kriging variance, LSTM confidence interval, and hydraulic variance, is assessed. The assessment structure adopts a three-level quality control strategy, including point-level quality control (dynamic threshold), regional quality control (different adoption strategies are used through anomaly detection algorithms, such as: 5-10 nodes, increase the LSTM historical window by 50%; 11-50 nodes, start hydraulic model simulation; >50 nodes, manual inspection), and system quality control (combining point-level and regional quality control, and controlling through system quality control strategies) to ensure data quality. For high-confidence data, interpolation calculation is performed directly, while for low-confidence data, the algorithm needs to be reselected and re-quantized, and then the interpolation results with confidence labels are output. S214: Interpolation calculation. The interpolation module performs calculations according to different interpolation algorithms and then outputs the completed running data.

[0042] The above technical solution, which combines multimodal neighborhood construction, physical constraint algorithm selection, and uncertainty quantification, solves the problem of repairing complex missing scenarios in sewage pipe networks, and provides a data foundation with high completeness and high reliability for prediction models.

[0043] See appendix Figure 7 The specific steps for integrating, cleaning, and archiving data according to a pre-defined data model include: S300: Spatiotemporal topology modeling. The operational data completed in step S214 is used to map discrete sensor data into a structured graph model via a topology module. This includes node mapping, edge relationship construction, attribute embedding, elevation compensation, and topology verification, outputting the constructed topology graph. The core of this solution is to address the problem that discrete point data cannot represent the hydraulic connections within a pipe network. For example, in a sewage pipe network, the topological relationships between pumping stations, manholes, and treatment plants require first establishing a node-edge model, supplementing the Z-axis coordinates (such as manhole cover elevation), and then using Dijkstra's algorithm to verify connectivity. The algorithm for establishing the node-edge model is as follows: #def build_topology(sensors): graph = nx.Graph() # Node Mapping (Sensor → Pipeline Node) For sensor in sensors: graph.add_node(sensor.id, type=sensor.type, pos=(sensor.lon, sensor.lat)) #Edge Relationship Construction (Pipeline Connection) for pipe in pipeline_db: graph.add_edge(pipe.start_node, pipe.end_node, length = pipe.length, diameter = pipe.diameter) #Elevation compensation (DEM data correction) dem = load_digital_elevation_map() for node in graph.nodes: z = dem.query(graph.nodes[node]['pos']) graph.nodes[node]['elevation']= z #Add elevation attribute # Topology verification (Dijkstra's algorithm for verifying connectivity) if not nx.is_connected(graph): isolate_nodes = list(nx.isolates(graph)) auto_repair_connections(isolate_nodes) # Automatically repair isolated nodes return graph. S301: Multimodal data fusion, based on a topology graph, aligns pressure, flow, and gas data spatiotemporally, then performs tensor construction, time delay compensation, and physical verification before fusion output; its fusion algorithm is as follows: #def multimodal_fusion(graph, pressure_data, flow_data, gas_data): # Spatiotemporal alignment (linear interpolation + time delay compensation) aligned_flow = align_data(flow_data, target_freq="1Hz" delay=calculate_hydraulic_delay(graph)) # Hydraulic delay calculation # Tensor construction [node × time × feature] tensor = np.zeros((len(graph.nodes), len(pressure_data), 3)) for i, node in enumerate(graph.nodes): tensor[i, :, 0] = pressure_data[node] # Pressure dimension tensor[i, :, 1] = aligned_flow[node] #flow dimension tensor[i, :, 2] = gas_data[node] # Hydrogen sulfide dimension #Physical verification (Bernoulli equation constraints) for t in range(tensor.shape[1]): For edge in graph.edges: node1, node2 = edge P1, Q1 = tensor[node1, t, 0], tensor[node1, t, 1] P2, Q2 = tensor[node2, t, 0], tensor[node2, t, 1] #Mass conservation check: Q1 should ≈ Q2 (no leakage in the pipe section) if abs(Q1 - Q2)>0.1 max(Q1, Q2): tensor[node1, t, 1] = (Q1 + Q2) / 2 # Adjust flow rate Verification of Bernoulli's equation: ΔP = f(Δv, Δh) delta_P = calculate_pressure_drop(graph, node1, node2, Q1) if abs(P1 - P2 - delta_P)>5:#5kPa tolerance mark_physical_anomaly(node1, node2, t) # Mark physical anomalies return tensor This scheme ensures that pressure / flow / gas data are synchronized over time; for example, changes in hydrogen sulfide concentration lag behind changes in flow rate. Implementation requires calculating the transmission time lag between sensors (e.g., a flow rate of 0.3 meters per second corresponds to 333 milliseconds per 100 meters), aligning timestamps using tensors, and relying on Bernoulli's equation for physical verification.

[0044] S302: Data Cleaning. The fused data undergoes three levels of cleaning via the cleaning module, including L1 point anomaly cleaning, L2 context anomaly cleaning, and L3 physical rule cleaning. L1 point anomaly cleaning uses the Isolation Forest algorithm, L2 context anomaly cleaning uses GNN neighbor verification, and L3 physical rule cleaning uses Bernoulli equation constraints. The three-level cleaning mechanism targets different anomalies: L1 Isolation Forest handles instantaneous pulses caused by pump start-up and shutdown (such as pressure gauge interference from air bubbles); L2 GNN verification can detect false faults such as "pressure anomaly but adjacent nodes are normal" (reducing false alarms by 60%); L3 Bernoulli equation can capture the physical contradiction of "liquid level rise and pressure drop" (timely detection of pipeline collapse), and then correct and mark them respectively. S303: Hierarchical archiving storage securely stores corrected and tagged data in time-series databases, graph databases, and data warehouses respectively. Time-series databases store pressure fluctuations every second for real-time early warning, graph databases store topological relationships for traceability analysis, and data warehouses store cleaned data and support report generation.

[0045] Through the above steps, the sewage pipe network data was transformed from "fragmented" to "knowledge-based," providing a high-quality data foundation for the predictive model.

[0046] See appendix Figure 8 Based on archived data, the specific steps for time-series feature extraction, spatial correlation analysis, event feature labeling, model training and optimization include: S400: Temporal Feature Extraction: The S303 cleaned and archived data is extracted in time series using the extraction module, employing methods such as sliding segmentation, statistical calculation, wavelet transform, and feature fusion. S401: Spatial correlation analysis: Based on a neural network (GNN), a spatial relationship diagram of the pipeline network topology is constructed. The weight of the transmission effect between nodes is dynamically calculated through a hydraulic model, abnormal transmission paths are identified, and a spatial correlation feature matrix is ​​output. S402: Event Feature Labeling: Define a pipeline event feature library, use a deep learning model to learn the spatiotemporal patterns of historical events, output event probabilities, and label suspected events with spatiotemporal tags; S403: Model training and optimization. Modeling is performed based on temporal features, spatial features, and event labels. Predictive results are output, and model parameters are optimized in reverse through a co-evolutionary mechanism. At the same time, the operation and maintenance plan is dynamically optimized.

[0047] See appendix Figure 9 This invention also provides an efficient prediction system for pipeline network operation and maintenance data. The system includes: a data acquisition and transmission module, a verification and completion module, an integration and cleaning module, an extraction and modeling module, a decision push module, and a visualization module. The data acquisition and transmission module is used to realize standardized acquisition and hierarchical transmission of multi-source heterogeneous data. The verification and completion module is used to ensure high accuracy and integrity of data. The integration and cleaning module is used to construct a physically consistent digital twin of the pipeline network. The extraction and modeling module is used to realize accurate fault prediction and model self-evolution. The decision push module is used to realize multi-role collaborative decision support. The visualization module provides an immersive decision support environment.

[0048] The data acquisition and transmission module includes a protocol access module, an edge computing module, and a hierarchical transmission module. The protocol access module is used for unified access and protocol conversion of multi-source heterogeneous devices. The edge computing module performs real-time preprocessing of regular data, filtering noise and outliers. The hierarchical transmission module transmits data differently according to priority, optimizing bandwidth resources. The verification and completion module includes a dynamic fusion module, a missing data detection module, and an interpolation module. The dynamic fusion module is used for weighted fusion of multi-sensor data and adaptively adjusts filtering parameters. The missing data detection module accurately identifies the missing data type (single point / continuous / regional). The interpolation module selects the optimal interpolation algorithm according to the missing data type and quantifies reliability. The integration and cleaning module includes a topology module, a multi-source fusion module, and a cleaning module. The topology module is used to construct a digital twin of the pipeline network. The multi-source fusion module realizes spatiotemporal alignment and physical fusion of multi-source data. The cleaning module is used for... The system implements a three-level anomaly detection and correction mechanism. The extraction and modeling module includes an extraction module, an analysis module, a labeling module, and a collaborative modeling module. The extraction module extracts temporal and spatial features from the cleaned data. The analysis module quantifies spatial correlations between nodes and identifies fault propagation paths. The labeling module defines and identifies pipeline event characteristics. The collaborative modeling module trains a prediction model and enables parameter self-evolution. The decision-making and push module includes message routing, an access control module, and a terminal adaptation module. The message routing module distributes warning information according to event type. The access control module controls data access granularity. The terminal adaptation module provides multi-terminal data format conversion and offline support. The visualization module includes a dashboard module, a report generation module, and an interaction module. The dashboard module provides 3D dynamic display. The report generation module quickly generates customized analysis reports. The interaction module supports SQL-free queries.

[0049] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. An efficient prediction method for pipe network operation and maintenance data, characterized in that, Includes the following steps: S1: Deploy IoT devices at key nodes of the entire pipeline network and acquire pipeline operation data through a protocol access module. Then, automatically convert the data into a unified format through an embedded protocol and transmit the operation data to the data warehouse using a hierarchical transmission module. The IoT devices include pressure transmitters, hydrostatic level gauges, electromagnetic flow meters, and hydrogen sulfide gas sensors. The operation data specifically includes key data and routine data. The embedded protocol supports automatic conversion between Modbus / RS485 / MQTT protocols through a parser, and then performs hierarchical transmission. During hierarchical transmission, routine data also needs to undergo edge computing through an edge computing module to initially filter noise and outliers, thereby realizing the collection and hierarchical transmission of collaborative data from multi-source heterogeneous devices. S2: The verification and completion module performs real-time verification and correction on the operational data in the data warehouse, and uses an interpolation algorithm to complete the missing data on the corrected data. The specific steps of the real-time verification and correction include: S200: Grouping and initialization. The sensors corresponding to the acquired operation data are grouped into high-precision sensor groups and low-precision sensor groups, and each sensor in the high-precision sensor group and low-precision sensor group is initialized individually. S201: Filtering. After initialization, local filtering and pre-filtering are performed on each high-precision sensor and low-precision sensor respectively. The local filtering adopts the Kalman filtering algorithm, and the pre-filtering adopts Bernoulli equation constraints. Then, the local optimal estimate is calculated respectively. S202: Global Fusion. A dynamic fusion module globally fuses the local optimal estimate with the estimated confidence weights to calculate the global optimal estimate. The estimated confidence weights are dynamically calculated from the historical error rate of each sensor. Residual analysis is then performed based on the global optimal estimate. When the residual > 3σ, the estimated confidence weights are automatically adjusted. When the residual is normal, a correction value is output. Before residual analysis, a novelty sequence is constructed in real-time based on the global optimal estimate, and noise is dynamically adjusted. The specific steps are as follows: First, the novelty variance is calculated based on the constructed novelty sequence. Then, it is determined whether the novelty variance exceeds the threshold for 10 consecutive times. If yes, the Q / R matrix is ​​adaptively adjusted using an adaptive adjustment algorithm, and the Q / R matrix is ​​updated and filtered again. If no, the final estimated value data stream is output. S3: The completed runtime data is transmitted to the integration and cleaning module, integrated according to the preset data model, and the integrated data is cleaned and archived. The specific steps include: S300: Spatiotemporal topology modeling, which maps the discrete sensor data into a structured graph model through the topology module, using the completed running data from step S2. Specifically, it includes node mapping, edge relationship construction, attribute embedding, elevation compensation, and topology verification, and outputs the constructed topology graph. S301: Multimodal data fusion, based on topology graph, aligns pressure data, flow data and gas data in time and space, and then fuses and outputs the data after tensor construction, time delay compensation and physical verification; S302: Data cleaning. The fused data is cleaned in three levels through the cleaning module, including L1 point anomaly cleaning, L2 context anomaly cleaning and L3 physical rule cleaning. L1 point anomaly cleaning uses the isolated forest algorithm, L2 context anomaly cleaning uses GNN neighbor verification, and L3 physical rule cleaning uses Bernoulli equation constraints. Then, they are corrected and labeled respectively. S303: Tiered archiving storage securely stores corrected and tagged data according to time-series databases, graph databases, and data warehouses respectively; S4: Based on archived data, it performs time-series feature extraction and spatial correlation analysis, then forms joint features through event feature labeling, and then trains and optimizes the model through the collaborative modeling module, which can predict potential fault points and optimize operation and maintenance plans; S5: Push the prediction results and real-time data to relevant personnel through the decision push module, and realize real-time data synchronization through the message queue to ensure data consistency among modules; S6: The prediction results are visualized through the visualization module, and a custom report function is provided to support users in generating data reports according to their needs and realizing interactive functions.

2. The method of claim 1, wherein, The formula for constructing the new information sequence is: where: is the innovation for time step k; is the actual observation value for time step k; H is the observation matrix; is the prediction of the state vector based on information up to time k - 1 prior to time step k; the innovation variance is calculated as: wherein: is the innovation variance; N is the sliding window size; is the innovation vector at the i-th time instant; k is the current time index; is the transpose of the innovation vector at the i-th time instant.

3. The method of claim 1, wherein, The specific steps of the interpolation algorithm to complete missing data include: S210: Identification and classification: The final estimated value data stream after global fusion is used to locate missing points through the missing detection module and identify the missing type. The judgment logic of the missing type is as follows: scan the data matrix, count the length of continuous missing points by node, classify the missing points into single missing points, continuous missing points and regional missing points according to the missing length, and then output the missing type label. S211: Construct a spatiotemporal neighborhood, and output the neighborhood dataset Ω based on the missing type label through a multimodal missing data recognition engine; the multimodal missing data recognition engine also uses different constraint types for condition constraints, including spatial constraints, temporal constraints, hydraulic constraints, and flow direction constraints. S212: Algorithm selection, triggering different interpolation algorithms based on the missing type; the interpolation algorithms include spatiotemporal kriging, LSTM transfer learning, and topological constraints, wherein the spatiotemporal kriging, LSTM transfer learning, and topological constraints are respectively modeled by semi-variogram dynamic modeling, fine-tuned by transfer learning, and solved by hydraulic equations; S213: Uncertainty quantification. Different interpolation algorithms require different uncertainty quantification methods. The reliability of the results generated by different interpolation methods is evaluated, and then the interpolation results with confidence labels are output. S214: Interpolation calculation. The interpolation module performs calculations according to different interpolation algorithms and then outputs the completed running data.

4. The method of claim 1, wherein, Based on archived data, the specific steps for time-series feature extraction, spatial correlation analysis, event feature labeling, model training, and optimization include: S400: Temporal Feature Extraction: The S3 cleaned and archived data is extracted in time series using the extraction module, employing sliding segmentation, statistical calculation, wavelet transform, and feature fusion. S401: Spatial correlation analysis: Based on the neural network GNN, a spatial relationship diagram of the pipeline topology is constructed. The weight of the transmission effect between nodes is dynamically calculated through the hydraulic model, abnormal transmission paths are identified, and a spatial correlation feature matrix is ​​output. S402: Event Feature Labeling: Define a pipeline event feature library, use a deep learning model to learn the spatiotemporal patterns of historical events, output event probabilities, and label suspected events with spatiotemporal tags; S403: Model training and optimization. Modeling is performed based on temporal features, spatial features, and event labels. Predictive results are output, and model parameters are optimized in reverse through a co-evolutionary mechanism. At the same time, the operation and maintenance plan is dynamically optimized.

5. An efficient pipe network operation data prediction system for implementing the efficient pipe network operation data prediction method of any one of claims 1-4, characterized in that, The system includes: a data acquisition and transmission module, a verification and completion module, an integration and cleaning module, an extraction and modeling module, a decision push module, and a visualization module. The data acquisition and transmission module is used to achieve standardized acquisition and hierarchical transmission of multi-source heterogeneous data. The verification and completion module is used to ensure high data accuracy and integrity. The integration and cleaning module is used to construct a physically consistent digital twin of the pipeline network. The extraction and modeling module is used to achieve accurate fault prediction and model self-evolution. The decision push module is used to achieve multi-role collaborative decision support. The visualization module provides an immersive decision support environment.

6. The system for efficient prediction of pipe network operation and maintenance data according to claim 5, characterized in that, The acquisition and transmission module further includes a protocol access module, an edge computing module, and a hierarchical transmission module; the verification and completion module includes a dynamic fusion module, a missing detection module, and an interpolation module; the integration and cleaning module includes a topology module, a multi-source fusion module, and a cleaning module; the extraction and modeling module includes an extraction module, an analysis module, a labeling module, and a collaborative modeling module; the decision push module includes a message routing module, a permission module, and a terminal adaptation module; and the visualization module includes a cockpit module, a report generation module, and an interaction module.

Images