Intelligent detection method, module, system and medium for underground pipeline network data based on multi-source data fusion

Abstract

The present invention belongs to the field of data processing technology and provides an intelligent detection method for underground pipeline network data using multi-source data fusion. The method first obtains multi-source detection data of the underground pipeline network; then, based on the multi-source detection data and an isolation forest algorithm based on Bayesian optimization, outlier information is determined; finally, the outlier information is input into a pre-established pipeline status model to determine the fault prediction value of the outlier point in the underground pipeline network; wherein, the parameters of the pipeline status model are updated in real time according to an online variational inference algorithm; the online variational inference algorithm is configured with a first learning rate for global parameters and a second learning rate for local parameters; the first learning rate is used to reflect the common characteristics of the entire monitoring system or pipeline network; the second learning rate is used to reflect changes in sensors caused by changes in the underground environment. By using differentiated learning rates for global and local parameters, the pipeline status model is continuously updated in real time, thereby accurately predicting future safety risks of the underground pipeline network.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular relates to an intelligent detection method for underground pipe network data by integrating multi-source data. Background Art

[0002] With the acceleration of urbanization, underground pipeline networks are becoming increasingly complex, encompassing multiple types of pipelines, including water supply, drainage, gas, electricity, and communications. Due to the high concealment of underground pipelines, their long-term exposure to corrosive media, and susceptibility to electromagnetic interference, traditional detection methods struggle to meet the demands for high-precision and high-efficiency monitoring. Therefore, a new intelligent underground pipeline detection technology that integrates multi-source data is urgently needed.

[0003] Traditional underground pipeline detection mainly relies on a single technical means, including: collecting reflection wave characteristics through geological radar to locate pipelines and estimate buried depth; using high-precision magnetometers and other equipment to track ferromagnetic pipelines based on electromagnetic induction; using piezoelectric sensors, distributed optical fibers, etc. to detect pipeline leakage through acoustic wave detection; relying on manual inspections or fixed sensor single-point monitoring; obtaining prior attribute data such as pipeline type and material through geographic information systems, but lacking real-time dynamic analysis capabilities.

[0004] However, a single technology is easily affected by the complex underground environment, and there is no cross-validation of multi-source data, resulting in large deviations in the detection results. Existing technologies lack the ability to intelligently analyze and provide real-time feedback on multi-source data, and are unable to dynamically model and provide fault warnings. Therefore, the current intelligent detection accuracy of underground pipeline data is relatively low. Summary of the Invention

[0005] In view of this, the present invention provides a method for intelligent detection of underground pipeline network data by fusion of multi-source data, aiming to solve the problem of low accuracy of intelligent detection of underground pipeline network data in the prior art.

[0006] A first aspect of an embodiment of the present invention provides a method for intelligent detection of underground pipe network data by fusion of multi-source data, comprising:

[0007] Obtain multi-source detection data of underground pipeline networks;

[0008] Determine outlier information based on multi-source detection data and the isolation forest algorithm based on Bayesian optimization;

[0009] Input the abnormal point information into the pre-established pipeline status model to determine the fault prediction value of the abnormal point in the underground pipeline network;

[0010] The parameters of the pipeline state model are updated in real time according to an online variational inference algorithm. The online variational inference algorithm is equipped with a first learning rate for global parameters and a second learning rate for local parameters. The first learning rate is used to reflect the common characteristics of the entire monitoring system or pipeline network. The second learning rate is used to reflect the changes in the sensor caused by changes in the underground environment.

[0011] In one possible implementation, an initial value of the second learning rate is greater than an initial value of the first learning rate, and the method further includes:

[0012] When the number of optimization rounds is greater than the first preset number of rounds and not greater than the second preset number of rounds, the first learning rate is kept unchanged and the second learning rate is controlled to decay exponentially; wherein the second preset number of rounds is greater than the first preset number of rounds;

[0013] When the number of optimization rounds is greater than a second preset number of rounds, the local parameters are frozen and the first learning rate is set to a minimum value.

[0014] In a possible implementation, after acquiring multi-source detection data of the underground pipe network, the method further includes:

[0015] De-noising the multi-source detection data based on the extended Kalman filter algorithm;

[0016] The method further includes:

[0017] The initial value of the second learning rate is determined according to the covariance in the denoising process of the extended Kalman filter algorithm.

[0018] In one possible implementation, the online variational inference algorithm is further provided with a forgetting factor; the method further includes:

[0019] According to the abnormal point information, calculate the data change rate of the abnormal point;

[0020] Determine the forgetting factor based on the data change rate.

[0021] In one possible implementation, the online variational inference algorithm is further configured with a forgetting factor; the pipeline network data intelligent detection phase includes a corrosion detection phase and a leakage response phase; and the method further includes:

[0022] According to the outlier information, calculate the data change rate of the outlier and determine the current stage;

[0023] Determine the forgetting factor based on the data change rate and the current stage.

[0024] In a possible implementation, after acquiring multi-source detection data of the underground pipe network, the method further includes:

[0025] De-noising the multi-source detection data based on the extended Kalman filter algorithm;

[0026] The method further includes:

[0027] According to the covariance of the denoising process of the extended Kalman filter algorithm, the forgetting factor adjustment value in the corrosion detection stage is determined;

[0028] Calculate the consistency index based on the time series variance of each sensor data;

[0029] Based on the consistency index, the forgetting factor adjustment value in the leakage response phase is determined.

[0030] In a possible implementation, the method further includes:

[0031] When the fault prediction value is greater than the preset threshold, an alarm message is generated.

[0032] The second aspect of an embodiment of the present invention provides a data processing module, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, the steps of the intelligent detection method for underground pipeline data by multi-source data fusion as described in the first aspect above are implemented.

[0033] A third aspect of an embodiment of the present invention provides an underground pipe network intelligent detection system, comprising a data acquisition module and the data processing module of the second aspect above.

[0034] The fourth aspect of an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the underground pipeline data intelligent detection method of multi-source data fusion as described in the first aspect above.

[0035] The intelligent detection method for underground pipeline network data using multi-source data fusion provided by an embodiment of the present invention first obtains multi-source detection data of the underground pipeline network; then, based on the multi-source detection data and an isolation forest algorithm based on Bayesian optimization, determines outlier information; finally, the outlier information is input into a pre-established pipeline status model to determine the fault prediction value of the outlier point in the underground pipeline network; wherein, the parameters of the pipeline status model are updated in real time according to an online variational inference algorithm; the online variational inference algorithm is configured with a first learning rate for global parameters and a second learning rate for local parameters; the first learning rate is used to reflect the common characteristics of the entire monitoring system or pipeline network; the second learning rate is used to reflect changes in the sensor caused by changes in the underground environment. By using differentiated learning rates for global and local parameters, the pipeline status model is continuously updated in real time, thereby accurately predicting future safety risks of the underground pipeline network. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0037] Figure 1 Schematic diagram of the structure of the underground pipe network intelligent detection system provided by an embodiment of the present invention;

[0038] Figure 2 This is a flowchart for implementing the method for intelligent detection of underground pipe network data using multi-source data fusion provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In the following description, specific details such as particular system structures and techniques are provided for purposes of illustration, not limitation, to facilitate a thorough understanding of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

[0040] Figure 1 FIG is a schematic diagram of the structure of the underground pipe network intelligent detection system provided by the embodiment of the present invention. Figure 1 As shown, in some embodiments, the underground pipe network intelligent detection system includes: a data acquisition module 11 and a data processing module 12.

[0041] The data acquisition module 11 integrates multiple types of sensors to cover the multi-dimensional physical characteristics and prior attributes of the underground pipeline network, realizes full-factor data collection, and then sends it to the data processing module 12 to realize intelligent detection of the underground pipeline network.

[0042] Among them, the data acquisition module 11 includes a variety of acquisition equipment such as ground penetrating radar, ground lidar, high-precision magnetometer, piezoelectric sensor, distributed optical fiber, acoustic wave detection equipment, infrared thermal imager (mounted on a drone), and geographic information system.

[0043] The data processing module 12 specifically includes:

[0044] Data preprocessing module: denoise, correct and unify the format of the collected raw data.

[0045] Data analysis: Data from different sources are integrated and processed, and then intelligent analysis is used to identify the type, location, depth and status of the pipeline network, and detect anomalies (such as damage, leakage, etc.).

[0046] Data management module: stores, queries and shares detection data, and supports integration with other geographic information systems.

[0047] Figure 2 This is a flowchart of the implementation of the method for intelligent detection of underground pipe network data by multi-source data fusion provided by the embodiment of the present invention. Figure 2 As shown, in some embodiments, a method for intelligent detection of underground pipe network data using multi-source data fusion includes:

[0048] S210, acquiring multi-source detection data of the underground pipe network;

[0049] S220, determining outlier information based on the multi-source detection data and an isolation forest algorithm based on Bayesian optimization;

[0050] S230, inputting the abnormal point information into a pre-established pipeline status model to determine a fault prediction value of the abnormal point of the underground pipeline network;

[0051] The parameters of the pipeline state model are updated in real time according to an online variational inference algorithm. The online variational inference algorithm is equipped with a first learning rate for global parameters and a second learning rate for local parameters. The first learning rate is used to reflect the common characteristics of the entire monitoring system or pipeline network. The second learning rate is used to reflect the changes in the sensor caused by changes in the underground environment.

[0052] In an embodiment of the present invention, a ground-penetrating radar collects the time, amplitude, and frequency characteristics of the reflected wave for pipeline positioning and burial depth estimation. A ground-based laser radar extracts surface deformation and the coordinates of pipeline entrance markers. A high-precision magnetometer collects changes in the magnetic field gradient of ferromagnetic pipelines to track deeply buried metal pipelines. Piezoelectric sensors and distributed optical fibers can collect pipeline leakage soundprint characteristics and vibration spectrum information to locate water / gas pipeline leaks. Infrared thermal imagers monitor areas of abnormal surface temperature to detect hot spots of heating pipeline leaks. Geographic information systems are used to provide key information such as pipeline type, diameter, material, and laying year, and can also be used for data verification and prior knowledge supplementation.

[0053] In the embodiment of the present invention, after the raw data is collected, it is preprocessed to eliminate noise and errors. Noise is eliminated and errors are corrected through a multi-stage processing flow to ensure that the data quality meets the requirements of subsequent fusion analysis.

[0054] Due to the complex environment of underground pipelines (such as high concealment (deep burial, non-visibility), long-term exposure to corrosive media (soil, groundwater), electromagnetic interference, signal transmission issues, etc.), more complex algorithms are required to process noise, perform real-time analysis, or perform predictive maintenance. At the same time, it is necessary to integrate multimodal data such as pressure, vibration, and acoustics for cross-validation. In other fields, such as agricultural monitoring (temperature, humidity + light), the data correlation is simple, and the algorithm is mainly based on linear regression; electricity is mostly coupled with multiple physical fields, such as the interweaving of electrical (current, voltage), thermal (conductor temperature), mechanical (transformer vibration), and chemical (gas in oil) data.

[0055] Underground pipeline monitoring involves numerous noise sources, including vehicle vibration, soil displacement, construction activity, soil temperature, humidity, climate temperature, and light. This contrasts with the relatively stable monitoring environment in other areas. Pipeline monitoring requires the integration of data from different physical quantities, such as pressure and strain, while other areas, such as power generation, may prioritize multi-sensor data from the same physical quantity.

[0056] Based on this, the present invention adopts a multi-sensor joint denoising method for denoising. This method is a technical method that eliminates noise and improves data quality by integrating data from multiple sensors. Compared with single-sensor denoising, it has the following significant advantages:

[0057] (1) Noise characteristic complementarity: Joint denoising of multi-sensor data can utilize the complementarity of noise characteristics of different sensors to comprehensively suppress various types of interference.

[0058] (2) Improving the signal-to-noise ratio: The signal-to-noise ratio (SNR) of a single sensor is limited by hardware performance and environmental interference. Multi-sensor data fusion can significantly improve the SNR through weighted averaging or filtering algorithms.

[0059] (3) Improve anti-interference capability: For example, in a strong electromagnetic interference environment, the magnetometer data may fail, but GPR and LiDAR can still provide reliable data.

[0060] (4) Improve fault tolerance: Even if some sensors fail, target information can still be recovered through data from other sensors. If the DAS fiber is broken, the pipeline can still be located using GPR and magnetometer data.

[0061] (5) Improve data accuracy: Multi-sensor joint denoising can improve data accuracy through cross-validation and error compensation algorithms.

[0062] (6) Multi-sensor joint denoising can dynamically adjust weights according to environmental conditions and data quality.

[0063] (7) Multi-sensor joint denoising can dynamically adjust weights based on environmental conditions and data quality. For example, GPR+LiDAR+magnetometer: Jointly extract electromagnetic reflection, geometric shape, and magnetic field gradient features to improve pipeline material recognition accuracy; DAS+infrared thermal imager: Jointly extract vibration spectrum and temperature distribution features to accurately locate leak points.

[0064] The specific contents of denoising are as follows:

[0065] 1. Multi-sensor joint denoising

[0066] 1.1 Extended Kalman Filter (EKF) The extended Kalman filter algorithm extends the Kalman filter to nonlinear systems and handles nonlinear problems by linearizing the nonlinear system model. It uses Taylor expansion to approximate nonlinear functions and updates the state estimate and covariance matrix at each time step. Its state equation is:

[0067] =

[0068] in, is the system state vector (such as GPR reflection wave amplitude, LiDAR point cloud coordinates), is the process noise (assuming it is zero-mean Gaussian noise, the covariance matrix ), is a nonlinear state transfer function.

[0069] The measurement equation is:

[0070]

[0071] in, is the sensor observation value, is the measurement noise (covariance matrix ), is a nonlinear measurement function.

[0072] 1.2 Denoising Filtering Steps

[0073] In a multi-sensor system, assuming there is N sensors, and the measurement value of each sensor is .

[0074] First, initialize the state estimate and the covariance matrix , use the state equation to predict the state and covariance at the next moment:

[0075]

[0076]

[0077] in, is the state transition function The Jacobian matrix of is the process noise covariance matrix.

[0078] For each sensor i , calculate the measurement residual:

[0079]

[0080] The covariance of the measurement residuals is also calculated:

[0081]

[0082] in, is the measurement function The Jacobian matrix of It is a sensor The measurement noise covariance matrix of .

[0083] The Kalman gain is then calculated:

[0084]

[0085] Update state estimates and covariances:

[0086]

[0087]

[0088] Finally, the state estimates are weighted averaged according to the measurement accuracy of each sensor (the covariance matrix of the measurement noise):

[0089]

[0090]

[0091] After denoising, multi-source data needs to be aligned to ensure consistency and comparability in terms of time, space, and semantics. Multi-source data also needs to be normalized to eliminate differences in dimensions and value ranges, making different types of data comparable.

[0092] The alignment of multi-source data is specifically as follows:

[0093] Spatial registration includes coarse registration and fine registration. Coarse registration converts the coordinates of each sensor into a unified engineering coordinate system, while fine registration uses the ICP (Iterative Closest Point) algorithm to optimize the registration parameters.

[0094] For example, when the spatial reference is unified, coordinate transformation and feature anchor point matching can be used to control the error of multi-source data to the centimeter level, laying the foundation for subsequent analysis. If the data involves a time dimension (such as dynamic monitoring), clock synchronization technology can be used to align the acquisition timestamps of different devices.

[0095] The present invention can also perform feature dimension fusion. Specifically, it can use Fourier transform, machine learning and other technologies to achieve cross-modal correlation of time domain / frequency domain / spatial domain data, thereby improving the accuracy of anomaly recognition.

[0096] Then, the multi-source data is normalized to eliminate the differences in data dimensions, value ranges, etc., so that different types of data are comparable. Specifically, normalization can be performed using the following formula:

[0097] Min-Max:

[0098]

[0099] This method is applicable to continuous variables such as GPR amplitude and LiDAR point cloud intensity.

[0100] Z-Score Normalization:

[0101]

[0102] This method is suitable for data such as magnetic field gradient and temperature that follow Gaussian distribution.

[0103] Next, the normalized data needs to be tested and removed for outliers. This paper uses an isolation forest algorithm and Bayesian optimization for the parameters in the isolation forest. This method offers the following advantages over other methods:

[0104] (1) Because its time complexity is , so it is suitable for processing large-scale data.

[0105] (2) No labeled data is required and it can be directly applied to unlabeled datasets.

[0106] (3) It has good robustness to noise and outliers.

[0107] (4) The degree of anomaly can be intuitively explained by path length.

[0108] Randomly select a feature from the dataset j , assuming the data set has d features, randomly select one of the features j In the features j Minimum value of and maximum value Randomly select a split value between p .

[0109] Based on the split value p Divide the dataset into two subsets: Left subset: features j The value is less than p Right subset: features j The value is greater than or equal to p .

[0110] Recursively perform the above steps for each subset until one of the following termination conditions is met: there is only one point in the subset; the tree reaches the maximum depth (Usually set to ,in is the size of the dataset).

[0111] For data points x , whose path length Is the number of edges from the root node to the leaf node. Construct multiple isolated trees (usually 100-200 trees) to calculate x Average path length in all trees E .

[0112] For the inclusion n Data set of points, average path length The approximate value of is:

[0113]

[0114] in It is a harmonic number.

[0115] Data Points x Anomaly score Defined as:

[0116]

[0117] Among them, if ,but x It is likely an outlier. ,butx It is normal. If ,but x Most likely a normal point (long path length).

[0118] In water supply network leakage monitoring, due to the multi-factor coupling characteristics of the failure mechanism, as shown in the above case, leakage events are often caused by the coupling of multiple physical quantities such as pressure fluctuations, flow anomalies, temperature changes, and vibration frequency offsets. However, in existing monitoring systems, the potential correlation between various sensor parameters is still uncertain. Based on this, this patented technical solution innovatively introduces a sensor data independence analysis mechanism:

[0119] When the sensor datasets meet the independence assumption, a univariate isolation forest algorithm is used for parallel training, with outlier detection achieved through weighted averaging of anomaly scores. If significant data correlation is detected (Pearson coefficient ρ > 0.6), a multidimensional feature vector space is constructed. Principal component analysis (PCA) is employed to reduce dimensionality to 90% variance preservation. This method allows for gradual model updates without requiring all data to be loaded at once. This means that if the data distribution changes, the PCA and isolation forest models must be refitted periodically. This avoids the curse of dimensionality caused by feature redundancy while ensuring model generalization under diverse operating conditions.

[0120] In some embodiments, the initial value of the second learning rate is greater than the initial value of the first learning rate, and the method further includes: when the number of optimization rounds is greater than the first preset number of rounds and not greater than the second preset number of rounds, maintaining the first learning rate unchanged and controlling the second learning rate to decay exponentially; wherein the second preset number of rounds is greater than the first preset number of rounds; when the number of optimization rounds is greater than the second preset number of rounds, freezing the local parameters and setting the first learning rate to the minimum value.

[0121] In an embodiment of the present invention, data from different sources will be fused and processed, and then the type, location, depth and status of the pipeline network will be identified through intelligent analysis, and abnormal conditions (such as damage, leakage, etc.) will be detected. After determining some abnormal points in the pipeline, these abnormal points will be repaired or further monitored. Specifically, an online variational inference algorithm can be introduced to effectively reduce the impact of sensor noise and data conflicts by performing real-time probability modeling and dynamic parameter updates on multi-source heterogeneous data, thereby generating monitoring results with higher confidence. That is, by optimizing a simple variational distribution q(z) to approximate the complex true posterior distribution p(z|x), where z represents the latent variable (such as pipeline-related parameters) and x is the observed data (sensor signal). The goal is to maximize the evidence lower bound (ELBO):

[0122] Underground pipelines (such as water and oil pipes) may leak or experience structural damage due to aging, corrosion, or external forces. Traditional detection methods (such as manual inspections and fixed sensors) are inefficient and cannot provide real-time analysis. Online variational inference can be used to process sensor data in real time, dynamically update pipeline status models, and quickly detect anomalies.

[0123] First, define the sensor data and the variables to be detected.

[0124] Input data (observation data ):

[0125] Pressure sensor data: Changes in pipeline pressure over time (pressure drops suddenly when leaking).

[0126] Acoustic signal: Leaks or corrosion will emit sound waves of a specific frequency, vibration energy (Leakage causes increased vibration)

[0127] Flow data: Differences in outlet flow and inlet flow may indicate a leak.

[0128] Temperature / Chemical Sensors: Corrosion may be accompanied by temperature changes or the release of specific chemicals. Infrared Area Temperature

[0129] Hidden variables (unknown quantities to be detected) ):

[0130] Leak location (1D coordinate or 3D coordinate x=(x,y,z)).

[0131] Leakage intensity (leak hole size, such as leak hole area A) directly affects the pressure change and flow difference in the pipeline.

[0132] The degree of corrosion (reduction in pipe wall thickness).

[0133] Pipeline health status (normal, slightly corroded, severely corroded).

[0134] Sometimes also referred to as leakage rate , sensor noise level .

[0135] Then, a probability model is built.

[0136] Assuming that the pipeline state conforms to physical laws (such as fluid mechanics models), construct a joint probability distribution:

[0137] (x,z)= (x|z)p(z)

[0138] Assume that the sensor data follows a Gaussian distribution with a mean determined by the leak intensity A. The variance is the sensor noise variance. The initial probabilities of leakage and corrosion (for example, corrosion is more likely in older pipelines) are given by the dynamic prior. The dynamic prior can also be a leakage rate prior based on pipeline pressure and material parameters. .

[0139] Next, select Variational Distribution.

[0140] Use simple distribution to approximate the true posterior distribution, for example: leakage location: use Gaussian distribution (mean represents the most likely location, variance represents uncertainty). Corrosion degree: use gamma distribution (positive value, suitable for representing gradually accumulated corrosion). , where the variational parameter , when updating, adjust the gamma distribution parameters Pipeline health status: using classification distribution (probability of normal, mild, and severe categories).

[0141] Finally, execute the online update process (taking data every minute as an example)

[0142] Assume that the pipeline health status is "normal", the corrosion level is 0, and the leakage probability is 0. Set the learning rate (such as ). Receive real-time sensor data, for example, the pressure sensor shows a sudden drop in pressure from 5MPa to 4.8MPa.

[0143] Among them, for local parameter optimization (leakage / corrosion speculation at the current time point):

[0144] Input: current pressure, sound waves and other data.

[0145] Calculation: Based on the current global model, estimate whether there is leakage at the current moment and the degree of corrosion.

[0146] For example, a sudden drop in pressure may correspond to an increase in the probability of leakage from 5% to 60%.

[0147] If the corrosion level increases slowly, it may be updated from 0.1mm to 0.12mm.

[0148] Given the unique characteristics of pipeline monitoring, global parameters reflect the common characteristics of the entire monitoring system or pipeline network. Examples include the global elastic modulus of the pipeline material and the overall pressure-flow model parameters for fluid transmission. Local parameters are susceptible to short-term environmental changes (such as sudden temperature fluctuations) or localized environmental disturbances (such as soil stress concentration within a pipeline) and require dynamic adjustment. These parameters may also include the zero drift or sensitivity factor of a sensor (such as the mV / V deviation of a pressure sensor) or the loss coefficient of each fiber optic sensor. Local parameters in pipeline monitoring often reflect changes in multi-source sensors caused by changes in the surface or underground environment. Monitoring in other areas, such as power monitoring, may focus on the status of individual devices and are relatively stable.

[0149] To address this, the present invention considers both global parameters (which apply to the entire dataset) and local parameters (which apply to each data point or sensor feature). The learning rate of global parameters may need to decay more slowly, while the learning rate of local parameters can be adjusted more quickly. This comprehensive consideration of global and local parameters, adjusting them together, leads to better convergence.

[0150] The following is an implementation example to illustrate the learning rate, but it is not intended to be limiting:

[0151] (1) Initial stage (first 10 rounds):

[0152] High local learning rate (such as sensor sensitivity, local correction factor, etc.) quickly fits the characteristics of each pipe section.

[0153] Low global learning rates (such as material properties, basic model parameters, corrosion rates, etc.) maintain model stability.

[0154] (2) Mid-term (10-50 rounds):

[0155] Exponential decay (×0.95 per epoch) is applied to the high local learning rate to avoid overfitting the noise.

[0156] Keep the low global learning rate unchanged and strengthen global feature extraction.

[0157] (3) Late stage (after 50 rounds):

[0158] Freeze local parameters and fine-tune global parameters (such as 0.0001) to improve generalization.

[0159] High learning rate local adjustment: Reduce segment-specific fitting errors.

[0160] Low learning rate global optimization: Improving cross-region prediction consistency.

[0161] Initial stage (little data): high learning rate (such as ), quickly absorb new information.

[0162] Later stage (data stability): the learning rate is reduced (e.g. 0.01) to prevent noise interference.

[0163] In some embodiments, after obtaining multi-source detection data of the underground pipeline network, the method also includes: denoising the multi-source detection data according to the extended Kalman filter algorithm; the method also includes: determining the initial value of the second learning rate based on the covariance in the denoising process of the extended Kalman filter algorithm.

[0164] In an embodiment of the present invention, the multi-sensor joint denoising effect will also have a certain impact on the learning rate. This process mainly involves the adjustment of the local parameter learning rate. Different sensor channels can be assigned independent learning rates, and the weights are inversely proportional to the covariance of each channel. The learning rates of multiple sensors are dynamically adjusted according to the Kalman filter covariance. When the covariance is large (high data noise), the learning rate is reduced to stabilize the update; when the covariance is small, the learning rate is increased to accelerate convergence. The deviation between observation and prediction is used to assess the degree of data anomaly and trigger the attenuation or reset of the learning rate. For example, in cases such as pipeline leakage, the residual sequence of the Kalman filter may mutate, which will trigger the reset of model parameters or a sudden increase in the learning rate.

[0165] In some embodiments, the online variational inference algorithm is further provided with a forgetting factor; the method further comprises: calculating the data change rate of the outlier point based on the outlier point information; and determining the forgetting factor based on the data change rate.

[0166] In the embodiment of the present invention, the traditional update formula for global model update (taking corrosion intensity as an example) is:

[0167] Estimation of current corrosion intensity

[0168] The present invention introduces the forgetting factor based on the learning rate Update, where the forgetting factor is adaptively and dynamically adjusted based on the data change rate:

[0169] Estimation of current corrosion intensity

[0170] For example, the degree of corrosion follows a gamma distribution, and the posterior distribution parameter of the degree of corrosion ( , ) is updated to:

[0171]

[0172]

[0173] The present invention considers the change of data points and introduces variance for measurement. The monitoring indicator is the variance of sensor data in the sliding window .

[0174] Adjustment rules:

[0175]

[0176] Among them, when the data mutates ( When (such as 0.6), accelerate the forgetting of old states. When the data is stable ( Hour, (such as 0.95), retaining long-term memory.

[0177] Piecewise constant forgetting factor: Corrosion detection phase (slowly changing): e.g. =0.95, emphasis on long-term trend leakage response stage (rapid changes): e.g. = 0.7, fast tracking mutation. If leakage signals are detected multiple times in a row, the global model will gradually increase the leakage probability.

[0178] In some embodiments, the online variational inference algorithm is further provided with a forgetting factor; the stages of intelligent detection of pipeline network data include a corrosion detection stage and a leakage response stage; the method further includes: calculating the data change rate of the abnormal point and determining the current stage based on the abnormal point information; and determining the forgetting factor based on the data change rate and the current stage.

[0179] In some embodiments, after obtaining multi-source detection data of the underground pipeline network, the method also includes: denoising the multi-source detection data according to the extended Kalman filter algorithm; the method also includes: determining the forgetting factor adjustment value of the corrosion detection stage according to the covariance in the denoising process of the extended Kalman filter algorithm; calculating the consistency index according to the time series variance of each sensor data; and determining the forgetting factor adjustment value of the leakage response stage according to the consistency index.

[0180] In an embodiment of the present invention, the online variational inference algorithm realizes dynamic modeling of the pipeline network status by setting a forgetting factor (γ), and combines the differentiated requirements of the corrosion detection stage and the leakage response stage to form an adaptive adjustment mechanism based on data characteristics. The pipeline network status changes slowly (such as the pipe wall thickness becomes thinner year by year), and the sensor data shows low variance and gradual trend, and the noise is mainly periodic interference (such as seasonal soil moisture changes). The pipeline network status suddenly changes (such as a sudden drop in pressure due to a pipeline rupture), and the sensor data shows high variance and multi-parameter coupling anomalies (pressure, vibration, and temperature deviate from the mean synchronously). The pre-processed multi-source data (such as pressure, flow, and vibration) is detected for anomalies through the isolation forest algorithm, and the anomaly score is output. s ( x) (The value range is 0-1, and a score greater than 0.8 is considered an outlier.) Data change rate calculation:

[0181] Change_rate=| x t - x t−n | / x t−n ×100%

[0182] in, x t is the data value at the current moment, x t−n For the front n The average value of the period ( n Take 10-30, and adjust according to the type of pipe network).

[0183] During the corrosion detection phase, if continuous m cycles (such as m =20) and the abnormal point density is less than 5% and the data change rate is less than a threshold of 1 (e.g., 3%), it is determined to be in normal operation and enters the corrosion detection stage.

[0184] In the leakage response phase, if the abnormal point density in a single cycle is greater than 10% or the data change rate is greater than threshold 2 (such as 15%), it is determined to be an abnormal burst state and enters the leakage response phase.

[0185] In the extended Kalman filter denoising process, the covariance matrix P trace( P )) reflects the noise intensity of the data. The larger the value of the trace, the higher the noise. The adjustment formula is:

[0186] c 腐蚀 = c base - k 1*trace( P max )*trace( P )

[0187] in: c base =0.95 (basic forgetting factor); k 1=0.2 (adjustment coefficient, ensure c Corrosion ≥ 0.8); trace( P max ) is the historical maximum covariance trace value. The higher the noise, c The smaller it is, the stronger the trust in the current data is and the corrosion trend is prevented from being masked by noise.

[0188] Define the consistency index of sensor data mutation: Consistency_index= N Number of synchronous mutation sensors (if the data of ≥2 types of sensors among pressure, DAS vibration, and infrared temperature exceed their respective variance thresholds at the same time, it is determined to be a "synchronous mutation").

[0189] Adjustment formula:

[0190] c Leakage = c min + k 2*Consistency_index

[0191] in, c min =0.6 (minimum forgetting factor); k 2=0.4 (adjustment coefficient, ensure c Leakage ≤0.8). The higher the consistency index, c The larger it is, the stronger the collaborative trust of multi-sensor data is, and the faster the model responds to leaks.

[0192] When switching from the corrosion detection phase to the leakage response phase, if the data change rate is greater than the threshold 2 but the consistency index is less than 0.5 (single sensor mutation), a buffer window (e.g., 5 cycles) is started to avoid misjudgment.

[0193] When recovering from the leak response phase to the corrosion detection phase, continuous m The abnormal point density of each cycle must be less than 5% and the consistency index must be less than 0.3 to prevent frequent switching caused by short-term interference.

[0194] When entering the leak response phase, reset c Leak to the initial value of 0.6, clear the weight accumulation of historical corrosion trends, and focus on the current anomaly.

[0195] When returning to the corrosion detection phase, reinitialize based on prior data (such as pipeline material, laying year) c Corrosion, such as in old pipelines c The base is set to 0.9 and the new pipeline is set to 0.95 to improve the utilization of prior knowledge.

[0196] In some embodiments, the method further includes: generating an alarm message when the fault prediction value is greater than a preset threshold.

[0197] In this embodiment of the present invention, an alarm is triggered when the leakage probability exceeds 90% or the corrosion level exceeds a safety threshold. Specifically, the decision output is the current leakage probability; if it exceeds the threshold, an alarm is triggered. The most likely leak location is determined by the mean of the variational distribution, combined with the time difference of the acoustic sensor. After the alarm is triggered, manual or robotic inspections are performed to confirm the presence of leakage / corrosion. In the event of a false alarm (e.g., a sudden pressure drop due to valve operation), the new information is incorporated into the model and the prior distribution is revised.

[0198] The present invention has the following advantages:

[0199] (1) Real-time: The model is updated every minute, earlier than traditional regular inspections.

[0200] (2) Noise immunity: A single sensor failure will not lead to a false alarm (through probabilistic smoothing).

[0201] (3) Adaptability: After the pipeline ages, the model automatically increases the prior probability of corrosion.

[0202] In some embodiments, a scenario classifier is also provided in the pipeline status model. The scenario classifier is specifically a neural network or a preset formula. After the historical fault prediction values and the data of each sensor are input into the scenario classifier, the scenario classifier will classify the current scenario. The specific scenario types may be high-load operation, slow development of corrosion, accelerated corrosion, sudden pipe burst, interface leakage, etc.

[0203] After identifying the scenario, the first learning rate, second learning rate and forgetting factor can be dynamically adjusted according to the current scenario, aiming to improve the model's adaptability to different working conditions through parameter optimization.

[0204] The following describes the adjustment process using the preset formula through several examples, but is not intended to be limiting.

[0205] In high-load operation scenarios, sensor data fluctuates periodically. The adjustment goal is to quickly respond to transient changes to avoid misjudging them as abnormalities. Specifically, the transient overload characteristics can be captured using the following formula:

[0206] =min( ×1.5,0.1)

[0207] in, p 0 is the initial value of the learning rate, p t This is the adjusted learning rate. A threshold is set to ensure that the learning rate does not exceed 0.1 to avoid overfitting. This learning rate can be the first learning rate or the second learning rate. In this scenario, adjusting the second learning rate according to this formula is more optimal.

[0208] In the corrosion acceleration scenario, the adjustment goal is to reduce the learning rate to prevent noise from interfering with the long-term trend. Specifically, the following formula can be used to stably track the slow corrosion process:

[0209] =max (η0×0.7,0.001)

[0210] A threshold is used to ensure that the learning rate does not fall below 0.001, maintaining the model's ability to continuously update. This learning rate can be either the first learning rate or the second learning rate. In this scenario, adjusting the first learning rate according to this formula is more optimal.

[0211] In the case of sudden pipe burst, the adjustment goal is to reset the learning rate to the maximum value, that is, =1.0, full-weight learning emergency state. The learning rate can be the first learning rate or the second learning rate. In this scenario, adjusting the second learning rate according to this formula is more optimal.

[0212] In the interface leakage scenario, the target is adjusted to a medium learning rate to balance steady state and slight changes, that is, = , to avoid missing tiny leakage signals.

[0213] In the scenario where corrosion develops slowly, the adjustment goal is to strengthen the weight of historical data. The long-term corrosion law can be obtained through the following formula:

[0214] =min( +0.2,0.99)

[0215] in, is the adjustment value of the forgetting factor, is the initial value of the forgetting factor, with an upper limit of 0.99 to preserve long-term corrosion trends.

[0216] In the case of sudden pipe burst, the adjustment goal is to sharply reduce λ and quickly forget the historical state, that is, =0.1 to focus on current emergencies.

[0217] In the corrosion acceleration scenario, the adjustment goal is to moderately reduce λ to weaken the impact of outdated corrosion data. The following formula can be used to prevent old data from masking the acceleration trend:

[0218] =max( -0.3,0.5)

[0219] It should be understood that the order of execution of the steps in the above embodiments does not necessarily mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0220] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A method for intelligent detection of underground pipe network data by multi-source data fusion, characterized in that: include: Obtain multi-source detection data of underground pipeline networks; Determining outlier information based on the multi-source detection data and an isolation forest algorithm based on Bayesian optimization; Inputting the abnormal point information into a pre-established pipeline status model to determine a fault prediction value of the abnormal point of the underground pipeline network; The parameters of the pipeline state model are updated in real time according to an online variational inference algorithm. The online variational inference algorithm is configured with a first learning rate for global parameters and a second learning rate for local parameters. The first learning rate is used to reflect the common characteristics of the pipeline network, and the second learning rate is used to reflect changes in the sensor caused by changes in the underground environment. An initial value of the second learning rate is greater than an initial value of the first learning rate, and the method further includes: When the number of optimization rounds is greater than a first preset number of rounds and not greater than a second preset number of rounds, maintaining the first learning rate unchanged and controlling the second learning rate to decay exponentially; wherein the second preset number of rounds is greater than the first preset number of rounds; When the number of optimization rounds is greater than a second preset number of rounds, the local parameters are frozen and the first learning rate is set to a minimum value.

2. The method for intelligent detection of underground pipe network data based on multi-source data fusion according to claim 1 is characterized in that: After acquiring the multi-source detection data of the underground pipe network, the method further includes: Performing denoising on the multi-source detection data according to an extended Kalman filter algorithm; The method further comprises: The initial value of the second learning rate is determined according to the covariance in the denoising process of the extended Kalman filter algorithm.

3. The method for intelligent detection of underground pipe network data based on multi-source data fusion according to claim 1, characterized in that: The online variational inference algorithm is further provided with a forgetting factor; the method further comprises: Calculating the data change rate of the abnormal point according to the abnormal point information; The forgetting factor is determined according to the data change rate.

4. The method for intelligent detection of underground pipe network data based on multi-source data fusion according to claim 2 is characterized in that: The online variational inference algorithm is further provided with a forgetting factor; the phases of intelligent detection of pipeline network data include a corrosion detection phase and a leakage response phase; the method further comprises: Calculate the data change rate of the abnormal point and determine the current stage based on the abnormal point information; The forgetting factor is determined according to the data change rate and the current stage.

5. The method for intelligent detection of underground pipe network data based on multi-source data fusion according to claim 3 is characterized in that: After acquiring the multi-source detection data of the underground pipe network, the method further includes: Performing denoising on the multi-source detection data according to an extended Kalman filter algorithm; The method further comprises: According to the covariance of the denoising process of the extended Kalman filter algorithm, the forgetting factor adjustment value in the corrosion detection stage is determined; Calculate the consistency index based on the time series variance of each sensor data; According to the consistency indicator, a forgetting factor adjustment value in the leakage response phase is determined.

6. The method for intelligent detection of underground pipe network data by multi-source data fusion according to any one of claims 1 to 5, characterized in that: The method further comprises: When the fault prediction value is greater than the preset threshold, an alarm message is generated.

7. A data processing device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method for intelligent detection of underground pipe network data by multi-source data fusion as described in any one of claims 1 to 6 are implemented.

8. An intelligent underground pipe network detection system, characterized in that: It comprises a data acquisition module and the data processing device as claimed in claim 7.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for intelligent detection of underground pipe network data by multi-source data fusion as described in any one of claims 1 to 6 above are implemented.

Images