A comprehensive pipe gallery intelligent monitoring data fusion method based on multi-source data

Abstract

This invention discloses a data fusion method for intelligent monitoring of integrated utility tunnels based on multi-source data. It includes: S1, multi-source data acquisition and preprocessing; S2, multi-source data fusion based on GCN and LSTM; S3, equipment failure rate modeling and feature extraction; S4, video analysis based on CenterNet and MIAUC; S5, data fusion and state assessment; and S6, multi-objective optimization control. The advantages are: it can effectively reduce information loss in feature representation, improve the accuracy of state characterization, improve the completeness of anomaly identification, improve the accuracy and robustness of anomaly detection, and reduce system energy consumption while reducing equipment failure risk.

Description

Technical Field

[0001] This invention relates to the field of intelligent monitoring and data fusion technology, and in particular to a data fusion method for intelligent monitoring of integrated utility tunnels based on multi-source data. Background Technology

[0002] Integrated utility tunnels, as an important component of urban infrastructure, are widely used for the centralized laying and management of various pipelines, including power, telecommunications, water supply and drainage, and gas. Due to the complex internal environment, diverse equipment, and variable operating conditions, these tunnels are susceptible to factors such as temperature and humidity changes, gas leaks, and equipment aging during long-term operation, posing certain safety hazards.

[0003] In existing technologies, the monitoring and management of integrated utility tunnels mainly rely on single sensor data or video surveillance systems for status assessment. This results in low data utilization and isolated information, making it difficult to comprehensively reflect the operational status of the tunnels. Furthermore, traditional methods often focus on anomaly detection or alarms, lacking quantitative analysis of equipment failure risks and comprehensive decision-making capabilities based on multi-source information.

[0004] In addition, in terms of control strategies, existing systems typically adopt fixed rules or single-objective control methods, which make it difficult to ensure safe operation while also optimizing energy consumption, resulting in low system operating efficiency. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a comprehensive intelligent monitoring data fusion method for utility tunnels that can effectively reduce information loss in feature representation, improve the accuracy of state characterization, improve the completeness of anomaly identification, improve the accuracy and robustness of anomaly detection, and reduce system energy consumption while reducing equipment failure risk.

[0006] To address the aforementioned technical problems, the integrated utility tunnel intelligent monitoring data fusion method based on multi-source data of the present invention includes the following steps:

[0007] S1. Multi-source data acquisition and preprocessing:

[0008] Collect multi-source data from the integrated utility tunnel, including environmental data, equipment operation data, and video surveillance data. Perform time alignment and normalization processing on the multi-source data, specifically as follows:

[0009] For the original data sequence x i Standardization process:

[0010]

[0011] Where μ is the mean, σ s Standard deviation;

[0012] S2. Multi-source data fusion based on GCN and LSTM:

[0013] Construct a topology graph G=(V,E) of the equipment and sensors in the integrated utility tunnel, where node V represents equipment or sensor and edge E represents spatial or functional relationships. Construct an adjacency matrix A and perform normalization processing.

[0014]

[0015] Where D is the degree matrix;

[0016] Graph convolutional networks are used to extract spatial features from multi-source data:

[0017]

[0018] Among them, H (l) For the features of layer l, W (l) Let ϕ(⋅) be the weight matrix, and let ϕ(⋅) be the nonlinear activation function.

[0019] After multi-layer graph convolution calculation, the final layer output feature H is obtained. (L) The final layer output feature H (L) Organizing the data according to the time series, we obtain the input feature vector at time t:

[0020] x t =H t (L)

[0021] Among them, H t (L) This represents the node features of the graph convolutional network at time t;

[0022] The time series feature x t Inputting data into a Long Short-Term Memory (LSTM) network for temporal modeling:

[0023]

[0024] Where, x t Let h represent the input feature vector at time t, where h is the output of the graph convolutional network. t−1 c represents the hidden state at the previous time step. t−1 f represents the cell state at the previous moment. t i t o t Let c represent the gate control vectors for the forget gate, input gate, and output gate, respectively. t W represents the cell state at the current moment. f W i W o W cThese represent the weight matrices of the corresponding gate structures, b f b i b o b c Let represent the bias vectors of the corresponding gate structures, and ϕ(⋅) represent the activation function.

[0025] tanh(⋅) denotes the hyperbolic tangent function, and [ , ] denotes the vector concatenation operation;

[0026] According to the output gate o t With cell state c t Calculate the hidden state h at the current time step. t :

[0027]

[0028] The hidden state h t As a fusion feature output;

[0029] S3. Equipment Failure Rate Modeling and Feature Extraction:

[0030] Calculate the failure rate of the device within the time interval T:

[0031]

[0032] Where N is the number of times the fault occurred;

[0033] fusing feature h t The feature vector z is formed by concatenating the feature vector with the failure rate λ. t :

[0034]

[0035] The fault prediction value y is obtained by mapping using a fully connected layer:

[0036]

[0037] Among them, W z The weight matrix representing the fully connected layer has dimensions equal to or greater than the input feature vector z. t Matching, b z The bias vector represents the bias of the fully connected layer. The weight matrix and bias vector are obtained by training on historical operating data.

[0038] S4. Video analysis based on CenterNet and MIAUC:

[0039] The surveillance video is divided into multiple time segments. Keyframes are extracted from each segment and input into CenterNet for target detection, resulting in a heatmap of the target center point.

[0040]

[0041] Where I is the input image, f det For detection model;

[0042] Construct a loss function based on multi-instance AUC optimization:

[0043]

[0044] Where n represents the number of samples involved in the loss calculation, and AUC i Let be the area under the curve for the i-th sample, and α be the weighting coefficient. This represents the target detection loss function, used to measure the error between the detection result and the ground truth label;

[0045] The video anomaly score v is obtained by weighted summation of the confidence scores of each target in the detection results. t :

[0046]

[0047] Where m represents the number of targets to be detected, p j γ represents the detection confidence of the j-th target. j This represents the anomaly weight coefficient for the corresponding target;

[0048] The anomaly score v t This is the result of video anomaly detection;

[0049] S5. Data Fusion and Status Assessment:

[0050] The fault prediction results obtained in step S3 are fused with the video analysis results obtained in step S4:

[0051]

[0052] Where β1 and β2 are weighting coefficients, and y represents the fault prediction value obtained in step S3;

[0053] The integrated utility tunnel operation status evaluation index s was obtained. t；

[0054] S6. Multi-objective optimization control:

[0055] Construct an optimization function targeting equipment failure rate and energy consumption:

[0056]

[0057] Where λ is the failure rate and E is the system energy consumption;

[0058] The control variable is represented as:

[0059]

[0060] Among them, u i Let u represent the control variable of the i-th device, where u is the set of feasible control variables.

[0061] The objective function is solved by optimization:

[0062]

[0063] Obtain the optimal control variable u ∗ .

[0064] In step S1, missing data is filled in using linear interpolation:

[0065]

[0066] In step S4, a threshold judgment is performed on the video anomaly score.

[0067]

[0068] Where θ is the anomaly detection threshold, 1 is abnormal and 0 is normal.

[0069] In step S5, y and vt are normalized before fusion.

[0070] The optimization process in step S6 is implemented using traversal search or gradient descent methods.

[0071] Advantages of this invention:

[0072] 1. This invention achieves joint analysis of equipment and environmental data in integrated utility tunnels by constructing a multi-source data fusion model based on graph structure and combining it with time-series modeling methods: using graph convolution to model the spatial relationship between equipment;

[0073] This invention utilizes time series models to model the dynamic characteristics of data changes over time. Compared to methods based solely on a single time series or independent feature analysis, this invention can simultaneously characterize spatial correlation (inter-device influence) and temporal dependence (evolution of operating state), thus effectively reducing information loss in feature representation and improving the accuracy of state characterization.

[0074] 2. This invention introduces video surveillance data, extracts anomaly information through a target detection model, and fuses it with sensor data. From a multimodal learning perspective, sensor data can reflect numerical changes, while video data can reflect visual anomalies. The two are complementary. Some anomalies (such as water accumulation or intrusion) are difficult to detect through sensors, and some equipment anomalies (such as current anomalies) are difficult to identify visually. By fusing the two types of data, this invention can reduce misjudgments caused by a single data source, thus improving the completeness of anomaly identification and also improving the accuracy and robustness of anomaly detection.

[0075] 3. This invention constructs an optimization objective (minF=w1λ+w2E). Traditional methods usually only focus on a single objective (such as safety or energy consumption), while this invention considers both safety (failure rate) and economy (energy consumption) and achieves multi-objective balance through weight combination. At the same time, through enumeration or search control strategies, it can find the optimal solution within a limited control space; ensures that the solution process is stable and achievable, and reduces system energy consumption while reducing equipment failure risk. Attached Figure Description

[0076] Figure 1 This is a flowchart of the intelligent monitoring data fusion method for integrated utility tunnels based on multi-source data, as described in this invention. Detailed Implementation

[0077] The following detailed description of the intelligent monitoring data fusion method for integrated utility tunnels based on multi-source data, in conjunction with the accompanying drawings and specific embodiments, further illustrates the present invention.

[0078] Example:

[0079] This invention relates to intelligent monitoring and control scenarios applicable to urban integrated pipe corridors, underground tunnels, and similar multi-device, multi-source data environments. The various weight parameters include W... z b z β1, β2, γ j w1, w2, etc., can be obtained through training with historical operating data or set based on engineering experience, and can be dynamically adjusted according to actual operating conditions.

[0080] I. The following equipment shall be installed within the integrated utility tunnel:

[0081] Installed within the utility tunnel:

[0082] Temperature and humidity sensor (data collected every 10 seconds);

[0083] Current sensor (data collected every 10 seconds);

[0084] Video camera (25 frames per second);

[0085] The data at a specific moment is as follows:

[0086] Temperature: 30℃;

[0087] Humidity: 80%;

[0088] Current: 5A;

[0089] 2. Align all types of data according to timestamps and perform standardization processing:

[0090] set up:

[0091] Mean μ = 25, Standard deviation σ s =5; then the temperature normalization result is:

[0092]

[0093] III. Data Fusion Based on GCN and LSTM

[0094] (1) GCN output, let the graph convolution output be:

[0095]

[0096] (2) The LSTM output, after time series modeling, is obtained as follows:

[0097]

[0098] IV. Failure Rate Calculation and Prediction

[0099] (1) Failure rate calculation

[0100] Let the statistical time be T = 1 hour, and the number of recorded failures be N = 2, then:

[0101]

[0102] (2) Feature splicing

[0103]

[0104] (3) Fault prediction

[0105] set up:

[0106] but:

[0107]

[0108] The fault risk value is 0.92.

[0109] V. Video Anomaly Detection

[0110] (1) Target detection

[0111] Using CenterNet, the detection results were: water accumulation: confidence level 0.7; personnel: confidence level 0.5.

[0112] (2) Abnormal scoring

[0113] Let the weights be: water accumulation γ1 = 0.6, personnel γ2 = 0.4;

[0114] but:

[0115]

[0116] VI. State Fusion

[0117] Let the weights be β1 = 0.6 and β2 = 0.4, then:

[0118]

[0119] The current status is 0.8 (higher risk).

[0120] VII. (1) Objective Function

[0121]

[0122] (2) Comparison of control strategies

[0123] Based on the operating status of the equipment in the integrated utility tunnel, the control variables are set as follows:

[0124]

[0125] Where u1∈{0,1}: represents the fan switch (1 for on, 0 for off);

[0126] u2∈{0,1}: represents the drain pump switch; by enumerating all possible control combinations, a set of candidate control strategies is obtained: unreasonable strategies (such as shutting down all equipment under high-risk conditions) are eliminated, and a set of effective control strategies is obtained.

[0127]

[0128] (3) Substitute each of the candidate control strategies into the objective function for calculation, and select the control strategy with the smallest objective function value as the optimal control variable. The optimal control strategy is: turn on the drainage pump and turn off the fan.

[0129] Comparative experiment

[0130] Method 1: Based solely on sensor data (no video);

[0131] Method 2: The method of the present invention (fusion video + failure rate);

[0132] Data from the same utility tunnel over 24 hours: Actual number of failures: 10;

[0133] Experimental results:

[0134] Fault identification:

[0135]

[0136] Energy consumption comparison:

[0137]

[0138] The method of this invention improves the accuracy of fault identification by about 20% and reduces energy consumption by about 25%, indicating that the method has a better effect in intelligent monitoring of integrated utility tunnels.

[0139] This invention's method can be deployed in server- or embedded monitoring systems and implemented through software. By combining graph convolutional networks (GCNNs) with long short-term memory (LSTM) networks to jointly model multi-source data from integrated utility tunnels, it fully leverages the advantages of both in spatial and temporal feature extraction. On one hand, GCNNs construct the topology of devices and sensors, introducing the spatial relationships between nodes into the feature extraction process, thus modeling the correlation information between multiple devices and overcoming the information loss problem caused by the isolated processing of various data types in traditional methods. On the other hand, LTM networks utilize their gating mechanism to model time-series data, effectively capturing the long-term and short-term changes in equipment operating status, thereby improving the ability to identify abnormal trends and potential faults. Furthermore, the output of the GCNN is used as the input to the LTM network, achieving the synergistic fusion of spatial and temporal features, enabling the model to simultaneously depict the correlation relationships between devices in the integrated utility tunnel and their dynamic evolution over time. In addition, the neighborhood aggregation mechanism of GCNNs can smooth noisy data to a certain extent, while the gating structure of LTM networks can suppress the impact of short-term fluctuations on the results, thereby improving the robustness of the system under complex operating conditions. In summary, this invention effectively improves the accuracy and stability of integrated utility tunnel operation status assessment through the combined application of GCN and LSTM.

Claims

1. A data fusion method for intelligent monitoring of integrated utility tunnels based on multi-source data, comprising the following steps: S1. Multi-source data acquisition and preprocessing: Collect multi-source data from the integrated utility tunnel, including environmental data, equipment operation data, and video surveillance data. Perform time alignment and normalization processing on the multi-source data, specifically as follows: The original data sequence x i is normalized as follows: ； Where μ is the mean, σ s Standard deviation; S2. Multi-source data fusion based on GCN and LSTM: Construct a topology graph G=(V,E) of the equipment and sensors in the integrated utility tunnel, where node V represents equipment or sensor and edge E represents spatial or functional relationships. Construct an adjacency matrix A and perform normalization processing. ； Where D is the degree matrix; Graph convolutional networks are used to extract spatial features from multi-source data: ； Among them, H (l) For the features of layer l, W (l) Let ϕ(⋅) be the weight matrix, and let ϕ(⋅) be the nonlinear activation function. After multi-layer graph convolution calculation, the final layer output feature H is obtained. (L) The final layer output feature H (L) Organizing the data according to the time series, we obtain the input feature vector at time t: x t =H t (L)； Among them, H t (L) This represents the node features of the graph convolutional network at time t; The time series feature x t Inputting data into a Long Short-Term Memory (LSTM) network for temporal modeling: ； Where, x t Let h represent the input feature vector at time t, where h is the output of the graph convolutional network. t−1 c represents the hidden state at the previous time step. t−1 f represents the cell state at the previous moment. t i t o t Let c represent the gate control vectors for the forget gate, input gate, and output gate, respectively. t W represents the cell state at the current moment. f W i W o W c These represent the weight matrices of the corresponding gate structures, b f b i b o b c Let represent the bias vectors of the corresponding gate structures, and ϕ(⋅) represent the activation function. tanh(⋅) denotes the hyperbolic tangent function, and [ , ] denotes the vector concatenation operation; According to the output gate o t With cell state c t Calculate the hidden state h at the current time step. t : ； The hidden state h t As a fusion feature output; S3. Equipment Failure Rate Modeling and Feature Extraction: Calculate the failure rate of the device within the time interval T: ； Where N is the number of times the fault occurred; fusing feature h t The feature vector z is formed by concatenating the feature vector with the failure rate λ. t : ； The fault prediction value y is obtained by mapping using a fully connected layer: ； Among them, W z The weight matrix representing the fully connected layer has dimensions equal to or greater than the input feature vector z. t Matching, b z The bias vector represents the bias of the fully connected layer. The weight matrix and bias vector are obtained by training on historical operating data. S4. Video analysis based on CenterNet and MIAUC: The surveillance video is divided into multiple time segments. Keyframes are extracted from each segment and input into CenterNet for target detection, resulting in a heatmap of the target center point. ； Where I is the input image, f det For detection model; Construct a loss function based on multi-instance AUC optimization: ； Where n represents the number of samples involved in the loss calculation, and AUC i Let be the area under the curve for the i-th sample, and α be the weighting coefficient. This represents the target detection loss function, used to measure the error between the detection result and the ground truth label; The video anomaly score v is obtained by weighted summation of the confidence scores of each target in the detection results. t : ； Where m represents the number of targets to be detected, p j γ represents the detection confidence of the j-th target. j This represents the anomaly weight coefficient for the corresponding target; The anomaly score v t This is the result of video anomaly detection; S5. Data Fusion and Status Assessment: The fault prediction results obtained in step S3 are fused with the video analysis results obtained in step S4: ； Where β1 and β2 are weighting coefficients, and y represents the fault prediction value obtained in step S3; The integrated utility tunnel operation status evaluation index s was obtained. t； S6. Multi-objective optimization control: Construct an optimization function targeting equipment failure rate and energy consumption: ； Where λ is the failure rate and E is the system energy consumption; The control variable is represented as: ； Among them, u i Let u represent the control variable of the i-th device, where u is the set of feasible control variables. The objective function is solved by optimization: ； Obtain the optimal control variable u ∗ .

2. The integrated utility tunnel intelligent monitoring data fusion method based on multi-source data as described in claim 1, characterized in that: In step S1, missing data is filled in using linear interpolation: 。 3. The integrated utility tunnel intelligent monitoring data fusion method based on multi-source data as described in claim 1, characterized in that: In step S4, a threshold judgment is performed on the video anomaly score. ； Where θ is the anomaly detection threshold, 1 is abnormal and 0 is normal.

4. The integrated utility tunnel intelligent monitoring data fusion method based on multi-source data as described in claim 1, characterized in that: In step S5, y and vt are normalized before fusion.

5. The integrated utility tunnel intelligent monitoring data fusion method based on multi-source data as described in claim 1, characterized in that: The optimization process in step S6 is implemented using traversal search or gradient descent methods.

Images