Subway tunnel water leakage monitoring method and system based on multi-modal data fusion

Abstract

The invention belongs to the technical field of safety monitoring of subway tunnels, and particularly relates to a subway tunnel water leakage monitoring method and system based on multi-modal data fusion, and the method comprises the steps: obtaining multi-source sensor time sequence data in a subway tunnel, carrying out the feature extraction, and extracting feature parameters related to water leakage; carrying out fusion analysis on the characteristic parameters through a data fusion algorithm, and evaluating the position and severity of water leakage, including extracting time-space correlation characteristics of data by using a dynamic time warping method; after feature extraction is performed on the data set and the space-time correlation features by using a convolutional neural network and a recurrent neural network, feature fusion is performed; and performing anomaly detection and classification on the fused high-level features by using a neural network method to obtain the position and severity of the leakage water. The advanced data fusion technology and the feature extraction algorithm are adopted, various collected data can be analyzed in real time, abnormal conditions can be found in time, and early warning is carried out.

Description

Technical Field

[0001] The present invention belongs to the technical field of safety monitoring of subway tunnels, and in particular relates to a subway tunnel water leakage monitoring method and system based on multimodal data fusion. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid advancement of urbanization, subways, as an important part of modern urban public transportation, are also expanding their tunnel networks. The safe operation of subway tunnels is crucial to ensuring the smooth flow of urban traffic and the safety of passengers. However, with the increasing aging of subway systems and the complex and changeable geological conditions, tunnel water leakage has become a major challenge affecting the safe operation of subways.

[0004] Water leakage may not only cause structural corrosion and increase maintenance costs, but may also lead to more serious safety accidents such as electrical equipment failure and tunnel collapse. Currently, traditional subway tunnel water leakage monitoring mainly relies on visual inspection and point-based monitoring systems such as closed-circuit television (CCTV) monitoring and basic moisture detectors.

[0005] These methods have several disadvantages: manual visual inspection relies on the experience and visual judgment of inspectors, is easily affected by subjective factors, and is difficult to achieve all-weather, all-round monitoring. Point monitoring technologies, such as humidity sensors, can only provide data at specific points and cannot fully cover the entire tunnel system. They also have slow response times and limited data information. Although closed-circuit television systems can provide visual monitoring, image quality and judgment efficiency are limited under complex environmental conditions, such as low light or high humidity environments. Summary of the invention

[0006] In order to overcome the above-mentioned deficiencies of the prior art, the present invention provides a subway tunnel water leakage monitoring method and system based on multimodal data fusion.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0008] A first aspect of the present invention provides a subway tunnel water leakage monitoring method based on multimodal data fusion, comprising:

[0009] Obtain time series data from multi-source sensors in subway tunnels;

[0010] Perform feature extraction on multi-source sensor time series data to extract characteristic parameters related to water leakage;

[0011] The extracted characteristic parameters are fused and analyzed through data fusion algorithm to evaluate the location and severity of water leakage, including:

[0012] The extracted feature parameters are initially fused using the standardization algorithm and principal component analysis method; the feature set after initial fusion is screened using the recursive feature elimination method to obtain the optimal feature subset;

[0013] Combined with the spatiotemporal information of subway tunnels, the dynamic time warping method is used to perform spatiotemporal correlation analysis on the optimal feature subset to obtain spatiotemporal correlation features.

[0014] Convolutional neural networks and recurrent neural networks are used to extract features of multi-source sensor time series data and spatiotemporal correlation features, and the extracted features are fused to obtain fused high-level features;

[0015] The neural network method is used to detect and classify anomalies on the fused high-level features to obtain the location and severity of the water leakage.

[0016] A second aspect of the present invention provides a subway tunnel water leakage monitoring system based on multimodal data fusion, comprising:

[0017] Multimodal data acquisition subsystem and data fusion analysis subsystem;

[0018] The multimodal data acquisition subsystem is used to obtain multi-source sensor time series data in the subway tunnel;

[0019] The data fusion analysis subsystem includes a feature extraction module and a fusion analysis module;

[0020] The feature extraction module is used to extract features from multi-source sensor time series data to extract feature parameters related to water leakage;

[0021] The fusion analysis module is used to perform fusion analysis on the extracted characteristic parameters through a data fusion algorithm to evaluate the location and severity of water leakage, including:

[0022] The extracted feature parameters are initially fused using the standardization algorithm and principal component analysis method; the feature set after initial fusion is screened using the recursive feature elimination method to obtain the optimal feature subset;

[0023] Combined with the spatiotemporal information of subway tunnels, the dynamic time warping method is used to perform spatiotemporal correlation analysis on the optimal feature subset to obtain spatiotemporal correlation features.

[0024] Convolutional neural networks and recurrent neural networks are used to extract features of multi-source sensor time series data and spatiotemporal correlation features, and the extracted features are fused to obtain fused high-level features;

[0025] The neural network method is used to detect and classify anomalies on the fused high-level features to obtain the location and severity of the water leakage.

[0026] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of a subway tunnel water leakage monitoring method based on multimodal data fusion as described in the first aspect of the present invention.

[0027] The fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps in the method for monitoring water leakage in a subway tunnel based on multimodal data fusion as described in the first aspect of the present invention are implemented.

[0028] One or more of the above technical solutions have the following beneficial effects:

[0029] 1. The present invention can capture and analyze environmental changes in subway tunnels in real time from multiple dimensions by integrating sound, image, temperature, humidity, pressure and chemical sensors, greatly improving the efficiency and accuracy of monitoring. Compared with the traditional method that relies on manual visual inspection and point monitoring, the present invention can identify and locate the specific location and extent of water leakage faster and more accurately.

[0030] 2. The present invention uses a variety of data fusion technologies and feature extraction algorithms to analyze various collected data in real time, detect abnormal situations in time and automatically trigger the early warning system. This real-time monitoring and response mechanism can quickly notify operation managers to take measures and effectively prevent accidents.

[0031] 3. The data analysis report generated by the present invention can provide accurate decision support for the subway operation management center or relevant departments, and help formulate more effective repair and maintenance strategies. Through the real-time feedback mechanism of the intelligent early warning system, managers can quickly understand the repair effect of leakage diseases and adjust maintenance strategies in time.

[0032] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0034] Figure 1 This is a flow chart of a subway tunnel water leakage monitoring method based on multimodal data fusion according to the first embodiment.

[0035] Figure 2 This is a schematic diagram of the layout of the multimodal data acquisition system of the first embodiment.

[0036] Figure 3 This is a data fusion analysis flow chart of the first embodiment.

[0037] Figure 4 This is a flow chart of the intelligent early warning of the first embodiment.

[0038] In the figure, 1. Multimodal data acquisition subsystem, 2. Data fusion analysis subsystem, 3. Intelligent early warning subsystem, 4. Sound sensor, 5. Image sensor, 6. Temperature sensor, 7. Humidity sensor, 8. Pressure sensor, 9. Chemical sensor. DETAILED DESCRIPTION

[0039] Embodiment 1

[0040] like Figure 1 As shown, this embodiment discloses a subway tunnel water leakage monitoring method based on multimodal data fusion, including:

[0041] Step 1: Obtain time series data from multi-source sensors in the subway tunnel;

[0042] Step 2: Extract features from the multi-source sensor time series data to extract characteristic parameters related to water leakage, including: the spectral characteristics of the sound, the water trace characteristics in the image, and the changing trends of temperature and humidity;

[0043] Step 3: Use data fusion algorithm to analyze the extracted characteristic parameters and evaluate the location and severity of water leakage;

[0044] Step 4: Conduct intelligent early warning and formulate repair and remediation measures based on the results of fusion analysis.

[0045] In step 1, multi-source sensor data in the subway tunnel is obtained, including: respectively obtaining data collected by the sound sensor 4, the image sensor 5, the temperature sensor 6, the humidity sensor 7, the pressure sensor 8 and the chemical sensor 9; Figure 2 As shown, the sound sensor 4 is installed on the top or wall of the tunnel to capture characteristic sounds such as water flow and dripping. These sensors analyze the characteristics of the sound by receiving sound wave signals and converting them into electrical signals.

[0046] The image sensor 5 is installed at key areas inside the tunnel, such as the wall or the roof, and uses optical or infrared imaging technology to capture images of the inside of the tunnel to identify signs of water seepage.

[0047] Temperature sensors 6 are distributed on the top, walls and / or ground of the tunnel to monitor temperature changes caused by leakage.

[0048] The humidity sensor 7 is arranged on the wall and / or the ground, and monitors the change of humidity by absorbing water vapor in the air and converting it into an electrical signal.

[0049] The pressure sensor 8 is installed in the area where the ground and / or underground water level is low to monitor the pressure change of the water body.

[0050] The chemical sensor 9 is installed on the wall and / or the ground to monitor the change of water quality through chemical reaction.

[0051] The above sensors work in coordination with each other. When a sensor detects abnormal data, other sensors are activated to collect data.

[0052] When the sound sensor detects a characteristic sound in the subway tunnel, it triggers the image sensor to start taking images of the inside of the subway tunnel to capture signs of water leakage associated with the sound, such as water marks and water stains;

[0053] When the temperature sensor detects a sudden change in the temperature inside the subway tunnel, it triggers the humidity sensor to detect the humidity and the pressure sensor to detect whether the pressure of the groundwater body has changed, so as to jointly determine whether a water leakage event has occurred;

[0054] When sound sensors, image sensors, temperature sensors, humidity sensors or pressure sensors detect signs of water leakage in the subway tunnel, chemical sensors are triggered to detect changes in water quality.

[0055] Coordinated work of sound sensors and image sensors: When the sound sensor detects characteristic sounds in the subway tunnel, such as flowing water or dripping water, it will trigger the image sensor to start taking images of the inside of the subway tunnel to capture signs of water leakage associated with the sound, such as water marks and water stains.

[0056] Coordination between temperature sensors, humidity sensors and pressure sensors: When the temperature sensor detects a sudden change in temperature inside the subway tunnel, it may indicate a cooling effect of water leakage. At this time, the humidity sensor will detect whether the humidity increases accordingly, and the pressure sensor will also monitor whether the pressure of the groundwater body changes, in order to jointly determine whether a water leakage event has occurred.

[0057] Coordination of chemical sensors with other sensors: When sound, image, temperature, humidity or pressure sensors detect signs of water leakage in the subway tunnel, the chemical sensor will be triggered to start detecting whether the water quality has changed, further confirming the existence of water leakage and whether the leakage has affected the water quality.

[0058] In the present invention, each sensor continuously collects data and monitors the data in real time. When a sensor detects abnormal data related to water leakage, such as the sound sensor captures the sound of water flow, the image sensor captures the image of water marks, the temperature sensor detects the temperature drop, etc., the system will mark the data as abnormal data and trigger subsequent processing.

[0059] Once a sensor detects abnormal data, the system will activate other sensors and call their data for fusion analysis. For example, when the sound sensor detects abnormal water flow, the system will activate the image sensor to capture images of the corresponding area and call the data of temperature, humidity, pressure and other sensors for comprehensive analysis.

[0060] In step 2, feature extraction of sensor data includes:

[0061] Step 201: For the sound sensor, use short-time Fourier transform (STFT) to convert the sound signal from the time domain to the frequency domain, and extract the spectral characteristics of the sound signal, including frequency components, amplitude, energy, etc.; by analyzing these spectral characteristics, the characteristic sounds generated by the leakage water (such as dripping water, flowing water, etc.) can be identified.

[0062] Step 202: For the image sensor, use a pre-trained convolutional neural network (such as ResNet, VGG, etc.) to process the image and extract feature maps in the image, including edges, textures, shapes, etc. By analyzing these feature maps, visual features such as water leakage traces and wet spots can be identified.

[0063] Step 203: Perform sliding window averaging processing on the time series data collected by the temperature sensor to extract trend characteristics of temperature changes, including sudden temperature changes and gradual changes. By analyzing the trend characteristics of temperature changes, abnormal temperature changes caused by water leakage can be identified.

[0064] Step 204: Perform ARIMA model analysis on the time series data collected by the humidity sensor to extract the time series characteristics of humidity changes, including sudden increases and decreases in humidity. By analyzing the time series characteristics of humidity changes, abnormal humidity changes caused by water leakage can be identified.

[0065] Step 205: Use wavelet transform to perform multi-scale decomposition on the data collected by the pressure sensor, and extract the change characteristics of the pressure signal at different scales, including instantaneous pressure changes, long-term pressure changes, etc.; by analyzing the pressure characteristics at different scales, abnormal conditions of groundwater pressure changes can be identified.

[0066] Step 206: Perform principal component analysis on the multi-dimensional water quality data (such as pH value, conductivity, dissolved oxygen concentration, etc.) collected by the chemical sensor to extract the main change components and feature vectors. By analyzing these main components, abnormal changes in water quality can be identified and the presence of water leakage can be further confirmed.

[0067] In step 3, the fusion analysis part comprehensively analyzes the feature parameters extracted by different sensors and comprehensively considers the relationship between various features; through the data fusion algorithm, the system can more accurately determine whether there is water leakage in the subway tunnel and evaluate the location and severity of the water leakage.

[0068] like Figure 3 As shown, the extracted feature parameters are fused and analyzed through the data fusion algorithm, including:

[0069] Step 301: Preliminary feature fusion.

[0070] Step 3011: feature standardization processing;

[0071] The characteristic parameters extracted by each sensor have different dimensions and ranges, so they need to be standardized. Min-Max normalization is used to convert different features to the same dimension and range.

[0072] Min-Max Normalization:

[0073]

[0074] In the formula, X is the original data, X min and X max are the minimum and maximum values ​​in the data set, respectively.

[0075] Step 3012: Preliminary feature fusion analysis.

[0076] A preliminary fusion analysis is performed on the standardized feature parameters, and the principal component analysis (PCA) method is used to extract the main components by calculating the feature covariance matrix and eigenvalue decomposition. The main purpose is to reduce noise and improve the representativeness of the features.

[0077] Step 3013: Feature selection.

[0078] The most representative features are selected from the initially fused feature set to reduce redundant information and computational complexity. The recursive feature elimination (RFE) method is used to recursively train the model and eliminate features with smaller weights to select the optimal feature subset.

[0079] Among them, the optimal feature subsets obtained based on the recursive feature elimination (RFE) method include:

[0080] (1) Using the initial feature set X = {x1, x2, ..., x n} and labels Y = {y1, y2, ..., y n}Train the support vector machine (SVM) model and obtain the model parameter β.

[0081]

[0082] (2) Calculate feature weight β i , for SVM, the feature weights are the coefficients of the model.

[0083] (3) Sort by the absolute value of the feature weight and select the feature x with the smallest weight j To be eliminated.

[0084] (4) Remove x from the feature set j , repeat the above steps until the preset number of features k is reached;

[0085] (5) The feature subset finally selected is the optimal feature set, which is used for subsequent analysis and fusion.

[0086] Step 302: Spatiotemporal correlation analysis.

[0087] The dynamic time warping (DTW) method is used to calculate the optimal matching path by nonlinearly aligning the time series data; combined with the spatiotemporal information of the subway tunnel, the spatiotemporal correlation analysis of the characteristic parameters is performed to obtain the spatiotemporal correlation features. Specifically, it includes:

[0088] (1) Collect the time series data X of each sensor i (t) and X j (t), where i and j represent different sensors and t represents the timestamp.

[0089] (2) Calculate the Euclidean distance matrix D between two time series. The matrix element D(i, j) represents the time series X. i (t) and X j (t) At time t i and t j The Euclidean distance between .

[0090] (3) Initialize the cumulative distance matrix C, where the matrix element C(i, j) represents the minimum cumulative distance from the starting point to (i, j).

[0091] Initialize the first element of the matrix C(0,0)=D(0,0)

[0092]

[0093] (4) Tracing back from the end point C(n, m) of the matrix C to the starting point C(0, 0), find the optimal alignment path Path = {(i k , j k )}.

[0094] (5) According to the optimal alignment path, the spatiotemporal correlation features are extracted, including the aligned timestamps and the corresponding feature values.

[0095] Step 303: high-level feature fusion;

[0096] Combined with the spatiotemporal information of the subway tunnel, the spatiotemporal correlation analysis of the characteristic parameters is carried out, and the deep learning network structure is used to integrate and analyze the spatiotemporal correlation features at a higher level; specifically, it includes:

[0097] Step 3031: Receive the time series data of each sensor and the spatiotemporal correlation features after DTW processing; use a method combining convolutional neural network (CNN) and recurrent neural network (RNN) to extract spatiotemporal features.

[0098] CNN layer: used to extract local features in time series data.

[0099] h (1) =ReLU(Conv1D(X,W (1) )+b (1) )

[0100] RNN layer: used to extract temporal features of time series data.

[0101] h (t) =LSTM(h (t-1) , X (t) , W (l) +b (l)

[0102] The features extracted by CNN and RNN are fused and the fully connected layer is used for feature combination.

[0103] h (f) =ReLU(W (f) [h (1) ,h (r) ]+b (f) )

[0104] Step 304: Anomaly detection and classification.

[0105] Step 3041: High-level feature anomaly detection and non-linear classification.

[0106] The neural network (NN) method is used to perform anomaly detection and nonlinear classification on the fused high-level features, including:

[0107] (1) High-level feature vector h extracted from multimodal data (f) As input; use the fully connected layer to process high-level features and extract deeper features.

[0108] z (1) =σ(W (1) h (f) +b (1) )

[0109] Among them, W (1) and b (1) are weight and bias parameters, σ is the activation function

[0110] (2) The extracted deep features are used to detect anomalies using an autoencoder, including:

[0111] The encoder part compresses the input features into a low-dimensional representation.

[0112] z=f encoder (z (1) )=σ(W enc z (1) +b enc )

[0113] The decoder part reconstructs the low-dimensional representation into high-dimensional features.

[0114]

[0115] Calculate the reconstruction error between the encoder and decoder It is used as the basis for anomaly detection, and samples with large reconstruction errors are considered anomalies.

[0116] (3) Use deep neural networks to perform nonlinear classification and output the water leakage risk level.

[0117] z (2) =σ(W (2) z (1) +b (2) )

[0118] The final output layer performs classification.

[0119] y=Softmax(W (o) z (2) +b (o) ) where z (2) is a deep feature, W (o) , W (2) and b (o) 、b (2)are the weight and bias parameters of the output layer, and the Softmax function is used to calculate the probability distribution of water leakage risk.

[0120] Step 3042: Determine the location and extent of the water leakage.

[0121] Using the comprehensive feature h after high-level feature fusion (c) And the classification result y, combined with the sensor position L, position inference is performed. Assume that the sensor position is L = {L1, L2, ..., L n}, and the conditional probability relationship between the sensor features is The Bayesian network model is:

[0122]

[0123] in, is the probability of the sensor feature calculated by the classification result y.

[0124] Based on the Bayesian network model, the formula for position inference is:

[0125]

[0126] That is, find the maximum conditional probability P(L|h (c) , y) position L.

[0127] Among them, by calculating the characteristics of each sensor and position L i The specific steps are as follows:

[0128] Calculate the abnormality level E of each sensor i :

[0129]

[0130] In the formula, is the eigenvector in the normal state.

[0131] Combined with the sensor location, calculate the association between the degree of anomaly and location:

[0132]

[0133] This weighted average method combines the abnormality degree and location of each sensor to obtain the specific location of the water leakage.

[0134] Combined with the comprehensive features h after high-level feature fusion (c) And the classification result y, use the Bayesian network to quantitatively evaluate the leakage degree S:

[0135]

[0136] in, For a given risk level y i In the case of The conditional probability of For a given sensor characteristic and risk level y i The conditional probability of water leakage degree S under the condition of .

[0137] Calculate the joint probability distribution P(S, h (c) , y), using the Bayesian formula:

[0138]

[0139] Marginalize the joint probability distribution and calculate the marginal probability distribution of the leakage level S:

[0140]

[0141] Among them, the denominator P(h (c) , y) is a normalization constant that can be calculated by marginalizing S:

[0142]

[0143] Choose the maximum posterior probability P(S|h (c) , y) Estimate the leakage level S:

[0144]

[0145] Step 305: Sensor label and image sensor assistance: The intelligent early warning system uses the label of the detection sensor and the image data provided by the image sensor to further confirm the location and extent of the water leakage disease and provide a reference for subsequent repair and remediation measures.

[0146] In step 4, intelligent early warning is carried out based on the results of fusion analysis, and repair and remediation measures are formulated, including:

[0147] The results of the fusion analysis are transmitted to the subway operation management center or relevant departments, and corresponding repair and remediation measures are formulated according to relevant regulatory requirements and emergency plans, including temporary isolation, emergency repairs and pipeline replacement;

[0148] During the implementation of repair and remediation measures, sensors continue to monitor changes in the location and extent of water leakage, as well as the effectiveness of repairs.

[0149] like Figure 4As shown in the figure, after the data is integrated and analyzed, the location and degree of water leakage are transmitted to the subway operation management center or relevant departments in real time. This information can be transmitted in real time through the network or other communication means so that timely response measures can be taken.

[0150] Formulate measures according to regulatory requirements: After receiving information on the location and extent of water leakage, the subway operation management center or relevant departments will formulate corresponding repair and remediation measures according to relevant regulatory requirements and emergency plans. These measures may include temporary isolation, emergency repairs, pipe replacement, etc.

[0151] Real-time monitoring and feedback: During the implementation of repair and remediation measures, the intelligent early warning system continues to monitor the changes in the location and extent of water leakage, as well as the repair effect. If necessary, the system can adjust the measures or take other countermeasures.

[0152] Embodiment 2

[0153] This embodiment discloses a subway tunnel water leakage monitoring system based on multimodal data fusion, comprising: a multimodal data acquisition subsystem 1, a data fusion analysis subsystem 2 and an intelligent early warning subsystem 3;

[0154] The multimodal data acquisition subsystem 1 is used to obtain multi-source sensor data in the subway tunnel;

[0155] The data fusion analysis subsystem 2 includes a feature extraction module and a fusion analysis module; the feature extraction module is used to extract features from multi-source sensor data and extract characteristic parameters related to water leakage, including: spectral features of sound, water trace features in images, and temperature and humidity change trends; the fusion analysis module is used to perform fusion analysis on the extracted characteristic parameters to evaluate the location and severity of water leakage;

[0156] The intelligent early warning subsystem 3 is used to determine the repair and remediation measures based on the results of the fusion analysis.

[0157] The multimodal data acquisition subsystem 1 includes: a sound sensor 4, an image sensor 5, a temperature sensor 6, a humidity sensor 7, a pressure sensor 8 and a chemical sensor 9;

[0158] Various sensors work in coordination with each other. When a sensor detects abnormal data, other sensors are activated to collect data.

[0159] The fusion analysis module includes: (1) Data preprocessing module, which first receives the raw data from the multimodal data acquisition subsystem, cleans, denoises and standardizes the data, and outputs the preprocessed data. (2) Feature conversion module, which extracts feature vectors representing different modal features from the preprocessed data and converts the data of each sensor into a feature form that is easy to analyze. (3) Feature dimension reduction module, which performs feature selection and dimension reduction on the extracted feature vectors, removes redundant information, and retains the most important feature subsets. (4) Spatiotemporal correlation module, which performs spatiotemporal correlation analysis on the selected feature subsets, and analyzes the spatiotemporal relationship of feature parameters in combination with the spatiotemporal information of the subway tunnel. (5) High-level feature vector fusion module, which performs high-level fusion of feature parameters of different modalities to generate a unified feature representation, comprehensively considering the feature information of each sensor. (6) Anomaly detection and classification module, which performs anomaly detection and classification on the fused high-level feature vectors, and generates a detailed water leakage disease report, including the location and degree of water leakage.

[0160] Intelligent early warning subsystem: intelligent early warning module, repair and remediation measures release module, and repair effect display module;

[0161] The intelligent early warning module is used to integrate the analysis results to determine the level of tunnel leakage and issue early warning information; the repair and remediation measures release module is used to formulate corresponding repair and remediation measures according to relevant specifications and emergency plans, including temporary isolation, emergency repairs and pipeline replacement;

[0162] The repair effect feedback module is used to process the data collected by sensors during the implementation of repair and remediation measures, monitor the changes in the location and degree of water leakage and the repair effect.

[0163] Embodiment 3

[0164] The purpose of this embodiment is to provide a computer-readable storage medium.

[0165] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in a method for monitoring water leakage in a subway tunnel based on multimodal data fusion as described in Example 1 of the present disclosure.

[0166] Embodiment 4

[0167] The purpose of this embodiment is to provide an electronic device.

[0168] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps in a method for monitoring water leakage in a subway tunnel based on multimodal data fusion as described in Example 1 of the present disclosure are implemented.

[0169] The steps involved in the apparatuses of the above embodiments 2, 3 and 4 correspond to the method embodiment 1, and the specific implementation methods can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0170] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0171] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A subway tunnel water leakage monitoring method based on multimodal data fusion, characterized in that: include: Obtain time series data from multi-source sensors in subway tunnels; Perform feature extraction on multi-source sensor time series data to extract characteristic parameters related to water leakage; The extracted characteristic parameters are fused and analyzed through data fusion algorithm to evaluate the location and severity of water leakage, including: The extracted feature parameters are initially fused using the standardization algorithm and principal component analysis method; the feature set after initial fusion is screened using the recursive feature elimination method to obtain the optimal feature subset; Combined with the spatiotemporal information of subway tunnels, the dynamic time warping method is used to perform spatiotemporal correlation analysis on the optimal feature subset to obtain spatiotemporal correlation features. Convolutional neural networks and recurrent neural networks are used to extract features of multi-source sensor time series data and spatiotemporal correlation features, and the extracted features are fused to obtain fused high-level features; The neural network method is used to detect and classify anomalies on the fused high-level features to obtain the location and severity of the water leakage.

2. A subway tunnel water leakage monitoring method based on multimodal data fusion as claimed in claim 1, characterized in that: The obtaining of multi-source sensor data in the subway tunnel includes: Respectively obtain data collected by sound sensors, image sensors, temperature sensors, humidity sensors, pressure sensors and chemical sensors; Various sensors work in coordination with each other. When a sensor detects abnormal data, other sensors are activated to collect data.

3. A subway tunnel water leakage monitoring method based on multimodal data fusion as claimed in claim 2, characterized in that: The sound sensor is installed on the top and / or wall of the tunnel to collect water flow and dripping sound data; The image sensor is installed in the inner area of ​​the tunnel and is used to capture image data of the inner part of the tunnel based on optical or infrared imaging technology; The temperature sensor is arranged on the top, wall and / or ground of the tunnel to collect temperature data on the surface of the object; The humidity sensor is arranged on the wall and / or the ground inside the tunnel, and is used to collect humidity data inside the tunnel; The pressure sensor is arranged in an area inside the tunnel where the ground and / or underground water level is low, and is used to collect water pressure data; The chemical sensor is arranged on the inner wall and / or the ground of the tunnel and is used to monitor the water quality change data through chemical reactions.

4. The method for monitoring water leakage in a subway tunnel based on multimodal data fusion according to claim 2, characterized in that: When a sensor detects abnormal data, other sensors are started to collect data, including: When the sound sensor detects a characteristic sound in the subway tunnel, it triggers the image sensor to start taking images of the inside of the subway tunnel to capture signs of water leakage associated with the sound; When the temperature sensor detects a sudden change in the temperature inside the subway tunnel, it triggers the humidity sensor to detect the humidity and the pressure sensor to detect whether the pressure of the groundwater body has changed, so as to jointly determine whether a water leakage event has occurred; When sound sensors, image sensors, temperature sensors, humidity sensors or pressure sensors detect signs of water leakage in the subway tunnel, chemical sensors are triggered to detect changes in water quality.

5. The method for monitoring water leakage in a subway tunnel based on multimodal data fusion according to claim 1, characterized in that: The characteristic parameters related to water leakage include: frequency spectrum characteristics of sound, water trace characteristics in the image, and variation trends of temperature and humidity.

6. The method for monitoring water leakage in a subway tunnel based on multimodal data fusion according to claim 1, characterized in that: Based on the results of fusion analysis, intelligent early warning and repair and remediation measures are formulated, including: The results of the fusion analysis are transmitted to the subway operation management center or relevant departments, and corresponding repair and remediation measures are formulated according to relevant regulatory requirements and emergency plans, including temporary isolation, emergency repairs and pipeline replacement; During the implementation of repair and remediation measures, sensors continue to monitor changes in the location and extent of water leakage, as well as the effectiveness of repairs.

7. The method for monitoring water leakage in a subway tunnel based on multimodal data fusion according to claim 1, characterized in that: The method of using a neural network to detect and classify anomalies on the fused high-level features includes: The fused high-level features are input into the autoencoder for anomaly detection, and the anomaly detection results are input into the deep neural network for nonlinear classification to obtain the water leakage risk level; The Bayesian network is used to quantitatively evaluate the degree of water leakage by combining the comprehensive features after high-level feature fusion and the water leakage risk level.

8. A subway tunnel water leakage monitoring system based on multimodal data fusion, characterized in that: include: Multimodal data acquisition subsystem and data fusion analysis subsystem; The multimodal data acquisition subsystem is used to obtain multi-source sensor time series data in the subway tunnel; The data fusion analysis subsystem includes a feature extraction module and a fusion analysis module; The feature extraction module is used to extract features from multi-source sensor time series data to extract feature parameters related to water leakage; The fusion analysis module is used to perform fusion analysis on the extracted characteristic parameters through a data fusion algorithm to evaluate the location and severity of water leakage, including: The extracted feature parameters are initially fused using the standardization algorithm and principal component analysis method; the feature set after initial fusion is screened using the recursive feature elimination method to obtain the optimal feature subset; Combined with the spatiotemporal information of subway tunnels, the dynamic time warping method is used to perform spatiotemporal correlation analysis on the optimal feature subset to obtain spatiotemporal correlation features. Convolutional neural networks and recurrent neural networks are used to extract features of multi-source sensor time series data and spatiotemporal correlation features, and the extracted features are fused to obtain fused high-level features; The neural network method is used to detect and classify anomalies on the fused high-level features to obtain the location and severity of the water leakage.

9. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps of a subway tunnel water leakage monitoring method based on multimodal data fusion as described in any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the subway tunnel water leakage monitoring method based on multimodal data fusion as described in any one of claims 1 to 7 are implemented.

Images