Tunnel environment monitoring method and system based on multi-source data fusion

Abstract

The embodiment of the invention relates to the technical field of tunnel environment monitoring, and particularly discloses a tunnel environment monitoring method and system based on multi-source data fusion. According to the embodiment of the invention, multi-source monitoring is carried out on the target tunnel, and data synchronization and preprocessing are carried out on the multi-source monitoring data; performing spatial-temporal feature extraction on the multi-source standard data, and performing unified feature mapping; performing multi-source fusion on the multi-modal feature sequence; performing environment classification identification on the fused feature data; and constructing an information visualization platform, and monitoring and displaying the abnormal event data and the plurality of environment identification data. Multi-source monitoring, data synchronization and pre-processing, spatial-temporal feature extraction, multi-source fusion and environment classification identification, construction of an information visualization platform for monitoring display, data fusion processing of multi-source monitoring and comprehensive perception of multi-source data can be performed, so that linkage coupling analysis can be performed on various data, and multi-source data can be acquired. The abnormity identification capability and the response capability are improved, and the false alarm rate is effectively reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of tunnel environment monitoring, and in particular relates to a tunnel environment monitoring method and system based on multi-source data fusion. Background Art

[0002] Tunnel environmental monitoring is a comprehensive monitoring process that uses a variety of sensing technologies and data collection methods to perceive, collect, analyze and evaluate the environmental parameters inside and around the tunnel in real time or periodically. Its purpose is to fully understand the environmental status during tunnel operation, ensure driving safety and infrastructure stability. It is widely used in different types of underground projects such as highway tunnels, railway tunnels, subway tunnels and mine tunnels, and is a core component for realizing intelligent tunnel management and maintenance.

[0003] In existing technologies, tunnel environmental monitoring usually relies on an independent single data source, lacks comprehensive perception of multi-source data, and often adopts a decentralized management approach. It is unable to perform unified multi-source fusion processing and unable to conduct linkage coupling analysis of multiple data. It has problems such as weak anomaly recognition ability, high false alarm rate and slow response. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a tunnel environment monitoring method and system based on multi-source data fusion, aiming to solve the problems raised in the background technology.

[0005] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions: A tunnel environment monitoring method based on multi-source data fusion, the method specifically comprising the following steps: Performing multi-source monitoring on the target tunnel to obtain multi-source monitoring data, and performing data synchronization and preprocessing on the multi-source monitoring data to obtain multi-source standard data; Extracting spatiotemporal features from the multi-source standard data and performing unified feature mapping to generate a multimodal feature sequence; Performing multi-source fusion on the multimodal feature sequence to generate fused feature data; Performing environmental classification and identification on the fused feature data, recording multiple environmental identification data, performing real-time anomaly identification, and recording abnormal event data; An information visualization platform is constructed, in which the abnormal event data and the plurality of environmental identification data are monitored and displayed.

[0006] A tunnel environment monitoring system based on multi-source data fusion is applied to the above-mentioned tunnel environment monitoring method based on multi-source data fusion. The system includes a multi-source monitoring processing unit, a spatiotemporal feature extraction unit, a feature multi-source fusion unit, an environment classification and identification unit, and a visual monitoring display unit, wherein: A multi-source monitoring processing unit, configured to perform multi-source monitoring on a target tunnel, obtain multi-source monitoring data, and synchronize and pre-process the multi-source monitoring data to obtain multi-source standard data; A spatiotemporal feature extraction unit, configured to extract spatiotemporal features from the multi-source standard data, perform unified feature mapping, and generate a multimodal feature sequence; A feature multi-source fusion unit, configured to perform multi-source fusion on the multimodal feature sequence to generate fused feature data; An environment classification and identification unit, configured to perform environment classification and identification on the fused feature data, record a plurality of environment identification data, perform real-time anomaly identification, and record abnormal event data; The visualization monitoring and display unit is used to build an information visualization platform, in which the abnormal event data and the plurality of environmental identification data are monitored and displayed.

[0007] Compared with the prior art, the present invention has the following beneficial effects: The embodiment of the present invention performs multi-source monitoring on the target tunnel, synchronizes and preprocesses the multi-source monitoring data, extracts spatiotemporal features from multi-source standard data, and performs unified feature mapping, performs multi-source fusion on multimodal feature sequences, classifies and identifies the environment of the fused feature data, and constructs an information visualization platform to monitor and display abnormal event data and multiple environmental identification data. The system is capable of performing multi-source monitoring, data synchronization and preprocessing, followed by spatiotemporal feature extraction, multi-source fusion, and environmental classification and identification, and constructing an information visualization platform for monitoring and display. It is capable of performing data fusion processing for multi-source monitoring, achieving comprehensive perception of multi-source data, and thus performing linkage and coupling analysis on multiple data, improving abnormality identification and response capabilities, and effectively reducing false alarm rates. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention.

[0009] Figure 1 A flow chart of a method provided by an embodiment of the present invention is shown.

[0010] Figure 2 The application architecture diagram of the system provided by the embodiment of the present invention is shown. DETAILED DESCRIPTION

[0011] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0012] It is understandable that in the existing technology, tunnel environment monitoring usually relies on an independent single data source, lacks comprehensive perception of multi-source data, and often adopts a decentralized management approach. It is unable to perform unified multi-source fusion processing and unable to perform linkage coupling analysis of multiple data. It has problems such as weak anomaly recognition ability, high false alarm rate and slow response.

[0013] To address the above-mentioned issues, the embodiments of the present invention perform multi-source monitoring on the target tunnel to obtain multi-source monitoring data, synchronize and preprocess the multi-source monitoring data, and obtain multi-source standard data. Spatiotemporal feature extraction and unified feature mapping are performed on the multi-source standard data to generate a multimodal feature sequence. Multi-source fusion is performed on the multimodal feature sequence to generate fused feature data. Environmental classification and identification are performed on the fused feature data to record multiple environmental identification data. Real-time anomaly identification is performed to record abnormal event data. An information visualization platform is constructed to monitor and display abnormal event data and multiple environmental identification data. The system is capable of performing multi-source monitoring, data synchronization and preprocessing, followed by spatiotemporal feature extraction, multi-source fusion, and environmental classification identification. The system is capable of performing data fusion processing on multi-source monitoring, achieving comprehensive perception of multi-source data, and thus performing linked coupled analysis of multiple data, improving anomaly identification and response capabilities, and effectively reducing false alarm rates.

[0014] Figure 1 A flow chart of a method provided by an embodiment of the present invention is shown.

[0015] Specifically, the tunnel environment monitoring method based on multi-source data fusion includes the following steps: Step S101 : performing multi-source monitoring on a target tunnel to obtain multi-source monitoring data, and performing data synchronization and preprocessing on the multi-source monitoring data to obtain multi-source standard data.

[0016] In an embodiment of the present invention, multi-source monitoring such as gas monitoring, temperature and humidity monitoring, video monitoring, airflow monitoring, vibration monitoring, and traffic flow monitoring is performed on the target tunnel to obtain multi-source monitoring data including gas monitoring data, temperature and humidity monitoring data, video monitoring data, airflow monitoring data, vibration monitoring data, and traffic flow monitoring data. The multi-source monitoring data are then aligned uniformly using NTP time to obtain multi-source aligned data. Thereafter, the multi-source aligned data are subjected to noise filtering (using low-pass filtering, wavelet denoising, and other methods to eliminate high-frequency interference), redundancy elimination (cross-validation of multi-source data to eliminate faulty data sources), and missing value filling (using linear interpolation, multivariate regression interpolation, KNN filling, etc.) to obtain multi-source complete data. The multi-source complete data are then standardized to unify the data units and coordinate references to obtain multi-source standard data.

[0017] Specifically, in a preferred embodiment of the present invention, performing multi-source monitoring on the target tunnel, obtaining multi-source monitoring data, and performing data synchronization and preprocessing on the multi-source monitoring data, obtaining multi-source standard data specifically includes the following steps: Conduct multi-source monitoring on the target tunnel and obtain multi-source monitoring data; Aligning the multi-source monitoring data uniformly using NTP time to obtain multi-source aligned data; Performing noise filtering, redundancy elimination, and missing value filling processing on the multi-source aligned data to obtain multi-source complete data; The multi-source complete data is standardized to obtain multi-source standard data.

[0018] Furthermore, the tunnel environment monitoring method based on multi-source data fusion further includes the following steps: Step S102 : extracting spatiotemporal features from the multi-source standard data, and performing unified feature mapping to generate a multimodal feature sequence.

[0019] In an embodiment of the present invention, multi-source feature data is obtained by performing time series features (maximum value, mean, standard deviation, and rate of change within a sliding window), frequency domain features (using FFT to extract periodic changes and discover regular pollution sources), spatial features (combining the distribution locations of multi-source monitoring points to construct a spatial thermal matrix to reflect the distribution of pollution sources), image features (using CNN to extract parameters such as smoke shape, density, and motion trajectory), and traffic feature extraction (congestion index, emission intensity model, instantaneous total traffic volume, etc.) on multi-source standard data. The multi-source feature data is then subjected to feature vectorization processing to obtain multi-source vector data. Afterwards, the multi-source vector data is subjected to feature normalization processing to eliminate dimensionality effects to obtain normalized vector data, and the normalized vector data is feature labeled to generate a multimodal feature sequence.

[0020] Specifically, in a preferred embodiment of the present invention, the step of extracting spatiotemporal features from the multi-source standard data and performing unified feature mapping to generate a multimodal feature sequence specifically includes the following steps: Extracting time series features, frequency domain features, spatial features, image features, and traffic features from the multi-source standard data to obtain multi-source feature data; Performing feature vectorization processing on the multi-source feature data to obtain multi-source vector data; Performing feature normalization processing on the multi-source vector data to eliminate dimension effects and obtain normalized vector data; Feature annotation is performed on the normalized vector data to generate a multimodal feature sequence.

[0021] Furthermore, the tunnel environment monitoring method based on multi-source data fusion further includes the following steps: Step S103: performing multi-source fusion on the multimodal feature sequence to generate fused feature data.

[0022] In an embodiment of the present invention, by loading a multi-source fusion model of a preset deep neural network (which can be a multi-source fusion model constructed by a multimodal Transformer or LSTM fusion network), the multimodal feature sequence is imported into the multi-source fusion model, and multi-source fusion processing is performed. After completing the multi-source fusion processing of the model, the fusion feature data is exported.

[0023] Specifically, in a preferred embodiment of the present invention, the multi-source fusion of the multimodal feature sequence to generate fused feature data specifically includes the following steps: Load the preset multi-source fusion model; Importing the multimodal feature sequence into a multi-source fusion model to perform multi-source fusion processing; Export fused feature data.

[0024] In a preferred embodiment of the present invention, the step of importing the multimodal feature sequence into the multi-source fusion model and performing multi-source fusion processing specifically includes the following steps: The multimodal feature sequence is subjected to separable convolution operation on each modal feature channel to obtain three sets of dimensionally aligned feature matrices; Calculate the average norm of the multimodal feature sequence in each time window, and dynamically generate the dimension correction coefficient through the multi-layer perceptron to obtain the dynamic dimension scaling factor of each time slice; Perform matrix dot product operations on the three sets of dimensionally aligned feature matrices and the dynamic dimension scaling factor of each time slice according to the modality type to obtain the attention parameter matrix of the correlation relationship between the modalities; Use depth-wise separable convolution to perform spatial filtering on multimodal features to obtain feature slices of local correlation; Multiply the feature slice of local correlation with the feature matrix aligned with three sets of dimensions to obtain the query vector, key vector and value vector; The attention parameter matrix of the inter-modal correlation relationship is used to calculate the scaled dot product of the query vector and the key vector, and then Softmax is applied to generate the attention distribution to obtain the spatial correlation weight map; Perform multi-head attention weighted summation on the spatial association weight map and the value vector to obtain a set of spatially enhanced features; The multimodal feature sequence is downsampled by a one-dimensional convolution kernel to obtain a compressed time series feature tensor; The compressed time series feature tensor is batch normalized and gated to obtain standardized time series features; The standardized time series features are input into the bidirectional LSTM network for recursive processing to obtain a time evolution feature vector sequence; The spatial enhancement feature set and the time evolution feature vector sequence are adaptively fused in sequence to obtain a preliminary fused feature tensor; the preliminary fused feature tensor is subjected to nonlinear feature optimization to obtain enhanced fused features.

[0025] In this embodiment of the present invention, separable convolution is used to generate three sets of dimensionally aligned feature matrices. This allows the weight matrix of each modality to be trained independently, learning its unique representational pattern in the tunnel environment. For example, gas sensor concentration fluctuations and smoke diffusion trajectories in the video are modeled using different matrices. The dimensional scaling factor of the attention calculation is dynamically adjusted based on the real-time mean of the eigenvector norm and a correction factor generated by a multi-layer perceptron (MLP). To account for the difference in the numerical ranges of temperature and humidity data (in percentages) and traffic flow (vehicles per minute), subtle changes in carbon monoxide concentration at the beginning of a fire require different scaling than fluctuations during normal periods.

[0026] To ensure optimal results, a time synchronization compensation mechanism was introduced when generating the attention parameter matrix. This addresses video frame processing delays and aligns the real-time data stream from the gas sensor through timestamp interpolation, eliminating timing misalignment caused by transmission delays in multi-source data. Finally, a Transformer-LSTM dual-stream architecture was used for spatial and temporal enhancement, followed by fusion of the temporal and spatial streams to achieve complementary spatial and temporal enhancement of the data.

[0027] In the Transformer flow, separable convolution is used to extract local spatial features (such as the concentration gradient between adjacent monitoring points), and then a cross-point attention weight matrix is ​​constructed to quantify the mutual influence of monitoring data at different locations. A multi-head attention mechanism is then used to hierarchically aggregate global spatial information. The output results can locate the core area of ​​the pollution source.

[0028] In the LSTM stream, the temporal evolution model is built based on LSTM. One-dimensional convolution compresses the temporal dimension, and a bidirectional LSTM is used to capture forward and backward causal relationships (such as the temperature rise that precedes a fire). Noise interference is filtered through a gating mechanism, and finally, the temporal and spatial streams are fused to achieve complementary and enhanced feature extraction.

[0029] In a preferred embodiment of the present invention, the adaptive feature fusion of the spatial enhancement feature set and the time evolution feature vector sequence to obtain a preliminary fused feature tensor specifically includes the following steps: Perform global average pooling on the spatial enhancement feature set to obtain the spatial significance score of each modality; The spatial significance scores of each modality are input into the compressed excitation network to generate initial weights, and the distribution sharpness of the initial weights is adjusted by the temperature coefficient to obtain the modal dynamic fusion coefficient. The time dimension variance of the time evolution feature vector sequence is calculated and used as a compensation factor to obtain a time series dynamic adjustment coefficient; Perform channel dimension interpolation alignment on the spatial enhancement feature set and the time evolution feature vector sequence to obtain the features to be fused with consistent dimensions; The features to be fused with the same dimension are weightedly superimposed according to the modal dynamic fusion coefficient and the temporal dynamic adjustment coefficient to obtain the preliminary fusion feature tensor.

[0030] In this embodiment of the present invention, global average pooling is used to extract the spatial significance of each modal feature. Combined with time-series variance analysis, dynamic weights are generated to reflect the health status of the monitoring equipment. When the variance of a sensor's data increases abnormally (e.g., temperature and humidity sensor drift), its weight is automatically reduced to below a threshold. By achieving spatial-temporal complementarity through a dual-stream fusion architecture, the generated features can capture the chain reaction of, for example, a sudden increase in traffic volume, an increase in exhaust gas concentration, and a decrease in visibility.

[0031] In a preferred embodiment of the present invention, the nonlinear feature optimization of the preliminary fused feature tensor to obtain the enhanced fused feature specifically includes the following steps: The channels of the preliminary fused feature tensor are grouped by sensor type to obtain cross-modal channels; Perform element-wise multiplication of the cross-modal channels to generate cross-terms and calculate the Pearson correlation coefficient between the cross-terms and the preliminary fused feature tensor; Eliminate the redundancy of the initial fusion feature tensor according to the size of the Pearson correlation coefficient to obtain the de-redundant fusion feature; The de-redundant fusion features are segmented into feature channels according to time steps to obtain several paths containing continuous channels; Several paths containing continuous channels are randomly shielded in a probabilistic manner, and adjacent channels of the shielded paths are compensated by linear interpolation to obtain an anti-overfitting feature that retains the nonlinear relationship of key channels. The anti-overfitting feature is used to construct the interaction terms of all feature pairs, and the bivalent cross calculation is performed to obtain the gradient amplitude of all interaction terms; Sort the gradient amplitudes of all interaction items in descending order and retain the first few to obtain the filtered interaction items. Then, concatenate the filtered interaction items with the anti-overfitting features to obtain the interaction enhancement features. Perform modal alignment on the interaction enhancement features according to timestamps to obtain aligned multimodal features; The aligned multimodal features within the same spatiotemporal unit are element-wise multiplied, and then exponential amplification is applied to the product result to obtain the enhanced fusion feature.

[0032] In this embodiment of the present invention, by randomly masking some feature channels with a probability p, the model is forced to learn correlations outside of redundant paths. A channel interpolation algorithm is used to effectively address the data fusion challenge of sensors with different sampling rates. Dynamic pruning and compression are performed by calculating the gradient amplitude of feature channels to achieve lightweighting. Finally, cross-modal cross-enhancement is performed to generate enhanced fused features.

[0033] Furthermore, the tunnel environment monitoring method based on multi-source data fusion further includes the following steps: Step S104: performing environmental classification and identification on the fused feature data, recording a plurality of environmental identification data, performing real-time anomaly identification, and recording abnormal event data.

[0034] In an embodiment of the present invention, environmental pollution is identified on the fused feature data to obtain a pollution identification state, fire risk is identified on the fused feature data to obtain risk identification data, and traffic anomaly is identified on the fused feature data to obtain traffic identification data. The pollution identification state, risk identification data, and traffic identification data constitute a plurality of environmental identification data. Then, a preset combined identification algorithm (composed of an isolation forest algorithm, autoencoder + reconstruction error analysis, ARIMA + Prophet prediction residual, and image / video stream anomaly detection) is used to perform real-time anomaly identification on the fused feature data and record abnormal event data.

[0035] Specifically, in a preferred embodiment of the present invention, the fusion feature data is subjected to environmental classification and identification, multiple environmental identification data are recorded, and real-time abnormality identification and recording of abnormal event data specifically include the following steps: Performing environmental pollution identification on the fused feature data to obtain a pollution identification state; Performing fire risk identification on the fused feature data to obtain risk identification data; Performing traffic anomaly recognition on the fused feature data to obtain traffic recognition data; A preset combined recognition algorithm is used to perform real-time anomaly recognition on the fused feature data and record abnormal event data.

[0036] In a preferred embodiment of the present invention, the use of a preset combined recognition algorithm to perform real-time anomaly recognition on the fused feature data specifically includes the following steps: From the current moment t, trace back 24 time slices, extract the feature subset corresponding to each recognition algorithm, perform variance calculation, and obtain the variance value of each recognition algorithm; The variance value of each recognition algorithm is subjected to exponential decay processing to obtain the normalized variance evaluation value of each recognition algorithm; The normalized variance evaluation values ​​of each recognition algorithm are processed in turn through weighting, exponential operation and Softmax function to obtain the real-time dynamic weight coefficient of each recognition algorithm; The fused feature data is input into the pre-trained isolation forest model, the average path length of the fused feature data in the decision tree is calculated, and the anomaly probability value is generated according to the preset threshold to obtain the anomaly score based on the tree structure; Input the fused feature data into the pre-trained autoencoder network for data reconstruction to obtain reconstructed data; Calculate the Euclidean distance between the fused feature data and the reconstructed data, and convert the Euclidean distance between the fused feature data and the reconstructed data into anomaly probability through the Sigmoid function to obtain the quantitative score of the reconstruction error; The time series data of the fused feature data is input into the ARIMA and Prophet models for prediction to obtain the first predicted value and the second predicted value at the current moment; The first predicted value and the second predicted value at the current moment are multiplied and compared with the measured value, and the positive deviation part is logarithmically transformed to obtain the nonlinear residual abnormality score; Input the video stream data portion of the fused feature data into the pre-trained GAN model to generate the predicted image of the current frame; Extract the depth features of the real frame and the predicted image, calculate the cosine similarity of the depth features of the real frame and the predicted image, and take the complement to obtain the visual anomaly probability score; The tree-based anomaly score, reconstruction error quantization score, nonlinear residual anomaly score, and visual anomaly probability score are weighted and summed using the real-time dynamic weight coefficients of each recognition algorithm. The weighted summation result is adaptively normalized to obtain a comprehensive anomaly score. Calculate the moving average of historical data of the same period at the current moment, take the average plus 3 times the standard deviation as the dynamic threshold, compare the comprehensive anomaly score with the dynamic threshold, generate a binary judgment result, and use the binary judgment result as the abnormal event trigger mark.

[0037] In an embodiment of the present invention, different algorithms are used to process the optimal data type for multi-source heterogeneous data such as gas, video, and vibration in tunnel monitoring. When a certain type of data is abnormal, other modules provide verification basis. For example, if smoke is detected in the video but the gas is normal, the autoencoder is started to check the sensor status. If the vibration is abnormal but the traffic volume is zero, the isolation forest is triggered to analyze the geological parameters, thereby improving fault tolerance.

[0038] Furthermore, the tunnel environment monitoring method based on multi-source data fusion further includes the following steps: Step S105: constructing an information visualization platform, and monitoring and displaying the abnormal event data and the plurality of environmental identification data on the information visualization platform.

[0039] In an embodiment of the present invention, an information visualization platform is constructed that is provided with a real-time heat map, a video linkage display window, an indicator trend chart, and a multi-source interactive comparison chart, and display-related data is extracted from abnormal event data and multiple environmental identification data. Then, the display-related data is visualized in the information visualization platform, and corresponding abnormal monitoring alarms are performed on the abnormal event data.

[0040] Specifically, in a preferred embodiment of the present invention, the construction of an information visualization platform, in which the abnormal event data and the plurality of environmental identification data are monitored and displayed, specifically comprises the following steps: Build an information visualization platform, which is equipped with real-time heat maps, video linkage display windows, indicator trend charts and multi-source interactive comparison charts; Extracting presentation-related data from the abnormal event data and the plurality of environmental identification data; In the information visualization platform, display-related data is visualized; Perform corresponding abnormal monitoring and alarm on the abnormal event data.

[0041] Further, Figure 2 The application architecture diagram of the system provided by the embodiment of the present invention is shown.

[0042] Among them, in another preferred embodiment provided by the present invention, a tunnel environment monitoring system based on multi-source data fusion is applied to the above-mentioned tunnel environment monitoring method based on multi-source data fusion, comprising: The multi-source monitoring processing unit 101 is configured to perform multi-source monitoring on a target tunnel, obtain multi-source monitoring data, and perform data synchronization and preprocessing on the multi-source monitoring data to obtain multi-source standard data.

[0043] In an embodiment of the present invention, the multi-source monitoring processing unit 101 performs multi-source monitoring on the target tunnel, including gas monitoring, temperature and humidity monitoring, video monitoring, airflow monitoring, vibration monitoring, and traffic flow monitoring, to obtain multi-source monitoring data including gas monitoring data, temperature and humidity monitoring data, video monitoring data, airflow monitoring data, vibration monitoring data, and traffic flow monitoring data. The multi-source monitoring data is then aligned uniformly using NTP time to obtain multi-source aligned data. Thereafter, the multi-source aligned data is subjected to noise filtering (using low-pass filtering, wavelet denoising, and other methods to eliminate high-frequency interference), redundancy elimination (multi-source data cross-validation to eliminate faulty data sources), and missing value filling (using linear interpolation, multivariate regression interpolation, KNN filling, etc.) to obtain multi-source complete data. The multi-source complete data is then standardized to unify the data units and coordinate references to obtain multi-source standard data.

[0044] The spatiotemporal feature extraction unit 102 is configured to extract spatiotemporal features from the multi-source standard data, perform unified feature mapping, and generate a multimodal feature sequence.

[0045] In an embodiment of the present invention, the spatiotemporal feature extraction unit 102 obtains multi-source feature data by performing time series features (maximum value, mean, standard deviation, and rate of change within a sliding window), frequency domain features (using FFT to extract periodic changes and discover regular pollution sources), spatial features (combining the distribution locations of multi-source monitoring points to construct a spatial thermal matrix to reflect the distribution of pollution sources), image features (using CNN to extract parameters such as smoke shape, density, and motion trajectory), and traffic feature extraction (congestion index, emission intensity model, instantaneous total traffic volume, etc.) on multi-source standard data, and then performs feature vectorization processing on the multi-source feature data to obtain multi-source vector data. Thereafter, the multi-source vector data is feature normalized to eliminate the dimensionality effect to obtain normalized vector data, and the normalized vector data is feature labeled to generate a multimodal feature sequence.

[0046] The feature multi-source fusion unit 103 is configured to perform multi-source fusion on the multimodal feature sequence to generate fused feature data.

[0047] In an embodiment of the present invention, the feature multi-source fusion unit 103 loads a preset multi-source fusion model of a deep neural network (which can be a multi-source fusion model constructed by a multi-modal Transformer or LSTM fusion network), imports the multi-modal feature sequence into the multi-source fusion model, performs multi-source fusion processing, and exports the fusion feature data after completing the multi-source fusion processing of the model.

[0048] The environment classification and identification unit 104 is used to perform environment classification and identification on the fused feature data, record multiple environment identification data, perform real-time anomaly identification, and record abnormal event data.

[0049] In an embodiment of the present invention, the environmental classification and identification unit 104 performs environmental pollution identification on the fused feature data to obtain a pollution identification state, performs fire risk identification on the fused feature data to obtain risk identification data, and performs traffic anomaly identification on the fused feature data to obtain traffic identification data. The pollution identification state, risk identification data, and traffic identification data constitute a plurality of environmental identification data. A preset combined identification algorithm (composed of an isolation forest algorithm, autoencoder + reconstruction error analysis, ARIMA + Prophet prediction residual, and image / video stream anomaly detection) is then used to perform real-time anomaly identification on the fused feature data and record abnormal event data.

[0050] The visualization monitoring and display unit 105 is used to build an information visualization platform, in which the abnormal event data and the plurality of environmental identification data are monitored and displayed.

[0051] In an embodiment of the present invention, the visual monitoring and display unit 105 constructs an information visualization platform provided with a real-time heat map, a video linkage display window, an indicator trend chart and a multi-source interactive comparison chart, and extracts display-related data from abnormal event data and multiple environmental identification data, and then visualizes the display-related data in the information visualization platform, and performs corresponding abnormal monitoring alarms on the abnormal event data.

[0052] It should be understood that, although the various steps in the flow chart of each embodiment of the present invention are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in order, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0053] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When executed, the program can include the processes of the above-described method embodiments. Any reference to memory, storage, database, or other media used in the various embodiments provided herein may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0054] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0055] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

[0056] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A tunnel environment monitoring method based on multi-source data fusion, characterized in that: The method specifically comprises the following steps: Performing multi-source monitoring on the target tunnel to obtain multi-source monitoring data, and performing data synchronization and preprocessing on the multi-source monitoring data to obtain multi-source standard data; Extracting spatiotemporal features from the multi-source standard data and performing unified feature mapping to generate a multimodal feature sequence; Performing multi-source fusion on the multimodal feature sequence to generate fused feature data; Performing environmental classification and identification on the fused feature data, recording multiple environmental identification data, performing real-time anomaly identification, and recording abnormal event data; Building an information visualization platform, in which the abnormal event data and the plurality of environmental identification data are monitored and displayed; The multi-source monitoring of the target tunnel, obtaining the multi-source monitoring data, synchronizing and preprocessing the multi-source monitoring data, and obtaining the multi-source standard data specifically include the following steps: Conduct multi-source monitoring on the target tunnel and obtain multi-source monitoring data; Aligning the multi-source monitoring data uniformly using NTP time to obtain multi-source aligned data; Performing noise filtering, redundancy elimination, and missing value filling processing on the multi-source aligned data to obtain multi-source complete data; performing standardization processing on the multi-source complete data to obtain multi-source standard data; The multi-source monitoring of the target tunnel includes: gas monitoring, temperature and humidity monitoring, video monitoring, airflow monitoring, vibration monitoring and traffic flow monitoring; the multi-source monitoring data includes: gas monitoring data, temperature and humidity monitoring data, video monitoring data, airflow monitoring data, vibration monitoring data and traffic flow monitoring data.

2. The tunnel environment monitoring method based on multi-source data fusion according to claim 1 is characterized in that: The step of extracting spatiotemporal features from the multi-source standard data and performing unified feature mapping to generate a multimodal feature sequence specifically includes the following steps: Extracting time series features, frequency domain features, spatial features, image features, and traffic features from the multi-source standard data to obtain multi-source feature data; Performing feature vectorization processing on the multi-source feature data to obtain multi-source vector data; Performing feature normalization processing on the multi-source vector data to eliminate dimension effects and obtain normalized vector data; Feature annotation is performed on the normalized vector data to generate a multimodal feature sequence.

3. The tunnel environment monitoring method based on multi-source data fusion according to claim 1 is characterized in that: The multi-source fusion of the multimodal feature sequence to generate fused feature data specifically includes the following steps: Load the preset multi-source fusion model; Importing the multimodal feature sequence into a multi-source fusion model to perform multi-source fusion processing; Export fused feature data.

4. The tunnel environment monitoring method based on multi-source data fusion according to claim 3 is characterized in that: The step of importing the multimodal feature sequence into the multi-source fusion model and performing multi-source fusion processing specifically includes the following steps: The multimodal feature sequence is subjected to separable convolution operation on each modal feature channel to obtain three sets of dimensionally aligned feature matrices; Calculate the average norm of the multimodal feature sequence in each time window, and dynamically generate the dimension correction coefficient through the multi-layer perceptron to obtain the dynamic dimension scaling factor of each time slice; Perform matrix dot product operations on the three sets of dimensionally aligned feature matrices and the dynamic dimension scaling factor of each time slice according to the modality type to obtain the attention parameter matrix of the inter-modal correlation relationship; use depthwise separable convolution to perform spatial filtering extraction on multimodal features to obtain feature slices of local correlation; The feature slice of local correlation is multiplied by the feature matrix aligned with three sets of dimensions to obtain the query vector, key vector and value vector; the scaled dot product of the query vector and key vector is calculated using the attention parameter matrix of the inter-modal association relationship, and then Softmax is applied to generate the attention distribution to obtain the spatial association weight map; The spatial correlation weight map and the value vector are subjected to multi-head attention weighted summation to obtain a spatial enhancement feature set; the multimodal feature sequence is downsampled by a one-dimensional convolution kernel to obtain a compressed time series feature tensor; The compressed time series feature tensor is batch normalized and gated to obtain standardized time series features. The standardized time series features are input into a bidirectional LSTM network for recursive processing to obtain a time evolution feature vector sequence. The spatial enhancement feature set and the time evolution feature vector sequence are adaptively fused in sequence to obtain a preliminary fused feature tensor; the preliminary fused feature tensor is subjected to nonlinear feature optimization to obtain enhanced fused features.

5. The tunnel environment monitoring method based on multi-source data fusion according to claim 4 is characterized in that: The adaptive feature fusion of the spatial enhancement feature set and the time evolution feature vector sequence in sequence to obtain a preliminary fused feature tensor specifically includes the following steps: Perform global average pooling on the spatial enhancement feature set to obtain the spatial significance score of each modality; The spatial significance scores of each modality are input into the compressed excitation network to generate initial weights, and the distribution sharpness of the initial weights is adjusted by the temperature coefficient to obtain the modal dynamic fusion coefficient. The time dimension variance of the time evolution feature vector sequence is calculated and used as a compensation factor to obtain a time series dynamic adjustment coefficient; Perform channel dimension interpolation alignment on the spatial enhancement feature set and the time evolution feature vector sequence to obtain the features to be fused with consistent dimensions; The features to be fused with the same dimension are weightedly superimposed according to the modal dynamic fusion coefficient and the temporal dynamic adjustment coefficient to obtain the preliminary fusion feature tensor.

6. The tunnel environment monitoring method based on multi-source data fusion according to claim 5 is characterized in that: The nonlinear feature optimization of the preliminary fusion feature tensor to obtain the enhanced fusion feature specifically includes the following steps: The channels of the preliminary fused feature tensor are grouped by sensor type to obtain cross-modal channels; the cross-modal channels are element-wise multiplied to generate cross-terms, and the Pearson correlation coefficient between the cross-terms and the preliminary fused feature tensor is calculated; The redundancy of the preliminary fusion feature tensor is eliminated according to the size of the Pearson correlation coefficient to obtain the de-redundant fusion feature; the de-redundant fusion feature is segmented into feature channels according to time steps to obtain several paths containing continuous channels; the several paths containing continuous channels are randomly shielded in a probabilistic manner, and the adjacent channels of the shielded paths are compensated by linear interpolation to obtain an anti-overfitting feature that retains the nonlinear relationship of the key channels; The anti-overfitting feature is used to construct the interaction terms of all feature pairs, and the bivalent cross calculation is performed to obtain the gradient amplitude of all interaction terms; Sort the gradient amplitudes of all interaction items in descending order and retain the first few to obtain the filtered interaction items. Then, concatenate the filtered interaction items with the anti-overfitting features to obtain the interaction enhancement features. Perform modal alignment on the interaction enhancement features according to timestamps to obtain aligned multimodal features; The aligned multimodal features within the same spatiotemporal unit are element-wise multiplied, and then exponential amplification is applied to the product result to obtain the enhanced fusion feature.

7. The tunnel environment monitoring method based on multi-source data fusion according to claim 6 is characterized in that: The step of performing environmental classification and identification on the fused feature data, recording multiple environmental identification data, performing real-time anomaly identification, and recording abnormal event data specifically includes the following steps: Performing environmental pollution identification on the fused feature data to obtain a pollution identification state; Performing fire risk identification on the fused feature data to obtain risk identification data; Performing traffic anomaly recognition on the fused feature data to obtain traffic recognition data; A preset combined recognition algorithm is used to perform real-time anomaly recognition on the fused feature data and record abnormal event data.

8. The tunnel environment monitoring method based on multi-source data fusion according to claim 7 is characterized in that: The use of a preset combined recognition algorithm to perform real-time anomaly recognition on the fused feature data specifically includes the following steps: From the current moment t, trace back 24 time slices, extract the feature subset corresponding to each recognition algorithm, perform variance calculation, and obtain the variance value of each recognition algorithm; perform exponential decay processing on the variance value of each recognition algorithm to obtain the normalized variance evaluation value of each recognition algorithm; The normalized variance evaluation values ​​of each recognition algorithm are processed in turn through weighting, exponential operation and Softmax function to obtain the real-time dynamic weight coefficient of each recognition algorithm; The fused feature data is input into the pre-trained isolation forest model, the average path length of the fused feature data in the decision tree is calculated, and the anomaly probability value is generated according to the preset threshold to obtain the anomaly score based on the tree structure; The fused feature data is input into the pre-trained autoencoder network for data reconstruction to obtain the reconstructed data; the Euclidean distance between the fused feature data and the reconstructed data is calculated, and the Euclidean distance between the fused feature data and the reconstructed data is converted into an abnormality probability through the Sigmoid function to obtain the reconstruction error quantitative score; The time series data of the fused feature data is input into the ARIMA and Prophet models for prediction to obtain the first predicted value and the second predicted value at the current moment; the first predicted value and the second predicted value at the current moment are multiplied and compared with the measured value, and the positive deviation part is logarithmically transformed to obtain the nonlinear residual abnormality score; The video stream data portion of the fused feature data is input into the pre-trained GAN model to generate a predicted image of the current frame; the depth features of the real frame and the predicted image are extracted, the cosine similarity of the depth features of the real frame and the predicted image is calculated and the complement is taken to obtain the visual anomaly probability score; The tree-based anomaly score, reconstruction error quantization score, nonlinear residual anomaly score, and visual anomaly probability score are weighted and summed using the real-time dynamic weight coefficients of each recognition algorithm. The weighted summation result is adaptively normalized to obtain a comprehensive anomaly score. Calculate the moving average of historical data of the same period at the current moment, take the average plus 3 times the standard deviation as the dynamic threshold, compare the comprehensive anomaly score with the dynamic threshold, generate a binary judgment result, and use the binary judgment result as the abnormal event trigger mark.

9. The tunnel environment monitoring method based on multi-source data fusion according to claim 8, characterized in that: The construction of the information visualization platform, in which the abnormal event data and the plurality of environmental identification data are monitored and displayed, specifically comprises the following steps: Build an information visualization platform, which is equipped with real-time heat maps, video linkage display windows, indicator trend charts and multi-source interactive comparison charts; Extracting presentation-related data from the abnormal event data and the plurality of environmental identification data; In the information visualization platform, display-related data is visualized; Perform corresponding abnormal monitoring and alarm on the abnormal event data.

10. A tunnel environment monitoring system based on multi-source data fusion, wherein the system is applied to the tunnel environment monitoring method based on multi-source data fusion according to any one of claims 1 to 9, characterized in that: The system includes a multi-source monitoring processing unit, a spatiotemporal feature extraction unit, a feature multi-source fusion unit, an environment classification and recognition unit, and a visual monitoring display unit, wherein: A multi-source monitoring processing unit, configured to perform multi-source monitoring on a target tunnel, obtain multi-source monitoring data, and synchronize and pre-process the multi-source monitoring data to obtain multi-source standard data; A spatiotemporal feature extraction unit, configured to extract spatiotemporal features from the multi-source standard data, perform unified feature mapping, and generate a multimodal feature sequence; A feature multi-source fusion unit, configured to perform multi-source fusion on the multimodal feature sequence to generate fused feature data; An environment classification and identification unit, configured to perform environment classification and identification on the fused feature data, record a plurality of environment identification data, perform real-time anomaly identification, and record abnormal event data; The visualization monitoring and display unit is used to build an information visualization platform, in which the abnormal event data and the plurality of environmental identification data are monitored and displayed.

Images