Tunnel Environmental Monitoring Method and System Based on Multi-Source Data Fusion

Abstract

This invention relates to the field of tunnel environmental monitoring technology, specifically disclosing a tunnel environmental monitoring method and system based on multi-source data fusion. This invention involves multi-source monitoring of the target tunnel, data synchronization and preprocessing of the multi-source monitoring data; spatiotemporal feature extraction and unified feature mapping of multi-source standard data; multi-source fusion of multi-modal feature sequences; environmental classification and identification of the fused feature data; and the construction of an information visualization platform to monitor and display abnormal event data and multiple environmental identification data. It enables multi-source monitoring, data synchronization and preprocessing, spatiotemporal feature extraction, multi-source fusion, and environmental classification and identification, and the construction of an information visualization platform for monitoring and display. It allows for multi-source monitoring data fusion processing, achieving comprehensive perception of multi-source data, thereby enabling linked and coupled analysis of various data, improving anomaly identification and response capabilities, and effectively reducing false alarm rates.

Description

Technical Field

[0001] This invention belongs to the field of tunnel environmental monitoring technology, and particularly relates to a tunnel environmental monitoring method and system based on multi-source data fusion. Background Technology

[0002] Tunnel environmental monitoring is a comprehensive monitoring process that uses various sensing technologies and data acquisition methods to perceive, collect, analyze, and evaluate environmental parameters inside and around the tunnel in real time or periodically. Its purpose is to fully understand the environmental status during tunnel operation, ensure traffic safety and the stability of infrastructure, and is widely used in different types of underground engineering such as highway tunnels, railway tunnels, subway tunnels and mine tunnels. It is a core component of realizing intelligent tunnel management and maintenance.

[0003] In existing technologies, tunnel environment monitoring usually relies on independent single data sources, lacks comprehensive perception of multi-source data, and often adopts a decentralized management approach, which cannot perform unified multi-source fusion processing, cannot perform linkage and coupling analysis of multiple data, and has problems such as weak anomaly identification ability, high false alarm rate and slow response. Summary of the Invention

[0004] The purpose of this invention is to provide a tunnel environment monitoring method and system based on multi-source data fusion, aiming to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0006] A tunnel environment monitoring method based on multi-source data fusion, the method specifically includes the following steps:

[0007] Multi-source monitoring is performed on the target tunnel to acquire multi-source monitoring data, and the multi-source monitoring data is synchronized and preprocessed to acquire multi-source standard data.

[0008] Spatiotemporal features are extracted from the multi-source standard data, and unified feature mapping is performed to generate multimodal feature sequences;

[0009] The multimodal feature sequences are fused from multiple sources to generate fused feature data;

[0010] The fused feature data is used for environmental classification and identification, multiple environmental identification data are recorded, and real-time anomaly identification is performed to record anomaly event data;

[0011] An information visualization platform is constructed to monitor and display the abnormal event data and multiple environmental identification data.

[0012] A tunnel environment monitoring system based on multi-source data fusion is applied to the aforementioned tunnel environment monitoring method based on multi-source data fusion. The system includes a multi-source monitoring processing unit, a spatiotemporal feature extraction unit, a feature multi-source fusion unit, an environmental classification and identification unit, and a visualization monitoring display unit, wherein:

[0013] The multi-source monitoring and processing unit is used to perform multi-source monitoring of the target tunnel, acquire multi-source monitoring data, and perform data synchronization and preprocessing on the multi-source monitoring data to acquire multi-source standard data.

[0014] The spatiotemporal feature extraction unit is used to extract spatiotemporal features from the multi-source standard data, perform unified feature mapping, and generate a multimodal feature sequence.

[0015] The feature multi-source fusion unit is used to perform multi-source fusion on the multimodal feature sequence to generate fused feature data;

[0016] An environmental classification and recognition unit is used to classify and recognize the environment based on the fused feature data, record multiple environmental recognition data, and perform real-time anomaly recognition and record abnormal event data.

[0017] The visualization monitoring and display unit is used to construct an information visualization platform, in which the abnormal event data and multiple environmental identification data are monitored and displayed.

[0018] Compared with the prior art, the beneficial effects of the present invention are:

[0019] This invention enables multi-source monitoring of target tunnels, data synchronization and preprocessing of multi-source monitoring data; spatiotemporal feature extraction and unified feature mapping of multi-source standard data; multi-source fusion of multimodal feature sequences; environmental classification and identification of fused feature data; and the construction of an information visualization platform to monitor and display abnormal event data and multiple environmental identification data. It allows for multi-source monitoring, data synchronization and preprocessing, spatiotemporal feature extraction, multi-source fusion, and environmental classification and identification, facilitating the construction of an information visualization platform for monitoring and display. This enables comprehensive perception of multi-source data, allowing for linked and coupled analysis of various data sources, improving anomaly identification and response capabilities, and effectively reducing false alarm rates. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention.

[0021] Figure 1 A flowchart of the method provided by an embodiment of the present invention is shown.

[0022] Figure 2 Shows the application architecture diagram of the system provided by the embodiments of the present invention. Detailed implementation manners

[0023] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0024] It can be understood that in the prior art, tunnel environment monitoring usually relies on independent single data sources, lacks comprehensive perception of multi-source data, and often adopts a decentralized management method, which cannot perform unified multi-source fusion processing, cannot perform linkage coupling analysis on multiple types of data, and has problems such as weak anomaly recognition ability, high false alarm rate and slow response.

[0025] To solve the above problems, the embodiments of the present invention perform multi-source monitoring on the target tunnel, obtain multi-source monitoring data, perform data synchronization and preprocessing on the multi-source monitoring data to obtain multi-source standard data; extract spatio-temporal features from the multi-source standard data, perform unified feature mapping, and generate multi-modal feature sequences; perform multi-source fusion on the multi-modal feature sequences to generate fusion feature data; perform environmental classification recognition on the fusion feature data, record multiple environmental recognition data, and perform real-time anomaly recognition to record anomaly event data; construct an information visualization platform, and in the information visualization platform, monitor and display the anomaly event data and multiple environmental recognition data. It can perform multi-source monitoring, data synchronization and preprocessing, then perform spatio-temporal feature extraction, multi-source fusion and environmental classification recognition, construct an information visualization platform for monitoring and display, can perform data fusion processing of multi-source monitoring, realize comprehensive perception of multi-source data, so as to perform linkage coupling analysis on multiple types of data, improve the anomaly recognition ability and response ability, and effectively reduce the false alarm rate.

[0026] Figure 1 Shows the flow chart of the method provided by the embodiments of the present invention.

[0027] Specifically, a tunnel environment monitoring method based on multi-source data fusion, the method specifically includes the following steps:

[0028] Step S101, perform multi-source monitoring on the target tunnel, obtain multi-source monitoring data, and perform data synchronization and preprocessing on the multi-source monitoring data to obtain multi-source standard data.

[0029] In this embodiment of the invention, multi-source monitoring is performed on the target tunnel, including gas monitoring, temperature and humidity monitoring, video monitoring, airflow monitoring, vibration monitoring, and traffic flow monitoring. This yields multi-source monitoring data, including gas monitoring data, temperature and humidity monitoring data, video monitoring data, airflow monitoring data, vibration monitoring data, and traffic flow monitoring data. The multi-source monitoring data is then aligned using NTP time to obtain multi-source aligned data. Subsequently, the multi-source aligned data undergoes noise filtering (using low-pass filtering, wavelet denoising, and other methods to eliminate high-frequency interference), redundancy removal (multi-source data cross-validation to remove faulty data sources), and missing value completion (using linear interpolation, multivariate regression interpolation, KNN completion, and other methods) to obtain complete multi-source data. Finally, the complete multi-source data is standardized to unify data units and coordinate references, resulting in multi-source standard data.

[0030] Specifically, in the preferred embodiment provided by the present invention, the step of performing multi-source monitoring of the target tunnel, acquiring multi-source monitoring data, and synchronizing and preprocessing the multi-source monitoring data to obtain multi-source standard data specifically includes the following steps:

[0031] Multi-source monitoring of the target tunnel was conducted to obtain multi-source monitoring data;

[0032] The multi-source monitoring data are uniformly aligned using NTP time to obtain multi-source aligned data;

[0033] The multi-source aligned data is subjected to noise filtering, redundancy removal, and missing value completion to obtain multi-source complete data;

[0034] The multi-source complete data is standardized to obtain multi-source standard data.

[0035] Furthermore, the tunnel environment monitoring method based on multi-source data fusion also includes the following steps:

[0036] Step S102: Spatiotemporal features are extracted from the multi-source standard data, and unified feature mapping is performed to generate a multimodal feature sequence.

[0037] In this embodiment of the invention, multi-source feature data is obtained by performing time-series features (maximum value, mean, standard deviation, and rate of change within a sliding window), frequency domain features (using FFT to extract periodic changes and discover regular pollution sources), spatial features (combining the location of multi-source monitoring points to construct a spatial thermodynamic matrix to reflect the distribution of pollution sources), image features (using CNN to extract parameters such as smoke shape, density, and movement trajectory), and traffic features (congestion index, emission intensity model, instantaneous traffic volume, etc.) on multi-source standard data. Then, feature vectorization processing is performed on the multi-source feature data to obtain multi-source vector data. After that, feature normalization processing is performed on the multi-source vector data to eliminate the influence of dimensions, resulting in normalized vector data. Finally, feature annotation is performed on the normalized vector data to generate a multimodal feature sequence.

[0038] Specifically, in the preferred embodiment provided by the present invention, the step of extracting spatiotemporal features from the multi-source standard data and performing unified feature mapping to generate a multimodal feature sequence specifically includes the following steps:

[0039] The multi-source standard data is subjected to time-series features, frequency domain features, spatial features, image features, and traffic features to obtain multi-source feature data;

[0040] The multi-source feature data is processed into feature vectorization to obtain multi-source vector data;

[0041] The multi-source vector data is subjected to feature normalization to eliminate the influence of dimensions and obtain normalized vector data.

[0042] The normalized vector data is labeled with features to generate a multimodal feature sequence.

[0043] Furthermore, the tunnel environment monitoring method based on multi-source data fusion also includes the following steps:

[0044] Step S103: Perform multi-source fusion on the multimodal feature sequence to generate fused feature data.

[0045] In this embodiment of the invention, a multi-source fusion model of a preset deep neural network (which may be a multi-source fusion model constructed by a multi-modal Transformer or LSTM fusion network) is loaded, and the multi-modal feature sequence is imported into the multi-source fusion model for multi-source fusion processing. After the multi-source fusion processing of the model is completed, the fused feature data is exported.

[0046] Specifically, in the preferred embodiment provided by the present invention, the step of multi-source fusion of the multimodal feature sequence to generate fused feature data specifically includes the following steps:

[0047] Load the preset multi-source fusion model;

[0048] The multimodal feature sequences are imported into a multi-source fusion model for multi-source fusion processing.

[0049] Export the fused feature data.

[0050] In a preferred embodiment of the present invention, the step of importing the multimodal feature sequence into a multi-source fusion model for multi-source fusion processing specifically includes the following steps:

[0051] The multimodal feature sequence is subjected to separable convolution operation on each modal feature channel to obtain three sets of dimension-aligned feature matrices;

[0052] Calculate the mean norm of the multimodal feature sequences within each time window, and use the mean norm of the multimodal feature sequences within each time window to dynamically generate dimension correction coefficients through a multilayer perceptron, thereby obtaining the dynamic dimension scaling factor for each time slice;

[0053] The attention parameter matrix for intermodal correlation is obtained by performing matrix dot product operation on the three sets of dimension-aligned feature matrices and the dynamic dimension scaling factor of each time slice according to modality type.

[0054] Depthwise separable convolution is used to perform spatial filtering extraction on multimodal features to obtain feature slices with local correlations;

[0055] Multiply the locally correlated feature slices with the feature matrices aligned to the three dimensions to obtain the query vector, key vector, and value vector;

[0056] The attention parameter matrix of intermodal association is used to calculate the scaled dot product of the query vector and the key vector, and then Softmax is applied to generate the attention distribution to obtain the spatial association weight map.

[0057] The spatial association weight graph and the value vector are subjected to multi-head attention weighted summation to obtain the spatial augmentation feature set;

[0058] The multimodal feature sequences are downsampled using a one-dimensional convolution kernel to obtain a compressed temporal feature tensor.

[0059] The compressed temporal feature tensors are batch normalized and gated to obtain standardized temporal features.

[0060] The standardized temporal features are input into a bidirectional LSTM network for recursive processing to obtain a sequence of time-evolved feature vectors.

[0061] The spatially enhanced feature set and the temporally evolved feature vector sequence are sequentially subjected to adaptive feature fusion to obtain a preliminary fused feature tensor; the preliminary fused feature tensor is then subjected to nonlinear feature optimization to obtain the enhanced fused features.

[0062] In this embodiment of the invention, three sets of dimension-aligned feature matrices are generated through separable convolution, allowing the weight matrix of each modality to be trained independently, learning its unique representational pattern in the tunnel environment. For example, the concentration fluctuations of gas sensors and the smoke diffusion trajectory in the video are modeled using different matrices. The dimensionality scaling factor for attention calculation is dynamically adjusted based on the real-time calculated mean norm of the feature vectors and the correction coefficients generated by the multilayer perceptron (MLP). This addresses the differences in numerical range between temperature and humidity data (percentage units) and traffic flow (vehicles / minute), and the need for different scaling ratios between the slight changes in carbon monoxide concentration at the initial stage of a fire and the fluctuations during normal periods.

[0063] To ensure better results, a time synchronization compensation mechanism was introduced when generating the attention parameter matrix. This mechanism addresses the video frame processing latency and aligns the real-time data stream from the gas sensor through timestamp interpolation, eliminating temporal misalignment caused by transmission delays in multi-source data. Finally, a Transformer-LSTM dual-stream architecture was employed for spatial and temporal enhancement, followed by fusion of the temporal and spatial streams to achieve complementary enhancement of the data in both spatial and temporal aspects.

[0064] In the Transformer stream, local spatial features (such as concentration gradients between adjacent monitoring points) are extracted through separable convolution, and then a cross-point attention weight matrix is ​​constructed to quantify the mutual influence of monitoring data at different locations. Finally, a multi-head attention mechanism is used to aggregate global spatial information in a hierarchical manner, and the output results can locate the core area of ​​the pollution source.

[0065] In the LSTM stream, temporal evolution modeling is performed based on LSTM. The temporal dimension is compressed by one-dimensional convolution, and then bidirectional LSTM is used to capture forward / backward causal relationships (such as temperature rise precursors before a fire). Noise interference is filtered out through a gating mechanism, and finally, the temporal and spatial streams are fused to achieve complementary and enhanced feature extraction.

[0066] In a preferred embodiment of the present invention, the step of adaptively fusing the spatially enhanced feature set and the temporally evolved feature vector sequence to obtain a preliminary fused feature tensor specifically includes the following steps:

[0067] The spatial augmentation feature set is subjected to global average pooling to obtain the spatial saliency score of each modality;

[0068] The spatial saliency scores of each modality are input into the compressed excitation network to generate initial weights, and the distribution sharpness of the initial weights is adjusted by the temperature coefficient to obtain the modality dynamic fusion coefficients.

[0069] The time dimension variance of the time evolution feature vector sequence is calculated, and the time dimension variance of the time evolution feature vector sequence is used as a compensation factor to obtain the time series dynamic adjustment coefficient.

[0070] The spatial enhancement feature set and the temporal evolution feature vector sequence are aligned by channel dimension interpolation to obtain features to be fused with consistent dimensions.

[0071] The features to be fused with consistent dimensions are weighted and superimposed according to the modal dynamic fusion coefficient and the temporal dynamic adjustment coefficient to obtain the preliminary fused feature tensor.

[0072] In this embodiment of the invention, the spatial significance of each modal feature is extracted by global average pooling, and combined with temporal variance analysis to generate dynamic weights reflecting the health status of the monitoring equipment. When the variance of a certain sensor data abnormally increases (such as temperature and humidity sensor drift), its weight is automatically reduced to below a threshold. By achieving spatial-temporal complementarity through a dual-stream fusion architecture, the generated features can capture chain effects such as "sudden increase in traffic flow → increase in exhaust gas concentration → decrease in visibility".

[0073] In a preferred embodiment of the present invention, the step of performing nonlinear feature optimization on the preliminary fused feature tensor to obtain the enhanced fused features specifically includes the following steps:

[0074] The channels of the preliminary fused feature tensor are grouped according to sensor type to obtain cross-modal channels;

[0075] Element-wise multiplication is performed across modal channels to generate cross terms, and the Pearson correlation coefficient between the cross terms and the preliminary fused feature tensor is calculated.

[0076] Redundancy is eliminated from the initial fusion feature tensor based on the magnitude of the Pearson correlation coefficient, resulting in redundancy-free fusion features.

[0077] The redundancy-free fused features are segmented into feature channels according to time steps to obtain several paths containing continuous channels.

[0078] Several paths containing continuous channels are randomly masked in a probabilistic manner, and linear interpolation compensation is performed on the adjacent channels of the masked paths to obtain anti-overfitting features that preserve the nonlinear relationship of key channels.

[0079] The interaction terms of all feature pairs are constructed using anti-overfitting features, and binary cross-validation is performed to obtain the gradient magnitude of all interaction terms.

[0080] Sort the gradient magnitudes of all interaction items in descending order and keep the top few to obtain the filtered interaction items. Then, concatenate the filtered interaction items with the anti-overfitting features to obtain the interaction enhancement features.

[0081] The interactive enhancement features are modally aligned according to timestamps to obtain aligned multimodal features;

[0082] The aligned multimodal features within the same spatiotemporal unit are multiplied element-wise, and then the product result is exponentially amplified to obtain the enhanced fused features.

[0083] In this embodiment of the invention, by randomly masking some feature channels with probability p, the model is forced to learn correlations outside of redundant paths. The channel interpolation algorithm used effectively solves the data fusion problem of sensors with different sampling rates. Dynamic pruning and compression are performed by calculating the gradient magnitude machine of the feature channels to achieve lightweighting. Finally, cross-modal enhancement is used to generate enhanced fused features.

[0084] Furthermore, the tunnel environment monitoring method based on multi-source data fusion also includes the following steps:

[0085] Step S104: Perform environmental classification and identification on the fused feature data, record multiple environmental identification data, and perform real-time anomaly identification to record abnormal event data.

[0086] In this embodiment of the invention, environmental pollution identification status is obtained by performing environmental pollution identification on the fused feature data, fire risk identification is obtained by performing fire risk identification on the fused feature data, and traffic anomaly identification is obtained by performing traffic anomaly identification on the fused feature data. The pollution identification status, risk identification data, and traffic identification data constitute multiple environmental identification data. Then, a preset combined identification algorithm (composed of the isolated forest algorithm, autoencoder + reconstruction error analysis, ARIMA + Prophet prediction residual, and image / video stream anomaly detection) is used to perform real-time anomaly identification on the fused feature data and record the anomaly event data.

[0087] Specifically, in the preferred embodiment provided by the present invention, the step of performing environmental classification and identification on the fused feature data, recording multiple environmental identification data, and performing real-time anomaly identification and recording anomaly event data specifically includes the following steps:

[0088] Environmental pollution identification is performed on the fused feature data to obtain the pollution identification status;

[0089] Fire risk identification is performed on the fused feature data to obtain risk identification data;

[0090] Traffic anomaly identification is performed on the fused feature data to obtain traffic identification data;

[0091] A preset combined recognition algorithm is used to perform real-time anomaly identification on the fused feature data and record the anomaly event data.

[0092] In a preferred embodiment of the present invention, the real-time anomaly identification of the fused feature data using a preset combined identification algorithm specifically includes the following steps:

[0093] Starting from the current time t, backtrack 24 time slices, extract the feature subset corresponding to each recognition algorithm, calculate the variance, and obtain the variance value of each recognition algorithm;

[0094] The variance value of each recognition algorithm is subjected to exponential decay processing to obtain the normalized variance evaluation value of each recognition algorithm.

[0095] The normalized variance evaluation values ​​of each recognition algorithm are processed sequentially by weighting, exponential operation and Softmax function to obtain the real-time dynamic weight coefficients of each recognition algorithm.

[0096] The fused feature data is input into a pre-trained isolated forest model, the average path length of the fused feature data in the decision tree is calculated, and anomaly probability values ​​are generated according to a preset threshold to obtain an anomaly score based on the tree structure.

[0097] The fused feature data is input into a pre-trained autoencoder network for data reconstruction to obtain reconstructed data;

[0098] Calculate the Euclidean distance between the fused feature data and the reconstructed data, and convert the Euclidean distance between the fused feature data and the reconstructed data into anomaly probability through the Sigmoid function to obtain the reconstruction error quantification score;

[0099] The time series data portion of the fused feature data is input into the ARIMA and Prophet models for prediction, resulting in the first and second predicted values ​​at the current time.

[0100] Multiply the first and second predicted values ​​at the current time and compare them with the measured values. Take the positive deviation part and perform logarithmic transformation to obtain the nonlinear residual abnormal score.

[0101] The video stream data portion with fused feature data is input into a pre-trained GAN model to generate the predicted image for the current frame;

[0102] Extract depth features from real frames and predicted images, calculate the cosine similarity between the depth features of real frames and predicted images and take the complement to obtain a visual anomaly probability score.

[0103] The anomaly score based on tree structure, reconstruction error quantification score, nonlinear residual anomaly score, and visual anomaly probability score are weighted and summed using the real-time dynamic weight coefficients of each recognition algorithm, and the weighted summation result is adaptively normalized to obtain a comprehensive anomaly score.

[0104] Calculate the moving average of historical data in the same period at the current time, take the average plus 3 times the standard deviation as the dynamic threshold, compare the comprehensive anomaly score with the dynamic threshold, generate a binary judgment result, and use the binary judgment result as the trigger mark for anomaly events.

[0105] In this embodiment of the invention, different algorithms are used to process the optimal data type for multi-source heterogeneous data such as gas, video, and vibration in tunnel monitoring. When a certain type of data is abnormal, other modules provide verification basis. For example, if smoke is detected in the video but the gas is normal, the self-encoder is started to check the sensor status. If the vibration is abnormal but the traffic flow is zero, the isolated forest analysis of geological parameters is triggered, thereby improving fault tolerance.

[0106] Furthermore, the tunnel environment monitoring method based on multi-source data fusion also includes the following steps:

[0107] Step S105: Construct an information visualization platform, in which the abnormal event data and multiple environmental identification data are monitored and displayed.

[0108] In this embodiment of the invention, an information visualization platform is constructed with real-time heatmaps, video-linked display windows, indicator trend charts, and multi-source interactive comparison charts. Relevant data is extracted and displayed from abnormal event data and multiple environmental identification data. The relevant data is then visualized and displayed on the information visualization platform, and corresponding abnormal event data is monitored and alarmed.

[0109] Specifically, in the preferred embodiment provided by the present invention, the construction of the information visualization platform, in which the abnormal event data and multiple environmental identification data are monitored and displayed, specifically includes the following steps:

[0110] An information visualization platform is constructed, which includes a real-time heat map, a video-linked display window, an indicator trend chart, and a multi-source interactive comparison chart.

[0111] Relevant data is extracted and displayed from the abnormal event data and multiple environmental identification data.

[0112] The information visualization platform displays relevant data in a visual format.

[0113] The abnormal event data is monitored and alarmed accordingly.

[0114] Furthermore, Figure 2 An application architecture diagram of the system provided in an embodiment of the present invention is shown.

[0115] In another preferred embodiment of the present invention, a tunnel environment monitoring system based on multi-source data fusion, wherein the system is applied to the aforementioned tunnel environment monitoring method based on multi-source data fusion, includes:

[0116] The multi-source monitoring and processing unit 101 is used to perform multi-source monitoring of the target tunnel, acquire multi-source monitoring data, and perform data synchronization and preprocessing on the multi-source monitoring data to acquire multi-source standard data.

[0117] In this embodiment of the invention, the multi-source monitoring processing unit 101 performs multi-source monitoring on the target tunnel, including gas monitoring, temperature and humidity monitoring, video monitoring, airflow monitoring, vibration monitoring, and traffic flow monitoring, to acquire multi-source monitoring data such as gas monitoring data, temperature and humidity monitoring data, video monitoring data, airflow monitoring data, vibration monitoring data, and traffic flow monitoring data. Then, the multi-source monitoring data is uniformly aligned using NTP time to obtain multi-source aligned data. Subsequently, the multi-source aligned data is processed by noise filtering (using low-pass filtering, wavelet denoising, and other methods to eliminate high-frequency interference), redundancy removal (multi-source data cross-validation to remove faulty data sources), and missing value completion (applying linear interpolation, multivariate regression interpolation, KNN completion, etc.) to obtain multi-source complete data. Finally, the multi-source complete data is standardized to unify data units and coordinate references to obtain multi-source standard data.

[0118] The spatiotemporal feature extraction unit 102 is used to extract spatiotemporal features from the multi-source standard data, perform unified feature mapping, and generate a multimodal feature sequence.

[0119] In this embodiment of the invention, the spatiotemporal feature extraction unit 102 extracts multi-source feature data by performing time-series features (maximum value, mean, standard deviation, and rate of change within a sliding window), frequency domain features (using FFT to extract periodic changes and discover regular pollution sources), spatial features (combining the location of multi-source monitoring points to construct a spatial thermal matrix to reflect the distribution of pollution sources), image features (using CNN to extract parameters such as smoke shape, density, and movement trajectory), and traffic features (congestion index, emission intensity model, instantaneous traffic volume, etc.) on the multi-source feature data. Then, the multi-source feature data is processed by feature vectorization to obtain multi-source vector data. After that, the multi-source vector data is processed by feature normalization to eliminate the influence of dimensions to obtain normalized vector data. The normalized vector data is then labeled with features to generate a multimodal feature sequence.

[0120] The feature multi-source fusion unit 103 is used to perform multi-source fusion on the multimodal feature sequence to generate fused feature data.

[0121] In this embodiment of the invention, the feature multi-source fusion unit 103 loads a preset deep neural network multi-source fusion model (which may be a multi-modal Transformer or LSTM fusion network-constructed multi-source fusion model), imports the multi-modal feature sequence into the multi-source fusion model, performs multi-source fusion processing, and after completing the multi-source fusion processing of the model, exports the fused feature data.

[0122] The environment classification and recognition unit 104 is used to classify and recognize the environment of the fused feature data, record multiple environment recognition data, and perform real-time anomaly recognition and record abnormal event data.

[0123] In this embodiment of the invention, the environmental classification and identification unit 104 performs environmental pollution identification on the fused feature data to obtain pollution identification status, performs fire risk identification on the fused feature data to obtain risk identification data, and performs traffic anomaly identification on the fused feature data to obtain traffic identification data. The pollution identification status, risk identification data, and traffic identification data constitute multiple environmental identification data. Then, a preset combined identification algorithm (composed of isolated forest algorithm, autoencoder + reconstruction error analysis, ARIMA + Prophet prediction residual and image / video stream anomaly detection combination) is used to perform real-time anomaly identification on the fused feature data and record abnormal event data.

[0124] The visualization monitoring and display unit 105 is used to construct an information visualization platform, in which the abnormal event data and multiple environmental identification data are monitored and displayed.

[0125] In this embodiment of the invention, the visualization monitoring and display unit 105 constructs an information visualization platform with a real-time heat map, a video linkage display window, an indicator trend chart, and a multi-source interactive comparison chart. It extracts and displays relevant data from abnormal event data and multiple environmental identification data, and then visualizes and displays the relevant data in the information visualization platform, and performs corresponding abnormal monitoring and alarms on the abnormal event data.

[0126] It should be understood that although the steps in the flowcharts of the various embodiments of the present invention are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the various embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0127] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0128] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0129] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

[0130] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A tunnel environment monitoring method based on multi-source data fusion, characterized in that, The method specifically includes the following steps: Multi-source monitoring is performed on the target tunnel to acquire multi-source monitoring data, and the multi-source monitoring data is synchronized and preprocessed to acquire multi-source standard data. Spatiotemporal features are extracted from the multi-source standard data, and unified feature mapping is performed to generate multimodal feature sequences; The multimodal feature sequences are fused from multiple sources to generate fused feature data; The fused feature data is used for environmental classification and identification, multiple environmental identification data are recorded, and real-time anomaly identification is performed to record anomaly event data; An information visualization platform is constructed, in which the abnormal event data and multiple environmental identification data are monitored and displayed; The process of multi-source fusion of the multimodal feature sequences to generate fused feature data specifically includes the following steps: Load the preset multi-source fusion model; The multimodal feature sequences are imported into a multi-source fusion model for multi-source fusion processing. Export fused feature data; The process of importing the multimodal feature sequence into the multi-source fusion model for multi-source fusion processing specifically includes the following steps: The multimodal feature sequence is subjected to separable convolution operation on each modal feature channel to obtain three sets of dimension-aligned feature matrices; Calculate the mean norm of the multimodal feature sequences within each time window, and use the mean norm of the multimodal feature sequences within each time window to dynamically generate dimension correction coefficients through a multilayer perceptron, thereby obtaining the dynamic dimension scaling factor for each time slice; The three sets of dimension-aligned feature matrices and the dynamic dimension scaling factor of each time slice are multiplied by matrix dot product according to modality type to obtain the attention parameter matrix of intermodal correlation; depthwise separable convolution is used to perform spatial filtering extraction on multimodal features to obtain feature slices with local correlation. The feature slices with local relevance are multiplied with the feature matrices aligned to three dimensions to obtain the query vector, key vector and value vector; the scaled dot product of the query vector and key vector is calculated using the attention parameter matrix of intermodal association; then Softmax is applied to generate the attention distribution to obtain the spatial association weight map. The spatial correlation weight graph and the value vector are subjected to multi-head attention weighted summation to obtain the spatial augmented feature set; the multimodal feature sequence is downsampled by a one-dimensional convolution kernel to obtain the compressed temporal feature tensor. The compressed temporal feature tensor is batch normalized and gated to obtain standardized temporal features; the standardized temporal features are then input into a bidirectional LSTM network for recursive processing to obtain a sequence of time evolution feature vectors. The spatially enhanced feature set and the temporally evolved feature vector sequence are sequentially subjected to adaptive feature fusion to obtain a preliminary fused feature tensor; the preliminary fused feature tensor is then subjected to nonlinear feature optimization to obtain the enhanced fused features.

2. The tunnel environment monitoring method based on multi-source data fusion according to claim 1, characterized in that, The process of performing multi-source monitoring of the target tunnel, acquiring multi-source monitoring data, and synchronizing and preprocessing the multi-source monitoring data to obtain multi-source standard data specifically includes the following steps: Multi-source monitoring of the target tunnel was conducted to obtain multi-source monitoring data; The multi-source monitoring data are uniformly aligned using NTP time to obtain multi-source aligned data; The multi-source aligned data is subjected to noise filtering, redundancy removal, and missing value completion to obtain multi-source complete data; The multi-source complete data is standardized to obtain multi-source standard data; The multi-source monitoring of the target tunnel includes: gas monitoring, temperature and humidity monitoring, video monitoring, airflow monitoring, vibration monitoring, and traffic flow monitoring; the multi-source monitoring data includes: gas monitoring data, temperature and humidity monitoring data, video monitoring data, airflow monitoring data, vibration monitoring data, and traffic flow monitoring data.

3. The tunnel environment monitoring method based on multi-source data fusion according to claim 1, characterized in that, The process of extracting spatiotemporal features from the multi-source standard data and performing unified feature mapping to generate a multimodal feature sequence specifically includes the following steps: The multi-source standard data is subjected to time-series features, frequency domain features, spatial features, image features, and traffic features to obtain multi-source feature data; The multi-source feature data is subjected to feature vectorization processing to obtain multi-source vector data; The multi-source vector data is subjected to feature normalization to eliminate the influence of dimensions and obtain normalized vector data. The normalized vector data is labeled with features to generate a multimodal feature sequence.

4. The tunnel environment monitoring method based on multi-source data fusion according to claim 1, characterized in that, The step of adaptively fusing the spatially enhanced feature set and the temporally evolved feature vector sequence to obtain the preliminary fused feature tensor specifically includes the following steps: The spatial augmentation feature set is subjected to global average pooling to obtain the spatial saliency score of each modality; The spatial saliency scores of each modality are input into the compressed excitation network to generate initial weights, and the distribution sharpness of the initial weights is adjusted by the temperature coefficient to obtain the modality dynamic fusion coefficients. The time dimension variance of the time evolution feature vector sequence is calculated, and the time dimension variance of the time evolution feature vector sequence is used as a compensation factor to obtain the time series dynamic adjustment coefficient. The spatial enhancement feature set and the temporal evolution feature vector sequence are aligned by channel dimension interpolation to obtain features to be fused with consistent dimensions. The features to be fused with consistent dimensions are weighted and superimposed according to the modal dynamic fusion coefficient and the temporal dynamic adjustment coefficient to obtain the preliminary fused feature tensor.

5. The tunnel environment monitoring method based on multi-source data fusion according to claim 4, characterized in that, The process of performing nonlinear feature optimization on the preliminary fused feature tensor to obtain the enhanced fused features specifically includes the following steps: The channels of the preliminary fused feature tensor are grouped according to sensor type to obtain cross-modal channels; the cross-modal channels are multiplied element-wise to generate cross terms, and the Pearson correlation coefficient between the cross terms and the preliminary fused feature tensor is calculated. Redundancy is eliminated from the initial fusion feature tensor based on the magnitude of the Pearson correlation coefficient to obtain redundancy-free fusion features. The redundancy-free fusion features are then segmented into feature channels according to time steps to obtain several paths containing continuous channels. These paths containing continuous channels are then randomly masked in a probabilistic manner, and linear interpolation compensation is performed on the adjacent channels of the masked paths to obtain anti-overfitting features that retain the nonlinear relationship of key channels. The interaction terms of all feature pairs are constructed using anti-overfitting features, and binary cross-validation is performed to obtain the gradient magnitude of all interaction terms. Sort the gradient magnitudes of all interaction items in descending order and keep the top few to obtain the filtered interaction items. Then, concatenate the filtered interaction items with the anti-overfitting features to obtain the interaction enhancement features. The interactive enhancement features are modally aligned according to timestamps to obtain aligned multimodal features; The aligned multimodal features within the same spatiotemporal unit are multiplied element-wise, and then the product result is exponentially amplified to obtain the enhanced fused features.

6. The tunnel environment monitoring method based on multi-source data fusion according to claim 5, characterized in that, The process of classifying and identifying the environment from the fused feature data, recording multiple environmental identification data points, and performing real-time anomaly identification and recording anomaly event data specifically includes the following steps: Environmental pollution identification is performed on the fused feature data to obtain the pollution identification status; Fire risk identification is performed on the fused feature data to obtain risk identification data; Traffic anomaly identification is performed on the fused feature data to obtain traffic identification data; A preset combined recognition algorithm is used to perform real-time anomaly identification on the fused feature data and record the anomaly event data.

7. The tunnel environment monitoring method based on multi-source data fusion according to claim 6, characterized in that, The real-time anomaly identification of the fused feature data using a preset combined recognition algorithm specifically includes the following steps: Starting from the current time t, backtrack 24 time slices, extract the feature subset corresponding to each recognition algorithm, calculate the variance, and obtain the variance value of each recognition algorithm; perform exponential decay processing on the variance value of each recognition algorithm to obtain the normalized variance evaluation value of each recognition algorithm. The normalized variance evaluation values ​​of each recognition algorithm are processed sequentially by weighting, exponential operation and Softmax function to obtain the real-time dynamic weight coefficients of each recognition algorithm. The fused feature data is input into a pre-trained isolated forest model, the average path length of the fused feature data in the decision tree is calculated, and anomaly probability values ​​are generated according to a preset threshold to obtain an anomaly score based on the tree structure. The fused feature data is input into a pre-trained autoencoder network for data reconstruction to obtain reconstructed data; the Euclidean distance between the fused feature data and the reconstructed data is calculated, and the Euclidean distance between the fused feature data and the reconstructed data is converted into anomaly probability through the Sigmoid function to obtain the reconstruction error quantification score; The time series data portion of the fused feature data is input into the ARIMA and Prophet models for prediction, obtaining the first and second predicted values ​​at the current time. The first and second predicted values ​​at the current time are multiplied and compared with the measured values. The positive deviation portion is then logarithmically transformed to obtain the nonlinear residual abnormal score. The video stream data portion with fused feature data is input into a pre-trained GAN model to generate a predicted image for the current frame; depth features of the real frame and the predicted image are extracted, the cosine similarity of the depth features of the real frame and the predicted image is calculated and the complement is taken to obtain a visual anomaly probability score. The anomaly score based on tree structure, reconstruction error quantification score, nonlinear residual anomaly score, and visual anomaly probability score are weighted and summed using the real-time dynamic weight coefficients of each recognition algorithm, and the weighted summation result is adaptively normalized to obtain a comprehensive anomaly score. Calculate the moving average of historical data in the same period at the current time, take the average plus 3 times the standard deviation as the dynamic threshold, compare the comprehensive anomaly score with the dynamic threshold, generate a binary judgment result, and use the binary judgment result as the trigger mark for anomaly events.

8. The tunnel environment monitoring method based on multi-source data fusion according to claim 7, characterized in that, The construction of the information visualization platform, in which the abnormal event data and multiple environmental identification data are monitored and displayed, specifically includes the following steps: An information visualization platform is constructed, which includes a real-time heat map, a video-linked display window, an indicator trend chart, and a multi-source interactive comparison chart. Relevant data is extracted and displayed from the abnormal event data and multiple environmental identification data. The information visualization platform displays relevant data in a visual format. The abnormal event data is monitored and alarmed accordingly.

9. A tunnel environment monitoring system based on multi-source data fusion, wherein the system is applied to the tunnel environment monitoring method based on multi-source data fusion as described in any one of claims 1 to 8, characterized in that, The system includes a multi-source monitoring and processing unit, a spatiotemporal feature extraction unit, a multi-source feature fusion unit, an environmental classification and recognition unit, and a visualization monitoring and display unit, wherein: The multi-source monitoring and processing unit is used to perform multi-source monitoring of the target tunnel, acquire multi-source monitoring data, and perform data synchronization and preprocessing on the multi-source monitoring data to acquire multi-source standard data. The spatiotemporal feature extraction unit is used to extract spatiotemporal features from the multi-source standard data, perform unified feature mapping, and generate a multimodal feature sequence. The feature multi-source fusion unit is used to perform multi-source fusion on the multimodal feature sequence to generate fused feature data; An environmental classification and recognition unit is used to classify and recognize the environment based on the fused feature data, record multiple environmental recognition data, and perform real-time anomaly recognition and record abnormal event data. The visualization monitoring and display unit is used to construct an information visualization platform, in which the abnormal event data and multiple environmental identification data are monitored and displayed.

Images