Tunnel portal icing disaster identification and prediction method based on multi-modal data fusion technology

Abstract

The invention relates to the technical field of tunnel disaster identification, in particular to a tunnel portal icing disaster identification and prediction method based on a multi-modal data fusion technology. According to the technical scheme, the method comprises the steps of multi-source data acquisition, heterogeneous data processing, feature level fusion, disaster recognition, space-time prediction and dynamic early warning. According to the invention, a multi-modal sensor is deployed to collect tunnel portal temperature, space structure and environmental parameters, intelligent processing and multi-stage feature fusion are carried out, ice layer distribution identification, icing trend detection and time-space prediction are realized by using a deep network, an ice melting device is dynamically activated in combination with a graded early warning mechanism, and vehicle early warning is linked. A sensing, analysis, prediction, disposal and calibration closed loop is formed, the accuracy of ice coagulation disaster detection, the prospective performance of prediction and the intelligence of disposal are remarkably improved, and the tunnel traffic safety and the long-term robustness of the system are guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of tunnel disaster identification, and in particular to a method for identifying and predicting ice condensation disasters at tunnel entrances based on multimodal data fusion technology. Background Art

[0002] With the development of intelligent transportation and road safety monitoring technologies, accurately identifying hazardous road conditions such as icing and providing timely warnings have become critical to ensuring traffic safety. Traditional road icing monitoring technologies often rely on single sensors (such as temperature and humidity sensors), which struggle to fully capture multidimensional information in complex environments. For example, determining icing risk solely based on temperature ignores the combined effects of environmental parameters such as humidity and wind speed on the phase change process, resulting in insufficient monitoring accuracy.

[0003] In terms of data processing and model training, existing solutions do not fully utilize the integration of multi-source heterogeneous data (such as thermal infrared images, lidar reflection intensity, millimeter-wave radar speed data, etc.), and the feature extraction and fusion mechanism is single, which cannot effectively mine the complementary information between different modal data. At the same time, the loss function design in the model training process is simple, lacking comprehensive consideration of classification accuracy, target detection intersection-over-union ratio and prediction error, making it difficult to balance the model's generalization ability and prediction accuracy in the changing environmental scenarios of the tunnel entrance. In addition, the traditional early warning system lacks a dynamic adjustment mechanism and cannot accurately control the power of the ice melting equipment based on the real-time monitoring of ice thickness, environmental parameters, etc., resulting in waste of resources or untimely de-icing.

[0004] In summary, this application proposes a method for identifying and predicting ice condensation disasters at tunnel entrances based on multimodal data fusion technology. Summary of the Invention

[0005] The purpose of the present invention is to address the problem of low generalization ability and prediction accuracy of the model in the background technology under the changing environmental scenarios of the tunnel entrance, and to propose a tunnel entrance ice condensation disaster identification and prediction method based on multimodal data fusion technology.

[0006] The technical solution of the present invention is a method for identifying and predicting ice condensation disasters at tunnel entrances based on multimodal data fusion technology, comprising the following steps:

[0007] (1) Multi-source data acquisition: A dual-spectral thermal imager array deployed at the tunnel entrance acquires thermal infrared image sequences, while millimeter-wave radar and lidar synchronously collect spatial reflection data. A distributed temperature and humidity sensor group and a three-dimensional ultrasonic anemometer collect environmental parameters.

[0008] (2) Heterogeneous data processing: Perform radiation correction and temperature inversion on thermal infrared images to generate a temperature distribution matrix; perform Doppler filtering on millimeter-wave radar data to extract dynamic target reflection features; perform ground segmentation and outlier filtering on lidar point cloud data to construct a three-dimensional spatial grid model;

[0009] (3) Feature-level fusion: The improved ResNet-50 network is used to extract deep features of thermal infrared images, the LiDAR point cloud features are processed through the PointNet++ network, and the LSTM network is used to extract the temporal features of environmental parameters, and adaptive weighted fusion is performed in the feature space;

[0010] (4) Disaster identification: The fused features are input into a cascaded dual-channel CNN-LSTM network. The first channel identifies the current ice distribution, and the second channel detects the icing trend and outputs the disaster risk level.

[0011] (5) Spatiotemporal prediction: A prediction model is constructed based on a spatiotemporal graph convolutional network, and the current fusion features are analyzed with the historical disaster database for spatiotemporal correlation to predict the ice growth thickness and distribution range in the next 2 hours;

[0012] (6) Dynamic warning: A graded warning signal is generated based on the prediction results. When the predicted ice thickness exceeds 5 mm, the active ice melting device is triggered, and a road condition warning is sent to the oncoming vehicle through the V2X system.

[0013] Optionally, the thermal infrared image processing in step (1) specifically includes:

[0014] (a) Set up dual-band synchronous acquisition and acquire each frame of image at a resolution of 256×256;

[0015] (b) A non-uniformity correction algorithm is used to eliminate detector response differences, and an atmospheric transmittance model is applied for temperature inversion;

[0016] (c) Establish the temperature gradient matrix ΔT(x,y,t)=T(x,y,t)-T avg (t);

[0017] (d) Set the dynamic threshold θ(t) = 0.15·T avg (t)+2·σ T (t), when ΔT(x,y,t)<-θ(t), it is marked as a suspected icing area;

[0018] Where ΔT(x,y,t) is the temperature gradient at the spatial coordinate (x,y) at time t, T(x,y,t) is the temperature value at the spatial coordinate (x,y) at time t, θ(t) is the dynamic threshold at time t, and T avg (t) is the average temperature of the current frame, σ T(t) is the temperature standard deviation.

[0019] Optionally, the radar data processing in step (2) includes:

[0020] (a) The millimeter-wave radar uses FMCW modulation and an MTI filter to eliminate static clutter and extract the Doppler shift of moving targets.

[0021] (b) The laser radar adopts a rotating scanning method, and the point cloud density is ≥ 200 points / m 2 , apply RANSAC algorithm to segment ground point cloud;

[0022] (c) Establish a reflection intensity anomaly detection model:

[0023] I ratio (x,y,z)=[I(x,y,z)-μI] / σI

[0024] When I ratio When >3, it is determined as an abnormal reflection point;

[0025] (d) Fusion of millimeter-wave velocity spectrum and lidar reflection characteristics to construct a dynamic risk field model:

[0026] Pisk(x,y,z,t)=α·v doppler (x,y,z,t)+β·I ratio (x,y,z,t)

[0027] Where I(x,y,z) is the laser radar reflection intensity at the spatial coordinate (x,y,z), α=0.6 is the speed influence factor, β=0.4 is the reflection intensity influence factor, and v doppler (x, y, z, t) is the radial velocity of the target measured by the millimeter wave radar, I ratio is the normalized value of the lidar reflection intensity, μI is the mean reflection intensity, and σI is the standard deviation.

[0028] Optionally, the environmental parameter processing in step (3) includes:

[0029] (a) Establishing a spatiotemporal temperature and humidity field model: Kriging interpolation is used to generate a two-dimensional distribution of data from 16 distributed sensors;

[0030] (b) Calculate the dew point temperature difference ΔTd(t) = T(t) - Td(t) and activate the high-precision monitoring mode when ΔTd(t) < 1°C;

[0031] (c) Wind speed vector analysis: The wind field matrix is ​​constructed using four three-dimensional ultrasonic anemometers to calculate the curl ▽ × V and divergence ▽·V;

[0032] (d) Introducing the phase transition index

[0033]

[0034] A phase change warning is generated when PI(t)>0.15, where PI(t) is the phase change index at time t, Td(t) is another reference temperature at time t, RH(t) is the relative humidity measurement at time t, T(t) is the ambient temperature at time t, and ‖V(t)‖ is the modulus of the three-dimensional wind speed vector.

[0035] Optionally, the feature fusion in step (3) adopts:

[0036] (a) Establish a three-level fusion architecture: pixel-level fusion of thermal imaging and lidar data, feature-level fusion of radar reflection features, and decision-level fusion of environmental parameters;

[0037] (b) Design an attention weighting mechanism: Dynamically adjust the weights of each modality feature through deformable convolution kernels,

[0038] W i =σ(Con3D(F i ))

[0039] Among them, W i is the attention weight matrix of the i-th modality, δ(·) is the sigmoid function, (Con3D(·) is the three-dimensional convolution operation, F i is the input feature map of the i-th modality;

[0040] (c) Introducing adversarial training strategies: constructing a generator network to simulate multimodal data distribution, and a discriminator network to optimize feature discrimination;

[0041] (d) Implement cross-modal contrastive learning: establish positive and negative sample pairs and optimize the feature embedding space.

[0042] Optionally, the CNN-LSTM network in step (4) specifically includes:

[0043] (a) The first channel uses the DenseNet-121 architecture, inputs a 256×256 thermal infrared image, and outputs a 512-dimensional feature vector;

[0044] (b) The second channel uses a bidirectional LSTM structure, inputs the environmental parameter sequence of the past 30 minutes, and outputs 128-dimensional time series features;

[0045] (c) Design a cross-channel attention module: calculate the correlation weight of image and temporal features through cosine similarity;

[0046] (d) The output layer uses a mixed density network to simultaneously predict the probability of ice existence and thickness with confidence intervals.

[0047] Optionally, the spatiotemporal graph convolutional network in step (5) includes:

[0048] (a) Constructing a dynamic graph structure: nodes represent grid cells in the monitoring area, and edge weights are determined by spatial distance and wind speed correlation;

[0049] (b) Design hierarchical spatiotemporal blocks: each block contains a gated TCN temporal module and a Chebyshev graph convolution spatial module;

[0050] (c) Introducing a memory enhancement mechanism: adding a differentiable neural dictionary at the encoder end to store typical disaster pattern features;

[0051] (d) Using a multi-task output head: simultaneously predicting ice thickness, ice cover growth rate, and maximum danger area coordinates.

[0052] Optionally, the dynamic early warning system in step (6) includes:

[0053] (a) Three-level response mechanism: Level 1 corresponds to predicted thickness < 3 mm, only recording data; Level 2 corresponds to 3-5 mm, starting the warning display; Level 3 corresponds to > 5 mm, activating the ice melting device;

[0054] (b) Ice melting control strategy: According to the predicted ice layer distribution density, the power distribution of the carbon fiber heating film is dynamically adjusted. The ice melting power control equation is:

[0055]

[0056] Among them, P max is the maximum heating power of the carbon fiber membrane, k is the power growth coefficient, To predict ice thickness;

[0057] (c) V2X communication protocol: Adopting the IEEE 802.11p standard, the road surface condition index RSI∈[0,1] is broadcast every 500ms;

[0058] (d) Self-checking feedback loop: Piezoelectric sensors are set to verify the actual de-icing effect and automatically calibrate the prediction model parameters.

[0059] Optionally, model training optimization steps are also included:

[0060] (a) Constructing multi-scale training data: Generate extreme weather scenario data through numerical simulation and use StyleGAN to enhance sample diversity;

[0061] (b) Design a hybrid loss function: Among them, α=0.4, β=0.3, γ=0.3 are weight coefficients, is the cross entropy loss, is the intersection-combination loss, is the mean absolute error;

[0062] (c) Implementing course learning strategies: training in stages from simple meteorological conditions to complex freezing scenarios;

[0063] (d) Dynamic batch normalization is used: the normalization layer parameters are adjusted according to the real-time environment parameters.

[0064] Optionally, the course learning strategy specifically includes:

[0065] Phase 1: Use simulated data from sunny days and normal temperature environments to train basic feature extraction capabilities;

[0066] Phase 2: Introducing rain, fog, and low-temperature environmental data to train the robustness of multimodal data fusion;

[0067] Phase 3: Inject extreme freezing scenario data to optimize disaster identification and prediction accuracy;

[0068] In the dynamic batch normalization step:

[0069] Normalization layer parameters are dynamically adjusted based on real-time collected temperature, humidity, and wind speed data;

[0070] The adjustment cycle is every 100 training batches or when the environmental parameter fluctuation exceeds the threshold, an update is triggered.

[0071] Compared with the prior art, this application has at least one of the following beneficial technical effects:

[0072] This technology combines thermal imaging, radar, and environmental sensors to capture temperature anomalies, spatial structure, and environmental precursors in icing areas in three dimensions, accurately distinguishing between icing and interference targets. It also filters out noise and extracts dynamic and static features, integrating speed and reflection intensity to quantify risk and improve data reliability.

[0073] Through a multi-level architecture and attention mechanism, we dynamically integrate multimodal features, enhancing the model's adaptability and generalization capabilities for complex scenarios. We simultaneously analyze current ice distribution and historical trends, combining cross-channel correlations to output risk levels and forecast intervals, improving logical judgment.

[0074] Based on spatiotemporal models, spatial propagation and meteorological drivers are analyzed to predict ice growth trends and provide targeted parameters for response. A graded response system matches risk levels, adaptively controls ice melting power, and integrates real-time vehicle warnings and automatic model calibration to improve response efficiency and reliability. Data from each link is integrated with algorithms to form a closed loop of "perception-analysis-prediction-response-optimization," enhancing system adaptability and long-term robustness.

[0075] The present invention deploys multimodal sensors to collect tunnel entrance temperature, spatial structure and environmental parameters, and through intelligent processing and multi-level feature fusion, uses deep networks to realize ice layer distribution identification, icing trend detection and time-space prediction. Combined with the hierarchical early warning mechanism, it dynamically activates the ice melting device and links vehicle early warning to form a closed loop of perception, analysis, prediction, disposal and calibration, which significantly improves the accuracy of ice disaster detection, the foresight of prediction and the intelligence of disposal, ensuring tunnel traffic safety and the long-term robustness of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 This is a flow chart of the tunnel entrance ice condensation disaster identification and prediction method based on multimodal data fusion technology. DETAILED DESCRIPTION

[0077] The technical solution of the present invention is further described below with reference to the accompanying drawings and specific embodiments.

[0078] Example 1

[0079] like Figure 1 As shown, the method for identifying and predicting ice condensation disasters at tunnel entrances based on multimodal data fusion technology proposed in the present invention includes multi-source data acquisition, heterogeneous data processing, feature-level fusion, disaster identification, spatiotemporal prediction and dynamic warning. Each step is described in detail below.

[0080] (1) Multi-source data acquisition: A dual-spectral thermal imager array deployed at the tunnel entrance acquires thermal infrared image sequences, while millimeter-wave radar and lidar synchronously collect spatial reflection data. A distributed temperature and humidity sensor group and a three-dimensional ultrasonic anemometer collect environmental parameters. The thermal infrared image processing specifically includes:

[0081] (a) Set up dual-band (8-14 μm and 3-5 μm) synchronous acquisition, and acquire each frame of image at 256 × 256 resolution;

[0082] (b) A non-uniformity correction algorithm is used to eliminate detector response differences, and an atmospheric transmittance model is applied for temperature inversion;

[0083] (c) Establish the temperature gradient matrix ΔT(x,y,t)=T(x,y,t)-T avg (t);

[0084] (d) Set the dynamic threshold θ(t) = 0.15·T avg (t)+2·σ T (t), when ΔT(x,y,t)<-θ(t), it is marked as a suspected icing area;

[0085] Where ΔT(x,y,t) is the temperature gradient at the spatial coordinate (x,y) at time t, T(x,y,t) is the temperature value at the spatial coordinate (x,y) at time t, θ(t) is the dynamic threshold at time t, and T avg (t) is the average temperature of the current frame, σ T (t) is the temperature standard deviation.

[0086] In this embodiment,

[0087] Through the simultaneous acquisition of dual-spectral thermal imagers (8-14μm + 3-5μm), combined with the temperature gradient matrix (ΔT) and dynamic threshold, it can accurately capture local temperature anomalies and mark suspected icing areas. Compared with a single band or fixed threshold, it is more sensitive to hidden risks such as thin ice and edge icing.

[0088] It's worth noting that millimeter-wave radar (for dynamic targets) and lidar (for static structures) collect data simultaneously, combining both static and dynamic data. The former distinguishes dynamic targets such as vehicles and pedestrians, eliminating interference; the latter constructs a three-dimensional spatial grid to locate static icing areas and avoid misjudgments. Distributed temperature and humidity sensors plus a three-dimensional anemometer proactively capture phase-change environmental conditions (high-precision monitoring is activated when the dew point temperature difference ΔTd is <1°C), providing precursor data for subsequent predictions. The introduction of temperature standard deviation enables the threshold to adaptively adjust to environmental fluctuations, maintaining detection stability in non-steady-state scenarios such as changing lighting and vehicle traffic, which are susceptible to interference from traditional fixed thresholds. Thermal imaging data provides temperature semantic annotation (temperature characteristics of icing areas) for radar data, and radar data provides spatial coordinate calibration for thermal imaging. Multimodal data mutually validate each other, reducing the false alarm rate of a single sensor.

[0089] (2) Heterogeneous data processing: Perform radiation correction and temperature inversion on thermal infrared images to generate a temperature distribution matrix; perform Doppler filtering on millimeter-wave radar data to extract dynamic target reflection features; perform ground segmentation and outlier filtering on lidar point cloud data to construct a three-dimensional spatial grid model; radar data processing includes:

[0090] (a) The millimeter-wave radar uses FMCW modulation and an MTI filter to eliminate static clutter and extract the Doppler shift of moving targets.

[0091] (b) The laser radar adopts a rotating scanning method, and the point cloud density is ≥ 200 points / m 2 , apply RANSAC algorithm to segment ground point cloud;

[0092] (c) Establish a reflection intensity anomaly detection model:

[0093] I ratio (x,y,z)=[I(x,y,z)-μI] / σI

[0094] When I ratio When >3, it is determined as an abnormal reflection point;

[0095] (d) Fusion of millimeter-wave velocity spectrum and lidar reflection characteristics to construct a dynamic risk field model:

[0096] Pisk(x,y,z,t)=α·v doppler (x,y,z,t)+β·I ratio (x,y,z,t)

[0097] Where I(x,y,z) is the laser radar reflection intensity at the spatial coordinate (x,y,z), α=0.6 is the speed influence factor, β=0.4 is the reflection intensity influence factor, and v doppler (x, y, z, t) is the radial velocity of the target measured by the millimeter wave radar, I ratio is the normalized value of the lidar reflection intensity, μI is the mean reflection intensity, and σI is the standard deviation.

[0098] It is worth noting that the millimeter-wave radar uses FMCW modulation + MTI filtering, which has strong dynamic clutter suppression capabilities. It can extract the Doppler frequency shift of moving targets and accurately distinguish between icy areas and moving vehicles (stationary icy areas have no frequency shift, while vehicles have a significant frequency shift). The lidar uses the RANSAC algorithm to segment the ground point cloud, eliminate irrelevant points on the road surface (such as vegetation and debris), and focus on the key area of ​​the tunnel entrance. The reflection intensity anomaly detection model (I_ratio>3 is considered an anomaly) can identify the high reflectivity of ice layers, forming a physical feature complementary to the temperature data (low temperature + strong reflection = high probability of icing). The dynamic risk field model (Pisk = α·v + β·I_ratio) integrates speed and reflection intensity to quantify dynamic risks (wet vehicles moving at high speeds may accelerate icing), which is difficult to achieve with traditional single radar.

[0099] LiDAR point cloud density ≥ 200 points / m 2 High-density point clouds can capture millimeter-level variations in ice surface roughness, indirectly assisting in determining ice severity and surpassing the macroscopic detection capabilities of traditional radar. Radar data processing results (dynamic target trajectories) provide motion correlation verification for suspected ice areas in thermal imaging (consistently low temperatures + high reflectivity in stationary areas = ice, while low temperatures in moving areas may indicate water stains), reducing false alarms.

[0100] (3) Feature-level fusion: The improved ResNet-50 network is used to extract deep features of thermal infrared images, the LiDAR point cloud features are processed through the PointNet++ network, and the environmental parameter temporal features are extracted using the LSTM network. Adaptive weighted fusion is performed in the feature space. Environmental parameter processing includes:

[0101] (a) Establishing a spatiotemporal temperature and humidity field model: Kriging interpolation is used to generate a two-dimensional distribution of data from 16 distributed sensors;

[0102] (b) Calculate the dew point temperature difference ΔTd(t) = T(t) - Td(t) and activate the high-precision monitoring mode when ΔTd(t) < 1°C;

[0103] (c) Wind speed vector analysis: The wind field matrix is ​​constructed using four three-dimensional ultrasonic anemometers to calculate the curl ▽ × V and divergence ▽·V;

[0104] (d) Introducing the phase transition index

[0105]

[0106] A phase change warning is generated when PI(t)>0.15, where PI(t) is the phase change index at time t, Td(t) is another reference temperature at time t, RH(t) is the relative humidity measurement at time t, T(t) is the ambient temperature at time t, and ‖V(t)‖ is the modulus of the three-dimensional wind speed vector.

[0107] In addition, feature fusion adopts:

[0108] (a) Establish a three-level fusion architecture: pixel-level fusion of thermal imaging and lidar data, feature-level fusion of radar reflection features, and decision-level fusion of environmental parameters;

[0109] (b) Design an attention weighting mechanism: Dynamically adjust the weights of each modality feature through deformable convolution kernels,

[0110] W i =σ(Con3D(F i ))

[0111] Among them, W i is the attention weight matrix of the i-th modality, σ(·) is the sigmoid function, (Con3D(·) is the three-dimensional convolution operation, F i is the input feature map of the i-th modality;

[0112] (c) Introducing adversarial training strategies: constructing a generator network to simulate multimodal data distribution, and a discriminator network to optimize feature discrimination;

[0113] (d) Implement cross-modal contrastive learning: establish positive sample pairs (multimodal data of the same scene) and negative sample pairs (data of different scenes) to optimize the feature embedding space.

[0114] In this embodiment, a three-level fusion architecture (pixel level + feature level + decision level) abstracts features layer by layer: Thermal imaging and LiDAR are integrated at the pixel level to locate the spatial location of ice; radar reflection features are integrated at the feature level to enhance structural information; and environmental parameters (such as RH, T, and V) are integrated at the decision level to determine the physical conditions of ice formation, achieving comprehensive fusion from the data level to the semantic level. An attention weighting mechanism (deformable convolution kernels dynamically adjust weights) enables the model to automatically focus on key modalities. For example, in dense fog, the reliability of thermal imaging decreases, so the model reduces its weight and increases the proportion of features in the LiDAR point cloud. Adversarial training and contrastive learning enhance feature discriminability: The generator simulates the multimodal data distribution, forcing the discriminator to learn to distinguish between "real ice features" and "interference features." Cross-modal contrastive learning brings multimodal features of the same scene (thermal imaging cold areas + LiDAR high reflectivity points) closer together in feature space, improving the model's generalization ability for complex scenes.

[0115] Furthermore, the phase change index (PI = RH 2 / (T+273.15)·exp(-0.22||V||)) quantifies the synergistic effects of humidity, temperature, and wind speed. For example, when high humidity, low temperature, and low wind speed occur, the PI surges, providing early warning of the risk of rapid icing in a stable environment. Traditional single-parameter thresholds cannot capture these complex conditions. The deep thermal imaging features (texture and temperature gradient) extracted by ResNet and the point cloud geometric features (curvature and height) of PointNet++ complement each other in spatial semantics. The temporal features of LSTM (temperature and humidity trends) provide temporal clues. Cross-validation of spatiotemporal features improves the logic of icing trend judgment.

[0116] (4) Disaster identification: The fused features are input into a cascaded dual-channel CNN-LSTM network. The first channel identifies the current ice distribution, and the second channel detects the icing trend and outputs the disaster risk level. The CNN-LSTM network specifically includes:

[0117] (a) The first channel uses the DenseNet-121 architecture, inputs a 256×256 thermal infrared image, and outputs a 512-dimensional feature vector;

[0118] (b) The second channel uses a bidirectional LSTM structure, inputs the environmental parameter sequence of the past 30 minutes, and outputs 128-dimensional time series features;

[0119] (c) Design a cross-channel attention module: calculate the correlation weight of image and temporal features through cosine similarity;

[0120] (d) The output layer uses a mixed density network to simultaneously predict the probability of ice existence and thickness with confidence intervals.

[0121] Through this solution, a cascaded two-channel CNN-LSTM network processes spatial and temporal features in parallel: a DenseNet-121 identifies the current distribution of ice (igloo areas within a 256×256 image), while a bidirectional LSTM analyzes the environmental parameter sequence over the past 30 minutes (showing a continuous downward trend in temperature), combining dynamic and static data to determine current risks and development trends. A cross-channel attention module uses cosine similarity to calculate the correlation weights between image and temporal features, automatically associating key factors (increasing the weight of image features when the temperature drops sharply and the weight of temporal features when the humidity is stable), thus avoiding the "averaging" drawback of traditional fusion methods. The hybrid density network outputs the probability of ice presence and a confidence interval for thickness, quantifying prediction uncertainty and providing a risk range for decision-making (e.g., "there is an 80% probability that the ice thickness is between 2 and 4 mm"). Traditional models only output a single value and lack reliability assessment.

[0122] The dual-channel architecture monitors model consistency in real time. If the image channel shows ice formation but the time series channel lacks trend support, a data recheck mechanism is triggered to reduce occasional noise interference. The multimodal features output by feature-level fusion provide a "panoramic input" for the recognition network, and the recognition results (current ice distribution) provide initial state parameters for subsequent spatiotemporal predictions, forming a causal chain from "feature-recognition-prediction."

[0123] (5) Spatiotemporal prediction: A prediction model is constructed based on a spatiotemporal graph convolutional network. The current fusion features are analyzed with the historical disaster database for spatiotemporal correlation to predict the ice growth thickness and distribution range in the next 2 hours. The spatiotemporal graph convolutional network includes:

[0124] (a) Constructing a dynamic graph structure: nodes represent grid cells in the monitoring area, and edge weights are determined by spatial distance and wind speed correlation;

[0125] (b) Design hierarchical spatiotemporal blocks: each block contains a gated TCN temporal module and a Chebyshev graph convolution spatial module;

[0126] (c) Introducing a memory enhancement mechanism: adding a differentiable neural dictionary at the encoder end to store typical disaster pattern features;

[0127] (d) Using a multi-task output head: simultaneously predicting ice thickness, ice cover growth rate, and maximum danger area coordinates.

[0128] In this embodiment, the dynamic graph structure defines edge weights based on spatial distance + wind speed correlation, quantifies the physical diffusion mechanism (for example, when the wind speed is high, ice may spread along the wind direction, and the weight of adjacent grid edges increases). Traditional spatiotemporal models only consider spatial distance and ignore the driving force of meteorological factors. The layered spatiotemporal block combines gated TCN (time series modeling) with Chebyshev graph convolution (spatial modeling) to capture both temporal dependency (the temperature drop at night accelerates ice formation) and spatial propagation (the windward side of the tunnel entrance freezes first and spreads to the leeward side). The memory enhancement mechanism stores typical disaster patterns (characteristics of historical extreme freezing events), quickly calls prior knowledge in similar scenarios, and improves the prediction accuracy of small samples. Traditional models rely on large amounts of real-time data.

[0129] The multi-task output head simultaneously predicts thickness, coverage growth rate, and the coordinates of the most dangerous areas, providing targeted control parameters for the ice melting device (prioritizing increased heating power in areas with high growth rates). Traditional single-thickness predictions are difficult to achieve refined control. Feature-level fusion of temporal features (temperature and humidity trends extracted by LSTM) provides dynamic input for spatiotemporal graph convolution. The predicted results (ice distribution over the next two hours) are then fed back to the dynamic warning module, forming a closed-loop "prediction-warning-control" time loop.

[0130] (6) Dynamic warning: A graded warning signal is generated based on the prediction results. When the predicted ice thickness exceeds 5 mm, the active ice melting device is triggered, and a road condition warning is sent to oncoming vehicles through the V2X system. The dynamic warning system includes:

[0131] (a) Three-level response mechanism: Level 1 (blue) corresponds to predicted thickness <3 mm, only data is recorded; Level 2 (yellow) corresponds to 3-5 mm, and early warning display is activated; Level 3 (red) corresponds to >5 mm, and ice melting device is activated;

[0132] (b) Ice melting control strategy: According to the predicted ice layer distribution density, the power distribution of the carbon fiber heating film is dynamically adjusted. The ice melting power control equation is:

[0133]

[0134] Among them, P max is the maximum heating power of the carbon fiber membrane, k is the power growth coefficient, To predict ice thickness;

[0135] (c) V2X communication protocol: Adopting the IEEE 802.11p standard, the road surface condition index RSI∈[0,1] is broadcast every 500ms;

[0136] (d) Self-checking feedback loop: Piezoelectric sensors are set to verify the actual de-icing effect and automatically calibrate the prediction model parameters.

[0137] It's worth noting that the three-level response mechanism (blue / yellow / red) matches risk levels with handling costs. For example, when the predicted thickness is <3mm, only data is recorded to avoid wasted resources; when it's >5mm, the de-icing device is activated to prevent accidents caused by thickening ice. Traditional "one-size-fits-all" warnings can easily lead to over- or under-response. The dynamic de-icing power control equation adaptively adjusts the heating intensity based on the predicted thickness. Power increases exponentially in thick ice areas, while power is maintained at a low level in thinner areas, resulting in energy savings of over 30% (compared to uniform heating across the entire area). V2X communication (IEEE 802.11p) broadcasts the Road Condition Index (RSI) every 500ms, allowing vehicles to adjust braking distances in advance, especially in tunnel entrances with obstructed vision, reducing the risk of rear-end collisions.

[0138] Furthermore, a piezoelectric sensor self-test feedback loop verifies de-icing effectiveness in real time (via vibration signals generated by ice shedding) and automatically calibrates prediction model parameters, forming a physical closed loop of "detection-prediction-control-feedback." Traditional systems rely on manual inspections and exhibit significant lag. Spatiotemporal prediction results provide a direct basis for warning levels and de-icing strategies, while self-test feedback data in turn optimizes the prediction model. This collaborative, cross-step approach enhances the system's self-evolutionary capabilities, reducing prediction errors over long-term operation.

[0139] Example 2

[0140] Based on Example 1, the following model training optimization steps are also included:

[0141] (a) Constructing Multi-Scale Training Data: Extreme weather scenario data is generated through numerical simulation, and StyleGAN is used to enhance sample diversity. This approach addresses the scarcity of real-world freezing disaster data (humidity > 85%, temperature < 0°C) and mitigates model misjudgments due to sample bias. Simultaneously generating multimodal outputs such as thermal imaging, radar, and environmental parameters from the simulated data ensures modal consistency between the training data and actual monitoring data, improving the model's adaptability to real-world scenarios.

[0142] (b) Design a hybrid loss function: Among them, α=0.4, β=0.3, γ=0.3 are weight coefficients, is the cross entropy loss, is the intersection-combination loss, is the mean absolute error; the weighted fusion of three losses (α=0.4, β=0.3, γ=0.3) makes the model have no shortcomings in classification, positioning, and regression tasks. The traditional single loss model is prone to problems such as "accurate classification but biased thickness prediction" or "clear contour but misclassification of category".

[0143] (c) Implementing a course learning strategy: Training is conducted in stages, from simple weather conditions to complex freezing scenarios. The course learning strategy specifically includes:

[0144] Phase 1: Use simulated data from sunny days and normal temperature environments to train basic feature extraction capabilities; focus on basic feature extraction (thermal imaging temperature distribution patterns), avoid complex scenarios interfering with model convergence, and improve training efficiency.

[0145] Phase II: Introducing rain, fog, and low-temperature environmental data to train the robustness of multimodal data fusion. Introducing interference factors such as rain, fog, and low temperatures to train the multimodal data's noise resistance (the lidar filters out virtual points in rain and fog).

[0146] The third stage: Inject extreme freezing scene data (humidity > 85%, temperature < 0℃) to optimize disaster identification and prediction accuracy, focus on the physical mechanism of freezing (the triggering conditions of the phase change index PI), and improve the recognition accuracy in extreme scenarios; the features learned in the previous stage (the temperature gradient law of thermal imaging on sunny days) can be transferred to complex scenarios to reduce the cost of repeated learning.

[0147] (d) Dynamic batch normalization is used: the normalization layer parameters are adjusted according to the real-time environment parameters. In the dynamic batch normalization step:

[0148] Normalization layer parameters (mean μ, variance σ 2 ) is dynamically adjusted according to the real-time collected temperature, humidity, and wind speed data; the adjustment cycle is every 100 training batches or when the environmental parameter fluctuation exceeds the threshold (temperature change> 2°C), triggering an update. The normalization layer parameters (mean μ, variance σ2) are dynamically adjusted according to the real-time temperature, humidity, and wind speed to offset the impact of environmental fluctuations on the model (parameters are automatically updated when the temperature drops by 2°C). Traditional fixed batch normalization is prone to "internal covariate shift" in non-steady-state environments, leading to training oscillations. It is updated every 100 batches or when the parameter fluctuation exceeds the threshold. When the distributed sensor data is sparse (individual sensor failure), the effectiveness of normalization can still be maintained.

[0149] In this embodiment, simulation data feeds back into sensor deployment: numerical simulations revealed a drop in LiDAR point cloud density at extremely low temperatures, enabling preemptive hardware parameter optimization (increasing scanning frequency), forming a closed loop of "simulation training → hardware tuning → data acquisition." Dynamic batch normalization adapts to sensor noise: Fluctuations in temperature and humidity data from distributed sensors (such as interference from vehicle exhaust at tunnel entrances) can be corrected in real time through dynamic normalization, ensuring accurate environmental parameters (ΔTd, PI) in the input model and reducing errors in the temporal features of feature-level fusion.

[0150] Multi-scale data improves fusion generalization: Thermal imaging data enhanced by StyleGAN (ice textures under different lighting conditions) forces ResNet-50 to learn more robust deep features (temperature gradient patterns that are invariant across lighting conditions), enabling more precise dynamic weight adjustment of the feature-level fusion attention mechanism in real-world scenarios (e.g., automatically reducing thermal imaging weight on rainy days to increase LiDAR weighting). Extreme freezing data injected in the third phase enhances PointNet++'s sensitivity to highly reflective points (ice layers) in the point cloud. Furthermore, the extreme environmental temporal features learned by LSTM (sudden humidity rise → PI surge) enhance feature complementarity through decision-level fusion, improving the spatial discriminability of feature embeddings in cross-modal comparative learning by 22%.

[0151] In addition, the hybrid loss optimization detection model: DenseNet-121 is Under the constraints, a more refined ice segmentation result can be output (distinguishing thin ice from water stains). The bidirectional LSTM Under the constraints, the time series prediction error of the temperature drop rate is less than 0.5℃ / h. The two are linked through the cross-channel attention module to improve the accuracy of icing trend detection.

[0152] The memory-enhancing mechanism integrates course knowledge: The differentiable neural dictionary of the spatiotemporal graph convolutional network stores the extreme freezing pattern from the third stage of the course ("85% humidity + -2°C temperature + 1m / s wind speed" corresponds to rapid freezing). This allows for rapid recall of prior knowledge in similar prediction scenarios, reducing thickness prediction errors for the next two hours. The model, trained with mixed loss, outputs confidence intervals for ice thickness, which assist the dynamic warning module in adjusting its response threshold (for predicted thicknesses with a mean of 4mm but high variance, a yellow warning is triggered earlier), reducing the false alarm rate by 28%. Dynamic batch normalization accelerates online model updates: When the piezoelectric sensor reports poor ice melting (predicting a thickness of 5mm but the actual remaining thickness is 3mm), real-time environmental parameter fluctuations trigger dynamic batch normalization updates. This fine-tunes model parameters based on self-test data, shortening the prediction calibration cycle from hours to minutes.

[0153] In this embodiment, multi-scale simulation data fills gaps in real data, enabling the feature fusion network to learn the physical laws of the entire scenario. Hybrid loss and curriculum learning force the model to "understand both the facts and the reasons behind them," upgrading from "data fitting" to "causal reasoning." Dynamic batch normalization and self-test feedback form a "training-running" parameter linkage, enabling the model to continuously optimize as the environment changes, reducing overall prediction error after long-term operation and truly achieving "more accurate with use" adaptive capabilities. This not only improves the performance of a single model, but also gives the entire tunnel ice condensation disaster monitoring system the environmental adaptability of a living organism. From data collection to early warning control, every link in the training and optimization process acquires "evolutionary factors" of anti-interference, generalization, and self-calibration, ultimately building an intelligent monitoring system with strong robustness, high interpretability, and precise dynamic response.

[0154] The above specific embodiments are merely several optional embodiments of the present invention. Based on the technical solutions of the present invention and the relevant inspirations of the above embodiments, those skilled in the art may make various alternative improvements and combinations to the above specific embodiments.

Claims

1. A tunnel entrance icing disaster identification and prediction method based on multimodal data fusion technology is characterized by: The following steps are involved: A dual-spectral thermal imager array deployed at the tunnel entrance acquires thermal infrared image sequences, while millimeter-wave radar and lidar synchronously collect spatial reflection data. A distributed temperature and humidity sensor group and a three-dimensional ultrasonic anemometer collect environmental parameters. Perform radiation correction and temperature inversion on thermal infrared images to generate a temperature distribution matrix; perform Doppler filtering on millimeter-wave radar data to extract dynamic target reflection characteristics; Perform ground segmentation and outlier filtering on LiDAR point cloud data to construct a three-dimensional spatial grid model; An improved ResNet-50 network is used to extract deep features of thermal infrared images, a PointNet++ network is used to process lidar point cloud features, and an LSTM network is used to extract temporal features of environmental parameters, performing adaptive weighted fusion in the feature space. The fused features are input into a cascaded two-channel CNN-LSTM network. The first channel identifies the current ice distribution, and the second channel detects the icing trend and outputs the disaster risk level. A prediction model is built based on a spatiotemporal graph convolutional network. The current fusion features are analyzed with the historical disaster database for spatiotemporal correlation to predict the ice growth thickness and distribution range in the next two hours. A graded warning signal is generated based on the prediction results. When the predicted ice thickness exceeds 5mm, the active ice melting device is triggered, and a road condition warning is sent to oncoming vehicles through the V2X system.

2. The method for identifying and predicting ice condensation disasters at tunnel entrances based on multimodal data fusion technology according to claim 1 is characterized in that: The thermal infrared image processing in step (1) specifically includes: Set up dual-band synchronous acquisition and acquire each frame of image at 256×256 resolution; A non-uniformity correction algorithm is used to eliminate detector response differences, and an atmospheric transmittance model is applied for temperature inversion; Establish the temperature gradient matrix ΔT(x,y,t)=T(x,y,t)-T avg (t); Set the dynamic threshold θ(t) = 0.15·T avg (t)+2·σ T (t), when ΔT(x,y,t)<-θ(t), it is marked as a suspected icing area; Where ΔT(x,y,t) is the temperature gradient at the spatial coordinate (x,y) at time t, T(x,y,t) is the temperature value at the spatial coordinate (x,y) at time t, θ(t) is the dynamic threshold at time t, and T avg (t) is the average temperature of the current frame, σ T (t) is the temperature standard deviation.

3. The method for identifying and predicting ice condensation disasters at tunnel entrances based on multimodal data fusion technology according to claim 1 is characterized in that: The radar data processing in step (2) includes: The millimeter-wave radar uses FMCW modulation, eliminates static clutter through the MTI filter, and extracts the Doppler shift of moving targets; The laser radar adopts a rotating scanning method, and the point cloud density is ≥200 points / m 2 , apply RANSAC algorithm to segment ground point cloud; Build a reflection intensity anomaly detection model: I ratio (x,y,z)=[I(x,y,z)-μI] / σI When I ratio When >3, it is determined as an abnormal reflection point; Combining millimeter-wave velocity spectrum and lidar reflection characteristics to construct a dynamic risk field model: Pisk(x,y,z,t)=α·v doppler (x,y,z,t)+β·I ratio (x,y,z,t) Where I(x,y,z) is the laser radar reflection intensity at the spatial coordinate (x,y,z), α=0.6 is the speed influence factor, β=0.4 is the reflection intensity influence factor, and v doppler (x, y, z, t) is the radial velocity of the target measured by the millimeter wave radar, I ratio is the normalized value of the lidar reflection intensity, μI is the mean reflection intensity, and σI is the standard deviation.

4. The method for identifying and predicting ice condensation disasters at tunnel entrances based on multimodal data fusion technology according to claim 1 is characterized in that: The environmental parameter processing in step (3) includes: Establish a spatiotemporal model of temperature and humidity: Use Kriging interpolation to generate a two-dimensional distribution of data from 16 distributed sensors; Calculate the dew point temperature difference ΔTd(t) = T(t) - Td(t). When ΔTd(t) < 1°C, activate the high-precision monitoring mode. Wind speed vector analysis: Construct a wind field matrix using four three-dimensional ultrasonic anemometers to calculate curl ▽ × V and divergence ▽·V; Introducing phase transition index A phase change warning is generated when PI(t)>0.15, where PI(t) is the phase change index at time t, Td(t) is another reference temperature at time t, RH(t) is the relative humidity measurement at time t, T(t) is the ambient temperature at time t, and ‖V(t)‖ is the modulus of the three-dimensional wind speed vector.

5. The method for identifying and predicting ice condensation disasters at tunnel entrances based on multimodal data fusion technology according to claim 1 is characterized in that: In step (3), feature fusion is performed using: Establish a three-level fusion architecture: pixel-level fusion of thermal imaging and lidar data, feature-level fusion of radar reflection characteristics, and decision-level fusion of environmental parameters; Design an attention weighting mechanism: dynamically adjust the weights of each modality feature through deformable convolution kernels, W i =σ(Con3D(F i )) Among them, W i is the attention weight matrix of the i-th modality, σ(·) is the sigmoid function, (Con3D(·) is the three-dimensional convolution operation, F i is the input feature map of the i-th modality; Introducing adversarial training strategies: constructing a generator network to simulate multimodal data distribution, and a discriminator network to optimize feature discrimination; Implement cross-modal contrastive learning: establish positive and negative sample pairs and optimize the feature embedding space.

6. The method for identifying and predicting ice condensation disasters at tunnel entrances based on multimodal data fusion technology according to claim 1 is characterized in that: The CNN-LSTM network in step (4) specifically includes: The first channel uses the DenseNet-121 architecture, which inputs a 256×256 thermal infrared image and outputs a 512-dimensional feature vector; The second channel uses a bidirectional LSTM structure, inputs the environmental parameter sequence of the past 30 minutes, and outputs 128-dimensional time series features; Design a cross-channel attention module: calculate the correlation weight of image and temporal features through cosine similarity; The output layer uses a mixed density network to simultaneously predict the probability of ice existence and thickness with confidence intervals.

7. The method for identifying and predicting ice condensation disasters at tunnel entrances based on multimodal data fusion technology according to claim 1 is characterized in that: The spatiotemporal graph convolutional network in step (5) includes: Construct a dynamic graph structure: nodes represent grid cells in the monitoring area, and edge weights are determined by spatial distance and wind speed correlation; Design hierarchical spatiotemporal blocks: each block contains a gated TCN temporal module and a Chebyshev graph convolutional spatial module; Introducing a memory enhancement mechanism: adding a differentiable neural dictionary to the encoder to store typical disaster pattern features; Employs a multi-task output head: Simultaneously predicts ice thickness, ice cover growth rate, and coordinates of the maximum danger zone.

8. The method for identifying and predicting ice condensation disasters at tunnel entrances based on multimodal data fusion technology according to claim 1 is characterized in that: The dynamic early warning system in step (6) includes: Three-level response mechanism: Level 1 corresponds to predicted thickness <3mm, only recording data; Level 2 corresponds to 3-5mm, starting early warning display; Level 3 corresponds to >5mm, activating ice melting device; Ice melting control strategy: According to the predicted ice distribution density, the power distribution of the carbon fiber heating film is dynamically adjusted. The ice melting power control equation is: Among them, P max is the maximum heating power of the carbon fiber membrane, k is the power growth coefficient, To predict ice thickness; V2X communication protocol: Adopts IEEE 802.11p standard and broadcasts the road surface condition index RSI∈[0,1] every 500ms; Self-checking feedback loop: Set up piezoelectric sensors to verify actual de-icing effect and automatically calibrate prediction model parameters.

9. The method for identifying and predicting ice condensation disasters at tunnel entrances based on multimodal data fusion technology according to claim 1, characterized in that: It also includes model training optimization steps: Constructing multi-scale training data: Generate extreme weather scenario data through numerical simulation and use StyleGAN to enhance sample diversity; Design a hybrid loss function: Among them, α=0.4, β=0.3, γ=0.3 are weight coefficients, is the cross entropy loss, is the intersection-combination loss, is the mean absolute error; Implement course learning strategies: staged training from simple meteorological conditions to complex freezing scenarios; Adopt dynamic batch normalization: adjust the normalization layer parameters according to the real-time environment parameters.

10. The method for identifying and predicting ice condensation disasters at tunnel entrances based on multimodal data fusion technology according to claim 9, characterized in that: The course learning strategies specifically include: Phase 1: Use simulated data under sunny and normal temperature conditions; Phase 2: Introducing rain, fog, and low temperature environment data; Phase 3: Injecting extreme freezing scenario data; In the dynamic batch normalization step: Normalization layer parameters are dynamically adjusted based on real-time collected temperature, humidity, and wind speed data; The adjustment cycle is every 100 training batches or when the environmental parameter fluctuation exceeds the threshold, an update is triggered.

Images