A method for intelligent identification and risk assessment of a solution cavity based on MSCNN and combination of multi-modal data

Abstract

The application discloses a kind of intelligent recognition and risk assessment method of roadway wall back solution cavity based on multi-scale convolutional neural network (MSCNN) and multi-modal data fusion.The method first synchronously collects ultrasonic wave, ground penetrating radar and microseismic data, after denoising, standardization processing, extracts five kinds of key features such as wave velocity, energy attenuation rate, dielectric constant anomaly, fracture density and energy cumulative slope.Through forward simulation to expand data and construct labeled data set.Finally, multi-modal features are input to MSCNN with three branches for fusion training and identification.The application results show that the method can effectively capture multi-scale complex features, significantly improve the accuracy and robustness of solution cavity recognition, and provide a new solution for concealed anomaly body detection.

Description

Technical Field

[0001] This invention relates to a method for intelligent identification and risk assessment of cavities behind the walls of potash mine roadways based on a combination of multi-scale convolutional neural networks (MSCNN) and multimodal data (ultrasound, microseismic data) in the interpretation of ground-penetrating radar data. It is applicable to real-time intelligent monitoring and dynamic assessment and prevention of disasters in areas with developed cavities, such as the Kaiyuan mining area in Laos. Background Technology

[0002] Due to the easily soluble and corrosive nature of the rock salt layer, the Kaiyuan potash mine in Laos often develops hidden solution cavities behind the tunnel walls. The formation of these cavities weakens the bearing capacity of the roof strata and leads to stress imbalance and structural deformation in the surrounding rock, potentially causing roof collapse and structural instability. If the cavity connects to an aquifer, the sudden influx of high-pressure water and sediment can flood the tunnel and trigger a severe water inrush accident. Therefore, efficient, non-destructive, real-time, and accurate detection of solution cavities behind the tunnel walls is crucial. However, relying solely on ground-penetrating radar cannot achieve small-scale, millimeter-level cavity identification; electromagnetic wave data is sensitive to noise and struggles to distinguish similar features such as cavities from rock fissures; and traditional radar data interpretation suffers from significant subjectivity. Existing deep learning models (with weak fusion capabilities for multi-source heterogeneous data (radar, ultrasound, microseismic), and feature alignment errors ≥0.3m) with fixed fields of view cannot simultaneously capture the macroscopic morphology (diameter ≥5m) and microscopic structure (pores, fissures) of cavities. Traditional early warning systems cannot achieve intelligent online monitoring of cavity expansion rates (e.g., ≥5cm / h), resulting in delayed disaster warnings. To address these issues, there is an urgent need to develop an intelligent cavity identification method combining MSCNN and multimodal data, along with an integrated solution for dynamic risk assessment. Summary of the Invention

[0003] The purpose of this invention is to propose an intelligent cavity identification and risk assessment method based on MSCNN and multimodal data. It fully considers the complex morphology of cavities (such as sac-like, tubular, and honeycomb-like), blurred boundaries (radar echo gradient < 0.1dB / m), and dynamic expansion, and achieves intelligent cavity identification and risk classification assessment with millimeter-level accuracy, thus solving the problems of real-time performance and accuracy in the detection of concealed cavities. The technical solution of the present invention is as follows: A cavity intelligent identification and risk assessment method based on MSCNN and multimodal data includes the following steps: First, three types of sensors were used to collect and process multimodal data of the target body behind the tunnel wall. Step (1): Deploy a high-frequency ultrasonic array along the tunnel wall to detect cavities in the near-wall area (0-1m) (resolution ±0.05m). Step (2): Use 400MHz GR ground penetrating radar to lay longitudinal survey lines along the tunnel direction, and use intelligent detection to obtain the cavity morphology and dielectric constant distribution B-Scan map in the medium-deep (0-5m) range; Step (3): Install triaxial microseismic sensors (10 sensors are arranged with a spacing of 10m between each sensor, and the position is updated every 24 hours) on the roof and surrounding rock of the tunnel. Sample the sensors around the clock to capture the low-frequency vibration signals (frequency band 1-50Hz) caused by the expansion of the cavity in a timely manner, and reflect the location of the rupture (cavity area) and the amount of energy. Step (4): The data collected by the above methods are denoised, registered, and aligned. The ultrasonic velocity distribution map is processed by trend tomography, and the energy distribution map is subjected to reverse compensation. The ground penetrating radar B-Scan data map is denoised and gain-processed. The density distribution map and cumulative energy slope map of microseismic rupture events (cavity area) are processed by STA algorithm and moment tensor inversion, respectively. Through these processes, a high-resolution target medium distribution image behind the wall is generated, laying the foundation for subsequent interpretation. Step (5): Design the data collected by the above methods from two dimensions: time reference unification and spatial registration, to ensure accurate synchronization of multimodal data; Step (6): Extract features from the five types of data processed above, extract velocity and energy attenuation rate from ultrasonic velocity distribution map and energy distribution map, extract dielectric constant anomaly from ground penetrating radar B-Scan data map, and extract rock salt fracture density and energy cumulative slope from microseismic fracture event (cavity area) density distribution map and energy cumulative slope map; Step (7): The key features extracted from the above three types of data are classified and subjected to forward modeling according to basic rules to expand the data, and then organized into their respective labeled datasets. Input the data into the convolutional neural network model according to the three branches. Ultrasonic data input shape: 256×256×2, dual-channel wave velocity distribution map and energy distribution map; Ground penetrating radar data input shape: 512×512×1, single-channel radar B-Scan image; Microseismic data input shape: 64×N×2, rupture density distribution map and energy accumulation slope map. Secondly, a multi-scale convolutional neural network architecture (MSCNN) is designed, which includes an input layer, convolutional layer, pooling layer, fully connected layer, optimizer and loss function, output layer, etc., connected in sequence. Step (1): Low-scale branching for local detail perception requires capturing the microstructure of the cavity; Input layer: Image size 256×256×2; Dilated convolutional layer: 3×3 kernel, dilation rate 3, 64 output channels, stride 1, automatic padding; Batch normalization (BN operation, standardizing the feature map of each channel to accelerate training convergence) and ReLU activation; Max pooling layer: 2×2 kernel, stride 2, padding 0; Convolutional layer: 3×3 kernel, 128 channels, stride 1, padding 1; Spatial pyramid pooling (SPP) achieves multi-scale fusion, capturing both local details (small scale) and global distribution (large scale) of the cavity, and is highly adaptable to subsequent fully connected layers. Regardless of the input feature map size, SPP outputs a fixed dimension (256-dimensional). Step (2): Extract the mesoscale branches of the main contour of the cavity, which requires identifying the macroscopic morphology of the cavity; Input layer: Input a single-channel B-Scan image, size 512×512×1; First convolutional layer: Uses a 3×3 convolutional kernel, 64 channels, stride 1, padding 1, output 512×512×64; Second convolutional layer: Uses a 3×3 convolutional kernel, 64 channels, stride 1, padding 1, output 512×512×64; Max pooling layer: Uses a 2×2 pooling kernel, stride 2, the number of channels remains 64, output: 256×256×64; Third convolutional layer: Uses a 3×3 convolutional kernel, 128 channels, stride 1, padding 1, final output size 256×256×128; Step (3): The high-scale branch of global modeling requires analysis of the relationship between the spatial distribution of the cavity and the stress field; Input size: 64×N×2, 64 samples, N time steps, 2 features; LSTM layer (Long Short-Term Memory network): 64 hidden units, 2 input dimensions (corresponding to 2 features), output mode: returns the hidden state of the final time step; fully connected layer: 64 input dimensions, 64 output dimensions, introduces the ReLU function; a self-attention mechanism is added to realize dynamic feature weighting, providing dynamic risk signals for multimodal fusion; Step (4): Multimodal and multiscale feature fusion, GPR and ultrasound are aligned, the low-scale branch output (256×256×128) is upsampled to 512×512 to match the resolution of the original GPR image; the 64-dimensional microseismic feature vector is copied and expanded to 512×512×64 to adapt to the image spatial dimension; this part can realize data image feature alignment and dimension matching; at the same time, Squeeze-and-Excitation attention weighted fusion of data is used. Finally, the output head features a layered design. It employs a two-branch design, corresponding to cavity identification and risk assessment respectively. Step (1): First output branch, based on the intelligent cavity recognition method combining MSCNN and multimodal data; Multi-scale cavity recognition output head design: Input layer 512×512×256, fused multimodal feature map; First, a 3×3 convolutional layer is used to further extract local details (such as cavity boundaries and pore distribution) and compress the 256-channel features to 128 channels; The feature map of each channel is normalized and ReLU activated; Convolutional layer is used again: 1×1 convolutional kernel, 1 channel, stride 1, padding 0; Then, the Sigmoid activation function and probability calculation function are used to realize channel fusion, mapping the 128-channel features to a single channel, integrating multi-scale information, weighted fusion, and compressing each pixel value to [0,1] according to the probability calculation formula to represent the probability of cavity existence. The threshold is set to 0.7, and pixels with a probability ≥0.7 are marked as cavities. Finally, the cavity segmentation map is output through the Gumbel-Softmax function. Step (2): The second output branch, a risk assessment method based on the combination of MSCNN and multimodal data, includes the following steps: First, microseismic early assessment and prediction; GPR scans along the survey network every 12 hours, and ultrasonic scanning of the entire cross section every 4 hours; microseismic sensors continuously monitor 24 / 7, and abnormal events trigger high-frequency GPR scanning. Secondly, design the risk index function DRI. Where V: cavity volume (m³), D: distance from tunnel wall (m), R: expansion rate (cm / h), S: surrounding rock strength (MPa), with S as the benchmark 15MPa; Cavity volume V: The volume of the connected region is calculated after thresholding the segmentation probability map (p>0.7); Spread rate R: The slope of the cumulative microseismic energy (linear fitting R²>0.85); Surrounding rock strength S: Can be estimated based on the ultrasonic wave velocity. Risk warning classification design: Level I (DRI≥0.75): Immediate evacuation and area closure are required; Level II (0.6<DRI<0.75): Time-limited reinforcement and intensified monitoring are required; Level III (DRI≤0.6): Routine inspections are required. Finally, a multi-scale risk grading output head is designed. Input layer: Global average pooling (GAP) is applied to the fused feature maps from the previous layer to preserve semantic information in the channel dimension while eliminating spatial dimension; 256-dimensional representation extracts high-level features from 256 channels; First fully connected layer: Nonlinearly maps the 256-dimensional features to 128 dimensions, learning the complex relationships between features through a weight matrix; ReLU activation function is also introduced; Second fully connected layer: Maps the 128-dimensional features to a 3-dimensional output (corresponding to low / medium / high risk); Softmax activation function is introduced to convert the 3-dimensional output into a probability distribution, resulting in a three-dimensional probability vector. The category corresponding to the highest probability is taken as the prediction result. Attached Figure Description

[0004] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. Those skilled in the art can obtain other drawings based on these drawings without creative effort. Figure 1 This is a schematic diagram of the workflow in one embodiment of the present invention; Figure 2 This is a schematic diagram of ultrasonic detection in one embodiment of the present invention; Figure 3 This is a schematic diagram of ground-penetrating radar detection in one embodiment of the present invention; Figure 4 This is a schematic diagram of a microseismic instrument detection in one embodiment of the present invention; Figure 5 This is a schematic diagram of the MSCNN structure in one embodiment of the present invention; Figure 6 This is a schematic diagram of a 3-branch channel convolution in one embodiment of the present invention; Figure 7 This is a schematic diagram of the MSCNN attention mechanism in one embodiment of the present invention; Figure 8 This is a graph showing the training results of a conventional convolutional neural network in one embodiment of the present invention; Figure 9 This is a graph showing the training results of MSCNN in one embodiment of the present invention; Figure 10 This is a schematic diagram of the training results of a conventional single radar cavity recognition ordinary convolutional neural network in one embodiment of the present invention; Figure 11 This is a schematic diagram of the training results of a multi-scale convolutional neural network for cavity recognition based on multimodal data in one embodiment of the present invention; Figure 12 This is a schematic diagram illustrating the actual situation of the cavity in one embodiment of the present invention; Detailed Implementation

[0005] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. In fact, the embodiments described below are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention. This invention implements a cavity intelligent recognition and evaluation method combining MSCNN and multimodal data. The workflow is as follows: Figure 1 As shown, it includes the data detection and acquisition stage, the data processing and feature extraction stage, the data expansion and dataset creation stage, and the multi-scale convolutional neural network application stage. 1. Data detection and acquisition The system uses 50kHz high-frequency ultrasound with a resolution of ±0.05m and a detection depth of 0-3m. The ultrasonic probe is installed using a magnetic base and silicone grease coupling agent to ensure efficient sound wave transmission. The probe tilt angle is 15°-30°, and the fan-shaped scan covers the 0-3m area behind the wall. Sensor nodes are arranged along the tunnel direction with a horizontal measurement line spacing of 2m and a vertical layered scan at 0.5m intervals. A full-section scan is performed every 4 hours, with each scan lasting 10 minutes. The GR ground-penetrating radar, equipped with a 400MHz high-frequency antenna, has a maximum detection depth of 5m and a vertical resolution of 5cm. The survey lines are spaced 0.5m apart and deployed along the tunnel axis, as shown in the figure, covering a width of 3m. It generates B-Scan data maps in real time, with a single detection time of 60 minutes and a daily data collection distance of 3km. Explosion-proof microseismic sensors with a frequency range of 1-200Hz, a sensitivity of 0.1μm / s, and a dynamic range ≥120dB are used. Sensor nodes are arranged along the roadway direction, with a distance of no less than 10m between every two microseismic sensors. The nodes are located on the roadway roof and both sides, forming a three-dimensional monitoring network to ensure that the detection range covers the roof, both sides, and the area behind the wall. The sensors on the roof face the roof rock strata to monitor vertical fractures; the sensors on the sides are arranged horizontally to monitor horizontal stress release. The positions are updated every 24 hours, and sampling is performed around the clock to promptly capture low-frequency vibration signals (frequency band 1-50Hz) caused by the expansion of the cavity, reflecting the fracture location (cavity area) and energy magnitude. 2. Data Processing and Feature Extraction Ultrasonic detection yields velocity and energy distribution maps. The velocity distribution map is then subjected to travel-time tomography to obtain a velocity model (including local velocity perturbation rates), which is used to adjust and optimize the velocity distribution map. Energy compensation corrects for energy attenuation. Since ultrasonic energy attenuates exponentially with propagation distance *d*, the theoretical attenuation is calculated for each ray path (a maximum gain threshold of 30 dB is set to avoid noise amplification), and this is used to inversely compensate the energy distribution map. The formula for attenuation compensation is: Where, : initial amplitude, : attenuation coefficient (approximately 0.2 dB / m in rock salt); Ground-penetrating radar (GPR) B-Scan data processing employs wavelet transform noise reduction, background removal, and gain adjustment. These steps help reduce noise and enhance the reflected signal of targets behind walls. The processed B-Scan data will serve as the basis for subsequent target detection and analysis behind walls, effectively improving data quality and detection accuracy, and providing reliable data support and analytical basis. Microseismic sensors detected density distribution maps and cumulative energy slope maps of rupture events (cavity regions). The STA algorithm was used for event authenticity verification, and moment tensor inversion was employed for quantitative energy analysis, thus achieving a refined interpretation of microseismic activity. The processed rupture event density distribution maps and cumulative energy slope maps will serve as the foundational data for subsequent backwall target detection and analysis, effectively improving data quality and the accuracy of backwall target detection, and providing reliable data support and analytical basis for subsequent work. The data processed by the above methods is designed from the dimensions of time reference unification and spatial registration to ensure accurate multimodal synchronization (time error < 1ms, spatial error < 0.05m). For time domain synchronization, dual-mode time synchronization redundancy design, timekeeping module compensation mechanism, and timestamp embedding strategy are adopted to achieve nanosecond-level time synchronization. For spatial alignment, multi-sensor coordinate system unification, multimodal sensor calibration, and spatial error control are implemented to ensure that both static registration error and dynamic tracking performance are less than 0.05m. Simultaneously, standardization processing is performed to achieve dimensional unification and regular distribution of multimodal features. Feature extraction: Features are extracted from the five types of preprocessed data. The ultrasonic velocity distribution map is extracted through velocity inversion. The formula for calculating the time difference between the transmitter and receiver is: in, , Let i be the i-th and j-th signals. Velocity inversion constructs a 3D mesh model, velocity Solve using the least squares method: in, For the propagation path length, The coefficient is the smoothing constraint coefficient. Calculate the spectral attenuation coefficient of the energy distribution map, and extract the envelope of the time-window signal using Hilbert transform: Calculate the instantaneous decay rate: in, The original time-domain signal, Represents the Hilbert transform. The instantaneous amplitude (envelope) of the analytic signal is represented by T, where T is the time window length and t0 is the start time of the signal. This represents the Fast Fourier Transform. Dielectric constant anomalies were extracted from ground-penetrating radar B-Scan data. An improved S-transform was used to extract the two-way travel time. The dielectric constant was calculated based on common midpoint (CMP) data, and the energy ratio was calculated along the survey line. The calculation formula is as follows: in, At the initial moment, Here, d represents the Ricker wavelet, and d represents the antenna spacing. To get around the time difference, This is the analysis window. Extracting the spatial density of rock salt fractures from the density distribution map of microseismic rupture events. Extract the energy fitting slope from the cumulative energy slope plot. The fracture density is calculated based on the STA algorithm. The calculation formula is as follows: in, The spatial density of the rock salt fracture is given by [reference needed], where h = 1m is the bandwidth, and K is the Epanechnikov kernel function. To prepare for Benioff's response, The slope is the energy fitting slope. 3. Data Expansion and Dataset Creation Phase The key features extracted from the three types of data were classified and subjected to forward modeling according to basic rules to expand the data. Then, they were organized into their respective labeled datasets as shown in Table 1, with 1000 data points in each group, totaling 3000. The data were input into a convolutional neural network model with three branches. Ground penetrating radar data input shape: 512×512×1, single-channel radar B-Scan image; ultrasonic data input shape: 256×256×2, dual-channel, containing wave velocity distribution and energy distribution maps respectively; microseismic data: input shape: 64×N×2, channels containing fracture density distribution map and energy accumulation slope map. Table 1 Dataset Detection instruments Data type a / number Data type b / number Total / Number ultrasound Wave velocity distribution map / 500 Energy distribution map / 500 1000 Ground penetrating radar B-Scan dielectric constant diagram / 1000 / 1000 Micro-vibration Fracture density distribution map / 500 Energy accumulation slope graph / 500 1000 4. Application Stage of Multi-Scale Convolutional Neural Networks Convolutional Neural Networks (CNNs) are a type of feedforward neural network with powerful feature extraction and image recognition capabilities, widely used in fields such as geomagnetic noise removal and tunnel wall hazard detection. CNNs typically consist of multiple convolutional layers, pooling layers, and fully connected layers. In this embodiment of the invention, a novel multi-scale convolutional neural network model (MSCNN) is constructed, including a three-branch convolutional neural network process. Training and performance analysis were performed on this model. The training process includes data processing, model construction, loss function selection, and the application of optimization algorithms. Evaluation was conducted based on the obtained training results. As shown in the figure, a schematic diagram of the convolutional neural network structure in this embodiment is provided, including an input layer, convolutional layers, pooling layers, fully connected layers, and an output layer, used for two application stages: intelligent identification of tunnel walls and risk assessment. During the training application process, to evaluate the model's performance, the dataset is divided into training and validation sets in an 8:2 ratio. The optimizer used in this invention is the Adam optimizer, the loss function is the joint loss function, the batch size is 64, the initial learning rate is 0.001, and the number of training epochs is 50. The MSCNN architecture is designed as follows: First, we design three input branches. Step (1): The first branch (ultrasound image, velocity distribution map, and energy distribution map) is a low-scale branch for local detail perception, which needs to capture the microstructure of the cavity. The number of convolutional kernels in each convolutional layer is 64. Input layer: Image size 256×256×2; Dilated convolutional layer: 3×3 kernel, dilation rate 3, 64 output channels, stride 1, automatic padding enabled to extract low-frequency edge and texture features, while perceiving a wider range of rock structures without increasing the number of parameters, expanding the receptive field to 7×7 (equivalent to a standard convolution of 13×13) and maintaining resolution; ReLU activation function is introduced to enhance feature representation; Max pooling layer: 2×2 kernel, stride 2, padding 0, output 128×128×64, achieving dimensionality reduction and anti-aliasing. To reduce noise, the feature map is compressed to 1 / 4 of its original size, preserving salient features while expanding the receptive field to 14×14. The convolutional layers use 3×3 kernels with 128 channels, a stride of 1, and padding of 1 to deepen features, encode more complex cavity morphologies (such as ellipsoidal and fissure-like shapes), and enhance the boundary contrast between the cavity and the surrounding rock, thus improving detail. Spatial pyramid pooling (SPP) achieves multi-scale fusion, capturing both local details and global distribution of the cavity, and is highly adaptable to subsequent fully connected layers. Regardless of the input feature map size, the SPP output has a fixed dimension (256 dimensions). Information about this branch of the convolutional neural network is shown in Table 2. Table 2 Information on the first branch of the convolutional neural network Layer Type Output Shape Number of Parameters Other Parameters Input Layer （64,256×256,2） 0 / Void Conv2D （64,256×256,128） 1216 F:64, K: 3×3A: ReLU MaxPooling2D （64,256×256,128） 0 S:1, P:1×1 Conv2D （64,256×256,128） 73856 F:64, K: 3×3A: ReLU Spatial Pyramid Pooling (SPP) （64,512×512,256） 98560 A: ReLU Step (2): Second branch (Ground-penetrating radar image, B-Scan data map), extract the mesoscale branch of the main contour of the cavity, which requires identifying the macroscopic morphology of the cavity. The number of convolutional kernels in each convolutional layer is 64. Input layer: Input a single-channel grayscale image of dielectric constant, size 512×512×1; First convolutional layer: Uses a 3×3 convolutional kernel, 64 channels, stride 1, padding 1, output 512×512×64; Second convolutional layer: Uses a 3×3 convolutional kernel, 64 channels, stride 1, padding 1, output 512×512×64, capable of recognizing cavity features. Max pooling layer: Uses a 2×2 pooling kernel, stride 2, maintaining the number of channels at 64, output: 256×256×64, which can expand the receptive field to 4 times the original image (4×4) and enhance robustness to cavity position shifts. Third convolutional layer: 3×3 convolutional kernel, 128 channels, stride 1, padding 1, final output size 256×256×128. This part, by increasing the number of channels, can recognize the macroscopic morphology of the cavity while capturing its microscopic morphology. Information about this branch of the convolutional neural network is shown in Table 3: Table 3 Information on the second branch of the convolutional neural network Layer Type Output Shape Number of Parameters Other Parameters Input Layer (64, 512×512,1) 0 1-Conv2D (64, 512×512,64) 1792 F:64, K: 3×3A: ReLU 2-Conv2D (64, 512×512,64) 36928 F:64, K: 3×3A: ReLU MaxPooling2D (64, 256×256,64) 0 S:2, P:2×2 Layer Type Output Shape Number of Parameters Other Parameters 3-Conv2D (64, 256×256,128) 73856 F:64, K: 3×3A: ReLU Step (3): The third branch (microseismic images, rupture event density distribution map, and energy cumulative slope map), a high-scale branch for global modeling, requires analysis of the spatial distribution of the cavity and its correlation with the stress field. Each convolutional layer has 64 convolutional kernels. Input layer size: 64×N×2, single sample number (64), N time steps, 2 features (rupture density, energy accumulation slope); LSTM layer (Long Short-Term Memory network): 64 hidden units, input dimension 2 (corresponding to 2 features), output mode: returns the hidden state of the final time step, which can capture the temporal dependence of microseismic events (gradual energy accumulation process) and filter random noise; Convolutional layer: 1×1 convolutional kernel, 1 channel, stride 1, padding 0; Fully connected layer: input dimension 128, output dimension 64, introduces the ReLU function to enhance nonlinearity, which can improve the model's ability to fit nonlinear relationships, mainly including the exponential relationship between cavity expansion rate and energy release; Add a self-attention mechanism to realize dynamic feature weighting, automatically focus on the time step of microseismic events in the cavity expansion surge stage, and establish the correlation of different time steps to provide dynamic risk signals for multimodal fusion. The information of this branch of convolutional neural network is shown in Table 4: Table 4 Information on the third branch of the convolutional neural network Layer Type Output Shape Number of Parameters Other Parameters Input Layer (64, 64×N,2) 0 / LSTM (64, 64×N,2) 33536 Hidden Number: 64 Conv2D (64, 64×N,128) 129 F:64, K: 1×1A: ReLU Dense (64, 64×64,64) 4160 A: ReLU Self-attention (64, 64×64,64) 16384 / Secondly, the multimodal data fusion design is implemented. After the data from the three-branch operation enters the fusion unit, it is necessary to ensure multimodal and multi-scale feature fusion, GPR alignment with ultrasound, and upsampling the second branch output from 256×256×128 to 512×512×256 to match the resolution of the original GPR image. The 64-dimensional microseismic feature vector is copied and expanded to 512×512×256 to adapt to the image spatial dimension. This part achieves data-image feature alignment and dimension matching. Squeeze-and-Excitation attention-weighted fusion is used, with the feature fusion formula as follows: in, , It is a fully connected layer. It is ReLU. Secondly, the output head features a layered design. It employs a two-branch design, corresponding to cavity identification and risk assessment respectively. Step (1): The first output branch is based on a cavity intelligent recognition method combining MSCNN and multimodal data. The inversion training mechanism of the ground-penetrating radar B-Scan data map is as follows: Figure 9 As shown. The multi-scale cavity recognition output head design uses 64 convolutional kernels in the convolutional layers. The input layer is 512×512×256, representing the fused multimodal feature map, with 256 channels representing cavity features at different scales. First, a 3×3 convolutional layer is used to further extract local details (such as cavity boundaries and pore distribution) and compress the 256-channel features to 128 channels, removing redundant information and improving computational efficiency. The feature map of each channel is standardized (mean=0, variance=1) and normalized to accelerate training convergence and ReLU activation, enhancing the ability to express complex boundary features of cavities and surrounding rocks. A convolutional layer is used again: a 1×1 convolutional kernel, 1 channel, stride 1, and padding 0. Then, the Sigmoid activation function and probability calculation function are used to achieve channel fusion, mapping the 128-channel features to a single channel. During this process, a feature fusion unit is added to integrate multi-scale information. Weights are assigned according to sensor reliability for weighted fusion. Based on the probability calculation formula, each pixel value is compressed to [0,1] (outputting the independent probability of each pixel, suitable for binary classification tasks (cavity / background)), representing the probability of cavity existence. A threshold of 0.7 is set, and pixels with probability P≥0.7 are marked as cavities. Finally, the cavity segmentation map is output through the Gumbel-Softmax function. The first output convolutional neural network information is shown in Table 5: Table 5 Information on the first output branch of the convolutional neural network Layer Type Output Shape Number of Parameters Other Parameters Input Layer (64,512×512,256) 0 / 1-Conv2D (64,512×512,128) 295040 F:64, K: 3×3A: ReLU 2-Conv2D (64,512×512,128) 129 F:64, K: 1×1A: ReLU Sigmoid (64,512×512,1) / Probability calculation function P, Gumbel-Softmax function Formula for calculating the probability P of a sinkhole occurrence: Among them, when the dielectric constant is abnormal When the dielectric constant of normal rock salt is >5 (approximately 8-10), the ultrasonic velocity... When the wave velocity is <2500 m / s (normal rock salt wave velocity is about 3500 m / s), and the microseismic event N ≥ 1 (normal rock salt N is 0), the probability of the occurrence of the cavity zone needs to be considered. Ultimately, the model is expected to achieve an accuracy rate of over 95% on the validation set, such as... Figure 9 As shown in the figure. This result demonstrates that a method for identifying retro-wall cavity by combining multimodal data fusion and convolutional neural networks can capture complex features at multiple scales and in multiple directions, effectively reducing the ambiguity of the fused feature map signal, and exhibiting higher recognition accuracy and robustness. The constructed MSCNN model demonstrates high effectiveness and reliability in the retro-wall cavity identification task, successfully extracting effective features from complex multimodal data, and providing an effective new scheme for detecting retro-wall anomalies. Figure 11 As shown, this embodiment provides a schematic diagram of the cavity identification results using a convolutional neural network based on a traditional single radar; as... Figure 12 As shown, this is a schematic diagram illustrating the cavity recognition results of a multi-scale convolutional neural network based on multimodal data fusion in an embodiment. Figure 12 As shown, a schematic diagram is provided to verify the actual situation of the solution cavity in this embodiment. Step (2): The second branch is a disaster assessment method based on the combination of MSCNN and multimodal data. First, the microseismic monitoring results are used as preliminary predictions for assessment. The ground-level seismic (GPR) system scans along the seismic network every 12 hours, while the ultrasonic system performs a full-section scan every 4 hours. The microseismic sensor provides continuous 24 / 7 monitoring, triggering a high-frequency GPR scan when an abnormal microseismic event occurs. (Abnormal event: When the microseismic system detects an event with energy > 1 × 10³ J, it sends a trigger command to both the GPR and ultrasonic systems; the GPR initiates a high-frequency scan, and the ultrasonic system switches to directional focusing mode.) Secondly, design the risk index function DRI: Where V: volume of the cavity (m³) 3 D: Distance from the tunnel wall (m), R: Expansion rate (cm / h), S: Surrounding rock strength (MPa), with S as the benchmark of 15 MPa. Cavity volume V: Cavity segmentation probability map thresholding ( Then calculate the volume of the connected region; the spread rate R: the slope of the microseismic energy accumulation (linear fitting). The surrounding rock strength S can be calculated based on the ultrasonic wave velocity. . Risk warning classification design: High risk level I (DRI≥0.75): immediate evacuation and area closure required; Medium risk level II (0.6<DRI<0.75): time-limited reinforcement and intensified monitoring required; Low risk level III (DRI≤0.6): routine inspection required. Secondly, the design of a multi-scale risk assessment output head. Input layer: Performs global average pooling (GAP) on the fused feature maps from the previous layer, preserving semantic information in the channel dimension while eliminating spatial dimension; this prevents overfitting and enhances feature robustness. The 256-dimensional representation extracts high-level features from 256 channels. First fully connected layer: Non-linearly maps the 256-dimensional features to 128 dimensions, learning complex relationships between features through a weight matrix; it also introduces the ReLU activation function to enhance the model's non-linear expressive power. Second fully connected layer: Maps the 128-dimensional features to a 3-dimensional output (corresponding to low / medium / high risk); it introduces the Softmax activation function to convert the 3-dimensional output into a probability distribution, satisfying... The final output value is a three-dimensional probability vector. Take the maximum probability p i The corresponding category is prediction result i. The output convolutional neural network information is shown in Table 6: Table 6 Information on the Second Output Branch Convolutional Neural Network Layer Type Output Shape Number of Parameters Other Parameters Input Layer (64,512×512,256) 0 / Dense (64,512×512,128) 32,896 A: ReLU Dense (64,512×512,3) 387 A: Softmax It should be understood that the specific implementation process of each module is described in the above method. This invention will not repeat the details here. The division of the functional modules is only for illustrative purposes. In some embodiments, some functional modules can be combined and some functional modules can be split. Each functional module can be implemented in software, hardware, or a combination of software and hardware. The software and hardware devices include, but are not limited to, general-purpose computer equipment, programmable gate arrays, digital signal processors, microprocessors and their corresponding programming software. Matters not covered in this invention are common knowledge. The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A cavity intelligent recognition method based on MSCNN and multimodal data combination, characterized in that, Includes the following steps: Step (1.1): The ultrasonic instrument detects the velocity distribution map and energy distribution map, the ground penetrating radar detects the B-Scan data map, and the microseismic sensor detects the density distribution map and energy cumulative slope map of the rupture event (cavity area); Step (1.2): Perform trend tomography on the ultrasonic velocity distribution map and reverse compensation on the energy distribution map; perform noise reduction and gain processing on the B-Scan data map of the ground penetrating radar; and perform STA algorithm and moment tensor inversion operations on the density distribution map and energy cumulative slope map of microseismic rupture events (cavity area), respectively. Step (1.3): Design the processed data from the dimensions of time reference unification and spatial registration to ensure accurate synchronization of multimodal data; at the same time, perform standardization processing to achieve dimensional unification and distribution regularity of multimodal feature data; Step (1.4): Perform feature extraction on the five types of data processed above. Extract velocity and energy decay rate from velocity distribution map and energy distribution map, extract dielectric constant outliers from B-Scan data map, and extract rock salt fracture density and energy cumulative slope from fracture event (cavity area) density distribution map and energy cumulative slope map. Step (1.5): Classify the key features extracted from the five types of data according to the basic rules and perform forward simulation to expand the data, and then organize them into a labeled dataset; Furthermore, the measured signals were processed according to steps (1.1)-(1.5). The processed energy distribution map, velocity distribution map, B-Scan data map, rupture event (cavity region) density distribution map, and energy cumulative slope map were imported into a multi-scale convolutional neural network (MSCNN) for cavity identification application in three branches.

2. The method as described in claim 1, characterized in that, The process of using three types of sensors to detect multimodal data of the cavity is as follows: Step (2.1): High-frequency ultrasound is used to automatically and periodically scan the tunnel through a regular grid. The software processes the data in real time and generates energy distribution maps and velocity distribution maps. Step (2.2): Use GR ground penetrating radar to continuously detect along the tunnel wall, and use host software to collect signals in real time and generate B-Scan data map; Step (2.3): Use explosion-proof micro-vibration sensors to monitor the entire tunnel in three dimensions, and generate a density distribution map and energy accumulation slope map of rupture events (cavity area) through all-weather data.

3. The method as described in claim 1, characterized in that, The process of data processing and standardization of signals from various sensors is as follows: Step (3.1): The ultrasonic velocity distribution map is subjected to travel-time tomography to obtain a velocity model for adjustment and optimization. The energy distribution map requires energy attenuation correction; the attenuation compensation A is calculated using the following formula: in, Initial amplitude, Attenuation coefficient (in rock salt) (Approximately 0.2 dB / m) Step (3.2): The B-Scan data image processing of the ground penetrating radar involves wavelet transform noise reduction, background removal, gain adjustment, and other operations. The processed B-Scan data image will serve as the basis for subsequent target detection and analysis behind walls. The formula for wavelet transform is: in, For reconstructing the signal, t is the time index. To capture high-frequency details, wavelet coefficients, The scaling factor indicating a low-frequency trend; Step (3.3): The density distribution map and energy accumulation slope map of microseismic rupture events are used to verify the authenticity of the events using the STA algorithm, and the energy is quantitatively analyzed using moment tensor inversion, so as to achieve a refined interpretation of microseismic activity; Step (3.4): Further, standardization operations are performed on the data processed in steps (1)-(3). For time domain synchronization, operations such as dual-mode time synchronization redundancy design, timekeeping module compensation mechanism and timestamp embedding strategy are adopted; for spatial domain alignment, operations such as multi-sensor coordinate system, multi-modal sensor calibration and spatial error control are adopted.

4. The method according to claim 1, characterized in that, The process of feature extraction and dataset establishment for various signals is as follows: Step (4.1): The ultrasonic velocity distribution map is extracted through velocity inversion, and the velocity is solved using the least squares method. in, For the transmit-receive pair time difference, , Let be the i-th, j-th channel signal, be the propagation path length, and λ be the smoothing constraint coefficient; Step (4.2): The ultrasonic energy distribution map is represented by the spectral attenuation coefficient. The envelope is extracted by performing a Hilbert transform on the time window signal, and then the instantaneous attenuation rate is calculated. : in, The original time-domain signal, Represents the Hilbert transform. The instantaneous amplitude (envelope) of the analytic signal is represented by T, where T is the time window length and t0 is the start time of the signal. Indicates Fast Fourier Transform; Step (4.3): The characteristics of the ground-penetrating radar B-Scan data map are reflected by the dielectric constant anomalies. The dielectric constant is calculated based on the common midpoint (CMP) data. : in, At the initial moment, Let c be the Ricker wavelet, c be the electromagnetic wave velocity, and d be the antenna spacing. To get around time differences; Step (4.4): Microseismic rupture event density distribution characteristics through rock salt rupture density This is reflected in the energy cumulative slope plot, which fits the slope through energy. The calculation formula is as follows: in, Let h be the spatial density of the rock salt fracture, h = 1m be the bandwidth, and K be the Epanechnikov kernel function. To prepare for Benioff The slope is the energy fitting slope.

5. The method according to claim 1, characterized in that, A novel multi-scale convolutional neural network model (MSCNN) is constructed, comprising a convolutional neural network flow with three input branches and two output branches, sequentially connected as input layer, convolutional layer, pooling layer, flattening layer, fully connected layer module, and output layer. During training and application, to evaluate the model's performance, the dataset is divided into training and validation sets in an 8:2 ratio. The optimizer used in this invention is the Adam optimizer, with a batch size of 64, an initial learning rate of 0.001, and 50 training epochs. The first branch (ultrasound image processing branch, 256×256×2) is primarily responsible for processing local detail features and enhancing the receptive field. Key information in ultrasound images often resides in specific shapes and textures. Dilated convolutional layers can expand the receptive field of feature extraction without sacrificing resolution, capturing a wider range of contextual information. Subsequent Spatial Pyramid Pooling (SPP) layers then perform multi-scale pooling on the feature maps, generating fixed-length outputs, effectively fusing features at different scales and making the network more robust to changes in target size and shape. The core function of this branch is multi-scale, high-resolution detail feature extraction. The second branch (Ground Penetrating Radar Image Processing Branch 512×512×1) has the core design idea of ​​performing deep feature abstraction and compression. Radar images are typically large and noisy. By sequentially stacking multiple convolutional and pooling layers, the network can progressively extract semantic features from low to high levels. Max pooling layers effectively reduce data dimensionality, enhance the translation invariance of features, and suppress noise. The cascading of multiple convolutional layers enables the network to learn complex and discriminative patterns in radar signals. This branch's function is to achieve hierarchical depth extraction and dimensionality reduction of features, providing high-level semantic information for classification. The third branch (microseismic image processing branch 64×N×2) is specifically designed for processing time-dependent microseismic signals. The LSTM layer, as a core component, effectively captures the long-term dynamic patterns and sequential correlations of the signal over time. Convolutional layers are used to extract local patterns from the time-series signal. The introduction of a self-attention mechanism allows the model to dynamically evaluate and weight the importance of features at different time points, focusing on the most critical time segments for the identification task, thereby enhancing the model's ability to capture key events in the time-series signal. The function of this branch is to achieve dynamic modeling of time-series features and focus on key information. After the three branches run, the data enters the fusion unit for multimodal and multiscale feature fusion, and the size of each image is adjusted to 512×512×256.

6. The method according to claim 5, characterized in that, Further design of the multi-scale cavity recognition output head (first output branch): This module takes the fused multimodal feature map as input, performs feature compression and enhancement through convolutional layers, and strengthens boundary features using activation functions. Finally, through channel fusion and probability calculation, it outputs the probability that each pixel is a cavity, thereby generating the final cavity segmentation map.

7. A risk assessment method based on the combination of MSCNN and multimodal data, characterized in that, The evaluation method design includes the following steps: Step (7.1): The ultrasonic instrument detects the velocity distribution map and energy distribution map, the ground penetrating radar detects the B-Scan data map, and the microseismic sensor detects the density distribution map and energy accumulation slope map of the rupture event (cavity area); Step (7.2): Perform trend tomography on the ultrasonic velocity distribution map and reverse compensation on the energy distribution map; perform noise reduction and gain processing on the B-Scan data map of the ground penetrating radar; and perform STA algorithm and moment tensor inversion operations on the density distribution map and energy cumulative slope map of microseismic rupture events (cavity area), respectively. Step (7.3): Design the processed data from the dimensions of time reference unification and spatial registration to ensure accurate synchronization of multimodal data; at the same time, perform standardization processing to achieve dimensional unification and distribution regularity of multimodal feature data; Step (7.4): Perform feature extraction on the five types of data processed above, extract velocity and energy decay rate from velocity distribution map and energy distribution map, extract dielectric constant outliers from B-Scan data map, and extract rock salt fracture density and energy cumulative slope from fracture event (cavity area) density distribution map and energy cumulative slope map; Step (7.5): Classify the key features extracted from the five types of data according to the basic rules and perform forward simulation to expand the data, and then organize them into a labeled dataset; Furthermore, the measured signals were processed according to steps (7.1)-(7.5). The processed energy distribution map, velocity distribution map, B-Scan data map, rupture event (cavity region) density distribution map, and energy cumulative slope map were then imported into a multi-scale convolutional neural network (MSCNN) for risk assessment application, with each branch consisting of three sub-branches.

8. According to claim 7, the feature is that, Microseismic monitoring results are used for preliminary assessment and prediction. When an abnormal microseismic event occurs, a high-frequency GPR scan is triggered, and the ultrasound signal is switched to directional focusing mode.

9. The method according to claim 7, characterized in that, Risk level assessment is achieved by designing a Risk Index Function (DRI): The risk index function integrates four parameters: cavity volume, the closest distance between the cavity and the tunnel wall, the cavity expansion rate, and the surrounding rock strength. Combined with a preset benchmark strength constant, it calculates a quantified risk index value. Based on the aforementioned risk index values, the risk of the cavity is divided into three warning categories: High risk level I (DRI≥0.75): immediate evacuation and area closure are required; Medium risk level II (0.6<DRI<0.75): time-limited reinforcement and intensified monitoring are required; Low risk level III (DRI≤0.6): routine inspections are required.

10. As claimed in claim 7, the feature is that, Design the output head for multi-scale risk assessment (second output branch): The classification module first performs global average pooling on the input feature map to compress spatial information while preserving channel semantics. Then, it uses two fully connected layers to perform non-linear mapping and dimensionality transformation of the features. Finally, it uses the Softmax function to output predicted probabilities for low, medium, and high risk levels, and selects the category corresponding to the highest probability as the classification result.

Images