Underground cavity recognition method and system combining multi-scale fusion and convolutional neural network

Abstract

The present invention discloses a method and system for identifying underground cavities that combines multiscale fusion with a convolutional neural network. The method comprises the following steps: first, using the Hilbert transform to process ground-penetrating radar (GPR) time series data to extract the instantaneous properties of the GPR signal: instantaneous amplitude (IA), instantaneous phase (IP), and instantaneous frequency (IF), thereby obtaining accurate local signal characteristics; second, applying the dual-tree complex wavelet transform to perform multiscale feature fusion on the IA, IP, and IF, effectively addressing the problem of insufficient geophysical feature information in the B-scan profile (BP). Finally, the fused dataset is input into a convolutional neural network for training. Training results show that the fusion method can capture multiscale and multidirectional features, effectively reduce the signal ambiguity of the BP, and has higher recognition accuracy and robustness, providing an effective new strategy for detecting underground cavity anomalies.

Description

Technical Field

[0001] The present invention relates to the technical field of underground anomaly detection using ground penetrating radar, and in particular to an underground cavity recognition method and system combining multi-scale fusion and convolutional neural network in ground penetrating radar data processing. Background Art

[0002] Detecting underground cavities is crucial for preventing ground subsidence, structural instability, and potential catastrophic failure. The formation of subsurface cavities can weaken the support of overlying materials, thereby compromising the integrity of underground structures and leading to subsidence, collapse, or damage to superjacent buildings and infrastructure (Ma et al., 2024). Therefore, nondestructive, efficient, and accurate detection of underground cavities is crucial (Hariri-Ardebili et al., 2023). As an efficient geophysical method for nondestructive testing (NDT), ground-penetrating radar (GPR) provides high-resolution imaging of subsurface anomalies by emitting electromagnetic waves and recording their reflections. It has been widely used in various fields, including geological exploration, environmental monitoring, and engineering construction (Yu et al., 2023; Feng et al., 2021; Sonkamble and Chandra, 2021). GPR plays a crucial role in identifying and characterizing these hidden anomalies (Li et al., 2022).

[0003] However, interpreting and analyzing GPR data remains a significant challenge in practical inspections. Subsurface structural materials are typically composed of a heterogeneous mixture of soil, rock, gravel, and other components. These materials vary significantly in volume proportion, spatial distribution, and physical properties, exhibiting multiphase, discrete, and random medium characteristics (Guo Shili et al., 2021). When high-frequency electromagnetic waves emitted by GPR propagate through the subsurface, the heterogeneity of the medium leads to complex GPR signal propagation paths, attenuation, scattering, and dispersion, which in turn reduces the resolution of the received signal and blurs the B-scan profile (Wang Hui et al., 2022). Experienced technicians are typically required to process complex GPR data to improve the signal-to-noise ratio and eliminate wavelet attenuation and dispersion. They utilize migration, deconvolution, filtering, and time-varying gain to reduce noise and enhance target reflections (Du Yuchuan et al., 2023). The efficiency and accuracy of data interpretation depend largely on the technician's expertise. However, ensuring timely and accurate interpretation becomes challenging when processing large amounts of data (Pryshchenko et al., 2022). Therefore, the effective integration of deep learning technology is of great significance for the automatic identification of subsurface anomalies using GPR ( Ma Changying et al., 2024 ). Summary of the Invention

[0004] The present invention aims to overcome the shortcomings of the prior art in detecting underground anomalies using ground penetrating radar and to provide a more efficient and accurate anomaly detection method.

[0005] The technical solutions provided by the present invention are as follows:

[0006] In a first aspect, a method for identifying underground cavities by combining multi-scale fusion with a convolutional neural network comprises the following steps:

[0007] Step (1): For the GPR time series data, extract the instantaneous properties of the GPR signal: instantaneous amplitude IA, instantaneous phase IP and instantaneous frequency IF;

[0008] Step (2): Dual-tree complex wavelet transform (DT-CWT) is used to perform feature fusion on the instantaneous amplitude (IA), instantaneous phase (IP) and instantaneous frequency (IF);

[0009] Step (3): The dataset IAF-datasets and its labels after the instantaneous attribute features are fused are input into the convolutional neural network model for training, and the trained convolutional neural network model based on multi-scale fusion is used to classify and identify the ground penetrating radar signals of underground cavities collected in real time;

[0010] During the recognition process, the ground penetrating radar signal of the underground cavity collected in real time is processed according to steps (1) and (2) to obtain instantaneous attribute feature fusion data and input it into the trained convolutional neural network model based on multi-scale fusion.

[0011] The label refers to the data having no anomalies or empty anomalies;

[0012] Furthermore, the instantaneous attribute extraction process of the GPR signal is as follows:

[0013] Step A1: Let the GPR time series data signal be x(t), and construct the complex signal z(t) of x(t):

[0014]

[0015] Where: x(t) is the ground penetrating radar time series data signal, which contains information about the underground structure; is the Hilbert transform of x(t), which represents the component related to the phase of x(t); i is the imaginary unit;

[0016] Perform Hilbert transform on x(t):

[0017]

[0018] Step A2: Based on the complex signal z(t) of the GPR time series data, the instantaneous properties of the GPR signal are calculated as follows:

[0019]

[0020] Among them: A(t), φ(t), They are the instantaneous amplitude IA, instantaneous phase IP and instantaneous frequency IF of the ground penetrating radar respectively.

[0021] Furthermore, the process of IA, IP and IF feature fusion using dual-tree complex wavelet transform is as follows:

[0022] The dual-tree complex wavelet transform (DT-CWT) utilizes a pair of complex filter banks to decompose and reconstruct signals. Compared to traditional wavelet transforms, the DT-CWT offers superior performance and feature extraction capabilities. The signal is first decomposed into its real and imaginary components, which are then processed using two wavelet transforms in a tree-like structure. This multi-tree structure better captures the multi-scale and multi-directional characteristics of the signal, enabling more accurate signal analysis and feature extraction.

[0023] DT-CWT consists of two main steps: decomposition and reconstruction. In the decomposition process, the signal is first filtered through a pair of complex filter banks, and then the wavelet transform of two tree structures is performed separately. In the reconstruction process, the results of each tree are inversely transformed and then combined into the final signal.

[0024] Step B1: Assuming that the GPR time series signal x(t) is a real-valued signal, where t represents the time index, construct a complex wavelet expression for x(t);

[0025] x(t)=ψ h (t)+iψ g (t)

[0026] Among them, ψ h (t) and ψ g (t) are real tree wavelet and complex tree wavelet respectively, i is the imaginary unit;

[0027] Step B2: Transform the wavelet coefficients and scaling coefficients to the dual-tree complex wavelet transform through the real and imaginary tree transforms:

[0028] Wavelet coefficients and scaling coefficients of the real dual-tree complex wavelet transform:

[0029]

[0030] in, and are the wavelet coefficients and scale coefficients of the real part dual-tree complex wavelet transform, j is the scale factor, J is the maximum scale factor, and k is the wavelet filter length;

[0031] Wavelet coefficients and scaling coefficients of the imaginary dual-tree complex wavelet transform:

[0032]

[0033] Step B3: Obtain the wavelet coefficient d that captures high-frequency details based on the wavelet coefficient and scale coefficient of the dual-tree complex wavelet transform j (t) and the scaling factor c representing the low-frequency trend J (t):

[0034]

[0035] Step B4: Wavelet coefficients d that capture high-frequency details j (t) and the scaling factor c representing the low-frequency trend J (t) is reconstructed, and the reconstructed signal is as follows:

[0036]

[0037] in, Represents the reconstructed signal.

[0038] After steps (1) and (2), the reconstructed signal is processed Multi-scale feature fusion is performed by calculating the difference in the corresponding wavelet coefficient positions in IA, IP, and IF to determine their fusion weights. Specifically, the fusion weight reflects the contribution of each instantaneous attribute at the wavelet coefficient position, and the difference in the three instantaneous attribute coefficients forms a new linear combination weight. Finally, these fusion weights are substituted into the above formula to calculate the fused wavelet coefficients. The fused signal is then subjected to the inverse dual-tree complex wavelet transform (IDT-CWT) to obtain the final reconstructed signal. The acquisition of this signal is a key part of this solution, effectively fusing feature information from different scales and directions, laying the foundation for subsequent model training.

[0039] Furthermore, the convolutional neural network structure includes an input layer, a convolution-pooling combination module, a flatten layer, a fully connected layer module and an output layer connected in sequence;

[0040] The convolution-pooling combination module consists of two parts. Each convolution combination contains two convolution blocks, a maximum pooling layer, a batch normalization layer, and a dropout layer. Each convolution block is followed by a ReLU activation function module. The number of convolution kernels in the first and second parts of the convolution block is 32 and 64 respectively.

[0041] The fully connected layer module includes two fully connected layer units, each of which includes a fully connected layer and a Dropout layer, and a ReLU activation function module is added after each fully connected layer;

[0042] The output layer uses the softmax function.

[0043] Furthermore, during the training process, the Adam optimizer was used, and the cross entropy loss function was used to measure the performance of the network in the classification task. The batch size was set to 64, and the initial learning rate was 0.001.

[0044] The second aspect is an underground cavity recognition system that combines multi-scale fusion and convolutional neural networks, including:

[0045] Instantaneous attribute extraction module: uses Hilbert transform to process GPR time series data and extracts the instantaneous attributes of GPR signals: instantaneous amplitude IA, instantaneous phase IP and instantaneous frequency IF;

[0046] Feature fusion module: Dual-tree complex wavelet transform (DT-CWT) is used to fuse the instantaneous amplitude (IA), instantaneous phase (IP) and instantaneous frequency (IF);

[0047] Model training and recognition module: The IAF-datasets dataset, which is a fusion of instantaneous attribute features, and its labels are input into the convolutional neural network model for training. The trained convolutional neural network model based on multi-scale fusion is used to classify and identify the ground penetrating radar signals of underground cavities collected in real time.

[0048] During the recognition process, the ground penetrating radar signal of the underground cavity collected in real time is processed according to the instantaneous attribute extraction module and the feature fusion module, and the instantaneous attribute feature fusion data is input into the trained convolutional neural network model based on multi-scale fusion.

[0049] Furthermore, the extraction process of the instantaneous attribute extraction module is as follows:

[0050] Step A1: Let the GPR time series data signal be x(t), and construct the complex signal z(t) of x(t):

[0051]

[0052] Where: x(t) is the ground penetrating radar time series data signal, which contains information about the underground structure; is the Hilbert transform of x(t), which represents the component related to the phase of x(t); i is the imaginary unit;

[0053] Perform Hilbert transform on x(t):

[0054]

[0055] Step A2: Based on the complex signal z(t) of the GPR time series data, the instantaneous properties of the GPR signal are calculated as follows:

[0056]

[0057] Among them: A(t), φ(t), They are the instantaneous amplitude IA, instantaneous phase IP and instantaneous frequency IF of the ground penetrating radar respectively.

[0058] Furthermore, the feature fusion module uses dual-tree complex wavelet transform to fuse the features of IA, IP and IF as follows:

[0059] Step B1: Assuming that the GPR time series signal x(t) is a real-valued signal, where t represents the time index, construct a complex wavelet expression for x(t);

[0060] x(t)=ψ h (t)+iψ g (t)

[0061] Among them, ψ h (t) and ψ g (t) are real tree wavelet and complex tree wavelet respectively, i is the imaginary unit;

[0062] Step B2: Transform the wavelet coefficients and scaling coefficients to the dual-tree complex wavelet transform through the real and imaginary tree transforms:

[0063] Wavelet coefficients and scaling coefficients of the real dual-tree complex wavelet transform:

[0064]

[0065] in, and are the wavelet coefficients and scale coefficients of the real part dual-tree complex wavelet transform, j is the scale factor, J is the maximum scale factor, and k is the wavelet filter length;

[0066] Wavelet coefficients and scaling coefficients of the imaginary dual-tree complex wavelet transform:

[0067]

[0068] Step B3: Obtain the wavelet coefficient d that captures high-frequency details based on the wavelet coefficient and scale coefficient of the dual-tree complex wavelet transform j (t) and the scaling factor c representing the low-frequency trend J (t):

[0069]

[0070] Step B4: Wavelet coefficients d that capture high-frequency details j (t) and the scaling factor c representing the low-frequency trend J (t) is reconstructed, and the reconstructed signal is as follows:

[0071]

[0072] in, Represents the reconstructed signal.

[0073] Furthermore, the convolutional neural network structure of the model training and recognition module includes an input layer, a convolution-pooling combination module, a flatten layer, a fully connected layer module, and an output layer connected in sequence;

[0074] The convolution-pooling combination module consists of two parts. Each convolution combination contains two convolution blocks, a maximum pooling layer, a batch normalization layer, and a dropout layer. Each convolution block is followed by a ReLU activation function module. The number of convolution kernels in the first and second parts of the convolution block is 32 and 64 respectively.

[0075] The fully connected layer module includes two fully connected layer units, each of which includes a fully connected layer and a Dropout layer, and a ReLU activation function module is added after each fully connected layer;

[0076] The output layer uses the softmax function.

[0077] In a third aspect, a computer-readable storage medium stores a computer program, which is called by a processor to execute: the steps of the above-mentioned underground cavity identification method combining multi-scale fusion and convolutional neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly describes the accompanying drawings required for use. It is apparent that the accompanying drawings described below represent only some embodiments of the present invention. A person skilled in the art can also derive other accompanying drawings based on these drawings without inventive effort.

[0079] Figure 1 It is a technical route diagram of a method for fusing instantaneous attribute features of a ground penetrating radar in one embodiment of the present invention;

[0080] Figure 2 is a flow chart of ground penetrating radar data preprocessing in one embodiment of the present invention;

[0081] Figure 3 Schematic diagram of ground penetrating radar data preprocessing in one embodiment of the present invention, where (a) is the B-scan Profile and (b) is the Preprocessed Profile;

[0082] Figure 4 is a schematic cross-sectional view of a ground penetrating radar B-scan according to an embodiment of the present invention;

[0083] Figure 5 is a schematic diagram of the instantaneous amplitude of a ground penetrating radar in one embodiment of the present invention;

[0084] Figure 6 is a schematic diagram of the instantaneous phase of a ground penetrating radar in one embodiment of the present invention;

[0085] Figure 7 is a schematic diagram of the instantaneous frequency of a ground penetrating radar in one embodiment of the present invention;

[0086] Figure 8 2. It is a schematic diagram of signal decomposition and reconstruction of dual-tree complex wavelet transform in one embodiment of the present invention;

[0087] Figure 9 This is a feature fusion flow chart of dual-tree complex wavelet transform in one embodiment of the present invention;

[0088] Figure 10 This is a feature fusion result diagram of wavelet transform in one embodiment of the present invention;

[0089] Figure 11 This is a feature fusion result diagram of the dual-tree complex wavelet transform in one embodiment of the present invention;

[0090] Figure 12 is a schematic diagram of a convolutional neural network structure in one embodiment of the present invention;

[0091] Figure 13 1 is a diagram showing the training results of a convolutional neural network using a traditional input method according to an embodiment of the present invention;

[0092] Figure 14 1 is a diagram showing the training results of a convolutional neural network using an instantaneous attribute feature fusion input method according to an embodiment of the present invention; DETAILED DESCRIPTION

[0093] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0094] The embodiment of the present invention provides an underground cavity identification method combining multi-scale fusion and convolutional neural network, and its technical route is as follows: Figure 1As shown in the figure, it includes the feature extraction stage (Feature extraction stage): forward modeling, data preprocessing, instantaneous attributes features extraction; feature fusion stage (Feature fusion Stage): applying dual-tree complex wavelet transform (Applying DT-CWT), calculating wavelet coefficients (Calculating wavelet coefficients), multi-scale feature fusion (Multi-scale features fusion); application stage (Application stage): dataset creation (IAF-datasets creation), model training (CNN Model architecture building and training), verification and evaluation (Evaluation, verification and application);

[0095] Specifically include:

[0096] Step (1): Use Hilbert transform to process the GPR time series data and extract the instantaneous properties of the GPR signal: instantaneous amplitude IA, instantaneous phase IP and instantaneous frequency IF;

[0097] Step (2): Dual-tree complex wavelet transform (DT-CWT) is used to perform feature fusion on the instantaneous amplitude (IA), instantaneous phase (IP) and instantaneous frequency (IF);

[0098] Step (3): The dataset IAF-datasets and its labels after the instantaneous attribute features are fused are input into the convolutional neural network model for training, and the trained convolutional neural network model based on multi-scale fusion is used to classify and identify the ground penetrating radar signals of underground cavities collected in real time;

[0099] During the recognition process, the ground penetrating radar signal of the underground cavity collected in real time is processed according to steps (1) and (2) to obtain instantaneous attribute feature fusion data and input it into the trained convolutional neural network model based on multi-scale fusion.

[0100] This recognition method can enhance the detection and identification of anomalies using ground-penetrating radar (GPR). The feature fusion method involves extracting and fusing instantaneous attribute features from GPR. Finally, the fused dataset is fed into a convolutional neural network for training. Training results demonstrate that this feature fusion method can effectively classify and identify underground anomalies with high accuracy. In this embodiment of the present invention, feature fusion of GPR time series data can effectively enhance the detection of underground anomalies.

[0101] First, instantaneous attribute features, such as instantaneous amplitude, instantaneous phase, and instantaneous frequency, are extracted from the GPR signal. These features effectively capture the reflection characteristics and frequency response of anomalies. The instantaneous amplitude represents the change in signal strength, highlighting the reflection characteristics of anomalies. The instantaneous phase reflects the phase change of the signal, helping to identify interfaces between different materials and structures. The instantaneous frequency displays the frequency change of the signal, enabling the differentiation of anomalies with different frequency responses.

[0102] like Figure 2 As shown, a flow chart of ground penetrating radar data preprocessing in this embodiment is provided, which specifically includes the following steps:

[0103] First, the data module includes data loading (GPR Data loading) and data parameter setting (Data Parameter Setting).

[0104] During the data loading phase, the GPR data file is loaded and radar waveform data is acquired. The data parameter setting phase includes setting parameters such as the antenna center frequency (AntenaFrequency), relative permittivity (Relative Permittivity), time window size (TimeWindow), sampling interval (Sampling Interval), and the profile start and end points (StartPoint and EndPoint).

[0105] The image processing module generates B-scan profiles, two-dimensional representations of radar waveform data in time and range coordinates, to visually demonstrate the structural characteristics of the subsurface medium. Subsequent preprocessing steps include direct wavelet removal, background removal, gain adjustment, and two-dimensional filtering, which help reduce noise and enhance the reflected signal from underground targets. The processed B-scan profiles (GPRB-scan profiles) serve as the foundation for subsequent underground target detection and analysis, effectively improving data quality and detection accuracy, and providing reliable data support and analysis.

[0106] However, in the context of inhomogeneous media, the B-scan profile of the ground penetrating radar will encounter some limitations, mainly reflection errors, signal attenuation and reduced resolution. The heterogeneity of the medium will affect the propagation path of the electromagnetic wave, resulting in refraction and reflection effects, which may produce artifacts and errors in the imaging. In addition, scattering and absorption in the medium will also cause signal attenuation, reducing the detection depth and sensitivity of the ground penetrating radar, thereby limiting the detection of underground targets. These factors largely cause the ambiguity of the B-scan profile, making data interpretation and analysis complicated. Although attempts have been made to preprocess the ground penetrating radar data to mitigate these effects, the inherent complexity of the medium often makes the ambiguity persist for a long time. Figure 3 (a) shows a schematic diagram of the original B-scan cross-section of the ground penetrating radar data in this embodiment. Figure 3 (b) shows a schematic diagram of the B-scan profile of the ground penetrating radar data preprocessing in this embodiment, where the horizontal axis is position (Position), in meters (m), and the vertical axis is depth (Deepth), in meters (m).

[0107] Scattering and absorption in the inhomogeneous medium background will reduce the spatial resolution of imaging, making it difficult to distinguish the boundaries and details of underground targets. In order to solve these limitations, the Hilbert transform is introduced to extract instantaneous features. Figure 4 The B-scan profile shown in the figure has a horizontal axis representing position (in meters) and a vertical axis representing depth (in meters), which shows the distribution of GPR data in depth and position dimensions, revealing its inherent ambiguity and complexity.

[0108] The Hilbert transform is used to process the GPR time series data. Its essence is to convolve the time series x(t) with 1 / πt, which can extract the instantaneous characteristics of the electromagnetic wave such as amplitude, phase and frequency to obtain the instantaneous properties of the GPR signal.

[0109] Step A1: Let the GPR time series data signal be x(t), and construct the complex signal z(t) of x(t):

[0110]

[0111] Where: x(t) is the ground penetrating radar time series data signal, which contains information about the underground structure; is the Hilbert transform of x(t), which represents the component related to the phase of x(t); i is the imaginary unit;

[0112] Perform Hilbert transform on x(t):

[0113]

[0114] Step A2: Based on the complex signal z(t) of the GPR time series data, the instantaneous properties of the GPR signal are calculated as follows:

[0115]

[0116] Among them: A(t), φ(t), They are the instantaneous amplitude IA, instantaneous phase IP and instantaneous frequency IF of the ground penetrating radar respectively.

[0117] The ground penetrating radar signal is processed by Hilbert transform to extract its instantaneous attribute features: IA, IP, IF. These features can better reflect the changes in time and frequency of the signal. Figures 5-7 As shown, the instantaneous amplitude diagram of the ground penetrating radar in this embodiment is provided, showing the distribution of the instantaneous amplitude in the depth and position dimensions; the instantaneous phase diagram of the ground penetrating radar shows the distribution of the instantaneous phase in the depth and position dimensions; the instantaneous frequency diagram of the ground penetrating radar shows the distribution of the instantaneous frequency in the depth and position dimensions, the horizontal axis is position (Position), the unit is meter (m), the vertical axis is depth (Deepth), the unit is meter (m). Figure 4As shown in the figure, the B-scan profile of the ground-penetrating radar in this embodiment is severely affected in inhomogeneous media, making it difficult to clearly display the morphology and distribution of anomalies. The inhomogeneity of the medium complicates the signal propagation path, and signal attenuation and scattering exacerbate signal blur, making it difficult to distinguish the target's boundaries. Furthermore, beam spreading and reduced resolution further blur the image, making the detection and identification of anomalies difficult. The instantaneous attributes IA, IP, and IF are obtained through the Hilbert transform. IA reflects the reflection intensity of the ground-penetrating radar signal and is proportional to the square root of the total signal energy. It can be seen that the strong energy in shallow layers and weak energy in deeper layers complicate the output and display of B-scan profile information. IP reflects the continuity of the event axis in the ground-penetrating radar signal. In isotropic homogeneous media, the phase of high-frequency electromagnetic waves is continuous, but underground media are typically inhomogeneous, and phase changes occur when passing through anomalies, regardless of the signal energy level. IF reflects the temporal rate of change of IF. When high-frequency electromagnetic waves pass through changing lithologies or strata, the frequency of the electromagnetic waves will change significantly. Therefore, IA and IF can determine the approximate location of underground anomalies, while IP can roughly describe their contours.

[0118] The dual-tree complex wavelet transform (DT-CWT) is used to fuse the features of IA, IP and IF. The dual-tree complex wavelet transform is characterized by the use of a pair of complex filter banks to decompose and reconstruct the signal. Compared with the traditional wavelet transform, DT-CWT has better performance and feature extraction capabilities. The signal is first decomposed into two parts, the real part and the imaginary part, and then processed by two tree-structured wavelet transforms respectively. This multi-tree structure can better capture the multi-scale and multi-directional characteristics of the signal, thereby achieving more accurate signal analysis and feature extraction. Figure 8 As shown, a schematic diagram of signal decomposition and reconstruction of the dual-tree complex wavelet transform in this embodiment is provided, which is a process of decomposing the signal into different scales and directions and finally reconstructing the signal.

[0119] DT-CWT consists of two main steps: decomposition and reconstruction. In the decomposition process, the signal is first filtered through a pair of complex filter banks, and then the wavelet transform of two tree structures is performed separately. In the reconstruction process, the results of each tree are inversely transformed and then combined into the final signal.

[0120] Step B1: Assuming that the GPR time series signal x(t) is a real-valued signal, where t represents the time index, construct a complex wavelet expression for x(t);

[0121] x(t)=ψ h (t)+iψ g (t) (4)

[0122] Among them, ψh (t) and ψ g (t) are real tree wavelet and complex tree wavelet respectively, i is the imaginary unit;

[0123] Step B2: Transform the wavelet coefficients and scaling coefficients to the dual-tree complex wavelet transform through the real and imaginary tree transforms:

[0124] Wavelet coefficients and scaling coefficients of the real dual-tree complex wavelet transform:

[0125]

[0126] in, and are the wavelet coefficients and scale coefficients of the real part dual-tree complex wavelet transform, j is the scale factor, J is the maximum scale factor, and k is the wavelet filter length;

[0127] Wavelet coefficients and scaling coefficients of the imaginary dual-tree complex wavelet transform:

[0128]

[0129] Step B3: Obtain the wavelet coefficient d that captures high-frequency details based on the wavelet coefficient and scale coefficient of the dual-tree complex wavelet transform j (t) and the scaling factor c representing the low-frequency trend J (t):

[0130]

[0131] Step B4: Wavelet coefficients d that capture high-frequency details j (t) and the scaling factor c representing the low-frequency trend J (t) is reconstructed, and the reconstructed signal is as follows:

[0132]

[0133] in, Represents the reconstructed signal.

[0134] As an effective signal processing technology, the dual-tree complex wavelet transform (DT-CWT) has been widely used in image compression, texture analysis, image restoration and other fields. By leveraging the advantages of complex wavelet transform and dual-tree structure, DT-CWT can more accurately capture the time-frequency characteristics of the signal and has good directional selectivity. In GPR data processing, instantaneous attribute characteristics are crucial for describing underground structure and disease characteristics. Traditional wavelet transform has limitations in extracting these features, and DT-CWT can overcome these problems. Utilizing the characteristics of complex wavelet transform, compared to WT, DT-CWT adopts a dual-tree structure to decompose the signal in different directions, which can more comprehensively capture the time-frequency characteristics of the signal and is particularly suitable for ground penetrating radar signals in inhomogeneous media.

[0135] When using the dual-tree complex wavelet transform (DT-CWT) for feature fusion, the input images are first converted to grayscale images (Input images reading Convert to grayscale), and feature extraction is performed using the DT-CWT (Execute DT-CWT). According to equations (5) and (6), wavelet coefficients of different scales and directions are extracted from each image. Then, for each wavelet coefficient position, the difference between the corresponding wavelet coefficients in the three images is calculated to determine their fusion weights (Execute fusion). Specifically, the fusion weight reflects the contribution weight of each image at that position through the coefficient difference of the three images, and then these weights are linearly combined to form a new fusion weight. Finally, these fusion weights are substituted into equation (7) to calculate the fused wavelet coefficients. Next, the fused high-frequency wavelet coefficients are synthesized using the inverse dual-tree complex wavelet transform (Inverse DT-CWT), and the final fused image is reconstructed using the average value of the low-frequency wavelet coefficients of the three images (Low-frequency subband fusion). This method can effectively capture and fuse the detailed time-frequency features of multiple images, and is particularly suitable for application scenarios in complex media such as ground penetrating radar (GPR) data processing. Figure 9 As shown, a feature fusion flow chart of the dual-tree complex wavelet transform in this embodiment is provided.

[0136] like Figure 10 As shown, the wavelet transform feature fusion of the instantaneous properties of the ground penetrating radar in this embodiment is provided, such as Figure 11As shown, the dual-tree complex wavelet transform feature fusion of the instantaneous attributes of the ground penetrating radar in this embodiment is provided. The feature fusion image after wavelet transform still has noise points, resulting in low resolution and information loss, while DT-CWT can fully and effectively fuse the extracted instantaneous attribute features to obtain a more comprehensive and accurate description of the underground structure. The response characteristics of the abnormal body can be better displayed. By comprehensively utilizing the fusion image of various feature information, the complexity and diversity of the underground structure can be better reflected, and pseudo-features and noise interference can be effectively suppressed, thereby improving the signal processing effect and the ability to identify underground diseases.

[0137] Convolutional Neural Network (CNN) is a feedforward neural network that is widely used in fields such as geomagnetic noise removal, natural blasting identification, and internal detection of tunnel linings. CNN is usually composed of multiple layers of convolutional layers and pooling layers stacked alternately, as well as fully connected layers. In the embodiment of the present invention, a new convolutional neural network model was constructed, and training and effect analysis were performed on the model. The training process includes data preprocessing, model construction, loss function selection, and application of optimization algorithms, and evaluation was performed based on the obtained training results. Figure 12 As shown, a schematic diagram of the convolutional neural network structure in this embodiment is provided, including an input layer, a convolution-pooling combination, a fully connected layer and an output layer, which is used to process and classify ground penetrating radar data features.

[0138] The convolutional neural network structure includes an input layer, a convolution-pooling combination module (Conv-Pooling Combination), a flatten layer, a fully connected layer module and an output layer connected in sequence;

[0139] The convolution-pooling combination module consists of two parts. Each convolution combination contains two convolution blocks, a maximum pooling layer, a batch normalization layer, and a dropout layer. Each convolution block is followed by a ReLU activation function module. The number of convolution kernels in the first and second parts of the convolution block is 32 and 64 respectively.

[0140] The fully connected layer module includes two fully connected layer units, each of which includes a fully connected layer and a Dropout layer, and a ReLU activation function module is added after each fully connected layer;

[0141] The output layer uses the softmax function.

[0142] Multiple convolution-pooling modules extract multi-level abstract features from the input data. At the network's input, the feature-fused dataset is processed and fed into the convolutional layer. It is then further processed through the activation function layer (ReLU) and the pooling layer before being output to the fully connected layer. In the fully connected layer, the output of the previous layer is used as input and calculated using the ReLU activation function to enhance the model's nonlinear representation capabilities.

[0143] To prevent overfitting during training, Batch Normalization and Dropout techniques are added after the convolutional and pooling layers to improve the model's training stability and generalization capabilities in underground cavity identification tasks. Specifically, Batch Normalization, applied after each convolutional layer, normalizes activation values ​​to speed up training and improve stability; Dropout, on the other hand, randomly drops neurons, reducing the model's reliance on specific nodes and thus enhancing its generalization capabilities. Table 1 shows the parameters of the convolutional neural network used in this example.

[0144] Table 1 Convolutional neural network parameter information

[0145]

[0146] During training, the input image size was uniformly set to 128×128 pixels. This size effectively balances computational resource requirements with feature extraction capabilities. A total of 27,000 images were collected, covering a wide range of underground cavity characteristics to ensure the model's generalization across diverse scenarios. To evaluate the model's performance, the dataset was split into a training set (21,600 images) and a validation set (5,400 images) in an 8:2 ratio. The model was trained using the Adam optimizer and the cross-entropy loss function with a batch size of 64, an initial learning rate of 0.001, and 80 epochs.

[0147] As shown in Table 2, the data set information in this embodiment is provided.

[0148] Table 2 Dataset information

[0149]

[0150] The Adam optimizer was chosen for optimization because it combines the advantages of momentum and adaptive learning rates, enabling faster convergence and adapting to varying gradients. Furthermore, the cross-entropy loss function was used to measure model performance in classification tasks, making it particularly suitable for classification problems. The batch size was set to 64, which effectively utilizes the GPU's computing power during training while avoiding memory overload.

[0151] The initial learning rate is 0.001, which has been experimentally verified to be an optimal parameter and helps maintain a stable learning process in the early stages of training. During the training process of 80 epochs, the performance of the validation set is continuously monitored to prevent overfitting and ensure the model's performance on unseen data.

[0152] Finally, the model achieved an accuracy of 97.8% on the validation set. This result shows that an underground cavity recognition method that combines multi-scale fusion with convolutional neural networks can capture multi-scale and multi-directional features, effectively reduce the ambiguity of B-scan profile (BP) signals, and has higher recognition accuracy and robustness. The constructed CNN model has high effectiveness and reliability in underground cavity recognition tasks, and can successfully extract useful features from complex GPR data, providing an effective new strategy for underground cavity anomaly detection. Figure 13 As shown, the convolutional neural network training result diagram of the traditional input method in this embodiment is provided, with the Acc Curve (accuracy curve) of training and verification on the left and the Loss Curve (loss curve) of training and verification on the right; Figure 14 As shown, a convolutional neural network training result diagram of the instantaneous attribute feature fusion input method in the embodiment is provided, with the Acc Curve (accuracy curve) of training and verification on the left and the Loss Curve (loss curve) of training and verification on the right;

[0153] Table 3 provides a classification report for this example. The metrics listed in Table 3, such as precision, recall, F1 score, and per-class support, are also insightful and help provide a comprehensive understanding of the model's performance across different categories. These metrics are crucial for thoroughly examining the model's ability to accurately classify different types of anomalies. In the field of machine learning, meticulous evaluation of classification model performance is crucial.

[0154] Table 3 Classification report

[0155]

[0156] In this process, precision, recall, F1 score, and support are commonly used key metrics. Precision refers to the proportion of samples correctly classified as positive among all samples classified as positive. In other words, among all samples predicted as positive, how many are actually positive? Precision provides important information about the accuracy of the classifier. Precision can be expressed as follows:

[0157]

[0158] Among them, TP stands for True Positives and FP stands for False Positives.

[0159] Recall is the proportion of samples that are correctly classified as positive among all samples that are actually positive. In other words, how many samples are correctly predicted to be positive among all samples that are actually positive? This measures the classifier's ability to correctly identify positive samples and provides an important measure of the classifier's integrity. Recall can be expressed as follows:

[0160]

[0161] Among them, TP represents true positives and FN represents false negatives.

[0162] The F1 score is the harmonic mean of precision and recall, combining precision and recall to comprehensively consider the accuracy and completeness of the classifier. This harmonic mean provides a quantitative assessment of the classifier's overall performance and is particularly useful in situations with unbalanced class distributions. The F1 score can be expressed as follows:

[0163]

[0164] In addition, support indicates the actual number of occurrences of each category in the dataset, which is a direct measure of the sample size of the category in the dataset. The higher the support, the more frequently the category appears in the dataset. Including these detailed indicators emphasizes the comprehensiveness of the evaluation process and provides significant insights for further improvement and optimization of the model.

[0165] The high accuracy achieved by the feature fusion method in this example fully demonstrates its potential for application in anomaly detection tasks. These results highlight the effectiveness of fusing ground-penetrating radar instantaneous attribute feature datasets (IAF-datasets), outperforming individual B-scan profile datasets across various evaluation metrics. This approach improves accuracy and simplifies the training process, highlighting the value of multimodal data fusion techniques in improving model capabilities.

[0166] This embodiment successfully improves the accuracy and robustness of identifying underground anomalies in non-destructive testing by applying a combination of dual-tree complex wavelet transform (DT-CWT) and convolutional neural network (CNN). By performing Hilbert transform on GPR data to extract the instantaneous properties of the signal, and using dual-tree complex wavelet transform for feature fusion, the challenges posed by the heterogeneity of the underground medium are overcome, and more accurate local signal features are extracted. This method effectively solves the problem of fuzzy B-scan profile signals and provides a new way to accurately detect and identify underground anomalies. The research results show that compared with traditional methods, this method has higher accuracy and robustness in anomaly identification, providing important technical support and application prospects for safety assessment and management.

[0167] A method for identifying underground voids that combines multiscale fusion with convolutional neural networks can improve the automatic identification and interpretation of underground anomalies, providing a reliable technical support for underground target detection. By extracting local features and using the Hilbert transform to extract instantaneous attribute features (IA, IP, and IF) from GPR signals, this method effectively addresses the problem of radar signals being affected by the inhomogeneous underground medium, resulting in complex signal propagation paths, attenuation, and scattering, which can blur B-scan profile (BP) signals. To address the lack of geophysical feature information in the sample, the instantaneous attributes are fused using the Dual-Tree Complex Wavelet Transform (DT-CWT), fully capturing the shared information between the IA, IP, and IF while preserving the local structural characteristics of the GPR data. Furthermore, the fused IAF dataset provides comprehensive and accurate anomaly feature information. This fused dataset can be trained with a convolutional neural network (CNN) to achieve better classification performance. Results demonstrate that the proposed method outperforms traditional methods, achieving a recognition accuracy of 97.8%. This provides a new approach for the rapid and accurate identification, detection, and interpretation of underground void anomalies.

[0168] In some embodiments, the present invention further provides an underground cavity identification system combining multi-scale fusion and convolutional neural network, comprising:

[0169] Instantaneous attribute extraction module: uses Hilbert transform to process GPR time series data and extracts the instantaneous attributes of GPR signals: instantaneous amplitude IA, instantaneous phase IP and instantaneous frequency IF;

[0170] Feature fusion module: Dual-tree complex wavelet transform (DT-CWT) is used to fuse the instantaneous amplitude (IA), instantaneous phase (IP) and instantaneous frequency (IF);

[0171] Model training and recognition module: The IAF-datasets dataset, which is a fusion of instantaneous attribute features, and its labels are input into the convolutional neural network model for training. The trained convolutional neural network model based on multi-scale fusion is used to classify and identify the ground penetrating radar signals of underground cavities collected in real time.

[0172] During the recognition process, the ground penetrating radar signal of the underground cavity collected in real time is processed according to the instantaneous attribute extraction module and the feature fusion module, and the instantaneous attribute feature fusion data is input into the trained convolutional neural network model based on multi-scale fusion.

[0173] It should be understood that the specific implementation process of each module please refer to the above method content, the present invention will not go into details here, and the division of the above functional modules is only for example illustration. In some embodiments, some functional modules can be merged, and some functional modules can be split. Each functional module can be implemented in software or hardware or a combination of software and hardware. Among them, the software and hardware equipment includes but is not limited to general-purpose computer equipment, programmable gate arrays, digital signal processors, microprocessors and their corresponding programming or burning software.

[0174] The present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a computer, the computer executes a method for identifying underground cavities that combines multi-scale fusion and convolutional neural networks as described in an embodiment of the present application.

[0175] For the specific implementation process of each step, please refer to the description of the above method.

[0176] The readable storage medium is a computer-readable storage medium, which can be an internal storage unit of the software and hardware device described in any of the aforementioned embodiments, such as a hard disk or memory of a controller. The readable storage medium can also be an external storage device of the controller, such as a plug-in hard disk equipped on the controller, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Furthermore, the readable storage medium can also include both an internal storage unit of the controller and an external storage device. The readable storage medium is used to store the computer program and other programs and data required by the controller. The readable storage medium can also be used to temporarily store data that has been output or is to be output.

[0177] The above content is merely an example and explanation of the concept of the present invention. Those skilled in the art may make various modifications or additions to the described specific embodiments or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined by the claims, they should all fall within the scope of protection of the present invention.

[0178] Throughout this specification, references to terms such as "one embodiment," "example," or "specific example" indicate that the specific features, structures, materials, or characteristics described in conjunction with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0179] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the present invention to specific embodiments. Obviously, many modifications and variations are possible based on the contents of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents.

[0180] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A method for identifying underground cavities by combining multi-scale fusion and convolutional neural networks, characterized in that: The following steps are involved: Step (1): For the GPR time series data, extract the instantaneous properties of the GPR signal: instantaneous amplitude IA, instantaneous phase IP and instantaneous frequency IF; Step (2): Dual-tree complex wavelet transform (DT-CWT) is used to perform feature fusion on the instantaneous amplitude (IA), instantaneous phase (IP) and instantaneous frequency (IF); Step (3): The dataset IAF-datasets and its cavity labels after the instantaneous attribute features are fused are input into the convolutional neural network model for training. The trained convolutional neural network model based on multi-scale fusion is used to classify and identify the ground penetrating radar signals of underground cavities collected in real time. During the recognition process, the ground penetrating radar signal of the underground cavity collected in real time is processed according to steps (1) and (2), and the instantaneous attribute feature fusion data is input into the trained convolutional neural network model based on multi-scale fusion; The process of feature fusion of IA, IP and IF using dual-tree complex wavelet transform is as follows: Step B1: Assuming that the GPR time series signal x(t) is a real-valued signal, where t represents the time index, construct a complex wavelet expression for x(t); x(t)=ψ h (t)+iψ g (t) Among them, ψ h (t) and ψ g (t) are real tree wavelet and complex tree wavelet respectively, i is the imaginary unit; Step B2: extracting wavelet coefficients of different scales and directions from each image, where each image refers to an instantaneous amplitude image, an instantaneous phase image, and an instantaneous frequency image; The wavelet coefficients and scaling coefficients are transformed from the real and imaginary tree transforms to the dual-tree complex wavelet transform: Wavelet coefficients and scaling coefficients of the real dual-tree complex wavelet transform: j=1,2,…,J in, and are the wavelet coefficients and scale coefficients of the real part dual-tree complex wavelet transform, j is the scale factor, J is the maximum scale factor, and k is the wavelet filter length; Wavelet coefficients of imaginary dual-tree complex wavelet transform and scale coefficient Step B3: for each wavelet coefficient position, calculate the difference between the corresponding wavelet coefficients in the three images, determine the fusion weights of the three images, and linearly combine the fusion weights to form new fusion weights; Substitute the new fusion weight into the following formula to obtain the fused wavelet coefficients, specifically: According to the wavelet coefficients and scale coefficients of the dual-tree complex wavelet transform, the wavelet coefficients d that capture high-frequency details are obtained. j (t) and the scaling factor c representing the low-frequency trend J (t): Step B4: synthesize the high-frequency wavelet coefficients of the fused image by inverse dual-tree complex wavelet transform, and use the average value of the low-frequency wavelet coefficients of the three images to reconstruct the final fused image, that is, reconstruct the fused image according to the following formula; The wavelet coefficients d that capture high-frequency details j (t) and the scaling factor c representing the low-frequency trend J (t) is reconstructed, and the reconstructed signal is as follows: in, Represents the reconstructed signal.

2. The method according to claim 1, wherein The process of extracting instantaneous attributes of GPR signals is as follows: Step A1: Let the GPR time series data signal be x(t), and construct the complex signal z(t) of x(t): Where: x(t) is the ground penetrating radar time series data signal, which contains information about the underground structure; is the Hilbert transform of x(t), which represents the component related to the phase of x(t); i is the imaginary unit; Perform Hilbert transform on x(t): Step A2: Based on the complex signal z(t) of the GPR time series data, calculate the instantaneous properties of the GPR signal, which are: Among them: A(t), φ(t), They are the instantaneous amplitude IA, instantaneous phase IP and instantaneous frequency IF of the ground penetrating radar respectively.

3. The method according to claim 1, characterized in that The convolutional neural network structure includes an input layer, a convolution-pooling combination module, a flatten layer, a fully connected layer module and an output layer connected in sequence; The convolution-pooling combination module consists of two parts. Each convolution combination contains two convolution blocks, a maximum pooling layer, a batch normalization layer, and a dropout layer. Each convolution block is followed by a ReLU activation function module. The number of convolution kernels in the first and second parts of the convolution block is 32 and 64 respectively. The fully connected layer module includes two fully connected layer units, each of which includes a fully connected layer and a Dropout layer, and a ReLU activation function module is added after each fully connected layer; The output layer uses the softmax function.

4. The method according to claim 3, characterized in that During the training process, the Adam optimizer was used, and the cross entropy loss function was used to measure the performance of the network in the classification task. The batch size was set to 64, and the initial learning rate was 0.

001.

5. An underground cavity recognition system combining multi-scale fusion and convolutional neural network, characterized by: include: Instantaneous attribute extraction module: uses Hilbert transform to process GPR time series data and extracts the instantaneous attributes of GPR signals: instantaneous amplitude IA, instantaneous phase IP and instantaneous frequency IF; Feature fusion module: Dual-tree complex wavelet transform (DT-CWT) is used to fuse the instantaneous amplitude (IA), instantaneous phase (IP) and instantaneous frequency (IF); Model training and recognition module: The IAF-datasets dataset, which is a fusion of instantaneous attribute features, and its labels are input into the convolutional neural network model for training. The trained convolutional neural network model based on multi-scale fusion is used to classify and identify the ground penetrating radar signals of underground cavities collected in real time. During the recognition process, the ground penetrating radar signals of underground cavities collected in real time are processed according to the instantaneous attribute extraction module and the feature fusion module. The instantaneous attribute feature fusion data is input into the trained convolutional neural network model based on multi-scale fusion. The feature fusion module uses dual-tree complex wavelet transform to fuse the features of IA, IP and IF as follows: Step B1: Assuming that the GPR time series signal x(t) is a real-valued signal, where t represents the time index, construct a complex wavelet expression for x(t); x(t)=ψ h (t)+iψ g (t) Among them, ψ h (t) and ψ g (t) are real tree wavelet and complex tree wavelet respectively, i is the imaginary unit; Step B2: extracting wavelet coefficients of different scales and directions from each image, where each image refers to an instantaneous amplitude image, an instantaneous phase image, and an instantaneous frequency image; The wavelet coefficients and scaling coefficients are transformed from the real and imaginary tree transforms to the dual-tree complex wavelet transform: Wavelet coefficients and scaling coefficients of the real dual-tree complex wavelet transform: in, and are the wavelet coefficients and scale coefficients of the real part dual-tree complex wavelet transform, j is the scale factor, J is the maximum scale factor, and k is the wavelet filter length; Wavelet coefficients and scaling coefficients of the imaginary dual-tree complex wavelet transform: Step B3: for each wavelet coefficient position, calculate the difference between the corresponding wavelet coefficients in the three images, determine the fusion weights of the three images, and linearly combine the fusion weights to form new fusion weights; Substitute the new fusion weight into the following formula to obtain the fused wavelet coefficients, specifically: According to the wavelet coefficients and scale coefficients of the dual-tree complex wavelet transform, the wavelet coefficients d that capture high-frequency details are obtained. j (t) and the scaling factor c representing the low-frequency trend J (t): Step B4: synthesize the high-frequency wavelet coefficients of the fused image by inverse dual-tree complex wavelet transform, and use the average value of the low-frequency wavelet coefficients of the three images to reconstruct the final fused image, that is, reconstruct the fused image according to the following formula; The wavelet coefficients d that capture high-frequency details j (t) and the scaling factor c representing the low-frequency trend J (t) is reconstructed, and the reconstructed signal is as follows: in, Represents the reconstructed signal.

6. The system according to claim 5, characterized in that The extraction process of the instantaneous attribute extraction module is as follows: Step A1: Let the GPR time series data signal be x(t), and construct the complex signal z(t) of x(t): Where: x(t) is the ground penetrating radar time series data signal, which contains information about the underground structure; is the Hilbert transform of x(t), which represents the component related to the phase of x(t); i is the imaginary unit; Perform Hilbert transform on x(t): Step A2: Based on the complex signal z(t) of the GPR time series data, the instantaneous properties of the GPR signal are calculated as follows: Among them: A(t), φ(t), They are the instantaneous amplitude IA, instantaneous phase IP and instantaneous frequency IF of the ground penetrating radar respectively.

7. The system according to claim 5, characterized in that The convolutional neural network structure of the model training and recognition module includes an input layer, a convolutional pooling combination module, a flatten layer, a fully connected layer module, and an output layer connected in sequence; The convolution-pooling combination module consists of two parts. Each convolution combination contains two convolution blocks, a maximum pooling layer, a batch normalization layer, and a dropout layer. Each convolution block is followed by a ReLU activation function module. The number of convolution kernels in the first and second parts of the convolution block is 32 and 64 respectively. The fully connected layer module includes two fully connected layer units, each of which includes a fully connected layer and a Dropout layer, and a ReLU activation function module is added after each fully connected layer; The output layer uses the softmax function.

8. A computer-readable storage medium, characterized in that: A computer program is stored, and the computer program is called by a processor to execute: the steps of the method according to any one of claims 1 to 4.

Images