Ground penetrating radar signal real-time imaging method based on multi-modal data fusion and deep learning

Abstract

The invention discloses a ground penetrating radar signal real-time imaging method based on multi-modal data fusion and deep learning, and the method specifically comprises the following steps: 1, collecting the multi-source data of a road through a plurality of data collection devices; 2, filtering and noise reduction processing is carried out on the ground penetrating radar echo signal, and meanwhile preprocessing is carried out on a road surface image; 3, fusing the multi-source data to form a multi-modal data set; 4, performing feature extraction and learning on the multi-modal data set by adopting a deep learning model to obtain a real-time imaging result; and 5, optimizing and adjusting the deep learning model to improve the imaging precision and efficiency. When the scheme is applied to the vertical field, the data processing effect, the high-precision detection, the real-time processing capability, the strong anti-interference capability, the multi-target identification capability, the user friendliness and the strong expansibility are improved from multiple aspects.

Description

Technical Field

[0001] This invention belongs to the field of ground-penetrating radar imaging technology, and specifically provides a real-time imaging method for ground-penetrating radar signals based on multimodal data fusion and deep learning. Background Technology

[0002] Traditional ground-penetrating radar (GPR) data processing methods primarily rely on physical model-based imaging algorithms, such as Reverse Time Migration (RTM) and Synthetic Aperture Radar (SAR) technologies. These methods face several technical bottlenecks in practical applications, as follows: Physical model dependency problem: Traditional methods rely heavily on accurate estimation of electromagnetic parameters of the subsurface medium, including dielectric constant, conductivity, and magnetic permeability. In real, complex geological environments, these parameters often exhibit spatial variability and uncertainty, leading to significant distortion in imaging results. In particular, when the dielectric constant varies by more than ±20%, the depth estimation error may reach over 15%.

[0003] Computational efficiency bottleneck: The computational complexity of the inverse time migration algorithm based on the wave equation increases exponentially with mesh refinement. For typical underground detection scenarios (such as a detection area of ​​10m×10m×5m), when using a mesh resolution of 0.1m, a single imaging requires processing more than 5 million mesh points, with a computation time of up to tens of minutes, which cannot meet the requirements of real-time detection.

[0004] Noise sensitivity issues: Ground reflections, internal system noise, and random environmental interference can severely reduce the signal-to-noise ratio. Traditional filtering methods (such as mean filtering and median filtering) have limited effectiveness in noisy environments, resulting in blurred target features, especially insufficient detection capabilities for weak reflection signals and deep targets.

[0005] The challenge of multi-target identification: When multiple adjacent or overlapping underground targets exist, traditional methods struggle to accurately distinguish the boundaries and attributes of each target. The multiple reflections and scattering of electromagnetic waves in complex media further complicate target identification.

[0006] Ground-penetrating radar (GPR) is a key tool in road inspection, providing non-destructive detection of underground media distribution. However, traditional detection methods have numerous drawbacks. Core drilling can damage roads and is time-consuming and labor-intensive; ultrasonic testing is greatly affected by environmental interference; while GPR has advantages, its detection relies on manual judgment, resulting in low accuracy and a lack of mature automated analysis solutions. Furthermore, GPR signal processing is complex, requiring the fusion of multi-source data to improve imaging accuracy, but current technologies lack sufficient application of multimodal data fusion. The specific technical bottlenecks of traditional GPR technology are as follows: 1. Severe noise interference: Ground clutter, system noise, and multipath reflections cause the effective signal-to-noise ratio (SNR) to be lower than -10dB.

[0007] 2. Insufficient resolution: The recognition rate for targets buried at a depth greater than 2m and with a lateral dimension less than 30cm is less than 60%.

[0008] 3. Interpretation relies on experience: B-Scan image interpretation requires professional operation and has a low degree of automation.

[0009] 4. Poor real-time performance: 3D imaging processing time > 10 minutes / 100 meters (sampling interval 5cm). Summary of the Invention

[0010] To solve the above-mentioned technical problems, the technical solution adopted by this invention is: a real-time imaging method for ground-penetrating radar signals based on multimodal data fusion and deep learning, the specific steps of which are as follows: Step 1: Collect multi-source data about the road using various data acquisition devices; Specifically, various data acquisition devices include ground-penetrating radar, pulsed lidar, optically pumped magnetometer, and image acquisition components; Correspondingly, the collected data included noisy ground-penetrating radar echo signals, LiDAR surface point cloud data, geomagnetic gradient data, and road surface images. Step 2: Filter and reduce noise from the ground-penetrating radar echo signal, and preprocess the road surface image at the same time; A multi-stage adaptive filtering scheme is adopted, including a primary filtering stage, a secondary filtering stage, and a high-level filtering stage; Step 3: Merge the multi-source data to form a multimodal dataset; By combining time-frequency analysis, the time-domain and frequency-domain features of the signal are extracted using a method that combines short-time Fourier transform and continuous wavelet transform. Ground penetrating radar data, lidar topographic data, and geomagnetic data are deeply integrated at the feature level; Finally, a deep learning model is used to learn the complex mapping relationship between multimodal features and underground target parameters; Step 4: Use a deep learning model to extract and learn features from the multimodal dataset to obtain real-time imaging results; Step 5: Optimize and adjust the deep learning model to improve imaging accuracy and efficiency.

[0011] Furthermore, in step one, the ground-penetrating radar uses a 4×4 cross-dipole antenna array, supporting four polarization modes: HH, HV, VH, and VV.

[0012] Furthermore, in step two, the primary filtering stage employs a signal decomposition method based on variational mode decomposition. Secondary filtering stage: Wavelet threshold denoising is performed on each modal component after VMD decomposition; Advanced filtering stage: Based on the physical characteristics of radar signals, algorithms specifically designed for suppressing ground-reflected waves are developed.

[0013] Furthermore, in step three, the short-time Fourier transform is first invoked to analyze the time and frequency domains, and then the continuous wavelet transform is invoked to capture transient features and singularities. The results of the two transforms are then fused to generate a time-spectrum diagram with rich physical meaning.

[0014] Furthermore, in step three, for the hyperbolic reflection characteristics generated by typical targets such as underground pipelines, a feature enhancement algorithm is invoked to detect potential hyperbolic trajectories through Hough transform or Radon transform, and then coherent superposition is performed along the trajectory direction to significantly enhance the energy of the target signal while suppressing random noise.

[0015] Furthermore, in step three, a color mapping scheme with perceived uniformity, such as the CIELAB color space, is adopted to ensure the perceptual consistency between image color differences and changes in physical quantities. Dedicated color palettes are designed for different application scenarios, such as using warm colors to represent high-reflection areas and cool colors to represent low-reflection areas, thereby improving the interpretability of the image.

[0016] Furthermore, in step five, specific optimizations include model compression and acceleration, parallel computing architecture, and adaptive computing resource allocation.

[0017] The beneficial effects of using this invention are: This solution improves data processing performance in multiple ways when applied in vertical industries. High-precision detection: Through multimodal data fusion and deep learning technology, the detection rate and positioning accuracy of underground targets have been significantly improved.

[0018] Real-time processing capability: Optimized algorithms and hardware acceleration enable imaging speeds down to the second, meeting the needs of real-time decision-making on-site.

[0019] Strong anti-interference capability: Adaptive filtering and deep learning models effectively suppress various noise interferences and maintain stable performance in complex environments.

[0020] Multi-target recognition capability: It can simultaneously detect and distinguish multiple underground targets of different types and depths.

[0021] User-friendliness: The intuitive visual interface and automated processing flow lower the operating threshold and improve work efficiency.

[0022] Strong scalability: The modular design supports functional expansion and algorithm updates, adapting to the needs of different application scenarios.

[0023] Through the integration of multidisciplinary technologies, a major breakthrough has been achieved in ground-penetrating radar imaging technology, providing strong technical support for the development and utilization of underground space. Attached Figure Description

[0024] Figure 1 This is a flowchart of the present invention; Figure 2 This is a comparison chart of the original data and envelope detection versus depth prediction of the present invention. Figure 3 This is a three-dimensional positioning result diagram of the underground target according to the present invention. Detailed Implementation

[0025] The present invention will now be described in detail with reference to the accompanying drawings.

[0026] Example 1

[0027] Reference Figure 1 The real-time imaging method for ground-penetrating radar signals based on multimodal data fusion and deep learning has the following specific steps: Step 1: Collect multi-source data about the road using various data acquisition devices; Specifically, various data acquisition devices include ground-penetrating radar, pulsed lidar, optically pumped magnetometer, and image acquisition components; Correspondingly, the collected data included noisy ground-penetrating radar echo signals, LiDAR surface point cloud data, geomagnetic gradient data, and road surface images.

[0028] Step 2: Filter and reduce noise from the ground-penetrating radar echo signal, and preprocess the road surface image at the same time; A multi-stage adaptive filtering scheme is adopted, including a primary filtering stage, a secondary filtering stage, and a high-level filtering stage. The filtering performance indicators are a system signal-to-noise ratio improvement of more than 20dB, a signal fidelity (measured by the correlation coefficient) greater than 0.95, and a single-channel signal processing time controlled within 5ms.

[0029] An adaptive filtering algorithm is used to smooth the noisy ground-penetrating radar echo signal, effectively removing noise components, improving the signal-to-noise ratio, and enabling the effective signal to be extracted and identified more clearly. This solves the problem that severe noise interference in traditional ground-penetrating radar technology leads to an effective signal-to-noise ratio of less than -10dB.

[0030] Step 3: Merge the multi-source data to form a multimodal dataset; By combining time-frequency analysis, the time-domain and frequency-domain features of the signal are extracted using a method that combines short-time Fourier transform (STFT) and continuous wavelet transform (CWT). Specifically, the short-time Fourier transform is first invoked to analyze the time and frequency domains, and then the continuous wavelet transform is invoked to capture transient features and singularities. The results of the two transforms are fused to generate a time-spectrum diagram with rich physical meaning. Ground penetrating radar data, lidar topographic data, and geomagnetic data are deeply integrated at the feature level; Finally, a deep learning model was used to learn the complex mapping relationship between multimodal features and underground target parameters.

[0031] Reference Figure 2 By combining ground-penetrating radar (GPR) signals with LiDAR surface point cloud data, geomagnetic gradient data, and road surface images, multi-source data is enriched in terms of data dimensionality and feature information. This provides a more comprehensive input for the deep learning model, helping to more accurately identify and locate underground targets. In the deep learning model, electromagnetic feature flow uses 1D-CNN to extract subtle features from GPR signals, while cross-modal attention fusion further highlights features relevant to target recognition. These measures enable the model to better capture the features of underground targets, improve the recognition rate of small targets with a burial depth >2m and a lateral dimension <30cm, and solve the problem of insufficient resolution in traditional technologies leading to a recognition rate of <60% for such targets.

[0032] In addition, the wave equation regularization constraint also ensures that the prediction results conform to the laws of electromagnetic propagation, further improving the imaging accuracy and resolution.

[0033] Step 4: Use a deep learning model to extract and learn features from the multimodal dataset to obtain real-time imaging results.

[0034] Step 5: Optimize and adjust the deep learning model to improve imaging accuracy and efficiency.

[0035] Innovative designs were implemented at both the algorithm optimization and hardware acceleration levels: Model Compression and Acceleration: Knowledge distillation is employed to transfer knowledge from a complex teacher network to a lightweight student network. By softening the output probability distribution and matching hidden layer features, the student network maintains over 95% of the original model's performance while reducing the number of parameters by 60%. Simultaneously, quantized perceptual training is applied, quantizing network weights and activation values ​​from 32-bit floating-point to 8-bit integers, resulting in a 3-5x speedup for inference.

[0036] A four-stage pipelined parallel processing architecture is adopted to achieve overlapping execution of data acquisition, preprocessing, network inference, and result visualization. The data acquisition thread is responsible for controlling the sensor hardware and buffering raw data; the preprocessing thread utilizes a multi-core CPU to perform filtering and feature extraction in parallel; the network inference thread uses a GPU to accelerate deep learning model computation; and the visualization thread is responsible for generating real-time imaging results and interacting with the human-machine interface. Through precise thread synchronization and memory management, the system's end-to-end latency is ensured to be less than 100ms.

[0037] Adaptive computing resource allocation: Computing resources are dynamically adjusted based on data complexity and real-time requirements. For simple scenarios, a fast inference mode is used, employing a simplified network structure and low-resolution input; for complex scenarios, a high-precision mode is activated, using the complete network and high-resolution input. This adaptive mechanism maximizes system throughput while ensuring accuracy.

[0038] By employing deep learning models to extract and learn features from multimodal datasets, the system achieves automatic identification and localization of underground targets, reducing reliance on the experience of professional personnel. Augmented reality display technology visually presents the imaging results in actual road scenes, enabling non-professionals to quickly understand the distribution of underground targets, further improving the system's automation and ease of use. This solves the problem of traditional ground-penetrating radar B-Scan image interpretation requiring professional operation and having a low degree of automation.

[0039] Example 2

[0040] Compared to Embodiment 1, the difference in this embodiment is that, specifically: In step one, the ground-penetrating radar employs a 4×4 cross-dipole antenna array, supporting four polarization modes: HH (horizontal transmit-horizontal receive), HV (horizontal transmit-vertical receive), VH (vertical transmit-horizontal receive), and VV (vertical transmit-vertical receive). The system operates in the frequency range of 0.1-4 GHz, with the low-frequency band (0.1-1 GHz) used for deep-sea detection and the high-frequency band (1-4 GHz) used for shallow-sea high-resolution imaging. The antenna array spacing has been optimized to ensure that spatial sampling satisfies Nyquist's theorem and avoids grating lobe effects.

[0041] Fully polarized antenna arrays have good directivity and polarization characteristics, which can reduce interference from ground clutter and multipath reflections.

[0042] Example 3

[0043] Compared to Embodiment 1, the difference in this embodiment is that, specifically: In step two, the primary filtering stage, secondary filtering stage, and advanced filtering stage are as follows: Primary filtering stage: A signal decomposition method based on Variational Mode Decomposition (VMD) is employed. The VMD algorithm adaptively decomposes the input signal into multiple Intrinsic Mode Functions (IMFs) by solving a constrained variational problem. Compared to traditional Empirical Mode Decomposition (EMD), VMD has a solid mathematical foundation and better noise robustness. By optimizing the bandwidth constraint parameters and the number of modes, it effectively separates useful components and noise components from the signal.

[0044] Secondary filtering stage: Wavelet thresholding is performed on each modal component after VMD decomposition. An improved threshold function is used to achieve a smooth transition between hard and soft thresholding, avoiding the pseudo-Gibbs phenomenon. The threshold selection is based on the principle of unbiased risk estimation, achieving adaptive estimation of noise variance. The wavelet basis function is dynamically selected according to the signal characteristics; the Db4 wavelet is preferentially used for pulse-type reflection signals, while the Sym8 wavelet is used for oscillating signals.

[0045] Advanced filtering stage: Combining the physical characteristics of radar signals, an algorithm specifically designed for suppressing ground reflection waves is developed. By estimating the hyperbolic characteristics of ground reflection waves, selective filtering is performed in the Radon transform domain to effectively suppress strong ground reflection interference while preserving the target reflection signal.

[0046] Among them, road surface image preprocessing involves performing grayscale conversion and histogram equalization on the acquired road surface images to enhance the contrast and brightness of the images, highlight the texture and structural features of the road surface, so as to better integrate with the ground penetrating radar signal and provide a clearer image basis for subsequent feature extraction.

[0047] Example 4

[0048] Compared to Embodiment 1, the difference in this embodiment is that, specifically: In step three, for the hyperbolic reflection characteristics generated by typical targets such as underground pipelines, a feature enhancement algorithm (a targeted method encapsulated in MATLAB) is invoked to detect potential hyperbolic trajectories through Hough transform or Radon transform, and then coherent superposition is performed along the trajectory direction to significantly enhance the energy of the target signal while suppressing random noise.

[0049] The enhanced hyperbolic feature can improve the signal-to-noise ratio by 10-15 dB.

[0050] Mature data processing methods are used to standardize the data of each modality, eliminate dimensional differences, and then principal component analysis (PCA) or independent component analysis (ICA) is used to extract the main feature components.

[0051] Specifically, various mature data preprocessing techniques are used to standardize the data for each modality: For numerical sensor data, the Z-score normalization method is applied to eliminate the influence of dimensions. For image data, histogram equalization is used to enhance contrast; For LiDAR surface point cloud data, voxel grid downsampling is used to unify the spatial resolution. After eliminating the dimensional differences and distribution inconsistencies between different modal data through the above processing, principal component analysis (PCA) is further used to extract the main variance features from the high-dimensional data, or independent component analysis (ICA) is used to separate statistically independent signal components. This significantly reduces the data dimensionality while retaining key information, providing high-quality feature input for subsequent multimodal feature fusion and deep learning models.

[0052] Reference Figure 3 A color mapping scheme with uniform perception, such as the CIELAB color space, is employed to ensure the perceptual consistency of color differences and changes in physical quantities in the image. Dedicated color palettes are designed for different application scenarios, such as using warm colors to represent high-reflection areas and cool colors to represent low-reflection areas, thereby improving the interpretability of the image.

[0053] Example 5

[0054] Compared to Embodiment 1, the difference in this embodiment is that, specifically: In step four, a deep learning network architecture specifically designed for ground-penetrating radar data processing was implemented, achieving high-precision mapping from raw signals to target parameters: A multi-scale feature extraction network (MSFENet) is employed, with multiple branches operating in parallel to process input features at different scales. The large-scale branch uses a large 11×11 convolutional kernel to focus on extracting the overall contour and spatial distribution features of the target; the medium-scale branch uses a medium 7×7 convolutional kernel to capture the texture and structural details of the target; and the small-scale branch uses a small 3×3 convolutional kernel combined with dilated convolutions to enhance the perception of edges and fine features. The features extracted by each branch are fused through a deep concatenation layer to fully utilize multi-scale information. A dual attention module is embedded in the network, including channel attention and spatial attention. Channel attention learns the importance weights of each feature channel through global average pooling and fully connected layers, enabling adaptive selection of feature channels. Spatial attention learns the spatial weight distribution of the feature map through convolutional operations, highlighting key regions relevant to the target. The introduction of the attention mechanism enables the network to focus on the features most relevant to the underground target, significantly improving the model's discriminative ability.

[0055] The network simultaneously learns multiple related tasks, including target localization, attribute recognition, and confidence estimation. By sharing the underlying feature extraction network and task-specific sub-networks, it achieves knowledge transfer and collaborative optimization across different tasks. Multi-task learning not only improves the performance of each task but also enhances the model's generalization ability.

[0056] A learning strategy that progresses from easy to difficult is adopted to gradually increase the complexity of the training data. In the initial stage, simple samples with a single objective and ideal medium conditions are used to train the basic capabilities of the network; in the intermediate stage, medium-complexity samples with multiple objectives and uniform medium are introduced; in the final stage, complex scene samples containing random noise and non-uniform medium are used to improve the robustness of the model in real-world environments.

[0057] To address the limited amount of ground-penetrating radar (GPR) data, various data augmentation techniques are employed, including time shifting, amplitude scaling, noise addition, and simulating multiple reflections.

[0058] At the same time, by combining regularization techniques such as Dropout, Batch Normalization, and weight decay, overfitting can be effectively prevented and the generalization performance of the model can be improved.

[0059] Comprehensive verification was conducted through laboratory simulations and field tests. The specific verification methods and results are as follows: Laboratory testing: Standard targets (metal pipes, PVC pipes, concrete blocks, etc.) of different materials and depths were placed in a controlled environment. Test results showed that the system achieved a detection rate of 99.5% for metal targets, with a depth estimation error of less than 2cm and a horizontal positioning error of less than 3cm. The accuracy rate for identifying non-metallic targets exceeded 95%.

[0060] Field application verification: Tests were conducted in multiple real-world scenarios, including urban roads, archaeological sites, and mines. In complex urban environments, the system successfully identified various underground pipelines buried at depths of 0.5-3m, with positioning accuracy meeting engineering requirements. In archaeological exploration, the system non-destructively detected underground foundations and tomb structures, providing precise guidance for archaeological excavations.

[0061] Compared with traditional reverse time migration methods, this invention improves imaging speed by 20-50 times and positioning accuracy by 3-5 times, especially showing significant advantages in noisy environments and complex geological conditions. Multimodal data fusion effectively solves the uncertainty problem of single data sources, improving the reliability of the detection results.

[0062] By introducing an edge computing unit and integrating NVIDIA Jetson AGX Orin, it possesses high computing performance, enabling rapid processing of large amounts of multimodal data and deep learning model inference. This significantly shortens the 3D imaging processing time, achieving real-time imaging and controlling latency to <50ms / frame. It solves the problem of traditional 3D imaging processing taking >10 minutes / 100 meters (sampling interval 5cm).

[0063] The above content is only a preferred embodiment of the present invention. For those skilled in the art, many changes can be made in the specific implementation and application scope based on the concept of the present invention. As long as these changes do not depart from the concept of the present invention, they all fall within the protection scope of the present invention.

Claims

1. A real-time imaging method for ground-penetrating radar signals based on multimodal data fusion and deep learning, the specific steps of which are as follows: Step 1: Collect multi-source data about the road using various data acquisition devices; Specifically, various data acquisition devices include ground-penetrating radar, pulsed lidar, optically pumped magnetometer, and image acquisition components; Correspondingly, the collected data included noisy ground-penetrating radar echo signals, LiDAR surface point cloud data, geomagnetic gradient data, and road surface images. Step 2: Filter and reduce noise from the ground-penetrating radar echo signal, and preprocess the road surface image at the same time; A multi-stage adaptive filtering scheme is adopted, including a primary filtering stage, a secondary filtering stage, and a high-level filtering stage; Step 3: Merge the multi-source data to form a multimodal dataset; By combining time-frequency analysis, the time-domain and frequency-domain features of the signal are extracted using a method that combines short-time Fourier transform and continuous wavelet transform. Ground penetrating radar data, lidar topographic data, and geomagnetic data are deeply integrated at the feature level; Finally, a deep learning model is used to learn the complex mapping relationship between multimodal features and underground target parameters; Step 4: Use a deep learning model to extract and learn features from the multimodal dataset to obtain real-time imaging results; Step 5: Optimize and adjust the deep learning model to improve imaging accuracy and efficiency.

2. The real-time imaging method for ground-penetrating radar signals based on multimodal data fusion and deep learning as described in claim 1, characterized in that: In step one, the ground-penetrating radar uses a 4×4 cross-dipole antenna array, supporting four polarization modes: HH, HV, VH, and VV.

3. The real-time imaging method for ground-penetrating radar signals based on multimodal data fusion and deep learning as described in claim 1, characterized in that: In step two, the primary filtering stage employs a signal decomposition method based on variational mode decomposition. Secondary filtering stage: Wavelet threshold denoising is performed on each modal component after VMD decomposition; Advanced filtering stage: Based on the physical characteristics of radar signals, algorithms specifically designed for suppressing ground-reflected waves are developed.

4. The real-time imaging method for ground-penetrating radar signals based on multimodal data fusion and deep learning as described in claim 1, characterized in that: In step three, the short-time Fourier transform is first invoked to analyze the time and frequency domains, and then the continuous wavelet transform is invoked to capture transient features and singularities. The results of the two transforms are fused to generate a time-spectrum diagram with rich physical meaning.

5. The real-time imaging method for ground-penetrating radar signals based on multimodal data fusion and deep learning as described in claim 1, characterized in that: In step three, for the hyperbolic reflection characteristics generated by typical targets such as underground pipelines, a feature enhancement algorithm is invoked to detect potential hyperbolic trajectories through Hough transform or Radon transform, and then coherent superposition is performed along the trajectory direction to significantly enhance the energy of the target signal while suppressing random noise.

6. The real-time imaging method for ground-penetrating radar signals based on multimodal data fusion and deep learning as described in claim 1, characterized in that: In step three, a color mapping scheme with perceived uniformity, such as the CIELAB color space, is adopted to ensure the perceptual consistency between image color differences and changes in physical quantities. Dedicated color palettes are designed for different application scenarios, such as using warm colors to represent high-reflection areas and cool colors to represent low-reflection areas, thereby improving the interpretability of the image.

7. The real-time imaging method for ground-penetrating radar signals based on multimodal data fusion and deep learning as described in claim 1, characterized in that: Step five includes specific optimizations such as model compression and acceleration, parallel computing architecture, and adaptive computing resource allocation.

Images