Method and system for detecting internal road surface defects using three-dimensional ground penetrating radar

Abstract

The present invention relates to the field of intelligent detection and analysis technology, and in particular to a method and system for detecting internal road defects using three-dimensional ground penetrating radar. The present invention includes multimodal data acquisition and preprocessing, multimodal data alignment and feature extraction, adaptive deep learning model training and recognition, dynamic statistical feature analysis and optimization, and automatic report generation and decision support. Through multimodal data fusion technology, more comprehensive and accurate disease detection is achieved. Deep neural networks and multi-task learning frameworks are introduced to improve data processing efficiency and accuracy. Combined with reinforcement learning technology, model parameters are dynamically optimized to enhance the robustness and adaptability of the model. Through intelligent filtering and probability optimization, the misjudgment rate is significantly reduced, and the credibility of the recognition results is improved. Finally, a detailed detection report is generated based on the detection results to provide scientific and reliable decision support for road maintenance and management. The present invention improves the accuracy, efficiency and reliability of internal road defect detection.

Description

Technical Field

[0001] The present invention relates to the field of intelligent detection and analysis technology, and in particular to a method and system for detecting internal road surface defects using three-dimensional ground penetrating radar. Background Art

[0002] Internal pavement defects such as cracks, voids, and water leaks not only affect the service life of roads but can also lead to serious traffic accidents. Therefore, accurately and efficiently detecting and identifying internal pavement defects is a critical task in road maintenance and management. Current internal pavement defect detection technologies primarily rely on single sensors (such as 3D ground-penetrating radar). This results in poor data quality and unsatisfactory detection results when faced with extreme conditions such as complex geological structures and strong interference signals. Furthermore, data preprocessing and feature extraction processes are complex and inefficient, especially when processing large amounts of data. Existing recognition models are unstable under varying environmental conditions, lack adaptive capabilities, and lack intelligent optimization and feedback mechanisms. This results in low detection accuracy and efficiency, and a high incidence of misjudgments and missed detections.

[0003] The existing technical solution (Chinese invention patent, publication number: CN118171144A, title: A method for optimizing the identification of internal road surface defects based on statistical characteristics of radar signals) has the following drawbacks when attempting to solve the current problem:

[0004] Existing technologies use a single sensor (such as 3D ground-penetrating radar) for data collection and analysis. However, single sensors perform poorly in complex geological structures and under strong interference signal conditions, resulting in poor data quality and inability to fully and accurately identify diseases.

[0005] Existing technologies use traditional signal processing methods (such as filtering, denoising, and spectrum analysis) for data preprocessing and feature extraction. However, these methods are cumbersome and computationally intensive, resulting in low processing efficiency and difficulty in processing large amounts of data quickly and accurately.

[0006] Existing technologies rely on fixed rules and parameters (such as threshold determination and pattern matching) for disease identification. These models lack the ability to adapt to different environmental conditions, resulting in unstable detection results and insufficient recognition accuracy and reliability.

[0007] The existing detection process is linear and lacks a feedback mechanism. Since the detection process cannot be dynamically adjusted and optimized, the overall detection efficiency and accuracy are low.

[0008] Existing technologies rely on fixed rules and thresholds to determine diseases. However, since these rules cannot guarantee high-precision recognition in complex environments, there are a large number of misjudgments and missed judgments in the recognition results. Summary of the Invention

[0009] In response to the many problems existing in the above-mentioned existing technologies, the present invention provides a method and system for detecting internal road defects using three-dimensional ground-penetrating radar. The present invention uses three-dimensional ground-penetrating radar, lidar, infrared imaging and ultrasonic sensors to synchronously collect road surface data and preprocess the data; then, a deep neural network and a multi-task learning framework are used to align the data and extract features to generate a comprehensive feature vector; the data set is expanded by generating an adversarial network, and the model parameters are dynamically optimized in combination with reinforcement learning technology to improve the robustness and adaptability of the recognition model; finally, through intelligent filtering and probabilistic optimization, the hidden Markov model and the adaptive discriminant network are used to further optimize the recognition results, significantly reduce the misjudgment rate, and improve the accuracy and efficiency of detection.

[0010] like Figure 1-Figure 3 As shown, a method for detecting internal road surface defects using a three-dimensional ground penetrating radar comprises the following steps:

[0011] Use 3D ground-penetrating radar, lidar, infrared imaging, and ultrasonic sensors to perform simultaneous non-destructive testing on the road surface, collect multimodal data, and perform denoising, filtering, and feature enhancement on the data to generate pre-processed multimodal data;

[0012] Perform temporal synchronization and spatial registration on the preprocessed multimodal data to generate aligned multimodal data. Then, extract radar spatial feature data, laser point cloud feature data, infrared thermal feature data, and ultrasonic frequency domain feature data. Then, use a deep neural network to perform feature fusion and generate a comprehensive feature vector.

[0013] A training dataset was constructed using comprehensive feature vectors, augmented with a generative adversarial network, and trained simultaneously on disease recognition and feature prediction tasks using a multi-task learning framework. Reinforcement learning was combined with model parameter optimization, and a self-attention mechanism and recurrent neural network were introduced to process time series data to generate preliminary recognition results.

[0014] The energy amplitude, frequency domain spectrum characteristics and electromagnetic wave attenuation characteristics of the disease identification and feature prediction data in the preliminary identification results are calculated to generate dynamic statistical feature data, which are fed back into the adaptive deep learning model for secondary training. The hidden Markov model is used to probabilistically optimize the dynamic statistical feature data, and the adaptive discriminant network based on the attention mechanism is used for intelligent filtering and optimization to finally generate the identification results.

[0015] Preferably, the denoising, filtering and feature enhancement processing includes:

[0016] De-noising the radar data, filtering out interference signals through a high-pass filter, retaining valid signals, and generating pre-processed radar data;

[0017] Filter the laser data, remove noise points through statistical filtering algorithms, enhance point cloud quality, and generate pre-processed laser data;

[0018] Perform image enhancement processing on infrared data, improve thermal map details and contrast through histogram equalization technology, and generate pre-processed infrared data;

[0019] The ultrasonic data is subjected to spectrum analysis and denoising, and useful echo signals are extracted through Fourier transform to generate preprocessed ultrasonic data.

[0020] Preferably, the time synchronization, spatial registration and feature fusion include:

[0021] The pre-processed multimodal data is time-synchronized using the spatiotemporal feature alignment algorithm to ensure the consistency of data from different sensors at the same time point and generate time-synchronized data;

[0022] Use a feature point-based spatial registration algorithm to spatially register the time-synchronized data to ensure alignment of different sensor data in spatial coordinates and generate aligned multimodal data;

[0023] A convolutional neural network is used to extract radar spatial feature data from aligned multimodal data. A point cloud processing algorithm is used to extract laser point cloud feature data from aligned multimodal data. An image processing algorithm is used to extract infrared thermal feature data from aligned multimodal data. A spectrum analysis method is used to extract ultrasonic frequency domain feature data from aligned multimodal data. These feature data are then input into a deep neural network for feature fusion to generate a comprehensive feature vector.

[0024] Preferably, the generative adversarial network expanded dataset includes:

[0025] Generate virtual data using a generative adversarial network using a training dataset containing synthetic feature vectors;

[0026] The generated virtual data is fused with the original training data set to expand the training data set and generate an expanded training comprehensive feature vector;

[0027] Use a multi-task learning model to simultaneously train disease identification tasks and feature prediction tasks to optimize model parameters;

[0028] In the multi-task learning process, the disease recognition task and the feature prediction task share some model parameters while retaining the dedicated parameters of each task to generate a multi-task learning model.

[0029] Preferably, the reinforcement learning optimization model parameters include:

[0030] Utilize the policy gradient algorithm to optimize the multi-task learning model and dynamically adjust the model parameters by simulating detection tasks under various environmental conditions;

[0031] In the reinforcement learning process, a reward function is constructed to give rewards based on the accuracy and robustness of the detection results to improve model performance and generate an adaptive deep learning model;

[0032] Introducing a self-attention mechanism into the adaptive deep learning model enables the model to focus on key features and enhance the ability to identify disease characteristics;

[0033] Use recurrent neural networks and long short-term memory networks to process time series data, improve the model's ability to capture time-varying features, and generate a time series enhanced adaptive deep learning model to generate preliminary recognition results.

[0034] Preferably, the calculation of the energy amplitude, frequency domain spectrum characteristics and electromagnetic wave attenuation characteristics includes:

[0035] Perform energy amplitude calculation on the disease identification and feature prediction data in the preliminary identification results to generate energy amplitude data;

[0036] Perform frequency domain spectrum feature calculation on the disease identification and feature prediction data in the preliminary identification results to generate frequency domain spectrum feature data;

[0037] The electromagnetic wave attenuation characteristics are calculated for the disease identification and feature prediction data in the preliminary identification results to generate electromagnetic wave attenuation characteristic data.

[0038] Preferably, the calculation expressions of the energy amplitude, frequency domain spectrum characteristics and electromagnetic wave attenuation characteristics are respectively:

[0039] Energy amplitude calculation:

[0040]

[0041] in, represents the energy amplitude, Indicates the amplitude of the radar signal;

[0042] Frequency domain spectrum feature calculation:

[0043]

[0044] in, represents the frequency domain spectrum characteristics, represents the Fourier transform of a signal;

[0045] Calculation of electromagnetic wave attenuation characteristics:

[0046]

[0047] in, Indicates distance The signal amplitude at represents the initial signal amplitude, Represents the attenuation coefficient.

[0048] Preferably, the feedback and probability optimization of the dynamic statistical feature data include:

[0049] Integrate energy amplitude data, frequency domain spectrum feature data, and electromagnetic wave attenuation characteristic data into dynamic statistical feature data, and feed it back into the adaptive deep learning model for secondary training and optimization to form enhanced preliminary recognition results;

[0050] The hidden Markov model is used to perform probability optimization on the enhanced preliminary recognition results, eliminate noise and misjudgment, and generate probability optimization results.

[0051] Preferably, the intelligent filtering and optimization includes:

[0052] The adaptive discriminant network based on the attention mechanism is used to intelligently filter and optimize the probability optimization results to generate intelligent filtering results;

[0053] Further optimization is performed based on the intelligent filtering results to generate the final recognition results.

[0054] A system for executing the method for detecting internal road surface defects using three-dimensional ground penetrating radar, comprising:

[0055] 3D ground-penetrating radar, lidar, infrared imaging sensors, and ultrasonic sensors for simultaneous non-destructive testing of road surfaces and collection of multimodal data;

[0056] The data preprocessing module is used to perform denoising, filtering and feature enhancement on the collected multimodal data to generate preprocessed multimodal data;

[0057] Data synchronization and registration module, used to perform temporal synchronization and spatial registration on preprocessed multimodal data to generate aligned multimodal data;

[0058] Feature extraction and fusion module, which is used to extract radar spatial feature data, laser point cloud feature data, infrared thermal feature data, and ultrasonic frequency domain feature data from the aligned multimodal data, and use deep neural network to perform feature fusion to generate a comprehensive feature vector;

[0059] A training data construction module is used to construct a training dataset using a comprehensive feature vector and to augment the dataset by generating an adversarial network;

[0060] A multi-task learning and optimization module is used to simultaneously train disease recognition and feature prediction tasks using a multi-task learning framework, combine reinforcement learning to optimize model parameters, and introduce a self-attention mechanism and recurrent neural network to process time series data to generate preliminary recognition results;

[0061] The dynamic statistical feature analysis and feedback module is used to calculate the energy amplitude, frequency domain spectrum characteristics, and electromagnetic wave attenuation characteristics of the disease identification and feature prediction data in the preliminary identification results, generate dynamic statistical feature data, and feed it back to the adaptive deep learning model for secondary training;

[0062] The probability optimization and intelligent filtering module is used to perform probability optimization on dynamic statistical feature data using a hidden Markov model, and to perform intelligent filtering and optimization using an adaptive discriminant network based on an attention mechanism to generate the final recognition result.

[0063] Compared with the prior art, the advantages and beneficial effects of the present invention are:

[0064] The present invention achieves more comprehensive and accurate pavement disease detection through multimodal data fusion;

[0065] The present invention improves the efficiency and accuracy of data processing by introducing deep neural networks and multi-task learning framework;

[0066] The present invention realizes dynamic optimization of model parameters through reinforcement learning and adaptive deep learning models, thereby improving the robustness and adaptability of the model;

[0067] The present invention realizes real-time feedback and continuous optimization through intelligent filtering, adaptive discrimination and closed-loop optimization system, thus improving the accuracy and efficiency of detection;

[0068] The present invention significantly reduces the misjudgment rate and improves the credibility of recognition results through dynamic statistical feature analysis, hidden Markov model and generative adversarial training. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 Schematic diagram of the process of the present invention;

[0070] Figure 2 Schematic diagram of multimodal data acquisition in the present invention;

[0071] Figure 3 This is a data preprocessing flow chart of the present invention;

[0072] Figure 4 This is a diagram of the feature fusion process in the present invention;

[0073] Figure 5 Construction of training data set and model training diagram in the present invention;

[0074] Figure 6 This is a diagram of the feature analysis and optimization process in the present invention;

[0075] Figure 7 It is a structural block diagram of the system of the present invention. DETAILED DESCRIPTION

[0076] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0077] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise," "include," etc. used herein indicate the presence of the features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0078] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0079] When expressions such as “at least one of A, B, and C, etc.” are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (for example, “a system having at least one of A, B, and C” should include but is not limited to systems having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, and C, etc.). When expressions such as “at least one of A, B, or C, etc.” are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (for example, “a system having at least one of A, B, or C” should include but is not limited to systems having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, and C, etc.).

[0080] The accompanying drawings illustrate some block diagrams and / or flow charts. It should be understood that some blocks in the block diagrams and / or flow charts, or combinations thereof, may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when executed by the processor, these instructions may create a device for implementing the functions / operations described in these block diagrams and / or flow charts. The techniques of the present disclosure may be implemented in the form of hardware and / or software (including firmware, microcode, etc.). In addition, the techniques of the present disclosure may take the form of a computer program product on a computer-readable storage medium having stored thereon instructions, which may be used by or in conjunction with an instruction execution system.

[0081] A method for detecting internal road surface defects using three-dimensional ground penetrating radar comprises the following steps:

[0082] Use 3D ground-penetrating radar, lidar, infrared imaging, and ultrasonic sensors to perform simultaneous non-destructive testing on the road surface, collect multimodal data, and perform denoising, filtering, and feature enhancement on the data to generate pre-processed multimodal data;

[0083] Preferably, the denoising, filtering and feature enhancement processing includes:

[0084] De-noising the radar data, filtering out interference signals through a high-pass filter, retaining valid signals, and generating pre-processed radar data;

[0085] Filter the laser data, remove noise points through statistical filtering algorithms, enhance point cloud quality, and generate pre-processed laser data;

[0086] Perform image enhancement processing on infrared data, improve thermal map details and contrast through histogram equalization technology, and generate pre-processed infrared data;

[0087] The ultrasonic data is subjected to spectrum analysis and denoising, and useful echo signals are extracted through Fourier transform to generate preprocessed ultrasonic data.

[0088] In this invention, three-dimensional ground-penetrating radar (GPR), lidar (LiDAR), infrared imaging, and ultrasonic sensors are used to perform simultaneous nondestructive testing of road surfaces. The goal is to obtain more comprehensive and accurate multimodal data through the collaborative operation of multiple sensors. Three-dimensional GPR is primarily used to detect underground structures and defects. It works by emitting electromagnetic waves and receiving reflected signals to image and analyze the distribution and characteristics of underground objects. LiDAR uses the time difference between laser pulse reflections to perform high-precision three-dimensional imaging, primarily used to capture high-resolution point cloud data of the surface. Infrared imaging generates thermal images by detecting infrared radiation emitted by objects, effectively identifying areas of temperature anomalies and reflecting the thermal characteristics of underground defects. Ultrasonic sensors utilize the propagation characteristics of high-frequency sound waves in a medium, detecting physical changes within the medium through echo signals, and are suitable for detecting fine structures such as cracks and cavities.

[0089] The collected multimodal data needs to be processed through denoising, filtering, and feature enhancement to generate preprocessed multimodal data. This step is critical to ensuring data quality and consistency. The specific processing process is as follows:

[0090] Radar data denoising: A high-pass filter removes interfering signals, retains the valid signal, and generates pre-processed radar data. The high-pass filter removes low-frequency noise and enhances the high-frequency portion of the signal, making the details of underground structures clearer.

[0091] Laser data filtering: Statistical filtering algorithms are used to remove noise points, enhance point cloud quality, and generate pre-processed laser data. Statistical filtering algorithms analyze the statistical characteristics of point cloud data to remove isolated and noisy points, ensuring the accuracy and integrity of point cloud data.

[0092] Infrared data image enhancement: Histogram equalization is used to enhance thermal image detail and contrast, generating pre-processed infrared data. Histogram equalization significantly improves image contrast by adjusting the grayscale distribution of the image, making it easier to identify areas of temperature anomalies.

[0093] Ultrasonic Data Spectral Analysis and De-noising: Useful echo signals are extracted through Fourier transforms to generate pre-processed ultrasonic data. Fourier transforms convert time-domain signals into frequency-domain signals, extracting useful frequency components and removing unwanted noise to improve signal quality.

[0094] In one embodiment, the present invention was applied to detect internal road surface defects in a municipal road maintenance project. Three-dimensional ground-penetrating radar was used to detect underground pipelines and cavities, and after processing with a high-pass filter, multiple underground cavities were discovered. LiDAR captured high-precision point cloud data from the surface, and after statistical filtering, identified areas of road surface subsidence. Infrared imaging detected thermal anomalies, and after histogram equalization processing, underground water leaks were located. Ultrasonic sensors detected cracks in the road surface, and after Fourier transform processing, the depth and location of the cracks were confirmed. Through comprehensive analysis and processing of multimodal data, accurate pre-processed multimodal data was generated, providing reliable data support for subsequent repair plans.

[0095] The preprocessed multimodal data generated through the above steps significantly improves data quality and consistency, ensuring the accuracy and reliability of subsequent analysis. The collaborative working of multimodal data not only comprehensively reflects the internal pavement damage situation but also improves the accuracy and efficiency of disease detection through data fusion technology. In practical applications, this method can quickly and accurately detect various internal pavement diseases, providing a scientific and reliable basis for repairs, significantly improving the efficiency and quality of municipal road maintenance.

[0096] Perform temporal synchronization and spatial registration on the preprocessed multimodal data to generate aligned multimodal data. Then, extract radar spatial feature data, laser point cloud feature data, infrared thermal feature data, and ultrasonic frequency domain feature data. Then, use a deep neural network to perform feature fusion and generate a comprehensive feature vector.

[0097] Preferably, the time synchronization, spatial registration and feature fusion include:

[0098] The pre-processed multimodal data is time-synchronized using the spatiotemporal feature alignment algorithm to ensure the consistency of data from different sensors at the same time point and generate time-synchronized data;

[0099] Use a feature point-based spatial registration algorithm to spatially register the time-synchronized data to ensure alignment of different sensor data in spatial coordinates and generate aligned multimodal data;

[0100] like Figure 4 As shown in the figure, a convolutional neural network is used to extract radar spatial feature data from the aligned multimodal data, a point cloud processing algorithm is used to extract laser point cloud feature data from the aligned multimodal data, an image processing algorithm is used to extract infrared thermal feature data from the aligned multimodal data, and a spectrum analysis method is used to extract ultrasonic frequency domain feature data from the aligned multimodal data. These feature data are then input into a deep neural network for feature fusion to generate a comprehensive feature vector.

[0101] In the present invention, time synchronization and spatial registration are performed on the pre-processed multimodal data to ensure the consistency of data from different sensors at the same time point and spatial coordinates, thereby generating aligned multimodal data. Time synchronization refers to aligning the data collected by each sensor to the same time reference to avoid data inconsistency due to time differences. Spatial registration refers to aligning data from different sensors to the same spatial coordinate system to ensure spatial consistency of the data. After completing time synchronization and spatial registration, radar spatial feature data, laser point cloud feature data, infrared thermal feature data, and ultrasonic frequency domain feature data are extracted from the aligned multimodal data, and feature fusion is performed using a deep neural network to generate a comprehensive feature vector.

[0102] The specific processing process is as follows:

[0103] Time synchronization: Preprocessed multimodal data is synchronized using a spatiotemporal feature alignment algorithm to ensure consistency of data from different sensors at the same time point, generating time-synchronized data. The spatiotemporal feature alignment algorithm analyzes the timestamps of each sensor data and adjusts them to a unified time base, ensuring temporal consistency.

[0104] Spatial Registration: A feature-based spatial registration algorithm is used to spatially register time-synchronized data, ensuring alignment of data from different sensors in spatial coordinates and generating aligned multimodal data. The feature-based spatial registration algorithm identifies and matches feature points in the data from each sensor, aligning the data to the same spatial coordinate system and ensuring spatial consistency.

[0105] Feature extraction:

[0106] Radar spatial feature data extraction: A convolutional neural network is used to extract radar spatial feature data from aligned multimodal data. Through convolution and pooling operations, the convolutional neural network extracts features with spatial structure information from radar data.

[0107] Laser point cloud feature data extraction: Point cloud processing algorithms are used to extract laser point cloud feature data from aligned multimodal data. Point cloud processing algorithms analyze the geometry and spatial distribution of point cloud data to extract features with high-resolution spatial information.

[0108] Infrared thermal signature data extraction: Image processing algorithms are used to extract infrared thermal signature data from aligned multimodal data. These algorithms perform edge detection and region segmentation on infrared images to extract features that indicate thermal anomalies.

[0109] Ultrasonic frequency domain feature data extraction: Spectral analysis methods are used to extract ultrasonic frequency domain feature data from aligned multimodal data. Spectral analysis methods extract signals with frequency domain characteristics by performing a Fourier transform on the ultrasonic signal.

[0110] Feature Fusion: Extracted radar spatial feature data, laser point cloud feature data, infrared thermal feature data, and ultrasonic frequency domain feature data are fed into a deep neural network for feature fusion to generate a comprehensive feature vector. Through nonlinear transformations across multiple layers of neurons, the deep neural network fuses the feature data from each modality, generating a comprehensive feature vector containing multimodal information.

[0111] Fusion algorithms need to be able to handle the heterogeneity of data from different modalities while preserving the unique characteristics of each modality. For example, radar data focuses on spatial structure, infrared data on temperature distribution, and ultrasonic data on material properties.

[0112] Fusion algorithms should possess strong nonlinear feature extraction capabilities to extract useful composite features from multimodal data. They should also be able to dynamically adjust the weights of each modality, allowing the model to adaptively adjust its focus on different modal data based on the actual situation. Fusion algorithms should be able to fuse features at multiple levels, comprehensively considering everything from low-level basic features to high-level abstract features.

[0113] In the application of the present invention, the applicable algorithms include:

[0114] Convolutional Neural Networks (CNNs): CNNs can effectively extract spatial features and are particularly effective for image and point cloud data. Data from different modalities can be extracted using separate CNNs and then fused in a fusion layer.

[0115] Long Short-Term Memory (LSTM): LSTM is suitable for processing time series data and can be used to fuse time series features with other modal features. For example, the time series features of ultrasound data can be combined with other spatial features.

[0116] Self-Attention Mechanism: The self-attention mechanism dynamically adjusts the fusion weights of features from different modalities by calculating the importance weight of each feature. It is suitable for handling the changing importance of features from different modalities at different times.

[0117] Generative Adversarial Networks (GANs): GANs can be used to generate virtual data to augment training datasets, thereby improving the robustness and generalization capabilities of the model.

[0118] Multi-task learning framework: Through the multi-task learning framework, multiple related tasks can be trained simultaneously, allowing the model to share features and improve overall performance.

[0119] In one embodiment, the present invention is applied to detect internal defects of the runway in a certain airport runway maintenance project. First, three-dimensional ground penetrating radar, lidar, infrared imaging and ultrasonic sensors are used to synchronously collect runway data. Time synchronization is performed using a spatiotemporal feature alignment algorithm to ensure the consistency of all data at the same time point. Then, spatial registration is performed using a feature point-based spatial registration algorithm to ensure the alignment of the data in spatial coordinates. Next, a convolutional neural network is used to extract spatial features from radar data, a point cloud processing algorithm is used to extract geometric features from laser point cloud data, an image processing algorithm is used to extract thermal features from infrared data, and a spectrum analysis method is used to extract frequency domain features from ultrasonic data. Finally, these feature data are input into a deep neural network for feature fusion to generate a comprehensive feature vector. By analyzing the comprehensive feature vector, cracks, cavities and temperature abnormality areas inside the runway are accurately identified, providing reliable data support for subsequent maintenance work.

[0120] Through time synchronization and spatial registration, the consistency of data from different sensors is ensured, allowing multimodal data to be analyzed under the same temporal and spatial references. Feature extraction and fusion steps combine the strengths of each modality to generate a comprehensive feature vector containing comprehensive information. This approach not only improves data accuracy and reliability but also significantly enhances the precision and efficiency of disease detection through the collaborative work of multimodal data. In practical applications, this comprehensive analysis method can quickly and accurately detect various road surface defects, providing a scientific and reliable basis for maintenance and significantly improving the efficiency and quality of infrastructure maintenance.

[0121] like Figure 5 As shown in the figure, a comprehensive feature vector is used to construct a training dataset, and the dataset is expanded by generating an adversarial network. A multi-task learning framework is used to simultaneously train disease recognition and feature prediction tasks. Reinforcement learning is combined to optimize model parameters, and a self-attention mechanism and recurrent neural network are introduced to process time series data to generate preliminary recognition results.

[0122] Preferably, the generative adversarial network expanded dataset includes:

[0123] Generate virtual data using a generative adversarial network using a training dataset containing synthetic feature vectors;

[0124] The generated virtual data is fused with the original training data set to expand the training data set and generate an expanded training comprehensive feature vector;

[0125] Use a multi-task learning model to simultaneously train disease identification tasks and feature prediction tasks to optimize model parameters;

[0126] In the multi-task learning process, the disease recognition task and the feature prediction task share some model parameters while retaining the dedicated parameters of each task to generate a multi-task learning model.

[0127] In this paper, we use generative adversarial networks (GANs) to augment datasets, aiming to increase the diversity and scale of training datasets by generating high-quality virtual data, thereby improving the generalization and robustness of the model. The specific process includes the following steps:

[0128] Generate virtual data: Using a training dataset containing synthetic feature vectors, a generative adversarial network (GAN) generates virtual data. A GAN consists of a generator and a discriminator. The generator generates realistic virtual data, while the discriminator determines the authenticity of the data. Through adversarial training, the generator is continuously optimized, and the generated virtual data gradually approaches real data.

[0129] Data fusion: The generated virtual data is fused with the original training dataset to expand the training dataset and generate an expanded training comprehensive feature vector. This step, by introducing virtual data, increases the diversity of the training data and helps improve the generalization ability of the model.

[0130] Multi-task learning model training: Utilize a multi-task learning model to simultaneously train disease identification and feature prediction tasks to optimize model parameters. Multi-task learning allows different tasks to complement each other by sharing some model parameters, improving overall model performance.

[0131] Task parameter sharing and dedicated parameter retention: During multi-task learning, the disease recognition and feature prediction tasks share some model parameters while retaining their dedicated parameters to generate a multi-task learning model. This parameter sharing and dedicated parameter retention approach improves inter-task synergy while maintaining the independence of each task.

[0132] In one embodiment, the present invention was applied to detect internal road surface defects in a highway maintenance project. First, a comprehensive feature vector was generated using multimodal data to construct an initial training dataset. Next, virtual data was generated using a generative adversarial network (GAN). The generator in the GAN continuously generates virtual data similar to real data by learning the distribution of real data, while the discriminator continuously optimizes the generator's generation capabilities by identifying differences between real and virtual data. After multiple rounds of adversarial training, the generated virtual data exhibited high quality and diversity.

[0133] The generated virtual data is fused with the original training dataset. The expanded training dataset contains more diverse features and better represents the actual situation. In the multi-task learning model, both the defect recognition task and the feature prediction task are trained simultaneously. The defect recognition task analyzes multimodal data to accurately identify defects such as cracks and cavities within the road surface. The feature prediction task analyzes historical data to predict areas likely to develop defects in the future. By sharing some model parameters, the two tasks complement each other during training, improving overall model performance.

[0134] The resulting multi-task learning model can not only accurately identify current pavement damage but also effectively predict future damage risks, providing scientific data support for highway maintenance.

[0135] By expanding the dataset using a generative adversarial network, the diversity and scale of the training dataset were effectively increased, enhancing the model's generalization capabilities. By sharing parameters, the multi-task learning model optimized the synergy between the tasks of defect identification and feature prediction, improving overall model performance. In practical applications, this approach not only improves the accuracy and reliability of pavement defect detection but also provides comprehensive data support for maintenance decision-making, significantly improving the efficiency and quality of highway maintenance.

[0136] Preferably, the reinforcement learning optimization model parameters include:

[0137] Utilize the policy gradient algorithm to optimize the multi-task learning model and dynamically adjust the model parameters by simulating detection tasks under various environmental conditions;

[0138] In the reinforcement learning process, a reward function is constructed to give rewards based on the accuracy and robustness of the detection results to improve model performance and generate an adaptive deep learning model;

[0139] Introducing a self-attention mechanism into the adaptive deep learning model enables the model to focus on key features and enhance the ability to identify disease characteristics;

[0140] Use recurrent neural networks and long short-term memory networks to process time series data, improve the model's ability to capture time-varying features, and generate a time series enhanced adaptive deep learning model to generate preliminary recognition results.

[0141] In this invention, the reinforcement learning optimization model parameter step aims to continuously adjust and optimize the model parameters by simulating detection tasks under various environmental conditions to improve the model's accuracy and robustness in pavement disease detection. The specific process includes the following steps:

[0142] Policy Gradient Algorithm Optimization: Utilizes the Policy Gradient Algorithm to optimize multi-task learning models, dynamically adjusting model parameters by simulating detection tasks under various environmental conditions. The Policy Gradient Algorithm calculates gradients to guide parameter adjustments, allowing the model to gradually find the optimal parameter configuration through continuous trial and error.

[0143] Constructing a reward function: During the reinforcement learning process, a reward function is constructed to award rewards based on the accuracy and robustness of the detection results. The reward function is the core of reinforcement learning. By setting a reasonable reward mechanism, the model can be guided towards higher performance optimization. Specifically, when the model accurately identifies road defects, positive rewards are given; when the model misidentifies or misses a defect, negative rewards are given. Through continuous rewards and penalties, the model gradually improves its performance, ultimately generating an adaptive deep learning model.

[0144] Introducing the self-attention mechanism: The self-attention mechanism is incorporated into the adaptive deep learning model, enabling the model to focus on key features and enhance its ability to identify disease characteristics. By assigning attention weights to different features, the self-attention mechanism enables the model to more effectively identify key features when processing multimodal data, thereby improving the accuracy of disease detection.

[0145] Recurrent Neural Network and Long Short-Term Memory Network Processing: Recurrent Neural Networks (RNNs) and Long Short-Term Memory Networks (LSTMs) are used to process time series data, improving the model's ability to capture time-varying features. RNNs and LSTMs effectively process time series data, capturing temporal dependencies within the data through memory and forgetting mechanisms, generating a time series-enhanced adaptive deep learning model. This model can better capture and understand the dynamic changes in pavement defects, thereby generating preliminary identification results.

[0146] In one embodiment, the present invention was applied to detect internal road surface defects during a highway maintenance project. First, multimodal sensors collected road surface data to generate a comprehensive feature vector. Next, a policy gradient algorithm was used to optimize the multi-task learning model. During the simulation of detection tasks under various weather, traffic flow, and road conditions, the policy gradient algorithm dynamically adjusted model parameters, improving the model's adaptability.

[0147] A reward function is constructed to award rewards based on the accuracy and robustness of defect detection. For example, a positive reward is given when the model accurately identifies a pavement crack, while a negative reward is given when the model mistakenly identifies the pavement as intact. Through continuous positive and negative rewards, the model gradually optimizes its parameter configuration, generating an adaptive deep learning model.

[0148] Introducing a self-attention mechanism into the adaptive deep learning model allows the model to focus more closely on key features, such as abnormal pavement temperature and reflectance signals, thereby improving the accuracy of disease identification. Recurrent neural networks and long-short-term memory networks are used to process time series data, such as historical traffic flow data and seasonal variation data, to capture the dynamic trends of pavement disease.

[0149] Ultimately, a time series enhanced adaptive deep learning model was generated that can accurately identify and predict pavement defects, providing scientific and reliable data support for highway maintenance.

[0150] Reinforcement learning is used to optimize model parameters. The policy gradient algorithm is used to dynamically adjust model parameters, combined with the construction of a reasonable reward function, to guide the model toward higher performance. The introduction of a self-attention mechanism enables the model to more effectively identify key features, improving the accuracy of defect detection. Recurrent neural networks and long-short-term memory networks are used to process time series data, enhancing the model's ability to capture time-varying features. The resulting time series-enhanced adaptive deep learning model not only accurately identifies current pavement defects but also effectively predicts future defect trends, providing comprehensive data support for road maintenance.

[0151] like Figure 6 As shown in the figure, the energy amplitude, frequency domain spectrum characteristics and electromagnetic wave attenuation characteristics of the disease identification and feature prediction data in the preliminary identification results are calculated to generate dynamic statistical feature data, which are fed back to the adaptive deep learning model for secondary training. The hidden Markov model is used to probability optimize the dynamic statistical feature data, and the adaptive discriminant network based on the attention mechanism is used for intelligent filtering and optimization to finally generate the identification results.

[0152] Preferably, the calculation of the energy amplitude, frequency domain spectrum characteristics and electromagnetic wave attenuation characteristics includes:

[0153] Perform energy amplitude calculation on the disease identification and feature prediction data in the preliminary identification results to generate energy amplitude data;

[0154] Perform frequency domain spectrum feature calculation on the disease identification and feature prediction data in the preliminary identification results to generate frequency domain spectrum feature data;

[0155] The electromagnetic wave attenuation characteristics are calculated for the disease identification and feature prediction data in the preliminary identification results to generate electromagnetic wave attenuation characteristic data.

[0156] In this invention, the purpose of calculating energy amplitude, frequency domain spectral characteristics, and electromagnetic wave attenuation characteristics is to conduct a detailed analysis of the defect identification and feature prediction data in the preliminary identification results, extract key information from different perspectives, and generate corresponding feature data. This feature data is used to further optimize and enhance the performance of the adaptive deep learning model, thereby improving the accuracy and reliability of pavement internal defect detection.

[0157] Energy amplitude calculation is performed on the defect identification and feature prediction data in the preliminary identification results to generate energy amplitude data. Energy amplitude calculation reflects the overall energy of the signal by squared and summing the signal amplitude. Energy amplitude data can reveal the energy distribution characteristics of the defect area, helping to identify and locate the defect area. For example, underground cavities and cracks will show significant energy variations in radar signals, and energy amplitude calculation can accurately identify these abnormal areas.

[0158] Frequency domain spectral feature calculation is performed on the defect identification and feature prediction data from the preliminary identification results to generate frequency domain spectral feature data. Frequency domain spectral feature calculation works by converting time domain signals into frequency domain signals through Fourier transform and extracting the signal's frequency characteristics. Frequency domain spectral feature data can reveal the spectral characteristics of the defect area and help distinguish different types of defects. For example, cracks and cavities may exhibit different frequency distributions in the spectrum, and frequency domain spectral feature calculation can effectively distinguish these defect characteristics.

[0159] Electromagnetic wave attenuation characteristics are calculated for the damage identification and feature prediction data in the preliminary identification results to generate electromagnetic wave attenuation characteristic data. The principle of electromagnetic wave attenuation characteristic calculation is to analyze the attenuation of the signal during propagation to reflect the absorption and scattering characteristics of the medium. Electromagnetic wave attenuation characteristic data can reveal the medium characteristics of the damage area and help determine the depth and nature of the damage. For example, areas with high humidity will cause electromagnetic wave signals to attenuate faster. Electromagnetic wave attenuation characteristic calculation can identify groundwater and humid areas.

[0160] In one embodiment, the present invention was applied to detect internal road surface defects during a municipal road maintenance project. First, multimodal sensors collected road surface data, which was then initially processed and identified. Next, detailed analysis was performed on the defect identification and feature prediction data from the initial identification results.

[0161] Energy amplitude calculations of the radar signal data revealed that the energy in one area was significantly lower than that in the surrounding area, leading to the initial identification of an underground cavity. Further analysis of the energy amplitude data allowed the precise location and extent of the cavity to be pinpointed.

[0162] Calculating the frequency domain spectral characteristics of the ultrasonic signal data revealed a significant anomaly in the spectral distribution of a certain area, with frequency components concentrated in the low-frequency band. This was initially identified as an underground crack. Further analysis of the frequency domain spectral characteristics data allowed the depth and length of the crack to be determined.

[0163] Calculating the electromagnetic wave attenuation characteristics of the radar signal data revealed a significant increase in signal attenuation in a certain area, initially identifying it as a groundwater leak. Further analysis of the electromagnetic wave attenuation characteristics data allowed the location and extent of the leak to be identified.

[0164] The above calculations generate energy amplitude data, frequency domain spectrum characteristics, and electromagnetic wave attenuation characteristics. These data are used to further optimize and enhance the adaptive deep learning model, enabling it to more accurately identify and predict internal pavement defects, providing scientific and reliable data support for municipal road maintenance.

[0165] By calculating energy amplitude, frequency domain spectrum characteristics, and electromagnetic wave attenuation characteristics, this approach extracts and analyzes data for defect identification and prediction from various perspectives, generating detailed feature data. This feature data not only reveals the energy distribution, spectral characteristics, and dielectric properties of the defect area but also helps further optimize the performance of the adaptive deep learning model, improving the accuracy and reliability of defect detection. In practical applications, this approach can quickly and accurately identify and predict various road surface defects, providing comprehensive data support for maintenance decisions and significantly improving the efficiency and quality of infrastructure maintenance.

[0166] Preferably, the calculation expressions of the energy amplitude, frequency domain spectrum characteristics and electromagnetic wave attenuation characteristics are respectively:

[0167] Energy amplitude calculation:

[0168]

[0169] in, represents the energy amplitude, Indicates the amplitude of the radar signal;

[0170] Energy amplitude calculations can be used to identify energy changes in abnormal underground areas. Underground cavities or cracks can cause significant energy changes in radar signals, and calculating energy amplitudes can effectively identify these abnormal areas.

[0171] Frequency domain spectrum feature calculation:

[0172]

[0173] in, represents the frequency domain spectrum characteristics, represents the Fourier transform of a signal;

[0174] Frequency domain spectral feature calculations can reveal the spectral characteristics of the defect area. For example, cracks and cavities may exhibit different frequency distributions on the spectrum, and frequency domain spectral feature calculations can distinguish these defect characteristics.

[0175] Calculation of electromagnetic wave attenuation characteristics:

[0176]

[0177] in, Indicates distance The signal amplitude at represents the initial signal amplitude, Represents the attenuation coefficient.

[0178] Calculating electromagnetic wave attenuation characteristics can reveal the medium properties of diseased areas. For example, areas with high humidity will cause electromagnetic wave signals to attenuate faster. By calculating electromagnetic wave attenuation characteristics, groundwater and humid areas can be identified.

[0179] In one embodiment, the present invention was applied to detect internal road surface defects during a city road maintenance project. First, multimodal sensors collected road surface data, which was then preliminarily processed and identified. Next, energy amplitude, frequency domain spectrum characteristics, and electromagnetic wave attenuation characteristics were calculated.

[0180] Energy amplitude calculation: Energy amplitude calculation formula

[0181] Analysis of radar signal data revealed that the energy in a certain area was significantly lower than that in the surrounding area, leading to the initial identification of an underground cavity. Further analysis of the energy amplitude data pinpointed the cavity's location and extent.

[0182] Frequency domain spectrum feature calculation: Frequency domain spectrum feature calculation formula

[0183] Analysis of the ultrasonic signal data revealed a significant anomaly in the spectral distribution of a certain area, with frequency components concentrated in the low-frequency band. This was initially identified as an underground crack. Further analysis of the frequency domain spectral characteristic data determined the crack's depth and length.

[0184] Calculation of electromagnetic wave attenuation characteristics: Calculation formula of electromagnetic wave attenuation characteristics

[0185] Analysis of radar signal data revealed a significant increase in signal attenuation in a certain area, leading to the initial diagnosis of groundwater leakage. Further analysis of electromagnetic wave attenuation characteristic data identified the location and extent of the leakage area.

[0186] The above calculations generate energy amplitude data, frequency domain spectrum characteristics, and electromagnetic wave attenuation characteristics. These data are used to further optimize and enhance the adaptive deep learning model, enabling it to more accurately identify and predict internal pavement defects, providing scientific and reliable data support for municipal road maintenance.

[0187] By calculating energy amplitude, frequency domain spectrum characteristics, and electromagnetic wave attenuation characteristics, this method analyzes defect identification and feature prediction data in detail from different perspectives, generating rich feature data. This feature data not only reveals the energy distribution, spectral characteristics, and dielectric properties of the defect area, but also helps further optimize the performance of the adaptive deep learning model, improving the accuracy and reliability of defect detection. In practical applications, this method can quickly and accurately identify and predict various road surface defects, providing comprehensive data support for maintenance decisions and significantly improving the efficiency and quality of infrastructure maintenance.

[0188] Preferably, the feedback and probability optimization of the dynamic statistical feature data include:

[0189] Integrate energy amplitude data, frequency domain spectrum feature data, and electromagnetic wave attenuation characteristic data into dynamic statistical feature data, and feed it back into the adaptive deep learning model for secondary training and optimization to form enhanced preliminary recognition results;

[0190] The hidden Markov model is used to perform probability optimization on the enhanced preliminary recognition results, eliminate noise and misjudgment, and generate probability optimization results.

[0191] In the present invention, the feedback and probability optimization steps of dynamic statistical feature data are intended to further improve the accuracy and robustness of the disease detection model by integrating multiple feature data into dynamic statistical feature data and performing probability optimization using a hidden Markov model.

[0192] Energy amplitude data, frequency domain spectrum data, and electromagnetic wave attenuation characteristic data are integrated into dynamic statistical feature data and fed back into the adaptive deep learning model for secondary training and optimization. The process of integrating dynamic statistical feature data combines different types of feature data to form a comprehensive feature vector, enabling the model to extract useful features from multi-dimensional information.

[0193] By integrating dynamic statistical feature data, the model can more comprehensively understand the characteristics of the diseased area and improve the accuracy of disease identification. This data is fed back into the adaptive deep learning model for secondary training and optimization, allowing the model to further learn and adapt to new features, improving its generalization and robustness.

[0194] A hidden Markov model (HMM) is used to probabilistically optimize the enhanced initial recognition results. A hidden Markov model is a statistical model that describes the generation process of observation data through a sequence of states. It can effectively handle noise and uncertainty in time series data. Probabilistic optimization using the hidden Markov model can calculate the optimal probability distribution for each state based on the sequence characteristics of the observation data, eliminating noise and misjudgments.

[0195] Through the probabilistic optimization of the Hidden Markov Model, the accuracy of disease identification can be improved, and misjudgments and missed detections can be reduced. The resulting probabilistic optimization results can more accurately reflect the actual disease situation and provide reliable data support for subsequent maintenance decisions.

[0196] In one embodiment, the present invention was applied to detect internal road surface defects in a municipal road maintenance project. First, multimodal sensors collected road surface data, which was then initially processed and identified to generate preliminary identification results. Dynamic statistical feature data was then integrated and fed back, along with probability optimization.

[0197] The energy amplitude data, frequency domain spectrum characteristics, and electromagnetic wave attenuation characteristics from the initial identification results are integrated into dynamic statistical feature data. This integrated dynamic statistical feature data contains multi-dimensional disease characteristic information, such as energy changes, frequency spectrum characteristics, and dielectric attenuation characteristics. This data is fed back into the adaptive deep learning model for secondary training and optimization, enabling the model to better learn and adapt to new characteristics and improve the accuracy of disease identification.

[0198] A hidden Markov model was used to probabilistically optimize the enhanced initial identification results. For example, during the inspection process, an abnormal energy amplitude was detected in a certain area. The spectral characteristics showed a concentration of low frequencies, while the electromagnetic wave attenuation characteristics indicated high humidity. The hidden Markov model was used to probabilistically optimize these characteristic data, calculating the optimal probability distribution of the presence of disease in the area and eliminating noise and false positives. The resulting probabilistic optimization results confirmed the existence of underground cavities and cracks in the area, providing a scientific and reliable basis for decision-making in municipal road maintenance.

[0199] By integrating and feeding back dynamic statistical feature data and optimizing the probabilistic nature of hidden Markov models, this approach provides a detailed, multi-dimensional analysis of defect characteristics, generating more accurate and reliable defect identification results. This integrated dynamic statistical feature data enables the model to more comprehensively understand the characteristics of the defect area, improving the accuracy of defect identification. Probabilistic optimization of the hidden Markov model reduces noise and misjudgments, improving the accuracy and reliability of defect identification results. In practical applications, this approach can quickly and accurately identify and predict various pavement defects, providing comprehensive data support for maintenance decisions and significantly improving the efficiency and quality of infrastructure maintenance.

[0200] Preferably, the intelligent filtering and optimization includes:

[0201] The adaptive discriminant network based on the attention mechanism is used to intelligently filter and optimize the probability optimization results to generate intelligent filtering results;

[0202] Further optimization is performed based on the intelligent filtering results to generate the final recognition results.

[0203] In the present invention, the purpose of the intelligent filtering and optimization steps is to intelligently filter and optimize the probability optimization results by utilizing an adaptive discriminant network based on the attention mechanism, thereby generating more accurate disease identification results.

[0204] An adaptive discriminant network based on the attention mechanism intelligently filters and optimizes the probability optimization results. The attention mechanism is a technology that dynamically adjusts the model's focus. By assigning attention weights to different features, the model can more effectively focus on key features. When processing the probability optimization results, the adaptive discriminant network learns and adjusts attention weights to filter out noise and irrelevant features, extracting the most useful information for disease identification.

[0205] By introducing the attention mechanism, the adaptive discriminant network can automatically identify and extract key features from complex multimodal data, improving the accuracy of disease identification. The intelligent filtering results further refine and optimize the original probability optimization results, making disease identification more reliable.

[0206] Further optimization is performed based on the intelligent filtering results to generate the final recognition results. This further optimization step involves retraining and optimizing the deep learning model based on the intelligent filtering results to improve the accuracy and robustness of the recognition results. This process involves fine-tuning and optimizing the model parameters to better adapt the model to the various complex situations in the actual detection environment.

[0207] Through further optimization, the model can more accurately identify various types of pavement defects. The final recognition results not only have significant improvements in accuracy, but also excel in robustness and generalization, providing reliable data support for pavement defect detection.

[0208] In one embodiment, the present invention was applied to detect internal road surface defects during a highway maintenance project. First, multimodal sensors collected road surface data, which was then preliminarily processed and identified to generate preliminary identification results and probability optimization results. Next, intelligent filtering and optimization were performed.

[0209] Through the attention mechanism, the adaptive discriminant network can dynamically adjust its focus on different features. For example, in an inspection of a certain area, radar data and infrared imaging data both show anomalies, while ultrasonic data is relatively normal. The adaptive discriminant network can use the attention mechanism to assign more weight to radar and infrared imaging data, while reducing reliance on ultrasonic data, thereby more accurately identifying possible underground cavities and cracks in the area. Intelligent filtering results extract and optimize these key features, further enhancing recognition accuracy.

[0210] Based on the intelligent filtering results, a deep learning model is used for further optimization. For example, through further optimization steps, the model can conduct more in-depth analysis and prediction of potential damage areas identified in the intelligent filtering results based on historical data and environmental variables. The resulting identification results not only confirm the location of underground cavities and cracks, but also provide a detailed description of the damage scope and possible development trends. This information provides a scientific and reliable basis for decision-making on highway maintenance.

[0211] Through intelligent filtering and optimization steps, an adaptive discriminant network based on an attention mechanism is used to intelligently filter and optimize the probabilistic optimization results. This allows for a detailed, multi-dimensional analysis of defect characteristic data, generating more accurate and reliable defect identification results. Further optimization steps further refine the deep learning model through further training and optimization, resulting in even better accuracy, robustness, and generalization of the identification results. In practical applications, this approach can quickly and accurately identify and predict various road surface defects, providing comprehensive data support for maintenance decisions and significantly improving the efficiency and quality of infrastructure maintenance.

[0212] like Figure 7 As shown, a system for executing the method for detecting internal road surface defects using a three-dimensional ground penetrating radar comprises:

[0213] 3D ground-penetrating radar, lidar, infrared imaging sensors, and ultrasonic sensors for simultaneous non-destructive testing of road surfaces and collection of multimodal data;

[0214] The data preprocessing module is used to perform denoising, filtering and feature enhancement on the collected multimodal data to generate preprocessed multimodal data;

[0215] Data synchronization and registration module, used to perform temporal synchronization and spatial registration on preprocessed multimodal data to generate aligned multimodal data;

[0216] Feature extraction and fusion module, which is used to extract radar spatial feature data, laser point cloud feature data, infrared thermal feature data, and ultrasonic frequency domain feature data from the aligned multimodal data, and use deep neural network to perform feature fusion to generate a comprehensive feature vector;

[0217] A training data construction module is used to construct a training dataset using a comprehensive feature vector and to augment the dataset by generating an adversarial network;

[0218] A multi-task learning and optimization module is used to simultaneously train disease recognition and feature prediction tasks using a multi-task learning framework, combine reinforcement learning to optimize model parameters, and introduce a self-attention mechanism and recurrent neural network to process time series data to generate preliminary recognition results;

[0219] The dynamic statistical feature analysis and feedback module is used to calculate the energy amplitude, frequency domain spectrum characteristics, and electromagnetic wave attenuation characteristics of the disease identification and feature prediction data in the preliminary identification results, generate dynamic statistical feature data, and feed it back to the adaptive deep learning model for secondary training;

[0220] The probability optimization and intelligent filtering module is used to perform probability optimization on dynamic statistical feature data using a hidden Markov model, and to perform intelligent filtering and optimization using an adaptive discriminant network based on an attention mechanism to generate the final recognition result.

[0221] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0222] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0223] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0224] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0225] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0226] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0227] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can be implemented using any method or technology for information storage. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change RAM (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0228] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0229] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for detecting internal road surface defects using three-dimensional ground penetrating radar, characterized in that: The following steps are involved: Use 3D ground-penetrating radar, lidar, infrared imaging, and ultrasonic sensors to perform simultaneous non-destructive testing on the road surface, collect multimodal data, and perform denoising, filtering, and feature enhancement on the data to generate pre-processed multimodal data; Perform temporal synchronization and spatial registration on the preprocessed multimodal data to generate aligned multimodal data. Then, extract radar spatial feature data, laser point cloud feature data, infrared thermal feature data, and ultrasonic frequency domain feature data. Then, use a deep neural network to perform feature fusion and generate a comprehensive feature vector. A training dataset was constructed using comprehensive feature vectors, augmented with a generative adversarial network, and trained simultaneously on disease recognition and feature prediction tasks using a multi-task learning framework. Reinforcement learning was combined with model parameter optimization, and a self-attention mechanism and recurrent neural network were introduced to process time series data to generate preliminary recognition results. The energy amplitude, frequency domain spectrum characteristics and electromagnetic wave attenuation characteristics of the disease identification and feature prediction data in the preliminary identification results are calculated to generate dynamic statistical feature data, which are fed back into the adaptive deep learning model for secondary training. The hidden Markov model is used to probabilistically optimize the dynamic statistical feature data, and the adaptive discriminant network based on the attention mechanism is used for intelligent filtering and optimization to finally generate the identification results.

2. The method for detecting internal road defects using three-dimensional ground penetrating radar according to claim 1, characterized in that: The denoising, filtering and feature enhancement processes include: De-noising the radar data, filtering out interference signals through a high-pass filter, retaining valid signals, and generating pre-processed radar data; Filter the laser data, remove noise points through statistical filtering algorithms, enhance point cloud quality, and generate pre-processed laser data; Perform image enhancement processing on infrared data, improve thermal map details and contrast through histogram equalization technology, and generate pre-processed infrared data; The ultrasonic data is subjected to spectrum analysis and denoising, and useful echo signals are extracted through Fourier transform to generate preprocessed ultrasonic data.

3. The method for detecting internal road surface defects using three-dimensional ground penetrating radar according to claim 1, characterized in that: The time synchronization, spatial registration and feature fusion include: The pre-processed multimodal data is time-synchronized using the spatiotemporal feature alignment algorithm to ensure the consistency of data from different sensors at the same time point and generate time-synchronized data; Use a feature point-based spatial registration algorithm to spatially register the time-synchronized data to ensure alignment of different sensor data in spatial coordinates and generate aligned multimodal data; A convolutional neural network is used to extract radar spatial feature data from aligned multimodal data. A point cloud processing algorithm is used to extract laser point cloud feature data from aligned multimodal data. An image processing algorithm is used to extract infrared thermal feature data from aligned multimodal data. A spectrum analysis method is used to extract ultrasonic frequency domain feature data from aligned multimodal data. These feature data are then input into a deep neural network for feature fusion to generate a comprehensive feature vector.

4. The method for detecting internal road surface defects using three-dimensional ground penetrating radar according to claim 1, wherein: The generative adversarial network expanded dataset includes: Generate virtual data using a generative adversarial network using a training dataset containing synthetic feature vectors; The generated virtual data is fused with the original training data set to expand the training data set and generate an expanded training comprehensive feature vector; Use a multi-task learning model to simultaneously train disease identification tasks and feature prediction tasks to optimize model parameters; In the multi-task learning process, the disease recognition task and the feature prediction task share some model parameters while retaining the dedicated parameters of each task to generate a multi-task learning model.

5. The method for detecting internal road surface defects using three-dimensional ground penetrating radar according to claim 1, characterized in that: The reinforcement learning optimization model parameters include: Utilize the policy gradient algorithm to optimize the multi-task learning model and dynamically adjust the model parameters by simulating detection tasks under various environmental conditions; In the reinforcement learning process, a reward function is constructed to give rewards based on the accuracy and robustness of the detection results to improve model performance and generate an adaptive deep learning model; Introducing a self-attention mechanism into the adaptive deep learning model enables the model to focus on key features and enhance the ability to identify disease characteristics; Use recurrent neural networks and long short-term memory networks to process time series data, improve the model's ability to capture time-varying features, and generate a time series enhanced adaptive deep learning model to generate preliminary recognition results.

6. The method for detecting internal road surface defects using three-dimensional ground penetrating radar according to claim 1, characterized in that: The calculation of energy amplitude, frequency domain spectrum characteristics and electromagnetic wave attenuation characteristics includes: Perform energy amplitude calculation on the disease identification and feature prediction data in the preliminary identification results to generate energy amplitude data; Perform frequency domain spectrum feature calculation on the disease identification and feature prediction data in the preliminary identification results to generate frequency domain spectrum feature data; The electromagnetic wave attenuation characteristics are calculated for the disease identification and feature prediction data in the preliminary identification results to generate electromagnetic wave attenuation characteristic data.

7. The method for detecting internal road surface defects using three-dimensional ground penetrating radar according to claim 6, characterized in that: The calculation expressions of the energy amplitude, frequency domain spectrum characteristics and electromagnetic wave attenuation characteristics are respectively: Energy amplitude calculation: in, represents the energy amplitude, Indicates the amplitude of the radar signal; Frequency domain spectrum feature calculation: in, represents the frequency domain spectrum characteristics, represents the Fourier transform of a signal; Calculation of electromagnetic wave attenuation characteristics: in, Indicates distance The signal amplitude at represents the initial signal amplitude, Represents the attenuation coefficient.

8. The method for detecting internal road surface defects using three-dimensional ground penetrating radar according to claim 1, characterized in that: The feedback and probability optimization of the dynamic statistical feature data include: Integrate energy amplitude data, frequency domain spectrum feature data, and electromagnetic wave attenuation characteristic data into dynamic statistical feature data, and feed it back into the adaptive deep learning model for secondary training and optimization to form enhanced preliminary recognition results; The hidden Markov model is used to perform probability optimization on the enhanced preliminary recognition results, eliminate noise and misjudgment, and generate probability optimization results.

9. The method for detecting internal road surface defects using three-dimensional ground penetrating radar according to claim 1, characterized in that: The intelligent filtering and optimization include: The adaptive discriminant network based on the attention mechanism is used to intelligently filter and optimize the probability optimization results to generate intelligent filtering results; Further optimization is performed based on the intelligent filtering results to generate the final recognition results.

10. A system for executing the method for detecting internal road surface defects using three-dimensional ground penetrating radar according to any one of claims 1 to 9, characterized in that: include: 3D ground-penetrating radar, lidar, infrared imaging sensors, and ultrasonic sensors for simultaneous non-destructive testing of road surfaces and collection of multimodal data; The data preprocessing module is used to perform denoising, filtering and feature enhancement on the collected multimodal data to generate preprocessed multimodal data; Data synchronization and registration module, used to perform temporal synchronization and spatial registration on preprocessed multimodal data to generate aligned multimodal data; Feature extraction and fusion module, which is used to extract radar spatial feature data, laser point cloud feature data, infrared thermal feature data, and ultrasonic frequency domain feature data from the aligned multimodal data, and use deep neural network to perform feature fusion to generate a comprehensive feature vector; A training data construction module is used to construct a training dataset using a comprehensive feature vector and to augment the dataset by generating an adversarial network; A multi-task learning and optimization module is used to simultaneously train disease recognition and feature prediction tasks using a multi-task learning framework, combine reinforcement learning to optimize model parameters, and introduce a self-attention mechanism and recurrent neural network to process time series data to generate preliminary recognition results; The dynamic statistical feature analysis and feedback module is used to calculate the energy amplitude, frequency domain spectrum characteristics, and electromagnetic wave attenuation characteristics of the disease identification and feature prediction data in the preliminary identification results, generate dynamic statistical feature data, and feed it back to the adaptive deep learning model for secondary training; The probability optimization and intelligent filtering module is used to perform probability optimization on dynamic statistical feature data using a hidden Markov model, and to perform intelligent filtering and optimization using an adaptive discriminant network based on an attention mechanism to generate the final recognition result.

Images