Pavement internal disease detection method and system for three-dimensional ground penetrating radar
Through multimodal data fusion and deep learning technology, the internal disease detection model of road surface is optimized, and the problems of low data quality and processing efficiency under a single sensor are solved, achieving more efficient and accurate disease identification and detection.
Patent Information
- Application Number
- CN202411377532.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-30
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-09-30
AI Technical Summary
The existing internal road disease detection technology relies on a single sensor, has poor data quality and low processing efficiency, and the model is unstable under different environmental conditions, resulting in serious misjudgment and misjudgment phenomena, and lacks adaptive and optimization mechanisms.
Data is collected synchronously by using three-dimensional ground penetrating radar, lidar, infrared imaging and ultrasonic sensors, data alignment and feature extraction are performed through deep neural networks and multi-task learning frameworks, combined with generative adversarial networks and reinforcement learning optimization model parameters, and using Hidden Markov Models and adaptive discriminant networks are intelligently filtered and optimized.
It significantly improves the accuracy and efficiency of road surface disease detection, reduces the misjudgment rate, enhances the robustness and adaptability of the model, and realizes real-time feedback and continuous optimization.
Smart Images

Figure CN119026036A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent detection and analysis technology, and in particular to a method and system for detecting internal road surface defects using a three-dimensional ground penetrating radar. Background Art
[0002] Internal pavement defects such as cracks, voids, and water leakage not only affect the service life of the road, but may also cause serious traffic accidents. Therefore, accurate and efficient detection and identification of internal pavement defects is an important task in road maintenance and management. Current pavement internal defect detection technology mainly relies on a single sensor (such as 3D ground penetrating radar). When faced with extreme conditions such as complex geological structures and strong interference signals, the data quality is poor and the detection effect is not ideal. In addition, the data preprocessing and feature extraction process is complex and inefficient, especially when processing large-scale data. Existing recognition models are unstable under different environmental conditions, lack adaptive capabilities, and lack intelligent optimization and feedback mechanisms, resulting in low detection accuracy and efficiency, and serious misjudgment and missed judgment.
[0003] The existing technical solution (Chinese invention patent, publication number: CN118171144A, title: A method for optimizing the identification of internal road surface defects based on statistical characteristics of radar signals) has the following defects when attempting to solve the current problems: Existing technologies use a single sensor (such as 3D ground penetrating radar) for data collection and analysis. Since a single sensor performs poorly under complex geological structures and strong interference signal conditions, the data quality is poor and it is impossible to fully and accurately identify the disease. Existing technologies use traditional signal processing methods (such as filtering, denoising and spectrum analysis) for data preprocessing and feature extraction. Since these methods are cumbersome and require large amounts of computation, they are inefficient and difficult to process large amounts of data quickly and accurately. Existing technologies identify diseases based on fixed rules and parameters (such as threshold determination and pattern matching). Since these models lack the ability to adapt to different environmental conditions, the detection effect is unstable and the recognition accuracy and reliability are insufficient. The detection process of existing technologies is linear and lacks a feedback mechanism. Since the detection process cannot be dynamically adjusted and optimized, the overall detection efficiency and accuracy are low. Existing technologies rely on fixed rules and thresholds to determine diseases; since these rules are difficult to ensure high-precision recognition in complex environments, there are a large number of misjudgments and missed judgments in the recognition results. Summary of the invention
[0004] In view of the many problems existing in the above-mentioned prior art, the present invention provides a method and system for detecting internal road defects using three-dimensional ground penetrating radar. The present invention uses three-dimensional ground penetrating radar, lidar, infrared imaging and ultrasonic sensors to synchronously collect road surface data and pre-process the data; then, a deep neural network and a multi-task learning framework are used to align the data and extract features to generate a comprehensive feature vector; the data set is expanded by generating an adversarial network, and the model parameters are dynamically optimized in combination with reinforcement learning technology to improve the robustness and adaptability of the recognition model; finally, through intelligent filtering and probability optimization, the hidden Markov model and the adaptive discriminant network are used to further optimize the recognition results, significantly reduce the misjudgment rate, and improve the accuracy and efficiency of detection.
[0005] like Figure 1-Figure 3 As shown, a method for detecting internal road surface defects using three-dimensional ground penetrating radar comprises the following steps: Use 3D ground penetrating radar, lidar, infrared imaging and ultrasonic sensors to perform synchronous non-destructive testing on the road surface, collect multimodal data, and perform denoising, filtering and feature enhancement on the data to generate pre-processed multimodal data; Perform time synchronization and spatial registration on the preprocessed multimodal data to generate aligned multimodal data, and extract radar spatial feature data, laser point cloud feature data, infrared thermal feature data, and ultrasonic frequency domain feature data respectively. Then, use a deep neural network to perform feature fusion and generate a comprehensive feature vector. The training data set was constructed using comprehensive feature vectors, and the data set was expanded by generating adversarial networks. The disease recognition and feature prediction tasks were trained simultaneously using a multi-task learning framework. Reinforcement learning was combined to optimize model parameters, and self-attention mechanisms and recurrent neural networks were introduced to process time series data to generate preliminary recognition results. The energy amplitude, frequency domain spectrum characteristics and electromagnetic wave attenuation characteristics of the disease identification and feature prediction data in the preliminary identification results are calculated to generate dynamic statistical feature data, which are fed back to the adaptive deep learning model for secondary training. The hidden Markov model is used to probability optimize the dynamic statistical feature data, and the adaptive discriminant network based on the attention mechanism is used for intelligent filtering and optimization to finally generate the identification results.
[0006] Preferably, the denoising, filtering and feature enhancement processing includes: De-noising the radar data, filtering out interference signals through a high-pass filter, retaining valid signals, and generating pre-processed radar data; Filter the laser data, remove noise points through statistical filtering algorithm, enhance the point cloud quality, and generate pre-processed laser data; Perform image enhancement processing on infrared data, improve thermal map details and contrast through histogram equalization technology, and generate pre-processed infrared data; The ultrasonic data is subjected to spectrum analysis and denoising, and useful echo signals are extracted through Fourier transform to generate preprocessed ultrasonic data.
[0007] Preferably, the time synchronization, space registration and feature fusion include: The pre-processed multimodal data is synchronized using the spatiotemporal feature alignment algorithm to ensure the consistency of different sensor data at the same time point and generate time-synchronized data; Use feature point-based spatial registration algorithm to spatially register time-synchronized data to ensure alignment of different sensor data in spatial coordinates and generate aligned multimodal data; Convolutional neural networks are used to extract radar spatial feature data from aligned multimodal data. Point cloud processing algorithms are used to extract laser point cloud feature data from aligned multimodal data. Image processing algorithms are used to extract infrared thermal feature data from aligned multimodal data. Spectral analysis methods are used to extract ultrasonic frequency domain feature data from aligned multimodal data. These feature data are then input into deep neural networks for feature fusion to generate a comprehensive feature vector.
[0008] Preferably, the generative adversarial network expanded data set includes: Generate virtual data through a generative adversarial network using a training dataset containing synthetic feature vectors; The generated virtual data is merged with the original training data set to expand the training data set and generate an expanded training comprehensive feature vector; Use a multi-task learning model to simultaneously train disease identification tasks and feature prediction tasks to optimize model parameters; In the multi-task learning process, the disease recognition task and the feature prediction task share some model parameters while retaining the dedicated parameters of each task to generate a multi-task learning model.
[0009] Preferably, the reinforcement learning optimization model parameters include: The multi-task learning model is optimized using the policy gradient algorithm, and the model parameters are dynamically adjusted by simulating detection tasks under various environmental conditions; In the reinforcement learning process, a reward function is constructed to give rewards based on the accuracy and robustness of the detection results to improve model performance and generate an adaptive deep learning model; Introducing the self-attention mechanism into the adaptive deep learning model enables the model to focus on key features and enhance the ability to identify disease characteristics; Use recurrent neural networks and long short-term memory networks to process time series data, improve the model's ability to capture time-varying features, and generate a time series enhanced adaptive deep learning model to generate preliminary recognition results.
[0010] Preferably, the calculation of the energy amplitude, frequency domain spectrum characteristics and electromagnetic wave attenuation characteristics includes: Perform energy amplitude calculation on the disease identification and feature prediction data in the preliminary identification results to generate energy amplitude data; Perform frequency domain spectrum feature calculation on the disease identification and feature prediction data in the preliminary identification results to generate frequency domain spectrum feature data; The electromagnetic wave attenuation characteristics are calculated for the disease identification and feature prediction data in the preliminary identification results to generate electromagnetic wave attenuation characteristic data.
[0011] Preferably, the calculation expressions of the energy amplitude, frequency domain spectrum characteristics and electromagnetic wave attenuation characteristics are respectively: Energy amplitude calculation:
[0012] in, represents the energy amplitude, Indicates the amplitude of the radar signal; Frequency domain spectrum feature calculation:
[0013] in, represents the frequency domain spectrum characteristics, represents the Fourier transform of a signal; Calculation of electromagnetic wave attenuation characteristics:
[0014] in, Indicates distance The signal amplitude at represents the initial signal amplitude, Represents the attenuation coefficient.
[0015] Preferably, the feedback and probability optimization of the dynamic statistical feature data include: Integrate energy amplitude data, frequency domain spectrum feature data, and electromagnetic wave attenuation characteristic data into dynamic statistical feature data, and feed them back into the adaptive deep learning model for secondary training and optimization to form enhanced preliminary recognition results; The hidden Markov model is used to optimize the probability of the enhanced preliminary recognition results, eliminate noise and misjudgment, and generate probability optimization results.
[0016] Preferably, the intelligent filtering and optimization includes: The adaptive discriminant network based on the attention mechanism is used to intelligently filter and optimize the probability optimization results to generate intelligent filtering results; Further optimization is performed based on the intelligent filtering results to generate the final recognition results.
[0017] A system for executing the method for detecting internal road surface defects using three-dimensional ground penetrating radar, comprising: 3D ground penetrating radar, lidar, infrared imaging sensors and ultrasonic sensors for simultaneous non-destructive testing of road surfaces and collection of multi-modal data; A data preprocessing module is used to perform denoising, filtering and feature enhancement processing on the collected multimodal data to generate preprocessed multimodal data; A data synchronization and registration module is used to perform time synchronization and spatial registration on the preprocessed multimodal data to generate aligned multimodal data; The feature extraction and fusion module is used to extract radar spatial feature data, laser point cloud feature data, infrared thermal feature data and ultrasonic frequency domain feature data from the aligned multimodal data, and use deep neural network to perform feature fusion to generate a comprehensive feature vector; A training data construction module is used to construct a training data set using a comprehensive feature vector and to expand the data set by generating an adversarial network; The multi-task learning and optimization module is used to simultaneously train disease recognition and feature prediction tasks using a multi-task learning framework, optimize model parameters using reinforcement learning, and introduce a self-attention mechanism and recurrent neural network to process time series data to generate preliminary recognition results; Dynamic statistical feature analysis and feedback module, used to calculate the energy amplitude, frequency domain spectrum characteristics and electromagnetic wave attenuation characteristics of the disease identification and feature prediction data in the preliminary identification results, generate dynamic statistical feature data, and feed it back to the adaptive deep learning model for secondary training; The probability optimization and intelligent filtering module is used to perform probability optimization on dynamic statistical feature data using a hidden Markov model, and to perform intelligent filtering and optimization using an adaptive discriminant network based on an attention mechanism to generate the final recognition result.
[0018] Compared with the prior art, the advantages and beneficial effects of the present invention are: The present invention realizes more comprehensive and accurate pavement disease detection through multi-modal data fusion; The present invention improves the efficiency and accuracy of data processing by introducing deep neural networks and multi-task learning frameworks; The present invention realizes dynamic optimization of model parameters through reinforcement learning and adaptive deep learning models, thereby improving the robustness and adaptability of the model; The present invention realizes real-time feedback and continuous optimization through intelligent filtering, adaptive discrimination and closed-loop optimization system, thereby improving the accuracy and efficiency of detection; The present invention significantly reduces the misjudgment rate and improves the credibility of the recognition results through dynamic statistical feature analysis, hidden Markov model and generative adversarial training. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 It is a schematic diagram of the process of the present invention; Figure 2 It is a schematic diagram of multimodal data collection in the present invention; Figure 3 It is a data preprocessing flow chart in the present invention; Figure 4 It is a feature fusion process diagram in the present invention; Figure 5 Constructing a training data set and a model training diagram in the present invention; Figure 6 It is a characteristic analysis and optimization process diagram in the present invention; Figure 7 It is a structural block diagram of the system of the present invention. DETAILED DESCRIPTION
[0020] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is obvious that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.
[0021] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.
[0022] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.
[0023] In the case of using expressions such as "at least one of A, B, and C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.). In the case of using expressions such as "at least one of A, B, or C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, or C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).
[0024] Some block diagrams and / or flow charts are shown in the accompanying drawings. It should be understood that some boxes or combinations thereof in the block diagrams and / or flow charts can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that these instructions can create a device for implementing the functions / operations described in these block diagrams and / or flow charts when executed by the processor. The technology of the present disclosure can be implemented in the form of hardware and / or software (including firmware, microcode, etc.). In addition, the technology of the present disclosure can take the form of a computer program product on a computer-readable storage medium storing instructions, which can be used by an instruction execution system or in combination with an instruction execution system.
[0025] A method for detecting internal road surface defects using three-dimensional ground penetrating radar comprises the following steps: Use 3D ground penetrating radar, lidar, infrared imaging and ultrasonic sensors to perform synchronous non-destructive testing on the road surface, collect multimodal data, and perform denoising, filtering and feature enhancement on the data to generate pre-processed multimodal data; Preferably, the denoising, filtering and feature enhancement processing includes: De-noising the radar data, filtering out interference signals through a high-pass filter, retaining valid signals, and generating pre-processed radar data; Filter the laser data, remove noise points through statistical filtering algorithm, enhance the point cloud quality, and generate pre-processed laser data; Perform image enhancement processing on infrared data, improve thermal map details and contrast through histogram equalization technology, and generate pre-processed infrared data; The ultrasonic data is subjected to spectrum analysis and denoising, and useful echo signals are extracted through Fourier transform to generate preprocessed ultrasonic data.
[0026] In the present invention, three-dimensional ground penetrating radar, laser radar, infrared imaging and ultrasonic sensors are used to perform synchronous nondestructive testing on the road surface, aiming to obtain more comprehensive and accurate multimodal data through the coordinated work of multiple sensors. Three-dimensional ground penetrating radar is mainly used to detect underground structures and defects. Its working principle is to image and analyze the distribution and characteristics of underground objects by emitting electromagnetic waves and receiving reflected signals; laser radar uses the time difference of laser pulse reflection to perform high-precision three-dimensional imaging, which is mainly used to capture high-resolution point cloud data on the surface; infrared imaging generates thermal images by detecting infrared rays radiated by objects, which can effectively identify temperature anomaly areas and reflect the thermal characteristics of underground defects; ultrasonic sensors use the propagation characteristics of high-frequency sound waves in the medium to detect physical changes inside the medium through echo signals, and are suitable for detecting fine structures such as cracks and cavities.
[0027] The collected multimodal data needs to be processed by denoising, filtering and feature enhancement to generate preprocessed multimodal data. This step is a key link to ensure data quality and consistency. The specific processing process is as follows: Radar data denoising: The interference signal is filtered out through a high-pass filter, the effective signal is retained, and the pre-processed radar data is generated. The function of the high-pass filter is to remove low-frequency noise and enhance the high-frequency part of the signal, making the details of the underground structure clearer.
[0028] Laser data filtering: Use statistical filtering algorithms to remove noise points, enhance point cloud quality, and generate pre-processed laser data. The statistical filtering algorithm analyzes the statistical characteristics of point cloud data, removes isolated points and noise points, and ensures the accuracy and integrity of point cloud data.
[0029] Infrared data image enhancement processing: Use histogram equalization technology to enhance thermal image details and contrast and generate pre-processed infrared data. Histogram equalization significantly improves the image contrast by adjusting the grayscale distribution of the image, making it easier to identify temperature anomaly areas.
[0030] Ultrasonic data spectrum analysis and denoising: Useful echo signals are extracted through Fourier transform to generate pre-processed ultrasonic data. Fourier transform is used to convert time domain signals into frequency domain signals, extract useful frequency components from them, remove useless noise, and improve signal quality.
[0031] In one embodiment, the present invention is applied to detect internal road surface diseases in a municipal road maintenance project. Three-dimensional ground penetrating radar is used to detect underground pipelines and cavities, and multiple underground cavities are found after processing with a high-pass filter. LiDAR captures high-precision point cloud data on the surface, and after statistical filtering, the road surface settlement area is identified. Infrared imaging detects thermal anomalies, and after histogram equalization processing, the underground water leakage point is located. Ultrasonic sensors detect cracks in the road surface, and after Fourier transform processing, the depth and location of the cracks are confirmed. Through the comprehensive analysis and processing of multimodal data, accurate pre-processed multimodal data is generated, providing reliable data support for subsequent repair plans.
[0032] The pre-processed multimodal data generated by the above steps greatly improves the quality and consistency of the data, ensuring the accuracy and reliability of subsequent analysis. The collaborative work of multimodal data can not only fully reflect the internal disease conditions of the road surface, but also improve the accuracy and efficiency of disease detection through data fusion technology. In practical applications, this method can quickly and accurately detect various diseases inside the road surface, provide scientific and reliable repair basis, and significantly improve the efficiency and quality of municipal road maintenance.
[0033] Perform time synchronization and spatial registration on the preprocessed multimodal data to generate aligned multimodal data, and extract radar spatial feature data, laser point cloud feature data, infrared thermal feature data, and ultrasonic frequency domain feature data respectively. Then, use a deep neural network to perform feature fusion and generate a comprehensive feature vector. Preferably, the time synchronization, space registration and feature fusion include: The pre-processed multimodal data is synchronized using the spatiotemporal feature alignment algorithm to ensure the consistency of different sensor data at the same time point and generate time-synchronized data; Use feature point-based spatial registration algorithm to spatially register time-synchronized data to ensure alignment of different sensor data in spatial coordinates and generate aligned multimodal data; like Figure 4 As shown, a convolutional neural network is used to extract radar spatial feature data from the aligned multimodal data, a point cloud processing algorithm is used to extract laser point cloud feature data from the aligned multimodal data, an image processing algorithm is used to extract infrared thermal feature data from the aligned multimodal data, and a spectrum analysis method is used to extract ultrasonic frequency domain feature data from the aligned multimodal data. These feature data are then input into a deep neural network for feature fusion to generate a comprehensive feature vector.
[0034] In the present invention, the pre-processed multimodal data is time synchronized and spatially registered to ensure the consistency of data from different sensors at the same time point and spatial coordinates, thereby generating aligned multimodal data. Time synchronization refers to aligning the data collected by each sensor to the same time reference to avoid data inconsistency due to time differences. Spatial registration refers to aligning the data of different sensors to the same spatial coordinate system to ensure the consistency of the data in space. After completing time synchronization and spatial registration, radar spatial feature data, laser point cloud feature data, infrared thermal feature data, and ultrasonic frequency domain feature data are extracted from the aligned multimodal data, and feature fusion is performed using a deep neural network to generate a comprehensive feature vector.
[0035] The specific processing process is as follows: Time synchronization: Use the spatiotemporal feature alignment algorithm to synchronize the preprocessed multimodal data, ensure the consistency of different sensor data at the same time point, and generate time-synchronized data. The spatiotemporal feature alignment algorithm ensures the consistency of data in time by analyzing the timestamps of each sensor data and adjusting it to a unified time base.
[0036] Spatial registration: Use feature point-based spatial registration algorithms to spatially register time-synchronized data, ensure alignment of different sensor data in spatial coordinates, and generate aligned multimodal data. Feature point-based spatial registration algorithms align data to the same spatial coordinate system by identifying and matching feature points in each sensor data, ensuring spatial consistency.
[0037] Feature extraction: Radar spatial feature data extraction: Use convolutional neural networks to extract radar spatial feature data from aligned multimodal data. Convolutional neural networks extract features with spatial structure information from radar data through convolution and pooling operations.
[0038] Laser point cloud feature data extraction: Use point cloud processing algorithms to extract laser point cloud feature data from aligned multimodal data. The point cloud processing algorithm extracts features with high-resolution spatial information by analyzing the geometric shape and spatial distribution of point cloud data.
[0039] Infrared thermal feature data extraction: The infrared thermal feature data in the aligned multimodal data is extracted through image processing algorithms. The image processing algorithm extracts features with thermal anomaly information by performing edge detection, region segmentation and other operations on the infrared image.
[0040] Ultrasonic frequency domain feature data extraction: The spectrum analysis method is used to extract the ultrasonic frequency domain feature data from the aligned multimodal data. The spectrum analysis method extracts the signal with frequency domain characteristics by Fourier transforming the ultrasonic signal.
[0041] Feature fusion: The extracted radar spatial feature data, laser point cloud feature data, infrared thermal feature data and ultrasonic frequency domain feature data are input into the deep neural network for feature fusion to generate a comprehensive feature vector. The deep neural network fuses the feature data of each modality through nonlinear transformation of multiple layers of neurons to generate a comprehensive feature vector containing multi-modal information.
[0042] The fusion algorithm needs to be able to handle the heterogeneity of data from different modalities and retain their respective characteristic advantages. For example, radar data focuses on spatial structure, infrared data focuses on temperature distribution, and ultrasonic data focuses on material properties.
[0043] The fusion algorithm should have a strong nonlinear feature extraction capability to extract useful composite features from multimodal data. The fusion algorithm should be able to dynamically adjust the weight of each modal data so that the model can adaptively adjust the attention to different modal data according to the actual situation. The fusion algorithm should be able to perform feature fusion at multiple levels, from low-level basic features to high-level abstract features.
[0044] In the application of the present invention, the applicable algorithms include: Convolutional Neural Network (CNN): CNN can effectively extract spatial features and is very effective for image and point cloud data. Data of different modalities can be extracted through independent CNNs and then fused in the fusion layer.
[0045] Long Short-Term Memory (LSTM): LSTM is suitable for processing time series data and can be used to fuse time series features with other modal features. For example, the time series features of ultrasound data can be combined with other spatial features.
[0046] Self-Attention Mechanism: The self-attention mechanism dynamically adjusts the fusion weight of each modal feature by calculating the importance weight of each feature. It is suitable for processing the importance changes of different modal features at different times.
[0047] Generative Adversarial Networks (GANs): GANs can be used to generate virtual data to augment training datasets, thereby improving the robustness and generalization capabilities of the model.
[0048] Multi-task learning framework: Through the multi-task learning framework, multiple related tasks can be trained simultaneously, allowing the model to share features and improve overall performance.
[0049] In one embodiment, in a certain airport runway maintenance project, the present invention is applied to detect internal defects of the runway. First, three-dimensional ground penetrating radar, laser radar, infrared imaging and ultrasonic sensors are used to synchronously collect runway data. Time synchronization is performed through a spatiotemporal feature alignment algorithm to ensure the consistency of all data at the same time point. Then, a spatial registration algorithm based on feature points is used for spatial registration to ensure the alignment of data in spatial coordinates. Next, a convolutional neural network is used to extract spatial features from radar data, a point cloud processing algorithm is used to extract geometric features from laser point cloud data, an image processing algorithm is used to extract thermal features from infrared data, and a spectrum analysis method is used to extract frequency domain features from ultrasonic data. Finally, these feature data are input into a deep neural network for feature fusion to generate a comprehensive feature vector. By analyzing the comprehensive feature vector, cracks, cavities and temperature abnormality areas inside the runway are accurately identified, providing reliable data support for subsequent maintenance work.
[0050] Through time synchronization and spatial registration, the consistency of data from different sensors is ensured, so that multimodal data can be analyzed under the same time and space benchmark. The feature extraction and fusion steps combine the advantages of each modal data to generate a comprehensive feature vector containing comprehensive information. This method not only improves the accuracy and reliability of the data, but also significantly improves the accuracy and efficiency of disease detection through the collaborative work of multimodal data. In practical applications, this comprehensive analysis method can quickly and accurately detect various diseases inside the road surface, provide a scientific and reliable maintenance basis, and significantly improve the efficiency and quality of infrastructure maintenance.
[0051] like Figure 5 As shown in the figure, a comprehensive feature vector is used to construct a training data set, and the data set is expanded by generating an adversarial network. The multi-task learning framework is used to simultaneously train the disease recognition and feature prediction tasks. Reinforcement learning is combined to optimize the model parameters, and a self-attention mechanism and a recurrent neural network are introduced to process time series data to generate preliminary recognition results. Preferably, the generative adversarial network expanded data set includes: Generate virtual data through a generative adversarial network using a training dataset containing synthetic feature vectors; The generated virtual data is merged with the original training data set to expand the training data set and generate an expanded training comprehensive feature vector; Use a multi-task learning model to simultaneously train disease identification tasks and feature prediction tasks to optimize model parameters; In the multi-task learning process, the disease recognition task and the feature prediction task share some model parameters while retaining the dedicated parameters of each task to generate a multi-task learning model.
[0052] In this paper, the generative adversarial network (GAN) is used to expand the data set, aiming to enhance the diversity and scale of the training data set by generating high-quality virtual data, thereby improving the generalization ability and robustness of the model. The specific process includes the following steps: Generate virtual data: Use a training dataset containing comprehensive feature vectors to generate virtual data through a generative adversarial network. The generative adversarial network consists of a generator and a discriminator. The generator is responsible for generating realistic virtual data, while the discriminator determines the authenticity of the data. Through adversarial training, the generator is continuously optimized, and the generated virtual data gradually approaches the real data.
[0053] Data fusion: The generated virtual data is fused with the original training data set to expand the training data set and generate an expanded training comprehensive feature vector. This step increases the diversity of training data by introducing virtual data, which helps improve the generalization ability of the model.
[0054] Multi-task learning model training: Use the multi-task learning model to simultaneously train the disease identification task and feature prediction task to optimize the model parameters. Multi-task learning enables different tasks to complement each other by sharing some model parameters, thus improving the performance of the overall model.
[0055] Task parameter sharing and dedicated parameter retention: In the multi-task learning process, the disease recognition task and feature prediction task share some model parameters, while retaining the dedicated parameters of each task to generate a multi-task learning model. This method of parameter sharing and dedicated parameter retention can maintain the independence of each task while improving the synergy between tasks.
[0056] In one embodiment, the present invention is applied to detect internal road surface defects in a highway maintenance project. First, a comprehensive feature vector is generated using multimodal data to construct an initial training data set. Then, virtual data is generated through a generative adversarial network (GAN). The generator in the GAN continuously generates virtual data similar to the real data by learning the distribution of the real data, while the discriminator continuously optimizes the generator's generation ability by identifying the differences between the real data and the virtual data. After multiple adversarial trainings, the generated virtual data has high quality and diversity.
[0057] The generated virtual data is fused with the original training data set. The expanded training data set contains more diverse features and can better represent the actual situation. In the multi-task learning model, the disease identification task and the feature prediction task are trained simultaneously. The disease identification task accurately identifies cracks, cavities and other diseases inside the road surface by analyzing multimodal data; the feature prediction task predicts the possible disease areas in the future by analyzing historical data. By sharing some model parameters, the two tasks complement each other during the training process, improving the performance of the overall model.
[0058] The multi-task learning model finally generated can not only accurately identify current pavement damage, but also effectively predict future damage risks, providing scientific data support for highway maintenance.
[0059] By expanding the dataset through generative adversarial networks, the diversity and scale of the training dataset are effectively improved, and the generalization ability of the model is enhanced. The multi-task learning model optimizes the synergy of disease identification and feature prediction tasks by sharing parameters, improving the performance of the overall model. In practical applications, this method can not only improve the accuracy and reliability of pavement disease detection, but also provide comprehensive data support for maintenance decisions, significantly improving the efficiency and quality of highway maintenance.
[0060] Preferably, the reinforcement learning optimization model parameters include: The multi-task learning model is optimized using the policy gradient algorithm, and the model parameters are dynamically adjusted by simulating detection tasks under various environmental conditions; In the reinforcement learning process, a reward function is constructed to give rewards based on the accuracy and robustness of the detection results to improve model performance and generate an adaptive deep learning model; Introducing the self-attention mechanism into the adaptive deep learning model enables the model to focus on key features and enhance the ability to identify disease characteristics; Use recurrent neural networks and long short-term memory networks to process time series data, improve the model's ability to capture time-varying features, and generate a time series enhanced adaptive deep learning model to generate preliminary recognition results.
[0061] In the present invention, the step of optimizing model parameters by reinforcement learning aims to continuously adjust and optimize model parameters by simulating detection tasks under various environmental conditions, so as to improve the accuracy and robustness of the model in pavement disease detection. The specific process includes the following steps: Policy gradient algorithm optimization: The policy gradient algorithm is used to optimize the multi-task learning model, and the model parameters are dynamically adjusted by simulating detection tasks under various environmental conditions. The policy gradient algorithm guides the adjustment direction of parameters by calculating the gradient, so that the model gradually finds the optimal parameter configuration through continuous trial and adjustment.
[0062] Constructing a reward function: During the reinforcement learning process, a reward function is constructed to give rewards based on the accuracy and robustness of the detection results. The reward function is the core of reinforcement learning. By setting a reasonable reward mechanism, the model can be guided to optimize towards higher performance. Specifically, when the model can accurately identify road defects, a positive reward is given; when the model misjudges or misses a defect, a negative reward is given. Through continuous rewards and penalties, the model gradually improves its performance and eventually generates an adaptive deep learning model.
[0063] Introduction of self-attention mechanism: The self-attention mechanism is introduced in the adaptive deep learning model, so that the model can focus on key features and enhance the ability to identify disease features. The self-attention mechanism allocates attention weights to different features, so that the model can more effectively identify key features when processing multimodal data, thereby improving the accuracy of disease detection.
[0064] Recurrent Neural Network and Long Short-Term Memory Network Processing: Use recurrent neural network (RNN) and long short-term memory network (LSTM) to process time series data and improve the model's ability to capture time-varying features. RNN and LSTM can effectively process time series data, capture the temporal dependencies in the data through memory and forgetting mechanisms, and generate a time series enhanced adaptive deep learning model. This model can better capture and understand the dynamic changes of pavement diseases, thereby generating preliminary recognition results.
[0065] In one embodiment, the present invention is applied to detect road surface internal defects in a highway maintenance project. First, road surface data is collected by multimodal sensors to generate comprehensive feature vectors. Then, the multi-task learning model is optimized using the policy gradient algorithm. In the process of simulating detection tasks under various weather, traffic flow and road conditions, the policy gradient algorithm dynamically adjusts the model parameters to improve the adaptability of the model.
[0066] Construct a reward function to give rewards based on the accuracy and robustness of disease detection. For example, when the model accurately identifies pavement cracks, a positive reward is given; when the model misjudges that the pavement is intact, a negative reward is given. Through continuous positive and negative rewards, the model gradually optimizes its parameter configuration and generates an adaptive deep learning model.
[0067] The self-attention mechanism is introduced into the adaptive deep learning model, so that the model can focus more on key features, such as abnormal road surface temperature and abnormal reflection signal, thereby improving the accuracy of disease identification. Recurrent neural networks and long short-term memory networks are used to process time series data, such as historical traffic flow data and seasonal change data, to capture the dynamic trend of road surface diseases.
[0068] Ultimately, a time series enhanced adaptive deep learning model is generated, which can accurately identify and predict pavement defects, providing scientific and reliable data support for highway maintenance.
[0069] The model parameters are optimized through reinforcement learning, and the policy gradient algorithm is used to dynamically adjust the model parameters. Combined with the construction of a reasonable reward function, the model is guided to optimize towards higher performance. The self-attention mechanism is introduced to enable the model to more effectively identify key features and improve the accuracy of disease detection. Recurrent neural networks and long short-term memory networks are used to process time series data to enhance the model's ability to capture time-varying features. The resulting time series enhanced adaptive deep learning model can not only accurately identify current road surface diseases, but also effectively predict future disease trends, providing comprehensive data support for road maintenance.
[0070] like Figure 6 As shown, the energy amplitude, frequency domain spectrum characteristics and electromagnetic wave attenuation characteristics of the disease identification and feature prediction data in the preliminary identification results are calculated to generate dynamic statistical feature data, which are fed back to the adaptive deep learning model for secondary training. The hidden Markov model is used to probability optimize the dynamic statistical feature data, and the adaptive discriminant network based on the attention mechanism is used for intelligent filtering and optimization to finally generate the identification results.
[0071] Preferably, the calculation of the energy amplitude, frequency domain spectrum characteristics and electromagnetic wave attenuation characteristics includes: Perform energy amplitude calculation on the disease identification and feature prediction data in the preliminary identification results to generate energy amplitude data; Perform frequency domain spectrum feature calculation on the disease identification and feature prediction data in the preliminary identification results to generate frequency domain spectrum feature data; The electromagnetic wave attenuation characteristics are calculated for the disease identification and feature prediction data in the preliminary identification results to generate electromagnetic wave attenuation characteristic data.
[0072] In the present invention, the purpose of calculating energy amplitude, frequency domain spectrum characteristics and electromagnetic wave attenuation characteristics is to conduct a detailed analysis of the disease identification and feature prediction data in the preliminary identification results, extract key information from different angles, and generate corresponding feature data. These feature data are used to further optimize and enhance the performance of the adaptive deep learning model, thereby improving the accuracy and reliability of pavement internal disease detection.
[0073] Energy amplitude calculation is performed on the defect identification and feature prediction data in the preliminary identification results to generate energy amplitude data. The principle of energy amplitude calculation is to reflect the overall energy of the signal by squaring and summing the amplitude of the signal. Energy amplitude data can reveal the energy distribution characteristics of the defect area and help identify and locate the defect area. For example, underground cavities and cracks will show obvious energy changes in radar signals, and these abnormal areas can be accurately identified through energy amplitude calculation.
[0074] The frequency domain spectrum feature calculation is performed on the defect identification and feature prediction data in the preliminary identification results to generate frequency domain spectrum feature data. The principle of frequency domain spectrum feature calculation is to convert the time domain signal into the frequency domain signal through Fourier transform and extract the frequency characteristics of the signal. The frequency domain spectrum feature data can reveal the spectrum characteristics of the defect area and help distinguish different types of defects. For example, cracks and cavities may show different frequency distributions in the spectrum, and these defect characteristics can be effectively distinguished through frequency domain spectrum feature calculation.
[0075] The electromagnetic wave attenuation characteristics are calculated for the disease identification and feature prediction data in the preliminary identification results to generate electromagnetic wave attenuation characteristics data. The principle of electromagnetic wave attenuation characteristics calculation is to reflect the absorption and scattering characteristics of the medium by analyzing the attenuation of the signal during the propagation process. The electromagnetic wave attenuation characteristics data can reveal the medium characteristics of the diseased area and help determine the depth and nature of the disease. For example, areas with high humidity will cause electromagnetic wave signals to attenuate faster. Through the calculation of electromagnetic wave attenuation characteristics, groundwater and humid areas can be identified.
[0076] In one embodiment, the present invention is applied to detect road surface internal defects in a municipal road maintenance project. First, road surface data is collected by multimodal sensors, and the data is preliminarily processed and identified. Then, the defect identification and feature prediction data in the preliminary identification results are analyzed in detail.
[0077] Energy amplitude calculations were performed on radar signal data, and it was found that the energy in a certain area was significantly lower than that in the surrounding areas, which was initially determined to be an underground cavity. Further analysis of the energy amplitude data was used to accurately locate the location and range of the cavity.
[0078] By calculating the frequency domain spectrum characteristics of the ultrasonic signal data, it was found that the spectrum distribution in a certain area was obviously abnormal, and the frequency components were concentrated in the low frequency band. It was initially determined to be an underground crack. By further analyzing the frequency domain spectrum characteristic data, the depth and length of the crack were determined.
[0079] By calculating the electromagnetic wave attenuation characteristics of the radar signal data, it was found that the signal attenuation rate in a certain area was significantly accelerated, and it was initially determined to be groundwater leakage. By further analyzing the electromagnetic wave attenuation characteristic data, the location and range of the leakage area were identified.
[0080] Through the above calculations, energy amplitude data, frequency domain spectrum feature data and electromagnetic wave attenuation characteristic data are generated. These data are used to further optimize and enhance the adaptive deep learning model, so that the model can more accurately identify and predict internal road diseases, providing scientific and reliable data support for municipal road maintenance.
[0081] By calculating the energy amplitude, frequency domain spectrum characteristics and electromagnetic wave attenuation characteristics, the disease identification and feature prediction data are extracted and analyzed from different angles to generate detailed feature data. These feature data can not only reveal the energy distribution, spectrum characteristics and medium characteristics of the diseased area, but also help to further optimize the performance of the adaptive deep learning model and improve the accuracy and reliability of disease detection. In practical applications, this method can quickly and accurately identify and predict various diseases inside the road surface, provide comprehensive data support for maintenance decisions, and significantly improve the efficiency and quality of infrastructure maintenance.
[0082] Preferably, the calculation expressions of the energy amplitude, frequency domain spectrum characteristics and electromagnetic wave attenuation characteristics are respectively: Energy amplitude calculation:
[0083] in, represents the energy amplitude, Indicates the amplitude of the radar signal; By calculating the energy amplitude, we can identify the energy changes in abnormal underground areas. Underground cavities or cracks can cause significant changes in the energy of radar signals. By calculating the energy amplitude, we can effectively identify these abnormal areas.
[0084] Frequency domain spectrum feature calculation:
[0085] in, represents the frequency domain spectrum characteristics, represents the Fourier transform of a signal; Frequency domain spectrum feature calculation can reveal the spectrum characteristics of the defect area. For example, cracks and cavities may show different frequency distributions in the spectrum, and these defect characteristics can be distinguished by frequency domain spectrum feature calculation.
[0086] Calculation of electromagnetic wave attenuation characteristics:
[0087] in, Indicates distance The signal amplitude at represents the initial signal amplitude, Represents the attenuation coefficient.
[0088] The calculation of electromagnetic wave attenuation characteristics can reveal the medium characteristics of the diseased area. For example, areas with high humidity will cause electromagnetic wave signals to attenuate faster. By calculating the electromagnetic wave attenuation characteristics, groundwater and humid areas can be identified.
[0089] In one embodiment, the present invention is applied to detect road surface internal defects in a certain urban road maintenance project. First, road surface data is collected through a multimodal sensor, and the data is preliminarily processed and identified. Then, energy amplitude, frequency domain spectrum characteristics and electromagnetic wave attenuation characteristics are calculated.
[0090] Energy amplitude calculation: Energy amplitude calculation formula
[0091] Analysis of radar signal data revealed that the energy in a certain area was significantly lower than that in the surrounding area, and it was initially determined to be an underground cavity. Further analysis of the energy amplitude data accurately located the location and range of the cavity.
[0092] Frequency domain spectrum feature calculation: Frequency domain spectrum feature calculation formula
[0093] Analysis of the ultrasonic signal data revealed that the frequency spectrum distribution in a certain area was significantly abnormal, with the frequency components concentrated in the low-frequency band, which was initially determined to be an underground crack. Further analysis of the frequency domain spectrum feature data determined the depth and length of the crack.
[0094] Calculation of electromagnetic wave attenuation characteristics: Calculation formula of electromagnetic wave attenuation characteristics
[0095] Analysis of radar signal data revealed that the signal attenuation rate in a certain area was significantly accelerated, which was initially determined to be groundwater leakage. Further analysis of electromagnetic wave attenuation characteristic data identified the location and range of the leakage area.
[0096] Through the above calculations, energy amplitude data, frequency domain spectrum feature data and electromagnetic wave attenuation characteristic data are generated. These data are used to further optimize and enhance the adaptive deep learning model, so that the model can more accurately identify and predict internal road diseases, providing scientific and reliable data support for municipal road maintenance.
[0097] By calculating the energy amplitude, frequency domain spectrum characteristics and electromagnetic wave attenuation characteristics, the disease identification and feature prediction data are analyzed in detail from different angles to generate rich feature data. These feature data can not only reveal the energy distribution, spectrum characteristics and medium characteristics of the diseased area, but also help further optimize the performance of the adaptive deep learning model and improve the accuracy and reliability of disease detection. In practical applications, this method can quickly and accurately identify and predict various diseases inside the road surface, provide comprehensive data support for maintenance decisions, and significantly improve the efficiency and quality of infrastructure maintenance.
[0098] Preferably, the feedback and probability optimization of the dynamic statistical feature data include: Integrate energy amplitude data, frequency domain spectrum feature data, and electromagnetic wave attenuation characteristic data into dynamic statistical feature data, and feed them back into the adaptive deep learning model for secondary training and optimization to form enhanced preliminary recognition results; The hidden Markov model is used to optimize the probability of the enhanced preliminary recognition results, eliminate noise and misjudgment, and generate probability optimization results.
[0099] In the present invention, the feedback and probability optimization steps of dynamic statistical feature data are intended to further improve the accuracy and robustness of the disease detection model by integrating multiple feature data into dynamic statistical feature data and performing probability optimization using a hidden Markov model.
[0100] The energy amplitude data, frequency domain spectrum feature data and electromagnetic wave attenuation characteristic data are integrated into dynamic statistical feature data and fed back to the adaptive deep learning model for secondary training and optimization. The process of integrating dynamic statistical feature data is to fuse different types of feature data together to form a comprehensive feature vector, so that the model can extract useful features from multi-dimensional information.
[0101] Through the integrated dynamic statistical feature data, the model can more comprehensively understand the characteristics of the diseased area and improve the accuracy of disease identification. Feedback to the adaptive deep learning model for secondary training and optimization enables the model to further learn and adapt to new features and improve the generalization and robustness of the model.
[0102] The hidden Markov model (HMM) is used to optimize the probability of the enhanced preliminary recognition results. The hidden Markov model is a statistical model that describes the generation process of observation data through a state sequence and can effectively handle the noise and uncertainty in time series data. By performing probability optimization with the hidden Markov model, the optimal probability distribution of each state can be calculated based on the sequence characteristics of the observation data, eliminating noise and misjudgment.
[0103] Through the probability optimization of the hidden Markov model, the accuracy of disease identification results can be improved, and misjudgment and missed judgment can be reduced. The final probability optimization results can more accurately reflect the actual situation of the disease and provide reliable data support for subsequent maintenance decisions.
[0104] In one embodiment, the present invention is applied to detect road surface internal defects in a municipal road maintenance project. First, road surface data is collected by multimodal sensors, and preliminary processing and recognition are performed to generate preliminary recognition results. Then, dynamic statistical feature data is integrated and fed back, and probability optimization is performed.
[0105] The energy amplitude data, frequency domain spectrum feature data and electromagnetic wave attenuation characteristic data in the preliminary identification results are integrated into dynamic statistical feature data. The integrated dynamic statistical feature data contains multi-dimensional disease feature information, such as energy changes, spectrum characteristics and medium attenuation characteristics. These data are fed back to the adaptive deep learning model for secondary training and optimization, so that the model can better learn and adapt to new features and improve the accuracy of disease identification.
[0106] The hidden Markov model is used to perform probability optimization on the enhanced preliminary recognition results. For example, during the detection process, it was found that the energy amplitude in a certain area was abnormal, the spectrum characteristics showed low-frequency concentration, and the electromagnetic wave attenuation characteristics indicated that there was high humidity in the area. The hidden Markov model was used to perform probability optimization on these feature data, calculate the optimal probability distribution of the presence of diseases in the area, and eliminate noise and misjudgment. The final probability optimization results showed that there were indeed underground cavities and cracks in the area, providing a scientific and reliable decision-making basis for municipal road maintenance.
[0107] Through the integration and feedback of dynamic statistical feature data and the probability optimization of the hidden Markov model, the disease feature data is analyzed in detail from multiple dimensions to generate more accurate and reliable disease identification results. The integrated dynamic statistical feature data enables the model to have a more comprehensive understanding of the characteristics of the diseased area and improve the accuracy of disease identification. Through the probability optimization of the hidden Markov model, noise and misjudgment are reduced, and the accuracy and reliability of disease identification results are improved. In practical applications, this method can quickly and accurately identify and predict various diseases inside the pavement, provide comprehensive data support for maintenance decisions, and significantly improve the efficiency and quality of infrastructure maintenance.
[0108] Preferably, the intelligent filtering and optimization includes: The adaptive discriminant network based on the attention mechanism is used to intelligently filter and optimize the probability optimization results to generate intelligent filtering results; Further optimization is performed based on the intelligent filtering results to generate the final recognition results.
[0109] In the present invention, the purpose of the intelligent filtering and optimization steps is to generate more accurate disease identification results by intelligently filtering and optimizing the probability optimization results using an adaptive discriminant network based on the attention mechanism.
[0110] The adaptive discriminant network based on the attention mechanism is used to intelligently filter and optimize the probability optimization results. The attention mechanism is a technology that can dynamically adjust the focus of the model. By assigning attention weights to different features, the model can focus on key features more effectively. When processing the probability optimization results, the adaptive discriminant network learns and adjusts the attention weights to filter out noise and irrelevant features and extract the most useful information for disease identification.
[0111] By introducing the attention mechanism, the adaptive discriminant network can automatically identify and extract key features in complex multimodal data, improving the accuracy of disease identification. The intelligent filtering result is a further refinement and optimization of the original probability optimization result, making the disease identification result more reliable.
[0112] Further optimization is performed based on the intelligent filtering results to generate the final recognition results. The further optimization step refers to improving the accuracy and robustness of the recognition results through retraining and optimization of the deep learning model based on the intelligent filtering results. This process includes fine-tuning and optimization of the model parameters so that the model can better adapt to various complex situations in the actual detection environment.
[0113] Through further optimization, the model can more accurately identify various types of pavement defects. The final recognition results not only have significant improvements in accuracy, but also excel in robustness and generalization ability, providing reliable data support for pavement defect detection.
[0114] In one embodiment, the present invention is applied to detect road surface defects in a highway maintenance project. First, road surface data is collected by multimodal sensors, and the data is preliminarily processed and identified to generate preliminary identification results and probability optimization results. Then, intelligent filtering and optimization are performed.
[0115] Through the attention mechanism, the adaptive discriminant network can dynamically adjust the focus on different features. For example, in the detection of a certain area, the radar data and infrared imaging data both show abnormalities, but the ultrasonic data is relatively normal. Through the adaptive discriminant network, more weight can be assigned to radar data and infrared imaging data according to the attention mechanism, while reducing the reliance on ultrasonic data, so as to more accurately identify the underground cavities and cracks that may exist in the area. The intelligent filtering results extract and optimize these key features, further enhancing the accuracy of recognition.
[0116] Based on the intelligent filtering results, the deep learning model is used for further optimization. For example, the potential disease areas identified in the intelligent filtering results can be further analyzed and predicted based on historical data and environmental variables through further optimization steps. The identification results generated in the end not only confirm the location of underground cavities and cracks, but also provide detailed disease ranges and possible development trends. This information provides a scientific and reliable basis for decision-making for highway maintenance.
[0117] Through intelligent filtering and optimization steps, the probability optimization results are intelligently filtered and optimized using an adaptive discriminant network based on an attention mechanism, and the disease feature data is analyzed in detail from multiple dimensions to generate more accurate and reliable disease identification results. Further optimization steps are performed through retraining and optimization of the deep learning model to make the recognition results more excellent in accuracy, robustness and generalization ability. In practical applications, this method can quickly and accurately identify and predict various diseases inside the road surface, provide comprehensive data support for maintenance decisions, and significantly improve the efficiency and quality of infrastructure maintenance.
[0118] like Figure 7 As shown, a system for executing the method for detecting internal road surface defects using three-dimensional ground penetrating radar comprises: 3D ground penetrating radar, lidar, infrared imaging sensors and ultrasonic sensors for simultaneous non-destructive testing of road surfaces and collection of multi-modal data; A data preprocessing module is used to perform denoising, filtering and feature enhancement processing on the collected multimodal data to generate preprocessed multimodal data; A data synchronization and registration module is used to perform time synchronization and spatial registration on the preprocessed multimodal data to generate aligned multimodal data; The feature extraction and fusion module is used to extract radar spatial feature data, laser point cloud feature data, infrared thermal feature data and ultrasonic frequency domain feature data from the aligned multimodal data, and use deep neural network to perform feature fusion to generate a comprehensive feature vector; A training data construction module is used to construct a training data set using a comprehensive feature vector and to expand the data set by generating an adversarial network; The multi-task learning and optimization module is used to simultaneously train disease recognition and feature prediction tasks using a multi-task learning framework, optimize model parameters using reinforcement learning, and introduce a self-attention mechanism and recurrent neural network to process time series data to generate preliminary recognition results; Dynamic statistical feature analysis and feedback module, used to calculate the energy amplitude, frequency domain spectrum characteristics and electromagnetic wave attenuation characteristics of the disease identification and feature prediction data in the preliminary identification results, generate dynamic statistical feature data, and feed it back to the adaptive deep learning model for secondary training; The probability optimization and intelligent filtering module is used to perform probability optimization on dynamic statistical feature data using a hidden Markov model, and to perform intelligent filtering and optimization using an adaptive discriminant network based on an attention mechanism to generate the final recognition result.
[0119] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0120] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0121] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0122] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0123] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0124] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0125] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0126] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0127] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A method for detecting internal road surface defects using three-dimensional ground penetrating radar, characterized in that: The following steps are involved: Use 3D ground penetrating radar, lidar, infrared imaging and ultrasonic sensors to perform synchronous non-destructive testing on the road surface, collect multimodal data, and perform denoising, filtering and feature enhancement on the data to generate pre-processed multimodal data; Perform time synchronization and spatial registration on the preprocessed multimodal data to generate aligned multimodal data, and extract radar spatial feature data, laser point cloud feature data, infrared thermal feature data, and ultrasonic frequency domain feature data respectively. Then, use a deep neural network to perform feature fusion and generate a comprehensive feature vector. The training data set was constructed using comprehensive feature vectors, and the data set was expanded by generating adversarial networks. The disease recognition and feature prediction tasks were trained simultaneously using a multi-task learning framework. Reinforcement learning was combined to optimize model parameters, and self-attention mechanisms and recurrent neural networks were introduced to process time series data to generate preliminary recognition results. The energy amplitude, frequency domain spectrum characteristics and electromagnetic wave attenuation characteristics of the disease identification and feature prediction data in the preliminary identification results are calculated to generate dynamic statistical feature data, which are fed back to the adaptive deep learning model for secondary training. The hidden Markov model is used to probability optimize the dynamic statistical feature data, and the adaptive discriminant network based on the attention mechanism is used for intelligent filtering and optimization to finally generate the identification results.
2. The method for detecting internal road surface defects using three-dimensional ground penetrating radar according to claim 1, characterized in that: The denoising, filtering and feature enhancement processing includes: De-noising the radar data, filtering out interference signals through a high-pass filter, retaining valid signals, and generating pre-processed radar data; Filter the laser data, remove noise points through statistical filtering algorithm, enhance the point cloud quality, and generate pre-processed laser data; Perform image enhancement processing on infrared data, improve thermal map details and contrast through histogram equalization technology, and generate pre-processed infrared data; The ultrasonic data is subjected to spectrum analysis and denoising, and useful echo signals are extracted through Fourier transform to generate preprocessed ultrasonic data.
3. The method for detecting internal road surface defects using three-dimensional ground penetrating radar according to claim 1, characterized in that: The time synchronization, space registration and feature fusion include: The pre-processed multimodal data is synchronized using the spatiotemporal feature alignment algorithm to ensure the consistency of different sensor data at the same time point and generate time-synchronized data; Use feature point-based spatial registration algorithm to spatially register time-synchronized data to ensure alignment of different sensor data in spatial coordinates and generate aligned multimodal data; Convolutional neural networks are used to extract radar spatial feature data from aligned multimodal data. Point cloud processing algorithms are used to extract laser point cloud feature data from aligned multimodal data. Image processing algorithms are used to extract infrared thermal feature data from aligned multimodal data. Spectral analysis methods are used to extract ultrasonic frequency domain feature data from aligned multimodal data. These feature data are then input into deep neural networks for feature fusion to generate a comprehensive feature vector.
4. The method for detecting internal road surface defects using three-dimensional ground penetrating radar according to claim 1, characterized in that: The generative adversarial network expanded dataset includes: Generate virtual data through a generative adversarial network using a training dataset containing synthetic feature vectors; The generated virtual data is merged with the original training data set to expand the training data set and generate an expanded training comprehensive feature vector; Use a multi-task learning model to simultaneously train disease identification tasks and feature prediction tasks to optimize model parameters; In the multi-task learning process, the disease recognition task and the feature prediction task share some model parameters while retaining the dedicated parameters of each task to generate a multi-task learning model.
5. The method for detecting internal road surface defects using three-dimensional ground penetrating radar according to claim 1, characterized in that: The reinforcement learning optimization model parameters include: The multi-task learning model is optimized using the policy gradient algorithm, and the model parameters are dynamically adjusted by simulating detection tasks under various environmental conditions; In the reinforcement learning process, a reward function is constructed to give rewards based on the accuracy and robustness of the detection results to improve model performance and generate an adaptive deep learning model; Introducing the self-attention mechanism into the adaptive deep learning model enables the model to focus on key features and enhance the ability to identify disease characteristics; Use recurrent neural networks and long short-term memory networks to process time series data, improve the model's ability to capture time-varying features, and generate a time series enhanced adaptive deep learning model to generate preliminary recognition results.
6. The method for detecting internal road surface defects using three-dimensional ground penetrating radar according to claim 1, characterized in that: The energy amplitude, frequency domain spectrum characteristics and electromagnetic wave attenuation characteristics calculation include: Perform energy amplitude calculation on the disease identification and feature prediction data in the preliminary identification results to generate energy amplitude data; Perform frequency domain spectrum feature calculation on the disease identification and feature prediction data in the preliminary identification results to generate frequency domain spectrum feature data; The electromagnetic wave attenuation characteristics are calculated for the disease identification and feature prediction data in the preliminary identification results to generate electromagnetic wave attenuation characteristic data.
7. The method for detecting internal road surface defects using three-dimensional ground penetrating radar according to claim 6, characterized in that: The calculation expressions of the energy amplitude, frequency domain spectrum characteristics and electromagnetic wave attenuation characteristics are respectively: Energy amplitude calculation: in, represents the energy amplitude, Indicates the amplitude of the radar signal; Frequency domain spectrum feature calculation: in, represents the frequency domain spectrum characteristics, represents the Fourier transform of a signal; Calculation of electromagnetic wave attenuation characteristics: in, Indicates distance The signal amplitude at represents the initial signal amplitude, Represents the attenuation coefficient.
8. The method for detecting internal road surface defects using three-dimensional ground penetrating radar according to claim 1, characterized in that: The feedback and probability optimization of the dynamic statistical feature data include: Integrate energy amplitude data, frequency domain spectrum feature data, and electromagnetic wave attenuation characteristic data into dynamic statistical feature data, and feed them back into the adaptive deep learning model for secondary training and optimization to form enhanced preliminary recognition results; The hidden Markov model is used to optimize the probability of the enhanced preliminary recognition results, eliminate noise and misjudgment, and generate probability optimization results.
9. The method for detecting internal road surface defects using three-dimensional ground penetrating radar according to claim 1, characterized in that: The intelligent filtering and optimization include: The adaptive discriminant network based on the attention mechanism is used to intelligently filter and optimize the probability optimization results to generate intelligent filtering results; Further optimization is performed based on the intelligent filtering results to generate the final recognition results.
10. A system for executing the method for detecting internal road surface defects using three-dimensional ground penetrating radar according to any one of claims 1 to 9, characterized in that: include: 3D ground penetrating radar, lidar, infrared imaging sensors and ultrasonic sensors for simultaneous non-destructive testing of road surfaces and collection of multi-modal data; A data preprocessing module is used to perform denoising, filtering and feature enhancement processing on the collected multimodal data to generate preprocessed multimodal data; A data synchronization and registration module is used to perform time synchronization and spatial registration on the preprocessed multimodal data to generate aligned multimodal data; The feature extraction and fusion module is used to extract radar spatial feature data, laser point cloud feature data, infrared thermal feature data and ultrasonic frequency domain feature data from the aligned multimodal data, and use deep neural network to perform feature fusion to generate a comprehensive feature vector; A training data construction module is used to construct a training data set using a comprehensive feature vector and to expand the data set by generating an adversarial network; The multi-task learning and optimization module is used to simultaneously train disease recognition and feature prediction tasks using a multi-task learning framework, optimize model parameters using reinforcement learning, and introduce a self-attention mechanism and recurrent neural network to process time series data to generate preliminary recognition results; Dynamic statistical feature analysis and feedback module, used to calculate the energy amplitude, frequency domain spectrum characteristics and electromagnetic wave attenuation characteristics of the disease identification and feature prediction data in the preliminary identification results, generate dynamic statistical feature data, and feed it back to the adaptive deep learning model for secondary training; The probability optimization and intelligent filtering module is used to perform probability optimization on dynamic statistical feature data using a hidden Markov model, and to perform intelligent filtering and optimization using an adaptive discriminant network based on an attention mechanism to generate the final recognition result.
Citation Information
Patent Citations
Road surface internal disease identification optimization method based on radar signal statistical characteristics
CN118171144A
Road pavement disease detection method and system
CN117541555A
Road disease real-time detection method based on radar, video and data analysis
CN117876329A
Road distress detection method and apparatus, electronic device, and storage medium
WO2022227405A1
Cited By
Detection method and device for steering wheel frame detection assembly line and readable storage medium thereof
CN119510002A
Track-based hollowing collection method and detection surface hollowing collection method
CN119595755A
Engineering surveying and mapping pavement flatness detection device and method
CN119642751A
Pavement structure damage detection method and device, storage medium and electronic equipment
CN119643566A
Precast concrete T-beam bubble detection method and system
CN119845964A
