Asphalt pavement interlayer bonding state recognition method based on deep learning

Abstract

The invention provides an asphalt pavement interlayer bonding state recognition method based on deep learning. The method comprises the following steps: data acquisition and image preprocessing, image labeling and sample construction, and network model design and training. Identification and classification, wherein the model automatically outputs an interlayer combination category represented by each pixel region in the image; and system deployment: embedding the model into a geological radar vehicle-mounted terminal to realize mobile detection. The invention aims to establish a set of lightweight automatic identification model for the interlayer bonding state through an improved deep learning method. The model can be deployed on real-time equipment, the automatic identification capability is improved by capturing diseases and background features thereof at the same time, and data support is provided for subsequent quantitative evaluation of the diseases.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, in particular to a method for identifying the interlayer bonding state of asphalt pavement based on deep learning. BACKGROUND

[0002] The asphalt pavement structure is composed of multiple layers of materials, and the interlayer bonding state has an important influence on the structural performance and service life of the pavement. Traditional detection methods such as core drilling, shear test, and interface pull-out test are destructive, low in efficiency, and have large construction interference, making it difficult to achieve large-scale, real-time, and non-destructive detection of the interlayer bonding state of the pavement.

[0003] In recent years, ground penetrating radar (GPR) has been widely used in pavement internal structure detection due to its non-destructive, fast, and efficient characteristics. However, it should be noted that although GPR technology has the advantage of non-destructive detection, its detection images lack intuitiveness, and data post-processing still relies on manual interpretation, which to some extent affects the accuracy of the detection results. In addition, the processing automation of GPR images is low, so it is urgent to develop an efficient and automated processing method to improve the detection accuracy and efficiency.

[0004] With the rapid development of artificial intelligence technology, especially the continuous breakthroughs in the field of deep learning, the application of neural networks in image recognition has achieved remarkable results. Convolutional neural networks, as a core component of deep learning methods, have the ability to automatically extract data features. Based on this, target detection technology based on convolutional neural networks has been widely used in GPR image data for hidden disease identification and positioning. Among them, YOLO is a highly efficient single-stage target detection algorithm that performs well in local feature extraction, has accurate region division and target positioning capabilities, and can automatically classify and locate disease features in GPR images. However, existing target detection technology still faces many challenges in practical applications. First, in complex backgrounds, especially in the recognition of hidden diseases such as interlayer defects in GPR images, the CNN architecture is difficult to fully capture long-distance dependency features, which can easily lead to false positives and false negatives, especially when disease features are blurred or affected by interference signals, making it difficult to guarantee detection accuracy and robustness. Second, although some algorithms perform well in experimental environments, due to limitations in computing resources and hardware performance, these algorithms are often difficult to meet the real-time detection needs in practical applications, which further affects their application in dynamic environments. In addition, existing target detection frameworks mainly focus on disease identification and positioning, but still lack in the fine quantitative analysis of diseases (such as width, area, etc.), and cannot provide sufficient quantitative data support for the comprehensive evaluation and management of diseases. Therefore, there is an urgent need for a method that combines GPR data and artificial intelligence analysis technology to automatically and accurately identify the interlayer bonding state of the pavement. SUMMARY

[0005] In view of the above problems in the prior art, the present application provides a deep learning-based asphalt pavement interlayer bonding state recognition method.

[0006] To achieve the above-mentioned purposes, the technical scheme adopted by the present application is as follows: The deep learning-based asphalt pavement interlayer bonding state recognition method comprises the following steps: S1, collecting radar image data of the interlayer structure of the asphalt pavement by using a ground penetrating radar device, and pre-processing the original image; S2, combining the measured core drilling data to label the image samples, and constructing a classification data set containing "good interlayer bonding" and "poor interlayer bonding" categories; S3, constructing a deep learning recognition model that integrates convolutional neural network and Transformer module, extracting interlayer poor diffraction waveform features through local feature modeling, capturing long-distance dependence relationship of disease background and structure boundary through global feature modeling, and learning global features between interlayer poor feature waves and background layer interface lines; S4, inputting the newly collected pre-processed radar image into the trained model, outputting the pixel-level interlayer bonding state classification result, and associating GNSS positioning information to realize spatial positioning of the disease.

[0007] Further, the pre-processing in S1 includes time zero point correction, exponential gain, amplitude compensation, overall method background filtering and FIR band pass filtering, wherein: The specific calculation method of the time zero point correction is:

[0008]

[0009] In the formula, is the corrected signal, r(t) is the reference signal, s(t) is the original signal, Δt is the time offset to be corrected, R(Δt) represents s(t) the correlation degree of the original model and r(t) the reference signal; The specific calculation method of the exponential gain is:

[0010] In the formula, s gain (t) is the signal amplitude after gain, s(t) is the original signal amplitude, β is the gain coefficient, which controls the enhancement degree of the signal, eβt is an exponential gain factor; The specific calculation method of amplitude compensation is:

[0011] In the formula, A compensated (t) is the signal strength after amplitude compensation, A(t) is the original amplitude of the signal at time t , and α is the attenuation coefficient, e αt is the compensation factor in the exponential attenuation model, t is a time variable; The specific method of the overall method background filtering is:

[0012] In the formula, I(t) is the original B-scan image signal, B(t) is the estimation of the background signal, is the total variation of the minimum image; The specific method of the FIR band-pass filter is:

[0013] In the formula, y(t) is the filter output model, x(t) is the original input signal, h(n) is the impulse response coefficient of the filter, n is the delay of the filter, M is the length of the filter.

[0014] Further, the S2 specifically comprises the following steps: S21, constructing a data set named DS2; S22, positioning the disease area by using the rectangular frame method, for the enhanced image, based on the high similarity with the original image, and combining the original annotation information to complete the re-annotation; S23, reducing the annotation deviation caused by the enhancement processing through multiple rounds of artificial checking and correction, and dividing the data set according to the good and poor interlayer combination according to the annotation, and dividing into training set, verification set and test set according to the set proportion.

[0015] Further, the S3 specifically comprises the following steps: S31, local feature modeling uses CNN convolution layer to slide and extract the local information of the interlayer poor diffraction waveform feature of the input image; S32, transform the channel dimension using a 1*1 convolution without changing the feature map size as a bridge between CNN and Transformer; S33, global feature modeling constructs an Unfold-Transformer-Fold structure by unfolding the secondary folding to extract disease background information and formation mechanism, divides the interlayer adverse disease scanning image with a size of d, H, W into multiple sub-blocks with a size of w, h, and uses the Transformer multi-head attention mechanism to calculate the correlation between the red pixel position and the surrounding blue pixel position in the graph across the sub-blocks; S34, learn the global features between the interlayer adverse feature wave and the background layer interface line, and reconstruct the original interlayer adverse disease feature map size through the modeled sub-blocks.

[0016] Further, the global feature in S34 is represented as:

[0017] In the formula, is a sequence set unfolded from the sub-block to the global feature representation; is the sub-block size, is the width and height thereof, is the pixel in the sub-block.

[0018] Further, S4 specifically includes the following steps: S41, convert the classification result into a heat map or a pseudo-color map, wherein the interlayer bonding adverse area is marked as red with a probability of >90%; S42, generate a spatial coordinate binding report by associating GNSS positioning information; S43, deploy the model on a vehicle-mounted embedded GPU unit, obtain transmission images by a detection vehicle equipped with an LTD-2600 ground penetrating radar, continuously collect B-Scan images along the driving direction of the road and perform identification. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is the flowchart of the interlayer bonding state recognition method of the asphalt pavement based on deep learning of the present application. Figure 2 is the schematic diagram of the interlayer adverse data collection process of the embodiment of the present application. Figure 3 is the pre-processing flowchart and effect diagram of the embodiment of the present application. Figure 4 is the deep learning structure schematic diagram of the embodiment of the present application. DETAILED DESCRIPTION The specific embodiments of the present application are described below to facilitate the understanding of the present application for those skilled in the art, but it should be clear that the present application is not limited to the scope of the specific embodiments, and for those skilled in the art, it is obvious that all the inventions utilizing the concept of the present application are within the scope of the present application as defined and limited by the appended claims.

[0020] A deep learning-based asphalt pavement interlayer bonding state recognition method, as shown in Figure 1 , includes the following steps: S1, using a ground penetrating radar device to collect radar image data of the interlayer structure of the asphalt pavement, and pre-processing the original image; Field data collection: using a high-resolution ground penetrating radar device with a frequency of 1.5 GHz, a test line is laid along the road longitudinal direction on a certain municipal road section for continuous scanning to collect original radar image data. The sampling interval is 0.05 m, the number of single-channel waveforms is 512, and the data obtained contains the echo reflection information between the upper and lower layers of the asphalt. The field GPR interlayer bad data collection process is shown in Figure 2 .

[0021] Image preprocessing: the original radar image has certain noise interference, which affects the subsequent recognition accuracy. The preprocessing steps are as follows: 1. Time zero correction: eliminate the trigger delay of different channel A-Scan signals and unify the starting point of each channel; first, determine the time offset by correlation analysis method, the calculation formula is as follows:

[0022] In the formula, r(t) is the reference signal, s(t) is the original signal, Δt is the time offset to be corrected, R(Δt) represents s(t) the correlation degree of the original model and r(t) the reference signal.

[0023] In the correction process, by calculating the correlation under different time offset Δt , find the time offset R(Δt) that makes Δt max , so as to complete the signal correction, and the expression of the corrected signal is as follows:

[0024] 2. Amplitude compensation: to compensate for the energy attenuation with the increase of detection depth, and to enhance the visualization effect of interlayer defects, amplitude compensation technology is introduced to improve the signal-to-noise ratio of weak reflection areas in the image. The mathematical expression is as follows:

[0025] wherein, A compensated (t) is the signal strength after amplitude compensation, A(t) is the original amplitude of the signal at time t is the decay coefficient, α αt is the compensation factor in the exponential decay model, e t is the time variable.

[0026] 3. Exponential gain: To compensate for the energy attenuation with the increase of the detection depth, the reflection signal of long-distance propagation is gradually enhanced by applying an exponential function to the time or propagation distance, so that its amplitude is raised to an identifiable level, thereby achieving effective extraction of deep diseases. The mathematical expression formula is as follows:

[0027] wherein, s gain (t) is the signal amplitude after gain, s(t) is the original signal amplitude, β is the gain coefficient, which controls the degree of signal enhancement, e βt is the exponential gain factor.

[0028] 4. Overall method background filtering: To remove the low-frequency background current in the B-Scan image and improve the interface contrast, the effective removal of background noise is achieved by minimizing the total variation (TV) of the image. The total variation function is expressed as follows:

[0029] wherein, I(t) is the original B-scan image signal, B(t) is the estimation of the background signal, and the optimization problem of minimizing the total variation can be expressed as: wherein is the gradient of the background signal.

[0030] 5. FIR band-pass filtering: To filter out high-frequency noise and ultra-low frequency interference, different frequency band FIR band-pass filtering method is used to remove the road texture, background noise and other frequency components of B-Scan image at three frequencies, further suppress the clutter interference of non-important frequencies, and the FIR band-pass filtering output is shown as follows:

[0031] wherein,​y(t) is the filter output model, x(t) is the original input signal, h(n) is the impulse response coefficient of the filter, n is the delay of the filter, M is the length of the filter.

[0032] After the above processing, the B-Scan image with denoising and contrast enhancement is obtained, and the processing flow and processing effect are as shown in Figure 3 The size is unified to 256x256 pixels, and saved in PNG format, which is used for model training.

[0033] S2, combine the measured core data to label the image samples, and construct a classification data set containing “good interlayer bonding” and “poor interlayer bonding” categories; After sample labeling and data set construction, data preprocessing and enhancement, a data set named Ds2 is constructed, and the interlayer poor disease position and type labeling is completed. In the labeling process, rectangular box method is used for accurate positioning of disease area to avoid labeling error interference on model training. For SAGAN enhanced images, based on its high similarity with the original image, combined with the original labeling information, the re-labeling operation is completed. In the labeling, the automatic labeling tool is used for auxiliary labeling, and through multiple rounds of manual checking and correction, the labeling deviation caused by enhancement processing is further reduced. The total amount of samples is 1200 frames of B-Scan images: 800 frames of good interlayer bonding, 400 frames of poor interlayer bonding, and the data set is divided into training set, validation set and test set according to the ratio of 8:1:1.

[0034] S3, construct a deep learning recognition model integrating convolutional neural network and Transformer module, extract interlayer poor diffraction waveform features through local feature modeling, capture long-distance dependence relationship of disease background and structure boundary line through global feature modeling, and learn global features between interlayer poor characteristic wave and background layer interface line; Deep learning model training. The Ds2 preprocessed in step three is used as the data set in the training process. In order to speed up the model convergence speed and improve the precision, the transfer learning process is used as the starting point of model training. The coco128 pre-training weight of YOLOv5 is used to initialize the weight parameters after training 1000 times on the original ground penetrating radar scan image. Table 1 shows the specific optimization parameters.

[0035] Table 1 Training optimization parameters

[0036] The deep learning model of the application is composed of local feature modeling and global feature modeling. The local feature modeling can use a convolution layer to slide and extract the local information of the interlayer adverse diffraction waveform features, such as M-shaped and wave-shaped, from the input image. Meanwhile, 1*1 convolution can solve the mismatch between the channel number of the feature map generated by the convolution layer and the feature dimension (256, 512) required by the Transformer by transforming the channel dimension without changing the size of the feature map, thereby serving as a bridge between CNN and Transformer and improving the expression ability of the entire model. The global feature modeling constructs an Unfold-Transformer-Fold structure by unfolding the secondary folding to extract the disease background information and formation mechanism, divides the interlayer adverse disease scanning image with a size of d, H, W into a plurality of sub-blocks Patches with a size of w, h, and the Patch contains but is not limited to part of the curve of the disease diffraction wave and the road surface structure boundary line. Then, the sub-blocks are flattened into a one-dimensional sequence set X U ∈R P×N×d , P= wh , N = HW / P, and the correlation between the red pixel position and the surrounding blue pixel position in the graph is calculated across the Patch by using the multi-head attention mechanism of the Transformer, so as to fully learn the global feature representation between the interlayer adverse feature wave and the background layer interface line X G ∈ R P×N×d , and the formula is as follows. Finally, the Patch block is reconstructed to the original interlayer adverse disease feature map size d, H, and W. The deep learning structure is shown in Figure 4 .

[0037]

[0038] In order to verify the recognition accuracy, the deep learning detection model is verified for the two types of interlayer bonding states under the same test set. The verification indexes include the precision P and the recall R, and the calculation formula is as follows

[0039]

[0040] In the formula, TP and FP respectively represent the number of correct predictions of the disease samples and the number of incorrect predictions of the background and other target samples as the disease samples, and FN represents the correct disease samples incorrectly predicted as the background and other target samples. The verification results are shown in Table 2.

[0041] Table 2 Precision verification indexes

[0042] S4, input the newly collected pretreated radar image into the trained model, output the pixel-level interlayer bonding state classification result, and realize disease spatial positioning by associating GNSS positioning information.

[0043] The transmission image is obtained by the detection vehicle equipped with LTD-2600 ground penetrating radar, real-time disease identification is carried out, the ground penetrating radar is installed at the bottom of the detection vehicle, and B-Scan images are continuously collected along the driving direction of the road; the embedded GPU computing unit (such as NVIDIA Jetson Xavier) deploys a pre-trained deep learning model, which can perform real-time preprocessing and interlayer bonding state recognition on the collected radar images; the GNSS positioning module provides high-precision geographic location information to ensure the spatial accuracy of the identification result; the data communication module supports uploading the processing result to the cloud platform, which is convenient for subsequent data analysis and decision support. The integrated design of the system fully considers the particularity of the vehicle-mounted environment, has the characteristics of anti-shock, dustproof and waterproof, and ensures stable operation under complex road conditions. At the same time, the system supports remote maintenance and upgrading, which is convenient for rapid deployment and application in different areas and road types. Through the integrated application of the vehicle-mounted system, efficient, accurate and real-time monitoring of the interlayer bonding state of the asphalt pavement is realized, which provides strong technical support for road maintenance and management.

[0044] The present application is described with reference to flowcharts and / or block diagrams that illustrate the method, device (system), and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as a combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that implements the functions specified in the flowcharts and / or block diagrams.

[0045] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including instruction apparatus, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that implements the functions specified in the flowcharts and / or block diagrams.

[0046] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 Figure 1

[0047] The principles and implementations of the present application are described in the specific embodiments, the above description of the embodiments is only for helping to understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation and application range, and the above description should not be understood as a limitation of the present application.

[0048] Those skilled in the art will realize that the embodiments described herein are for the purpose of understanding the principles of the present application and should be understood as not limited to such specific embodiments and examples. Various other specific adaptations and combinations of features can be made in accordance with the teachings of the present application without departing from the spirit of the present application, and these adaptations and combinations are also within the scope of the present application.​​

Claims

1. A method for identifying the interlayer bonding state of asphalt pavement based on deep learning, characterized in that: The steps include: S1. Collect radar image data of the interlayer structure of the asphalt pavement using geological radar equipment and pre-process the original image; S2. Combine the measured drill core data with labeled image samples to construct a classification dataset containing the categories of "good interlayer bonding" and "poor interlayer bonding"; S3. Build a deep learning recognition model that integrates convolutional neural networks and Transformer modules. This model extracts interlayer poor diffraction waveform features through local feature modeling, captures the long-range dependency between the defect background and the structural boundary line through global feature modeling, and learns the global features between the interlayer poor characteristic waves and the background layer interface line. S4. Input the newly collected pre-processed radar image into the trained model, output the pixel-level inter-layer combined state classification result, and associate it with the GNSS positioning information to realize the spatial positioning of the disease.

2. The method for identifying the interlayer bonding state of asphalt pavement based on deep learning according to claim 1 is characterized in that: The preprocessing in S1 includes time zero point correction, exponential gain, amplitude compensation, overall background filtering and FIR bandpass filtering, wherein: The specific calculation method of time zero point correction is: Where, is the corrected signal, r(t) is the reference signal, s(t) is the original signal, Δt is the time offset to be corrected, R(Δt) express s(t) Original model and r(t) the degree of correlation of the reference signal; The specific calculation method of exponential gain is: Where, s gain (t) is the signal amplitude after gain, s(t) is the original signal amplitude, β is the gain coefficient, which controls the degree of signal enhancement. e βt is the exponential gain factor; The specific calculation method of amplitude compensation is: Where, A compensated (t) is the signal strength after amplitude compensation, A(t) For the signal at time t The original amplitude of the moment, α is the attenuation coefficient, e αt is the compensation factor in the exponential decay model, t is the time variable; The specific method of overall background filtering is: Where, I(t) is the original B-scan image signal, B(t) is the estimate of the background signal, is the total variation of the minimum image; The specific method of FIR bandpass filtering is: Where, y(t) is the filter output model, x(t) is the original input signal, h(n) are the impulse response coefficients of the filter, n is the delay of the filter, M is the length of the filter.

3. The method for identifying the interlayer bonding state of asphalt pavement based on deep learning according to claim 1 is characterized in that: The S2 specifically includes the following steps: S21, build a data set named DS2; S22. Locate the diseased area using a rectangular frame method. For the enhanced image, re-label it based on its high similarity with the original image and in combination with the original labeling information. S23. Through multiple rounds of manual verification and correction, the annotation deviation caused by the enhancement process is reduced, and the data sets with good inter-layer integration and poor inter-layer integration are divided according to the annotations, and divided into training sets, validation sets and test sets according to the set ratio.

4. The method for identifying the interlayer bonding state of asphalt pavement based on deep learning according to claim 1 is characterized in that: The S3 specifically includes the following steps: S31, local feature modeling uses the CNN convolutional layer to slide the input image to extract local information of the inter-layer bad diffraction waveform characteristics; S32, using 1*1 convolution to transform the channel dimension without changing the size of the feature map as a bridge between CNN and Transformer; S33, global feature modeling constructs the Unfold-Transformer-Fold structure by unfolding the sub-folding method to extract the disease background information and formation mechanism, and converts the size of d, H, W The inter-layer defect scan image is divided into multiple sub-blocks of size w and h, and the Transformer multi-head attention mechanism is used to calculate the correlation between the red pixel position in the image and the surrounding blue pixel positions across the sub-blocks; S34. Learn the global features between the interlayer defect characteristic wave and the background layer interface line, and reconstruct the original interlayer defect feature map size through the modeled sub-blocks.

5. The method for identifying the interlayer bonding state of asphalt pavement based on deep learning according to claim 4 is characterized in that: The global feature in S34 is expressed as: Where, The sub-block is expanded into a sequence set of Global feature representation; is the sub-block size, For its width and height, is the pixel in the sub-block.

6. The method for identifying the interlayer bonding state of asphalt pavement based on deep learning according to claim 4 is characterized in that: The S4 specifically includes the following steps: S41. Convert the classification results into a heat map or a pseudo-color map, wherein the interlayer poor bonding area is marked in red with a probability greater than 90%; S42, associating GNSS positioning information to generate a spatial coordinate binding report; S43. The model is deployed on a vehicle-mounted embedded GPU unit. The transmission image is obtained by a detection vehicle equipped with an LTD-2600 ground penetrating radar. B-Scan images are continuously collected along the road driving direction and recognized.

Images