Road near-surface hidden defect detection method and system

Abstract

The invention relates to the technical field of road near-surface hidden defect detection, and particularly discloses a road near-surface hidden defect detection method and system.The method comprises the steps that concrete test blocks containing different defects are obtained, and experimental surface temperature distribution data of the test blocks under the surface thermal wave loading condition are recorded; simulating simulated surface temperature distribution data of the concrete test blocks with different defects under a thermal wave loading condition; enhanced simulation surface temperature distribution data is obtained through a conditional variation auto-encoder; performing defect marking on the experimental surface temperature distribution data and the enhanced simulation surface temperature distribution data, and training a space-time dual-coding model; and obtaining an infrared thermal imaging image sequence of a to-be-predicted road, and inputting the infrared thermal imaging image sequence into the trained space-time dual-coding model to obtain a defect detection result of the to-be-predicted road. According to the method, the defect area is guided through the time features, then the defect form is refined through the spatial features, and the accuracy of road near-surface invisible defect detection is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of road near-surface hidden defect detection, and particularly to a method and system for detecting road near-surface hidden defects. Background Art

[0002] The statements in this part only provide background technical information related to the present invention and do not necessarily constitute prior art.

[0003] Currently, the urban road network is becoming increasingly perfect, greatly facilitating travel. However, with the increase in the service life of roads, maintenance and repair have become more and more important, especially the threats of hidden defects such as cavities, voids, and base damage are becoming increasingly prominent. Therefore, adopting advanced detection technologies and scientific maintenance means to timely discover and repair these hidden dangers is crucial for ensuring road safety and extending service life.

[0004] Road near-surface hidden defect detection mainly refers to the detection of defects such as cavities, voids, cracks, or water accumulation inside the road. These defects are all internal road defects and cannot be identified by observation or machine vision methods.

[0005] In the traditional field of road defect detection, research mainly focuses on the identification of pavement surface cracks, with less research on road hidden defects and relatively scarce identification technologies. Manual inspections such as the core drilling method will damage the road surface structure and have low efficiency, and are often used to verify the existence of defects. The signals of emerging ground-penetrating radars are relatively complex and are still in the development process.

[0006] With the development of non-destructive testing technologies, infrared thermal imaging technology has been widely used in asphalt water permeability detection, road and material surface damage detection due to its advantages such as strong intuitiveness and non-contact. However, most of the existing technologies use infrared thermal imaging technology to detect early tiny defects on the road surface, and there is still little research on the detection of road internal hidden defects.

[0007] In the existing technology, most infrared thermal imaging analysis methods often only focus on images at a single moment and ignore time series information. Applying them to road hidden defect detection often brings the following technical problems: (1) It is impossible to effectively distinguish the defect area from the normal area: The temperature anomaly in the defect area is often a dynamic process and will show obvious characteristics only over time. For example, the air layer under the cavity will cause heat conduction lag, forming a temperature anomaly, but this anomaly may not be obvious or stable at different time points. At a certain moment, the temperatures of the defect area and the non-defect area may be similar, resulting in incorrect detection of defects. Images at a single moment cannot depict this dynamic change, easily misjudging the normal area as a defect or missing real defects.

[0008] (2) Greatly affected by environmental interference and reduced robustness: External environmental factors (such as sunlight, shadow, wind speed, humidity, etc.) will affect infrared thermal images. For example, the influence of shadow: If a certain frame is affected by shadow, resulting in abnormal temperature distribution, it may be misidentified as a defect. Another example is weather change: If only the images at a certain moment are used, it is impossible to determine whether the temperature anomaly is caused by a defect or a weather factor.

[0009] (3) Higher false detection rate and missed detection rate: Since a single-frame image can only provide instantaneous temperature distribution and cannot verify the cause of temperature anomaly, it is easy to misjudge non-defect areas as defects. The temperature anomalies in some defect areas are not obvious at certain moments, but will only become apparent after evolving over time.

[0010] Although the prior art discloses the use of deep learning technology for road surface defect recognition, however, the input data of the neural network model is often also a single-frame image. The model can only rely on spatial information and cannot learn key temporal dynamic features, resulting in poor detection effects.

[0011] In addition, the use of deep learning technology requires a large amount of high-quality training data. However, the experimental data of hidden defects is limited by factors such as acquisition cost and measurement conditions, making it difficult to manufacture samples, with a small number of samples, and it is difficult to meet the requirements of the deep learning data volume. Although simulation can be used to obtain the required data, however, due to the lack of noise characteristics in the experimental environment in the simulation data, directly using the simulation data for training may lead to a decline in the adaptability of the model in actual applications. Summary of the Invention

[0012] In order to solve the above problems, the present invention proposes a method and system for detecting hidden defects near the surface of a road. The conditional variational autoencoder (CVAE) technology is used to process the experimental noise of the simulation data to enhance the authenticity of the data. Based on the spatio-temporal dual-encoding strategy of Transformer and CNN, the temporal and spatial features of the temperature data are fully mined to improve the accuracy and generalization ability of detecting hidden defects near the surface of the road and materials.

[0013] In some embodiments, the following technical solutions are adopted: A method for detecting hidden defects near the surface of a road, comprising: Obtain concrete test blocks with different defects and record the experimental surface temperature distribution data of these test blocks under the condition of surface thermal wave loading; establish a simulation model to simulate the simulated surface temperature distribution data of concrete test blocks with different defects under the condition of thermal wave loading; Add experimental noise to the simulated surface temperature distribution data through a conditional variational autoencoder (CVAE) to obtain enhanced simulated surface temperature distribution data; Defect labels are added to the experimental surface temperature distribution data and the enhanced simulation surface temperature distribution data, which are used as the training dataset to train the spatio-temporal dual-encoding model; An infrared thermal imaging image sequence of the road to be predicted is obtained. The obtained image sequence is input into the trained spatio-temporal dual-encoding model. The defect area is guided by the time-domain encoding, and then the defect morphology is refined by the space-domain encoding. Finally, the defect detection result of the road to be predicted is obtained.

[0014] As an optional solution, the spatio-temporal dual-encoding model preliminarily extracts features from the input surface temperature distribution data through a single-channel convolutional layer and an average pooling layer. The extracted features are input into a series of Transformer encoding modules to complete the time-domain encoding; the feature information after the time-domain encoding is input into an encoder-decoder structure based on CNN after tensor reconstruction. The upper and lower layer features are connected by copying, and after re-extracting and restoring the input features, the defect prediction result is output.

[0015] As an optional solution, the simulation surface temperature distribution data and the experimental surface temperature distribution data of concrete specimens with the same defects are obtained to construct the training dataset of the conditional variational autoencoder; The simulation surface temperature distribution data and the experimental surface temperature distribution data are provided to the conditional variational autoencoder at the same time. Among them, the simulation surface temperature distribution data is used as the conditional input to the encoder of the conditional variational autoencoder, and the experimental surface temperature distribution data is used as the target data to calculate the reconstruction loss, so that the model can learn the noise characteristics of the experimental data. After training, only the decoder part of the conditional variational autoencoder (CVAE) is used for data conversion. Specifically, first, a latent vector z is sampled from the standard normal prior distribution, and at the same time, the simulation surface temperature distribution data is used as the condition to input into the conditional branch of the decoder; subsequently, the Latent branch (i.e., the latent vector branch) of the decoder performs a fully connected expansion and upsampling on the latent vector, and fuses it with the simulation image features extracted by the conditional branch in the channel dimension, and finally generates the enhanced simulation surface temperature distribution data with the noise characteristics of the experiment.

[0016] As an optional solution, the conditional variational autoencoder includes an encoder and a decoder, and a fully connected layer is combined in the middle to learn the distribution of the latent variables; the encoder gradually extracts features and compresses them into a compact representation, maps them to the latent variable space through a fully connected layer, and forms a latent vector representation in combination with the conditional input, and then is gradually restored to the original input size through the decoder.

[0017] The loss function of the conditional variational autoencoder is specifically: ; ; ; Among them, is the loss function of the conditional variational autoencoder, is the reconstruction loss, is the KL divergence loss; and are the weights of the reconstruction loss and the KL divergence loss respectively; , are the original input image and the reconstructed image respectively; N represents the total number of pixels in the image; and are the mean of and the standard deviation of output by the encoder respectively, where d is the dimension of the latent variable.

[0018] As an optional solution, the time-domain encoding process is specifically as follows: The feature map passing through the single-channel convolutional layer is sliced along the channel dimension into multiple inter-sampling feature blocks, the size of each feature block is fixed, each feature block contains the temperature feature at a certain fixed moment, and is rearranged into a one-dimensional vector; The rearranged feature blocks are projected through the inter-sampling fully connected layer, time series encoding is performed on each inter-sampling feature block, and the encoded inter-sampling feature blocks are input into the Transformer encoder, and the output dimension of the Transformer encoder is reconstructed through the tensor reconstruction layer.

[0019] As an optional solution, the spatial-domain encoding process is specifically as follows: The reconstructed tensor after time-domain encoding is used as the input of the spatial-domain encoder. The spatial-domain encoder includes multiple convolutional layers. A copy connection method is adopted between some convolutional layers to realize the splicing of shallow features and deep features. Two convolutional operations are performed within each convolutional layer, and a ReLU activation function is adopted for non-linear mapping before the convolutional operation. The mapping from the feature map to the recognition result is realized through the output layer; The road surface temperature response is learned through spatial-domain encoding to realize the detection of hidden defects near the road surface.

[0020] In some other embodiments, the following technical solutions are adopted: A system for detecting hidden defects near the road surface includes: A data acquisition module, configured to acquire concrete test blocks with different defects, record the experimental surface temperature distribution data of these test blocks under the condition of surface thermal wave loading; establish a simulation model to simulate the simulated surface temperature distribution data of concrete test blocks with different defects under the condition of thermal wave loading; A simulation enhancement module, which is used to add experimental noise to the simulation surface temperature distribution data through a conditional variational autoencoder to obtain enhanced simulation surface temperature distribution data; A model training module, which is used to perform defect marking on the experimental surface temperature distribution data and the enhanced simulation surface temperature distribution data, and use them as a training data set to train a spatio-temporal dual-encoding model; A defect detection module, which is used to obtain an infrared thermal imaging image sequence of a road to be predicted, input the obtained image sequence into the trained spatio-temporal dual-encoding model, guide the defect area through time-domain encoding, and then refine the defect morphology through spatial-domain encoding, and finally obtain the defect detection result of the road to be predicted.

[0021] In some other embodiments, the following technical solutions are adopted: A terminal device, which includes a processor and a memory. The processor is used to implement instructions; the memory is used to store multiple instructions, and the instructions are suitable for being loaded and executed by the processor to perform the above-mentioned road near-surface hidden defect detection method.

[0022] In some other embodiments, the following technical solutions are adopted: A computer-readable storage medium, in which multiple instructions are stored, and the instructions are suitable for being loaded and executed by the processor of the terminal device to perform the above-mentioned road near-surface hidden defect detection method.

[0023] Compared with the prior art, the beneficial effects of the present invention are: (1) The present invention processes the simulation temperature samples through a conditional variational autoencoder, extracts the experimental noise features for subsequent noise addition processing of the simulation temperature samples, so as to generate enhanced simulation temperature samples containing experimental noise, thereby better simulating real test conditions, solving the problem of insufficient experimental sample data, and improving the generalization ability of the model.

[0024] (2) Since in the temperature response of the thermal wave excitation, there are usually distinguishable differences in the spatial temperature field distribution and the time-domain thermal signal characteristics between the defect area and the non-defect area on the near surface of the road. This difference is mainly manifested as the thermal relaxation time shift caused by the change in the equivalent heat capacity of the defect area, and the abnormal surface temperature gradient caused by the change in the heat conduction path. Therefore, the present invention first performs time-domain encoding on the infrared thermal imaging image sequence, which can better utilize the temperature change trend to distinguish the defect area and the normal area, enhance the robustness, reduce the interference of environmental factors, and reduce the errors caused by factors such as light and materials; then extract the spatial-domain features of the image sequence after time-domain encoding, which can better focus on the real defect area rather than the instantaneous temperature change.

[0025] The present invention first guides the defect area through time features and then refines the defect morphology through spatial features, significantly improving the accuracy of detecting near-surface invisible defects on roads.

[0026] (3) The defect detection method of the present invention can maintain a high recognition accuracy under various conditions, can accurately obtain the position and morphology of defects, and is suitable for various actual road detection scenarios; accurate damage detection helps to take repair measures in a timely manner, reduce traffic accidents and maintenance costs, and improve the safety and durability of road use at the same time.

[0027] (4) The defect detection method of the present invention can also be applied to other near-surface damage detections, such as paint debonding detection, honeycomb composite panel debonding detection, and internal cavity detection of alloy materials.

[0028] Other features and advantages of the present invention will be partially given in the following description, partially become obvious from the following description, or be understood through the practice of this aspect. Description of the Drawings

[0029] Figure 1 It is a flowchart of the method for detecting near-surface hidden defects on roads in an embodiment of the present invention; Figure 2 It is a schematic structural diagram of a conditional variational autoencoder in an embodiment of the present invention; Figure 3 It is a flowchart of the preprocessing and data partitioning of the experimental data set in an embodiment of the present invention; Figure 4 It is a flowchart of the preprocessing and data partitioning of the simulation data set in an embodiment of the present invention; Figure 5 It is a flowchart of constructing training data for the spatio-temporal double encoder in an embodiment of the present invention; Figure 6 It is a schematic diagram of the data processing process of the spatio-temporal double encoder in an embodiment of the present invention; Figure 7 It is a schematic structural diagram of the model of the time-domain encoder in an embodiment of the present invention; Figure 8 It is a visualization recognition result diagram of different types of defects in the test set. Detailed Embodiments

[0030] It should be noted that the following detailed description is illustrative and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present application belongs.

[0031] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular forms are also intended to include the plural forms. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they specify the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0032] Embodiment 1 In one or more embodiments, a method for detecting hidden defects on the near surface of a road is disclosed. In combination with Figure 1 , the specific process is as follows: S101: Obtain concrete specimens with different defects, and record the experimental surface temperature distribution data of these specimens under the condition of surface thermal wave loading; establish a simulation model to simulate the simulated surface temperature distribution data of concrete specimens with different defects under the condition of thermal wave loading.

[0033] In this embodiment, in combination with Figure 3 , through the method of experimental measurement, obtain the experimental surface temperature distribution data of concrete specimens with different defect types under the condition of surface thermal wave loading and its evolution over time; first, trim and compress the experimental surface temperature distribution data to obtain node experimental temperature data with a dimension of 80×80×4000. Secondly, divide the node temperature data collected within 4000s along the time dimension. The experimental surface thermal wave loading period is 100s, the length of the division window is 100s, and a single experimental model provides 40 experimental temperature samples, and the dimension of each experimental temperature sample is 80×80×100. Visualize the binarized labels after annotation, and the white area is the defect area with obvious edge features.

[0034] In combination with Figure 4 , record the simulated surface node temperature distribution data of the defective concrete specimens under the condition of thermal wave loading and its evolution over time under the simulation method. For a single simulation model, node simulation temperature data with a dimension of 80×80×4000 can be obtained. Secondly, divide the node simulation temperature data collected within 4000s along the time dimension. The modulation frequency of the thermal wave loading is 0.01Hz, and the length of the division window is selected as 100s to obtain 40 simulation temperature samples, and the dimension of each temperature data sample is 80×80×100.

[0035] S102: Add experimental noise to the simulated surface temperature distribution data through a conditional variational autoencoder to obtain enhanced simulated surface temperature distribution data.

[0036] In this embodiment, the structure of the conditional variational autoencoder (CVAE) is as Figure 2As shown, it includes an encoder and a decoder, and a fully connected layer is incorporated in the middle to learn the probability distribution of the latent vector.

[0037] Obtain the simulated surface temperature distribution data and experimental surface temperature distribution data of concrete specimens with the same defects, and construct a training dataset for the conditional variational autoencoder.

[0038] In this embodiment, the simulated surface temperature distribution data and experimental surface temperature distribution data are simultaneously provided to the conditional variational autoencoder (CVAE). In the encoding stage, the experimental surface temperature distribution data (target) and the simulated surface temperature distribution data (condition) are concatenated in the channel dimension and then input into the encoder. The encoder extracts features through multi-layer convolution operations and outputs the mean vector and logarithmic variance vector of the latent variable through flattening and fully connected layers respectively; the reparameterization is used to generate a specific latent vector z. During the training process, the experimental surface temperature distribution data is used as the target data to calculate the reconstruction loss, and combined with the KL divergence loss that constrains the latent variable distribution to be a standard normal distribution, so that the model learns the noise characteristics of the experimental data. After training is completed, in the inference stage, only the simulated surface temperature distribution data needs to be input as the condition into the decoder of the CVAE, and at the same time, a latent vector z is sampled from the prior distribution. After being processed by the decoder, enhanced data with experimental noise characteristics can be generated.

[0039] The processing process of the conditional variational autoencoder for data is specifically as follows: The input of the conditional variational autoencoder is an 80×80×2 multi-channel image obtained by concatenating the experimental image and the simulated image. It gradually extracts features through a convolutional encoder and compresses them into a compact representation (such as 10×10×4), and then maps them to the latent variable space through a fully connected layer; subsequently, the reparameterization technique is used to generate the latent variable z; the decoder gradually restores to the original input size (80×80×1) through a series of symmetric deconvolution operations, and at the same time uses the simulated image as the conditional input during the decoding stage to jointly generate the target experimental image. By training the model with the experimental sample temperature as the target data and using the weighted combination of the reconstruction loss and the KL divergence as the loss function, it can reflect the error between the reconstructed temperature data and the experimental temperature data, and can constrain the latent variable distribution to be a standard normal distribution; thus, when the model extracts the residual part between encoding and reconstruction, it can fully capture the experimental noise characteristics and apply them to the subsequent noise addition processing of the simulated temperature samples to generate enhanced simulated temperature samples containing experimental noise, better simulating the real test conditions and improving the generalization ability of the model. Table 1 shows the CVAE model structure parameters.

[0040] Table 1 CVAE model structure parameters

[0041] In this embodiment, the conditional variational autoencoder (CVAE) adopted receives, during the encoding stage, an input obtained by concatenating the experimental temperature distribution map and the simulated temperature distribution map along the channel dimension, so as to simultaneously capture the noise characteristics of the experimental data and the structural information of the simulated data in the latent variables. After generating a specific latent vector by using the reparameterization trick, during the generation stage, the decoder inputs the latent vector and the simulated temperature distribution data (as conditional information) together to generate enhanced simulated temperature samples with the noise characteristics of the experiment. After such training, the model can not only reconstruct an image close to the experimental data, but also make the latent variable distribution close to the standard normal distribution, thereby extracting the experimental noise features and using them for subsequent noise addition processing of the simulated temperature samples to better simulate the real test conditions and improve the generalization ability of the model.

[0042] In this embodiment, the reparameterization trick is used to decouple the random sampling distribution of the latent variables into a deterministic part and a noise part, ensure the gradient can be passed, and achieve end-to-end training. Its formula is:

[0043] where z represents the latent vector, μ represents the mean, σ is the standard deviation, ε represents the standard normal distribution noise, conforming to N (0, 1).

[0044] In this embodiment, by training the model with the experimental sample temperature as the target data, a weighted combination of the reconstruction loss and the KL divergence is used as the loss function. Among them, the reconstruction loss is used to measure the error between the reconstructed image and the experimental temperature sample (target data). Therefore, the mean squared error is used as the reconstruction loss function. Let the original input image be , and the reconstructed image obtained after passing through the encoder-decoder process is , then the reconstruction loss can be defined as: ; where N represents the total number of pixels in the image.

[0045] Let the mean output by the encoder be and the standard deviation be , then the calculation formula of the KL divergence loss is: ; where d is the dimension of the latent variable; the calculation method of the KL divergence loss in this embodiment can approximate the distribution of the latent variable z to the standard normal distribution N (0, 1).

[0046] The above two parts are combined with weights. After repeated tests, it is found that when the weights are 0.5 and 1.0, the model has the best processing effect, which is the total loss function of CVAE. : .

[0047] Through training, using this loss function with the experimental temperature samples as the target data, the model can not only reconstruct images close to the experimental data but also make the latent variable distribution converge to the standard normal distribution, thereby extracting the experimental noise characteristics. After training, in the inference stage, only the simulated temperature distribution data needs to be input as a condition to the decoder of CVAE, and at the same time, a latent vector z is sampled from the prior distribution, and the decoder can generate enhanced simulated temperature samples containing experimental noise characteristics.

[0048] For the segmented simulated surface node temperature distribution data, the CVAE experimental noise characteristics are added to each data unit to generate enhanced simulated temperature samples containing experimental noise. The completed binary labels are displayed in a visual way, and the white area is the defect area with obvious edge features.

[0049] S103: Mark the defects (i.e., binary labels) for the experimental surface temperature distribution data and the enhanced simulated surface temperature distribution data, and use them as the training data set to train the spatio-temporal double-encoding model.

[0050] Combined with Figure 5 , the enhanced simulated temperature samples and the experimental temperature samples are used as the data set samples for deep learning. These samples can be flexibly divided according to test requirements and used for the training, validation, and testing of neural networks to ensure that the model has good generalization ability and accuracy in various scenarios.

[0051] In this embodiment, the spatio-temporal double-encoding model is a deep learning model including time-domain encoding and space-domain encoding. Among them, the time-domain encoding uses a Transformer encoding module, and the space-domain encoding uses a CNN encoding module to achieve the learning of different-dimensional features in time and space.

[0052] Combined with Figure 6 , the processing process of the spatio-temporal double-encoding model for data is specifically as follows: For the collected temperature thermal sequence samples, input them into the single-channel convolutional layer module and the average pooling layer for preliminary feature extraction to compress the dimension of the samples. The feature extraction method uses a single-channel convolutional layer. Input the sampled feature map obtained after the initial feature extraction into consecutive Transformer encoding modules, and the time-domain encoding in the defect recognition process can be completed through the Transformer encoding modules. Subsequently, the feature information after time-domain encoding is reconstructed into a feature matrix with the same dimension as the input sampled feature map through tensor reconstruction and used as the input of the spatial-domain encoding module. In the spatial-domain encoding module, an encoder-decoder structure based on CNN is constructed, and the upper and lower layer features are connected through copy connection. Through the re-extraction and restoration of the input features, the spatial-domain encoding finally realizes the output of predicting defects.

[0053] Hidden defects such as cavities or delaminations in the road will cause changes in the heat conduction characteristics of local areas; when the road surface is thermally excited (such as solar radiation or artificial heat source), the heat conduction in the normal area is faster and the temperature change is relatively stable; while in the defect area, due to the existence of hidden defects such as cavities or delaminations, the heat conduction is blocked, resulting in an increase in the thermal inertia of this area, manifested as a lag in temperature change or temperature anomaly. It can be seen that the main information of hidden defects is first reflected in the change trend of temperature over time, rather than the spatial distribution of a single frame, that is, the characteristic difference of temperature in the time dimension is more obvious.

[0054] Based on this, in this embodiment, a spatio-temporal dual-encoding model is constructed. First, time encoding is performed on the input temperature sample sequence to extract the dynamic features of the time series, such as: the temperature change curve over time, the occurrence time of the temperature peak, the temperature change rate (such as the slope in the heating and cooling stages), etc. These features play a very important role in the identification of hidden defects. Through time encoding, it is easier to extract this abnormal temperature change pattern and improve the distinguishability of defects.

[0055] In addition, if spatial feature extraction is directly performed, it may be affected by factors such as road surface texture, pollution, and shadows, resulting in an increase in noise. In this embodiment, the information at different time points is first integrated through time encoding to form stable time features, thereby reducing the interference of environmental factors.

[0056] After time encoding, the data has been converted into a representation rich in time information; at this time, through spatial encoding, the spatial position of the defect, the shape of the defect (size, depth), and the connectivity of the defect are further identified. This can avoid the problem of false detection easily caused by being affected by environmental factors when directly using a single-frame image for defect detection, and at the same time improve the expression ability of spatial features.

[0057] After time encoding in this embodiment, the model can identify areas with abnormal temperature changes, and then use spatial feature extraction to determine the morphology and location of these abnormal areas; it can better focus on real defect areas rather than instantaneous temperature changes; it helps to more accurately locate defects, rather than just detecting "something is abnormal somewhere".

[0058] As a specific implementation, time-domain encoding is a time-series analysis method based on the Transformer architecture. As Figure 7 shown, first, the feature map passing through the single-channel convolutional layer is sliced along the channel dimension into multiple inter-sampling feature blocks. The size of each feature block is fixed and rearranged into a one-dimensional vector. According to the dimension of the previously input training samples, each sampling feature map can be sliced into 100 inter-sampling feature blocks, and each feature block contains the temperature feature at a certain fixed moment. Subsequently, the rearranged feature blocks are projected through the inter-sampling fully connected layer to perform time-series encoding on each inter-sampling feature block, and the encoded inter-sampling feature blocks are input into the Transformer encoder. The Transformer encoder consists of two normalization layers, a multi-head self-attention calculation layer, and a feed-forward neural network. The formula for the multi-head self-attention mechanism in the Transformer encoder is: ; where Q, K and V represent Query, Key, and Value respectively, dk is the dimension of the key vector. The multi-head attention calculation mechanism allows the model to learn information from different representation subspaces simultaneously, while the feed-forward neural network further processes the features at each moment before output. At the end of time-domain encoding, the output dimension of the Transformer encoder is reconstructed through the tensor reconstruction layer to enable subsequent spatial-domain encoding.

[0059] Spatial-domain encoding takes over the reconstructed tensor after time-domain encoding. Table 2 shows the model structure parameters of the spatial-domain encoder.

[0060] Table 2 Model Structure Parameters of the Spatial-Domain Encoder

[0061] The spatial domain encoder consists of five convolutional layers, two max pooling layers, two upsampling layers, and a final output layer. Among them, some convolutional layers are connected by copy connection to realize the splicing of shallow features and deep features. Each convolutional layer performs two convolutional operations, and the kernel size, stride, and padding are 3, 1, and 1 respectively. And a ReLU activation function is used for non-linear mapping before the convolutional operation. Finally, the mapping from the feature map to the recognition result is realized through the output layer composed of 1×1 convolutions. By learning the surface temperature response through spatial domain encoding, the internal defects of concrete specimens can ultimately be identified under the excitation of sinusoidal thermal signals. The SGD algorithm is used to optimize the hyperparameters. The initial learning rate is set to 0.001, the batch size is set to 10, and a total of 100 epochs of training are carried out. In terms of the selection of the loss function, a linear combination of the BCE loss function and the Dice loss function is adopted, and the weights of the former and the latter are 0.5 and 1 respectively.

[0062] The learning rate adjustment formula of SGD can be expressed as: ; where η new is the new learning rate, η old is the old learning rate, decay_rate is the decay rate, epoch is the current iteration number, total_epochs is the total iteration number.

[0063] The calculation formula of the BCE loss function is: ; where, y true and y pred represent the true label and the predicted output after sigmoid transformation.

[0064] The Dice loss function focuses on reducing the set difference between the prediction and the true label, and its calculation formula is: ; where, X and Y represent the true label and the prediction result respectively.

[0065] After repeated tests on the linear combination of the BCE loss function and the Dice loss function, it is found that when the weights are 0.5 and 1.0, the recognition effect of the model is the best, and the formula can be written as: ; In this embodiment, the spatio-temporal dual-coding model aims to accurately detect the edges of defects in the sample and finally output in the form of a binary map.

[0066] Common accuracy metrics in the field of image segmentation are used for evaluation, such as intersection over union (IoU), recall, precision, accuracy, and F1 score.

[0067] Table 3 records the detection accuracy of various types of defects in the test set under different loading powers.

[0068] Table 3 Statistical accuracy of identifying different types of defects in the test set

[0069] Among them, center, upper, right, and upper right represent that the position of the center point of the defect is at different positions, and the numbers 123 represent the size of the defect. Figure 8 Displays the labels of each defect sample in the above test set and the visualized results after recognition. Table 4 records the accuracy metrics of the deep learning model when this non-preset loading power is used for testing.

[0070] Table 4 Statistical accuracy of defect detection by the deep learning model under non-preset loading power

[0071] In this embodiment, the accuracy metrics achieved by the deep learning model when facing defects of different sizes or positions are tested. The results show that for specific non-preset defects, defect localization and recognition can be achieved under different loading powers, and as the loading power increases, each recognition accuracy metric gradually increases and tends to be stable. Further, this study tests the accuracy values that various types of defects can reach when the loading power deviates. It is found that when the loading power of the samples in the test set is high, the obtained IoU and F1 score metrics can both exceed 95%, showing excellent recognition performance, and the model can still generally perform effective measurements. Whether it is for non-preset size defects that have not been trained or when there is a deviation between the test power and the training power, the proposed deep learning model can accurately identify the corresponding defect areas. These experimental results fully prove that the proposed model has excellent generalization performance.

[0072] S104: Obtain the infrared thermal imaging image sequence of the road to be predicted, input the obtained image sequence into the trained spatio-temporal dual-coding model, guide the defect area through time-domain coding, and then refine the defect morphology through space-domain coding to obtain the final defect detection result of the road to be predicted, including the type of defect (such as cavity, crack, or water accumulation, etc.), the center position, and the boundary.

[0073] The output result is shown in the form of a binary image, and it can be clearly seen that there are defective areas and non-defective areas, and the boundaries are relatively obvious.

[0074] Embodiment 2 In one or more embodiments, a road near-surface hidden defect detection system is disclosed, which specifically includes: A data acquisition module, which is used to acquire concrete test blocks with different defects, record the experimental surface temperature distribution data of these test blocks under the condition of surface thermal wave loading; establish a simulation model to simulate the simulated surface temperature distribution data of concrete test blocks with different defects under the condition of thermal wave loading; A simulation enhancement module, which is used to add experimental noise to the simulated surface temperature distribution data through a conditional variational autoencoder to obtain enhanced simulated surface temperature distribution data; A model training module, which is used to mark the experimental surface temperature distribution data and the enhanced simulated surface temperature distribution data as defective, and use them as a training data set to train a spatio-temporal dual-encoding model; A defect detection module, which is used to acquire an infrared thermal imaging image sequence of the road to be predicted, input the acquired image sequence into the trained spatio-temporal dual-encoding model, guide the defective area through time-domain encoding, and then refine the defective shape through space-domain encoding, and finally obtain the defect detection result of the road to be predicted.

[0075] It should be noted that the specific implementation manners of the above modules are the same as those in Embodiment 1 and will not be elaborated here.

[0076] Embodiment 3 In one or more embodiments, a terminal device is disclosed, which includes a processor and a memory. The processor is used to implement instructions; the memory is used to store multiple instructions, and the instructions are suitable for being loaded and executed by the processor to perform the road near-surface hidden defect detection method described in Embodiment 1.

[0077] It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0078] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A part of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0079] In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor or the instructions in the form of software.

[0080] Embodiment 4 In one or more embodiments, a computer-readable storage medium is disclosed, in which multiple instructions are stored, and the instructions are adapted to be loaded and executed by the processor of the terminal device to perform the road near-surface hidden defect detection method described in Embodiment 1.

[0081] Although the specific embodiments of the present invention are described above in conjunction with the accompanying drawings, it is not a limitation to the protection scope of the present invention. Those skilled in the art should understand that, based on the technical solutions of the present invention, various modifications or deformations that can be made by those skilled in the art without creative efforts are still within the protection scope of the present invention.

Claims

1. A method for detecting hidden defects near the road surface, characterized in that: include: Acquire concrete test blocks with different defects and record the experimental surface temperature distribution data of these test blocks under surface thermal wave loading conditions; establish a simulation model to simulate the simulated surface temperature distribution data of concrete test blocks with different defects under thermal wave loading conditions; By adding experimental noise to the simulated surface temperature distribution data through a conditional variational autoencoder, enhanced simulated surface temperature distribution data is obtained; The experimental surface temperature distribution data and the enhanced simulation surface temperature distribution data are marked with defects and used as training data sets to train the spatiotemporal dual encoding model; Obtain the infrared thermal imaging image sequence of the road to be predicted, input the acquired image sequence into the trained spatiotemporal dual coding model, guide the defect area through time domain coding, and then refine the defect morphology through space domain coding, and finally obtain the defect detection result of the road to be predicted.

2. A method for detecting hidden defects near the road surface as claimed in claim 1, characterized in that: The spatiotemporal dual encoding model performs preliminary feature extraction on the input surface temperature distribution data through a single-channel convolution layer and an average pooling layer, and the extracted features are input into a continuous Transformer encoding module to complete time domain encoding; the feature information after time domain encoding is reconstructed by tensor and input into the CNN-based encoder-decoder structure, which connects the upper and lower layer features through copying connections, and outputs the defect prediction results after re-extracting and restoring the input features.

3. A method for detecting hidden defects near the road surface as claimed in claim 1, characterized in that: The simulated surface temperature distribution data and the experimental surface temperature distribution data of concrete test blocks with the same defects are obtained to construct a training data set for the conditional variational autoencoder; The simulated surface temperature distribution data and the experimental surface temperature distribution data are simultaneously provided to the conditional variational autoencoder, wherein the simulated surface temperature distribution data is used as the conditional input conditional variational autoencoder's encoder, and the experimental surface temperature distribution data is used as the target data to calculate the reconstruction loss, so that the model learns the noise characteristics of the experimental data; First, the latent vector z is sampled from the standard normal prior distribution, and the simulated surface temperature distribution data is used as a condition and input into the conditional branch of the decoder of the conditional variational autoencoder. Subsequently, the latent vector branch of the decoder performs full-connection expansion and upsampling on the latent vector, and concatenates and fuses it with the simulated image features extracted by the conditional branch in the channel dimension, finally generating enhanced simulated surface temperature distribution data with experimental noise characteristics.

4. A method for detecting hidden defects near the road surface as claimed in claim 1, characterized in that: The conditional variational autoencoder includes: an encoder and a decoder, in which a fully connected layer is combined to learn the probability distribution of the potential vector, and a simulated image is introduced as conditional information; The encoder gradually extracts features and compresses them into compact representations, maps them to the latent variable space through a fully connected layer, and then gradually deconvolves them to restore them to the original input size through a decoder combined with the conditional information.

5. A method for detecting hidden defects near the road surface as claimed in claim 4, characterized in that: The loss function of the conditional variational autoencoder is specifically: ； ； ； in, is the loss function of the conditional variational autoencoder, is the reconstruction loss, is the KL divergence loss; and are the weights of reconstruction loss and KL divergence loss respectively; , are the original input image and the reconstructed image respectively; N Represents the total number of pixels in the image; and The mean of the encoder output is and the standard deviation is , d is the dimension of the latent variable.

6. A method for detecting hidden defects near the road surface as claimed in claim 1, characterized in that: The time domain coding process is specifically as follows: The feature map after the single-channel convolution layer is divided into multiple inter-sample feature blocks along the channel dimension. The size of each feature block is fixed. Each feature block contains the temperature feature at a fixed time and is rearranged into a one-dimensional vector. The rearranged feature blocks are projected through the inter-sample fully connected layer, each inter-sample feature block is time series encoded, the encoded inter-sample feature blocks are input into the Transformer encoder, and the output dimension of the Transformer encoder is reconstructed through the tensor reconstruction layer.

7. A method for detecting hidden defects near the road surface as claimed in claim 1, characterized in that: The spatial domain encoding process is specifically as follows: The reconstructed tensor after time domain encoding is used as the input of the spatial domain encoder. The spatial domain encoder includes multiple convolutional layers. Some convolutional layers use a copy connection method to realize the splicing of shallow-level features and deep-level features. Two convolution operations are performed in each convolutional layer, and the ReLU activation function is used for nonlinear mapping before the convolution operation. The mapping from feature map to recognition result is realized through the output layer. The road surface temperature response is learned through spatial domain coding to realize the detection of hidden defects near the road surface.

8. A road near-surface hidden defect detection system, characterized in that: include: The data acquisition module is used to acquire concrete test blocks with different defects and record the experimental surface temperature distribution data of these test blocks under the condition of surface thermal wave loading; establish a simulation model to simulate the simulated surface temperature distribution data of concrete test blocks with different defects under the condition of thermal wave loading; A simulation enhancement module, used to add experimental noise to the simulated surface temperature distribution data through a conditional variational autoencoder to obtain enhanced simulated surface temperature distribution data; The model training module is used to mark defects in the experimental surface temperature distribution data and the enhanced simulation surface temperature distribution data, and use them as training data sets to train the spatiotemporal dual encoding model; The defect detection module is used to obtain the infrared thermal imaging image sequence of the road to be predicted, input the obtained image sequence into the trained time-space dual coding model, guide the defect area through time domain coding, and then refine the defect morphology through space domain coding, and finally obtain the defect detection result of the road to be predicted.

9. A terminal device, comprising a processor and a memory, wherein the processor is used to implement instructions; and the memory is used to store multiple instructions, characterized in that: The instructions are suitable for being loaded by a processor and executing the method for detecting hidden defects near the road surface as described in any one of claims 1-7.

10. A computer-readable storage medium storing a plurality of instructions, characterized in that: The instructions are suitable for being loaded by a processor of a terminal device and executing the method for detecting hidden defects near the road surface as described in any one of claims 1-7.

Images