A method and system for detecting hidden defects near the road surface

Abstract

The present invention relates to the technical field of road near-surface hidden defect detection, and specifically discloses a road near-surface hidden defect detection method and system. The method comprises: obtaining concrete test blocks containing different defects, recording experimental surface temperature distribution data of these test blocks under surface thermal wave loading conditions; simulating simulated surface temperature distribution data of concrete test blocks with different defects under thermal wave loading conditions; obtaining enhanced simulated surface temperature distribution data through a conditional variational autoencoder; marking defects in the experimental surface temperature distribution data and the enhanced simulated surface temperature distribution data, and training a spatiotemporal dual encoding model; obtaining an infrared thermal imaging image sequence of the road to be predicted, inputting the image into the trained spatiotemporal dual encoding model, and obtaining defect detection results for the road to be predicted. The present invention first guides the defect area through temporal features, and then refines the defect morphology through spatial features, significantly improving the accuracy of road near-surface hidden defect detection.

Description

Technical Field

[0001] The present invention relates to the technical field of road near-surface hidden defect detection, and in particular to a road near-surface hidden defect detection method and system. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Urban road networks are increasingly well-developed, greatly facilitating travel. However, as roads age, maintenance and repair become increasingly important. In particular, hidden defects such as cavities, voids, and damaged subgrades pose a growing threat. Therefore, employing advanced inspection technologies and scientific maintenance methods to promptly detect and repair these hidden dangers is crucial to ensuring road safety and extending their service life.

[0004] Detection of hidden defects near the road surface mainly refers to the detection of defects such as cavities, voids, cracks or water accumulation inside the road. These defects are internal defects of the road and cannot be identified by observation or machine vision methods.

[0005] Traditionally, research in the field of road defect detection has primarily focused on identifying surface cracks in the pavement. Research on hidden road defects has been limited, and identification technology is relatively scarce. Manual inspection methods, such as core drilling, damage the pavement structure and are inefficient, often used only to verify the presence of defects. Emerging ground-penetrating radar (GPR) signals are more complex and are still under development.

[0006] With the development of nondestructive testing technology, infrared thermal imaging has gained widespread application in asphalt permeability testing and road and material surface damage detection due to its intuitive and contactless nature. However, existing technologies mostly use infrared thermal imaging to detect early-stage, minute defects on the road surface, while research on detecting hidden defects within the road remains limited.

[0007] In existing technologies, most infrared thermal imaging analysis methods tend to focus only on images at a single moment, while ignoring time series information. Applying these methods to detect hidden road defects often leads to the following technical problems:

[0008] (1) Inability to effectively distinguish defective areas from normal areas: Temperature anomalies in defective areas are often a dynamic process, and only become apparent over time. For example, the air layer beneath the cavity can cause a lag in heat conduction, resulting in temperature anomalies. However, these anomalies may not be obvious or may be unstable at different time points. At a certain moment, the temperatures of defective and non-defective areas may be similar, making it impossible to correctly detect defects. Images at a single moment cannot capture these dynamic changes, making it easy to misjudge normal areas as defects or miss real defects.

[0009] (2) Affected by environmental interference, resulting in reduced robustness: External environmental factors (such as sunlight, shadows, wind speed, humidity, etc.) can affect infrared thermal images. For example, shadow effects: If a frame is affected by shadows, resulting in abnormal temperature distribution, it may be mistakenly identified as a defect. Another example is weather changes: If only images at a certain moment are used, it is impossible to determine whether the temperature anomaly is caused by a defect or weather factors.

[0010] (3) High false detection and missed detection rates: Since a single-frame image can only provide instantaneous temperature distribution, it is impossible to verify the cause of temperature anomalies, and non-defective areas are easily misjudged as defects. Temperature anomalies in some defective areas may not be obvious at certain times, but will only become apparent after evolving over time.

[0011] Although existing technologies disclose the use of deep learning technology to identify road surface defects, the input data of the neural network model is often a single-frame image. The model can only rely on spatial information and cannot learn key temporal dynamic features, resulting in poor detection results.

[0012] Furthermore, deep learning requires a large amount of high-quality training data. However, experimental data for hidden defects is limited by factors such as acquisition cost and measurement conditions. Sample production is difficult, and the number of samples is small, making it difficult to meet the data volume requirements of deep learning. While simulation can be used to obtain the required data, it lacks the noise characteristics found in experimental environments, so using simulation data directly for training can reduce the model's adaptability in real-world applications. Summary of the Invention

[0013] To address the above issues, the present invention proposes a method and system for detecting hidden defects near the road surface. The method uses conditional variational autoencoder (CVAE) technology to perform experimental noise processing on simulation data to enhance the authenticity of the data. Based on the spatiotemporal dual encoding strategy of Transformer and CNN, the temporal and spatial characteristics of temperature data are fully exploited to improve the accuracy and generalization ability of detecting hidden defects near the road surface and materials.

[0014] In some embodiments, the following technical solutions are adopted:

[0015] A method for detecting hidden defects near the road surface, comprising:

[0016] Obtain concrete test blocks with different defects and record the experimental surface temperature distribution data of these test blocks under surface thermal wave loading conditions; establish a simulation model to simulate the simulated surface temperature distribution data of concrete test blocks with different defects under thermal wave loading conditions;

[0017] Experimental noise is added to the simulated surface temperature distribution data through the conditional variational autoencoder (CVAE) to obtain enhanced simulated surface temperature distribution data;

[0018] The experimental surface temperature distribution data and the enhanced simulation surface temperature distribution data are marked with defects and used as training data sets to train the spatiotemporal dual encoding model;

[0019] Obtain a sequence of infrared thermal imaging images of the road to be predicted, input the acquired image sequence into the trained spatiotemporal dual coding model, guide the defect area through time domain coding, and then refine the defect morphology through spatial domain coding, and finally obtain the defect detection results of the road to be predicted.

[0020] As an optional solution, the spatiotemporal dual encoding model performs preliminary feature extraction on the input surface temperature distribution data through a single-channel convolutional layer and an average pooling layer. The extracted features are input into multiple consecutive Transformer encoding modules to complete time domain encoding. The feature information after time domain encoding is reconstructed through tensors and input into the CNN-based encoder-decoder structure. The upper and lower layer features are connected by copying and connecting. After re-extracting and restoring the input features, the defect prediction results are output.

[0021] As an optional solution, obtain the simulated surface temperature distribution data and experimental surface temperature distribution data of concrete specimens with the same defects to construct a training dataset for the conditional variational autoencoder;

[0022] Simulated and experimental surface temperature distribution data are simultaneously fed into a conditional variational autoencoder (CVAE). The simulated surface temperature distribution data serves as the conditional input to the CVAE's encoder, while the experimental surface temperature distribution data serves as the target data for calculating the reconstruction loss, enabling the model to learn the noise characteristics of the experimental data. After training, only the decoder portion of the CVAE (conditional variational autoencoder) is required for data conversion. Specifically, a latent vector z is first sampled from a standard normal prior distribution, while the simulated surface temperature distribution data is fed as the conditional input to the conditional branch of the decoder. The latent branch of the decoder (i.e., the latent vector branch) then fully connects and upsamples the latent vector. This latent vector is then concatenated and fused with the simulated image features extracted by the conditional branch along the channel dimension, ultimately generating enhanced simulated surface temperature distribution data with experimental noise characteristics.

[0023] As an optional solution, the conditional variational autoencoder includes an encoder and a decoder, with a fully connected layer in the middle to learn the distribution of latent variables; the encoder gradually extracts features and compresses them into a compact representation, maps them to the latent variable space through the fully connected layer, and combines them with the conditional input to form a latent vector representation, which is then gradually restored to the original input size through the decoder.

[0024] The loss function of the conditional variational autoencoder is specifically:

[0025] ;

[0026] ;

[0027] ;

[0028] in, is the loss function of the conditional variational autoencoder, is the reconstruction loss, is the KL divergence loss; and are the weights of reconstruction loss and KL divergence loss respectively; 、 are the original input image and the reconstructed image respectively; N Represents the total number of pixels in the image; and The mean of the encoder output is and the standard deviation is , d is the dimension of the latent variable.

[0029] As an optional solution, the time domain coding process is specifically as follows:

[0030] The feature map after the single-channel convolution layer is divided into multiple inter-sample feature blocks along the channel dimension. The size of each feature block is fixed. Each feature block contains the temperature feature at a fixed moment and is rearranged into a one-dimensional vector.

[0031] The rearranged feature blocks are projected through the inter-sample fully connected layer, each inter-sample feature block is time series encoded, and the encoded inter-sample feature blocks are input into the Transformer encoder, and the output dimension of the Transformer encoder is reconstructed through the tensor reconstruction layer.

[0032] As an optional solution, the spatial domain encoding process is specifically as follows:

[0033] The reconstructed tensor after time-domain encoding serves as the input of the spatial domain encoder, which includes multiple convolutional layers. Some convolutional layers use replicated connections to achieve the splicing of shallow-level features and deep-level features. Each convolutional layer performs two convolution operations, and a ReLU activation function is used for nonlinear mapping before the convolution operation. The output layer maps the feature map to the recognition result. The road surface temperature response is learned through spatial domain coding to achieve the detection of hidden defects near the road surface.

[0034] In other embodiments, the following technical solutions are adopted:

[0035] A road near-surface hidden defect detection system, comprising:

[0036] The data acquisition module is used to obtain concrete test blocks with different defects and record the experimental surface temperature distribution data of these test blocks under surface thermal wave loading conditions; establish a simulation model to simulate the simulated surface temperature distribution data of concrete test blocks with different defects under thermal wave loading conditions;

[0037] A simulation enhancement module is used to add experimental noise to the simulated surface temperature distribution data through a conditional variational autoencoder to obtain enhanced simulated surface temperature distribution data;

[0038] The model training module is used to mark defects in the experimental surface temperature distribution data and the enhanced simulation surface temperature distribution data, and use them as training data sets to train the spatiotemporal dual encoding model;

[0039] The defect detection module is used to obtain a sequence of infrared thermal imaging images of the road to be predicted, input the obtained image sequence into a trained spatiotemporal dual coding model, guide the defect area through time domain coding, and then refine the defect morphology through spatial domain coding, finally obtaining the defect detection results of the road to be predicted.

[0040] In other embodiments, the following technical solutions are adopted:

[0041] A terminal device includes a processor and a memory, wherein the processor is used to implement instructions; the memory is used to store multiple instructions, and the instructions are suitable for being loaded by the processor and executed by the above-mentioned road near-surface hidden defect detection method.

[0042] In other embodiments, the following technical solutions are adopted:

[0043] A computer-readable storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded and executed by a processor of a terminal device to implement the above-mentioned method for detecting hidden defects near the road surface.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] (1) The present invention processes the simulated temperature samples through a conditional variational autoencoder, extracts the experimental noise features for subsequent noise processing of the simulated temperature samples, and generates enhanced simulated temperature samples containing experimental noise, thereby better simulating the actual test conditions, solving the problem of insufficient experimental sample data, and improving the generalization ability of the model.

[0046] (2) Due to the temperature response of thermal wave excitation, there are usually identifiable differences in the spatial temperature field distribution and time domain thermal signal characteristics between the defective area and the non-defective area near the road surface. This difference is mainly manifested in the thermal relaxation time offset caused by the change in the equivalent heat capacity of the defective area, and the surface temperature gradient anomaly caused by the change in the heat conduction path. Therefore, the present invention first performs time domain encoding on the infrared thermal imaging image sequence, which can better utilize the temperature change trend to distinguish between defective areas and normal areas, enhance robustness, reduce interference from environmental factors, and reduce errors caused by factors such as lighting and materials; then, the spatial domain features of the image sequence after time domain encoding are extracted, which can better focus on the real defective area rather than the instantaneous temperature change.

[0047] The present invention firstly guides the defect area through time characteristics and then refines the defect morphology through spatial characteristics, which significantly improves the accuracy of detecting invisible defects near the road surface.

[0048] (3) The defect detection method of the present invention can maintain a high recognition accuracy under various conditions, accurately obtain the location and shape of defects, and adapt to various actual road detection scenarios; accurate damage detection helps to take repair measures in a timely manner, reduce traffic accidents and maintenance costs, and at the same time improve the safety and durability of road use.

[0049] (4) The defect detection method of the present invention can also be applied to other near-surface damage detection, such as paint debonding detection, honeycomb composite panel debonding detection, and internal void detection of alloy materials.

[0050] Other features and advantages of additional aspects of the present invention will be given in part in the following description and in part will become obvious from the following description or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flow chart of a method for detecting hidden defects near the road surface in an embodiment of the present invention;

[0052] Figure 2 Schematic diagram of the structure of a conditional variational autoencoder according to an embodiment of the present invention;

[0053] Figure 3 This is a flow chart of the pre-processing and data partitioning of the experimental data set in an embodiment of the present invention;

[0054] Figure 4 Flowchart of pre-processing and data partitioning of simulation data sets in an embodiment of the present invention;

[0055] Figure 5 A flow chart for constructing training data for a spatiotemporal dual encoder according to an embodiment of the present invention;

[0056] Figure 6 Schematic diagram of the data processing process of the spatiotemporal dual encoder in an embodiment of the present invention;

[0057] Figure 7 Schematic diagram of the model structure of the time domain encoder in an embodiment of the present invention;

[0058] Figure 8 This is a visualization of the recognition results of different types of defects in the test set. DETAILED DESCRIPTION

[0059] It should be noted that the following detailed description is illustrative and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0060] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0061] Example 1

[0062] In one or more embodiments, a method for detecting hidden defects near the road surface is disclosed, combining Figure 1 , specifically including the following process:

[0063] S101: Acquire concrete test blocks with different defects, record experimental surface temperature distribution data of these test blocks under surface thermal wave loading conditions; establish a simulation model to simulate the simulated surface temperature distribution data of concrete test blocks with different defects under thermal wave loading conditions.

[0064] In this embodiment, combined with Figure 3 Through experimental measurement, we obtained experimental surface temperature distribution data for concrete specimens containing different defect types under surface thermal wave loading, as well as their temporal evolution. First, we clipped and compressed the experimental surface temperature distribution data to obtain node experimental temperature data with a dimension of 80×80×4000. Secondly, we segmented the node temperature data collected within 4000 seconds along the time dimension. The experimental surface thermal wave loading period was 100 seconds, and the segmentation window length was 100 seconds. A single experimental model provided 40 experimental temperature samples, each with a dimension of 80×80×100. The binary labels after annotation are displayed visually, showing that white areas represent defects with distinct edge features.

[0065] Combine Figure 4 The simulated surface nodal temperature distribution data and its temporal evolution were recorded for a defective concrete specimen under thermal wave loading using a simulation method. For a single simulation model, simulated nodal temperature data with dimensions of 80 × 80 × 4000 was obtained. Next, the nodal simulated temperature data collected over a 4000-second period was segmented along the time dimension. The thermal wave loading was modulated at a frequency of 0.01 Hz, and the segmentation window length was set to 100 seconds. This yielded 40 simulated temperature samples, each with dimensions of 80 × 80 × 100.

[0066] S102: Adding experimental noise to the simulated surface temperature distribution data through a conditional variational autoencoder to obtain enhanced simulated surface temperature distribution data.

[0067] In this embodiment, the structure of the conditional variational autoencoder (CVAE) is as follows Figure 2 As shown, it includes two parts, the encoder and the decoder, with a fully connected layer in the middle to learn the probability distribution of the potential vector.

[0068] The simulated surface temperature distribution data and experimental surface temperature distribution data of concrete specimens with the same defects are obtained to construct a training dataset for the conditional variational autoencoder.

[0069] This embodiment simultaneously feeds simulated and experimental surface temperature distribution data into a conditional variational autoencoder (CVAE). During the encoding phase, the experimental surface temperature distribution data (target) and the simulated surface temperature distribution data (condition) are concatenated along the channel dimension and then fed into the encoder. The encoder extracts features through multiple layers of convolution operations and, after flattening and fully connected layers, outputs the mean vector and log-variance vector of the latent variables. Reparameterization is then used to generate a specific latent vector z. During training, the experimental surface temperature distribution data is used as the target data to calculate the reconstruction loss. This is combined with the KL divergence loss, which constrains the latent variable distribution to a standard normal distribution, to enable the model to learn the noise characteristics of the experimental data. After training, during the inference phase, the simulated surface temperature distribution data is simply fed into the CVAE decoder as the condition, while the latent vector z is sampled from the prior distribution. After processing by the decoder, enhanced data with the experimental noise characteristics is generated.

[0070] The specific process of data processing by the conditional variational autoencoder is as follows:

[0071] The input of the conditional variational autoencoder is an 80×80×2 multi-channel image concatenated from the experimental and simulated images. The convolutional encoder gradually extracts features and compresses them into a compact representation (e.g., 10×10×4), which is then mapped to a latent variable space via a fully connected layer. Subsequently, a reparameterization technique is used to generate the latent variable z. The decoder then gradually restores the original input size (80×80×1) through a series of symmetric deconvolution operations. Simultaneously, the simulated image is used as a conditional input during the decoding phase to collaboratively generate the target experimental image. The model is trained using the experimental sample temperature as the target data, using a weighted combination of reconstruction loss and KL divergence as the loss function. This loss function reflects the error between the reconstructed temperature data and the experimental temperature data, constraining the latent data and ensuring that the variable distribution is standard normal. This allows the model to fully capture the experimental noise characteristics when extracting the residual between encoding and reconstruction. This is then applied to the subsequent noise addition process of the simulated temperature samples to generate enhanced simulated temperature samples containing experimental noise, better simulating real-world test conditions and improving the model's generalization ability. Table 1 shows the structural parameters of the CVAE model.

[0072] Table 1 CVAE model structure parameters

[0073]

[0074] In this embodiment, the conditional variational autoencoder (CVAE) employed in the encoding phase receives input consisting of the experimental and simulated temperature distribution maps concatenated along the channel dimension. This allows the latent variables to capture both the noise characteristics of the experimental data and the structural information of the simulated data. After generating a specific latent vector using a reparameterization technique, the decoder, in the generation phase, inputs this latent vector along with the simulated temperature distribution data (as conditional information) to generate enhanced simulated temperature samples with experimental noise characteristics. After training in this manner, the model can reconstruct images that approximate the experimental data while also making the latent variable distribution approach a standard normal distribution. This extracts the experimental noise characteristics, which are then used to add noise to the simulated temperature samples, better simulating real-world test conditions and improving the model's generalization capabilities.

[0075] In this embodiment, the reparameterization technique is used to decouple the random sampling distribution of the latent variable into a deterministic part and a noise part, ensuring that the gradient is transferable and achieving end-to-end training. The formula is:

[0076]

[0077] Where z represents the latent vector, μ represents the mean, σ is the standard deviation, ε Represents standard normal distribution noise, which conforms to N (0,1).

[0078] In this embodiment, the model is trained using the experimental sample temperature as the target data, and a weighted combination of reconstruction loss and KL divergence is used as the loss function. The reconstruction loss is used to measure the error between the reconstructed image and the experimental temperature sample (target data). Therefore, the mean square error is used as the reconstruction damage function. Suppose the original input image is , after the encoder-decoder process, the reconstructed image is , then the reconstruction loss can be defined as:

[0079] ;

[0080] in N Represents the total number of pixels in the image.

[0081] Assume that the mean value of the encoder output is and the standard deviation is , then the calculation formula of KL divergence loss is:

[0082] ;

[0083] in d is the dimension of the latent variable; the KL divergence loss calculation method of this embodiment can be used to calculate the latent variable z The distribution of approximates the standard normal distribution N (0,1).

[0084] After repeated testing, we found that the model has the best processing effect when the weights are 0.5 and 1.0, which is the total loss function of CVAE. :

[0085] .

[0086] By training with experimental temperature samples as target data and utilizing this loss function, the model not only reconstructs images close to the experimental data but also converges the latent variable distribution to a standard normal distribution, thereby extracting the experimental noise characteristics. After training, the inference phase simply requires inputting the simulated temperature distribution data into the CVAE decoder as a conditional input, while simultaneously sampling the latent vector z from the prior distribution. The decoder then generates enhanced simulated temperature samples that incorporate the experimental noise characteristics.

[0087] For the segmented simulated surface node temperature distribution data, the CVAE experimental noise feature is added to each data unit to generate enhanced simulated temperature samples containing experimental noise. The binary labels after annotation are visualized, showing that white areas are defect areas with obvious edge features.

[0088] S103: Defects are marked (i.e., binary labels) on the experimental surface temperature distribution data and the enhanced simulation surface temperature distribution data, and used as training data sets to train the spatiotemporal dual encoding model.

[0089] Combine Figure 5 , the enhanced simulation temperature samples and experimental temperature samples are used as data set samples for deep learning. These samples can be flexibly divided according to test requirements and used for training, verification and testing of neural networks to ensure that the model has good generalization ability and accuracy in various scenarios.

[0090] In this embodiment, the spatiotemporal dual coding model is a deep learning model that includes time domain coding and space domain coding, wherein the time domain coding adopts the Transformer coding module and the space domain coding adopts the CNN coding module to realize the learning of features of different dimensions of time and space.

[0091] Combine Figure 6 The data processing process of the spatiotemporal dual coding model is as follows:

[0092] The collected temperature thermal sequence samples are input into a single-channel convolutional layer module and an average pooling layer for preliminary feature extraction to compress the sample dimensions. The feature extraction method uses a single-channel convolutional layer. The sampling feature map obtained after the initial feature extraction is input into a continuous Transformer encoding module, which can complete the time domain encoding in the defect recognition process. The feature information after time domain encoding is then reconstructed into a feature matrix with the same dimension as the input sampling feature map through tensor reconstruction, and used as the input of the spatial domain encoding module. In the spatial domain encoding module, a CNN-based encoder-decoder structure is constructed, and the upper and lower layer features are connected through copy connections. By re-extracting and restoring the input features, spatial domain encoding ultimately achieves the output of predicted defects.

[0093] Hidden defects such as voids or peelings in roads can cause changes in the thermal conductivity characteristics of local areas. When the road surface is subjected to thermal stimulation (such as solar radiation or artificial heat sources), heat conduction in normal areas is faster and the temperature changes are relatively stable. However, due to the presence of hidden defects such as voids or peelings, heat conduction in defective areas is hindered, resulting in increased thermal inertia in these areas, which manifests as delayed temperature changes or temperature anomalies. Therefore, the main information of hidden defects is reflected first in the temperature change trend over time, rather than the spatial distribution of a single frame. In other words, the temperature characteristics in the time dimension are more obvious.

[0094] Based on this, this embodiment constructs a spatiotemporal dual-coding model. First, the input temperature sample sequence is time-coded to extract dynamic features of the time series, such as the temperature variation curve over time, the time of temperature peaks, and the temperature change rate (such as the slope during the heating and cooling phases). These features are crucial for identifying latent defects. Time coding makes it easier to extract abnormal temperature variation patterns, improving defect distinguishability.

[0095] In addition, if spatial features are directly extracted, they may be affected by factors such as road surface texture, pollution, and shadows, resulting in increased noise. This embodiment integrates information from different time points through time coding to form stable time features, thereby reducing interference from environmental factors.

[0096] After temporal encoding, the data is converted into a representation rich in temporal information. Spatial encoding is then used to further identify the spatial location, morphology (size, depth), and connectivity of the defect. This avoids the problem of false detection caused by environmental factors when directly using a single-frame image for defect detection, while also improving the expressiveness of spatial features.

[0097] After time encoding, the model in this embodiment can identify areas with abnormal temperature changes and then use spatial feature extraction to determine the shape and location of these abnormal areas. It can better focus on the actual defect area rather than instantaneous temperature changes, and help to locate defects more accurately rather than just detecting "an abnormality somewhere."

[0098] As a specific implementation method, time domain coding is a time series analysis method based on the Transformer architecture. Figure 7 As shown, the feature map that has passed the single-channel convolutional layer is first divided into multiple inter-sample feature blocks along the channel dimension. The size of each feature block is fixed and is rearranged into a one-dimensional vector. According to the dimension of the previously input training sample, each sampling feature map can be divided into 100 inter-sample feature blocks, each feature block contains the temperature features at a fixed moment. Subsequently, the rearranged feature blocks are projected through the inter-sample fully connected layer, and each inter-sample feature block is time series encoded. The encoded inter-sample feature blocks are input into the Transformer encoder. The Transformer encoder consists of two normalization layers, a multi-head self-attention calculation layer, and a feedforward neural network. The formula of the multi-head self-attention mechanism in the Transformer encoder is:

[0099] ;

[0100] in, Q, K 、V Represents query (Query), key (Key) and value (Value), dk is the dimension of the key vector. The multi-head attention computation mechanism allows the model to simultaneously learn information from different representation subspaces, while the feedforward neural network further processes the features at each moment before outputting them. At the end of temporal encoding, the tensor reconstruction layer reconstructs the output dimensions of the Transformer encoder to enable subsequent spatial encoding.

[0101] The spatial domain encoding inherits the reconstructed tensor after the time domain encoding. Table 2 gives the model structure parameters of the spatial domain encoder.

[0102] Table 2 Model structure parameters of spatial domain encoder

[0103]

[0104] The spatial domain encoder consists of five convolutional layers, two max pooling layers, two upsampling layers, and a final output layer. Some convolutional layers use replicated connections to concatenate shallow and deep features. Each convolutional layer performs two convolution operations, with kernel sizes, strides, and padding of 3, 1, and 1, respectively. A ReLU activation function is used before the convolution operation for nonlinear mapping. Finally, an output layer consisting of 1×1 convolutions maps the feature map to the recognition result. Learning the surface temperature response through spatial domain encoding ultimately enables the identification of internal defects in concrete specimens under sinusoidal thermal signal excitation. SGD is used to optimize hyperparameters, with an initial learning rate of 0.001 and a batch size of 10, for a total of 100 epochs. A linear combination of the BCE loss and the Dice loss is used, with weights of 0.5 and 1, respectively.

[0105] The learning rate adjustment formula of SGD can be expressed as:

[0106] ;

[0107] in η new is the new learning rate, η old is the old learning rate, decay_rate is the decay rate, epoch is the current iteration number, total_epochs is the total number of iterations.

[0108] The calculation formula of BCE loss function is:

[0109] ;

[0110] in, y true and y pred Represents the true label and the predicted output after sigmoid transformation.

[0111] The Dice loss function focuses on reducing the set difference between the predicted and true labels, and its calculation formula is:

[0112] ;

[0113] in, X and Y represent the true labels and predicted results respectively.

[0114] After repeated tests of the linear combination of the BCE loss function and the Dice loss function, it was found that when the weights were 0.5 and 1.0, the model had the best recognition effect. The formula can be written as:

[0115] ;

[0116] In this embodiment, the spatiotemporal dual encoding model is intended to achieve accurate detection of defect edges in samples and ultimately output them in the form of a binary image.

[0117] The commonly used accuracy indicators in the field of image segmentation are used for evaluation, such as Intersection over Union (IoU), Recall, Precision, Accuracy and F1 score.

[0118] Table 3 records the detection accuracy of various types of defects in the test set under different loading powers.

[0119] Table 3 Statistics of recognition accuracy of different types of defects in the test set

[0120]

[0121] Among them, center, top, right and top right represent the different positions of the defect center point, and the numbers 123 represent the size of the defect. Figure 8 The labels of each defect sample in the above test set and the visualization results after identification are shown. Table 4 records the accuracy indicators of the deep learning model when this non-preset loading power is used as a test.

[0122] Table 4. Statistics of deep learning model detection accuracy under non-preset loading power

[0123]

[0124] This embodiment tests the accuracy index achieved by the deep learning model when facing defects of different sizes or positions. The results show that for specific non-preset defects, the location and identification of defects can be achieved under different loading powers, and as the loading power increases, the various recognition accuracy indicators gradually increase and tend to stabilize. Furthermore, this study tested the accuracy values ​​that can be achieved for each type of defect when the loading power deviates. The results show that when the loading power of the samples in the test set is high, the obtained IoU and F1score indicators can exceed 95%, showing excellent recognition performance, and the model can still perform effective measurements as a whole. Whether it is for untrained non-preset size defects or when the test power and training power deviate, the proposed deep learning model can accurately identify the corresponding defect area. These experimental results fully demonstrate that the proposed model has excellent generalization performance.

[0125] S104: Obtain an infrared thermal imaging image sequence of the road to be predicted, input the obtained image sequence into the trained spatiotemporal dual coding model, guide the defect area through time domain coding, and then refine the defect morphology through spatial domain coding to obtain the final defect detection results of the road to be predicted, including the defect type (such as voids, cracks, or water accumulation), center location, and boundary.

[0126] The output results are shown in the form of a binary image, which clearly shows the defective areas and the non-defective areas, and the boundaries are relatively clear.

[0127] Example 2

[0128] In one or more embodiments, a road near-surface hidden defect detection system is disclosed, specifically comprising:

[0129] The data acquisition module is used to obtain concrete test blocks with different defects and record the experimental surface temperature distribution data of these test blocks under surface thermal wave loading conditions; establish a simulation model to simulate the simulated surface temperature distribution data of concrete test blocks with different defects under thermal wave loading conditions;

[0130] A simulation enhancement module is used to add experimental noise to the simulated surface temperature distribution data through a conditional variational autoencoder to obtain enhanced simulated surface temperature distribution data;

[0131] The model training module is used to mark defects in the experimental surface temperature distribution data and the enhanced simulation surface temperature distribution data, and use them as training data sets to train the spatiotemporal dual encoding model;

[0132] The defect detection module is used to obtain a sequence of infrared thermal imaging images of the road to be predicted, input the obtained image sequence into a trained spatiotemporal dual coding model, guide the defect area through time domain coding, and then refine the defect morphology through spatial domain coding, finally obtaining the defect detection results of the road to be predicted.

[0133] It should be noted that the specific implementation of the above modules is the same as that in Example 1 and will not be described in detail.

[0134] Example 3

[0135] In one or more embodiments, a terminal device is disclosed, which includes a processor and a memory, wherein the processor is used to implement instructions; the memory is used to store multiple instructions, and the instructions are suitable for being loaded by the processor and executed by the road near-surface hidden defect detection method described in Example 1.

[0136] It should be understood that in this embodiment, the processor may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), off-the-shelf field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0137] The memory may include a read-only memory and a random access memory, and provides instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0138] During implementation, each step of the above method may be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software.

[0139] Example 4

[0140] In one or more embodiments, a computer-readable storage medium is disclosed, in which a plurality of instructions are stored. The instructions are suitable for being loaded by a processor of a terminal device and executed by the method for detecting hidden defects near the road surface described in Example 1.

[0141] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without any creative work are still within the scope of protection of the present invention.

Claims

1. A method for detecting hidden defects near the road surface, characterized in that: include: Obtain concrete test blocks with different defects and record the experimental surface temperature distribution data of these test blocks under surface thermal wave loading conditions; establish a simulation model to simulate the simulated surface temperature distribution data of concrete test blocks with different defects under thermal wave loading conditions; Experimental noise is added to the simulated surface temperature distribution data through a conditional variational autoencoder to obtain enhanced simulated surface temperature distribution data; The experimental surface temperature distribution data and the enhanced simulation surface temperature distribution data are marked with defects and used as training data sets to train the spatiotemporal dual encoding model; Obtain a sequence of infrared thermal imaging images of the road to be predicted, input the acquired image sequence into a trained spatiotemporal dual encoding model, use temporal domain encoding to identify defect areas, and then use spatial domain encoding to refine defect morphology, ultimately obtaining defect detection results for the road to be predicted. The spatiotemporal dual encoding model performs preliminary feature extraction on the input surface temperature distribution data through a single-channel convolutional layer and an average pooling layer. The extracted features are input into a continuous Transformer encoding module to complete time domain encoding. The feature information after time domain encoding is reconstructed through tensors and input into a CNN-based encoder-decoder structure. The upper and lower layer features are connected by copying and connecting. After re-extracting and restoring the input features, the defect prediction results are output.

2. A method for detecting hidden defects near the road surface according to claim 1, characterized in that: The simulated surface temperature distribution data and experimental surface temperature distribution data of concrete specimens with the same defects are obtained to construct a training dataset for the conditional variational autoencoder. The simulated surface temperature distribution data and the experimental surface temperature distribution data are simultaneously provided to the conditional variational autoencoder, wherein the simulated surface temperature distribution data is used as the conditional input to the encoder of the conditional variational autoencoder, and the experimental surface temperature distribution data is used as the target data for calculating the reconstruction loss, so that the model learns the noise characteristics of the experimental data; First, the latent vector z is sampled from the standard normal prior distribution. At the same time, the simulated surface temperature distribution data is used as a condition and input into the conditional branch of the decoder of the conditional variational autoencoder. Subsequently, the latent vector branch of the decoder performs full-connection expansion and upsampling on the latent vector, and splices and fuses it with the simulated image features extracted by the conditional branch in the channel dimension, finally generating enhanced simulated surface temperature distribution data with experimental noise characteristics.

3. A method for detecting hidden defects near the road surface according to claim 1, characterized in that: The conditional variational autoencoder includes an encoder and a decoder, with a fully connected layer in between to learn the probability distribution of the latent vector and introduce a simulated image as conditional information. The encoder gradually extracts features and compresses them into compact representations, maps them to the latent variable space through a fully connected layer, and then gradually deconvolutes them to restore them to the original input size in combination with the conditional information through the decoder.

4. A method for detecting hidden defects near the road surface according to claim 3, characterized in that: The loss function of the conditional variational autoencoder is specifically: ； ； ； in, is the loss function of the conditional variational autoencoder, is the reconstruction loss, is the KL divergence loss; and are the weights of reconstruction loss and KL divergence loss respectively; 、 are the original input image and the reconstructed image respectively; N Represents the total number of pixels in the image; and The mean of the encoder output is and the standard deviation is , d is the dimension of the latent variable.

5. The method for detecting hidden defects near the road surface according to claim 1, wherein: The time domain coding process is specifically as follows: The feature map after the single-channel convolution layer is divided into multiple inter-sample feature blocks along the channel dimension. The size of each feature block is fixed. Each feature block contains the temperature feature at a fixed moment and is rearranged into a one-dimensional vector. The rearranged feature blocks are projected through the inter-sample fully connected layer, each inter-sample feature block is time series encoded, and the encoded inter-sample feature blocks are input into the Transformer encoder, and the output dimension of the Transformer encoder is reconstructed through the tensor reconstruction layer.

6. A method for detecting hidden defects near the road surface according to claim 1, characterized in that: The spatial domain coding process is specifically as follows: The reconstructed tensor after temporal domain encoding serves as the input to the spatial domain encoder, which includes multiple convolutional layers. Some convolutional layers use replicated connections to combine shallow and deep features. Each convolutional layer performs two convolution operations, and a ReLU activation function is used for nonlinear mapping before the convolution operation. The output layer maps the feature map to the recognition result. The road surface temperature response is learned through spatial domain coding to detect hidden defects near the road surface.

7. A road surface hidden defect detection system, characterized in that: include: The data acquisition module is used to obtain concrete test blocks with different defects and record the experimental surface temperature distribution data of these test blocks under surface thermal wave loading conditions; establish a simulation model to simulate the simulated surface temperature distribution data of concrete test blocks with different defects under thermal wave loading conditions; A simulation enhancement module is used to add experimental noise to the simulated surface temperature distribution data through a conditional variational autoencoder to obtain enhanced simulated surface temperature distribution data; The model training module is used to mark defects in the experimental surface temperature distribution data and the enhanced simulation surface temperature distribution data, and use them as training data sets to train the spatiotemporal dual encoding model; The defect detection module is used to obtain a sequence of infrared thermal imaging images of the road to be predicted, input the obtained image sequence into a trained spatiotemporal dual coding model, use temporal domain coding to identify defect areas, and then use spatial domain coding to refine the defect morphology, ultimately obtaining defect detection results for the road to be predicted; The spatiotemporal dual encoding model performs preliminary feature extraction on the input surface temperature distribution data through a single-channel convolutional layer and an average pooling layer. The extracted features are input into a continuous Transformer encoding module to complete time domain encoding. The feature information after time domain encoding is reconstructed through tensors and input into a CNN-based encoder-decoder structure. The upper and lower layer features are connected by copying and connecting. After re-extracting and restoring the input features, the defect prediction results are output.

8. A terminal device comprising a processor and a memory, wherein the processor is used to implement instructions; the memory is used to store multiple instructions, characterized in that: The instructions are suitable for being loaded by a processor and executing the method for detecting hidden defects near the road surface according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a plurality of instructions, characterized in that: The instructions are suitable for being loaded by a processor of a terminal device and executing the method for detecting hidden defects near the road surface according to any one of claims 1 to 6.

Images