A method, medium, and system for enhancing the characteristics of surface cracks in infrastructure.

Abstract

This invention provides a method, medium, and system for enhancing the features of cracks on infrastructure surfaces, belonging to the field of crack detection technology. The invention uses a structured light sensor and an inertial navigation device to scan the infrastructure surface and acquire a corrected three-dimensional depth image. An incremental encoder synchronously triggers a high-speed camera to acquire the corresponding two-dimensional grayscale image. A stereo calibration model is established, and a sub-pixel-level phase correlation algorithm is used to achieve pixel-level precise alignment between the two-dimensional grayscale image and the three-dimensional depth image. Crack contour features and crack centerlines are extracted and superimposed and fused into the three-dimensional depth image to generate an enhanced crack feature image. This image is then input into a multi-scale crack recognition model for identification and classification. This invention solves the technical problem of poor enhancement effect of crack features on infrastructure surfaces due to the difficulty in achieving pixel-level precise fusion of two-dimensional image texture features and three-dimensional depth data.

Description

Technical Field

[0001] This invention belongs to the field of crack detection technology, and more specifically, relates to a method, medium, and system for enhancing the characteristics of cracks on the surface of infrastructure. Background Technology

[0002] Road surface cracks are one of the most common types of defects in highway maintenance. Their timely and accurate identification is crucial for ensuring road safety and extending road lifespan. With the continuous increase in highway construction mileage and the growing demand for intelligent maintenance, the accuracy of road surface crack detection directly impacts the scientific and timely nature of maintenance decisions. Current mainstream road surface crack detection technologies are mainly divided into two categories: two-dimensional image-based and three-dimensional image-based methods. However, both have significant technical limitations and struggle to meet the high-precision detection requirements in complex scenarios.

[0003] Two-dimensional image-based recognition technology is the most widely used. Its core principle is to acquire two-dimensional images of the road surface using a camera, and then use image processing algorithms (such as edge detection, threshold segmentation, and machine learning) to extract crack features from the image and complete the recognition. However, this type of technology has significant limitations: the actual road environment is complex and variable, and factors such as changes in light intensity, shadow occlusion, road dust coverage, and oil pollution can all seriously affect the quality of two-dimensional images. At the same time, the appearance features of road cracks are highly diverse, including common black cracks as well as white cracks (such as cracks with more dust on low-grade roads). Some cracks may also appear blurred due to light and shadow interference. These problems make two-dimensional image-based automatic recognition methods prone to missed detections and false detections, resulting in generally low recognition accuracy.

[0004] To overcome the limitations of two-dimensional image recognition, crack identification technology based on three-dimensional images has emerged. This type of technology acquires three-dimensional spatial information of the road surface using three-dimensional sensing devices (such as structured light technology and binocular vision technology), extracting crack features by utilizing the elevation difference between the cracked area and the normal road surface. This effectively avoids interference factors in two-dimensional images, such as light and shadow. However, in certain special cases, the elevation difference of fine surface cracks is not significant, leading to a lack of crack features in the three-dimensional image, thus causing missed crack identification under specific conditions. For example, during the rainy season or after the road surface has been watered, rainwater fills narrow cracks, making the elevation difference between the cracked area and the surrounding road surface approach zero; fine cracks and shrinkage cracks on cement pavement surfaces are shallow, and the crack depth features are not obvious in the three-dimensional image. In these cases, the color contrast of the crack edges in the two-dimensional image is relatively large, which actually provides better crack features.

[0005] In summary, existing single 2D or 3D image crack detection technologies cannot fully address the crack detection needs under complex road surface conditions. They suffer from difficulties in crack feature extraction, high false negative rates, and consequently, low recognition accuracy. Therefore, there is an urgent need for a technical solution that can integrate the advantages of 2D and 3D images, enhance the representation of crack features under different conditions, and improve the crack detection rate to overcome the shortcomings of existing technologies. Because existing fusion methods suffer from spatial offset between 2D images and 3D depth images due to the vibration of the detection vehicle, the registration accuracy is typically only at the pixel level or higher, failing to achieve sub-pixel level precise alignment. In other words, existing technologies suffer from the technical problem of poor enhancement of crack features on infrastructure surfaces due to the difficulty in achieving pixel-level precise fusion of 2D image texture features and 3D depth data. Summary of the Invention

[0006] In view of this, the present invention provides a method, medium and system for enhancing the surface crack features of infrastructure, which can solve the technical problem in the prior art that the difficulty in achieving pixel-level accurate fusion of two-dimensional image texture features and three-dimensional depth data leads to poor enhancement effect of surface crack features of infrastructure.

[0007] This invention is implemented as follows: It provides a method, medium, and system for enhancing the surface crack features of infrastructure. The method includes: initiating a line structured light sensor to scan the infrastructure surface and acquire the original laser point cloud elevation matrix; simultaneously initiating an inertial navigation system to collect pitch, roll, and yaw angle data of a detection vehicle; performing a six-degree-of-freedom coordinate transformation correction on the original laser point cloud elevation matrix based on the pitch, roll, and yaw angle data; converting the corrected original laser point cloud elevation matrix into a three-dimensional depth image; installing incremental encoders on the wheel axles of the detection vehicle; the pulse signals output by the incremental encoders simultaneously trigger the scanning action of the line structured light sensor and the shooting action of a high-speed camera through a synchronous controller; the high-speed camera acquiring a two-dimensional grayscale image corresponding to the three-dimensional depth image; establishing a stereo calibration model of the line structured light sensor and the high-speed camera for offline calibration; and combining this with attitude compensation from the inertial navigation system. The matrix dynamically registers a 2D grayscale image and a 3D depth image. A sub-pixel-level phase correlation algorithm is used to calculate the spatial offset between the two images. Based on this spatial offset, a geometric transformation is performed on the 2D grayscale image to achieve pixel-level alignment with the 3D depth image. The 2D grayscale image is preprocessed to extract crack contour features. A skeleton extraction is performed on the crack contour features to obtain the crack centerline. The crack contour features and crack centerline are then superimposed and fused to the corresponding pixel positions in the 3D depth image to generate a crack feature-enhanced image. This enhanced image is input into a multi-scale crack recognition model for crack region identification, outputting pixel-level crack segmentation results. Based on the pixel-level segmentation results and the crack centerline, the crack width distribution is calculated. A probability density function is fitted to the crack width distribution, and crack feature parameters are extracted. The cracks are then classified based on the comparison results between the crack feature parameters and a preset threshold.

[0008] Among them, the six-degree-of-freedom coordinate transformation correction constructs a rotation transformation matrix from pitch angle data, roll angle data, and heading angle data, performs rotation and translation transformations on each point cloud coordinate in the original laser point cloud elevation matrix to obtain the corrected point cloud coordinates, and arranges the corrected point cloud coordinates into a two-dimensional matrix according to the horizontal and vertical positions.

[0009] The 3D depth image has a horizontal resolution of 1 mm and a vertical resolution of 1 mm.

[0010] The synchronization controller controls the trigger time difference between the line structured light sensor and the high-speed camera to within 30 nanoseconds.

[0011] The establishment of the stereo calibration model is carried out by offline calibration using a checkerboard calibration board to obtain the intrinsic and extrinsic parameter matrices. The intrinsic parameter matrix includes focal length, principal point coordinates, and distortion coefficients, while the extrinsic parameter matrix includes rotation matrix and translation vector.

[0012] The reprojection error of the stereo calibration model is less than 0.3 pixels.

[0013] The subpixel-level phase correlation algorithm performs a two-dimensional Fourier transform on the two-dimensional grayscale image and the three-dimensional depth image to calculate the frequency domain representation. It calculates the conjugate product of the frequency domain representation and normalizes it to obtain the cross power spectrum. It performs a two-dimensional inverse Fourier transform on the cross power spectrum to obtain the phase correlation matrix. It uses a parabolic fitting method to refine the peak positions in the phase correlation matrix at the subpixel level to obtain the subpixel-level offset.

[0014] The geometric transformation correction is achieved using bilinear interpolation.

[0015] Among them, the skeleton extraction adopts the crack skeleton extraction algorithm based on Fermat point theory. The crack pixel coordinates in the crack contour features are used to form a planar point set. The iterative reweighted least squares method is used to solve the weighted distance and minimum value from each crack pixel coordinate to the candidate skeleton point coordinate in the planar point set.

[0016] In the iterative reweighted least squares method, the weight coefficients are updated according to the current Euclidean distance in each iteration. The iteration terminates when the change in the coordinates of the candidate skeleton points is less than 0.1 mm or the number of iterations reaches 100.

[0017] The multi-scale crack recognition model includes an input layer, a multi-scale feature extraction layer, a hierarchical attention aggregation layer, and an output layer. The multi-scale feature extraction layer includes four parallel convolutional branches with kernel sizes of 3×3, 5×5, 7×7, and 9×9, and dilatation rates of 1, 2, 4, and 8, respectively.

[0018] The hierarchical attention aggregation layer comprises a channel attention submodule and a spatial attention submodule. The channel attention submodule calculates the channel weights of the output features of the four parallel convolutional branches, while the spatial attention submodule calculates the spatial weights of the output features of the four parallel convolutional branches. The hierarchical attention aggregation layer adaptively weights and fuses the output features of the four parallel convolutional branches using learnable scale weights. The scale weights are determined based on the average gradient intensity, the proportion of high-frequency components, and the depth change rate of the crack feature enhancement image.

[0019] Specifically, the probability density function fitting employs an adaptive kernel density estimation algorithm for crack width distribution modeling. The kernel density estimation uses a Gaussian kernel function, and the bandwidth parameter is initialized using the Silverman empirical rule and optimized through cross-validation. The adaptive kernel density estimation algorithm decreases the bandwidth parameter in regions with high crack width sample density and increases it in regions with low crack width sample density.

[0020] The crack characteristic parameters include peak position, peak width, and skewness. When the peak position is greater than 5 mm, it is determined to be a main crack; when the peak position is less than 2 mm, it is determined to be a micro crack; when the absolute value of the skewness is greater than 1.5, it is determined to be an asymmetric crack; and when the peak width is greater than 3 mm, it is determined to be a network crack with high width variability.

[0021] This invention establishes a stereo calibration model of a line-structured light sensor and a high-speed camera, combines it with attitude compensation from an inertial navigation system, and employs a sub-pixel-level phase correlation algorithm to achieve precise registration of a two-dimensional grayscale image and a three-dimensional depth image. By synchronously triggering scanning and shooting actions with an incremental encoder, the trigger time difference is controlled within 30 nanoseconds, eliminating spatial offset errors caused by time asynchrony. The sub-pixel-level phase correlation algorithm improves the registration accuracy from pixel-level to sub-pixel-level, enabling precise superposition and fusion of two-dimensional crack contour features and three-dimensional depth information at corresponding pixel positions, generating a crack feature-enhanced image containing texture details and depth information. In summary, this invention solves the technical problem mentioned in the background art, where the difficulty in achieving pixel-level precise fusion of two-dimensional image texture features and three-dimensional depth data leads to poor enhancement effects of crack features on infrastructure surfaces. Attached Figure Description

[0022] Figure 1 This is a flowchart of the method of the present invention.

[0023] Figure 2 This is a diagram showing the spatial offset calculation results during sub-pixel-level phase correlation registration.

[0024] Figure 3 The results of skeleton extraction and branch recognition in the network crack region are shown in the figure.

[0025] Figure 4 This is a comparison of the segmentation results of the multi-scale crack identification model for cracks of different scales.

[0026] Figure 5 The graph shows the probability density function curve for the adaptive kernel density estimation of crack width distribution.

[0027] Figure 6 This is a schematic diagram of the first set of 2D grayscale images of the road surface.

[0028] Figure 7 This is a schematic diagram of the first set of 3D depth images of the road surface.

[0029] Figure 8 This is the result image of the enhanced identification of road surface crack features in the first group.

[0030] Figure 9 This is a schematic diagram of the second set of 2D grayscale images of the road surface.

[0031] Figure 10This is a schematic diagram of the second set of 3D depth images of the road surface.

[0032] Figure 11 This is the result of enhanced identification of the second group of road surface crack features. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0034] like Figure 1 The diagram shows a flowchart of a method, medium, and system for enhancing the surface crack characteristics of infrastructure provided by this invention. The method includes the following steps: S1: The structured light sensor on the start line continuously scans the surface of the infrastructure to obtain the original laser point cloud elevation matrix. Simultaneously, the inertial navigation device is started to collect the pitch angle data, roll angle data, and heading angle data of the detection vehicle. Based on the pitch angle data, the roll angle data, and the heading angle data, the original laser point cloud elevation matrix is ​​corrected by a six-degree-of-freedom coordinate transformation to eliminate the elevation measurement error caused by the bumps and tilt of the detection vehicle. The corrected original laser point cloud elevation matrix is ​​converted into a three-dimensional depth image with a horizontal resolution of 1mm and a vertical resolution of 1mm. S2: An incremental encoder is installed on the wheel axle of the detection vehicle. The pulse signal output by the incremental encoder triggers the scanning action of the line structured light sensor and the shooting action of the high-speed camera simultaneously through the synchronization controller. The high-speed camera acquires a two-dimensional grayscale image corresponding to the three-dimensional depth image. The synchronization controller controls the trigger time difference between the line structured light sensor and the high-speed camera to within 30ns. S3: Establish a stereo calibration model of the line structured light sensing device and the high-speed camera. Use a checkerboard calibration board for offline calibration to obtain the intrinsic and extrinsic parameter matrices. Combine the attitude compensation matrix of the inertial navigation device to perform dynamic registration of the two-dimensional grayscale image and the three-dimensional depth image. Use a sub-pixel-level phase correlation algorithm to calculate the spatial offset between the two-dimensional grayscale image and the three-dimensional depth image. Perform geometric transformation correction on the two-dimensional grayscale image based on the spatial offset to achieve pixel-level alignment between the two-dimensional grayscale image and the three-dimensional depth image. S4: The two-dimensional grayscale image is preprocessed using an adaptive threshold segmentation algorithm to extract crack contour features. The crack skeleton extraction algorithm based on Fermat point theory is used to extract the crack contour features to obtain the crack centerline. Based on the pixel-level aligned spatial correspondence, the crack contour features and the crack centerline are superimposed and fused to the corresponding pixel positions of the three-dimensional depth image to generate a crack feature enhancement image. S5: Input the enhanced crack feature image into the multi-scale crack recognition model to identify the crack region. The multi-scale crack recognition model uses a multi-scale feature fusion algorithm based on hierarchical attention aggregation to extract crack features at different scales and outputs crack pixel-level segmentation results. S6: Calculate the crack width distribution based on the crack pixel-level segmentation results and the crack centerline. Use an adaptive kernel density estimation crack width distribution modeling algorithm to fit the probability density function of the crack width distribution. Extract the peak position, peak width, and skewness from the probability density function as crack feature parameters. Classify the cracks according to the comparison results between the crack feature parameters and preset thresholds. When the peak position is greater than 5mm, it is determined to be a main crack; when the peak position is less than 2mm, it is determined to be a fine crack; when the absolute value of the skewness is greater than 1.5, it is determined to be an asymmetric crack; and when the peak width is greater than 3mm, it is determined to be a network crack with high width variability.

[0035] The specific steps of the six-degree-of-freedom coordinate transformation correction include: designating the pitch angle data as (), the roll angle data as (), and the heading angle data as (); constructing a rotation transformation matrix; performing rotation and translation transformations on each point cloud coordinate in the original laser point cloud elevation matrix; and designating the transformed point cloud coordinates as the corrected point cloud coordinates. The corrected point cloud coordinates are then arranged into a two-dimensional matrix according to their lateral and longitudinal positions, where each element of the two-dimensional matrix is ​​the elevation value of its corresponding position. This two-dimensional matrix is ​​the corrected original laser point cloud elevation matrix. The rotation transformation matrix is ​​the product of rotation matrices around the three axes, and the translation transformation is compensated for based on the position offset of the inertial navigation device.

[0036] The steps for establishing the stereo calibration model specifically include: placing the checkerboard calibration board on the surface of the infrastructure, wherein the grid size of the checkerboard calibration board is 20mm×20mm; simultaneously acquiring three-dimensional depth images and two-dimensional grayscale images of the checkerboard calibration board using the line structured light sensor and the high-speed camera; extracting the corner coordinates of the checkerboard calibration board from the two-dimensional grayscale image, and extracting the corresponding three-dimensional spatial coordinates of the corner coordinates from the three-dimensional depth image; calculating the intrinsic parameter matrix of the high-speed camera based on the corner coordinates and the three-dimensional spatial coordinates, wherein the intrinsic parameter matrix includes focal length, principal point coordinates, and distortion coefficients; calculating the extrinsic parameter matrix of the high-speed camera relative to the line structured light sensor based on the corner coordinates and the three-dimensional spatial coordinates, wherein the extrinsic parameter matrix includes a rotation matrix and a translation vector; repeatedly acquiring images of the checkerboard calibration board at different positions and angles, and calculating the average value of the intrinsic parameter matrix and the extrinsic parameter matrix as the final calibration parameter, wherein the reprojection error of the final calibration parameter is less than 0.3 pixels.

[0037] The attitude compensation matrix is ​​constructed based on the pitch angle data, roll angle data and heading angle data collected by the inertial navigation device, and is used to compensate for the impact of the attitude changes of the detection vehicle during the driving process on the image registration accuracy.

[0038] The calculation steps of the sub-pixel-level phase correlation algorithm specifically include: performing a two-dimensional Fourier transform on the two-dimensional grayscale image and the three-dimensional depth image to calculate the frequency domain representation of the two-dimensional grayscale image and the frequency domain representation of the three-dimensional depth image; calculating the conjugate product of the frequency domain representation of the two-dimensional grayscale image and the frequency domain representation of the three-dimensional depth image, and normalizing the conjugate product to obtain a cross-power spectrum; performing a two-dimensional inverse Fourier transform on the cross-power spectrum to obtain a phase correlation matrix, searching for a peak position in the phase correlation matrix, the peak position corresponding to the spatial offset between the two-dimensional grayscale image and the three-dimensional depth image; using a parabolic fitting method to refine the peak position at the sub-pixel level, fitting a parabolic surface to the phase correlation values ​​corresponding to the integer coordinates of the peak position and its eight surrounding neighborhood coordinates, the extreme point coordinates of the parabolic surface being the sub-pixel-level offset; and performing the geometric transformation correction on the two-dimensional grayscale image based on the sub-pixel-level offset, the geometric transformation correction being implemented using a bilinear interpolation method.

[0039] The specific steps of the crack skeleton extraction algorithm based on Fermat point theory include: constructing a planar point set from all crack pixel coordinates in the crack contour features, and initializing the candidate skeleton point coordinates as the centroid coordinates of the planar point set; calculating the Euclidean distance from each crack pixel coordinate in the planar point set to the candidate skeleton point coordinates, and performing a weighted summation of the Euclidean distances to obtain the objective function value, with the weight coefficient initialized to 1; using an iterative reweighted least squares method to solve for the minimum value of the objective function value, updating the candidate skeleton point coordinates to the minimum point of the current weighted distance sum in each iteration, and updating the weight coefficient to the reciprocal of the current Euclidean distance; the iteration termination condition is that the change in the candidate skeleton point coordinates is less than 0.1 mm or the number of iterations reaches 100, and the trajectory of the candidate skeleton point coordinates after the iteration termination is the crack centerline; for cracks with branches, the above iterative process is performed in each branch region to obtain multiple crack centerlines.

[0040] The crack skeleton extraction algorithm based on Fermat point theory treats crack pixels as a set of planar points. It finds the geometric center skeleton line by solving the problem of minimizing the weighted distances from all crack pixels to candidate skeleton points. This method is robust to noise points and can naturally handle branching structures. The use of iterative reweighted least squares allows the algorithm to quickly converge to a stable skeleton line position. Furthermore, the dynamic adjustment of the weight coefficients enables adaptive processing of crack pixels at different positions, resulting in a smoother skeleton line located at the crack center, providing an accurate baseline for subsequent crack width measurement and feature parameter extraction. In each iteration, the iterative reweighted least squares method updates the weight coefficients based on the current Euclidean distance. Crack pixels with smaller distances receive larger weights, and those with larger distances receive smaller weights, causing the candidate skeleton point coordinates to gradually move towards the crack center region. After multiple iterations, the candidate skeleton point coordinates converge to the optimal position, forming the crack center line. The computational complexity of the crack skeleton extraction algorithm based on Fermat point theory is logarithmically related to the number of crack pixels, showing a significant advantage in computational efficiency compared to traditional thinning algorithms, making it suitable for real-time processing of large-scale crack images.

[0041] The specific structure of the multi-scale crack recognition model is as follows: The multi-scale crack recognition model includes an input layer, a multi-scale feature extraction layer, a hierarchical attention aggregation layer, and an output layer. The input layer receives the crack feature enhancement image and performs normalization processing. The multi-scale feature extraction layer includes four parallel convolutional branches with kernel sizes of 3×3, 5×5, 7×7, and 9×9, and dilatation rates of 1, 2, 4, and 8, respectively. These four parallel convolutional branches extract crack features at different receptive field scales. The hierarchical attention aggregation layer fuses the output features of the four parallel convolutional branches. The force aggregation layer includes a channel attention submodule and a spatial attention submodule. The channel attention submodule calculates the channel weights of the output features of the four parallel convolutional branches, and the spatial attention submodule calculates the spatial weights of the output features of the four parallel convolutional branches. The channel weights and spatial weights are applied to the output features of the four parallel convolutional branches to obtain weighted features. The weighted features are then summed pixel by pixel to obtain fused features. The output layer performs a 1×1 convolution and sigmoid activation on the fused features to obtain the crack pixel-level segmentation result. Each pixel value of the crack pixel-level segmentation result represents the probability that the pixel belongs to the crack.

[0042] The steps for establishing the training dataset of the multi-scale crack recognition model specifically include: acquiring enhanced crack feature images of different types of infrastructure surfaces, including asphalt pavement, cement pavement, colored asphalt pavement, airport runway, bridge pavement layer, and tunnel inner wall; manually annotating the enhanced crack feature images to label crack pixels and non-crack pixels, generating pixel-level label images; performing data augmentation on the enhanced crack feature images and the pixel-level label images, including random rotation, random flipping, random cropping, brightness adjustment, and contrast adjustment; and dividing the data-augmented enhanced crack feature images and the pixel-level label images into training set, validation set, and test set in an 8:1:1 ratio.

[0043] The training steps of the multi-scale crack recognition model specifically include: using the cross-entropy loss function as the objective function of the multi-scale crack recognition model, using the Adam optimizer for parameter optimization, setting the learning rate to 0.001, and the batch size to 16; during training, for each batch, randomly sampling the crack feature enhancement image and its corresponding pixel-level label image from the training set, inputting the crack feature enhancement image into the multi-scale crack recognition model to obtain the predicted crack pixel-level segmentation result, calculating the cross-entropy loss function value between the predicted crack pixel-level segmentation result and the pixel-level label image, calculating the gradient based on the cross-entropy loss function value, and updating the parameters of the multi-scale crack recognition model; evaluating the performance of the multi-scale crack recognition model on the validation set after every 10 batches of training, stopping training when the cross-entropy loss function value on the validation set does not decrease for 20 consecutive evaluations; and evaluating the final performance of the trained multi-scale crack recognition model on the test set.

[0044] The multi-scale feature fusion algorithm based on hierarchical attention aggregation achieves simultaneous extraction of multi-scale crack features by constructing four parallel convolutional branches with different receptive fields. These four parallel convolutional branches with different receptive fields employ convolutional operations with different kernel sizes and dilation rates, enabling the multi-scale crack recognition model to simultaneously capture multi-level feature dependencies ranging from 0.3mm microcracks to 100mm wide network cracks. The channel attention submodule adaptively strengthens channel features that contribute significantly to crack features and suppresses irrelevant channel features by learning the channel importance weights of each scale branch. The spatial attention submodule adaptively focuses on crack region locations and suppresses background region locations by learning the spatial location importance weights of each scale branch.

[0045] The hierarchical attention aggregation layer adaptively weights and fuses the output features of the four parallel convolutional branches using learnable scale weights. These scale weights are dynamically adjusted based on the scale distribution of cracks in the crack feature enhancement image. When the crack feature enhancement image mainly contains micro-cracks, the scale weight of the small receptive field branch is increased; when the crack feature enhancement image mainly contains wide cracks, the scale weight of the large receptive field branch is increased. This multi-scale feature fusion algorithm based on hierarchical attention aggregation solves the problem that a single convolutional kernel cannot adequately handle crack features at different scales by combining multi-scale parallel extraction with an attention mechanism. It avoids the memory surge problem caused by storing multiple layers of images in traditional pyramid methods. This allows the multi-scale crack recognition model to significantly reduce computational resource consumption while maintaining full-scale detection capability. Furthermore, the cross-scale feature alignment mechanism eliminates semantic biases between features of different resolutions, improving the model's accuracy and robustness in recognizing multi-scale cracks.

[0046] The scale weights in the hierarchical attention aggregation layer are determined based on three parameters: the average gradient intensity, the proportion of high-frequency components, and the depth change rate of the crack feature enhancement image. The average gradient intensity is calculated using Sobel edge detection on the crack feature enhancement image, reflecting the richness of edge details. The proportion of high-frequency components is calculated by performing a two-dimensional Fourier transform on the crack feature enhancement image, determining the ratio of energy in high-frequency regions to total energy, reflecting the distribution of detail textures. The depth change rate is obtained by calculating the standard deviation of the depth difference between adjacent pixels in the three-dimensional depth image, reflecting the degree of depth fluctuation in the crack. After normalization, the average gradient intensity, the proportion of high-frequency components, and the depth change rate are input into a scale weight calculation function. This function is a multilayer perceptron, outputting scale weight values ​​for four scale branches, the sum of which is 1. When the average gradient intensity is large and the proportion of high-frequency components is high, the scale weight calculation function increases the scale weight value of the small receptive field branch to capture the features of fine cracks; when the depth change rate is large, the scale weight calculation function increases the scale weight value of the large receptive field branch to capture the global structural features of wide cracks and network cracks.

[0047] The specific steps of the adaptive kernel density estimation crack width distribution modeling algorithm include: calculating the vertical distance from each point on the crack centerline to the nearest crack edge based on the crack pixel-level segmentation results and the crack centerline; twice the vertical distance is the crack width at that location; constructing a crack width sample set from the crack widths at all locations; performing kernel density estimation on the crack width sample set using a Gaussian kernel function, with the bandwidth parameter of the Gaussian kernel function initialized using the Silverman empirical rule; optimizing the bandwidth parameter through cross-validation by randomly dividing the crack width sample set into 5 subsets, selecting 4 subsets for kernel density estimation each time, calculating the log-likelihood value on the remaining 1 subset, repeating this process 5 times, and taking the average of the log-likelihood values ​​as the evaluation index; and selecting the subset that best optimizes the evaluation index. The larger bandwidth parameter is used as the optimal bandwidth parameter. An adaptive bandwidth adjustment strategy is adopted to decrease the bandwidth parameter in regions with high crack width sample density to improve resolution, and to increase the bandwidth parameter in regions with low crack width sample density to smooth noise. The crack width sample density is obtained by calculating the number of sample points in a certain neighborhood around each sample point. The kernel density is estimated using the optimal bandwidth parameter and the adaptive bandwidth adjustment strategy to obtain the probability density function curve of the crack width. The peak position, peak width, and skewness are extracted from the probability density function curve. The peak position is the crack width corresponding to the maximum value of the probability density function curve, the peak width is the width of the probability density function curve at half-peak height, and the skewness is the degree of asymmetry of the probability density function curve relative to the peak position.

[0048] The adaptive kernel density estimation crack width distribution modeling algorithm addresses the non-uniform distribution of crack widths by employing an adaptive bandwidth kernel density estimation method to fit the probability density function of the crack width. It achieves accurate modeling of the width distribution of main and secondary cracks by decreasing the bandwidth parameter in high-density regions and increasing it in low-density regions. The cross-validation optimization of the bandwidth parameter and the adaptive bandwidth adjustment strategy enable the algorithm to automatically adjust model parameters according to the actual crack width distribution characteristics, avoiding overfitting or underfitting problems caused by a fixed bandwidth. The extracted peak positions reflect the typical width of the main crack, the peak width reflects the variability of crack width, and the skewness reflects the symmetry of the crack width distribution. These crack feature parameters provide a reliable basis for the fine classification of crack defects, enabling the algorithm to distinguish different defect types such as main cracks, microcracks, asymmetric cracks, and network cracks, providing richer defect feature information for infrastructure maintenance decisions. The Silverman rule calculates the initial bandwidth parameter based on the sample size and standard deviation of the crack width sample set, providing a reasonable starting value for subsequent cross-validation optimization.

[0049] The specific implementation methods of the above steps are described in detail below.

[0050] The specific implementation of step S1 is as follows: The line structured light sensing device continuously scans the surface of the infrastructure at a scanning frequency of 10000Hz, generating 8192 laser point cloud data points per scan. These laser point cloud data points contain three dimensions: horizontal coordinates, vertical coordinates, and elevation coordinates, forming the original laser point cloud elevation matrix. Simultaneously, the inertial navigation device collects the motion attitude data of the detection vehicle at a sampling frequency of 1000Hz, obtaining the pitch angle data, roll angle data, and heading angle data. The six-degree-of-freedom coordinate transformation correction establishes a three-dimensional rotation transformation relationship, converting the pitch angle data, roll angle data, and heading angle data into a rotation matrix. This matrix transforms the coordinate system of each point cloud coordinate in the original laser point cloud elevation matrix, eliminating the influence of pitch angle changes caused by road bumps, roll angle changes caused by road slope, and heading angle changes caused by turning on the elevation measurement during the vehicle's movement. The rotation matrix is ​​constructed using Euler angle representation, and rotations in the three directions are applied sequentially to the original coordinates to obtain the corrected point cloud coordinates. The corrected point cloud coordinates are resampled into a regular grid with a horizontal interval of 1 mm and a vertical interval of 1 mm. The elevation value of each grid cell is the average elevation of the point cloud data near that location, forming the three-dimensional depth image. Each pixel of the three-dimensional depth image represents the elevation value of the corresponding location, and the pixel gray value has a linear correspondence with the elevation value.

[0051] The specific implementation of step S2 is as follows: The incremental encoder is installed on the wheel axle of the detection vehicle. When the wheel rotates, the incremental encoder outputs a periodic pulse signal, the frequency of which is proportional to the speed of the detection vehicle. The synchronization controller receives the pulse signal and performs frequency division processing. When the accumulated number of pulses reaches a preset trigger interval, the synchronization controller simultaneously sends a trigger signal to the line structured light sensor and the high-speed camera. The preset trigger interval is determined based on the longitudinal resolution of the three-dimensional depth image (1 mm), ensuring that the travel distance of the detection vehicle corresponding to two adjacent triggers is 1 mm. The synchronization controller adopts a hardware triggering method, and the trigger signal is generated through level transition. The line structured light sensor and the high-speed camera immediately start scanning and shooting actions after receiving the rising edge of the trigger signal. The trigger time difference is controlled within 30 ns, corresponding to a displacement error of less than 0.001 mm when the detection vehicle travels at 80 km / h, ensuring that the two-dimensional grayscale image and the three-dimensional depth image are accurately aligned in the longitudinal position. The high-speed camera uses an area array sensor to capture a complete cross-sectional image of the road surface in a single exposure. The lateral resolution of the two-dimensional grayscale image and the lateral resolution of the three-dimensional depth image are both 1 mm.

[0052] The specific implementation of step S3 is as follows: The stereo calibration model is established using the Zhang Zhengyou calibration method. The checkerboard calibration plate is placed at multiple different positions and angles, and the known geometric structure of the checkerboard calibration plate is used to solve for the intrinsic and extrinsic parameters of the high-speed camera. The intrinsic parameter matrix includes the focal length parameter, principal point coordinate parameter, radial distortion coefficient, and tangential distortion coefficient of the high-speed camera. The focal length parameter describes the magnification of the high-speed camera, the principal point coordinate parameter describes the positional offset of the image center, and the distortion coefficient describes the nonlinear effect of lens optical distortion on the image. The extrinsic parameter matrix describes the spatial positional relationship between the high-speed camera coordinate system and the line structured light sensing device coordinate system, including rotation and translation relationships. The attitude compensation matrix is ​​constructed based on the attitude data collected in real time by the inertial navigation device to compensate for the influence of the attitude deviation of the detection vehicle at the moment of image acquisition on image registration. The dynamic registration process first uses the intrinsic parameter matrix to correct the distortion of the 2D grayscale image, eliminating image deformation caused by lens distortion. Then, it uses the extrinsic parameter matrix and the pose compensation matrix to establish the correspondence between the pixel coordinates of the 2D grayscale image and the pixel coordinates of the 3D depth image. The subpixel-level phase correlation algorithm calculates the precise offset between the 2D grayscale image and the 3D depth image using frequency domain analysis. The algorithm uses Fourier transform to convert the image from the spatial domain to the frequency domain, and calculates the phase difference between the two images in the frequency domain. This phase difference directly corresponds to the image translation. The parabolic fitting method performs subpixel-level localization of the phase correlation peak, obtaining continuous peak positions by fitting discrete sampling points near the peak. The accuracy of the subpixel-level offset reaches 0.1 pixels. The geometric transformation correction performs translation and rotation transformations on the 2D grayscale image based on the subpixel-level offset. The bilinear interpolation method is used to calculate the transformed pixel grayscale values, ensuring grayscale continuity during the image transformation process, ultimately achieving pixel-level alignment between the 2D grayscale image and the 3D depth image in spatial position.

[0053] The specific implementation of step S4 is as follows: The adaptive thresholding algorithm calculates a local threshold for each pixel of the two-dimensional grayscale image. The local threshold is determined based on the grayscale statistical characteristics within the neighborhood window surrounding the pixel, and the size of the neighborhood window is set to 31×31 pixels. The algorithm calculates the average grayscale value within the neighborhood window, compares the pixel grayscale value with the average grayscale value, and determines the pixel as a crack pixel when the pixel grayscale value is lower than the average grayscale value minus an offset of 15, and determines the pixel as a background pixel when the pixel grayscale value is higher than the average grayscale value minus an offset of 15. The adaptive thresholding algorithm can adapt to the illumination changes and background grayscale differences in different regions of the two-dimensional grayscale image and extract the crack contour features. The crack skeleton extraction algorithm based on Fermat point theory treats the crack pixels in the extracted crack contour features as a discrete set of points on a plane, and solves for the geometric center point to minimize the weighted sum of the distances from the geometric center point to all crack pixels. The algorithm employs an iterative reweighted least squares method for solution. Initially, the weight coefficients are set to a uniform distribution. In each iteration, the weight coefficients are updated based on the distance from the current candidate skeleton point coordinates to each crack pixel. Crack pixels that are closer receive a larger weight, while those that are farther away receive a smaller weight. Through multiple iterations, the candidate skeleton point coordinates gradually converge to the geometric center of the crack, forming the crack centerline. The iteration termination condition is set to the candidate skeleton point coordinates moving a distance of less than 0.1 mm or the number of iterations exceeding 100, ensuring that the algorithm converges within a finite time. For cracks with branching structures, the algorithm first performs connected component analysis on the crack contour features to identify each branch region. Then, it independently performs the skeleton extraction process within each branch region to obtain multiple crack centerlines while maintaining the topological connectivity of the branches. The overlay and fusion process maps the crack contour features and crack centerlines extracted from the 2D grayscale image to their corresponding positions in the 3D depth image based on the pixel-level aligned spatial correspondence. The edge information of the crack contour features and the skeleton information of the crack centerlines are then overlaid on the 3D depth image to generate the crack feature enhancement image. The crack feature enhancement image contains both the elevation undulation information provided by the three-dimensional depth image and the crack edge contrast information provided by the two-dimensional grayscale image, achieving complementary enhancement of two-dimensional and three-dimensional features.

[0054] The specific implementation of step S5 is as follows: The multi-scale crack recognition model receives the crack feature enhancement image as input. The input layer normalizes the crack feature enhancement image, mapping pixel values ​​to the range of 0 to 1, eliminating the influence of grayscale differences between different images on the model. The four parallel convolutional branches of the multi-scale feature extraction layer simultaneously extract features from the crack feature enhancement image. The branch with a 3×3 kernel size has a smaller receptive field, used to capture local detail features of fine cracks, while the branch with a 9×9 kernel size has a larger receptive field, used to capture global structural features of wide cracks and mesh cracks. The setting of the dilatancy rate further expands the receptive field range; the convolutional branch with a dilatancy rate of 8 can cover a larger image area without increasing the number of parameters. The channel attention submodule weights the output features of the four parallel convolutional branches according to the channel dimension, calculates the importance score of each channel through global average pooling and fully connected layers, and normalizes the importance score into the channel weight. The channel weight reflects the degree of contribution of different feature channels to crack recognition. The spatial attention submodule weights the output features of the four parallel convolutional branches spatially, extracts spatially saliency features through max pooling and average pooling, and generates the spatial weight map using convolutional layers. This spatial weight map highlights the location of crack regions and suppresses background regions. The channel weights and spatial weights are applied to the output features to obtain the attention-enhanced weighted features. The weighted features of the four parallel convolutional branches are summed pixel-by-pixel to fuse multi-scale features, resulting in the fused features. The output layer performs a 1×1 convolution on the fused features to compress the feature dimension into a single channel, and then uses a sigmoid activation function to map the output value to between 0 and 1. The output value represents the probability that each pixel belongs to a crack; pixels with a probability greater than 0.5 are identified as crack pixels, and pixels with a probability less than or equal to 0.5 are identified as background pixels, forming the crack pixel-level segmentation result.

[0055] The specific implementation of step S6 is as follows: Pixel coordinates of the crack region are extracted based on the crack pixel-level segmentation results. Combined with the crack centerline obtained in step S4, for each point on the crack centerline, a straight line segment perpendicular to the crack centerline intersects the crack edge. The distance between the intersection points is calculated as the vertical distance, and twice the vertical distance is used as the crack width at the corresponding position on the crack centerline. All points on the crack centerline are traversed to obtain the crack width sample set, which contains thousands to tens of thousands of sample points. The adaptive kernel density estimation crack width distribution modeling algorithm uses the Gaussian kernel function to estimate the probability density of the crack width sample set. The center position of the Gaussian kernel function corresponds to the width value of each sample point, and the bandwidth parameter controls the broadening degree of the Gaussian kernel function. The Silverman rule calculates the initial bandwidth parameter based on the sample size and standard deviation of the crack width sample set. The initial bandwidth parameter is typically set as the sample standard deviation multiplied by the negative fifth power of the sample size, then multiplied by a constant factor of 1.06. The cross-validation optimization divides the crack width sample set into five subsets. Each time, four subsets are used for kernel density estimation, and the log-likelihood value is calculated on the remaining subset. The log-likelihood value measures the fit of the estimated probability density function to the test samples. This process is repeated five times to obtain five log-likelihood values, and the average value is used as the evaluation metric. The bandwidth parameter is adjusted by searching near its initial value, and the bandwidth parameter that maximizes the evaluation metric is selected as the optimal bandwidth parameter. The adaptive bandwidth adjustment strategy calculates the local density of each sample point in the crack width sample set. For high-density regions, the bandwidth parameter is reduced to 0.7 times the optimal bandwidth parameter, and for low-density regions, the bandwidth parameter is increased to 1.5 times the optimal bandwidth parameter, achieving adaptive bandwidth adjustment. The adjusted bandwidth parameter is used to perform kernel density estimation on the crack width sample set, resulting in the probability density function curve, which describes the probability distribution of different width values. The maximum value point is searched on the probability density function curve, and the width value corresponding to the maximum value point is the peak position, which reflects the main width characteristics of the crack. The two intersection points of the probability density function curve at half the height of its maximum value are calculated. The width difference between the two intersection points is the peak width, which reflects the degree of dispersion of crack width. The third central moment of the probability density function curve relative to the peak position is calculated and normalized to obtain the skewness. A positive skewness indicates a right-skewed distribution, and a negative skewness indicates a left-skewed distribution. The absolute value of the skewness indicates the degree of asymmetry in the distribution.The peak position is compared with the threshold of 5mm. If it is greater than 5mm, it is determined to be the main crack. The peak position is compared with the threshold of 2mm. If it is less than 2mm, it is determined to be the fine crack. The absolute value of the skewness is compared with the threshold of 1.5. If it is greater than 1.5, it is determined to be the asymmetric crack. The peak width is compared with the threshold of 3mm. If it is greater than 3mm, it is determined to be the network crack with high width variability.

[0056] It should be noted that the key technical ideas of this invention include three aspects. The first key technical idea is a crack skeleton extraction algorithm based on Fermat point theory. By treating crack pixels as a set of planar points and solving the weighted distance and minimum value problem, the crack centerline can be accurately extracted. Compared with the traditional thinning algorithm based on the layer-by-layer peeling method of morphological erosion operation, the Fermat point theory method starts from the perspective of global optimization and has stronger robustness to noise points and fracture regions. It avoids the problems of pseudo-branching and skeleton fracture that are easy to produce when dealing with irregular crack morphology. Moreover, the computational complexity of the iterative reweighted least squares method is significantly lower than the pixel-by-pixel traversal process of the traditional method, thus improving the efficiency and accuracy of skeleton extraction. The second key technical approach is a multi-scale feature fusion algorithm based on hierarchical attention aggregation. By constructing parallel convolutional branches with different receptive fields, it achieves simultaneous extraction of features across all scales, from micro-cracks to wide cracks. Combining channel attention and spatial attention mechanisms, it adaptively weights and fuses features at each scale. Compared to the traditional image pyramid method, which requires storing image copies at multiple resolutions, the parallel multi-scale structure achieves multi-scale perception at a single resolution through different convolutional kernels and dilation rates. This significantly reduces memory usage while maintaining full-scale detection capability. The attention mechanism dynamically adjusts the scale weights based on the average gradient intensity, high-frequency component proportion, and depth change rate of the crack feature-enhanced image, enabling the model to adaptively strengthen key features and suppress redundant features for different scenarios. The third key technical approach is an adaptive kernel density estimation crack width distribution modeling algorithm. Addressing the non-uniform distribution of crack widths, characterized by wide main cracks and narrow branch cracks, an adaptive bandwidth kernel density estimation method is used to fit the probability density function. Compared to the discrete bucket representation of traditional histogram statistical methods, kernel density estimation provides a continuous and smooth probability density function curve, enabling a more refined characterization of the width distribution. The adaptive bandwidth adjustment strategy reduces bandwidth in high-density regions to improve resolution and increases bandwidth in low-density regions to smooth noise, avoiding overfitting or underfitting problems caused by fixed bandwidth. The extracted peak position, peak width, and skewness—three quantitative parameters—provide a reliable basis for the fine classification of crack defects. The synergistic effect of these three key technologies is as follows: the skeleton extraction based on Fermat point theory provides an accurate baseline for width measurement; the multi-scale feature fusion algorithm ensures that cracks of different scales can be effectively identified; and kernel density estimation transforms the identification results into quantifiable statistical parameters for disease classification. The three technologies form a complete technical chain from feature enhancement to identification segmentation and parameter extraction. Compared with traditional methods where each step is processed independently and error accumulation is easy to occur, this invention achieves mutual error compensation through the deep integration of the three key technologies, which significantly improves the accuracy, robustness and automation level of infrastructure surface crack detection.

[0057] It should be noted that this invention also solves the following technical problem: Existing technologies suffer from low crack recognition accuracy due to the difficulty of simultaneously capturing multi-scale features ranging from microcracks to wide cracks using a single-scale convolutional kernel. This invention constructs a multi-scale feature extraction layer containing four parallel convolutional branches, employing different kernel sizes and dilatancy rates to simultaneously extract crack features at different receptive field scales. It utilizes the channel attention submodule and spatial attention submodule of a hierarchical attention aggregation layer to adaptively weight and fuse multi-scale features. The scale weights are dynamically adjusted based on the average gradient intensity, high-frequency component proportion, and depth change rate of the crack feature-enhanced image, enabling the model to adaptively focus on crack features at different scales, achieving accurate full-scale recognition from 0.3 mm microcracks to 100 mm wide cracks.

[0058] Specifically, the principle of this invention is as follows: This invention uses a six-degree-of-freedom coordinate transformation to correct and eliminate elevation measurement errors caused by changes in the attitude of the detection vehicle, ensuring the spatial consistency of the three-dimensional depth image. An incremental encoder enables hardware-level synchronous triggering between the line structured light sensing device and the high-speed camera, ensuring the synchronicity of data acquisition from a temporal perspective. A stereo calibration model is established to obtain the camera's intrinsic and extrinsic parameters, and the spatial correspondence between the two-dimensional image and the three-dimensional depth image is established by combining the attitude compensation matrix of the inertial navigation device. A sub-pixel-level phase correlation algorithm is used to calculate the spatial offset between the two images in the frequency domain and perform parabolic fitting refinement, improving the registration accuracy to the sub-pixel level. Through pixel-level precise alignment, the crack contour features and crack centerlines extracted from the two-dimensional image can be accurately superimposed onto the corresponding positions in the three-dimensional depth image, achieving precise fusion of texture features and depth features, providing feature-enhanced input data for subsequent multi-scale crack identification.

[0059] The following provides a specific embodiment 1 of the present invention, and the specific implementation of each step in this embodiment 1 is described in detail below.

[0060] The specific implementation of step S1 is as follows: The structured light sensor is activated to continuously scan the surface of the infrastructure to obtain the original laser point cloud elevation matrix. Simultaneously, the inertial navigation device is activated to collect the pitch, roll, and yaw angle data of the detection vehicle. Based on the pitch, roll, and yaw angle data, the original laser point cloud elevation matrix is ​​corrected using a six-degree-of-freedom coordinate transformation. The rotation transformation matrix for the six-degree-of-freedom coordinate transformation correction is then described. The statement is as follows: ; In the formula, This is pitch angle data, in radians; This is the roll angle data, in radians; This is the heading angle data, in radians; , , These are rotation matrices about the three coordinate axes, respectively. Dimensionless matrix. Corrected point cloud coordinates. The statement is as follows: ; In the formula, The coordinates of the point cloud in the original laser point cloud elevation matrix are represented by a three-dimensional column vector in mm. The corrected point cloud coordinates are represented by a 3D column vector in mm. The translation vector, determined based on the position offset of the inertial navigation device, is a three-dimensional column vector with units in mm. The corrected point cloud coordinates are arranged into a two-dimensional matrix according to their horizontal and vertical positions. Each element of the two-dimensional matrix is ​​the elevation value of the corresponding position, with both horizontal and vertical resolutions of 1 mm, forming a three-dimensional depth image.

[0061] The specific implementation method of step S2 is the same as described above, and will not be repeated in detail here.

[0062] The specific implementation of step S3 involves establishing a stereo calibration model of the line structured light sensor and the high-speed camera, and using a checkerboard calibration board for offline calibration to obtain the intrinsic and extrinsic parameter matrices. Intrinsic parameter matrix The statement is as follows: ; In the formula, and This refers to the focal length of the high-speed camera in both the horizontal and vertical directions, expressed in pixels. and The coordinates are the principal point coordinates, in pixels. The extrinsic parameter matrix includes the rotation matrix. Translation vector , for A dimensionless rotation matrix that describes the attitude relationship between the high-speed camera and the line-structured light sensor. The 3D translation vector, in mm, describes the positional relationship between the high-speed camera and the line structured light sensor. It is calculated by acquiring images of a checkerboard calibration board at different positions and angles, with a reprojection error of less than 0.3 pixels. A sub-pixel-level phase correlation algorithm calculates the spatial offset between the 2D grayscale image and the 3D depth image. The specific steps involve processing the 2D grayscale image... and 3D depth images Perform a two-dimensional Fourier transform to calculate the cross-power spectrum. The statement is as follows: ; In the formula, This represents the frequency domain of a two-dimensional grayscale image. The conjugate of the frequency domain representation of a 3D depth image; and Frequency domain coordinates, in units of ; To normalize the cross-power spectrum, which is dimensionless, a two-dimensional inverse Fourier transform is performed on the cross-power spectrum to obtain the phase correlation matrix. Peak positions are then searched within the phase correlation matrix, and the corresponding spatial offsets are calculated. A parabolic fitting method is used to refine the peak positions at the sub-pixel level, with sub-pixel offsets... The calculation is achieved by fitting the phase correlation values ​​of the peak position and its surrounding eight neighborhoods. and The unit is pixels, and the accuracy of the fitted offset reaches 0.1 pixels. Geometric transformation correction is performed on the 2D grayscale image based on the sub-pixel level offset, and pixel-level alignment is achieved using bilinear interpolation.

[0063] The specific implementation of step S4 is as follows: An adaptive threshold segmentation algorithm is used to preprocess the two-dimensional grayscale image to extract crack contour features. Then, a crack skeleton extraction algorithm based on Fermat's point theory is used to extract the crack centerline from the crack contour features. The crack skeleton extraction algorithm based on Fermat's point theory constructs a planar point set from the crack pixel coordinates of all cracks in the crack contour features. ,in This represents the total number of pixels in the crack. For the first Two-dimensional coordinates of each crack pixel, in mm, initializing candidate skeleton point coordinates. Let be the centroid coordinates of the planar point set, calculated using the following formula: ; In the formula, These are the coordinates of the initial candidate skeleton points, in mm. Objective function The statement is as follows: ; In the formula, These are the coordinates of the candidate skeleton points, in mm. For the first The weight coefficient for each crack pixel, in units of ; This is the Euclidean distance, in mm. Let be the objective function value, which is dimensionless. The minimum value of the objective function is found using an iterative reweighted least squares method, with the coordinates of the candidate skeleton points updated in each iteration. The statement is as follows: ; In the formula, The number of iterations is dimensionless. For the first The coordinates of the candidate skeleton points in the next iteration, in mm; For the first The coordinates of the candidate skeleton points in the next iteration, in mm; For the first The normalized weight coefficients for the next iteration are dimensionless, and the update formula is expressed as follows: ; In the formula, To prevent the small constant from being divided by zero, the empirical value is... mm, The summation index ranges from 1 to... During the initial iteration Dimensionless. The iteration termination condition is the change in the coordinates of the candidate skeleton points. For cracks smaller than 0.1 mm or after 100 iterations, the coordinate trajectory of the candidate skeleton points after the iteration terminates is the crack centerline. Based on the pixel-level aligned spatial correspondence, the crack contour features and crack centerline are superimposed and fused to the corresponding pixel positions in the 3D depth image to generate a crack feature-enhanced image.

[0064] The specific implementation of step S5 is as follows: the enhanced crack feature image is input into a multi-scale crack recognition model for crack region identification. The multi-scale crack recognition model uses a multi-scale feature fusion algorithm based on hierarchical attention aggregation to extract crack features at different scales and outputs pixel-level crack segmentation results. The multi-scale crack recognition model includes an input layer, a multi-scale feature extraction layer, a hierarchical attention aggregation layer, and an output layer. The multi-scale feature extraction layer includes four parallel convolutional branches with kernel sizes of [sizes to be filled in]. , , and The void ratios are 1, 2, 4, and 8, respectively. Scale weights in the hierarchical attention aggregation layer. The calculation is expressed as follows: ; In the formula, This is the scale branch index, with values ​​ranging from 1 to 4; For the first The scale weights of each scale branch are dimensionless. For the first The normalized average gradient strength corresponding to each scale branch is dimensionless. For the first The proportion of high-frequency components corresponding to each scale branch is dimensionless. For the first The normalized depth change rate corresponding to each scale branch is dimensionless. This is a multilayer perceptron function that outputs dimensionless numerical values. Normalized average gradient strength. The calculation formula is expressed as follows: ; In the formula, For the first Each scale branch corresponds to the width of the image, in pixels; For the first Each scale branch corresponds to the height of the image, in pixels; For position The horizontal Sobel gradient at the given point, expressed in grayscale values; For position The vertical Sobel gradient at point is expressed in grayscale values; 255 is the maximum grayscale value of the 8-bit grayscale image, making... Normalized to a dimensionless parameter. Normalized depth change rate. The calculation formula is expressed as follows: ; In the formula, For position The depth difference between adjacent pixels, in mm; The average depth difference, in mm, is calculated using the following formula: ; This represents the maximum depth value in the 3D depth image, in mm. Let be the minimum depth value in the 3D depth image, in mm, such that Normalized to dimensionless parameters. High-frequency component proportion. The dimensionless value is obtained by calculating the proportion of high-frequency region energy to total energy after performing a two-dimensional Fourier transform on the crack feature enhancement image.

[0065] The specific implementation of step S6 is as follows: Based on the crack pixel-level segmentation results and the crack centerline, the crack width distribution is calculated; then, an adaptive kernel density estimation crack width distribution modeling algorithm is used to fit the probability density function of the crack width distribution. Crack width The statement is as follows: ; In the formula, The first one on the center line of the crack The crack width at each point is expressed in mm. For the first The vertical distance from each point to the nearest crack edge, in mm. The crack widths at all locations form a crack width sample set. ,in The number of points along the crack centerline is dimensionless. The probability density function of the kernel density estimate. The statement is as follows: ; In the formula, This represents the crack width variable, in mm. The value is the probability density function value, in units of ; The Gaussian kernel function is dimensionless. For the first The adaptive bandwidth parameter for each sample point, in mm. Gaussian kernel function. The statement is as follows: ; In the formula, Standardized variables, dimensionless. Adaptive bandwidth parameter. The statement is as follows: ; In the formula, The optimal bandwidth parameters obtained through cross-validation optimization are in mm. For the first The sample density within the neighborhood of each sample point, in units of ; The global sample density is expressed in units of... Optimal bandwidth parameters The calculation is performed using Silverman's rule of thumb, and is expressed as follows: ; In the formula, 1.06 represents the standard deviation of the crack width sample set, in mm; and These are the dimensionless coefficients of Silverman's rule of thumb. Local sample density. and global sample density The calculation formula is expressed as follows: ; ; In the formula, For the first The number of sample points in the neighborhood of a sample point, dimensionless; The neighborhood radius is empirically defined as 3 mm. This represents the maximum value of the crack width sample set, in mm; This represents the minimum crack width in the sample set, expressed in mm. Peak locations are extracted from the probability density function curve. Peak width and skewness Peak location, as a characteristic parameter of cracks The crack width, in mm, corresponds to the maximum value of the probability density function curve. The probability density function curve at half-peak height The corresponding width, in mm, is the skewness. The calculation formula is expressed as follows: ; In the formula, The skewness is dimensionless. Cracks are classified according to the comparison between crack characteristic parameters and preset thresholds. When the peak position is greater than 5 mm, it is identified as a main crack; when the peak position is less than 2 mm, it is identified as a micro crack; when the absolute value of the skewness is greater than 1.5, it is identified as an asymmetric crack; and when the peak width is greater than 3 mm, it is identified as a network crack with high width variability.

[0066] To better understand and implement this invention, the following is a specific application scenario of this invention, Example 2: To verify the effectiveness of this invention, technicians constructed a test environment and evaluated the practical application performance of the method by conducting crack detection tests on an asphalt pavement of a highway. The test section was 500m long and 3.75m wide, with a service life of 8 years, and showed varying degrees of cracking. Technicians mounted a line structured light sensor and a high-speed camera on the detection vehicle. The scanning frequency of the line structured light sensor was set to 2000Hz, the scan line width to 4096 pixels, and the high-speed camera resolution to 4096×3000 pixels with a frame rate of 2000fps. Incremental encoders with a resolution of 10000 pulses / revolution were installed on the wheel axles of the detection vehicle. The wheel diameter was 650mm, and the vehicle's speed was set to 20km / h. An inertial navigation system with a sampling frequency of 200Hz was used to collect real-time data on the vehicle's pitch, roll, and yaw angles.

[0067] During testing, technicians activated the system to continuously scan the road surface. The raw laser point cloud elevation matrix acquired by the line structured light sensor contained approximately 8.2 million point cloud data points. The inertial navigation system simultaneously recorded pitch angle data ranging from -2.3° to 1.8°, roll angle data ranging from -1.5° to 1.2°, and heading angle data ranging from -0.8° to 0.6°. Based on this attitude data, technicians constructed a rotation transformation matrix and performed a six-degree-of-freedom coordinate transformation correction on the raw laser point cloud elevation matrix, eliminating elevation measurement errors caused by vehicle bumps and tilting. The corrected laser point cloud elevation matrix was converted into a three-dimensional depth image with a lateral resolution of 1 mm and a longitudinal resolution of 1 mm, with an image size of 3750 × 500,000 pixels. The synchronization controller precisely controlled the trigger time difference between the line structured light sensor and the high-speed camera to within 25 ns, ensuring a strict temporal correspondence between the acquired two-dimensional grayscale image and the three-dimensional depth image.

[0068] Technicians used a checkerboard calibration board for offline calibration. The board's grid size was 20mm × 20mm, containing 10 × 14 corner points. Thirty sets of calibration images were acquired at different positions and angles. The intrinsic parameter matrix of the high-speed camera was calculated using a stereo calibration algorithm, with a focal length of [missing information]. =4250 pixels and =4248 pixels, principal point coordinates are =(2048, 1500) pixels, distortion coefficient =-0.0342、 =0.0028、 =0.0005、 =-0.0003. The extrinsic parameter matrix, including the rotation matrix and translation vector, reflects the spatial positional relationship of the high-speed camera relative to the line structured light sensor, with a reprojection error of 0.28 pixels. Combined with the attitude compensation matrix of the inertial navigation system, technicians perform dynamic registration of the 2D grayscale image and the 3D depth image. For example... Figure 2 As shown, the spatial offset was calculated to be 3.62 pixels horizontally and 1.85 pixels vertically using a sub-pixel-level phase correlation algorithm. Based on this offset, the two-dimensional grayscale image was corrected by bilinear interpolation geometric transformation, achieving pixel-level alignment with a registration accuracy of 0.15 pixels.

[0069] Technicians preprocessed the 2D grayscale image using an adaptive thresholding segmentation algorithm to extract crack contour features. The adaptive threshold window size was set to 31×31 pixels, and the constant parameter was set to 5. Based on the extracted crack contour features, technicians applied a crack skeleton extraction algorithm based on Fermat point theory to extract the skeleton. For a typical longitudinal main crack, the number of crack pixels is 45680, and the initial candidate skeleton point coordinates are set as the coordinates of the crack contour centroid. The iterative reweighted least squares method converged after 38 iterations, and the change in the candidate skeleton point coordinates was reduced to 0.08 mm, resulting in a crack centerline with a length of 156 m. Figure 3 As shown, for a branched network of cracks, the algorithm performs an iterative process in each branch region, successfully extracting the centerlines of the main crack and four branch cracks. Based on the pixel-level aligned spatial correspondence, the technicians superimpose and fuse the crack contour features and crack centerlines onto the corresponding pixel positions of the 3D depth image to generate a crack feature-enhanced image.

[0070] The enhanced crack feature image is input into a multi-scale crack recognition model for crack region identification. This model contains four parallel convolutional branches with kernel sizes of 3×3, 5×5, 7×7, and 9×9, and hole ratios of 1, 2, 4, and 8, respectively. Each branch has 128 output channels. A hierarchical attention aggregation layer fuses the output features of the four branches, with channel attention and spatial attention submodules calculating channel and spatial weights, respectively. Based on the enhanced crack feature image, the average gradient intensity is calculated to be 18.5, the proportion of high-frequency components is 0.42, and the depth change rate is 2.8 mm. After inputting these values ​​into the scale weight calculation function, the weights for the four scale branches are 0.35, 0.28, 0.22, and 0.15, respectively. Figure 4 As shown, the output layer obtains pixel-level segmentation results for the cracks through 1×1 convolution and sigmoid activation, with a total detected crack area of ​​78.6. .

[0071] Technicians calculated the crack width distribution based on the pixel-level segmentation results and the crack centerline. For the main crack region, 15,600 width measurement points were sampled along the crack centerline, with crack widths ranging from 3.2 mm to 8.9 mm. Technicians used an adaptive kernel density estimation crack width distribution modeling algorithm to fit the probability density function to the width distribution. The initial bandwidth parameter of the Gaussian kernel function was calculated to be 0.62 mm according to the Silverman rule. After optimization through 5-fold cross-validation, the optimal bandwidth parameter was determined to be 0.48 mm. Figure 5 As shown, the adaptive bandwidth adjustment strategy reduces the bandwidth parameter to 0.32 mm in areas with high sample density and increases it to 0.71 mm in areas with low sample density. Crack feature parameters were extracted from the fitted probability density function curve, with a peak position of 5.6 mm, a peak width of 3.8 mm, and a skewness of 0.85. Based on the comparison results with preset thresholds, this crack was identified as a main crack and a network crack with high width variability. Twenty-three fine cracks were also detected in the test section, with an average width of 1.3 mm, and eight asymmetric cracks, with absolute skewness values ​​ranging from 1.6 to 2.1, as shown in Table 1.

[0072] Table 1 Statistical results of detection of different types of cracks

[0073] The technical team tested two sets of road surfaces, among which Figure 6 and Figure 9The first and second sets of 2D grayscale images of the road surface are shown. These images were acquired by a high-speed camera on the detection vehicle. Pulse signals were output by an incremental encoder mounted on the wheel axle, and the synchronous controller achieved precise synchronization with the line structured light sensor. The trigger time difference was strictly controlled within 30ns to ensure the spatiotemporal correspondence between the 2D and 3D images. Because 2D grayscale images can only capture the texture and grayscale variation information of the road surface, the contrast between cracks and the road background is low. The visible crack features are only about 30% of the actual cracks, making it difficult to accurately identify minute cracks less than 2mm wide, as well as crack areas obscured by pollution or shadows. Figure 7 The first set of 3D depth images of the road surface was displayed. These images were obtained by continuously scanning the original laser point cloud elevation matrix using a line structured light sensor. Simultaneously, pitch, roll, and yaw angle data collected by an inertial navigation system were combined to perform a six-degree-of-freedom coordinate transformation correction, effectively eliminating elevation measurement errors caused by vehicle bumps and tilting. The final result is a high-precision 3D depth image with a lateral and longitudinal resolution of 1mm. The 3D depth images accurately reflect the depth information and three-dimensional morphology of the cracks, exhibiting strong depth contrast. The visible crack features are completely consistent with the actual cracks, providing a reliable geometric basis for accurate crack identification. Figure 10 The second set of 3D depth images of the road surface was shown. These images were severely affected by water stains on the road surface. The specular reflection properties of the water stains prevented the laser beam from returning to the sensor normally, resulting in a complete loss of depth features. The visible crack features were reduced to zero. This fully reflects the limitations of a single 3D depth imaging mode under special environmental conditions and highlights the necessity of multimodal information fusion. Figure 8 and Figure 11 Two sets of crack feature enhancement and recognition results generated after a complete road surface processing workflow are presented. First, a stereo calibration model of the line structured light sensor and high-speed camera is established using a checkerboard calibration board to obtain the intrinsic and extrinsic parameter matrices. Then, the attitude compensation matrix of the inertial navigation device is combined to dynamically register the 2D grayscale image and the 3D depth image. A sub-pixel-level phase correlation algorithm is used to accurately calculate the spatial offset between the two and perform geometric transformation correction to achieve pixel-level alignment. Based on this, an adaptive threshold segmentation algorithm is used to extract crack contour features from the 2D grayscale image, and a crack skeleton extraction algorithm based on Fermat point theory is used to obtain the crack centerline. Finally, the crack contour features and crack centerline are superimposed and fused to the corresponding pixel positions in the 3D depth image. This fully combines the texture details of the 2D image and the geometric depth information of the 3D image, effectively compensating for the shortcomings of a single imaging mode. This generates a crack feature enhancement image with rich features, high contrast, and strong anti-interference ability, providing high-quality input data for subsequent multi-scale crack recognition models and significantly improving the detection accuracy, robustness, and reliability of different types of cracks in complex environments.

[0074] The technological advancements of this invention compared to traditional methods are mainly reflected in the following aspects: Traditional crack detection methods rely solely on two-dimensional image processing, failing to acquire crack depth information and leading to inaccurate assessments of crack severity. This invention, through the collaborative acquisition of a line-structured light sensor and a high-speed camera, fuses two-dimensional grayscale features with three-dimensional depth information to construct an enhanced crack feature image, enabling simultaneous and accurate measurement of both crack width and depth features. The inertial navigation device, combined with six-degree-of-freedom coordinate transformation correction technology, effectively eliminates systematic errors caused by the bumps and tilts of the detection vehicle, ensuring the reliability of elevation data. The crack skeleton extraction algorithm based on Fermat point theory overcomes the limitations of traditional thinning algorithms, such as sensitivity to noise and difficulty in branch processing. It achieves robust extraction of the crack centerline through iterative reweighted least squares, providing an accurate baseline for subsequent width measurement. The multi-scale crack recognition model, through a hierarchical attention aggregation mechanism, solves the problem that a single convolutional kernel cannot adequately handle cracks of different scales, making full-scale detection from micro-cracks to wide network cracks possible. The adaptive kernel density estimation crack width distribution modeling algorithm addresses the non-uniform distribution of crack width by employing an adaptive bandwidth adjustment strategy to accurately model the width distribution of main and branch cracks. The extracted peak position, peak width, and skewness, among other feature parameters, provide a quantitative basis for the fine classification of crack defects, supporting more scientific maintenance decisions.

[0075] It should be noted that the variables involved in this invention are explained in detail in Tables 2 and 3.

[0076] Table 2. Variable Explanation Table (Part 1)

[0077] Table 3. Variable Explanation Table (Part Two)

[0078] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for enhancing the surface crack characteristics of infrastructure, characterized in that, The process includes: activating a line structured light sensor to scan the infrastructure surface and acquire the original laser point cloud elevation matrix; simultaneously activating an inertial navigation system to collect pitch, roll, and yaw angle data of the detection vehicle; performing a six-degree-of-freedom coordinate transformation to correct the original laser point cloud elevation matrix based on the pitch, roll, and yaw angle data; converting the corrected original laser point cloud elevation matrix into a three-dimensional depth image; installing incremental encoders on the wheel axles of the detection vehicle; the pulse signals output by the incremental encoders triggering the scanning action of the line structured light sensor and the shooting action of the high-speed camera through a synchronous controller; the high-speed camera acquiring a two-dimensional grayscale image corresponding to the three-dimensional depth image; establishing a stereo calibration model for the line structured light sensor and the high-speed camera for offline calibration; and combining the attitude compensation matrix of the inertial navigation system to dynamically adjust the two-dimensional grayscale image and the three-dimensional depth image. The process involves state registration, employing a sub-pixel-level phase correlation algorithm to calculate the spatial offset between a 2D grayscale image and a 3D depth image. Based on this spatial offset, a geometric transformation is performed on the 2D grayscale image to achieve pixel-level alignment with the 3D depth image. The 2D grayscale image is preprocessed to extract crack contour features. A skeleton extraction is then performed on these features to obtain the crack centerline. The crack contour features and crack centerline are then superimposed and fused to the corresponding pixel positions in the 3D depth image to generate a crack feature enhancement image. This enhanced image is input into a multi-scale crack recognition model for crack region identification, outputting pixel-level crack segmentation results. Based on the pixel-level segmentation results and the crack centerline, the crack width distribution is calculated. A probability density function is fitted to the crack width distribution, and crack feature parameters are extracted. Finally, the cracks are classified according to the comparison results between the crack feature parameters and a preset threshold.

2. The method for enhancing the surface crack characteristics of infrastructure according to claim 1, characterized in that, The six-degree-of-freedom coordinate transformation correction constructs a rotation transformation matrix from pitch angle data, roll angle data, and heading angle data. It then performs rotation and translation transformations on each point cloud coordinate in the original laser point cloud elevation matrix to obtain the corrected point cloud coordinates. Finally, it arranges the corrected point cloud coordinates into a two-dimensional matrix according to their horizontal and vertical positions.

3. The method for enhancing the surface crack characteristics of infrastructure according to claim 2, characterized in that, The 3D depth image has a horizontal resolution of 1 mm and a vertical resolution of 1 mm.

4. The method for enhancing the surface crack characteristics of infrastructure according to claim 3, characterized in that, The synchronous controller keeps the trigger time difference between the line structured light sensor and the high-speed camera within 30 nanoseconds.

5. The method for enhancing the surface crack characteristics of infrastructure according to claim 4, characterized in that, The stereo calibration model was established by offline calibration using a checkerboard calibration board to obtain the intrinsic and extrinsic parameter matrices. The intrinsic parameter matrix includes focal length, principal point coordinates, and distortion coefficients, while the extrinsic parameter matrix includes rotation matrix and translation vector.

6. The method for enhancing the surface crack characteristics of infrastructure according to claim 5, characterized in that, The reprojection error of the stereo calibration model is less than 0.3 pixels.

7. The method for enhancing the surface crack characteristics of infrastructure according to claim 6, characterized in that, The subpixel-level phase correlation algorithm performs a two-dimensional Fourier transform on a two-dimensional grayscale image and a three-dimensional depth image to calculate the frequency domain representation. The conjugate product of the frequency domain representation is calculated and normalized to obtain the cross power spectrum. The cross power spectrum is then subjected to a two-dimensional inverse Fourier transform to obtain the phase correlation matrix. A parabolic fitting method is used to refine the peak positions in the phase correlation matrix at the subpixel level to obtain the subpixel-level offset.

8. The method for enhancing the surface crack characteristics of infrastructure according to claim 7, characterized in that, Geometric transformation corrections are achieved using bilinear interpolation.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions that, when executed in a computer, perform the method for enhancing the surface crack features of infrastructure according to any one of claims 1-8.

10. A system for enhancing the surface crack characteristics of infrastructure, characterized in that, The system comprises the computer-readable storage medium of claim 9, wherein the system is a computer, the computer-readable storage medium is disposed within the system, and the system is provided with a microprocessor that executes program instructions stored in the computer-readable storage medium.

Images