A method and system for detecting cracks in a road surface

Abstract

The application provides a kind of road surface crack detection method and system, method includes collecting the road surface image of target road surface and laser radar point cloud data, respectively to road surface image and laser radar point cloud data are preprocessed, to obtain processing image and processing point cloud;Based on processing image and processing point cloud are extracted to target area, to obtain target area image and target area point cloud;Target area image is carried out multi-scale enhancement processing, to obtain enhanced image;Based on enhanced image and target area point cloud are detected to crack, to output crack image;Crack image is carried out parameter identification and coordinate conversion, to output crack specific parameter, the application effectively solves the detection problem caused by shadow, uneven illumination and point cloud sparse, realizes the full process automation of crack from identification, segmentation to geometric parameter high-precision quantization.

Description

Technical Field

[0001] This invention belongs to the technical field of image processing, and specifically relates to a method and system for detecting road surface cracks. Background Technology

[0002] Currently, automated pavement crack detection technology faces three main challenges: 1) Poor environmental robustness: Image-based methods are susceptible to uneven lighting, shadow occlusion, and pavement stains; 2) Low geometric accuracy: Crack width measurement methods based on single-source data often ignore local nonlinear changes in crack orientation, resulting in simplistic measurement direction determination; 3) Insufficient automation: Existing solutions often require manual intervention or preset parameters in crack localization, image preprocessing, and feature fusion, making them difficult to adapt to diverse real-world scenarios. Although some research has attempted to fuse images and point clouds, these efforts have largely remained at the level of simple feature stitching, failing to construct an integrated, adaptive processing flow from data registration, target extraction, feature enhancement to precise quantification. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention provides a method and system for detecting road surface cracks, which solves the technical problems in the prior art.

[0004] In a first aspect, the present invention provides the following technical solution: a method for detecting road surface cracks, comprising:

[0005] Collect road surface images and lidar point cloud data of the target road surface, and preprocess the road surface images and lidar point cloud data respectively to obtain processed images and processed point clouds.

[0006] Based on the processed image and the processed point cloud, the target region is extracted to obtain the target region image and the target region point cloud.

[0007] The target region image is subjected to multi-scale enhancement processing to obtain an enhanced image;

[0008] Crack detection is performed based on the enhanced image and the point cloud of the target region to output a crack image;

[0009] The crack image is subjected to parameter identification and coordinate transformation to output specific crack parameters.

[0010] Compared to existing technologies, the beneficial effects of this invention are as follows: This invention utilizes the illumination-insensitive characteristic of lidar point cloud reflection intensity, deeply integrates it with image texture, and jointly optimizes it in a CRF model. This fundamentally overcomes the problem of traditional visual algorithms failing due to shadows, reflections, and stains. The "macro-micro directional fusion" crack width measurement method proposed in this invention breaks through the limitations of traditional orthogonal or fixed-direction methods, making the direction of each measurement point adaptive to the local morphology of the crack. Combined with refined skeleton extraction, the width measurement error is minimized. This invention achieves adaptive extraction of ROI rotation, eliminating the need for manual threshold setting. From GMM-CRF classification of values ​​to width measurement for automatic direction determination, the entire process greatly reduces manual intervention and parameter tuning, with a high degree of intelligence and strong universality. This invention not only provides a judgment on the "presence" or "absence" of cracks, but also provides quantitative data on width and length accurate to the millimeter level, as well as intuitive visualization results. It can be directly used for road surface technical condition assessment, maintenance decision-making, and engineering quantity calculation. This invention adopts strategies such as MobileSAM, ROI pruning, and parallel processing, which significantly improves processing speed while ensuring algorithm accuracy, enabling the system to be deployed on vehicle-mounted mobile platforms to achieve rapid inspection of large-scale road networks.

[0011] Preferably, the step of preprocessing the road surface image and the lidar point cloud data respectively to obtain the processed image and the processed point cloud specifically includes:

[0012] The road surface image is subjected to image distortion correction to obtain a processed image. The lidar point cloud data is subjected to statistical filtering to remove outlier noise points. Then, the lidar point cloud data is transformed to the camera coordinate system using the calibrated extrinsic parameter matrix to obtain the processed point cloud.

[0013] Preferably, the step of extracting the target region based on the processed image and the processed point cloud to obtain a target region image and a target region point cloud includes:

[0014] A rectangular search window is set in the processed point cloud, and the rectangular search window is moved longitudinally with a preset distance as the search step size.

[0015] Calculate the sum of the reflection intensities of all point clouds within the rectangular search window after each movement, extract the sub-region image corresponding to the rectangular search window in the processed image, and calculate the gradient magnitude variance of the grayscale image corresponding to the sub-region image.

[0016] The search value of each rectangular search window is determined based on the reflection intensity of all point clouds within the window and its variance relative to the gradient magnitude. :

[0017] ;

[0018] In the formula, To balance the weights, The first The reflection intensity and gradient magnitude variance corresponding to the rectangular search window after each movement;

[0019] The rectangular search window corresponding to the maximum search value is recorded as the reference window. The processed point cloud within the reference window is clustered and the centroid position is extracted to obtain the target centroid.

[0020] The reference window is rotated within a preset angle range with the target centroid as the rotation center and a preset angle as the rotation interval, and the reflection intensity of all point clouds in the reference window after each rotation is extracted.

[0021] The target rotation angle is determined based on the reflection intensity of all point clouds within the rotated reference window. :

[0022] ;

[0023] In the formula, Rotation angle The sum of the reflection intensities of all point clouds within the corresponding reference window;

[0024] Using the reference window corresponding to the target rotation angle as the target window, the target region image and the corresponding target region point cloud are extracted from the processed image and the processed point cloud, respectively, based on the target window.

[0025] Preferably, the step of performing multi-scale enhancement processing on the target region image to obtain an enhanced image includes:

[0026] The target region image is converted into a grayscale image, which is then segmented into several non-overlapping sub-images. A grayscale histogram is calculated for each sub-image, and a segmentation value is calculated based on the grayscale histogram. :

[0027] ;

[0028] In the formula, The total number of pixels in the sub-image. For gray levels, The dividing coefficient;

[0029] For each sub-image, the portion of the grayscale histogram that exceeds the segmentation value is cropped, and the cropped pixels are evenly redistributed to all grayscale levels to obtain a redistribution histogram.

[0030] The cumulative distribution function is calculated and mapped on the redistribution histogram to obtain a mapped image. All the mapped images are then fused using bilinear interpolation to obtain a fused image.

[0031] Construct several non-scale Gaussian kernels, and perform convolution processing on the grayscale image based on several Gaussian kernels of different scales to obtain several smooth images;

[0032] Extract the detail image of the smoothed image :

[0033] ; ;...; ;

[0034] In the formula, For the first A smooth image, It is a grayscale image. For the first A detailed image;

[0035] An enhanced image is determined based on the detailed image and the fused image. :

[0036] ;

[0037] In the formula, To merge images, To enhance the weight.

[0038] Preferably, the step of detecting cracks based on the enhanced image and the point cloud of the target region to output a crack image includes:

[0039] The enhanced image is input into a pre-trained first preset model, and the image editor in the first preset model is used to extract the features of the enhanced image and generate an initial segmentation mask.

[0040] For each connected region in the segmentation mask, extract its pixel grayscale set. Calculate the pixel grayscale set belonging to the first... Posterior probability of Gaussian components :

[0041] ;

[0042] ;

[0043] In the formula, The set of pixel grayscale values ​​belongs to the first... The posterior probability of each Gaussian component , For the first , The mixing weights of Gaussian components, , These are the first Gaussian density function and the second Gaussian density function, respectively. , The first The mean vector and covariance matrix of each Gaussian component. The dimension of the pixel grayscale set;

[0044] Update parameters based on the posterior probability:

[0045] ; ; ;

[0046] In the formula, This represents the total number of connected regions.

[0047] Repeat the posterior probability calculation and parameter update process until convergence to obtain a probability map of each pixel belonging to the crack. ;

[0048] For each pixel of the enhanced image Constructing feature fusion vectors The feature fusion vector includes the color space components of the enhanced image, texture response features, and the reflection intensity value, mean and variance of the local neighborhood reflection intensity of the point cloud of the target region.

[0049] Construct the objective function :

[0050] ;

[0051] ;

[0052] ;

[0053] In the formula, , These are unidimensional potential energy and dualdimensional potential energy, respectively. Represents pixels Labeled The probability, This indicates a tag compatibility function. , These are the first balancing weights and the second balancing weights, respectively. , Each pixel Color space components in an image, These are the bandwidth parameters for the first and second Gaussian kernels, respectively. It is a set of adjacent pixel pairs. A set of pixels;

[0054] Minimize the objective function to output a crack image.

[0055] Preferably, the step of performing parameter identification and coordinate transformation on the crack image to output specific crack parameters includes:

[0056] A parallel thinning algorithm is used to generate a crack skeleton for the crack image to obtain an initial skeleton. Points in the initial skeleton are identified, and it is determined whether the points in the initial skeleton meet the background judgment conditions. If the background judgment conditions are met, the points in the initial skeleton are used as background points. The judgment process is repeated until no new background points are generated, and the optimized skeleton is output.

[0057] Identify the first type of points, the second type of points, and the third type of points in the optimized skeleton. Starting from each first type of point, trace along the optimized skeleton until a second type of point or a third type of point is encountered. Record the chain code direction sequence and pixel step size of the corresponding tracing branch.

[0058] For each second-class point, check the branches it connects to. If the pixel step size of its branch is less than the step size threshold, then iteratively delete the current shortest score based on the chain code direction sequence until the pixel step size of all remaining branches is not less than the step size threshold or the point is no longer a second-class point, and output the crack skeleton.

[0059] Extract the main axis points of the crack skeleton, fit the main axis points to obtain an m-th degree polynomial curve, obtain the tangent slope of the m-th degree polynomial curve, and determine the final measurement direction angle based on the tangent slope.

[0060] A measurement line is defined for the skeleton points in the crack skeleton along the direction of the final measurement direction angle. The nearest edge points are searched on both sides of the measurement line on the crack image to obtain the edge point set.

[0061] Extreme points are filtered from the set of edge points according to preset rules to obtain the upper edge points. With the lower edge point :

[0062] The measurement width is determined based on the upper edge point and the lower edge point. :

[0063] ;

[0064] Determine the actual width of the crack Compared with the actual total length of the crack To obtain the specific parameters of the crack:

[0065] ;

[0066] ;

[0067] ; ;

[0068] In the formula, , These are the camera focal length and the object distance, respectively. These are the width and height of the image, respectively. These are the physical width and height of the sensor, respectively. These are the pixel physical width and height, respectively. For the final measurement of the direction angle, For the first With the The absolute value of the difference between the x-coordinates of each pixel. For the first With the The absolute value of the difference between the ordinates of each pixel.

[0069] Secondly, the present invention provides the following technical solution: a road surface crack detection system, the system comprising:

[0070] The acquisition module is used to acquire road surface images and lidar point cloud data of the target road surface, and to preprocess the road surface images and lidar point cloud data respectively to obtain processed images and processed point clouds.

[0071] The extraction module is used to extract the target region based on the processed image and the processed point cloud to obtain the target region image and the target region point cloud.

[0072] An enhancement module is used to perform multi-scale enhancement processing on the target region image to obtain an enhanced image;

[0073] The detection module is used to detect cracks based on the enhanced image and the point cloud of the target region, so as to output a crack image;

[0074] The output module is used to perform parameter recognition and coordinate transformation on the crack image to output specific crack parameters.

[0075] Preferably, the acquisition module is specifically used for:

[0076] The road surface image is subjected to image distortion correction to obtain a processed image. The lidar point cloud data is subjected to statistical filtering to remove outlier noise points. Then, the lidar point cloud data is transformed to the camera coordinate system using the calibrated extrinsic parameter matrix to obtain the processed point cloud.

[0077] Thirdly, the present invention provides the following technical solution: a computer, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the road surface crack detection method as described above.

[0078] Fourthly, the present invention provides the following technical solution: a storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the road surface crack detection method as described above. Attached Figure Description

[0079] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0080] Figure 1 This is a flowchart of the road surface crack detection method provided in Embodiment 1 of the present invention;

[0081] Figure 2 This is a structural block diagram of the road surface crack detection system provided in Embodiment 2 of the present invention;

[0082] Figure 3 This is a schematic diagram of the hardware structure of a computer provided for another embodiment of the present invention.

[0083] The embodiments of the present invention will be further described below with reference to the accompanying drawings. Detailed Implementation

[0084] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain embodiments of the present invention, and should not be construed as limiting the present invention.

[0085] Example 1

[0086] In Embodiment 1 of the present invention, as Figure 1 As shown, a method for detecting road surface cracks includes:

[0087] S1. Collect road surface images and lidar point cloud data of the target road surface, and preprocess the road surface images and lidar point cloud data respectively to obtain processed images and processed point clouds.

[0088] The preprocessing steps specifically include:

[0089] The road surface image is subjected to image distortion removal processing to obtain a processed image. The lidar point cloud data is subjected to statistical filtering to remove outlier noise points. Then, the lidar point cloud data is transformed to the camera coordinate system using the calibrated extrinsic parameter matrix to obtain the processed point cloud.

[0090] It should be noted that the methods used in the above preprocessing are all commonly used methods in the prior art, so they will not be described in detail here.

[0091] S2. Based on the processed image and the processed point cloud, the target region is extracted to obtain the target region image and the target region point cloud;

[0092] Step S2 includes:

[0093] S21. Set a rectangular search window in the processed point cloud, and move the rectangular search window longitudinally with a preset distance as the search step size;

[0094] S22. Calculate the sum of the reflection intensities of all point clouds within the rectangular search window after each movement, extract the sub-region image corresponding to the rectangular search window in the processed image, and calculate the gradient magnitude variance of the grayscale image corresponding to the sub-region image.

[0095] S23. Determine the search value of each rectangular search window based on the reflection intensity and variance of the gradient magnitude of all point clouds within the rectangular search window. :

[0096] ;

[0097] In the formula, To balance the weights, The first The reflection intensity and gradient magnitude variance corresponding to the rectangular search window after each movement;

[0098] Specifically, the search value is determined by combining the intensity features of the point cloud and the texture features of the image. The optimal window position can be determined by determining the search value. In this application, the balance weight is specifically 0.5.

[0099] S24. The rectangular search window corresponding to the maximum search value is recorded as the reference window. The processed point cloud within the reference window is clustered and the centroid position is extracted to obtain the target centroid.

[0100] Specifically, the clustering algorithm used here is the K-means clustering method.

[0101] S25. Rotate the reference window within a preset angle range with the target centroid as the rotation center and a preset angle as the rotation interval, and extract the reflection intensity of all point clouds in the reference window after each rotation.

[0102] Specifically, in this application, the reflection intensity of all point clouds after each 1-degree rotation is determined to determine the subsequent target area.

[0103] S26. Determine the target rotation angle based on the reflection intensity of all point clouds within the rotated reference window. :

[0104] ;

[0105] In the formula, Rotation angle The sum of the reflection intensities of all point clouds within the corresponding reference window;

[0106] S27. Using the reference window corresponding to the target rotation angle as the target window, extract the target region image and the corresponding target region point cloud from the processed image and the processed point cloud respectively based on the target window.

[0107] S3. Perform multi-scale enhancement processing on the target region image to obtain an enhanced image;

[0108] Step S3 includes:

[0109] S31. Convert the target region image into a grayscale image, segment the grayscale image into several non-overlapping sub-images, calculate the grayscale histogram of each sub-image, and calculate the segmentation value based on the grayscale histogram. :

[0110] ;

[0111] In the formula, The total number of pixels in the sub-image. For gray levels, The dividing coefficient;

[0112] S32. The portion of the grayscale histogram of each sub-image that exceeds the segmentation value is cropped, and the cropped pixels are evenly redistributed to all grayscale levels to obtain a redistribution histogram.

[0113] S33. Calculate and map the cumulative distribution function of the redistribution histogram to obtain a mapped image. Then, fuse all the mapped images using bilinear interpolation to obtain a fused image.

[0114] S34. Construct several Gaussian kernels of different scales, and perform convolution processing on the grayscale image according to several Gaussian kernels of different scales to obtain several smooth images.

[0115] Specifically, this application employs three different-scale Gaussian kernels for convolution operations to reflect the characteristics of the image at different scales, and also provides a rich information foundation for subsequent fusion enhancement.

[0116] S35. Extract the detail image of the smoothed image. :

[0117] ; ;...; ;

[0118] In the formula, For the first A smooth image, It is a grayscale image. For the first A detailed image;

[0119] S36. Determine the enhanced image based on the detailed image and the fused image. :

[0120] ;

[0121] In the formula, To merge images, To enhance the weight.

[0122] Specifically, the enhancement weight here is 0.7.

[0123] S4. Based on the enhanced image and the point cloud of the target region, perform crack detection to output a crack image;

[0124] Step S4 includes:

[0125] S41. Input the enhanced image into the pre-trained first preset model, and use the image editor in the first preset model to extract the features of the enhanced image and generate an initial segmentation mask;

[0126] Specifically, the first preset model here is the MobileSAM model, and the image editor is Tiny-ViT. After step S41, the image can be initially divided into foreground (crack candidate) and background.

[0127] S42. Extract the pixel grayscale set for each connected region in the segmentation mask. Calculate the pixel grayscale set belonging to the first... Posterior probability of Gaussian components :

[0128] ;

[0129] ;

[0130] In the formula, The set of pixel grayscale values ​​belongs to the first... The posterior probability of each Gaussian component , For the first , The mixing weights of Gaussian components, , These are the first Gaussian density function and the second Gaussian density function, respectively. , The first The mean vector and covariance matrix of each Gaussian component. The dimension of the pixel grayscale set;

[0131] S43. Update the parameters based on the posterior probability:

[0132] ; ; ;

[0133] In the formula, This represents the total number of connected regions.

[0134] S44. Repeat the posterior probability calculation and parameter update process until convergence to obtain the probability map of each pixel belonging to the crack. ;

[0135] Specifically, after iterating through the above posterior probability calculation and parameter update, the pixel grayscale set of each region is assigned to the category with the highest probability based on the final parameters and probabilities.

[0136] S45, for each pixel of the enhanced image Constructing feature fusion vectors The feature fusion vector includes the color space components of the enhanced image, texture response features, and the reflection intensity value, mean and variance of the local neighborhood reflection intensity of the point cloud of the target region.

[0137] Specifically, by fusing the color space components, texture response features, and reflection intensity values, mean and variance of the local neighborhood reflection intensity of the target region point cloud in the above-mentioned enhanced image through attention weights, a feature fusion vector can be obtained.

[0138] S46. Constructing the objective function :

[0139] ;

[0140] ;

[0141] ;

[0142] In the formula, , These are unidimensional potential energy and dualdimensional potential energy, respectively. Represents pixels Labeled The probability, This indicates a tag compatibility function. , These are the first balancing weights and the second balancing weights, respectively. , Each pixel Color space components in an image, These are the bandwidth parameters for the first and second Gaussian kernels, respectively. It is a set of adjacent pixel pairs. It is a set of pixels.

[0143] S47. Minimize the objective function to output a crack image;

[0144] Specifically, in this step, the objective function is minimized using algorithms such as Bellman-Ford to obtain the globally optimal label map, which can then output the binary segmented crack image.

[0145] S5. Perform parameter identification and coordinate transformation on the crack image to output specific crack parameters.

[0146] Step S5 includes:

[0147] S51. A parallel thinning algorithm is used to generate a crack skeleton for the crack image to obtain an initial skeleton. Points in the initial skeleton are identified, and it is determined whether the points in the initial skeleton meet the background judgment conditions. If the background judgment conditions are met, the points in the initial skeleton are used as background points. The judgment process is repeated until no new background points are generated, and the optimized skeleton is output.

[0148] Specifically, during the crack skeletonization process using the fast parallel thinning algorithm, the generated central axis is prone to pixel protrusions with a 90-degree angle. The presence of these points leads to inconsistent skeleton line widths, failing to meet the requirement of a single-pixel width crack skeleton central axis. These protrusions typically appear in four directions. Therefore, taking one point as an example, we name it point 1. Starting from point 1, we sequentially name its eight neighboring points clockwise from point 2 to point 9. Specifically, the top, bottom, left, and right of point 1 are points 2, 6, 8, and 4, respectively; the top left, bottom left, top right, and bottom right of point 1 are points 9, 7, 3, and 5, respectively. The background determination conditions are set according to the protrusions in different directions, resulting in four conditions:

[0149] Pixels 1, 2, and 8 have a value of 1, while pixels 4, 5, and 6 have a value of 0.

[0150] Pixels 1, 2, and 4 have a value of 1, while pixels 6, 7, and 8 have a value of 0.

[0151] Pixels 1, 4, and 6 have a value of 1, while pixels 2, 8, and 9 have a value of 0.

[0152] Pixels 1, 6, and 8 have a value of 1, while pixels 2, 3, and 4 have a value of 0.

[0153] Once any of the above conditions are met, point 1 is used as the background point, and the judgment process is repeated until no new background points are generated, and the optimized skeleton is output.

[0154] S52. Identify the first type of points, the second type of points and the third type of points of the optimized skeleton, start from each first type of point and trace along the optimized skeleton until the second type of point or the third type of point is encountered, and record the chain code direction sequence and pixel step size of the corresponding tracing branch.

[0155] Specifically, the first type of point, the second type of point, and the third type of point are endpoints, nodes, and growth points, respectively. A node is a pixel that exists on two or more central axes within its eight-neighborhood; a growth point is a pixel that exists on three or more central axes within its eight-neighborhood and has a growth branch, and is a type of node; an endpoint is a pixel that exists only on the central axis within its eight-neighborhood.

[0156] S53. For each second-class point, check the branches it connects to. If the pixel step size of its branch is less than the step size threshold, iteratively delete the current shortest score based on the chain code direction sequence until the pixel step size of all remaining branches is not less than the step size threshold or the point is no longer a second-class point, and output the crack skeleton.

[0157] S54. Extract the main axis points of the crack skeleton, fit the main axis points to obtain an m-th degree polynomial curve, obtain the tangent slope of the m-th degree polynomial curve, and determine the final measurement direction angle based on the tangent slope.

[0158] Specifically, the least squares method is used for fitting here. Furthermore, in determining the measurement direction angle, it varies according to the change in the slope dv of the tangent direction, as shown in the table below:

[0159]

[0160] Specifically, the slope range setting interval (-5, 5) covers all possible crack directions from almost horizontal to almost vertical, ensuring width measurement in any direction. Each discrimination range within this interval corresponds to a specific dv value range and a set of logical relationships for the eight neighboring pixels of a Pi point. For example, when dv is greater than 5 or less than -5, the crack trunk is considered to be in the horizontal direction, i.e., 0°. When dv is between -5 and -4.33, the direction is determined to be 11.25° based on a specific combination of Pi points, and so on. Using the 16 macro- and micro-directional feature matching rules defined by this logic, by comprehensively considering the conditions of dv and Pi points, the measurement direction can be determined for each point on the main axis of the crack, thereby realizing the width measurement direction determination by fusing macro- and micro-directional features of the crack skeleton's central axis pixels, where Pi points are the main axis points.

[0161] S55. Define a measurement line for the skeleton points in the crack skeleton along the direction of the final measurement direction angle, and search for the nearest edge points on both sides of the measurement line on the crack image to obtain the edge point set;

[0162] Specifically, after defining the measurement line, the distance between the edge points on both sides of the measurement line and the measurement line is calculated, and the edge points whose distance does not exceed the set threshold are stored in the edge point set.

[0163] S56. Filter extreme points from the set of edge points according to preset rules to obtain the upper edge points. With the lower edge point :

[0164] Specifically, the extreme point selection process is as follows: when the final measured direction angle is between 0 and 90 degrees, then... , , , When the final measured orientation angle is between 90 and 180 degrees, then , , , ,in, Represents the maximum and minimum horizontal coordinates of the edge point set. Represents the maximum and minimum vertical coordinates of the edge point set.

[0165] S57. Determine the measurement width based on the upper edge point and the lower edge point. :

[0166] ;

[0167] S58. Determine the actual width of the crack. Compared with the actual total length of the crack To obtain the specific parameters of the crack:

[0168] ;

[0169] ;

[0170] ; ;

[0171] In the formula, , These are the camera focal length and the object distance, respectively. These are the width and height of the image, respectively. These are the physical width and height of the sensor, respectively. These are the pixel physical width and height, respectively. For the final measurement of the direction angle, For the first With the The absolute value of the difference between the x-coordinates of each pixel. For the first With the The absolute value of the difference between the ordinates of each pixel.

[0172] The road surface crack detection method provided in Embodiment 1 of this invention utilizes the illumination-insensitive characteristic of lidar point cloud reflection intensity, which is deeply fused with image texture and jointly optimized within a CRF model. This fundamentally overcomes the problem of traditional visual algorithms failing due to shadows, reflections, and stains. The "macro-micro direction fusion" crack width measurement method proposed in this invention breaks through the limitations of traditional orthogonal or fixed-direction methods, making the direction of each measurement point adaptive to the local morphology of the crack. Combined with refined skeleton extraction, the width measurement error is minimized. This invention progresses from adaptive extraction of ROI rotation to eliminating the need for manual setting. From threshold GMM-CRF classification to width measurement for automatic direction determination, the entire process greatly reduces manual intervention and parameter tuning, exhibiting high intelligence and strong universality. This invention not only provides a judgment on the "presence" or "absence" of cracks, but also offers millimeter-level quantitative data on width and length, as well as intuitive visualization results. It can be directly used for road surface technical condition assessment, maintenance decision-making, and engineering quantity calculation. This invention adopts strategies such as MobileSAM, ROI pruning, and parallel processing, which significantly improves processing speed while ensuring algorithm accuracy. This allows the system to be deployed on vehicle-mounted mobile platforms to achieve rapid inspection of large-scale road networks.

[0173] Example 2

[0174] like Figure 2 As shown, in Embodiment 2 of the present invention, a pavement crack detection system is provided, the system comprising:

[0175] Acquisition module 1 is used to acquire road surface images and lidar point cloud data of the target road surface, and to preprocess the road surface images and lidar point cloud data respectively to obtain processed images and processed point clouds.

[0176] Extraction module 2 is used to extract the target region based on the processed image and the processed point cloud to obtain the target region image and the target region point cloud;

[0177] Enhancement module 3 is used to perform multi-scale enhancement processing on the target region image to obtain an enhanced image;

[0178] Detection module 4 is used to detect cracks based on the enhanced image and the point cloud of the target region, so as to output a crack image;

[0179] Output module 5 is used to perform parameter recognition and coordinate transformation on the crack image to output specific crack parameters.

[0180] Specifically, the acquisition module 1 is used for:

[0181] The road surface image is subjected to image distortion correction to obtain a processed image. The lidar point cloud data is subjected to statistical filtering to remove outlier noise points. Then, the lidar point cloud data is transformed to the camera coordinate system using the calibrated extrinsic parameter matrix to obtain the processed point cloud.

[0182] The extraction module 2 is used for:

[0183] A rectangular search window is set in the processed point cloud, and the rectangular search window is moved longitudinally with a preset distance as the search step size.

[0184] Calculate the sum of the reflection intensities of all point clouds within the rectangular search window after each movement, extract the sub-region image corresponding to the rectangular search window in the processed image, and calculate the gradient magnitude variance of the grayscale image corresponding to the sub-region image.

[0185] The search value of each rectangular search window is determined based on the reflection intensity of all point clouds within the window and its variance relative to the gradient magnitude. :

[0186] ;

[0187] In the formula, To balance the weights, The first The reflection intensity and gradient magnitude variance corresponding to the rectangular search window after each movement;

[0188] The rectangular search window corresponding to the maximum search value is recorded as the reference window. The processed point cloud within the reference window is clustered and the centroid position is extracted to obtain the target centroid.

[0189] The reference window is rotated within a preset angle range with the target centroid as the rotation center and a preset angle as the rotation interval, and the reflection intensity of all point clouds in the reference window after each rotation is extracted.

[0190] The target rotation angle is determined based on the reflection intensity of all point clouds within the rotated reference window. :

[0191] ;

[0192] In the formula, Rotation angle The sum of the reflection intensities of all point clouds within the corresponding reference window;

[0193] Using the reference window corresponding to the target rotation angle as the target window, the target region image and the corresponding target region point cloud are extracted from the processed image and the processed point cloud, respectively, based on the target window.

[0194] The enhancement module 3 is used for:

[0195] The target region image is converted into a grayscale image, which is then segmented into several non-overlapping sub-images. A grayscale histogram is calculated for each sub-image, and a segmentation value is calculated based on the grayscale histogram. :

[0196] ;

[0197] In the formula, The total number of pixels in the sub-image. For gray levels, The dividing coefficient;

[0198] For each sub-image, the portion of the grayscale histogram that exceeds the segmentation value is cropped, and the cropped pixels are evenly redistributed to all grayscale levels to obtain a redistribution histogram.

[0199] The cumulative distribution function is calculated and mapped on the redistribution histogram to obtain a mapped image. All the mapped images are then fused using bilinear interpolation to obtain a fused image.

[0200] Construct several non-scale Gaussian kernels, and perform convolution processing on the grayscale image based on several Gaussian kernels of different scales to obtain several smooth images;

[0201] Extract the detail image of the smoothed image :

[0202] ; ;...; ;

[0203] In the formula, For the first A smooth image, It is a grayscale image. For the first A detailed image;

[0204] An enhanced image is determined based on the detailed image and the fused image. :

[0205] ;

[0206] In the formula, To merge images, To enhance the weight.

[0207] The detection module 4 is used for:

[0208] The enhanced image is input into a pre-trained first preset model, and the image editor in the first preset model is used to extract the features of the enhanced image and generate an initial segmentation mask.

[0209] For each connected region in the segmentation mask, extract its pixel grayscale set. Calculate the pixel grayscale set belonging to the first... Posterior probability of Gaussian components :

[0210] ;

[0211] ;

[0212] In the formula, The set of pixel grayscale values ​​belongs to the first... The posterior probability of each Gaussian component , For the first , The mixing weights of Gaussian components, , These are the first Gaussian density function and the second Gaussian density function, respectively. , The first The mean vector and covariance matrix of each Gaussian component. The dimension of the pixel grayscale set;

[0213] Update parameters based on the posterior probability:

[0214] ; ; ;

[0215] In the formula, This represents the total number of connected regions.

[0216] Repeat the posterior probability calculation and parameter update process until convergence to obtain a probability map of each pixel belonging to the crack. ;

[0217] For each pixel of the enhanced image Constructing feature fusion vectors The feature fusion vector includes the color space components of the enhanced image, texture response features, and the reflection intensity value, mean and variance of the local neighborhood reflection intensity of the point cloud of the target region.

[0218] Construct the objective function :

[0219] ;

[0220] ;

[0221] ;

[0222] In the formula, , These are unidimensional potential energy and dualdimensional potential energy, respectively. Represents pixels Labeled The probability, This indicates a tag compatibility function. , These are the first balancing weights and the second balancing weights, respectively. , Each pixel Color space components in an image, These are the bandwidth parameters for the first and second Gaussian kernels, respectively. It is a set of adjacent pixel pairs. A set of pixels;

[0223] Minimize the objective function to output a crack image.

[0224] The output module 5 is used for:

[0225] A parallel thinning algorithm is used to generate a crack skeleton for the crack image to obtain an initial skeleton. Points in the initial skeleton are identified, and it is determined whether the points in the initial skeleton meet the background judgment conditions. If the background judgment conditions are met, the points in the initial skeleton are used as background points. The judgment process is repeated until no new background points are generated, and the optimized skeleton is output.

[0226] Identify the first type of points, the second type of points, and the third type of points in the optimized skeleton. Starting from each first type of point, trace along the optimized skeleton until a second type of point or a third type of point is encountered. Record the chain code direction sequence and pixel step size of the corresponding tracing branch.

[0227] For each second-class point, check the branches it connects to. If the pixel step size of its branch is less than the step size threshold, then iteratively delete the current shortest score based on the chain code direction sequence until the pixel step size of all remaining branches is not less than the step size threshold or the point is no longer a second-class point, and output the crack skeleton.

[0228] Extract the main axis points of the crack skeleton, fit the main axis points to obtain an m-th degree polynomial curve, obtain the tangent slope of the m-th degree polynomial curve, and determine the final measurement direction angle based on the tangent slope.

[0229] A measurement line is defined for the skeleton points in the crack skeleton along the direction of the final measurement direction angle. The nearest edge points are searched on both sides of the measurement line on the crack image to obtain the edge point set.

[0230] Extreme points are filtered from the set of edge points according to preset rules to obtain the upper edge points. With the lower edge point :

[0231] The measurement width is determined based on the upper edge point and the lower edge point. :

[0232] ;

[0233] Determine the actual width of the crack Compared with the actual total length of the crack To obtain the specific parameters of the crack:

[0234] ;

[0235] ;

[0236] ; ;

[0237] In the formula, , These are the camera focal length and the object distance, respectively. These are the width and height of the image, respectively. These are the physical width and height of the sensor, respectively. These are the pixel physical width and height, respectively. For the final measurement of the direction angle, For the first With the The absolute value of the difference between the x-coordinates of each pixel. For the first With the The absolute value of the difference between the ordinates of each pixel.

[0238] In other embodiments of the present invention, the present invention provides the following technical solution: a computer, including a memory 102, a processor 101, and a computer program stored in the memory 102 and executable on the processor 101, wherein the processor 101 executes the computer program to implement the road surface crack detection method as described above.

[0239] Specifically, the processor 101 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of the present invention.

[0240] The memory 102 may include a large-capacity memory for data or instructions. For example, and not limitingly, the memory 102 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 102 may include removable or non-removable (or fixed) media. Where appropriate, the memory 102 may be internal or external to a data processing device. In a particular embodiment, the memory 102 is non-volatile memory. In a particular embodiment, the memory 102 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random Access Memory (FPMDRAM), Extended Data Out Dynamic Random Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0241] The memory 102 can be used to store or cache various data files that need to be processed and / or used for communication, as well as possible computer program instructions executed by the processor 101.

[0242] The processor 101 implements the above-mentioned road surface crack detection method by reading and executing computer program instructions stored in the memory 102.

[0243] In some embodiments, the computer may further include a communication interface 103 and a bus 100. For example, Figure 3 As shown, the processor 101, memory 102, and communication interface 103 are connected through bus 100 and complete communication with each other.

[0244] The communication interface 103 is used to enable communication between the various modules, devices, units, and / or equipment in the embodiments of the present invention. The communication interface 103 can also enable data communication with other components such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations.

[0245] Bus 100 includes hardware, software, or both, that couples components of a computer device together. Bus 100 includes, but is not limited to, at least one of the following: data bus, address bus, control bus, expansion bus, and local bus. For example, and not as a limitation, bus 100 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 100 may include one or more buses. Although specific buses are described and illustrated in the embodiments of the present invention, the present invention is contemplated by any suitable bus or interconnect.

[0246] The computer can execute the road crack detection method of the present invention based on the acquired road crack detection system, thereby realizing road crack detection.

[0247] In some further embodiments of the present invention, in conjunction with the above-described pavement crack detection method, the present invention provides the following technical solution: a storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the above-described pavement crack detection method.

[0248] Those skilled in the art will understand that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0249] More specific examples of readable media (a non-exhaustive list) include: electrical connections (electronic devices) with one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0250] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0251] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0252] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for detecting road surface cracks, characterized in that, include: Collect road surface images and lidar point cloud data of the target road surface, and preprocess the road surface images and lidar point cloud data respectively to obtain processed images and processed point clouds. Based on the processed image and the processed point cloud, the target region is extracted to obtain the target region image and the target region point cloud. The target region image is subjected to multi-scale enhancement processing to obtain an enhanced image; Crack detection is performed based on the enhanced image and the point cloud of the target region to output a crack image; The crack image is subjected to parameter identification and coordinate transformation to output specific crack parameters; The step of detecting cracks based on the enhanced image and the point cloud of the target region to output a crack image includes: The enhanced image is input into a pre-trained first preset model, and the image editor in the first preset model is used to extract the features of the enhanced image and generate an initial segmentation mask. The first preset model is the MobileSAM model, and the image editor is Tiny-ViT; For each connected region in the segmentation mask, extract its pixel grayscale set. Calculate the pixel grayscale set belonging to the first... Posterior probability of Gaussian components : ； ； In the formula, The set of pixel grayscale values ​​belongs to the first... The posterior probability of each Gaussian component , For the first , The mixing weights of Gaussian components, , These are the first Gaussian density function and the second Gaussian density function, respectively. , The first The mean vector and covariance matrix of each Gaussian component. The dimension of the pixel grayscale set; Update parameters based on the posterior probability: ； ； ； In the formula, This represents the total number of connected regions. Repeat the posterior probability calculation and parameter update process until convergence to obtain a probability map of each pixel belonging to the crack. ; For each pixel of the enhanced image Constructing feature fusion vectors The feature fusion vector includes the color space components of the enhanced image, texture response features, and the reflection intensity value, mean and variance of the local neighborhood reflection intensity of the point cloud of the target region. Construct the objective function : ； ； ； In the formula, , These are unidimensional potential energy and dualdimensional potential energy, respectively. Represents pixels Labeled The probability, This indicates a tag compatibility function. , These are the first balancing weights and the second balancing weights, respectively. , Each pixel Color space components in an image, These are the bandwidth parameters for the first and second Gaussian kernels, respectively. It is a set of adjacent pixel pairs. A set of pixels; Minimize the objective function to output a crack image; The steps of performing parameter identification and coordinate transformation on the crack image to output specific crack parameters include: A parallel thinning algorithm is used to generate a crack skeleton for the crack image to obtain an initial skeleton. Points in the initial skeleton are identified, and it is determined whether the points in the initial skeleton meet the background judgment conditions. If the background judgment conditions are met, the points in the initial skeleton are used as background points. The judgment process is repeated until no new background points are generated, and the optimized skeleton is output. Identify the first type of points, the second type of points, and the third type of points in the optimized skeleton. Starting from each first type of point, trace along the optimized skeleton until a second type of point or a third type of point is encountered. Record the chain code direction sequence and pixel step size of the corresponding tracing branch. For each second-class point, check the branches it connects to. If the pixel step size of its branch is less than the step size threshold, then iteratively delete the current shortest score based on the chain code direction sequence until the pixel step size of all remaining branches is not less than the step size threshold or the point is no longer a second-class point, and output the crack skeleton. Extract the main axis points of the crack skeleton, fit the main axis points to obtain an m-th degree polynomial curve, obtain the tangent slope of the m-th degree polynomial curve, and determine the final measurement direction angle based on the tangent slope. A measurement line is defined for the skeleton points in the crack skeleton along the direction of the final measurement direction angle. The nearest edge points are searched on both sides of the measurement line on the crack image to obtain the edge point set. Extreme points are filtered from the set of edge points according to preset rules to obtain the upper edge points. With the lower edge point : The measurement width is determined based on the upper edge point and the lower edge point. : ； Determine the actual width of the crack Compared with the actual total length of the crack To obtain the specific parameters of the crack: ； ； ； ； In the formula, , These are the camera focal length and the object distance, respectively. These are the width and height of the image, respectively. These are the physical width and height of the sensor, respectively. These are the pixel physical width and height, respectively. For the final measurement of the direction angle, For the first With the The absolute value of the difference between the x-coordinates of each pixel. For the first With the The absolute value of the difference between the ordinates of each pixel.

2. The method for detecting road surface cracks according to claim 1, characterized in that, The steps of preprocessing the road surface image and the lidar point cloud data respectively to obtain the processed image and the processed point cloud specifically include: The road surface image is subjected to image distortion correction to obtain a processed image. The lidar point cloud data is subjected to statistical filtering to remove outlier noise points. Then, the lidar point cloud data is transformed to the camera coordinate system using the calibrated extrinsic parameter matrix to obtain the processed point cloud.

3. The method for detecting road surface cracks according to claim 1, characterized in that, The step of extracting the target region based on the processed image and the processed point cloud to obtain the target region image and the target region point cloud includes: A rectangular search window is set in the processed point cloud, and the rectangular search window is moved longitudinally with a preset distance as the search step size. Calculate the sum of the reflection intensities of all point clouds within the rectangular search window after each movement, extract the sub-region image corresponding to the rectangular search window in the processed image, and calculate the gradient magnitude variance of the grayscale image corresponding to the sub-region image. The search value of each rectangular search window is determined based on the reflection intensity of all point clouds within the window and its variance relative to the gradient magnitude. : ； In the formula, To balance the weights, The first The reflection intensity and gradient magnitude variance corresponding to the rectangular search window after each movement; The rectangular search window corresponding to the maximum search value is recorded as the reference window. The processed point cloud within the reference window is clustered and the centroid position is extracted to obtain the target centroid. The reference window is rotated within a preset angle range with the target centroid as the rotation center and a preset angle as the rotation interval, and the reflection intensity of all point clouds in the reference window after each rotation is extracted. The target rotation angle is determined based on the reflection intensity of all point clouds within the rotated reference window. : ； In the formula, Rotation angle The sum of the reflection intensities of all point clouds within the corresponding reference window; Using the reference window corresponding to the target rotation angle as the target window, the target region image and the corresponding target region point cloud are extracted from the processed image and the processed point cloud, respectively, based on the target window.

4. The method for detecting road surface cracks according to claim 1, characterized in that, The step of performing multi-scale enhancement processing on the target region image to obtain an enhanced image includes: The target region image is converted into a grayscale image, which is then segmented into several non-overlapping sub-images. A grayscale histogram is calculated for each sub-image, and a segmentation value is calculated based on the grayscale histogram. : ； In the formula, The total number of pixels in the sub-image. For gray levels, The dividing coefficient; For each sub-image, the portion of the grayscale histogram that exceeds the segmentation value is cropped, and the cropped pixels are evenly redistributed to all grayscale levels to obtain a redistribution histogram. The cumulative distribution function is calculated and mapped on the redistribution histogram to obtain a mapped image. All the mapped images are then fused using bilinear interpolation to obtain a fused image. Construct several non-scale Gaussian kernels, and perform convolution processing on the grayscale image based on several Gaussian kernels of different scales to obtain several smooth images; Extract the detail image of the smoothed image : ； ；…； ； In the formula, For the first A smooth image, It is a grayscale image. For the first A detailed image; An enhanced image is determined based on the detailed image and the fused image. : ； In the formula, To merge images, To enhance the weight.

5. A road surface crack detection system, characterized in that, The system employs the pavement crack detection method as described in claim 1, and the system comprises: The acquisition module is used to acquire road surface images and lidar point cloud data of the target road surface, and to preprocess the road surface images and lidar point cloud data respectively to obtain processed images and processed point clouds. The extraction module is used to extract the target region based on the processed image and the processed point cloud to obtain the target region image and the target region point cloud. An enhancement module is used to perform multi-scale enhancement processing on the target region image to obtain an enhanced image; The detection module is used to detect cracks based on the enhanced image and the point cloud of the target region, so as to output a crack image; The output module is used to perform parameter recognition and coordinate transformation on the crack image to output specific crack parameters.

6. The pavement crack detection system according to claim 5, characterized in that, The acquisition module is specifically used for: The road surface image is subjected to image distortion correction to obtain a processed image. The lidar point cloud data is subjected to statistical filtering to remove outlier noise points. Then, the lidar point cloud data is transformed to the camera coordinate system using the calibrated extrinsic parameter matrix to obtain the processed point cloud.

7. A computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the road surface crack detection method as described in any one of claims 1 to 4.

8. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the pavement crack detection method as described in any one of claims 1 to 4.

Images