A crack segmentation method based on a Hessian matrix and a target recognition network

Abstract

The application provides a crack segmentation method based on a Hessian matrix and a target recognition network, comprising the following steps: S1, acquiring a color pavement image and pre-processing the acquired color pavement image; S2, performing crack region detection on the pre-processed image by using a target detection method; S3, generating a gradient image based on a Hessian matrix Gaussian filter bank according to the image after the crack region detection; and S4, performing selective segmentation on the generated gradient image based on a threshold of a pavement texture template. The crack segmentation method based on the Hessian matrix and the target recognition network can enhance the crack region and part of the texture region, and can largely reduce texture false detection by using the texture template mean value as the threshold, and the edge is closer to the real edge of the crack.

Description

Technical Field

[0001] This invention relates to the field of auxiliary road daily inspection and maintenance technology, and more particularly to a crack segmentation method based on Hessian matrix and target recognition network. Background Technology

[0002] Currently, methods for pavement crack segmentation are mainly divided into deep learning methods, machine learning methods, and traditional methods. Deep learning methods primarily include pavement crack segmentation methods based on deep learning networks. Previous crack segmentation methods mainly employed traditional image processing techniques, such as thresholding, edge detection, and region growing. The main ideas of each method are as follows:

[0003] Thresholding segmentation: Thresholding segmentation methods use various image processing techniques to assign appropriate pixel intensity thresholds to divide image pixels into two categories, thereby separating the crack target from the background.

[0004] Edge detection: Edge detection methods use operators such as Sobel, Prewitt, and Canny to detect the edges of road cracks, thereby separating the crack target from the background.

[0005] Region growing: The region growing method describes specific information inside the crack by grouping pixels with similar features into a region, thereby separating the crack target from the background.

[0006] Machine learning methods are divided into two main categories: unsupervised learning and supervised learning.

[0007] Unsupervised learning: In unsupervised models based on sample learning, the training samples are unlabeled and have no explicit output. The computer needs to learn the similarities between samples and classify them on its own.

[0008] Supervised learning includes logistic regression, Naive Bayes, support vector machines, artificial neural networks, and random forests. It involves making probabilistic predictions on the data using modeling or non-modeling methods, then making direct or indirect probability judgments, and finally segmenting the gaps.

[0009] Problems with existing crack segmentation algorithms:

[0010] (1) Traditional image processing methods based on threshold segmentation, edge detection, and region growing are all based on the important assumption that the intensity of crack pixels is darker than the background and is usually continuous. This makes these methods difficult to use effectively in environments with complex background noise. They do not make discrimination based on crack features.

[0011] (2) Although machine learning-based methods can learn crack detection from training samples, their detection performance is not superior when dealing with cracks with uneven gray levels or complex topologies. The failure of these methods can be attributed to the lack of robust feature representation due to segmentation by probabilistic modeling, and the neglect of the interdependencies between cracks by thresholding pixel by pixel.

[0012] (3) In deep learning methods, although the crack segmentation method based on convolutional neural networks makes a judgment through feature extraction, the network's receptive field is small because it only performs convolution operations in local regions and lacks the ability to perceive the global context. In addition, it is highly dependent on the training data. Summary of the Invention

[0013] To address the aforementioned technical problems, this invention provides a crack segmentation method based on a Hessian matrix and a target recognition network. The method comprises a Transformer target recognition network RTDETR and a Gaussian filter bank based on a Hessian matrix. It can be used for image processing from highway data acquisition vehicles, assisting in routine road inspection and maintenance.

[0014] The technical means employed in this invention are as follows:

[0015] A crack segmentation method based on Hessian matrix and target recognition network includes:

[0016] S1. Acquire a colored road surface image and preprocess the acquired colored road surface image;

[0017] S2. Crack region detection is performed on the preprocessed image using target detection methods.

[0018] S3. Based on the image after crack region detection, generate a gradient map using a Gaussian filter bank based on the Hessian matrix.

[0019] S4. Based on the threshold of the road surface texture template, selectively segment the generated gradient map.

[0020] Furthermore, step S1 specifically includes:

[0021] S11. Convert the colored road surface image into a grayscale image P1;

[0022] S12. Perform shadow and raster removal on the grayscale image P1 to obtain the processed image P2.

[0023] Furthermore, step S2 specifically includes:

[0024] S21. Using the target detection method, crack areas are detected in the preprocessed image P2 to obtain the image P3 with cracks and the road surface texture template.

[0025] S22. Denote the obtained road surface texture template as B. i , i = 1, 2, 3...

[0026] Furthermore, step S3 specifically includes:

[0027] S31. Using a master-slave Gaussian filter bank, the cracked image P3 is enhanced to obtain the gradient map P4. The master filter is represented as follows:

[0028] r m (p, θ, σ) = g xx cos 2 θ+g yy sin 2 θ+g xy sin2θ

[0029] Among them, gxx, g yy g xy σ is the second-order partial derivative of the Gaussian convolution kernel, σ is the spatial scale factor of the Gaussian filter, θ is the direction of the main filter, and p = (x, y) is the coordinates of a point on the image.

[0030] S32. A multi-scale fusion method is used to enhance cracks at different scales. Specifically, the maximum response value of different scale factors is selected as the response at that point. The response is maximized when σ matches the crack width at that point. I(p) is the input two-dimensional grayscale image of the road surface, where S = {σ1, ..., σ...} n In the spatial scale, the obtained crack response map R m (p, θ), the calculation formula is as follows:

[0031]

[0032] S33. To improve the response intensity at the crack bifurcation point, the response of the crack region with a larger response in the adjacent area is used as the response, and a forward filter r is introduced. f and backward filter r b The calculation formula for collaborative detection is as follows:

[0033] r f (p, σ, ω1) = r d (xd cos(θ+ω1), y+d sin(θ+ω1))

[0034] r b (p, σ, ω2) = r d(x+d cos(θ+ω2), yd sin(θ+ω2))

[0035] Where ω1 and ω2 are angular offset coefficients, and d is the distance offset coefficient;

[0036] S34. Calculate the forward filter r f and backward filter r b The response is calculated using the following formula:

[0037]

[0038]

[0039] S35. In D = {d1, ..., d...} n In the offset scale space of}, calculate the response map R of the filter bank in the θ direction. h (p, θ), the calculation formula is as follows:

[0040]

[0041] S36. In ζ = {θ1, ..., θ} n In the direction space, calculate the final Gaussian filter bank response, that is, the maximum value R of the response at each direction. r (p), the calculation formula is as follows:

[0042]

[0043] The response result is the pixel value of each point in the generated gradient map P4.

[0044] Furthermore, step S4 specifically includes:

[0045] S41. Create masks J0 and J1 of the same size as the original image, and then use the bounding box B from the target detection network. i P4 is divided into a road surface texture gradient map J0 and a crack area gradient map J1.

[0046] S42. Calculate the mean gradient value of the road surface texture gradient map J0, that is, the response value of the road surface texture under the action of the Gaussian filter bank based on the Hessian matrix.

[0047] S43. Calculate the OTSU threshold of the gradient map J1 of the crack region, which is the maximum inter-class difference between the crack and the pavement in the crack region.

[0048] S44. Take the larger of the average gradient value and the OTSU threshold as the threshold, and binarize the gradient map P4 to obtain the final result P5.

[0049] Compared with the prior art, the present invention has the following advantages:

[0050] The crack segmentation method based on Hessian matrix and target recognition network provided by this invention can enhance crack regions and some texture regions. By using the mean of texture template as a threshold, it can greatly reduce texture false detections while retaining most of the crack regions, and the edges are closer to the real crack edges.

[0051] Based on the above reasons, this invention can be widely applied in fields such as crack segmentation. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a flowchart of the method of the present invention.

[0054] Figure 2 The obtained color road surface image is provided for an embodiment of the present invention.

[0055] Figure 3 The Gaussian filter bank result diagram and the defect boundary box are provided for the embodiments of the present invention.

[0056] Figure 4 The template mask provided in the embodiments of the present invention.

[0057] Figure 5 The extracted texture template region is provided in the embodiments of the present invention.

[0058] Figure 6 The OTSU threshold segmentation result is provided in the embodiment of the present invention.

[0059] Figure 7 The results of texture template extraction threshold segmentation provided in the embodiments of the present invention.

[0060] Figure 8 The diagram shows the effect obtained by using the method of the present invention in an embodiment of the present invention. Detailed Implementation

[0061] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0062] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0063] like Figure 1 As shown, this invention provides a crack segmentation method based on Hessian matrix and target recognition network, including:

[0064] S1. Acquire a color road surface image and preprocess the acquired color road surface image; such as... Figure 2 The image shown is a captured color road surface image.

[0065] S2. Crack region detection is performed on the preprocessed image using target detection methods.

[0066] S3. Based on the image after crack region detection, generate a gradient map using a Gaussian filter bank based on the Hessian matrix.

[0067] S4. Based on the threshold of the road surface texture template, selectively segment the generated gradient map.

[0068] In a specific implementation, as a preferred embodiment of the present invention, step S1 specifically includes:

[0069] S11. Convert the colored road surface image into a grayscale image P1;

[0070] S12. Perform shadow and raster removal on the grayscale image P1 to obtain the processed image P2.

[0071] In a specific implementation, as a preferred embodiment of the present invention, step S2 specifically includes:

[0072] S21. Using the target detection method, crack areas are detected in the preprocessed image P2 to obtain the image P3 with cracks and the road surface texture template.

[0073] S22. Denote the obtained road surface texture template as B. i ,i=1,2,3.... like Figure 4 , 5 As shown, these are the template mask and the road surface texture template, respectively.

[0074] In a specific implementation, as a preferred embodiment of the present invention, step S3 specifically includes:

[0075] S31. Using a master-slave Gaussian filter bank, the cracked image P3 is enhanced to obtain the gradient map P4. The master filter is represented as follows:

[0076] r m (p, θ, σ) = g xx cos 2 θ+g yy sin 2 θ+g xy sin2θ

[0077] Among them, gxx, g yy g xy σ is the second-order partial derivative of the Gaussian convolution kernel, σ is the spatial scale factor of the Gaussian filter, θ is the direction of the main filter, and p = (x, y) is the coordinates of a point on the image.

[0078] S32. A multi-scale fusion method is used to enhance cracks at different scales. Specifically, the maximum response value of different scale factors is selected as the response at that point. The response is maximized when σ matches the crack width at that point. I(p) is the input two-dimensional grayscale image of the road surface, where S = {σ1, ..., σ...} n In the spatial scale, the obtained crack response map R m (p, θ), the calculation formula is as follows:

[0079]

[0080] S33. To improve the response intensity at the crack bifurcation point, the response of the crack region with a larger response in the adjacent area is used as the response, and a forward filter r is introduced. f and backward filter r b The calculation formula for collaborative detection is as follows:

[0081] r f (p, σ, ω1) = r d(xd cos(θ+ω1), y+d sin(θ+ω1))

[0082] r b (p, σ, ω2) = r d (x+d cos(θ+ω2), yd sin(θ+ω2))

[0083] Where ω1 and ω2 are angular offset coefficients, and d is the distance offset coefficient;

[0084] S34. Calculate the forward filter r f and backward filter r b The response is calculated using the following formula:

[0085]

[0086]

[0087] S35. In D = {d1, ..., d...} n In the offset scale space of}, calculate the response map R of the filter bank in the θ direction. h (p, θ), the calculation formula is as follows:

[0088]

[0089] S36. In ζ = {θ1, ..., θ} n In the direction space, calculate the final Gaussian filter bank response, that is, the maximum value R of the response at each direction. r (p), the calculation formula is as follows:

[0090]

[0091] The response result is the pixel value of each point in the generated gradient map P4. For example... Figure 3 As shown, the Gaussian filter bank results and the disease boundary box are displayed. It can be seen that the crack area and some texture areas have been enhanced. The boundary box is the crack boundary box detected by the disease target detection network.

[0092] In a specific implementation, as a preferred embodiment of the present invention, step S4 specifically includes:

[0093] S41. Create masks J0 and J1 of the same size as the original image, and then use the bounding box B from the target detection network. i P4 is divided into a road surface texture gradient map J0 and a crack area gradient map J1.

[0094] S42. Calculate the mean gradient value of the road surface texture gradient map J0, that is, the response value of the road surface texture under the action of the Gaussian filter bank based on the Hessian matrix.

[0095] S43. Calculate the OTSU threshold of the gradient map J1 of the crack region, which is the maximum inter-class difference between the crack and the pavement in the crack region; such as Figure 6 The image shows the OTSU threshold segmentation results.

[0096] S44. Take the larger of the mean gradient value and the OTSU threshold as the threshold, and binarize the gradient map P4 to obtain the final result P5. For example... Figure 7 The image shows the threshold segmentation result extracted from the texture template. Figure 8 The image shown is an effect obtained using the method of this invention. It can be seen that the crack area and some textured areas are enhanced. Using the mean of the texture template as a threshold can significantly reduce false texture detections while preserving most of the crack area, and the edges are closer to the true crack edges.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A crack segmentation method based on Hessian matrix and target recognition network, characterized in that, include: S1. Acquire a colored road surface image and preprocess the acquired colored road surface image; S2. Crack region detection is performed on the preprocessed image using target detection methods. S3. Based on the image after crack region detection, generate a gradient map using a Gaussian filter bank based on the Hessian matrix, specifically including: S31. Use a master-slave Gaussian filter bank to process images with cracks. Enhancement of the crack region was performed to obtain the gradient map. The main filter is represented as follows: in, , , For the second-order partial derivative of the Gaussian convolution kernel, The spatial scale factor of the Gaussian filter. In the direction of the main filter, The coordinates of a point on the image; S32. A multi-scale fusion method is used to enhance cracks at different scales, that is, the maximum value of the factor response at different scales is selected as the response at this point. The response is greatest when it matches the crack width at this point. Given a 2D grayscale image of the road surface as input, Crack response map obtained at a spatial scale The calculation formula is as follows: S33. To improve the response intensity at the crack bifurcation point, the response of the crack region with a larger response in the adjacent area is used as the response, and a forward filter is introduced. and backward filter The calculation formula for collaborative detection is as follows: in and This is the angular offset coefficient. This is the distance offset coefficient; S34. Calculate the forward filter. and backward filter The response is calculated using the following formula: S35, in In the offset scale space, the calculation is performed in Response diagram of directional filter bank The calculation formula is as follows: S36, In In the direction space, calculate the final Gaussian filter bank response, that is, the maximum value of the response at each direction. The calculation formula is as follows: The response result is the generated gradient map. Pixel value per point; S4. Based on the threshold of the road surface texture template, perform selective segmentation on the generated gradient map, specifically including: S41. Create a mask of the same size as the original image. J 0 and J 1. Based on the bounding box results from the object detection network Will Divided into road surface texture gradient maps J 0 and gradient map of crack region J 1; S42. Calculate the road surface texture gradient map J The mean gradient value of 0 is the response value of the road surface texture under the action of a Gaussian filter bank based on the Hessian matrix. S43. Calculate the gradient map of the crack region. J The OTSU threshold of 1 is the maximum inter-class difference between the crack and the pavement in the crack region; S44. Take the larger of the average gradient value and the OTSU threshold as the threshold for the gradient map. Binarization is performed to obtain the final result. .

2. The crack segmentation method based on Hessian matrix and target recognition network according to claim 1, characterized in that, Step S1 specifically includes: S11. Convert the colored road surface image to a grayscale image. ; S12, For grayscale images Shadow and raster removal are performed to obtain the processed image. .

3. The crack segmentation method based on Hessian matrix and target recognition network according to claim 1, characterized in that, Step S2 specifically includes: S21. Using object detection methods, the preprocessed image... Crack detection was performed, and images showing cracks were obtained. and road surface texture templates; S22. Denote the obtained road surface texture template as... , .

Images