A pixel-level segmentation effect evaluation method based on road crack length and shape information

Abstract

The application discloses a pixel-level segmentation effect evaluation method based on road crack length and shape information, which takes a true value image as a benchmark, compares a crack orthographic image with the true value image, and judges whether the judgment effect of the crack orthographic image on the crack is accurate or not. The judgment of the method is started from the length accuracy of the detected road crack pixel-level segmentation and the fitting of the shape description, focuses on considering the length and shape indexes which are relatively important for the road crack, and can describe and quantitatively evaluate the spatial distribution of the length and shape indexes.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology and relates to a pixel-level segmentation effect evaluation method based on road crack length and morphology information. Background Technology

[0002] Pixel-level segmentation is one of the most fundamental techniques in image understanding and a key research focus in computer vision. It groups pixels from different semantic regions in an image, allowing for a more accurate description of the category or object of each pixel. This is crucial for understanding the content and details of an image. For example, pixel-level segmentation can quickly and accurately identify cracks and damage on roads, thus evaluating road health. Therefore, accurate pixel-level segmentation is paramount. However, current evaluation metrics largely follow traditional pixel-level segmentation standards such as accuracy, precision, and recall, which represent the point-to-surface relationship between correctly and incorrectly identified pixels. For linear targets like road cracks, continuing to use these metrics lacks rationality. Firstly, in road crack recognition tasks, the width scale changes very little compared to length, yet has a significant impact on traditional detection evaluation metrics. Furthermore, since the ground truth of pixel-level segmentation results in road crack detection tasks is often obtained through manual annotation, the annotation of road crack width can be significantly inaccurate due to the often small width of the crack. Therefore, using traditional pixel-level segmentation performance evaluation metrics to evaluate the pixel-level segmentation results of road cracks lacks objectivity and accuracy. Secondly, in road crack identification tasks, the purpose is often to assess road health or to track and repair road cracks; width has a relatively small impact on the task. For example, the "Highway Technical Condition Assessment Standard (JTG 5210—2018)" only analyzes the length of road cracks for automated identification, calculating the width as 0.2m. Therefore, using traditional pixel-level segmentation performance evaluation metrics to evaluate the pixel-level segmentation results of road cracks lacks applicability. Based on this, a more suitable evaluation metric for evaluating the pixel-level segmentation performance of road cracks is needed. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art and provide a pixel-level segmentation effect evaluation method based on road crack length and morphology information, so as to solve the problem that the evaluation indicators in the existing road crack pixel-level segmentation tasks lack objectivity, accuracy and rationality of application.

[0004] To achieve the above objectives, the present invention employs the following technical solution:

[0005] A pixel-level segmentation effect evaluation method based on road crack length and morphology information includes the following steps:

[0006] Step 1: Annotate the road crack image to obtain a pixel-level segmentation ground truth image of the road crack;

[0007] Step 2: Process the orthophoto image of the road crack using a segmentation algorithm to obtain a pixel-level segmented image of the road crack;

[0008] Step 3: Process the ground truth image and the pixel-level segmented image of the road crack using the skeletonization function to obtain the skeletonized ground truth image and the skeletonized crack image respectively.

[0009] The skeletonized ground truth image contains 0° pixel clusters, 30° pixel clusters, and 45° pixel clusters;

[0010] The skeletonized crack image contains 0° pixel clusters, 30° pixel clusters, and 45° pixel clusters;

[0011] Step 4: Search for the first crack region in the skeletonized crack image. The first crack region includes all cracks in the skeletonized crack image and is a rectangular region.

[0012] Step 5: Divide the first crack region into M equal-area partitions, map the first crack region and the M partitions into a skeletonized ground truth image, and obtain the second crack region in the skeletonized ground truth image. The second crack region is also divided into M partitions.

[0013] Step 6: Using the convolution kernels corresponding to the 0° pixel cluster, 30° pixel cluster, and 45° pixel cluster, traverse each partition of the first crack region to obtain the number of occurrences of each pixel cluster in each partition; using the convolution kernels corresponding to the 0° pixel cluster, 30° pixel cluster, and 45° pixel cluster, traverse each partition of the second crack region to obtain the number of occurrences of each pixel cluster in each partition; calculate the sum of errors in the number of occurrences of each pixel cluster in the same partitions of the first and second crack regions.

[0014] Step 7: Sum all the errors to obtain the evaluation index for the pixel-level segmentation effect of road cracks.

[0015] A further improvement of the present invention is that:

[0016] Preferably, in step 1, the annotation software is Labelme, CVAT, or VOTT.

[0017] Preferably, both the ground truth image of the road crack pixel-level segmentation and the orthophoto image of the road crack are images containing only pixel values ​​0 and 1; pixel value 0 is the background and pixel value 1 is the crack area.

[0018] Preferably, in step 3, there are two types of 0° pixel clusters, five types of 30° pixel clusters, and five types of 45° pixel clusters.

[0019] Preferably, in step 4, each side of the first crack region has the outermost end point of the crack in the direction of the edge; each side of the second crack region has the outermost end point of the crack in the direction of the edge.

[0020] Preferably, in step 4, the vertical center line and the horizontal center line of the first crack region are divided to obtain four uniformly sized partitions; the vertical center line and the horizontal center line of the second crack region are divided to obtain four uniformly sized partitions.

[0021] Preferably, in step 6, the convolution kernel corresponding to the 0° pixel cluster is With a bias value of 0 and a stride of 1, traverse each partition in the first crack region and each partition in the second crack region. If a convolution result equals 3 once, it is considered that a 0° pixel cluster has appeared. The number of 0° pixel clusters appearing in the first crack region and the second crack region are obtained respectively.

[0022] Preferably, in step 6, the convolution kernel corresponding to the 30° pixel cluster is With a bias value of 0 and a stride of 1, traverse each partition in the first crack region and each partition in the second crack region. If a convolution result equals 2 once, it is considered that a 30° pixel cluster has occurred. The number of occurrences of 30° pixel clusters in the first crack region and the second crack region are obtained respectively.

[0023] Preferably, in step 6, the convolution kernel corresponding to the 45° pixel cluster is With a bias value of 0 and a stride of 1, traverse each partition in the first crack region and each partition in the second crack region. If a convolution result equals 3 once, it is considered that a 45° pixel cluster has occurred. The number of occurrences of 45° pixel clusters in the first crack region and the second crack region are obtained respectively.

[0024] Preferably, in step 6, the sum of errors is:

[0025]

[0026] Among them, D i T represents the number of pixel clusters appearing in the first crack region. i This represents the number of pixel clusters appearing in the second crack region.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] This invention discloses a pixel-level segmentation performance evaluation index based on road crack length and morphology information. The method uses a ground truth image as a benchmark, comparing the crack orthophoto image with the ground truth image to determine the accuracy of crack identification in the orthophoto image. This method assesses the accuracy of the length segmentation and the fit between the detected road crack's shape description, focusing on the relatively important length and shape indicators of road cracks and providing a description and quantitative evaluation of their spatial distribution. This will be of great value in crack tracing, tracking, and road surface health assessment. Regarding evaluation accuracy, a block-based approach is used to mitigate and compensate for errors, which is of significant importance in the application of pixel-level road crack segmentation performance evaluation. Attached Figure Description

[0029] Figure 1 Flowchart of the method proposed in this invention;

[0030] Figure 2 The present invention proposes a total of 12 pixel clusters in 3 categories: 0°, 30°, and 45°.

[0031] Figure 3 A schematic diagram of pixel-level image segmentation proposed in this invention. Detailed Implementation

[0032] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0033] See Figure 1 This invention discloses a pixel-level segmentation effect evaluation method based on road crack length and morphology information, specifically including the following steps:

[0034] Step 1: Manually annotate the crack image to obtain the true image of road crack pixel-level segmentation. This image is considered the correct result of crack segmentation and is used as the true image of road crack pixel-level segmentation.

[0035] Preferably, the annotation of the crack image can be accomplished using various software such as Labelme, CVAT, or VOTT.

[0036] Step 2: Process the orthophoto image of the road crack using a road crack pixel-level segmentation algorithm to obtain a road crack pixel-level segmentation image;

[0037] The ground truth image and the pixel-level segmentation image of the road crack obtained in steps 1 and 2 contain only two types of pixels with pixel values ​​of 0 and 1. Pixels with a pixel value of 0 represent the background, which is the non-crack area; and pixels with a pixel value of 1 represent the crack area.

[0038] Step 3: Use the skeletonization function to process the pixel-level segmentation ground truth image of the road crack to obtain the skeletonization information of the real crack and the skeletonized ground truth image; process the pixel-level segmentation image of the road crack using the skeletonization function to obtain the skeletonized crack image; the specific skeletonization function is the skeletonization function skeletonization function in the skimage library in Python.

[0039] Specifically, the obtained skeletonized crack image and skeletonized ground truth image should only contain the following: Figure 2 The pixel clusters shown are divided into three categories: 0° pixel clusters, 30° pixel clusters, and 45° pixel clusters. There are two categories of 0° pixel clusters, five categories of 30° pixel clusters, and five categories of 45° pixel clusters, for a total of 12 types of pixel clusters, in order to perform crack length statistics.

[0040] Specifically, in a 0° pixel cluster, the value of the three consecutive horizontal or vertical middle pixel blocks is 1, and the value of the remaining pixel blocks is 0; in a 30° pixel cluster, the value of the two consecutive horizontal or vertical middle pixel blocks is 1, and the pixel block next to the pixel block with a value of 0 in the horizontal or vertical middle pixel block is 1; in a 45° pixel cluster, the value of the middle pixel block is 1, the value of the two end pixel blocks in a row or column on the edge is 1, or the value of the three pixel blocks on the diagonal is 1.

[0041] Step 4: Draw the first crack region in the skeletonized crack image. This crack region encompasses all cracks in the image, and each edge of the first crack region has a crack endpoint, which is the outermost endpoint of all cracks in that direction. This ensures that there are no other crack points on the crack image besides the crack region. The lower left corner of the entire skeletonized crack image is the origin of the coordinate system. The coordinates of the lower left and upper right corners of the first crack region are denoted as (X...). min ,Y min ),(X max ,Y max ), making the crack point (X) min ,Y min ),(X max ,Y max Points X and Y are interior points of a rectangular region formed by two points as diagonal points, where (X) is an interior point of the region formed by two points as diagonal points. i ,Y j ) indicates such as Figure 3 The pixel in the i-th column and j-th row of the Cartesian coordinate system is shown.

[0042] The set of planar points in the first crack region is R. 2 ={(x,y)|x∈[X]} min ,X max ],y∈[Y min ,Ymax ]}, so that the crack point is denoted as but

[0043] Step 5: Divide the first crack region in the skeletonized crack image into vertical and horizontal center lines. Each crack region is then evenly divided into four partitions: region a, region b, region c, and region d. The coordinates of the midpoint are... The coordinates of the cracks in each crack region are shown in the following formula:

[0044]

[0045]

[0046]

[0047]

[0048] Based on the first crack region and its corresponding four partitions, the second crack region is divided in the ground truth image of road crack pixel-level segmentation. The second crack region is further divided into four parts, namely region a', region b', region c' and region d'.

[0049] Step 6: Count the number of crack pixel clusters in the four regions of the two images respectively, and calculate the sum of errors in the number of each type of crack pixel cluster in the region. The sum of errors in the four regions are denoted as E1, E2, E3 and E4 respectively.

[0050] The crack length is calculated based on the crack length extraction algorithm, specifically as follows:

[0051] Step 6.1, using a single-channel single convolution kernel With a bias value of 0 and a stride of 1, no padding operation is performed. The road crack pixel-level segmentation image is traversed. If a convolution result equals 3 once, it is considered a 0° pixel cluster. After traversing the entire image, the cumulative number of occurrences of 3 is recorded as D0. The same method is used to process the ground truth image of the road crack pixel-level segmentation to obtain T0.

[0052] Step 6.2, using a single-channel single convolution kernel With a bias value of 0 and a stride of 1, no padding is performed. The image is segmented at the pixel level for road cracks. If a convolution result equals 2 occurs once, it is considered a 30° pixel cluster. After traversing the entire image, the cumulative number of occurrences of 2 is recorded as D. 30 The same method was used to process the ground truth image of road crack pixel-level segmentation to obtain T. 30 ;

[0053] Step 6.3, using a single-channel single convolution kernel With a bias value of 0 and a stride of 1, no padding is performed. The image is segmented at the pixel level for road cracks. If a convolution result equals 3 occurs once, it is considered a 45° pixel cluster. After traversing the entire image, the cumulative number of occurrences of 3 is recorded as D. 45 The same method was used to process the ground truth image of road crack pixel-level segmentation to obtain T. 45 ;

[0054] Step 6.4, calculate the sum of errors in the number of each type of crack pixel cluster in each region:

[0055]

[0056] In step 5, both the pixel-level segmented image of the road crack and the corresponding labeled pixel-level segmented ground truth image of the road crack are divided into four parts. This allows for segmentation evaluation of the segmentation effect in blocks, which to some extent alleviates the compensation of errors and has a positive effect on the evaluation accuracy. When the accuracy requirement is higher, the number of segmented regions can be further increased.

[0057] Step 7: Obtain the evaluation index for the pixel-level segmentation effect of road cracks:

[0058]

[0059] This evaluation metric can be used to determine the segmentation effect of pixel-level segmentation images of road cracks.

[0060] This invention provides a pixel-level segmentation performance evaluation index based on road crack length and morphology information, which can be used to evaluate the segmentation effect of road crack pixel-level segmentation images. Compared with existing technologies, the method of this invention has significant advantages: in terms of evaluation focus, it emphasizes the relatively important length and shape indicators of road cracks and can describe and quantitatively evaluate their spatial distribution, which will play a significant role in crack tracing, tracking, and road surface health assessment; in terms of evaluation accuracy, it uses a block-based method to mitigate error accumulation and compensation, which is of great significance in the application of road crack pixel-level segmentation performance evaluation.

[0061] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A pixel-level segmentation effect evaluation method based on road crack length and morphology information, characterized in that, Includes the following steps: Step 1: Annotate the road crack image to obtain a pixel-level segmentation ground truth image of the road crack; Step 2: Process the orthophoto image of the road crack using a segmentation algorithm to obtain a pixel-level segmented image of the road crack; Step 3: Process the ground truth image and the pixel-level segmented image of the road crack using the skeletonization function to obtain the skeletonized ground truth image and the skeletonized crack image respectively. The skeletonized ground truth image contains 0° pixel clusters, 30° pixel clusters, and 45° pixel clusters; The skeletonized crack image contains 0° pixel clusters, 30° pixel clusters, and 45° pixel clusters; Step 4: Search for the first crack region in the skeletonized crack image. The first crack region includes all cracks in the skeletonized crack image and is a rectangular region. Step 5: Divide the first crack region into M equal-area partitions, map the first crack region and the M partitions into a skeletonized ground truth image, and obtain the second crack region in the skeletonized ground truth image. The second crack region is also divided into M partitions. Step 6: Use the convolution kernels corresponding to the 0° pixel cluster, 30° pixel cluster and 45° pixel cluster to traverse each partition in the first crack region to obtain the number of occurrences of each pixel cluster in each partition. Using the convolution kernels corresponding to the 0° pixel cluster, 30° pixel cluster and 45° pixel cluster respectively, we traverse each partition in the second slit to obtain the number of times each pixel cluster appears in each partition. Calculate the sum of errors in the number of occurrences of each pixel cluster in the same partition of the first and second crack regions; In step 6, the convolution kernel corresponding to the 0° pixel cluster is With a bias value of 0 and a stride of 1, we traverse each partition in the first crack region and each partition in the second crack region. If a convolution result equals 3 once, we consider that a 0° pixel cluster has appeared once. We obtain the number of 0° pixel clusters in the first crack region and the second crack region respectively. In step 6, the convolution kernel corresponding to the 30° pixel cluster is With a bias value of 0 and a stride of 1, we traverse each partition in the first crack region and each partition in the second crack region. If a convolution result equals 2 once, we consider that a 30° pixel cluster has appeared. We obtain the number of 30° pixel clusters appearing in the first crack region and the second crack region, respectively. In step 6, the convolution kernel corresponding to the 45° pixel cluster is With a bias value of 0 and a stride of 1, we traverse each partition in the first crack region and each partition in the second crack region. If a convolution result equals 3 once, we consider that a 45° pixel cluster has appeared. We obtain the number of 45° pixel clusters in the first crack region and the second crack region respectively. Step 7: Sum all the errors to obtain the evaluation index for the pixel-level segmentation effect of road cracks. 2.The pixel-level segmentation evaluation method based on road crack length and shape information according to claim 1, wherein, In step 1, the labeling software is Labelme, CVAT, or VOTT.

3. The pixel-level segmentation effect evaluation method based on road crack length and morphology information according to claim 1, characterized in that, Both the ground truth image of the road crack pixel-level segmentation and the orthophoto image of the road crack contain only pixel values ​​0 and 1; pixel value 0 is the background and pixel value 1 is the crack area.

4. The pixel-level segmentation effect evaluation method based on road crack length and morphology information according to claim 1, characterized in that, In step 3, there are two types of 0° pixel clusters, five types of 30° pixel clusters, and five types of 45° pixel clusters.

5. The pixel-level segmentation effect evaluation method based on road crack length and morphology information according to claim 1, characterized in that, In step 4, each side of the first crack region has the outermost end point of the crack in the direction of the edge; each side of the second crack region has the outermost end point of the crack in the direction of the edge.

6. The pixel-level segmentation effectiveness evaluation method based on road crack length and morphology information according to claim 1, characterized in that, In step 4, the vertical and horizontal center lines of the first crack region are divided to obtain four uniformly sized partitions; the vertical and horizontal center lines of the second crack region are also divided to obtain four uniformly sized partitions.

7. The pixel-level segmentation effectiveness evaluation method based on road crack length and morphology information according to claim 1, characterized in that, In step 6, the sum of errors is: in, The number of pixel clusters appearing in the first crack region. This represents the number of pixel clusters appearing in the second crack region.

Images