Interpretable classification and measurement algorithm for multiple types of concrete crack defects

Abstract

An interpretable classification and measurement algorithm for multiple types of bridge crack defects belongs to the field of bridge structure detection technology. It aims to solve the problems of single crack type in public datasets, difficulty in identifying complex cracks, and inaccurate classification in existing bridge crack detection, especially the difficulties in distinguishing between cracks and non-cracks and classifying multiple types of mixed cracks. The technical solution includes: using a deep learning semantic segmentation network model to segment bridge crack images; using traditional image processing technology to distinguish between cracks and non-cracks; further subdividing various types of cracks in non-cracks; and using a custom seed point growth method to measure the length and average width of conventional cracks. The present invention improves the accuracy of crack identification, realizes the effective classification of complex cracks, and provides a scientific basis for the health monitoring and maintenance of bridge structures. Through this algorithm, large amounts of image data can be automatically processed, crack information can be updated in real time, and the safety and stability of bridge structures can be ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of bridge structure health monitoring, and in particular to an interpretable classification and measurement algorithm for multiple types of concrete crack diseases. Background Art

[0002] As critical transportation infrastructure, the structural health and safety of bridges are directly linked to smooth transportation and the safety of people's lives and property. In my country, bridges are numerous, often exposed to complex and changing environmental conditions. Therefore, regular inspection and maintenance are crucial for improving bridge health and ensuring structural safety. Among bridge defects, concrete cracks are the most common. These cracks not only indicate structural stress concentration or material aging, but if allowed to continue to expand, they can also lead to structural damage, exposing and corroding internal steel reinforcement, reducing bridge stability and shortening its service life.

[0003] Traditional bridge structure inspections rely on experienced inspectors manually marking cracks and visually assessing their severity to facilitate subsequent maintenance. However, this method is not only time-consuming and labor-intensive, but also carries significant safety risks. Furthermore, test results are susceptible to subjective influences from inspectors, making it difficult to ensure accurate and consistent results.

[0004] With the advancement of science and technology, especially the emergence of bridge surface image acquisition equipment such as drones and bridge inspection vehicles, the methods of bridge appearance inspection have undergone a revolutionary change. These devices have enabled bridge appearance inspection to be moved from outdoor to indoor locations, shifting from physical inspection to image analysis, significantly improving inspection efficiency and safety. Based on this, deep learning methods, due to their powerful feature extraction and classification capabilities, have been widely used in bridge crack detection. Compared with traditional manual inspection methods, deep learning methods demonstrate significant advantages in robustness and universality. They can handle crack images of diverse morphologies and complex backgrounds, providing strong support for automated bridge surface inspection.

[0005] However, despite significant progress in bridge crack detection, deep learning methods still face numerous challenges. For one thing, crack images in current public datasets primarily focus on simple, single horizontal and vertical cracks, lacking effective coverage of complex and diverse cracks, particularly the distinction between chapped and non-chapped cracks. This dataset limitation limits the accuracy of deep learning models in identifying and classifying actual bridge crack images, particularly when faced with complex and varied crack morphologies. Furthermore, for traditional computer vision methods, the complexity of bridge crack images and the various noise interferences present in the shooting environment make it difficult to manually design effective image features to address these challenges, resulting in poor crack classification results.

[0006] Furthermore, existing research has primarily focused on detecting single, simple cracks, whereas actual bridge cracks are far more complex. A single image often contains multiple crack types, such as crazing cracks, conventional cracks, and a mixture of these types. These different crack types vary in morphology, size, and distribution, placing higher demands on crack detection, classification, and measurement. However, research and publications addressing the presence of multiple crack types in a single image remain scarce, and effective solutions are lacking.

[0007] For example, CN113506281B discloses a bridge crack detection method based on a deep learning framework. This method achieves automatic detection of bridge cracks through steps such as acquiring bridge images, preprocessing, and model training. However, although this technical solution mentions preprocessing the original bridge crack images to obtain an image dataset, it does not describe in detail the dataset construction process, especially how to ensure the diversity and representativeness of the dataset. Although it describes the structure of the segmentation model, it does not explain in depth the functions of the various parts of the model and how they work together to achieve crack detection. In addition, there is a lack of detailed explanation of the hyperparameter selection (such as learning rate, optimization algorithm, etc.) during the model training process. Although its deep learning model can automatically detect cracks, the model's decision-making process lacks explainability, making it difficult for users to understand why the model makes specific judgments.

[0008] For example, CN116297472A discloses a deep learning-based drone bridge crack detection method and system, which combines drone technology and deep learning algorithms to achieve accurate detection of bridge cracks. However, there are also some technical defects and shortcomings. 1) The system is complex. The system includes multiple parts such as acquisition module, 3D modeling module, and path planning module, which makes the entire system relatively complex. This not only increases the difficulty of system implementation, but may also affect the stability and reliability of the system; 2) The positioning accuracy is not high: Although the use of UWB auxiliary positioning module to improve the positioning accuracy of drones is mentioned, in some complex environments (such as the bottom of the bridge), the positioning accuracy may still be interfered with and affected; 3) The detection accuracy needs to be improved: Although the SSD algorithm is improved to improve the detection accuracy, it does not provide detailed information on how the improved algorithm performs in dealing with fine cracks; 4) In addition, there is a lack of targeted classification and measurement strategies for different types of cracks (such as cracks and non-cracks).

[0009] In summary, while existing bridge crack detection technologies have improved detection efficiency and safety to a certain extent, they still have many shortcomings. In particular, more sophisticated, accurate, and interpretable methods are needed for crack classification and measurement to meet the needs of practical engineering applications. Therefore, this paper proposes an interpretable classification and measurement algorithm for multiple types of bridge crack damage. This algorithm aims to overcome the limitations of existing technologies, improve the accuracy of crack identification and classification, and provide strong support for bridge structure health monitoring and maintenance. Summary of the Invention

[0010] The technical problem to be solved by the present invention is to provide an interpretable classification and measurement algorithm for multiple types of concrete crack diseases, to solve the limitations of existing technologies in the field of bridge crack detection and assessment, especially the problems of inaccurate and inefficient classification and measurement of multiple types of bridge crack diseases, and the technical defects of the existing technology such as low crack identification accuracy, fuzzy classification and large measurement errors.

[0011] To solve the above technical problems, the present invention adopts a technical solution: an interpretable classification and measurement algorithm for multiple types of bridge crack defects, comprising the following steps:

[0012] Step 1: Use a deep learning semantic segmentation network model to segment the bridge crack image and obtain a binary image of the crack;

[0013] Step 2: Perform hole filling operation on the segmented binary crack image, then subtract the original binary image to obtain the hole area, and set the threshold according to the number of hole areas to distinguish between crack-type cracks and non-crack-type cracks;

[0014] Step 3: For non-cracking cracks, traditional image processing methods are further used to analyze the crack shape and size characteristics to classify them into small curved cracks, flaky cracks similar to cracks, and conventional cracks. The remaining cracks are distinguished by the number of intersections.

[0015] Step 4: For the classified common cracks, use the custom seed point growth method to measure their length and average width.

[0016] In the preferred solution, the deep learning semantic segmentation network model in Step 1 is a U-Net semantic segmentation network, but is not limited to U-Net. Network models suitable for semantic segmentation such as FCN (Fully Convolutional Networks), Deep Lab, and Seg-Net can also be used.

[0017] In a preferred solution, the hole filling operation in Step 2 refers to filling the holes in the segmented binary crack image to highlight the hollow area formed by the crack, and identifying the characteristics of the crack type crack by comparing the difference between the images before and after filling.

[0018] In the preferred scheme, the specific method for distinguishing between crack-type cracks and non-crack-type cracks in Step 2 is: filling the holes in the segmented binary crack image to obtain a filled image, then subtracting the original binary image from the filled image to obtain a hole area image, counting the number of hole areas, and when the number of hole areas is greater than or equal to a preset threshold, it is determined to be a crack-type crack, otherwise it is determined to be a non-crack-type crack.

[0019] In a preferred solution, the step of classifying non-cracking cracks using traditional image processing methods in Step 3 includes:

[0020] Step 3.1: Filter out each crack region using the connected domain method, draw a bounding rectangle, calculate the ratio of the bounding rectangle area to the region area, and set a threshold to distinguish small curved cracks from other cracks.

[0021] Step 3.2: For the remaining cracks, calculate the ratio of the crack area to the area of ​​the circumscribed rectangle and set a threshold to distinguish between cracks and flake cracks;

[0022] Step 3.3: For the remaining cracks, extract the crack skeleton and count the number of intersections, and set a threshold to distinguish between tortoise-type cracks and regular cracks.

[0023] In a preferred solution, the connected domain method in Step 3.1 is to search for interconnected pixel regions by traversing the image and record the positions of the pixels in the regions, and draw a circumscribed rectangle using the positions of the boundary pixels.

[0024] In a preferred solution, the crack skeleton extraction in Step 3.3 adopts the Zhang-Suen thinning algorithm, which can accurately obtain the fine line structure of the crack while maintaining the consistency and connectivity of the shape.

[0025] In a preferred solution, the step of measuring the length and average width of conventional cracks using the custom seed point growth method in Step 4 includes:

[0026] Step 4.1: Extract the skeleton of the conventional crack and the coordinates of each vertex of the skeleton;

[0027] Step 4.2: Determine the growth seed point and the regional growth stop condition, and traverse other vertices using the seed point growth method to obtain the longest distance, which is the length of the regular crack;

[0028] Step 4.3: Calculate the average width as the crack area divided by the skeleton length.

[0029] In a preferred solution, the growth seed point in Step 4.2 is the vertex in the crack skeleton closest to the upper left corner of the image, and the growth stop condition is to stop when the last vertex is encountered. The number of growth rounds when it stops represents the maximum length of the crack.

[0030] In a preferred solution, the bridge crack image in Step 1 is obtained by taking pictures with a drone, and the dataset used by the algorithm is a self-built dataset constructed by slicing a specific bridge with a predetermined size after taking pictures with a drone.

[0031] The interpretable classification and measurement algorithm for multiple types of concrete crack diseases provided by the present invention has the following beneficial effects:

[0032] 1. This invention addresses the limitations of existing technologies in bridge crack detection and assessment, particularly the inaccurate and inefficient classification and measurement of multiple types of bridge crack defects, by proposing an effective solution. Traditional methods are ineffective when dealing with complex backgrounds and diverse cracks, resulting in low crack identification accuracy, ambiguous classification, and large measurement errors. Deep learning, while robust and universal, struggles to achieve optimal results when insufficient data is available. This invention effectively overcomes these technical limitations by combining deep learning with traditional image processing techniques, improving the accuracy and efficiency of crack detection.

[0033] 2. To address the problem of lack of data on tortoise-type cracks in existing data sets, the present invention uses traditional image processing technology and a new method of hole filling and regional analysis to effectively distinguish tortoise-type cracks from non-tortoise-type cracks. This innovation not only significantly improves the accuracy of identifying tortoise-type cracks, but also enriches the crack data set, providing more diverse learning materials for model training, thereby improving the accuracy and efficiency of overall crack identification.

[0034] 3. This invention combines deep learning with traditional image processing to achieve precise identification and classification of bridge cracks. First, a deep learning model (such as the U-Net semantic segmentation network) is used to perform preliminary segmentation of bridge crack images, generating binary images of the cracks. Traditional image processing techniques are then used for detailed classification and measurement. This method effectively improves the accuracy of crack identification and classification, particularly for cracks that are flaking. Through hole filling and regional analysis, this invention can accurately distinguish between flaking and non-flaking cracks, providing a clear and scientific basis for subsequent crack monitoring and assessment.

[0035] 3. In response to the problem of insufficient crack and fissure data in public data sets, the present invention proposes a new crack and fissure identification method. This method performs hole filling and regional analysis based on the binary image after crack segmentation. By counting the number of void areas and setting appropriate thresholds, it can effectively distinguish cracks from non-cracks. This innovative method not only improves the accuracy of crack and fissure identification, but also provides a rich data basis for subsequent related research. In addition, the implementation of this method is simple and easy, and high-precision results can be obtained in a relatively short time, which has broad application prospects.

[0036] 4. For mixed crack areas, the present invention proposes a comprehensive classification method that combines crack morphology, size and other characteristics. This method first uses a circumscribed rectangle to draw the size characteristics of the crack, and then distinguishes curved small area cracks; then calculates the ratio of the crack area to the circumscribed rectangle area, extracts the morphological characteristics of the crack, and further distinguishes between type cracks and flaky cracks; finally, the number of intersections is counted through skeleton processing to accurately distinguish between type cracks and conventional cracks. This comprehensive classification method not only improves the accuracy of crack classification, but also provides a clear classification basis for subsequent crack monitoring and evaluation.

[0037] 5. To address the complexity of mixed-type crack classification, this paper proposes a comprehensive classification method that combines crack morphology, size, and other characteristics. This method utilizes methods such as bounding rectangle drawing, area ratio calculation, and skeletonization to meticulously distinguish between small curved cracks, flaky cracks similar to tortoise-like cracks, tortoise-type cracks, and conventional cracks. This comprehensive classification strategy not only enhances the flexibility of crack classification but also ensures accuracy, providing a scientific basis for subsequent crack monitoring, assessment, and repair and reinforcement.

[0038] 6. For conventional crack length and average width measurements, this paper proposes a custom seed point growth method. This method records the vertex coordinates of the crack skeleton and uses a seed point growth algorithm to determine the longest distance of the crack, thereby accurately locating the actual crack length. Simultaneously, the average crack width is calculated by calculating the ratio of the crack area to the skeleton length. This method not only improves the accuracy of crack measurement but also automates the processing of large amounts of image data, updating crack information in real time and providing a timely basis for structural health monitoring. Furthermore, the flexibility of the custom seed point growth method allows for dynamic parameter adjustment based on the characteristics of the specific crack, ensuring high-precision and reliable measurement results.

[0039] 7. By combining deep learning with traditional image processing technology, the present invention proposes an interpretable classification and measurement algorithm for multiple types of bridge crack diseases, which effectively solves the limitations of existing technologies, improves the accuracy and efficiency of crack detection, provides important data support for the safety assessment and maintenance of bridge structures, and lays a solid foundation for the safety assurance of bridge structures.

[0040] 8. The detailed classification of the present invention helps engineers and researchers to gain a deeper understanding of the causes of different types of cracks and their impact on structural safety. By taking more effective maintenance and reinforcement measures, the service life and safety of roads and buildings can be further improved. This comprehensive crack classification method has broad application prospects and important practical significance in engineering practice.

[0041] 9. The method of the present invention not only improves the accuracy of classification of chasm-type cracks, but also significantly improves work efficiency. The implementation process is simple and easy, and high-precision results can be obtained in a relatively short time. In addition, the method improves the data set, provides richer learning data for related segmentation and classification models, and further improves the detection accuracy of chasm-type cracks. This technological breakthrough helps to solve the difficulties in crack detection and provides strong support for the health monitoring of structures such as bridges. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0043] Figure 1 It is a technical flow chart of the algorithm of the present invention;

[0044] Figure 2 Examples of cracks in public datasets;

[0045] Figure 3 This is an example of cracks in the self-built dataset;

[0046] Figure 4 This is an example of mixed cracks;

[0047] Figure 5 This is an example of a conventional crack;

[0048] Figure 6 The non-crack type crack binary original image, filling image, and segmentation cavity image of the present invention;

[0049] Figure 7 The present invention provides a binary original image of crack type fracture, a filling image, and a segmented cavity image;

[0050] Figure 8 It is a binary image of a crack type crack of the present invention;

[0051] Figure 9 Schematic diagram of the cracked skeleton of the present invention;

[0052] Figure 10 A schematic diagram of the crack skeleton and intersection markings of the present invention;

[0053] Figure 11 A schematic diagram of a conventional crack skeleton and intersection markings of the present invention;

[0054] Figure 12 This is an example of a crack classification statistical diagram of the present invention;

[0055] Figure 13 This is a flow chart of the algorithm for measuring crack length using custom seed points in the present invention;

[0056] Figure 14 This is an example diagram showing the classification and measurement results of the present invention. DETAILED DESCRIPTION

[0057] The technical solutions of the present invention are further described below with reference to the accompanying drawings and embodiments:

[0058] Example 1

[0059] The interpretable classification and measurement algorithm for multiple types of bridge crack damage includes the following steps:

[0060] Step 1: Use a deep learning semantic segmentation network model to segment the bridge crack image and obtain a binary image of the crack;

[0061] Step 2: Perform hole filling operation on the segmented binary crack image, then subtract the original binary image to obtain the hole area. By counting the number of hole areas and setting a threshold, the crack type and non-crack type cracks are distinguished.

[0062] Step 3: For non-cracking cracks, traditional image processing methods are further used to screen out each crack area through the connected domain method, draw a circumscribed rectangle, calculate the ratio of the circumscribed rectangle area to the area of ​​the region, and the ratio of the crack area to the circumscribed rectangle area, and combine the number of intersections of the crack skeleton to subdivide the non-cracking cracks into small curved area cracks, flaky cracks similar to cracks, conventional cracks and / or remaining crazing cracks;

[0063] Step 4: For the classified regular cracks, extract their skeleton and the coordinates of each skeleton vertex. Use the custom seed point growth algorithm to obtain the longest distance of the crack as the crack length. Divide the crack area by the skeleton length to get the average width.

[0064] In this embodiment, the deep learning semantic segmentation network model in Step 1 is a U-Net semantic segmentation network, but is not limited to U-Net. Network models suitable for semantic segmentation such as FCN, DeepLab, and Seg-Net can also be used.

[0065] Furthermore, the hole filling operation in Step 2 refers to filling the holes in the segmented binary crack image to highlight the hollow area formed by the cracks, and identifying the characteristics of the crack type cracks by comparing the difference between the images before and after filling.

[0066] Furthermore, the specific method for distinguishing between crack-type cracks and non-crack-type cracks in Step 2 is: filling the holes in the segmented binary crack image to obtain a filled image, then subtracting the filled image from the original binary image to obtain a hole area image, and counting the number of hole areas. When the number of hole areas is greater than or equal to a preset threshold, it is determined to be a crack-type crack, otherwise it is determined to be a non-crack-type crack.

[0067] Furthermore, the step of classifying non-cracking cracks using traditional image processing methods in Step 3 includes:

[0068] Step 3.1: Record the area S of the entire region;

[0069] Step 3.2: Filter out each crack region using the connected domain method, draw a circumscribed rectangle for each crack, and record the area S1 of the circumscribed rectangle of each crack in the region;

[0070] Step 3.3: Set the threshold value threshold1. When S1 / S < threshold1, the crack is recorded as a small curved crack.

[0071] Step 3.4: Set the threshold threshold2 and count the crack area S2 (i.e., the number of white pixels) of the remaining cracks. When S2 / S1 < threshold2, the crack is recorded as a flaky crack of the quasi-cracking type.

[0072] Step 3.5: After obtaining the skeleton of the remaining cracks, the number of intersections S3 is counted and a threshold threshold 3 is set. When S3 > threshold 3, this type of crack is recorded as a tortoise crack, otherwise it is recorded as a regular crack.

[0073] Furthermore, the connected domain method in Step 3.1 is to search for interconnected pixel regions by traversing the image and record the positions of the pixels in the regions, and draw a circumscribed rectangle based on the positions of the boundary pixels.

[0074] Furthermore, the crack skeleton extraction in Step 3.3 adopts the Zhang-Suen thinning algorithm, which can accurately obtain the fine line structure of the crack while maintaining the consistency and connectivity of the shape.

[0075] Furthermore, the steps of measuring the length and average width of conventional cracks using the custom seed point growth method in Step 4 include:

[0076] Step 4.1: Extract the skeleton of the conventional crack and the coordinates of each vertex of the skeleton;

[0077] Step 4.2: Determine the growth seed point and the regional growth stop condition, and traverse other vertices using the seed point growth method to obtain the longest distance, which is the length of the regular crack;

[0078] Step 4.3: Calculate the average width as the crack area divided by the skeleton length.

[0079] Furthermore, in Step 4.2, the growth seed point is selected as the vertex in the crack skeleton closest to the upper left corner of the image. The growth stop condition is to stop when the last vertex is encountered. The number of growth rounds when it stops represents the maximum length of the crack.

[0080] Furthermore, the growth seed point and the regional growth stopping condition determined in Step 4.2 are: the initial seed point is the vertex in the skeleton closest to the upper left corner of the image; the growth stopping condition is to stop when the last vertex is encountered; the number of growth rounds at the time of stopping represents the maximum length of the crack; during the growth process, it pauses when encountering another vertex, and the number of growth rounds is counted, that is, how many times it has grown, and the number of growth rounds is the distance between the two vertices; then growth continues until the last vertex, and the cumulative number of all growth rounds is the maximum length of the conventional crack.

[0081] Furthermore, the bridge crack image in Step 1 is obtained by taking pictures with a drone, and the dataset used by the algorithm is a self-built dataset constructed by slicing a specific bridge with a predetermined size after taking pictures with a drone.

[0082] Example 2

[0083] In another preferred embodiment, based on the above embodiment 1, this embodiment will describe the technical solution of the present invention in detail in combination with specific embodiments.

[0084] Dataset preparation: The dataset used in this example is a self-constructed dataset constructed by photographing a bridge using the Matrice 350 RTK drone. The captured bridge images were preprocessed and cut into 448×448 pixel slices for subsequent crack detection, classification, and measurement.

[0085] Step 1: Crack Image Segmentation

[0086] First, we used the U-Net semantic segmentation network to segment the bridge crack images from our self-built dataset. The U-Net network, with its unique U-shaped structure, excels in image segmentation and is particularly well-suited for the crack segmentation task in this example. The resulting binary image shows the crack region as white and the background as black.

[0087] Step 2: Classification of cracks and non-cracks

[0088] Hole-filling is performed on the segmented binary crack image, and then the original binary image is subtracted to obtain distinct hole regions. A threshold is set based on the number of holes. When the number of holes exceeds the threshold, the crack is identified as a chasm-type crack; otherwise, it is identified as a non-chasm-type crack. In this example, the threshold is set to 5, meaning that when the number of holes exceeds 5, the crack is identified as a chasm-type crack.

[0089] Step 3: Further classification of non-cracking cracks

[0090] For non-cracks, traditional computer vision methods are further used for detailed classification. The specific steps are as follows:

[0091] 3.1. Record the area S of the entire crack region;

[0092] 3.2. Filter out each crack region using the connected domain method, draw a circumscribed rectangle for each crack, and record the circumscribed rectangle area S1 of each crack in the region;

[0093] 3.3. Set the threshold value threshold1. When S1 / S < threshold1, the crack is recorded as a small curved crack. In this embodiment, threshold1 is set to 0.1.

[0094] 3.4. For the remaining cracks, set a threshold value threshold2, and count the crack area (i.e., the number of white pixels) as S2. When S2 / S1 < threshold2, the crack is recorded as a flaky crack of the quasi-cracking type. In this embodiment, threshold2 is set to 0.5.

[0095] 3.5. For the remaining cracks, use the Zhang-Suen algorithm to extract the crack skeleton and then count the number of intersections S3. Set a threshold value, threshold3. When S3 is greater than threshold3, the crack is determined to be a tortoise-type crack. (Note that the determination of tortoise-type cracks in this step is based on the number of intersections, which is different from the determination method in step 2. However, here, it should actually be distinguished between conventional cracks and complex tortoise-type cracks. If the number of intersections is small, the crack is a conventional crack. The description of tortoise-type cracks here is based on the description of the previous process, and should actually be understood as further distinguishing the complex cases among the remaining cracks. Otherwise, the crack is determined to be a conventional crack. In this embodiment, threshold3 is set to 10.

[0096] Step 4: Conventional crack length and average width measurement

[0097] For the conventional cracks obtained by classification, the custom seed point growth method is used to measure their length and average width. The specific steps are as follows:

[0098] 4.1. Extract regular fracture areas and extract regular fractures separately through the maximum connected area;

[0099] 4.2. Use the Zhang-Suen thinning algorithm to extract the thinned skeleton of the crack image and record the coordinates of each vertex on the skeleton;

[0100] 4.3. Select the point in the upper left corner as the growth seed point;

[0101] 4.4. Use the seed point growth algorithm to expand the growth seed point to other vertices, and set the growth stop condition to "stop when encountering a vertex";

[0102] 4.5. By repeating the growth process, traversing all vertices and recording the longest growth round number, the maximum length of the regular crack is obtained;

[0103] 4.6. Calculate the average width as crack area / frame length.

[0104] Example 3

[0105] In another preferred embodiment, based on the above embodiments 1 and 2, as Figure 1 As shown, this embodiment will further illustrate the technical solution of the present invention in combination with the drawings and specific embodiments.

[0106] Dataset preparation: The dataset used in this example is a self-built dataset constructed by photographing a bridge using the Matrice 350 RTK drone. The captured bridge images were pre-processed and cut into 448×448 pixel slices for subsequent crack detection, classification, and measurement. The overall process is as follows: Figure 1 shown.

[0107] 1. Traditional image processing methods to classify cracks and non-cracks

[0108] To distinguish between cracks and non-cracks, this example uses traditional image processing methods to classify these types of cracks based on their characteristics. To avoid the influence of the bridge background on the classification results, this example first uses a semantic segmentation model to process the self-built dataset to obtain crack segmentation results, which are binary images. This example uses the U-Net semantic segmentation network, but this is not limited to this network. Any network model suitable for semantic segmentation can be used, such as FCN, DeepLab, Seg-Net, etc.

[0109] For cracks and non-cracks, such as Figure 2As shown in the figure, a comparison reveals that chasm-type cracks often exhibit multi-directional, interlaced, irregular crack patterns, dividing the crack area into several independent small regions of varying sizes. The boundaries between these small regions are blurred, adding complexity to the overall structure. In contrast, conventional cracks and mixed cracks of various types exhibit different characteristics. Although these cracks may also enclose cavities, their number is often small and their coverage is relatively limited. Therefore, identifying and classifying these two types of cracks is particularly important during crack analysis and treatment.

[0110] Based on the analysis of the above crack characteristics, this embodiment proposes a method based on traditional computer vision to effectively distinguish between crack-type cracks and non-crack-type cracks. Specifically, the hole filling operation is first performed on the segmented binary crack image, and then the original binary image is subtracted to obtain the obvious hole area. Figure 3 、 4 As shown, the binary image after segmentation of non-crack type cracks and crack type cracks, the image after hole filling processing, and the final image of the void area are displayed.

[0111] The experimental results show that there are only two significant cavities in non-crack-type cracks, while there are seven cavities in crack-type cracks. This significant difference provides an effective basis for distinguishing between crack-type and non-crack-type cracks, which can be further distinguished by appropriately setting thresholds. This method not only improves the accuracy of crack analysis but also provides a basis for subsequent structural assessment and maintenance.

[0112] 1. Use the characteristics of various cracks to distinguish mixed crack areas

[0113] For non-crack types, in addition to conventional cracks, there are also crack areas where conventional cracks and multiple types of cracks are mixed. The characteristics of each crack in the mixed cracks have been described above and divided into four categories: curved small-area cracks, crack-like flaky cracks, crack-type cracks, and conventional cracks. This embodiment further distinguishes the various types of cracks in the mixed cracks by analyzing the characteristics of each type of crack, such as size and shape. The specific algorithm is shown in Table 1:

[0114] Table 1

[0115]

[0116] Since small curved cracks have a relatively small area, to distinguish these cracks, this embodiment uses a connected domain algorithm to find each crack region. Then, based on the boundary coordinates of each region, the bounding rectangle of each crack region is obtained. The proportion of each crack within the region is determined by statistically calculating the ratio of the bounding rectangle area to the region area. A threshold is set, and cracks smaller than the threshold are classified as small curved cracks and marked with a red frame, while cracks larger than the threshold proceed to the next round of classification. For flaky cracks, these cracks occupy a larger area within the region, but their shape often appears curled, so they occupy a larger proportion of the bounding rectangle area. This embodiment uses the ratio of the crack area to the bounding rectangle area to map the curled and divergent states of the cracks. A threshold is set to distinguish between flaky cracks and flaky cracks. Cracks above the threshold are classified as flaky cracks and marked with a yellow frame, while those below the threshold are classified as flaky cracks and regular areas for the final round of classification.

[0117] For the remaining tortoise-type cracks and conventional cracks, in addition to being judged by the number of cavities mentioned above, they can also be judged by the number of intersections. Turtle-type cracks often intersect horizontally and vertically, resulting in a large number of intersections, while conventional cracks are mostly single horizontal and vertical cracks, and even if there are a small number of branches, the number of intersections is not large. Based on this feature, this embodiment distinguishes between tortoise-type cracks and conventional cracks by detecting intersections and recording the number of intersections after extracting the crack skeleton. Due to the irregular extension of cracks, this embodiment uses the Zhang-Suen algorithm for crack skeleton extraction. Zhang-Suen skeleton extraction algorithm

[15] It is an efficient image processing technology, especially suitable for extracting crack skeletons. Its advantage is that it can accurately obtain the fine line structure of an object while maintaining the consistency and connectivity of the shape. The algorithm effectively removes redundant pixels through iterative processing steps to generate a refined skeleton, which not only reduces the computational complexity but also improves the accuracy of the results. In addition, the Zhang-Suen algorithm has strong robustness to noise and small area interference, and can stably extract crack skeletons in various environments, providing a reliable data basis for subsequent crack analysis and classification, such as Figure 8 、 9 is a schematic diagram of a crack-type crack binary image and its skeleton, as shown in Figure 10 、 11 The following are schematic diagrams of typical tortoise-type and conventional crack skeletons and their intersection markings. It can be seen that the number of intersections in tortoise-type cracks is significantly higher than that in conventional cracks. This feature can be used to effectively distinguish tortoise-type cracks from conventional cracks, and tortoise-type cracks are marked with blue frames, and conventional cracks are marked with green frames.

[0118] Figure 12 This is a bar chart that counts and plots the number of various types of cracks in the left data set ( Figure 12(As shown on the right), by marking each type of crack in detail, not only can different types of cracks be clearly identified and classified, but it also lays a solid foundation for subsequent statistical analysis. After the cracks are marked, the data can be subjected to in-depth statistical analysis. This process is not just a simple quantitative count, but also includes an analysis of the nature of the cracks, such as the frequency of crack occurrence, the speed of expansion, and the proportion of different types of cracks in the area. These statistical results will provide important decision-making basis for subsequent maintenance, and can accurately assess the structural health and potential risks of the bridge. In addition, the characteristic information of each crack, including its location, the length and width of the circumscribed rectangle, the shape, and the relationship with the surrounding structure, can be recorded and included in the database, facilitating a more comprehensive understanding of the distribution of cracks in the area and the formulation of more targeted maintenance and protection strategies.

[0119] 2. Custom seed point growth method to measure the length and average width of conventional cracks

[0120] After classifying regular cracks, this embodiment proposes a method for skeleton extraction and length calculation for regular cracks to improve the accuracy and efficiency of crack analysis. First, regular crack regions are extracted. Using the maximum connected area method, these cracks are individually isolated. Thinning algorithms such as Zhang-Suen are then used to extract a thinned skeleton from the crack image. During this process, the coordinates of each vertex on the crack skeleton are recorded. These vertices are important geometric features of the crack and provide key data for the subsequent growth algorithm. Since regular cracks have relatively simple structures, to reduce computational complexity, this embodiment selects the top-left corner point (i.e., the point closest to the left vertex) as the growth seed point. Next, using the seed point growth algorithm, the growth seed point is extended to other vertices. At this stage, the growth stop condition is set to "stop at vertex." Specifically, when a seed point starts growing from one vertex and reaches another, the growth process is terminated. This process is performed by gradually checking neighboring pixels or adjacent vertices to ensure that each growth step is along the connected portion of the skeleton. Since the skeleton area width of a conventional crack is only 1 pixel, the number of growth rounds represents the distance between two vertices. By repeating the above growth process, all vertices can be traversed and the longest number of growth rounds can be recorded to obtain the maximum length of the conventional crack. The average width obtained is recorded as crack area / skeleton length. The specific process is as follows: Figure 13 shown.

[0121] The above method is to classify various types of cracks in mixed cracks and measure the length and average width of conventional cracks. Figure 14 Examples of classification results and length and average width results for regular cracks are shown.

[0122] Example 4

[0123] In another preferred embodiment, based on the above-mentioned embodiments 1, 2, and 3, in order to demonstrate the diversity and practicality of the technical solution of the present invention, this embodiment uses another semantic segmentation network FCN to segment the bridge crack image, and uses the subsequent processing steps in embodiment 1 for classification and measurement.

[0124] In step 1, the FCN network is used to segment the bridge crack images in the self-built dataset. The FCN network, with its fully convolutional structure and effective fusion of multi-scale features, is also suitable for crack segmentation tasks. The binary image obtained after segmentation is also used for subsequent crack classification and measurement.

[0125] The subsequent steps (steps 2 to 4) are exactly the same as those in Example 2 and will not be repeated here.

[0126] Compared with the closest existing technology, the present invention proposes a crack classification method based on the combination of traditional image processing technology and deep learning. By filling holes and counting the number of void areas, it effectively distinguishes between cracks and non-cracks, thereby improving the accuracy of classification. For non-crack cracks, it further uses crack morphology, size and other characteristics for classification, and realizes the effective distinction between curved small area cracks, crack-like flaky cracks, crack-type cracks and conventional cracks. In terms of measuring the length and average width of conventional cracks, a custom seed point growth method is adopted to improve the measurement accuracy and efficiency.

[0127] Example 5

[0128] In another preferred embodiment, based on the above-mentioned embodiments 1 to 4, a computer-readable storage medium stores a computer program, which, when executed by a processor, implements the explainable classification and measurement algorithm for multiple types of bridge crack diseases described in any one of the above-mentioned embodiments 1 to 4.

[0129] Example 6

[0130] In another preferred embodiment, based on the above-mentioned embodiment 5, an electronic device includes a processor and a storage medium, wherein the storage medium stores the computer program described in embodiment 5, and when the processor executes the computer program, it implements the explainable classification and measurement algorithm for multiple types of bridge crack diseases described in any one of embodiments 1 to 4.

[0131] In the preferred solution, the deep learning semantic segmentation network model in Step 1 is a U-Net semantic segmentation network, but is not limited to U-Net. Network models suitable for semantic segmentation such as FCN (Fully Convolutional Networks), Deep Lab, and Seg-Net can also be used. The above settings can be selected according to specific task requirements and data characteristics. At the same time, in order to improve the generalization ability and segmentation accuracy of the model, technical means such as data enhancement and loss function optimization can be used during the training process.

[0132] In the preferred solution, the hole filling operation in Step 2 refers to filling the holes in the segmented binary crack image to highlight the hollow area formed by the crack, and identifying the characteristics of the crack-type crack by comparing the differences between the images before and after filling; the above settings effectively improve the accuracy of crack identification, especially in complex backgrounds, and can more clearly define the crack boundaries, providing a reliable data basis for subsequent crack type classification and severity assessment.

[0133] In the preferred scheme, the specific method for distinguishing between crack-type cracks and non-crack-type cracks in Step 2 is: filling the holes in the segmented binary crack image to obtain a filled image, then subtracting the original binary image from the filled image to obtain a hole area image, counting the number of hole areas, and when the number of hole areas is greater than or equal to a preset threshold, it is determined to be a crack-type crack, otherwise it is determined to be a non-crack-type crack; the above settings can effectively distinguish different types of cracks and improve the accuracy and efficiency of crack identification. In addition, the crack classification can be further refined according to the size, shape and other characteristics of the hole area, providing more detailed information support for subsequent crack processing and repair.

[0134] In the preferred solution, the connected domain method in Step 3.1 is to search for interconnected pixel areas by traversing the image and record the positions of the pixel points in the area, and draw a circumscribed rectangle based on the positions of the boundary pixels; the above setting can accurately identify and mark each connected area in the image, and at the same time use the circumscribed rectangle to effectively represent the position and size of each area, providing accurate geometric information for subsequent processing steps.

[0135] In the preferred solution, the crack skeleton extraction in Step 3.3 adopts the Zhang-Suen thinning algorithm, which can accurately obtain the fine line structure of the crack while maintaining the consistency and connectivity of the shape. The above settings ensure the accuracy and stability of crack identification. On this basis, the crack skeleton is further optimized through morphological processing to reduce noise interference, laying a solid foundation for subsequent crack feature extraction and analysis.

[0136] In the preferred solution, the growth seed point in Step 4.2 is the vertex in the crack skeleton closest to the upper left corner of the image, and the growth stop condition is to stop when the last vertex is encountered. The number of growth rounds at the time of stopping represents the maximum length of the crack. The above settings ensure the accuracy and efficiency of crack detection. By calculating the number of growth rounds and comparing it with the preset threshold, it is possible to quickly determine whether the crack exceeds the safety range, providing a reliable basis for subsequent crack repair and structural safety assessment.

[0137] In the preferred solution, the growth seed point and the regional growth stopping condition are determined in Step 4.2 as follows: the initial seed point is the vertex in the skeleton closest to the upper left corner of the image; the growth stopping condition is to stop when the last vertex is encountered; the number of growth rounds at the time of stopping represents the maximum length of the crack; during the growth process, when another vertex is encountered, it is paused, and the number of growth rounds is counted, that is, how many times it has grown, and the number of growth rounds is the distance between the two vertices; then growth continues until the last vertex, and the cumulative number of all growth rounds is the maximum length of the conventional crack; the above settings can accurately track the crack path, while effectively avoiding misjudgment and omissions, and improving the accuracy and efficiency of crack detection. In addition, the scheme also has good adaptability and robustness, and can cope with crack detection needs of different complexities and morphologies.

[0138] In the preferred solution, the bridge crack image in Step 1 is obtained by drone photography, and the dataset used by the algorithm is a self-built dataset constructed by slicing a specific bridge with a predetermined size after photographing it by a drone. The above settings effectively improve the efficiency and accuracy of data acquisition, enabling the crack detection algorithm to more accurately identify and analyze minor damage in the bridge structure, providing strong data support for bridge safety assessment and maintenance.

[0139] In summary, this paper addresses the technical limitations in the field of bridge crack detection and assessment, especially the inaccurate and inefficient classification and measurement of multiple types of cracks. It proposes an interpretable classification and measurement algorithm for multiple types of concrete crack diseases. This algorithm successfully combines the advantages of deep learning and traditional image processing techniques, bringing an innovative solution to the field of bridge crack detection.

[0140] First, the present invention uses deep learning to perform preliminary crack identification and segmentation. This step allows for rapid location and extraction of crack regions, laying the foundation for subsequent processing. Subsequently, it combines traditional image processing techniques, such as the connected domain method and the Zhang-Suen skeleton extraction algorithm, to perform detailed crack classification and feature extraction. This combined approach not only improves crack identification accuracy but also enhances the interpretability of the entire process.

[0141] This paper proposes a novel identification method for the special type of cracks. This method performs a hole-filling operation on the binary image after crack segmentation, then subtracts the original binary image to generate multiple independent hollow regions. By counting these hollow regions and setting an appropriate threshold, it can effectively distinguish between cracks and non-cracks, significantly improving the accuracy of crack classification.

[0142] To measure crack length and average width, this paper proposes a custom seed point growth method. By defining growth seed points and regional growth stopping conditions, this method leverages the crack skeleton and vertex information to achieve precise crack length measurement. This method is not only highly automated but also ensures measurement accuracy, providing reliable data support for bridge structure safety assessments.

[0143] Furthermore, the present invention proposes a comprehensive classification method that combines characteristics such as crack morphology and size. This method can effectively distinguish between various types of cracks, including small curved cracks, flaky cracks similar to tortoise cracks, tortoise-type cracks, and conventional cracks. This classification method not only improves the accuracy of crack identification but also provides a clear classification basis for subsequent crack monitoring and assessment, helping engineers and researchers gain a deeper understanding of the causes of different crack types and their impact on structural safety.

[0144] To address the problem of insufficient collection of cracks and fissures in existing public datasets, this paper effectively overcomes the difficulty of complex crack classification by building a self-built dataset and using innovative identification methods. This innovation not only enriches the crack dataset but also provides new insights for the development of crack detection technology.

[0145] In summary, this invention innovatively proposes multiple aspects, including the innovative application of deep learning combined with traditional image processing methods, a new identification method for cracks, a customized seed point growth method for crack measurement, and a comprehensive classification method. By deeply exploring crack characteristics and combining traditional image processing techniques with deep learning algorithms, this invention achieves interpretable classification and precise measurement of multiple types of bridge crack diseases, providing strong support for the safety assessment and maintenance of bridge structures. This innovative achievement not only promotes the development of bridge crack detection technology but also provides a solid foundation for the safety of bridge structures.

Claims

1. An interpretable classification and measurement algorithm for multiple types of bridge crack damage, characterized by: The following steps are involved: Step 1: Use a deep learning semantic segmentation network model to segment the bridge crack image and obtain a binary image of the crack; Step 2: Fill the holes in the segmented binary crack image to obtain a filled image. Then, subtract the filled image from the original binary image to obtain a hole area image. Count the number of hole areas. When the number of hole areas is greater than or equal to the preset threshold, it is determined to be a crack type crack, otherwise it is determined to be a non-crack type crack. Step 3: For non-cracking cracks, traditional image processing methods are further used to analyze the shape and size characteristics of the cracks to classify them into small curved cracks, flaky cracks similar to cracks, and conventional cracks. The remaining cracks are distinguished by the number of intersections. The specific steps are as follows: Step 3.1: Filter out each crack region using the connected domain method, draw a bounding rectangle, calculate the ratio of the bounding rectangle area to the region area, and set a threshold to distinguish small curved cracks from other cracks. Step 3.2: For the remaining cracks, calculate the ratio of the crack area to the area of ​​the circumscribed rectangle and set a threshold to distinguish between cracks and flake cracks; Step 3.3: For the remaining cracks, extract the crack skeleton, count the number of intersections, and set a threshold to distinguish between crazing cracks and regular cracks. Step 4: For the classified common cracks, use the custom seed point growth method to measure their length and average width.

2. The interpretable classification and measurement algorithm for multiple types of bridge crack defects according to claim 1 is characterized by: The deep learning semantic segmentation network model in Step 1 is U-Net, FCN, DeepLab or Seg-Net.

3. The interpretable classification and measurement algorithm for multiple types of bridge crack defects according to claim 1 is characterized by: The bridge crack images in Step 1 are obtained by drone photography, and after photography, slices are cut into predetermined sizes to construct a self-built dataset.

4. The interpretable classification and measurement algorithm for multiple types of bridge crack defects according to claim 1 is characterized by: The connected domain method in Step 3.1 is to search for interconnected pixel areas by traversing the image and record the positions of the pixels in the area, and draw a circumscribed rectangle based on the positions of the boundary pixels.

5. The interpretable classification and measurement algorithm for multiple types of bridge crack defects according to claim 1 is characterized by: The crack skeleton extraction in Step 3.3 adopts the Zhang-Suen thinning algorithm, which can accurately obtain the fine line structure of the crack while maintaining the consistency and connectivity of the shape.

6. The interpretable classification and measurement algorithm for multiple types of bridge crack defects according to claim 1 is characterized in that: The steps of measuring the length and average width of conventional cracks using the custom seed point growth method in Step 4 include: Step 4.1: Extract the skeleton of the conventional crack and the coordinates of each vertex of the skeleton; Step 4.2: Determine the growth seed point and the regional growth stop condition, and traverse other vertices using the seed point growth method to obtain the longest distance, which is the length of the regular crack; Step 4.3: Calculate the average width as the crack area divided by the skeleton length.

7. The interpretable classification and measurement algorithm for multiple types of bridge crack defects according to claim 6 is characterized by: In Step 4.2, the growth seed point is selected as the vertex in the crack skeleton closest to the upper left corner of the image. The growth stop condition is to stop when the last vertex is encountered. The number of growth rounds at the time of stopping represents the maximum length of the crack. During the growth process, it pauses when another vertex is encountered, and the number of growth rounds is counted. The number of growth rounds represents the length of the distance between the two vertices, and then growth continues until the last vertex is reached, and all growth rounds are accumulated.

Images