Inplanatable classification and measurement algorithm for multiple types of concrete crack diseases

Abstract

The invention discloses an interpretable classification and measurement algorithm for multiple types of bridge crack diseases, and belongs to the technical field of bridge structure detection. The method aims at solving the problems that in existing bridge crack detection, the crack type of a common data set is single, complex cracks are difficult to recognize, and classification is inaccurate, and particularly, the problems of distinguishing of cracking cracks and non-cracking cracks and classification of multiple types of mixed cracks are solved. According to the technical scheme, the method comprises the steps of segmenting a bridge crack image by using a deep learning semantic segmentation network model; a traditional image processing technology is utilized to distinguish cracking and non-cracking cracks; further subdividing various fractures in the non-cracking fractures; measuring the length and the average width of the conventional crack by adopting a self-defined seed point growth method; according to the method, the accuracy of crack identification is improved, effective classification of complex cracks is realized, and a scientific basis is provided for health monitoring and maintenance of a bridge structure; through the algorithm, a large amount of image data can be automatically processed, crack information is updated in real time, and safety and stability of a bridge structure are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of bridge structure health monitoring, and particularly to an interpretable classification and measurement algorithm for multiple types of concrete crack diseases. Background Art

[0002] As an important transportation infrastructure, the structural health and safety of bridges are directly related to the smoothness of transportation and the safety of people's lives and property. In China, there are a large number of bridges, and most of them face complex and changeable environmental conditions. Therefore, regular inspections and maintenance play a crucial role in improving bridge health and ensuring structural safety. Among bridge diseases, concrete structure cracks are the most common type. They not only indicate structural stress concentration or material aging, but if allowed to continue to expand, they will also cause damage to the bridge structure, expose and corrode the internal steel bars, thereby reducing the bridge stability and shortening the service life of the bridge.

[0003] In traditional bridge structure detection, the detection method of cracks mainly relies on experienced inspectors for manual marking, and the severity of cracks is judged by the naked eye to facilitate subsequent maintenance. However, such detection methods are not only time-consuming and laborious, with high safety risks, but also the detection results are easily affected by the subjective factors of inspectors, making it difficult to ensure the accuracy and consistency of the detection results.

[0004] With the development of science and technology, especially the emergence of bridge surface image acquisition devices such as unmanned aerial vehicles and bridge inspection vehicles, the method of bridge appearance detection has undergone a revolutionary change. These devices enable bridge appearance detection to be transferred from outdoors to indoors and from physical detection to image analysis, greatly improving the detection efficiency and safety. On this basis, deep learning methods have been widely used in bridge crack detection due to their powerful feature extraction and classification capabilities. Compared with traditional manual detection methods, deep learning methods show significant advantages in terms of robustness and universality, and can process crack images with various shapes and complex backgrounds, providing strong support for the automatic detection of bridge surfaces.

[0005] However, despite the significant progress made by deep learning methods in bridge crack detection, they still face many challenges. On the one hand, the crack images in current public datasets mainly consist of single simple cracks in the horizontal and vertical directions, lacking effective coverage of complex and diverse cracks, especially the distinction between cracked and non-cracked cracks. This limitation of the dataset leads to the restricted accuracy of recognition and classification of deep learning models when dealing with actual bridge crack images, especially when facing complex and changeable crack morphologies. On the other hand, for traditional computer vision methods, due to the complexity of bridge crack images and various noise interferences in the shooting environment, it is difficult to artificially design effective image features to cope with these challenges, resulting in poor crack classification effects.

[0006] In addition, existing research mainly focuses on the detection of single simple cracks, while the actual bridge crack situation is much more complex. There are often multiple types of cracks in an image, such as turtle shell cracks, conventional cracks, and situations where multiple cracks are mixed. These different types of cracks vary in morphology, size, distribution, etc., posing higher requirements for crack detection, classification, and measurement. However, for the situation where there are multiple types of cracks in an image, relevant literature reports and research are still relatively scarce, lacking effective solutions.

[0007] For example, CN113506281B discloses a bridge crack detection method based on a deep learning framework. This method realizes the automatic detection of bridge cracks through steps such as obtaining bridge images, preprocessing, and model training. However, although this technical solution mentions preprocessing the original bridge crack image to obtain an image dataset, it does not elaborate on the construction process of the dataset, especially how to ensure the diversity and representativeness of the dataset. Although it describes the structure of the segmentation model, it does not deeply explain the functions of each part of the model and how they work together to achieve crack detection. In addition, there is also a lack of detailed description of the selection of hyperparameters (such as learning rate, optimization algorithm, etc.) during the model training process. Although its deep learning model can automatically detect cracks, the decision-making process of the model lacks interpretability, making it difficult for users to understand why the model makes specific judgments.

[0008] Another example is that CN116297472A discloses a drone bridge crack detection method and system based on deep learning, which combines drone technology and deep learning algorithms to achieve precise detection of bridge cracks. However, there are also some technical defects and deficiencies: 1) The system is complex. The system includes multiple parts such as an acquisition module, a 3D modeling module, and a path planning module, making the entire system relatively complex. This not only increases the implementation difficulty of the system but may also affect the stability and reliability of the system. 2) The positioning accuracy is not high. Although it mentions using a UWB-assisted positioning module to improve the positioning accuracy of the drone, in some complex environments (such as the bottom of the bridge), the positioning accuracy may still be interfered with and affected. 3) The detection accuracy needs to be improved. Although the SSD algorithm is improved to enhance the detection accuracy, it does not elaborate on how the improved algorithm performs in dealing with micro-cracks. 4) In addition, there is also a lack of targeted classification and measurement strategies for different types of cracks (such as turtle shell cracks and non-turtle shell cracks).

[0009] In summary, although the existing bridge crack detection technologies have improved the detection efficiency and safety to a certain extent, there are still many deficiencies. Especially in the classification and measurement of cracks, more refined, accurate, and interpretable methods are needed to meet the requirements of actual engineering applications. Therefore, the present invention proposes an interpretable classification and measurement algorithm for multi-type bridge crack diseases, aiming to overcome the limitations of the existing technologies, improve the accuracy of crack identification and classification, and provide strong support for the health monitoring and maintenance of bridge structures. Summary of the Invention

[0010] The technical problem to be solved by the present invention is to provide an interpretable classification and measurement algorithm for multi-type concrete crack diseases, to solve the limitations of the existing technologies in the field of bridge crack detection and evaluation, especially the problems of inaccurate classification and measurement, low efficiency for multi-type bridge crack diseases, and the technical defects of low crack identification accuracy, fuzzy classification, and large measurement errors in the existing technologies.

[0011] To solve the above technical problems, the technical solution adopted by the present invention is as follows: An interpretable classification and measurement algorithm for multi-type bridge crack diseases, including the following steps: Step1: Use a deep learning semantic segmentation network model to segment the bridge crack image to obtain a binary image of the crack; Step2: Perform a hole filling operation on the segmented binary crack image, then subtract the original binary image to obtain the hollow area, and set a threshold according to the number of hollow areas to distinguish between cracked cracks and non-cracked cracks; Step3: For non-cracked cracks, further use traditional image processing methods to classify them by analyzing the morphological and size characteristics of the cracks, including curved small-area cracks, cracked-like flaky cracks, regular cracks, and distinguish the remaining cracked cracks by the number of intersection points; Step4: For the classified regular cracks, use the custom seed point growth method to measure their length and average width.

[0012] In a preferred solution, the deep learning semantic segmentation network model in Step1 is the U-Net semantic segmentation network, but it is not limited to U-Net, and other network models suitable for semantic segmentation such as FCN (Fully Convolutional Networks), Deep Lab, and Seg-Net can also be used.

[0013] In a preferred solution, the hole filling operation in Step2 refers to filling the holes in the segmented binary crack image to highlight the hollow area formed by the crack, and identifying the characteristics of the cracked cracks by comparing the differences between the images before and after filling.

[0014] In a preferred solution, the specific method for distinguishing between cracked cracks and non-cracked cracks in Step 2 is as follows: perform hole filling on the segmented binary crack image to obtain the filled image, then subtract the original binary image from the filled image to obtain the hole region image, count the number of hole regions, and when the number of hole regions is greater than or equal to a preset threshold, it is determined as a cracked crack; otherwise, it is determined as a non-cracked crack.

[0015] In a preferred solution, the steps for classifying non-cracked cracks using traditional image processing methods in Step 3 include: Step 3.1: Screen out each crack region through the connected component method, draw the circumscribed rectangle, calculate the ratio of the area of the circumscribed rectangle to the area of the region, and set a threshold to distinguish between small curved region cracks and other cracks; Step 3.2: For the remaining cracks, calculate the ratio of the crack area to the area of the circumscribed rectangle, and set a threshold to distinguish between quasi-cracked sheet cracks; Step 3.3: For the finally remaining cracks, count the number of intersection points after extracting the crack skeleton, and set a threshold to distinguish between cracked cracks and regular cracks.

[0016] In a preferred solution, the connected component method in Step 3.1 searches for connected pixel regions by traversing the image and records the pixel positions within the region, and draws the circumscribed rectangle through the boundary pixel positions.

[0017] In a preferred solution, the Zhang-Suen thinning algorithm is used to extract the crack skeleton in Step 3.3. This algorithm can accurately obtain the thin line structure of the crack while maintaining shape consistency and connectivity.

[0018] In a preferred solution, the steps for measuring the length and average width of regular cracks using the custom seed point growth method in Step 4 include: Step 4.1: Extract the skeleton of the regular crack and the coordinates of each vertex of the skeleton; Step 4.2: Determine the growth seed point and the region growth stop condition, and traverse other vertices through the seed point growth method to obtain the longest distance, which is the length of the regular crack; Step 4.3: Calculate the average width as the crack area divided by the skeleton length.

[0019] In a preferred solution, in Step 4.2, the growth seed point is selected as the vertex closest to the upper left corner of the image in the crack skeleton, and the growth stop condition is to stop when encountering the last vertex. The number of growth rounds at the stop represents the maximum length of the crack.

[0020] In a preferred embodiment, the bridge crack image in Step1 is obtained by shooting with a drone, and the dataset used in the algorithm is a self-built dataset constructed by slicing the images taken by the drone on a specific bridge into a predetermined size.

[0021] The interpretable classification and measurement algorithm for multi-type concrete crack diseases provided by the present invention has the following beneficial effects: 1. In view of the limitations of the prior art in the field of bridge crack detection and evaluation, especially the problems of inaccurate classification and measurement of multi-type bridge crack diseases and low efficiency, the present invention proposes an effective solution. Traditional methods have poor effects in dealing with complex backgrounds and diverse cracks, with low crack recognition accuracy, fuzzy classification and large measurement errors. Although deep learning has robustness and universality, it is also difficult to achieve the best results in the absence of sufficient data. By combining deep learning with traditional image processing techniques, the present invention effectively overcomes these technical defects and improves the accuracy and efficiency of crack detection.

[0022] 2. Aiming at the problem of lack of data on cracked cracks in the existing dataset, the present invention adopts traditional image processing techniques and effectively distinguishes cracked and non-cracked cracks through a new method of hole filling and region analysis. This innovation not only significantly improves the recognition accuracy of cracked cracks, but also enriches the crack dataset, providing more diverse learning materials for model training, thereby improving the overall crack recognition accuracy and efficiency.

[0023] 3. The present invention combines deep learning with traditional image processing to achieve accurate recognition and classification of bridge cracks. First, a deep learning model (such as the U-Net semantic segmentation network) is used to preliminarily segment the bridge crack image to obtain a binary image of the crack. Then, traditional image processing techniques are used for fine classification and measurement. This method effectively improves the accuracy of crack recognition and classification, especially the recognition effect of cracked cracks is significantly improved. Through hole filling and region analysis, the present invention can accurately distinguish cracked and non-cracked cracks, providing a clear and scientific basis for subsequent crack monitoring and evaluation.

[0024] 3. Aiming at the problem of insufficient data on cracked cracks in the public dataset, the present invention proposes a new method for identifying cracked cracks. This method performs hole filling and region analysis based on the binary image after crack segmentation. By counting the number of void regions and setting appropriate thresholds, it effectively distinguishes cracked and non-cracked cracks. This innovative method not only improves the accuracy of cracked crack recognition, but also provides a rich data basis for subsequent related research. In addition, the implementation of this method is simple and easy, and can obtain high-precision results in a short time, with broad application prospects.

[0025] 4. For the mixed crack area, the present invention proposes a comprehensive classification method that combines features such as crack morphology and size. This method first uses the circumscribed rectangle to draw the size features of the crack, and then distinguishes the small curved area cracks. Then, it calculates the ratio of the crack area to the circumscribed rectangle area to extract the morphological features of the crack, and further distinguishes the cracks similar to turtle shell flakes. Finally, it counts the number of intersection points through skeletonization to accurately distinguish the turtle shell cracks and conventional cracks. This comprehensive classification method not only improves the accuracy of crack classification but also provides a clear classification basis for subsequent crack monitoring and evaluation.

[0026] 5. Facing the complexity of mixed crack classification, the present invention proposes a comprehensive classification method that combines features such as crack morphology and size. This method uses means such as circumscribed rectangle drawing, area ratio calculation, and skeletonization to carefully distinguish small curved area cracks, cracks similar to turtle shell flakes, turtle shell type cracks, and conventional cracks. This comprehensive classification strategy not only enhances the flexibility of crack classification but also ensures the accuracy of classification, providing a scientific basis for subsequent crack monitoring, evaluation, repair, and reinforcement.

[0027] 6. In the measurement of the length and average width of conventional cracks, the present invention proposes a custom seed point growth method. This method records the vertex coordinates on the crack skeleton and uses the seed point growth algorithm to obtain the longest distance of the crack, thereby accurately positioning the actual length of the crack. At the same time, by calculating the ratio of the crack area to the skeleton length, the average width of the crack is obtained. This method not only improves the accuracy of crack measurement but also can automatically process a large amount of image data, update crack information in real time, and provide a timely basis for the health monitoring of the structure. In addition, the flexibility of the custom seed point growth method enables us to dynamically adjust parameters according to the characteristics of specific cracks to ensure the high precision and high reliability of the measurement results.

[0028] 7. By combining deep learning and traditional image processing techniques, the present invention proposes an interpretable classification and measurement algorithm for multi-type bridge crack diseases, effectively solving the limitations in the prior art, improving the accuracy and efficiency of crack detection, providing important data support for the safety assessment and maintenance of bridge structures, and providing a solid foundation for the safety guarantee of bridge structures.

[0029] 8. The detailed classification of the present invention helps engineers and researchers deeply understand the causes of different types of cracks and their impacts on structural safety. By taking more effective repair and reinforcement measures, the service life and safety of roads and buildings can be further improved. This comprehensive crack classification method has broad application prospects and important practical significance in engineering practice.

[0030] 9. The method of the present invention not only improves the accuracy of the classification of cracked cracks, but also significantly improves the working efficiency. The implementation process is simple and easy, and high-precision results can be obtained in a short time. In addition, this method improves the dataset, provides richer learning data for relevant segmentation and classification models, and further improves the detection accuracy of cracked cracks. This technological breakthrough helps to solve the difficulties in crack detection and provides strong support for the health monitoring of structures such as bridges. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The present invention will be further described below in conjunction with the drawings and embodiments: Figure 1 is the technical flow chart of the algorithm of the present invention; Figure 2 is an example of cracks in the public dataset; Figure 3 is an example of cracks in the self-built dataset; Figure 4 is an example of mixed cracks; Figure 5 is an example of regular cracks; Figure 6 are the binary original image, filled image, and segmented cavity image of non-cracked cracks of the present invention; Figure 7 are the binary original image, filled image, and segmented cavity image of cracked cracks of the present invention; Figure 8 is the binary image of cracked cracks of the present invention; Figure 9 is a schematic diagram of the cracked skeleton of the present invention; Figure 10 is a schematic diagram of the cracked crack skeleton and intersection annotation of the present invention; Figure 11 is a schematic diagram of the regular crack skeleton and intersection annotation of the present invention; Figure 12 is an example of the crack classification statistical chart of the present invention; Figure 13 is the algorithm flow chart of the custom seed point for measuring crack length of the present invention; Figure 14 is an example diagram of the classification and measurement results display of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0032] The technical solutions in the present invention will be further described below in conjunction with the drawings and embodiments: Embodiment 1 An interpretable classification and measurement algorithm for multi-type bridge crack diseases, comprising the following steps: Step1: Use a deep learning semantic segmentation network model to segment the bridge crack image to obtain a binary image of the crack; Step 2: Perform hole filling operation on the segmented binary crack image, then subtract the original binary image to obtain the void area. By counting the number of void areas and setting a threshold, distinguish between cracked cracks and non-cracked cracks; Step 3: For non-cracked cracks, further use traditional image processing methods. Through the connected component method, screen out each crack area, draw a circumscribed rectangle, calculate the ratio of the area of the circumscribed rectangle to the area of the region and the ratio of the crack area to the area of the circumscribed rectangle, and combine the number of intersection points of the crack skeleton to subdivide non-cracked cracks into curved small area cracks, cracked-like flaky cracks, regular cracks, and / or remaining cracked cracks; Step 4: For the classified regular cracks, extract their skeletons and the coordinates of each vertex of the skeleton. Use the custom seed point growth algorithm to obtain the longest distance of the crack as the crack length, and divide the crack area by the skeleton length to obtain the average width.

[0033] In this embodiment, the deep learning semantic segmentation network model in Step 1 is the U-Net semantic segmentation network, but it is not limited to U-Net. Other network models suitable for semantic segmentation such as FCN, DeepLab, and Seg-Net can also be used.

[0034] Furthermore, the hole filling operation in Step 2 refers to filling the voids in the segmented binary crack image to highlight the hollow area formed by the cracks, and identifying the characteristics of cracked cracks by comparing the differences in the images before and after filling.

[0035] Furthermore, the specific method for distinguishing between cracked cracks and non-cracked cracks in Step 2 is as follows: Perform hole filling on the segmented binary crack image to obtain the filled image, then subtract the original binary image from the filled image to obtain the void area image. Count the number of void areas. When the number of void areas is greater than or equal to the preset threshold, it is determined as a cracked crack; otherwise, it is determined as a non-cracked crack.

[0036] Furthermore, the steps for classifying non-cracked cracks using traditional image processing methods in Step 3 include: Step 3.1: Record the entire area S; Step 3.2: Screen out each crack area through the connected component method, draw a circumscribed rectangle for each crack, and record the area S1 of the circumscribed rectangle of each crack in the region; Step 3.3: Set a threshold threshold1. When S1 / S < threshold1, record this type of crack as a curved small area crack; Step 3.4: Set the threshold threshold2, and count the crack area S2 of the remaining cracks (i.e., the number of white pixels). When S2 / S1 < threshold2, this type of crack is recorded as a cracked sheet-like crack. Step 3.5: After obtaining the skeleton of the finally remaining cracks, count the number of intersection points S3, and set the threshold threshold3. When S3 > threshold3, this type of crack is recorded as a cracked crack; otherwise, it is recorded as a regular crack.

[0037] Furthermore, the connected domain method in Step 3.1 is to search for interconnected pixel regions by traversing the image and record the positions of the pixel points within the regions, and draw a circumscribed rectangle through the positions of the boundary pixel points.

[0038] Furthermore, the Zhang-Suen thinning algorithm is used to extract the crack skeleton in Step 3.3. This algorithm can accurately obtain the thin line structure of the cracks while maintaining the consistency and connectivity of the shape.

[0039] Furthermore, the steps of measuring the length and average width of regular cracks by the custom seed point growth method in Step 4 include: Step 4.1: Extract the skeleton of the regular crack and the coordinates of each vertex of the skeleton. Step 4.2: Determine the growth seed point and the region growth stop condition. By traversing other vertices through the seed point growth method, the longest distance is obtained, and this distance is the length of the regular crack. Step 4.3: Calculate the average width as the crack area divided by the skeleton length.

[0040] Furthermore, in Step 4.2, the growth seed point is selected as the vertex closest to the upper left corner of the image where the crack skeleton is located, and the growth stop condition is to stop when encountering the last vertex. The number of growth rounds at the stop represents the maximum length of the crack.

[0041] Furthermore, in Step 4.2, the determination of the growth seed point and the region growth stop condition is that the initial seed point is the vertex closest to the upper left corner of the image where the skeleton is located, and the growth stop condition is to stop when encountering the last vertex. The number of growth rounds at the stop represents the maximum length of the crack. During the growth process, when encountering another vertex, pause and count the number of growth rounds, that is, how many times it has grown. The number of growth rounds is the distance between these two vertices, and then continue to grow until the last vertex. The cumulative number of all growth rounds is the maximum length of this regular crack.

[0042] Furthermore, the bridge crack image in Step 1 is obtained by shooting with a drone, and the dataset used in the algorithm is a self-built dataset constructed by slicing the images taken by the drone on a specific bridge into a predetermined size.

[0043] Example 2 In another preferred embodiment, based on the above Embodiment 1, this embodiment will describe the technical solution of the present invention in detail in combination with specific embodiments.

[0044] Dataset preparation: The dataset used in this embodiment is a self-built dataset constructed by shooting a certain bridge with a drone Matrice 350 RTK; the captured bridge images are preprocessed and cut into slices of 448×448 pixel size for subsequent crack detection, classification and measurement.

[0045] Step 1: Crack image segmentation First, use the U-Net semantic segmentation network to segment the bridge crack images in the self-built dataset. The U-Net network, with its unique U-shaped structure, performs excellently in the field of image segmentation and is particularly suitable for the crack segmentation task in this embodiment. In the binary image obtained after segmentation, the crack area is white and the background is black.

[0046] Step 2: Classification of cracked and non-cracked cracks Perform a hole filling operation on the segmented binary crack image, and then subtract the original binary image to obtain obvious hole areas. Set a threshold according to the number of hole areas. When the number of holes is greater than the threshold, it is determined as a cracked crack; otherwise, it is determined as a non-cracked crack. In this embodiment, the threshold is set to 5, that is, when the number of holes is greater than 5, it is determined as a cracked crack.

[0047] Step 3: Further classification of non-cracked cracks For non-cracked cracks, further use traditional computer vision methods for detailed classification. The specific steps are as follows: 3.1. Record the area S of the entire crack area; 3.2. Screen out each crack area through the connected component method, draw a circumscribed rectangle for each crack, and record the area S1 of the circumscribed rectangle of each crack in the area; 3.3. Set a threshold threshold1. When S1 / S < threshold1, record this type of crack as a curved small area crack. In this embodiment, threshold1 is set to 0.1; 3.4. For the remaining cracks, set a threshold threshold2, count the crack area (i.e., the number of white pixels) and record it as S2. When S2 / S1 < threshold2, record this type of crack as a quasi-cracked sheet-like crack. In this embodiment, threshold2 is set to 0.5; 3.5. For the finally remaining cracks, after extracting the crack skeleton using the Zhang-Suen algorithm, count the number of intersection points S3. Set a threshold threshold3. When S3 is greater than threshold3, it is determined as a cracked-type crack (note that the determination of the cracked-type crack in this step is based on the number of intersection points, which is different from the determination method in step 2. However, here it is actually a distinction between regular cracks and complex cracked-type cracks. If the number of intersection points is small, it is a regular crack. The description of the cracked-type crack here is based on the previous process description and should actually be understood as a further distinction of the complex situation in the remaining cracks); otherwise, it is determined as a regular crack. In this embodiment, threshold3 is set to 10.

[0048] Step 4: Measurement of the length and average width of regular cracks For the classified regular cracks, use the custom seed point growth method to measure their length and average width. The specific steps are as follows: 4.1. Extract the regular crack area and extract the regular cracks separately through the largest connected region. 4.2. Use the Zhang-Suen thinning algorithm to extract the thinned skeleton of the crack image and record the coordinates of each vertex on the skeleton. 4.3. Select the top-leftmost point as the growth seed point. 4.4. Use the seed point growth algorithm to expand the growth seed point to other vertices, and set the growth stop condition as "stop when encountering a vertex". 4.5. By continuously repeating the growth process, traverse all vertices and record the longest growth round number to obtain the maximum length of the regular crack. 4.6. Calculate the average width as the crack area / skeleton length.

[0049] Embodiment 3 In another preferred embodiment, based on the above Embodiments 1 and 2, as Figure 1 shown, this embodiment will further elaborate on the technical solution of the present invention in combination with the accompanying drawings and specific embodiments.

[0050] Dataset preparation: The dataset used in this embodiment is a self-built dataset constructed by shooting a certain bridge with a drone Matrice 350 RTK; the captured bridge images are preprocessed and cut into slices of 448×448 pixel size for subsequent crack detection, classification, and measurement. The overall process is as Figure 1 shown.

[0051] 1. Classify cracked and non-cracked cracks using traditional image processing methods To distinguish between cracked and non-cracked fissures, in this embodiment, a traditional image processing method is used to classify them according to the characteristics of cracked and non-cracked fissures. To avoid the influence of the bridge background on the classification results, in this embodiment, a semantic segmentation model is first used to process the self-built dataset to obtain the segmentation result of the fissures, which is a binary image. In this embodiment, the U-Net semantic segmentation network is adopted, but it is not limited to this network, and any network model suitable for semantic segmentation can be used, such as FCN, DeepLab, Seg-Net, etc.

[0052] For cracked and non-cracked fissures, as Figure 2 shown, by comparison, it can be found that cracked fissures often present a multi-directional and irregular fissure pattern. As a result, the fissure area will be divided into several small regions of different sizes and independent of each other. The boundaries between these small regions are blurred, bringing complexity to the overall structure. Relatively speaking, conventional fissures and multi-type mixed fissures present different characteristics. Although these fissures may also enclose to form cavities, their number is often small and the coverage range is relatively limited. Therefore, in the process of fissure analysis and treatment, it is particularly important to identify and classify these two types of fissures.

[0053] Based on the analysis of the above fissure characteristics, this embodiment proposes an effective method for distinguishing cracked and non-cracked fissures based on traditional computer vision methods. Specifically, first, a hole filling operation is performed on the segmented binary fissure image, and then the original binary image is subtracted to obtain the obvious cavity area. As Figure 3 、 4 shown, it shows the binary image after segmentation of non-cracked and cracked fissures, the image after hole filling treatment, and the finally obtained cavity area image.

[0054] It can be seen from the experimental results that in non-cracked fissures, there are only 2 significant cavities, while in cracked fissures, the number of cavities is 7. This significant difference provides an effective basis for distinguishing between cracked and non-cracked fissures, that is, by appropriately setting thresholds to further distinguish these two fissure types. This method not only improves the accuracy of fissure analysis but also provides a basis for subsequent structural assessment and maintenance.

[0055] 1. Distinguish the mixed fissure area using the characteristics of various fissures In the non-cracked type, in addition to conventional fissures, there are also fissure areas with a mixture of conventional fissures and various fissures. The characteristics of each fissure in the mixed fissures have been described above and they are divided into four categories, namely, small curved area fissures, cracked-like flaky fissures, cracked fissures, and conventional fissures. In this embodiment, by analyzing the characteristics such as the size and shape of various fissures, the various fissures in the mixed fissures are further distinguished. The specific algorithm is shown in Table 1: Table 1

[0056] Since the area of the curved small-area cracks is small, in order to distinguish this type of cracks, in this embodiment, the connected component algorithm is used to find each crack area, and then the circumscribed rectangle of each crack area is obtained according to the boundary coordinate values of each area. By statistically calculating the ratio of the circumscribed rectangle area to the area of the region, the proportion of the crack in the region is judged. A threshold is set, and the cracks smaller than the threshold are determined to be curved small-area cracks and marked with a red box, while the cracks larger than the threshold are classified in the next round; for the crack-like scaly cracks, the proportion of this type of crack in the region is large, but its shape often appears curled, so it accounts for a relatively large proportion in the circumscribed rectangle area. In this embodiment, the ratio of the crack area of this type of crack to the circumscribed rectangle area is used to map the curled and divergent states of the crack, and a threshold is set to classify the crack-like scaly cracks. Those higher than the threshold are the crack-like scaly cracks and are marked with a yellow box, while those lower than the threshold are the scaly crack regions and the conventional regions for the last round of classification.

[0057] For the remaining scaly cracks and conventional cracks, in addition to being determined by the above-mentioned number of holes, they can also be determined by the number of intersection points. Scaly cracks often intersect horizontally and vertically, so the number of intersection points generated is also large, while conventional cracks are mostly single horizontal and vertical cracks, and even if there are a small number of branch intersection points, the number is not large. Based on this feature, in this embodiment, after extracting the crack skeleton, the intersection points are detected and the number of intersection points is recorded to distinguish scaly cracks and conventional cracks. Due to the irregular extension characteristics of cracks, for the extraction of the crack skeleton, the Zhang-Suen algorithm is selected in this embodiment. The Zhang-Suen skeleton extraction algorithm

[15] is an efficient image processing technology, especially suitable for the extraction of crack skeletons. Its advantage lies in being able to accurately obtain the thin line structure of the object while maintaining the shape consistency and connectivity. Through iterative processing steps, the algorithm effectively removes redundant pixels, thereby generating a refined skeleton, which not only reduces the computational complexity but also improves the accuracy of the results. In addition, the Zhang-Suen algorithm has strong robustness to noise and small-area interference and can stably extract crack skeletons in various environments, providing a reliable data basis for subsequent crack analysis and classification, such as Figure 8 、 9 is a schematic diagram of the binary image of scaly cracks and its skeleton, such as Figure 10 、 11 are respectively schematic diagrams of the skeletons of typical scaly cracks and conventional cracks and their intersection markings. It can be seen that the number of intersection points in scaly cracks is significantly higher than that in conventional cracks. Based on this feature, scaly cracks and conventional cracks can be effectively distinguished, and the scaly cracks are marked with a blue box, and the conventional cracks are marked with a green box.

[0058] Figure 12 is a bar chart that counts and plots the number of various types of cracks in the left - hand dataset ( Figure 12 shown on the right). By making detailed annotations for various types of cracks, not only can different types of cracks be clearly identified and classified, but also a solid foundation can be laid for subsequent statistical analysis. After completing the crack annotation, in - depth statistical analysis can be carried out on these data. This process is not just a simple count of numbers, but also includes the analysis of the nature of the cracks, such as the occurrence frequency of the cracks, the propagation speed, and the proportion of different types of cracks in the area, etc. These statistical results will provide important decision - making basis for subsequent maintenance, and can accurately evaluate the structural health status and potential risks of the bridge. In addition, the characteristic information of each crack, including its location, the length and width of the circumscribed rectangle, the shape, and the relationship with the surrounding structure, can be recorded and incorporated into the database later, which is convenient for a more comprehensive understanding of the crack distribution in this area and formulating more targeted repair and protection strategies.

[0059] 2. Measuring the length and average width of regular cracks using the self - defined seed - point growth method After classifying the regular cracks, in order to detect the length of the regular cracks, this embodiment proposes a method for skeleton extraction and length calculation of regular cracks, aiming to improve the accuracy and efficiency of crack analysis. First, extract the regular crack area. After separating the regular cracks by the largest connected region, use thinning algorithms such as Zhang - Suen to extract the thinned skeleton of the crack image. During this process, record the coordinates of each vertex on the crack skeleton. These vertices are important geometric feature points in the crack and can provide key data for the subsequent growth algorithm. Among all the extracted vertices, since the structure of regular cracks is relatively simple, in order to reduce the computational amount, this embodiment selects the point in the upper - left corner (i.e., the point closest to the left vertex) as the growth seed point. Then, use the seed - point growth algorithm to expand the growth seed point to other vertices. At this stage, set the growth stop condition as "stop when encountering a vertex". Specifically, when the seed point starts growing from one vertex until it reaches another vertex, the growth process will be terminated. This process will be executed by gradually checking the neighborhood pixels or adjacent vertices to ensure that each growth is along the connected part of the skeleton. Since the width of the skeleton area of regular cracks is only 1 pixel value, the number of growth rounds represents the distance length between two vertices. By continuously repeating the above growth process, all vertices can be traversed and the longest growth round can be recorded to obtain the maximum length of the regular crack. The average width obtained is recorded as crack area / skeleton length, and the specific process is as Figure 13 shown.

[0060] The above - mentioned method is to classify various types of cracks in the mixed cracks and measure the length and average width of regular cracks. Figure 14Examples of classification results and the lengths and average widths of regular cracks.

[0061] Example 4 In another preferred embodiment, on the basis of the above Examples 1, 2, and 3, in order to demonstrate the diversity and practicality of the technical solution of the present invention, in this embodiment, another semantic segmentation network FCN is used to segment bridge crack images, and the subsequent processing steps in Example 1 are followed for classification and measurement.

[0062] In step 1, the FCN network is used to segment bridge crack images in the self-built dataset. The FCN network, with its fully convolutional structure and effective fusion of multi-scale features, is also applicable to the crack segmentation task. The binary image obtained after segmentation is also used for subsequent crack classification and measurement.

[0063] The subsequent steps (steps 2 to 4) are exactly the same as those in Example 2 and will not be elaborated here.

[0064] Compared with the closest prior art, the present invention proposes a crack classification method based on the combination of traditional image processing technology and deep learning. By hole filling and counting the number of void regions, it effectively distinguishes between cracked and non-cracked cracks, improving the accuracy of classification. For non-cracked cracks, further classification is carried out using features such as crack morphology and size, achieving effective distinction of curved small-area cracks, quasi-cracked sheet-like cracks, cracked cracks, and regular cracks. In terms of measuring the length and average width of regular cracks, a custom seed point growth method is adopted, improving the accuracy and efficiency of measurement.

[0065] Example 5 In another preferred embodiment, on the basis of the above Examples 1 to 4, a computer-readable storage medium stores a computer program thereon. When the program is executed by a processor, it implements the interpretable classification and measurement algorithm for multi-type bridge crack diseases described in any one of the above Examples 1 to 4.

[0066] Example 6 In another preferred embodiment, on the basis of the above Example 5, an electronic device includes a processor and a storage medium. The storage medium stores the computer program described in Example 5, and when the processor executes the computer program, it implements the interpretable classification and measurement algorithm for multi-type bridge crack diseases described in any one of Examples 1 to 4.

[0067] In a preferred solution, the deep learning semantic segmentation network model in Step1 is a U-Net semantic segmentation network, but it is not limited to U-Net. Other network models suitable for semantic segmentation such as FCN (Fully Convolutional Networks), Deep Lab, and Seg-Net can also be used. The above settings can be selected according to specific task requirements and data characteristics. At the same time, in order to improve the generalization ability and segmentation accuracy of the model, technical means such as data augmentation and loss function optimization can be adopted during the training process.

[0068] In a preferred solution, the hole filling operation in Step2 refers to filling the holes in the segmented binary crack image to highlight the hollow area formed by the cracks, and identifying the characteristics of the cracked cracks by comparing the differences in the images before and after filling. The above settings effectively improve the accuracy of crack identification. Especially in complex backgrounds, the crack boundaries can be more clearly defined, providing a reliable data basis for subsequent crack type classification and severity assessment.

[0069] In a preferred solution, the specific method for distinguishing between cracked cracks and non-cracked cracks in Step2 is as follows: perform hole filling on the segmented binary crack image to obtain the filled image, then subtract the original binary image from the filled image to obtain the hole area image, and count the number of hole areas. When the number of hole areas is greater than or equal to a preset threshold, it is determined as a cracked crack; otherwise, it is determined as a non-cracked crack. The above settings can effectively distinguish different types of cracks, improve the accuracy and efficiency of crack identification. In addition, the crack classification can be further refined according to the characteristics such as the size and shape of the hole area, providing more detailed information support for subsequent crack treatment and repair.

[0070] In a preferred solution, the connected component method in Step3.1 searches for connected pixel regions by traversing the image and records the positions of the pixel points within the region, and draws a bounding rectangle through the positions of the boundary pixel points. The above settings can accurately identify and mark each connected region in the image, and at the same time effectively represent the positions and sizes of each region using the bounding rectangle, providing accurate geometric information for subsequent processing steps.

[0071] In a preferred solution, the Zhang-Suen thinning algorithm is used to extract the crack skeleton in Step3.3. This algorithm can accurately obtain the thin line structure of the crack while maintaining the consistency and connectivity of the shape. The above settings ensure the accuracy and stability of crack identification. On this basis, the crack skeleton is further optimized through morphological processing to reduce noise interference, laying a solid foundation for subsequent crack feature extraction and analysis.

[0072] In a preferred solution, in Step 4.2, the growth seed point is selected as the vertex closest to the upper left corner of the image in the crack skeleton, and the growth stop condition is to stop when encountering the last vertex. The number of growth rounds at the stop represents the maximum length of the crack. The above settings ensure the accuracy and efficiency of crack detection. By calculating and comparing the number of growth rounds with a preset threshold, it can be quickly determined whether the crack exceeds the safe range, providing a reliable basis for subsequent crack repair and structural safety assessment.

[0073] In a preferred solution, the determination of the growth seed point and the region growth stop condition in Step 4.2 are as follows: the initial seed point is the vertex closest to the upper left corner of the image in the skeleton, and the growth stop condition is to stop when encountering the last vertex. The number of growth rounds at the stop represents the maximum length of the crack. During the growth process, when encountering another vertex, it pauses, and the number of growth rounds is counted, that is, how many times it has grown. The number of growth rounds is the distance between these two vertices, and then it continues to grow until the last vertex. The cumulative number of all growth rounds is the maximum length of the conventional crack. The above settings can accurately track the crack path, effectively avoid misjudgment and omission, and improve the accuracy and efficiency of crack detection. In addition, this solution also has good adaptability and robustness, and can meet the crack detection requirements of different complexities and morphologies.

[0074] In a preferred solution, the bridge crack image in Step 1 is obtained by drone shooting, and the dataset used by the algorithm is a self-built dataset constructed by slicing the images taken by the drone on a specific bridge into a predetermined size. The above settings effectively improve the efficiency and accuracy of data collection, enabling the crack detection algorithm to more accurately identify and analyze the minor damages in the bridge structure, providing strong data support for bridge safety assessment and maintenance.

[0075] In summary, in view of the technical limitations in the field of bridge crack detection and assessment, especially the problems of inaccurate classification and measurement of multiple types of cracks and low efficiency, the present invention proposes an interpretable classification and measurement algorithm for multiple types of concrete crack diseases. This algorithm successfully combines the advantages of deep learning and traditional image processing technologies, bringing an innovative solution to the field of bridge crack detection.

[0076] First of all, the present invention uses deep learning for the preliminary identification and segmentation of cracks. This step can quickly locate and extract the crack area, laying a foundation for subsequent processing. Subsequently, combined with traditional image processing technologies, such as the connected component method, Zhang-Suen skeleton extraction algorithm, etc., for the fine classification and feature extraction of cracks. This combined method not only improves the accuracy of crack identification but also enhances the interpretability of the entire processing process.

[0077] For this special type of cracked fissures, the present invention proposes a new identification method. This method performs hole filling operations on the binary image after crack segmentation, and then subtracts the original binary image to generate multiple independent hollow regions. By counting the number of these hollow regions and setting appropriate thresholds, it is possible to effectively distinguish between cracked and non-cracked fissures, significantly improving the accuracy of cracked fissure classification.

[0078] In the measurement of crack length and average width, the present invention proposes a custom seed point growth method. By determining the growth seed points and the region growth stop conditions, relying on the crack skeleton and vertex information, this method achieves accurate measurement of crack length. This method not only has a high degree of automation but also ensures the accuracy of measurement, providing reliable data support for the safety assessment of bridge structures.

[0079] In addition, the present invention also proposes a comprehensive classification method that combines features such as crack morphology and size. This method can effectively distinguish various types of cracks, such as curved small-area cracks, quasi-cracked flaky cracks, cracked fissures, and conventional cracks. This classification method not only improves the accuracy of crack identification but also provides a clear classification basis for subsequent crack monitoring and assessment, helping engineers and researchers to deeply understand the causes of different types of cracks and their impacts on structural safety.

[0080] Aiming at the problem of insufficient collection of cracked fissures in existing public datasets, the present invention effectively overcomes the problem of complex crack classification through self-built datasets and innovative identification methods. This innovation not only enriches the crack dataset but also provides new ideas for the development of crack detection technology.

[0081] In summary, the present invention innovatively proposes multiple aspects, such as the innovative application of combining deep learning and traditional image processing methods, the new identification method for cracked fissures, the proposed custom seed point growth method for measuring cracks, and the comprehensive classification method. By deeply exploring crack features and combining traditional image processing techniques with deep learning algorithms, the present invention realizes the interpretable classification and accurate measurement of multi-type bridge crack diseases, providing strong support for the safety assessment and maintenance of bridge structures. This innovative achievement not only promotes the development of bridge crack detection technology but also provides a solid foundation for the safety guarantee of bridge structures.

[0082] In summary, aiming at the technical limitations in the field of bridge crack detection and assessment, especially the problems of inaccurate classification and measurement and low efficiency of multi-type cracks, the present invention proposes an interpretable classification and measurement algorithm for multi-type concrete crack diseases. This algorithm successfully integrates the advantages of deep learning and traditional image processing techniques, bringing an innovative solution to the field of bridge crack detection.

[0083] First, the present invention uses deep learning for the preliminary identification and segmentation of cracks. This step can quickly locate and extract the crack area, laying a foundation for subsequent processing. Subsequently, combined with traditional image processing techniques, such as the connected component method, Zhang-Suen skeleton extraction algorithm, etc., fine classification and feature extraction of cracks are carried out. This combined approach not only improves the accuracy of crack identification but also enhances the interpretability of the entire processing process.

[0084] For the special type of reticulated cracks, the present invention proposes a new identification method. This method performs hole filling operations on the binary image after crack segmentation, and then subtracts the original binary image to generate multiple independent hollow regions. By counting the number of these hollow regions and setting appropriate thresholds, it is possible to effectively distinguish between reticulated and non-reticulated cracks, significantly improving the accuracy of reticulated crack classification.

[0085] In the measurement of crack length and average width, the present invention proposes a custom seed point growth method. This method realizes the accurate measurement of crack length by determining the growth seed points and the region growth stop conditions, relying on the crack skeleton and vertex information. This method not only has a high degree of automation but also ensures the accuracy of the measurement, providing reliable data support for the safety assessment of bridge structures.

[0086] In addition, the present invention also proposes a comprehensive classification method that combines crack morphology, size and other features. This method can effectively distinguish various types of cracks, such as small curved area cracks, reticulated-like flaky cracks, reticulated cracks, and conventional cracks. This classification method not only improves the accuracy of crack identification but also provides a clear classification basis for subsequent crack monitoring and assessment, helping engineers and researchers to deeply understand the causes of different types of cracks and their impact on structural safety.

[0087] Aiming at the problem of insufficient collection of reticulated cracks in existing public datasets, the present invention effectively overcomes the problem of complex crack classification through self-built datasets and innovative identification methods. This innovation not only enriches the crack dataset but also provides new ideas for the development of crack detection technology.

[0088] In summary, the present invention innovatively proposes in multiple aspects, such as the innovative application of combining deep learning and traditional image processing methods, the new identification method for reticulated cracks, the proposed custom seed point growth method for measuring cracks, and the comprehensive classification method. By deeply exploring crack features and combining traditional image processing techniques with deep learning algorithms, the present invention realizes the interpretable classification and accurate measurement of multi-type bridge crack diseases, providing strong support for the safety assessment and maintenance of bridge structures. This innovative achievement not only promotes the development of bridge crack detection technology but also provides a solid foundation for the safety guarantee of bridge structures.

Claims

1. An interpretable classification and measurement algorithm for multiple types of bridge crack damage, characterized by: The following steps are involved: Step 1: Use the deep learning semantic segmentation network model to segment the bridge crack image and obtain a binary image of the crack; Step 2: Perform hole filling operation on the segmented binary crack image, then subtract the original binary image to obtain the hole area, and set the threshold according to the number of hole areas to distinguish between crack-type cracks and non-crack-type cracks; Step 3: For non-cracking type cracks, traditional image processing methods are further used to classify them by analyzing the shape and size characteristics of the cracks, including curved small area cracks, quasi-cracking sheet cracks, conventional cracks, and the remaining cracking type cracks are distinguished by the number of intersections; Step 4: For the classified common cracks, use the custom seed point growth method to measure their length and average width.

2. The explainable classification and measurement algorithm for multiple types of bridge cracks according to claim 1 is characterized by: The deep learning semantic segmentation network model in the Step 1 is a U-Net semantic segmentation network, but is not limited to U-Net. Network models suitable for semantic segmentation such as FCN, DeepLab, and Seg-Net may also be used.

3. The interpretable classification and measurement algorithm for multiple types of bridge cracks according to claim 1 is characterized by: The hole filling operation in Step 2 refers to filling the holes in the segmented binary crack image to highlight the hollow area formed by the crack, and identifying the characteristics of the crack-type crack by comparing the difference between the images before and after filling.

4. The explainable classification and measurement algorithm for multiple types of bridge crack damage according to claim 3 is characterized in that: The specific method for distinguishing between tortoise-type cracks and non-turtle-type cracks in the Step 2 is: fill the holes in the segmented binary crack image to obtain a filled image, then subtract the filled image from the original binary image to obtain a hole area image, count the number of hole areas, and when the number of hole areas is greater than or equal to a preset threshold, it is determined to be a tortoise-type crack, otherwise it is determined to be a non-turtle-type crack.

5. The explainable classification and measurement algorithm for multiple types of bridge crack damage according to claim 1 is characterized in that: The step of classifying non-cracking type cracks using the traditional image processing method in Step 3 includes: Step 3.1: Filter out each crack area through the connected domain method, draw the circumscribed rectangle, calculate the ratio of the circumscribed rectangle area to the regional area, and set the threshold to distinguish between curved small regional cracks and other cracks; Step 3.2: For the remaining cracks, calculate the ratio of the crack area to the area of ​​the circumscribed rectangle and set a threshold to distinguish between cracks and flake cracks; Step 3.3: For the remaining cracks, extract the crack skeleton, count the number of intersections, and set a threshold to distinguish between tortoise-type cracks and conventional cracks.

6. The interpretable classification and measurement algorithm for multiple types of bridge crack damage according to claim 4 is characterized by: The connected domain method in Step 3.1 is to search for interconnected pixel areas by traversing the image and record the positions of the pixels in the area, and draw a circumscribed rectangle based on the positions of the boundary pixels.

7. The interpretable classification and measurement algorithm for multiple types of bridge cracks according to claim 4 is characterized by: The crack skeleton extraction in Step 3.3 adopts the Zhang-Suen thinning algorithm, which can accurately obtain the fine line structure of the crack while maintaining the consistency and connectivity of the shape.

8. The explainable classification and measurement algorithm for multiple types of bridge crack damage according to claim 1 is characterized in that: The steps of measuring the length and average width of conventional cracks by using the custom seed point growth method in Step 4 include: Step 4.1: Extract the skeleton of conventional cracks and the coordinates of each vertex of the skeleton; Step 4.2: Determine the growth seed point and the regional growth stop condition, and traverse other vertices through the seed point growth method to obtain the longest distance, which is the length of the conventional crack; Step 4.3: Calculate the average width as the crack area divided by the frame length.

9. The interpretable classification and measurement algorithm for multiple types of bridge cracks according to claim 7 is characterized by: In the Step 4.2, the growth seed point is selected as the vertex in the crack skeleton that is closest to the upper left corner of the image. The growth stop condition is to stop when the last vertex is encountered. The number of growth rounds when it stops represents the maximum length of the crack. The point at the upper left corner of the skeleton. The growth stopping condition is to stop when encountering a vertex. The number of growth rings represents the distance between the two vertices.

10. The interpretable classification and measurement algorithm for multiple types of bridge crack damage according to claim 1 is characterized by: The bridge crack image in Step 1 is obtained by drone photography, and the dataset used by the algorithm is a self-built dataset constructed by slicing with a predetermined size after the drone photographs the specific bridge.

Images