A parallel intelligent quantification method and device for concrete bridge defects
Through object detection network model and deep learning technology, combined with image preprocessing and connectivity domain screening, the problem of low accuracy of micro-crack quantization is solved, and efficient micro-crack quantization and visualization is achieved, which is suitable for micro-crack detection of concrete bridges.
Patent Information
- Application Number
- CN202210620526.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-02
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2042-06-02
AI Technical Summary
The prior art is difficult to accurately quantify and visualize micro-cracks with widths less than 1 mm, and the traditional detection methods are inefficient and costly, and cannot meet the needs of information timeliness.
The target detection network model (such as YOLOv5) is used in combination with deep learning, and the accurate quantification and visualization of micro-cracks are achieved through technologies such as image preprocessing, region threshold segmentation, connection domain screening and noise removal.
It improves the accuracy and efficiency of micro-crack quantization and visualization, can effectively remove noise and extract pixel-level crack information, which is suitable for simultaneous quantization and visualization of multiple images.
Smart Images

Figure CN114897873B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of bridge defect detection quantification and image processing, and in particular to a parallel intelligent quantification method and device for concrete bridge defects. Background Art
[0002] Most of the bridges built in the world are concrete bridges. During use, bridges need to be inspected regularly to facilitate the formulation of corresponding maintenance and repair measures. Cracks are one of the main diseases of concrete bridges and have become the most important content of concrete bridge inspection and maintenance. Traditional manual inspection and quantification methods have disadvantages such as high cost and low efficiency, and can no longer meet the needs of information timeliness. In recent years, many scholars have used the advantages of deep learning in image recognition and used convolutional neural networks to locate and classify bridge diseases, and have achieved good results.
[0003] In existing research, deep learning is used to identify and detect cracks, but deep learning-related ideas have not been used alone to achieve the task of quantifying microcracks (crack width less than 1 mm). In addition, the accurate quantification and visualization of microcracks (width less than 1 mm) in images also needs to be improved. Quantifying cracks is very important in actual engineering applications. Summary of the invention
[0004] The purpose of the present invention is to provide a parallel intelligent quantification method for concrete bridge defects in view of the fact that the prior art currently lacks the idea of quantifying microcracks using deep learning alone, as well as the problem of low accuracy in quantifying microcracks. The method uses a target detection network model to extract crack areas, and quantifies and visualizes cracks through digital image processing using deep learning, thereby improving the accuracy of quantification and visualization of microcracks and improving work efficiency.
[0005] In order to achieve the above object, the technical solution adopted by the present invention is:
[0006] A parallel intelligent quantification method for concrete bridge defects, comprising:
[0007] Step S1, identifying and cutting out the crack area from the original image to obtain a crack area image;
[0008] Step S2, graying, median filtering and contrast-limited adaptive histogram equalization processing are performed on the crack area image;
[0009] Step S3, based on the processed crack region image, divide it into several continuous regions, take the product of the maximum crack width and the diagonal length in the region as the threshold of the most pixels occupied by the crack, perform binarization processing on the image in the region with the threshold in the histogram, and obtain a binary image of the crack region after binarization processing of each region;
[0010] Step S4, removing the painted mark noise of the crack area binary image;
[0011] Step S5, converting the crack region binary image with the painted mark noise removed into a connected domain image, and filtering the connected domains to obtain a filtered connected domain image;
[0012] Step S6, based on the filtered connected domain graph, quantify and connect the cracks to obtain a complete crack binary graph;
[0013] Step S7, obtaining the contour coordinates of the complete crack binary image, performing a visualization operation on the cracks, and obtaining a crack visualization image;
[0014] Step S8, based on the position information of the cropped image in the original image, restore the crack visualization image to the original image.
[0015] As a preferred solution of the present invention, the target detection network model YOLOv5 is used to identify the crack area.
[0016] As a preferred solution of the present invention, step S3 further includes dividing the crack region image into a plurality of continuous square regions and non-square regions using the short side of the crack region image as the side length.
[0017] As a preferred embodiment of the present invention, step S4 includes:
[0018] Step S41, extracting blue graffiti mark information in the HSV color space to obtain a mask of the blue graffiti mark;
[0019] Step S42, using image morphological opening operation to remove noise from the mask and then perform pixel inversion processing to obtain a processed mask;
[0020] Step S43, multiplying the crack region binary image and the processed mask.
[0021] As a preferred solution of the present invention, in step S5, the binary image of the crack area with the painted mark noise removed is converted into a connected domain image through a connected domain labeling algorithm, and connected domains with a width-to-length ratio of the top 60% and an area greater than 20 pixels are screened out.
[0022] As a preferred solution of the present invention, in step S6, quantification includes crack length quantification, and the formula is as follows:
[0023]
[0024] in, is the total number of connected domains, , For the The area of a connected domain, No. The length of the connected domain, for The actual distance represented by each pixel in the axial direction, for The actual distance represented by each pixel in the axis direction.
[0025] As a preferred solution of the present invention, in step S6, quantification includes crack width quantification, and the formula is as follows:
[0026]
[0027] in, is the total number of connected domains, , For the The area of a connected domain, No. The length of the connected domain, for The actual distance represented by each pixel in the axial direction, for The actual distance represented by each pixel in the axis direction.
[0028] As a preferred solution of the present invention, in step S7, the findContours function in the OpenCV library is used to obtain the contour coordinates of the crack binary image, and then the contour is drawn by the drawContours function according to the contour coordinates.
[0029] A parallel intelligent quantification device for concrete bridge defects comprises a memory and a processor, wherein the memory is used to store at least one program, and the processor is used to load at least one program to execute a parallel intelligent quantification method for concrete bridge defects.
[0030] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:
[0031] This method has strong adaptability and robustness, and can effectively remove noise from crack images on the surface of concrete structures. At the same time, more pixel-level crack information can be extracted through local threshold segmentation. Secondly, the detection network is combined to locate the crack position, and the digital image processing method can be used to accurately quantify and visualize the microcracks in the image. Finally, multiple images can be quantified and visualized at the same time, which improves the efficiency of microcrack quantification and visualization. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 It is a schematic flow chart of the method of the present invention.
[0033] Figure 2 It is a result diagram of the present invention based on each step. DETAILED DESCRIPTION
[0034] The present invention will be described in detail below in conjunction with the accompanying drawings.
[0035] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0036] Example 1
[0037] A parallel intelligent quantification device for concrete bridge defects comprises a memory and a processor, wherein the memory is used to store at least one program, and the processor is used to load at least one program to execute a parallel intelligent quantification method for concrete bridge defects.
[0038] like Figure 1 As shown in the flowchart, a parallel intelligent quantification method for concrete bridge defects includes the following steps:
[0039] Step S1, identifying and cutting out the crack area from the original image to obtain a crack area image;
[0040] The original image resolution is 8688×4888. Through the deep learning-based target detection network model trained in this dataset, such as the YOLOv5 model, the detection box of the crack is obtained. The coordinates of the upper left vertex of the bounding box are (1500, 97) and the coordinates of the lower right vertex are (3325, 8080). Figure 2 (a) As shown. The crack area image is cropped out according to the detection frame on the original image to obtain the area image with cracks retained, as shown in Figure 2 (b) shown.
[0041] Step S2, graying, median filtering and contrast-limited adaptive histogram equalization processing are performed on the crack area image;
[0042] Specifically, the crack region image is preprocessed, including grayscale, median filtering and contrast-limited adaptive histogram equalization (CLAHE). The cropped crack region image is a three-channel color image. The information of these three channels is redundant for the subsequent crack quantization operation and will only increase the amount of calculation. Therefore, the three-channel image is first converted into a single-channel grayscale image by formula (1).
[0043] Gray=0.30R+0.59G+0.11B (1)
[0044] The grayscaled image still contains a lot of noise, because the surface of the concrete bridge itself is not smooth, but full of various undulations. These undulations will appear as shadows, salt and pepper noise, etc. when photographed under light. Therefore, the grayscaled image is first filtered using a median filter. In this embodiment 1, the kernel size of the filter is set to 5×5. When traversing each pixel on the image, the kernel will move on the entire image, and the median of the 25 pixels will be used as the value of the pixel. The image after grayscale and median filtering is shown in the figure. Figure 2 (c) is shown. Then the image after median filtering is processed using the limited contrast adaptive histogram equalization function in the OpenCV library to suppress noise while enhancing image contrast. The result is shown in Figure 2 (d) shown.
[0045] Step S3, based on the processed crack area image, divide it into several continuous areas, take the product of the maximum crack width and the diagonal length in the area as the threshold of the most pixels occupied by the crack, and use the threshold to binarize the image in the area in the histogram, and obtain a binary image of the crack area after binarization of each area, which specifically includes:
[0046] Step S31, as Figure 2 As shown in (d), the size of the processed crack area image is 7983 × 1825. The image is divided from top to bottom into multiple square areas and the remaining non-square areas with the short side of the image as the side length.
[0047] Step S32, count the grayscale distribution of all pixels in each square area. The side length of the square area is 1825. In our data set, the number of pixels occupied by the width of the crack is at most 12 pixels. Therefore, in each square area, theoretically, the number of pixels occupied by the cracks distributed along the diagonal of the square is the largest, and its value is ; Further, the number of pixels in the non-square area is calculated separately.
[0048] Step S33, counting the grayscale histogram of each square area, and in the grayscale histogram of each area The corresponding grayscale value is used as the threshold to perform image binarization processing; similarly, the non-square area is subjected to image binarization processing separately.
[0049] Step S34, all square areas and non-square areas are spliced to obtain a complete crack area binary image containing crack pixels, and the result is as follows: Figure 2 (e) shown.
[0050] Step S4, removing the painted mark noise of the crack region binary image, specifically includes:
[0051] Step S41, extracting blue graffiti mark information in the HSV color space to obtain a mask of the blue graffiti mark;
[0052] Step S42: Since the extracted blue painted mark mask also contains other speckle noises, which are much smaller than the area of the painted mark, the mask is denoised using image morphological opening operation, i.e., the image is first eroded and then expanded, leaving only the painted mark part, and then pixel inversion processing is performed to obtain the processed mask;
[0053] Step S43, multiplying the crack region binary image and the processed mask to remove the graffiti mark noise, the result is as follows: Figure 2 (f) shown.
[0054] Step S5, converting the crack region binary image with the painted mark noise removed into a connected domain image, and filtering the connected domains to obtain a filtered connected domain image;
[0055] Specifically, it can be seen from the crack region binary image of the painted mark noise that there are still many non-crack noises in the image. In order to better deal with these non-crack noises, firstly, the crack binary image is converted into the corresponding connected domain image using the connected domain marking algorithm. The connected domain image is a marked image formed by numbering the connected regions in the foreground of the binary image from top to bottom and from left to right, starting from 1; secondly, the marked image is filtered by connected domains, including the following steps:
[0056] Step S51, counting the number of pixels contained in each connected domain and calculating the area of each connected domain; since the connected domain belonging to the crack is approximately linear, the point-like connected domain, that is, the connected domain with an area less than 20 pixels, is first removed.
[0057] Step S52, calculating the length of each connected domain by extracting the skeleton of the connected domain;
[0058] Step S53, by calculating the ratio of the area of the connected domain to the length of the connected domain, the width of each connected domain is obtained, and further the ratio of the length to the width of each connected domain, that is, the width-to-length ratio is obtained;
[0059] Step S54: From the perspective of the crack morphology, the length of the connected domain belonging to the crack is generally much greater than the width, so the connected domain with a smaller ratio of short side to long side is more likely to be a connected domain belonging to the crack. In the crack connected domain graph, all connected domains are sorted from small to large according to the width-to-length ratio of the connected domain, and the first 60% of the connected domains are taken to replace the cracks. The selected connected domains can maintain the characteristics of the crack morphology, and using these connected domains to quantify and visualize the cracks is the most accurate choice.
[0060] Step S6, based on the filtered connected domain graph, quantify and connect the cracks to obtain a complete crack binary graph;
[0061] Specifically, crack quantification refers to calculating the length and average width of cracks. In our dataset, the image resolution is 8688×4688, the focal length of the camera is 35mm, and the shooting distance is 50cm. Taking the direction of the short side of the image as the x-axis and the direction of the long side of the image as the y-axis, we can get the actual distance represented by each pixel in the y-axis direction of the image. , the actual distance represented by each pixel in the x-axis direction of the image The total number of connected domains after screening in this example image is N=968. The area of a connected domain is , length is .
[0062] (1) The formula for quantifying crack length is:
[0063] (2)
[0064] The lengths of the 968 connected domains after screening are accumulated to obtain the total length of all connected domains. , and then the actual distance represented by each pixel in the vertical direction of the image Multiply to get the actual length of the crack .
[0065] (2) The formula for quantifying crack width is:
[0066] (3)
[0067] Since the shape of the connected domain after screening is approximately a line segment, the width of each tiny connected domain is approximately equal to the ratio of the area of the connected domain to the length of the connected domain.
[0068] The average width of the crack connection domain is The actual average width of the crack is equal to the average width of the entire connected domain multiplied by the actual distance represented by each pixel in the horizontal direction of the image ,Right now .
[0069] Furthermore, the fracture connectivity domains are connected:
[0070] The filtered connected domain image contains some intermittent connected domain fragments. In order to visualize the cracks, these connected domains need to be connected. Select the longest connected domain from the filtered connected domains, and start from this connected domain to continuously connect upward and downward according to the principle of the shortest distance, directly connecting to the top connected domain and the bottom connected domain. After the connection is completed, a complete binary image of the crack is obtained, as shown in the following figure. Figure 2 (h) shown.
[0071] Step S7, obtaining the contour coordinates of the complete crack binary image, performing a visualization operation on the cracks, and obtaining a crack visualization image;
[0072] Specifically, the contour coordinates of the crack binary image are obtained according to the findContours() function in the OpenCV library, and then the contour is drawn in the crack area image according to the obtained contour coordinates through the drawContours() function to realize the visualization of the crack in the cropped image. The result is as follows: Figure 2 (i) shown.
[0073] Step S8, based on the position information of the cropped image in the original image, the crack visualization image is restored to the original image. The result is as follows: Figure 2 (j) shown.
[0074] In summary, the present invention first obtains a binary image containing crack pixels by cropping the image, image preprocessing and image binarization, and then removes the painted marks and non-crack noise by mask filtering and connected domain filtering respectively, and obtains the filtered connected domain to achieve the maximum maintenance of the crack morphology; based on the filtered connected domain, the cracks are quantified to obtain the average width and length of the cracks. Finally, the crack connected domain is connected and visualized and restored to the original image; accurate quantification and visualization of microcracks in the image are achieved, and multiple images can be quantified and visualized at the same time to improve efficiency. The result allows us to observe the crack damage and damage data more intuitively.
[0075] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A parallel intelligent quantification method for concrete bridge defects, characterized in that: include: Step S1, identifying and cutting out the crack area from the original image to obtain a crack area image; Step S2, graying, median filtering and contrast-limited adaptive histogram equalization processing are performed on the crack area image; Step S3, based on the processed crack region image, divide it into several continuous regions, take the product of the maximum crack width and the diagonal length in the region as the threshold of the most pixels occupied by the crack, perform binarization processing on the image in the region with the threshold in the histogram, and obtain a binary image of the crack region after binarization processing of each region; Step S4, removing the painted mark noise of the binary image of the crack area; Step S5, converting the crack region binary image with the painted mark noise removed into a connected domain image, and filtering the connected domains to obtain a filtered connected domain image; Step S6, based on the filtered connected domain graph, quantify and connect the cracks to obtain a complete crack binary graph; Step S7, obtaining the contour coordinates of the complete crack binary image, performing a visualization operation on the cracks, and obtaining a crack visualization image; Step S8, based on the position information of the cropped image in the original image, restore the crack visualization image to the original image.
2. The parallel intelligent quantification method for concrete bridge defects according to claim 1 is characterized in that: The target detection network model YOLOv5 is used to identify crack areas.
3. The parallel intelligent quantification method for concrete bridge defects according to claim 1 is characterized in that: Step S3 further includes dividing the crack region image into a plurality of continuous square regions and non-square regions using the short side of the crack region image as the side length.
4. The parallel intelligent quantification method for concrete bridge defects according to claim 1 is characterized in that: Step S4 includes: Step S41, extracting blue graffiti mark information in the HSV color space to obtain a mask of the blue graffiti mark; Step S42, using image morphological opening operation to remove noise from the mask and then perform pixel inversion processing to obtain a processed mask; Step S43, multiplying the crack region binary image and the processed mask.
5. The parallel intelligent quantification method for concrete bridge defects according to claim 1 is characterized in that: In step S5, the crack region binary map with the graffiti mark noise removed is converted into a connected domain map by a connected domain labeling algorithm, and connected domains with a width-to-length ratio of the first 60% and an area greater than 20 pixels are screened out.
6. According to the method for parallel intelligent quantification of concrete bridge defects in claim 1, in step S6, the quantification includes crack length quantification, and the formula is as follows: in, is the total number of connected domains, , For the The area of a connected domain, No. The length of the connected domain, for The actual distance represented by each pixel in the axial direction, for The actual distance represented by each pixel in the axis direction.
7. According to the method for parallel intelligent quantification of concrete bridge defects in claim 1, in step S6, the quantification includes crack width quantification, and the formula is as follows: in, is the total number of connected domains, , For the The area of a connected domain, No. The length of the connected domain, for The actual distance represented by each pixel in the axial direction, for The actual distance represented by each pixel in the axis direction.
8. According to the parallel intelligent quantification method for concrete bridge defects as described in claim 1, in step S7, the findContours function in the OpenCV library is used to obtain the contour coordinates of the crack binary image, and then the contour is drawn by the drawContours function according to the contour coordinates.
9. A parallel intelligent quantification device for concrete bridge defects, characterized in that: The method comprises a memory and a processor, wherein the memory is used to store at least one program, and the processor is used to load the at least one program to execute the parallel intelligent quantification method for concrete bridge defects according to any one of claims 1 to 8.
