A digital detection method for surface cracks of concrete segments

Through the full coverage field of view and crack detection model equipped with robotic arms and industrial cameras, the problems of low efficiency and poor accuracy of traditional detection methods are solved, and the rapid and accurate detection of cracks on the surface of concrete pipe sheets is achieved, and the efficiency of quality control is improved. and precision.

CN118037713BActive Publication Date: 2024-11-26SUZHOU CITY YILVJIAN TECH CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202410358693.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-03-27
Publication Date
2024-11-26
Estimated Expiration
2044-03-27

AI Technical Summary

Technical Problem

The detection of cracks on the surface of traditional concrete pipe sheets relies on artificial visual inspection, which is inefficient, limited in accuracy, and is difficult to adapt to large sizes or different lighting conditions, resulting in inaccurate detection results.

Method used

A full-coverage field of view based on robotic arms and industrial cameras is used to build a crack detection model, extract crack features through re-parameter convolution module and robust downsampling layer, and generate detection reports to achieve automated, fast and accurate crack detection.

Benefits of technology

It improves the detection speed and accuracy, enhances the adaptability to different types of pipe sheets and different lighting conditions, solves the problem of large-size pipe sheet measurement errors, and improves the quality control efficiency and accuracy in the production process of concrete pipe sheets.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118037713B_ABST
    Figure CN118037713B_ABST
Patent Text Reader

Abstract

The present invention relates to the technical field of concrete segment quality inspection, and solves the technical problems of low efficiency and large error of traditional inspection methods, low coverage of inspection indicators, and low level of overall process informatization. In particular, it relates to a digital inspection method for surface cracks of concrete segments, including: building a full-coverage field of view based on a robotic arm and an industrial camera; constructing a crack detection model for detecting surface cracks of concrete segments; extracting crack skeletons of cracks based on predicted label images output by the crack detection model, and generating morphological information of cracks according to the crack skeletons; calculating the actual crack length and average width of the cracks; and generating an inspection report. The present invention satisfies the requirements for efficient, accurate and automated inspection of surface cracks of concrete segments, and not only improves the speed and accuracy of inspection, but also enhances the adaptability of the system to different segment sizes and different lighting conditions. The application of this method will greatly improve the quality control efficiency and accuracy in the production process of concrete segments.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of concrete segment quality detection, and in particular to a digital detection method for surface cracks of a concrete segment. Background Art

[0002] In industrial production, the quality inspection of concrete segments is a key link to ensure the safety and durability of buildings. Traditional surface crack detection of concrete segments mostly relies on manual visual inspection, which is not only inefficient but also easily affected by the skill level and subjective judgment of operators, resulting in limited accuracy and reliability of the test results. In addition, manual inspection faces huge challenges when dealing with large-sized or large numbers of concrete segments in a high-intensity industrial production environment. Therefore, the development of an automated, efficient and accurate method for detecting surface cracks in concrete segments has become an urgent need in the industry.

[0003] In the prior art, although there have been attempts to assist crack detection by using digital image processing technology, these methods are often still limited in algorithm processing speed and accuracy, and cannot fully cover large-sized segments. In addition, under different lighting conditions, the adaptability and stability of segment crack detection are seriously affected, making it difficult to ensure the accuracy of the detection results. Therefore, a new technical solution is needed that can comprehensively, quickly and accurately detect cracks on the surface of concrete segments. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides a digital detection method for surface cracks of concrete pipe segments, which solves the technical problems of low efficiency, large errors, low coverage of detection indicators, and low level of overall process informatization of traditional detection methods, so as to meet the needs of efficient, accurate and automated detection of surface cracks of concrete pipe segments.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: a method for digitally detecting cracks on the surface of a concrete segment, the method comprising the following steps:

[0006] S1. Build a full-coverage field of view based on a robotic arm and an industrial camera to automatically collect surface image data of concrete segments in all directions and generate a detection data set divided into a training set and a test set;

[0007] S2. Construct a crack detection model for detecting cracks on the surface of concrete segments, and use a training set and a test set to train and evaluate the performance of the crack detection model;

[0008] S3, using the surface image data of the concrete segment to be detected as the input of the crack detection model to obtain an output predicted label image;

[0009] S4, judging whether there are cracks on the surface of the concrete segment to be inspected according to the predicted label image;

[0010] If yes, proceed to step S5;

[0011] If not, directly proceed to the judgment of the next predicted label image;

[0012] S5, extracting a crack skeleton of the crack based on the predicted label image, and generating morphological information of the crack according to the crack skeleton, the morphological information including the pixel-level length of the crack, the pixel-level average width, and the crack distribution direction;

[0013] S6. Calculate the actual crack length and average width of the crack according to the morphological information and the scale transformation coefficient;

[0014] S7. Generate a test report based on the morphological information as well as the actual crack length and average width.

[0015] Furthermore, in step S1, the specific process includes the following steps:

[0016] S11, setting the center of the inner curved surface of the concrete segment as the starting point of the acquisition route, and identifying the serial number of the concrete segment to obtain the corresponding model;

[0017] S12, dividing the surface of the concrete segment into multiple small areas according to its model, and planning a specific data collection route and robot arm movement for each small area;

[0018] S13, setting the shooting distance between the lens of the industrial camera and the surface of the concrete segment to 300 mm;

[0019] S14, collecting surface image data of the concrete segment by adjusting the exposure time of the industrial camera to simulate different lighting conditions and saving the data in RAW format;

[0020] S15, dividing the collected surface image data into a training set and a test set in a ratio of 8:2;

[0021] S16. Use Labelme software to create a label file for instance segmentation based on the training set and test set, and place the label file corresponding to the original data into the corresponding training set and test set folders.

[0022] Furthermore, in step S2, the crack detection model adopts a U-shaped structure, which consists of two parts: a crack feature extraction structure and a decoding structure. The feature extraction structure is composed of a heavy parameter convolution module and a robust downsampling layer stacked together, which can effectively extract crack features. The decoding structure is composed of a heavy parameter convolution module and an upsampling layer stacked together for reconstructing the crack position segmentation label.

[0023] Furthermore, the construction of the re-parameterized convolution module is divided into a training phase and an inference phase, specifically including:

[0024] Two branches are constructed during the training phase. The main convolution branch uses a standard 3*3 convolution layer. The weights are initialized and updated in each training cycle. The main convolution branch is responsible for capturing the basic features of the concrete surface.

[0025] The sub-convolution branch consists of a 1*1 convolution layer and a smooth convolution kernel, which is used to reduce the dimension of the feature map to reduce the amount of calculation. The smooth convolution kernel is then used to smooth the feature map to enhance the ability to capture details.

[0026] In the inference stage, the reparameterization technology is used to merge the main convolution branch and the secondary convolution branch into a single 3*3 convolution layer. The outputs of the main convolution branch and the secondary convolution branch are linearly combined to integrate the basic feature extraction capability of the main convolution branch and the detail enhancement capability of the secondary convolution branch.

[0027] Furthermore, the expression of the re-parameterized convolution module is:

[0028] ;

[0029] in, is the weight of the folded convolution used for inference; , , They are the convolution kernel of the 3*3 convolution layer of the main convolution branch during training, the convolution kernel of the 1*1 convolution layer of the secondary convolution branch, and the smoothing convolution kernel of the secondary convolution branch; and are the number of input channels and output channels of the convolution respectively; , It is the convolution kernel parameter position indicator.

[0030] Furthermore, the robust downsampling layer is a spatially variable and channel-variable content-aware filter, which is used to flexibly adjust the weight of the filter according to the local features and global information of the feature map, thereby suppressing unnecessary noise while maintaining important features of the image.

[0031] Furthermore, the expression of the content-aware filter is:

[0032] ;

[0033] In the above formula, , Represents each pixel position The surrounding spatial location set; For each pixel position The weight of is the input feature map.

[0034] Furthermore, in step S5, the specific process includes the following steps:

[0035] S51, calculating the distance from each pixel in the predicted label image to the nearest 0 pixel;

[0036] S52, drawing a circle with each pixel as the center and the distance to the nearest 0 pixel as the radius to extract the pixel points located on the central axis of the crack;

[0037] S53, replacing the pixel value of the pixel point located on the central axis of the crack from 1 to twice the distance from the point to the nearest 0 pixel point, to obtain the pixel points constituting the central axis of the crack, that is, the crack skeleton;

[0038] S54, extracting the starting point and the end point of the central axis of the crack, and calculating the pixel-level distance between the two points as the pixel-level length of the crack;

[0039] S55. Counting the average pixel value of all pixel points on the central axis of the crack is the pixel-level average width of the crack.

[0040] Furthermore, in step S6, the specific process includes the following steps:

[0041] S61, fixing a calibration ruler on the surface of the concrete segment, taking photos with an industrial camera lens at a distance of 300 mm from the calibration ruler on the surface of the concrete segment, and obtaining a plurality of calibration images;

[0042] S62, using the calibration graph to calculate a scale transformation coefficient for representing an actual distance value corresponding to a single pixel in the image;

[0043] S63. Calculate the actual crack length and average width of the crack according to the scale transformation coefficient.

[0044] Furthermore, in step S63, the calculation formula for the actual crack length and average width is:

[0045] ;

[0046] ;

[0047] in, and Represents the actual average width and length of the crack; and Represents the average width and length of cracks at the pixel level in the predicted label image; Represents the scaling coefficient.

[0048] By means of the above technical solution, the present invention provides a method for digitally detecting cracks on the surface of a concrete segment, which has at least the following beneficial effects:

[0049] 1. The present invention can overcome the difficulties of large surface area of ​​concrete segments, long manual detection time, large detection error, etc. Based on digital detection technology, it has the advantages of fast detection speed, high accuracy, strong adaptability, etc. It can be widely used in quality control and maintenance management of concrete segments, which not only improves the speed and accuracy of detection, but also enhances the adaptability of the system to different segment sizes and different lighting conditions. The application of this method will greatly improve the quality control efficiency and accuracy in the production process of concrete segments.

[0050] 2. The present invention establishes a full coverage field of view for concrete segments, and covers all areas of large-size segments by carrying industrial cameras on a robotic arm, thereby ensuring the comprehensiveness and accuracy of surface data collection of large-size segments, and solving the problems of the prior art that large-size segments are difficult to measure and have large measurement errors. The present invention significantly enhances its adaptability to different types of segments, and has important practical significance and broad application prospects for improving the quality control of concrete segments in industrial production through automated data collection and processing procedures.

[0051] 3. The crack detection model constructed by the present invention can preserve more image details in dim light or with short exposure time compared with traditional RGB images, which solves the limitation of existing methods on high requirements for detection conditions; the algorithm is lightweight and efficient, which improves the accuracy and responsiveness of concrete segment surface crack detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0053] Figure 1 It is a flow chart of the digital detection method of concrete segment surface cracks of the present invention;

[0054] Figure 2 A schematic diagram of a fully covered field of view constructed by the present invention;

[0055] Figure 3 A schematic diagram of the network structure of the crack detection model of the present invention;

[0056] Figure 4 This is a framework diagram of the re-parameterized convolution module of the present invention. DETAILED DESCRIPTION

[0057] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods, so that the implementation process of how the present application uses technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly.

[0058] Please refer to Figure 1-Figure 4 , showing a specific implementation of this embodiment, this embodiment builds an industrial camera carrying platform based on a robotic arm to achieve field coverage of large-sized concrete segments, automatically collect segment surface data; produce a detection data set in RAW format; construct a crack detection model with a lightweight crack segmentation algorithm CrackNet to detect cracks on the surface of concrete segments in real time; the segment surface data is inferred by the crack segmentation algorithm CrackNet, and if cracks exist, detailed information on the crack morphological characteristics is extracted; and a segment detection report is automatically generated. The digital detection method for concrete segment surface cracks proposed in the present invention further improves the degree of automation of production based on digital detection technology, can overcome the difficulties of large surface area of ​​concrete segments, long manual detection time, and large detection errors, has the advantages of fast detection speed, high accuracy, and strong adaptability, and can be widely used in quality control and maintenance management of concrete segments.

[0059] Please refer to Figure 1 This embodiment proposes a method for digitally detecting cracks on the surface of a concrete segment, which includes the following steps:

[0060] S1. Build a full coverage field of view based on a robotic arm and industrial camera, such as Figure 2 As shown, the surface image data of the concrete segment is automatically collected in all directions and a detection data set is generated, which is divided into a training set and a test set. In this step, the construction of a full coverage field of view can be mainly divided into steps such as space planning, equipment configuration, and route planning, which are as follows:

[0061] For space planning, the scanning field of view analysis and calculation are performed according to the maximum size of the concrete segment, the equipment volume and the site environment. The site environment is required to be a general industrial indoor environment, and direct sunlight should be avoided as much as possible. The ground is required to be relatively flat and the working path should be free of grooves. According to the motion range of the robotic arm used and the effective distance of the industrial camera, the system footprint is reasonably set, and the field of view coverage is calculated in detail to achieve full coverage of the field of view of the largest-sized concrete segment.

[0062] Equipment configuration, based on the analysis results combined with information such as the system footprint and equipment attributes, uses an I-type fixed tooling on the robotic arm, installs industrial cameras on both arms, and reserves space for other data acquisition equipment that may need to be installed in the future, completing the construction of the scanning field of view.

[0063] Route planning: divide the surface of the segment into multiple small areas according to the concrete segment model, plan specific data collection routes and robot arm movements for each area, and ensure that each area can be fully covered by the industrial camera without omissions. The optimal shooting distance between the industrial camera lens and the segment surface is 300mm. When planning the path, it is necessary to consider possible obstacles around the robot arm to ensure that the path planning can avoid obstacles and ensure safe operation; at the same time, it is necessary to try to ensure that the shooting distance between the industrial camera lens and the segment surface is 300mm. Specifically, in this step, the detailed implementation steps are as follows:

[0064] S11. The center of the inner curved surface of the concrete segment is set as the starting point of the data collection route, and the number of the concrete segment is identified to obtain the corresponding model. When performing path planning, the starting point is set as the center of the inner curved surface of the concrete segment. This center point is usually the number position of the segment, which provides a unique identification for the segment. The number of the center of the inner curved surface of the segment is collected to match the segment model to realize the planning of the data collection path. The step of collecting the segment number is also crucial for subsequent data processing and test report generation, because it ensures the traceability and manageability of data and specific concrete segments, which helps quality control and subsequent maintenance.

[0065] S12. Divide the surface of the concrete segment into multiple small areas according to the model of the concrete segment, and plan a specific data collection route and robot arm action for each small area; when planning the path, divide the surface of the concrete segment into five surfaces (the contact surface between the concrete segment and the ground is not collected), and divide each surface of the concrete segment into multiple small areas for more detailed scanning to ensure that each small area can be scanned. Among them, for each surface of the concrete segment, a scanning path is defined to cover all areas of the surface. The following strategies can be adopted:

[0066] The area divided by each face of the concrete segment adopts a serpentine scanning path: starting from the starting point, scanning along the horizontal direction, then when reaching the boundary, moving to the next row and changing the scanning direction, and so on, until reaching the end point.

[0067] Based on the path planning results, the path points for the robot's movement are generated. Each path point contains the position and posture information of the robot, as well as the position and orientation information of the industrial camera.

[0068] S13. Set the shooting distance between the industrial camera lens and the surface of the concrete segment to 300 mm. During the acquisition process, the shooting distance between the industrial camera lens and the surface of the concrete segment should be controlled at 300 mm, so as to ensure the clarity and accuracy of the image.

[0069] S14. The surface image data of the concrete segment is collected and saved in RAW format by adjusting the exposure time of the industrial camera to simulate different lighting conditions. When collecting the surface image data of the concrete segment, the exposure time of the industrial camera should be adjusted at each shooting posture to simulate different lighting conditions. A short exposure time can simulate insufficient lighting on the surface of the concrete segment. In this case, the brightness of the image is low, which may pose a challenge to the detection capability of the crack detection model. A moderate exposure time can simulate suitable lighting conditions on the surface of the concrete segment. In this case, the brightness of the image is moderate, which is conducive to the detection of the crack detection model.

[0070] Among them, the exposure time of the industrial camera is adjusted to be short. The insufficient lighting is simulated because the surface area of ​​the concrete pipe segment is large. When the shooting distance between the industrial camera lens and the surface of the concrete pipe segment is controlled at 300mm, the shooting field of view is small. Multiple shots are required to fully collect all surface area data. In actual inspection, in order to ensure the inspection efficiency, the exposure time of each data collection is short and the imaging effect is dark. Therefore, it is necessary to simulate a data set containing dark light data under similar working conditions to train the crack detection model in a targeted manner.

[0071] By adjusting the exposure time of the industrial camera to normal, data with appropriate lighting conditions on the surface of the concrete segment is collected. This is because the surface image data contains data with normal lighting, which allows the crack detection model to learn more background and crack characteristics, and improves the robustness of the crack detection model under different working conditions. The data should be saved in RAW format, which can save image details when the light is insufficient or the exposure time is insufficient to ensure the performance of the crack detection model.

[0072] S15. Divide the collected surface image data into a training set and a test set in a ratio of 8:2; the collected surface image data should be divided into a training set and a test set, where the training set accounts for 80% of all samples and the test set accounts for 20% of all samples. The training set is used to train the crack detection model in the subsequent steps, and the test set is used to evaluate the performance of the crack detection model in the subsequent steps, so that the accuracy and robustness of the crack detection model can be effectively tested.

[0073] S16. Use Labelme software to create instance segmentation label files based on the training set and the test set, and place the original data corresponding to the label files into the corresponding training set and test set folders. The original data is the surface image data collected in step S14. For the collected surface image data of the concrete segment, use Labelme software to create instance segmentation labels. Accurate segmentation labels are particularly important for subsequent model learning of crack features. Place the original data corresponding to the label files into the corresponding training set and test set folders.

[0074] When making instance segmentation labels, name the crack "crack". According to the shape of the crack, use the mouse to draw a polygon on the outer contour of the crack so that the polygon just covers the crack. Labelme will automatically generate a json file containing the coordinate positions and marks of each polygon point. Convert the label's json file into a png file with a background pixel value of 0 and a crack pixel value of 1 as the label.

[0075] The present invention establishes a full coverage field of view for concrete segments, and covers all areas of large-size segments by carrying an industrial camera on a robotic arm, thereby ensuring the comprehensiveness and accuracy of surface data collection of large-size segments, and solving the problems of difficult measurement and large measurement errors in the prior art for large-size segments. The present invention significantly enhances its adaptability to segments of different models, and has important practical significance and broad application prospects for improving the quality control of concrete segments in industrial production through automated data collection and processing procedures.

[0076] S2. Construct a crack detection model for detecting cracks on the surface of concrete segments, and use a training set and a test set to train and evaluate the performance of the crack detection model; Figure 3 The figure shows the CrackNet structure of the crack detection model. The crack detection model adopts a U-shaped structure, which consists of a crack feature extraction structure and a decoding structure. The feature extraction structure is composed of a heavy parameter convolution module and a robust downsampling layer stacked together to effectively extract crack features. The decoding structure is composed of a heavy parameter convolution module and an upsampling layer stacked together to reconstruct the crack position segmentation label. The structural composition of the crack detection model takes into account the diverse forms of cracks and the detection effects under different lighting conditions to adapt to the different forms and lighting conditions of cracks, ensuring detection speed and accuracy.

[0077] like Figure 4 As shown in the figure, the construction of the heavy parameter convolution module is divided into a training phase and an inference phase, which specifically includes: constructing two branches in the training phase, the main convolution branch adopts a standard 3*3 convolution layer, initializing the weights and updating them in each training cycle, and the main convolution branch is responsible for capturing the basic features of the concrete surface; the auxiliary convolution branch is composed of a 1*1 convolution layer and a smooth convolution kernel, which is used to reduce the dimension of the feature map to reduce the amount of calculation, and then the smooth convolution kernel is used to smooth the feature map to enhance the ability to capture details;

[0078] In the inference stage, the reparameterization technology is used to merge the main convolution branch and the secondary convolution branch into a single 3*3 convolution layer, and the outputs of the main convolution branch and the secondary convolution branch are linearly combined to fuse the basic feature extraction capability of the main convolution branch and the detail enhancement capability of the secondary convolution branch. It should be noted that the inference stage is the process of steps S3, S4 and S5 in this embodiment.

[0079] By innovatively using a linear combination of two branches in the training phase, the main convolution branch is consistent with the original 3*3 convolution layer, and the secondary convolution branch consists of a 1*1 convolution layer and a smooth convolution kernel. In the inference phase, the reparameterization technology is used to merge the two branches into a single 3*3 convolution layer, aiming to improve the crack detection model's ability to process crack depth features in practical applications and enhance its robustness to different lighting conditions. The specific expression of the reparameterized convolution module is:

[0080] ;

[0081] in, is the weight of the folded convolution used for inference; , , They are the convolution kernel of the 3*3 convolution layer of the main convolution branch during training, the convolution kernel of the 1*1 convolution layer of the secondary convolution branch, and the smoothing convolution kernel of the secondary convolution branch; and are the number of input channels and output channels of the convolution respectively; , It is the convolution kernel parameter position indicator.

[0082] The robust downsampling layer is a spatially variable and channel-variable content-aware filter that is used to flexibly adjust the weight of the filter according to the local features and global information of the feature map, suppressing unnecessary noise while maintaining the important features of the image. The robust downsampling layer is innovatively designed to replace traditional downsampling methods such as maximum pooling and nearest neighbor interpolation. By predicting the content-aware filter at each position and channel of the feature map, it can adaptively combine global and local features, suppress interference noise generated by low-light environments, and keep the details of the feature map from being lost during downsampling. Specifically:

[0083] Local information Generating function by weight and conditional indexes Get global information It uses global pooling operation Extract global information and generate functions through weights The calculation process is as follows:

[0084] ;

[0085] in, is the channel, width, and height position of the pixel, Represents a conditional index pixel set .

[0086] For each pixel position Weight By combining local information and global information The specific calculation formula is as follows:

[0087] ;

[0088] in, , Represents each pixel position The collection of surrounding spatial locations.

[0089] The content-aware filter can be expressed as follows:

[0090] ;

[0091] In the above formula, , Represents each pixel position The surrounding spatial location set; For each pixel position The weight of is the input feature map.

[0092] Use the calculated weights For the input feature map Weighted, get the output feature map ,The weighting process can be regarded as a low-pass filtering operation, which effectively smoothes the noise and retains important features.

[0093] When training the crack detection model, the input of the crack detection model is the RAW image of the training set, and the output of the crack detection model is the crack position segmentation label. The loss corresponding to the label of the RAW image of the training set is used to guide the update of the algorithm parameters. The performance evaluation of the crack detection model can be achieved using existing technical means and will not be described in detail here.

[0094] The crack detection model constructed by the present invention uses RAW images as input, which can preserve more image details than traditional RGB images in dim light or with short exposure time, thus solving the limitation of existing methods that have high requirements on detection conditions. It can realize the simultaneous completion of the acquisition of segment surface data and algorithm reasoning by the industrial camera carried by the robotic arm, and the crack detection model has the characteristics of lightweight and high efficiency. The crack detection model is designed for slender cracks, and consists of a robust downsampling layer and a heavy parameter convolution module, which improves the accuracy and responsiveness of concrete segment surface crack detection.

[0095] S3. Use the surface image data of the concrete segment to be detected as the input of the crack detection model to obtain the output predicted label image; specifically, use the collected surface image data of the concrete segment to be detected as the input of the crack detection model after preprocessing, and the crack detection model performs inference to obtain the predicted label image. The label image output by the crack detection model contains only two types of pixels, crack and background, wherein the pixel value of the crack is 1 and the pixel value of the background is 0. The predicted label image here reflects the presence or absence of cracks on the surface of the concrete segment and the location information of the cracks.

[0096] S4. Determine whether there are cracks on the surface of the concrete segment to be detected based on the predicted label image; if so, proceed to step S5; if not, directly proceed to the judgment of the next predicted label image; in this judgment process, it can be achieved based on the predicted label image output by step S3. A pixel value of 1 indicates that there are cracks on the surface of the concrete segment to be detected, and a pixel value of 0 indicates that there are no cracks.

[0097] S5, extracting the crack skeleton of the crack based on the predicted label image, and generating the morphological information of the crack based on the crack skeleton, the morphological information includes the pixel-level length of the crack, the pixel-level average width and the crack distribution direction; for the predicted label image with cracks, first extract the crack skeleton, i.e. the crack centerline of the crack, calculate the length and width of the crack using the crack skeleton, and then obtain the pixel-level length of the crack, the pixel-level average width and the crack distribution direction and other morphological information through statistical analysis. Therefore, in step S5, the specific process includes the following steps:

[0098] S51, calculating the distance from each pixel in the predicted label image to the nearest 0 pixel;

[0099] S52, drawing a circle with each pixel as the center and the distance to the nearest 0 pixel as the radius to extract the pixel points located on the central axis of the crack;

[0100] S53, replacing the pixel value of the pixel point located on the central axis of the crack from 1 to twice the distance from the point to the nearest 0 pixel point, to obtain the pixel points constituting the central axis of the crack, that is, the crack skeleton;

[0101] S54, extracting the starting point and the end point of the central axis of the crack, and calculating the pixel-level distance between the two points as the pixel-level length of the crack;

[0102] S55. Counting the average pixel value of all pixel points on the central axis of the crack is the pixel-level average width of the crack.

[0103] Among them, when extracting the crack skeleton, first, for each pixel point in the predicted label image where the crack exists, the crack detection model outputs the distance to the nearest 0 pixel point. Obviously, the distance to the 0 pixel point is 0; then a circle is drawn with each pixel as the center and the distance to the nearest 0 pixel point as the radius. If both sides of the circle touch the 0 pixel point, it means that the pixel point is located on the central axis of the crack; the pixel value on the central axis of the crack is replaced from the original 1 to twice the distance from the point to the nearest 0 pixel point (that is, the crack width of the pixel point); through the above method, the pixel points that constitute the central axis of the crack can be obtained, and these pixel points constitute the skeleton of the crack.

[0104] When calculating the average pixel-level width of the crack, the original value 0 of the pixel point located on the central axis of the crack has been replaced by twice the distance from the point to the nearest 0 pixel point. This new value represents the width of the crack at that point. By counting the values ​​of all the pixel points located on the central axis of the crack (that is, the crack width of each point) and calculating their average value, the pixel-level average width of the crack can be obtained. The specific calculation method is as follows:

[0105] ;

[0106] In the above formula, represents the average pixel-level width of the crack; k represents the total number of pixels located on the center axis of the crack; Represents the crack width of a single pixel point located on the centerline of the crack (i.e., twice the distance from the point to the nearest 0 pixel point).

[0107] When calculating the pixel-level length of the crack, the starting point and the end point of the crack centerline are extracted and the distance between the two points is calculated. This distance is in pixels and represents the pixel-level length of the crack.

[0108] S6, calculating the actual crack length and average width of the crack according to the morphological information and the scale transformation coefficient; specifically, multiplying the calculated crack morphological information by the scale transformation coefficient to obtain the actual crack length and average width, and storing the actual crack length, average width, and crack distribution direction for subsequent analysis and processing. Therefore, in step S6, the specific process includes the following steps:

[0109] S61. Fix a calibration ruler on the surface of the concrete segment, take photos with an industrial camera lens at a distance of 300 mm from the surface of the concrete segment, and obtain multiple calibration images. Specifically, select a calibration ruler with a known size and clear scale, such as a calibration ruler with millimeter scale, and take photos with an industrial camera lens at a distance of 300 mm from the surface of the segment, and obtain multiple calibration images.

[0110] S62. Use the calibration image to calculate the scale transformation coefficient used to represent the actual distance value corresponding to a single pixel in the image. Specifically, each calibration image needs to be processed first, such as denoising, grayscale, binarization, etc., and then the correspondence between the pixels in the calibration image and the scale on the calibration ruler is measured by image processing software to obtain the scale transformation coefficient, which represents the actual distance value corresponding to a single pixel in the image. The unit of the scale transformation coefficient is mm / pixel. In this step, the measurement accuracy needs to be ensured, and the accuracy is improved by taking the average value of multiple measurements.

[0111] When calculating the scale transformation coefficient, the industrial camera is taken at a distance of 300 mm from the surface of the segment. The shooting angle of the industrial camera is perpendicular to the plane where the surface of the concrete segment is located. The scale transformation coefficient is calculated as follows:

[0112] ;

[0113] ;

[0114] in, represents the scale transformation coefficient; n is the total number of calibration images; represents the scale transformation coefficient calculated from the i-th calibration image; Represents the actual length shown by the calibration ruler on the i-th calibration diagram (unit: mm); Represents the number of pixels occupied by the length shown by the calibration ruler on the i-th calibration image.

[0115] S63, calculating the actual crack length and average width of the crack according to the scale transformation coefficient; when calculating the actual crack length and average width, the average width and length at the pixel level in the predicted label image are multiplied by the scale transformation coefficient to obtain the actual average width and length value of the crack, and store them. The specific calculation method is as follows:

[0116] ;

[0117] ;

[0118] in, and Represents the actual average width and length of the crack (in mm); and Represents the average width and length of cracks at the pixel level in the predicted label image (in pixels); Represents the scaling coefficient.

[0119] S7. Generate a test report based on morphological information, actual crack length and average width. Specifically, the above method can obtain the output results of the crack detection model and various crack attribute results of all areas of the entire concrete segment. This information will serve as the basis for the subsequent generation of the test report; after obtaining the various attribute results of the concrete segment, these results are converted into a defect information table according to relevant production standards and specifications. This information table should include all key crack information, the crack location map on the segment surface, and the evaluation and analysis of the cracks based on this information;

[0120] Finally, a test report is generated, which contains basic information such as the concrete segment number, test time, and defect information table, which can clearly reflect the crack situation and the impact of the crack on the concrete segment. At the same time, the report should also include repair suggestions for the cracks to guide subsequent production optimization.

[0121] This embodiment can digitally detect cracks on the surface of concrete segments, solve the problems of low quality inspection efficiency of concrete segments in industrial production and subjective errors caused by human operation, improve the accuracy and efficiency of the measurement process, and further enhance the degree of digitalization of production.

[0122] The present invention requires no human intervention in the entire process of digital detection of cracks on the surface of concrete segments, except for loading and unloading of concrete segments, thereby improving the degree of automation and reducing the demand for human resources. It uses digital technology and algorithms to accurately calculate the actual size and shape of cracks, provide reliable quality assessment, and reduce the subjectivity and errors of human visual judgment. It automatically generates a test report and matches the segment model by numbering, thereby enhancing the traceability and manageability of data, and facilitating quality control and subsequent maintenance.

[0123] The present invention provides a new concrete segment surface crack detection solution by combining advanced robotic arm technology, industrial camera and specially designed crack segmentation algorithm CrackNet. The present invention uses a robotic arm equipped with an industrial camera to construct a set of full coverage field of view of the segment surface; collects images of the concrete segment surface to produce a concrete segment surface detection data set covering various practical application scenarios; develops a lightweight and efficient concrete segment surface crack segmentation algorithm CrackNet, which combines a U-shaped structure, a robust downsampling layer and a heavy parameter convolution module to accurately identify and segment cracks on the surface of the concrete segment in real time; realizes accurate mapping of pixels in the image to actual distances; further, the collected data is processed using the CrackNet algorithm to extract morphological information of crack morphological features; combined with the algorithm processing results and the specific information of the crack morphology, a quality inspection report of the concrete segment is automatically generated to provide a basis for subsequent quality control and maintenance management.

[0124] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, so the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0125] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the above embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0126] The above implementation methods have been described in detail. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A digital detection method for surface cracks of concrete segments, It is characterized in that The method comprises the following steps: S1. Build a full-coverage field of view based on a robotic arm and an industrial camera to automatically collect surface image data of concrete segments in all directions and generate a detection data set divided into a training set and a test set. The specific process includes the following steps: S11, setting the center of the inner curved surface of the concrete segment as the starting point of the acquisition route, and identifying the serial number of the concrete segment to obtain the corresponding model; S12, dividing the surface of the concrete segment into multiple small areas according to its model, and planning a specific data collection route and robot arm movement for each small area; S13, setting the shooting distance between the lens of the industrial camera and the surface of the concrete segment to 300 mm; S14, collecting surface image data of the concrete segment by adjusting the exposure time of the industrial camera to simulate different lighting conditions and saving the data in RAW format; S15, dividing the collected surface image data into a training set and a test set in a ratio of 8:2; S16. Use Labelme software to create a label file for instance segmentation based on the training set and the test set, and place the label file corresponding to the original data into the corresponding training set and test set folders; S2. Construct a crack detection model for detecting cracks on the surface of concrete segments, and use a training set and a test set to train and evaluate the performance of the crack detection model; The crack detection model adopts a U-shaped structure, which consists of a crack feature extraction structure and a decoding structure. The feature extraction structure is composed of a heavy parameter convolution module and a robust downsampling layer stacked together to effectively extract crack features. The decoding structure is composed of a heavy parameter convolution module and an upsampling layer stacked together to reconstruct crack position segmentation labels. The construction of the re-parameterized convolution module is divided into a training phase and an inference phase, which specifically includes: Two branches are constructed during the training phase. The main convolution branch uses a standard 3*3 convolution layer. The weights are initialized and updated in each training cycle. The main convolution branch is responsible for capturing the basic features of the concrete surface. The secondary convolution branch consists of a 1*1 convolution layer and a smooth convolution kernel, which is used to reduce the dimension of the feature map to reduce the amount of calculation. The smooth convolution kernel is then used to smooth the feature map to enhance the ability to capture details. In the inference stage, the reparameterization technology is used to merge the main convolution branch and the secondary convolution branch into a single 3*3 convolution layer. The outputs of the main convolution branch and the secondary convolution branch are linearly combined to integrate the basic feature extraction capability of the main convolution branch and the detail enhancement capability of the secondary convolution branch. The expression of the re-parameterized convolution module is: ; in, is the weight of the folded convolution used for inference; , , They are the convolution kernel of the 3*3 convolution layer of the main convolution branch during training, the convolution kernel of the 1*1 convolution layer of the secondary convolution branch, and the smoothing convolution kernel of the secondary convolution branch; and are the number of input channels and output channels of the convolution respectively; , It is the convolution kernel parameter position indicator; S3, using the surface image data of the concrete segment to be detected as the input of the crack detection model to obtain an output predicted label image; S4, judging whether there are cracks on the surface of the concrete segment to be inspected according to the predicted label image; If yes, proceed to step S5; If not, directly proceed to the judgment of the next predicted label image; S5, extracting a crack skeleton of the crack based on the predicted label image, and generating morphological information of the crack according to the crack skeleton, the morphological information including the pixel-level length of the crack, the pixel-level average width, and the crack distribution direction; S6. Calculate the actual crack length and average width of the crack according to the morphological information and the scale transformation coefficient; S7. Generate a test report based on the morphological information and the actual crack length and average width.

2. The detection method according to claim 1, It is characterized in that The robust downsampling layer is a spatially variable and channel-variable content-aware filter, which is used to flexibly adjust the weight of the filter according to the local features and global information of the feature map, thereby suppressing unnecessary noise while maintaining important features of the image.

3. The detection method according to claim 2, It is characterized in that The expression of the content-aware filter is: ; In the above formula, , Represents each pixel position The surrounding spatial location set; For each pixel position The weight of is the input feature map.

4. The detection method according to claim 1, It is characterized in that In step S5, the specific process includes the following steps: S51, calculating the distance from each pixel in the predicted label image to the nearest 0 pixel; S52, drawing a circle with each pixel as the center and the distance to the nearest 0 pixel as the radius to extract the pixel points located on the central axis of the crack; S53, replacing the pixel value of the pixel point located on the central axis of the crack from 1 to twice the distance from the point to the nearest 0 pixel point, to obtain the pixel points constituting the central axis of the crack, that is, the crack skeleton; S54, extracting the starting point and the end point of the central axis of the crack, and calculating the pixel-level distance between the two points as the pixel-level length of the crack; S55. Counting the average pixel value of all pixel points on the central axis of the crack is the pixel-level average width of the crack.

5. The detection method according to claim 1, It is characterized in that In step S6, the specific process includes the following steps: S61, fixing a calibration ruler on the surface of the concrete segment, taking photos with an industrial camera lens at a distance of 300 mm from the calibration ruler on the surface of the concrete segment, and obtaining a plurality of calibration images; S62, using the calibration graph to calculate a scale transformation coefficient for representing an actual distance value corresponding to a single pixel in the image; S63. Calculate the actual crack length and average width of the crack according to the scale transformation coefficient.

6. The detection method according to claim 5, It is characterized in that In step S63, the calculation formula for the actual crack length and average width is: ; ; in, and Represents the actual average width and length of the crack; and Represents the average width and length of cracks at the pixel level in the predicted label image; Represents the scaling coefficient.

Citation Information

Patent Citations

  • Concrete dam surface crack detection method based on pixel-level deep learning

    CN116485717A

  • Detection method for pixel-level concrete crack based on deep learning

    CN117291902A

  • Method and device for automatically drawing structural cracks and precisely measuring widths thereof

    US20200364849A1

  • Method and apparatus for identifying concrete crack based on video semantic segmentation technology

    US20210319547A1