A method for constructing apparent crack dataset and extracting its width based on deep learning

Abstract

The present application provides a method for constructing and extracting the apparent crack data set based on deep learning. The method splits the real cracks based on the idea of ​​splitting the cracks into two, effectively solving the problem of crack manufacturing, improving the efficiency of crack manufacturing, and maintaining a high degree of crack restoration; the method relies on a machine vision apparent crack manufacturing and splicing device to solve the problem of crack width and angle control, and adopts the control variable method to obtain multiple groups of crack image data more efficiently and accurately, improving the efficiency of crack data set construction; the method proposes a scheme for adding crack interference factors and a method for making interference factors, expanding the crack data set, and effectively enhancing the anti-interference ability of the total model for crack width extraction; the method adopts a variety of deep learning classification algorithm models, and uses the weighted mean of the multi-model classification value after denoising as the crack width, reducing the error rate and improving robustness.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine vision apparent crack recognition, and in particular relates to a method for constructing an apparent crack data set and extracting its width based on deep learning. Background Art

[0002] With the continuous increase in the number and mileage of existing highway bridges and tunnels, the pressure on highway bridge and tunnel maintenance has increased dramatically year by year. Traditional manual inspection can no longer meet the needs of future maintenance. Given the industry's urgent need for rapid and intelligent detection technology, it has become increasingly important to solve the problem of rapid perception and diagnosis of highway diseases.

[0003] The apparent crack intelligent detection technology based on machine vision has emerged. Its purpose is to use machines to replace human eyes to identify apparent diseases. Its means is to record disease information through cameras and extract disease results through artificial intelligence algorithms. Among them, machine vision crack recognition technology based on deep learning algorithms is widely used in bridge and tunnel inspection.

[0004] However, there are two major difficulties in the current crack identification technology based on deep learning algorithms:

[0005] On the one hand, there is the problem of constructing a crack dataset. The existing methods for constructing training datasets are mainly divided into on-site collection of crack images and indoor test equipment to create cracks for shooting. The former is expensive and inefficient, while the latter often uses mechanical or non-mechanical means, but has problems such as complex sample preparation process, difficulty in simulating real cracks, or difficulty in controlling crack width.

[0006] On the other hand, existing methods have difficulties in extracting apparent crack width and have low overall accuracy.

[0007] Therefore, it is urgent to propose a method for constructing an apparent crack dataset and extracting its width based on deep learning, to efficiently establish a crack width sample set and to improve the accuracy of crack width identification based on deep learning algorithms. Summary of the invention

[0008] In order to solve the problems existing in the prior art, improve the efficiency and quality of constructing a deep learning sample set of machine vision apparent crack width, and improve the accuracy of machine vision apparent crack width extraction, the present invention provides a method for constructing an apparent crack data set and extracting width based on deep learning, the method comprising:

[0009] Step 1: Take a high-definition grayscale image of the crack, extract the relative coordinates of the crack in the image, connect the relative coordinates in sequence to restore the crack edge strike line, stretch the strike line to construct the crack opening surface and the crack complementary surface.

[0010] Step 2: After making upper and lower crack morphology base membranes respectively according to the generated three-dimensional crack opening surface and the crack complementary surface, the upper and lower crack templates are cast through the molding container.

[0011] Step 3: Make sure the cracks on the two templates are in relative positions. After determining the required crack width and crack angle, move the upper and lower crack templates toward each other and merge to form a crack.

[0012] Step 4: Set the distance between the crack acquisition system and the crack within the working distance range, select multiple working distances at fixed intervals for acquisition, and use the control variable method to acquire multiple sets of crack image data.

[0013] Step 5: Classify the crack images and perform data enhancement on the images to construct a crack width dataset. According to the accuracy of the test set, different weights are assigned to different models to form a summary model of the crack width extraction algorithm.

[0014] Step 6: Input the crack images taken in the actual project into the crack width extraction algorithm summary model, use multiple crack width classification algorithm models respectively to obtain the crack corresponding category width, calculate the weighted mean, which is the final extracted crack width.

[0015] The present invention has the following beneficial effects:

[0016] (1) The present invention splits the real crack into two parts based on the idea of ​​splitting the crack into two parts, and casts the upper and lower crack templates through molding containers respectively, and then piece the templates together to construct the crack according to the idea of ​​combining the two into one, which effectively solves the problem of crack manufacturing, improves the efficiency of crack manufacturing, and maintains a high degree of crack restoration.

[0017] (2) The present invention relies on a machine vision apparent crack manufacturing and splicing device to solve the problem of crack width and angle control. At the same time, the control variable method is used to more efficiently and accurately obtain crack image data under different crack widths, different crack angles, and different interference factors, thereby improving the efficiency of crack data set construction.

[0018] (3) The present invention proposes a scheme for adding crack interference factors in the template making process and a method for making some interference factors, which expands the crack data set and can effectively enhance the anti-interference ability of the overall model for crack width extraction.

[0019] (4) The present invention adopts a variety of deep learning classification algorithm models and uses the weighted mean of the multi-model classification values ​​after denoising as the crack width. While realizing the transformation from crack width classification to regression, it can give full play to the advantages of different deep learning classification algorithm models, integrate the shallow and deep feature analysis of the image, improve accuracy and stability, reduce the risk of overfitting, and combine the classification results of multiple models to reduce the error rate and improve robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a specific flow chart of an embodiment of the present application.

[0021] Figure 2 It is a schematic diagram of the molding mechanism composition of an embodiment of the present application.

[0022] Figure 3 This is a connection diagram between the assembly plate and the adjustment slider in the embodiment of the present application.

[0023] Figure 4 It is a schematic diagram of the support base composition of an embodiment of the present application.

[0024] Figure 5 It is a schematic diagram of the crossbar adjustment mechanism of the embodiment of the present application.

[0025] Figure 6 It is a schematic diagram of the overall assembly of the device of the embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] The present application provides a method for constructing an apparent crack dataset and extracting its width based on deep learning, comprising the following steps:

[0028] Step 1: Take a high-definition grayscale image of the crack, extract the relative coordinates of the crack in the image, connect the relative coordinates in sequence to restore the crack edge strike line, stretch the strike line to construct the crack opening surface and the crack complementary surface.

[0029] Step 2: After making upper and lower crack morphology base membranes respectively according to the generated three-dimensional crack opening surface and the crack complementary surface, the upper and lower crack templates are cast through the molding container.

[0030] Step 3: Make sure the cracks on the two templates are in relative positions. After determining the required crack width and crack angle, move the upper and lower crack templates toward each other and merge to form a crack.

[0031] Step 4: Build a crack acquisition system, set the distance between the crack acquisition system and the crack within the commonly used working distance range, select multiple working distances for acquisition at fixed intervals, and use the control variable method to collect multiple sets of crack image data.

[0032] Step 5: Classify the crack images and perform data enhancement on the images to construct a crack width dataset. According to the accuracy of the test set, different weights are assigned to different models to form a summary model of the crack width extraction algorithm.

[0033] Step 6: Input the crack images taken in the actual project into the crack width extraction algorithm summary model, use multiple crack width classification algorithm models respectively to obtain the crack corresponding category width, calculate the weighted mean, which is the final extracted crack width.

[0034] Further, in step 1:

[0035] Select a single-direction crack to take a high-definition grayscale image of the crack, and

[0036] When extracting the relative coordinates of the crack from the image, select a clear edge on one side of the crack, establish a plane rectangular coordinate system with one end point of the crack edge as the starting point, draw points one by one along the direction of the crack edge and record the coordinates of each point. The drawn points should be detailed and completely include the direction characteristics of the crack.

[0037] Furthermore, the step 2 splits the real crack into two based on the idea of ​​dividing the crack into two, and casts the upper and lower crack templates through the molding container respectively, and then the templates are assembled into one to construct the crack.

[0038] Furthermore, in step 2, the casting material is gypsum, and

[0039] In the process of making the template, in order to simulate the actual scene of the bridge and tunnel, crack interference factors are added to the template to expand the data set.

[0040] Particularly, the crack interference factors include stains, spider webs, water seepage marks and scratches.

[0041] Furthermore, the stains are simulated by smearing paint and spraying dust; the spider web is simulated by weaving a spider web-shaped structure with fine silk threads and bonding with transparent glue; the water seepage marks are simulated by draining colored water with fine threads, which form water seepage marks after drying; and the scratches are simulated by scraping the surface of the crack template with hard metal.

[0042] Further, in step 4:

[0043] The crack acquisition system is composed of an industrial camera and a zoom lens, and

[0044] At each working distance, first fix the crack collection system, then adjust the height so that the crack is in the middle of the picture, adjust the lens magnification to keep the actual shooting range of the image fixed, adjust the focal length of the lens to make the image clear, adjust the camera shutter speed to keep the image at a good brightness, and then start image data collection.

[0045] Further, the control variable method in step 4 is specifically:

[0046] (1) The crack width is kept constant and the multi-angle crack images with fixed crack width are obtained by rotating the template;

[0047] (2) Control the crack angle to remain unchanged and obtain multiple crack width images with a fixed crack angle by adjusting the crack width;

[0048] (3) Fix the crack width and angle, adjust the type and number of interference factors appearing in the template, and obtain a multi-interference factor image under fixed crack width and angle.

[0049] Further, the step 5 comprises:

[0050] Step 5-1: Classify the crack images of the same width with different shooting distances, different crack angles, and different interference factors into the same category.

[0051] Step 5-2: Add crack width information to each category, and perform data enhancement on the image by rotation, histogram equalization, translation, noise addition, blurring, etc., to construct a crack width dataset.

[0052] Step 5-3: Based on the constructed crack width dataset, a deep learning-based image classification algorithm is used to train multiple crack width classification algorithm models. Different weights are assigned to different models according to the accuracy of the test set to form a crack width extraction algorithm summary model.

[0053] Further, in step 6:

[0054] Multiple crack width classification algorithm models are used to obtain the corresponding crack category widths, and then the mean μ and standard deviation σ of the classification widths of each model are calculated, and the width values ​​less than μ-σ and greater than μ+σ are deleted. Finally, the remaining value model weights are normalized and the weight mean is calculated, which is the final extracted crack width.

[0055] The following is a further description of the implementation steps of this application with reference to the accompanying drawings and specific examples:

[0056] like Figure 1 As shown, the embodiment of the present application provides a method for constructing an apparent crack data set and extracting width based on deep learning, and the specific implementation process is as follows:

[0057] (1) Crack acquisition:

[0058] In order to facilitate the production of crack templates, a crack with a single direction is selected, and a high-definition grayscale image of the crack is taken. Then, a clear edge on one side of the crack is selected, and a plane rectangular coordinate system is established with one end point of the crack edge as the starting point (0, 0). Points are drawn one by one along the direction of the crack edge and the coordinates of each point are recorded. The drawn points should be as detailed and complete as possible to include the direction characteristics of the crack (such as inflection points, sawtooth, etc.) to truly reflect the morphological characteristics of the crack.

[0059] (2) Structural crack opening surface and crack complementary surface:

[0060] Import the relative coordinates of the crack into 3D software (such as Rhino, Solidworks, AutoCAD, etc.), connect and restore the crack edge trend lines in sequence, then stretch the trend lines to construct the crack surface, and then use the reverse command to construct the crack complementary surface.

[0061] (3) Making crack-shaped bottom film:

[0062] According to the generated three-dimensional crack cracking surface and crack cracking complementary surface, 3D printing technology is used to produce upper and lower crack morphological base membranes respectively.

[0063] (4) Making crack template:

[0064] like Figure 2 As shown, based on the idea of ​​dividing the crack into two, the upper and lower crack templates are cast twice by the molding mechanism 1.

[0065] During molding, the horizontal baffle 8 and the vertical baffle 9 are first connected with nuts 11 and horizontal assembly screws 13 to form a baffle around the container. When connected, one side of the lap strip 14 faces inward, and then it is fixed to the bottom plate 10 with nuts 11 and vertical assembly screws 12, thereby forming a molded container 5 as a whole.

[0066] When pouring the formwork, first stick the crack morphology bottom membrane on the inner side of the smooth horizontal baffle 8 (without the overlap strip 14), and then pour it to 1 / 2 of the height inside the molding container 5; then, place the molding embedded plate 4 (composed of the embedded plate column 6 and the through hole 7) horizontally at 1 / 2 of the height inside the molding container 5 through the overlap strip 14, and then pour the remaining slurry until it fills the molding container 5 and vibrates, so that the upper and lower layers of concrete of the molding embedded plate 4 are in good contact; after the slurry is completely solidified, the crack formwork and the molding embedded plate 4 are integrated.

[0067] At this time, the horizontal baffle 8, the vertical baffle 9, the bottom plate 10, the nut 11, the vertical assembly screw 12, and the horizontal assembly screw 13 are removed to obtain the upper and lower crack templates respectively.

[0068] Finally, the formwork can be sprayed with paint to simulate the appearance of concrete in actual scenes such as bridges and tunnels.

[0069] Furthermore, gypsum is preferably used as the casting material, as the formwork is fast, high quality, light and the material is reusable.

[0070] Furthermore, in order to simulate the actual scene of bridges and tunnels during the template making process, common crack interference factors (such as stains, spider webs, traces of water seepage, scratches) were added to the template to expand the data set and improve the subsequent crack width identification and anti-interference capabilities.

[0071] (5) Crack creation and control:

[0072] like Figure 3 As shown, the assembly plate 17 is provided with an assembly through hole 25 for assembling the mold embedding plate 4. The ball joint 24 is adjusted to ensure that the cracks of the two templates are in relative positions. After determining the required crack width, the coarse adjustment knob 22 is first used to control the adjustment slider 16 to move greatly, so that the upper and lower crack templates move toward each other and merge to form a crack; when the distance between the upper and lower crack templates is close to the required crack width, the fine adjustment knob 23 is used to control the adjustment slider 16 for fine adjustment. During the adjustment process, the crack width is continuously observed with a crack width meter until the crack width reaches the requirement, thereby realizing crack width control.

[0073] like Figure 5 As shown, a row of coarse teeth 20 and a row of fine teeth 21 are arranged along the long axis direction inside the unslotted part of the cross bar 15 for matching the coarse adjustment knob 22 and the fine adjustment knob 23; at the same time, slots 19 are arranged on three sides of the cross bar 15 to ensure that the adjustment slider 16 can slide unobstructed inside the cross bar 15.

[0074] After the required crack angle is determined, the crack angle can be controlled by rotating the cross bar 15 around the center of the rotation axis.

[0075] (6) Obtain machine vision crack image data:

[0076] ① Build a crack acquisition system: Use a crack acquisition system composed of an industrial camera and a zoom lens, set the distance of the crack acquisition system within the commonly used working distance range, and select multiple working distances for shooting at fixed intervals. At each working distance, the crack acquisition system must be fixed first, such as Figure 4 As shown, the support base 3 includes a support column 26, a spring clamping slot 27, and an anti-overturning base 28. The support column 26 is provided with a spring clamping slot 27 and is welded to the anti-overturning base 28.

[0077] Press the crossbar adjustment mechanism 2 Figure 6After being assembled on the support base 3 in the assembly direction A shown, the height of the crossbar adjustment mechanism 2 is adjusted through the conversion joint 18 so that the crack is located in the middle of the picture, the magnification of the lens is adjusted to keep the actual shooting range of the image fixed, the focal length of the lens is adjusted to make the image clear, and the camera shutter speed is adjusted to keep the image at a good brightness, and then image data acquisition can be started.

[0078] ②Use the control variable method to collect multiple sets of crack image data:

[0079] The crack width is controlled to be constant, and multi-angle crack images with fixed crack width are obtained by rotating the template;

[0080] The crack angle is controlled to remain unchanged, and multiple crack width images with fixed crack angles are obtained by adjusting the crack width;

[0081] The crack width and angle are fixed, and the types and quantities of interference factors appearing in the template are adjusted to obtain images with multiple interference factors under fixed crack width and angle.

[0082] A single set of crack templates can generate a large number of sets of image data with different crack widths, crack angles, and different interference factors, providing reliable data for the subsequent construction of data sets and the use of deep learning algorithms to identify crack widths.

[0083] (7) Construction and training of crack image dataset:

[0084] ① Classify the crack images of the same width with different shooting distances, different crack angles and different interference factors into the same category;

[0085] ②Add crack width label information to each category, and perform data enhancement on the image by rotation, histogram equalization, translation, noise addition, blurring, etc., so as to construct a crack width dataset;

[0086] ③ Based on the constructed crack width dataset, a variety of deep learning-based image classification algorithms are used, including but not limited to CNN, RNN, Resnet, Transformers and other network structures, as well as different network layer models of a single network structure, to train multiple crack width classification algorithm models. According to the accuracy of the test set, different models are assigned different weights to form a summary model of the crack width extraction algorithm.

[0087] (8) Crack width extraction:

[0088] For crack images taken in actual projects, when the shooting working distance and shooting range are consistent with the constructed dataset image, the crack images are directly input into the crack width extraction algorithm summary model. Multiple crack width classification algorithm models are used to obtain the corresponding crack category widths, and then the mean of the classification widths of each model is calculated. μand standard deviation σ , delete the values ​​less than μ-σ and greater than μ + σ Finally, the residual value model weights are normalized and the weight mean is calculated, which is the final extracted crack width.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for constructing an apparent crack dataset and extracting its width based on deep learning, characterized in that: The following steps are involved: Step 1: Take a high-definition grayscale image of the crack, extract the relative coordinates of the crack in the image, connect the relative coordinates in sequence to restore the crack edge strike line, stretch the strike line to construct the crack surface and the crack complementary surface; Step 2: After making upper and lower crack morphological base films respectively according to the generated three-dimensional crack cracking surface and the crack cracking complementary surface, the upper and lower crack templates are cast through the molding container; Step 3: Make sure the cracks of the two templates are in relative positions, determine the required crack width and crack angle, and then move the upper and lower crack templates toward each other to merge and form a crack; Step 4: Set the distance between the crack acquisition system and the crack within the working distance range, select multiple working distances at fixed intervals for acquisition, and use the control variable method to acquire multiple sets of crack image data; Step 5: Classify the crack images and perform data enhancement on the images to construct a crack width dataset. According to the accuracy of the test set, different models are assigned different weights to form a summary model of the crack width extraction algorithm. Step 6: Input the crack images taken in the actual project into the crack width extraction algorithm summary model, use multiple crack width classification algorithm models respectively to obtain the crack corresponding category width, calculate the weighted mean, which is the final extracted crack width.

2. The method for constructing an apparent crack dataset and extracting its width based on deep learning according to claim 1, characterized in that: In step 1: Select a single-direction crack to take a high-definition grayscale image of the crack, and When extracting the relative coordinates of the crack from the image, select a clear edge on one side of the crack, establish a plane rectangular coordinate system with one end point of the crack edge as the starting point, draw points one by one along the direction of the crack edge and record the coordinates of each point. The drawn points should be detailed and completely include the direction characteristics of the crack.

3. The method for constructing an apparent crack dataset and extracting its width based on deep learning according to claim 1, characterized in that: The step 2 is based on the idea of ​​splitting the real crack into two, and the upper and lower crack templates are cast through the molding container respectively, and then the templates are assembled to construct the crack according to the idea of ​​combining the two into one.

4. The method for constructing an apparent crack dataset and extracting its width based on deep learning according to claim 3, characterized in that: In step 2: The casting material is gypsum, and In the process of making the template, in order to simulate the actual scene of the bridge and tunnel, crack interference factors are added to the template to expand the data set.

5. The method for constructing an apparent crack dataset and extracting its width based on deep learning according to claim 4, characterized in that: The crack interference factors include stains, spider webs, water seepage marks and scratches.

6. The method for constructing an apparent crack dataset and extracting its width based on deep learning according to claim 5, characterized in that: The stains are simulated by applying paint and spraying dust; The spider web is woven with fine silk threads to form a spider web-shaped structure and glued with transparent glue for simulation; The water seepage trace uses a thin line to drain the colored water, and forms a water seepage trace after drying; The scratches are simulated by scratching the surface of a crack template with hard metal.

7. The method for constructing an apparent crack dataset and extracting its width based on deep learning according to claim 1, characterized in that: In step 4: The crack acquisition system is composed of an industrial camera and a zoom lens, and At each working distance, first fix the crack collection system, then adjust the height so that the crack is in the middle of the picture, adjust the lens magnification to keep the actual shooting range of the image fixed, adjust the focal length of the lens to make the image clear, adjust the camera shutter speed to keep the image at the set brightness, and then start image data collection.

8. A method for constructing an apparent crack dataset and extracting its width based on deep learning according to claim 1 or 7, characterized in that: The control variable method in step 4 is specifically: (1) The crack width is kept constant and the multi-angle crack images with fixed crack width are obtained by rotating the template; (2) Control the crack angle to remain unchanged and obtain multiple crack width images with a fixed crack angle by adjusting the crack width; (3) Fix the crack width and angle, adjust the type and number of interference factors appearing in the template, and obtain a multi-interference factor image under fixed crack width and angle.

9. The method for constructing an apparent crack dataset and extracting its width based on deep learning according to claim 1, characterized in that: The step 5 comprises: Step 5-1: Classify the crack images of the same width including different shooting distances, different crack angles, and different interference factors into the same category; Step 5-2: Add crack width information to each category, and perform data enhancement on the image by rotation, histogram equalization, translation, noise addition, and blurring to construct a crack width dataset; Step 5-3: Based on the constructed crack width dataset, a deep learning-based image classification algorithm is used to train multiple crack width classification algorithm models. Different weights are assigned to different models according to the accuracy of the test set to form a crack width extraction algorithm summary model.

10. The method for constructing an apparent crack dataset and extracting its width based on deep learning according to claim 1, characterized in that: In step 6: Multiple crack width classification algorithm models are used to obtain the corresponding crack category widths, and then the mean of the classification widths of each model is calculated. μ and standard deviation σ , delete the values ​​less than μ-σ and greater than μ + σ Finally, the residual value model weights are normalized and the weight mean is calculated, which is the final extracted crack width.

Images