Road shadow crack identification and quantification method
Through Mask-ShadowGAN and improved UNet network model, deshading processing and crack feature extraction are performed, combined with chain code and in-circle method, the accuracy of crack identification and quantification in shadow environment is solved, and high-precision crack detection and quantization are achieved , simplifies operation and improves robustness.
Patent Information
- Application Number
- CN202411048521.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-11-08
AI Technical Summary
Existing computer vision technology is difficult to accurately identify and quantify cracks in road shadow environments, resulting in missed detection and inaccurate positioning.
Mask-ShadowGAN and improved UNet network model are used to measure crack length and width through deshading processing and crack feature extraction, combining chain code and incision circle methods, improving the accuracy of identification and quantification.
High-precision crack identification and quantification in shadow environments are achieved, which enhances the robustness and accuracy of detection, avoids tedious shadow annotation and compensation one by one, simplifies operations and reduces costs.
Smart Images

Figure CN118918168A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of cross-integration of computer vision and civil engineering, and in particular to a method for identifying and quantifying road shadow cracks. Background Art
[0002] For transportation agencies in various provinces and cities, timely detection and repair of road cracks is the main task of maintaining transportation safety. Traditional crack detection methods are based on manual inspections, which require a lot of manpower and material resources. In addition, due to geographical environment, weather and other reasons, it is difficult for inspectors to detect all cracks on the road. At present, although some automated detection methods using high-tech equipment such as lasers and radars have achieved certain success, they cannot be applied on a large scale due to expensive equipment and difficult operation. In recent years, defect detection methods based on computer vision have become an important road crack inspection method due to their simple operation and low cost. However, due to the shadows created by trees and buildings around the road, road crack inspection methods based on computer vision often have false detections and missed detections, which is not conducive to later quantification and repair. Summary of the invention
[0003] The purpose of the present invention is to provide a method for identifying and quantifying road shadow cracks, which is based on Mask-ShadowGAN and improved UNet, and is used to solve the problems of missed detection and inaccurate positioning and quantification of road cracks in shadow environments.
[0004] To achieve the above object, the present invention provides the following technical solutions:
[0005] A method for identifying and quantifying road shadow cracks comprises the following steps:
[0006] S1: Obtain crack images collected in real time and create datasets for training Mask-ShadowGAN and improving UNet network models;
[0007] S2: Use the dataset in S1 to train Mask-ShadowGAN and improved UNet network models;
[0008] S3: Use the Mask-ShadowGAN network trained in S2 to remove shadows from the crack image to obtain a crack image without shadows;
[0009] S4: Using the improved UNet network model trained in S2 to perform crack recognition on the de-shadowed crack image, a binary image containing crack information is obtained;
[0010] S5: Measure the crack length and width of the binary image containing crack information in S4.
[0011] Furthermore, the dataset for training the Mask-ShadowGAN network model in S1 is unpaired shadow crack images and unshadowed crack images with similar backgrounds.
[0012] Furthermore, the data set for training the improved UNet network model in S1 is a shadow-free crack image and a corresponding binary mask image, and the binary mask image is obtained by annotating the shadow-free crack image using PS software.
[0013] Furthermore, the improved UNet network model in S1 includes an encoder built based on double convolution, a decoder built based on deconvolution and attention mechanism, and a neural network cognitive module located between the fourth downsampling of the encoder and the first deconvolution of the decoder.
[0014] Furthermore, in S5, the crack length measurement is achieved through a calculation method based on a chain code, and the crack width measurement is achieved through a method based on an inscribed circle.
[0015] Furthermore, the construction method of the attention mechanism is as follows:
[0016] S101: Perform global average pooling on the input feature map in the width and height directions respectively;
[0017] S102: Combine the two parallel features of width and height and transpose them to the same dimension for stacking, and use convolution, normalization and activation functions to enhance the features;
[0018] S103: Separate the width and height into two parallel features, obtain the attention status in the width and height dimensions through transposition, convolution and use of sigmoid function, and finally multiply them with the original features to form an attention mechanism.
[0019] Furthermore, the neural network cognitive module divides the input feature map into two parts through a 1x1 convolution operation. The first part is directly transmitted to the output end without any additional operation. The second part first extracts the detail information of the basic feature map through a 3×3 convolution; then the extracted detail information is divided into four parts through a 1x1 convolution, and three hole convolutions with different expansion rates are used to simulate the receptive fields of different eccentricities in the human nervous system to learn the detail features. Then, a 1x1 convolution and a 3x3 convolution are used to highlight the importance of the area close to the center, thereby improving the insensitivity to small spatial changes; finally, it is spliced with the first part without any additional operation, and the channel correlation of the two parts is decoupled through a 1x1 convolution to obtain the final output.
[0020] Furthermore, the method for calculating the crack length based on the chain code is as follows:
[0021] (1) Obtaining the skeleton image of the crack from the binary image of the crack;
[0022] (2) Encode in a counterclockwise direction, traverse all pixels, and obtain the chain code of the crack;
[0023] (3) Calculate the crack chain length. The crack chain length is calculated as follows:
[0024] L=cG a +dG b
[0025] Where: L is the length of the chain code; G a is the number of even chain codes; G b is the number of odd chain codes; c and d are the corresponding weight parameters, respectively.
[0026] (4) Calculate the physical size of the crack length based on the image pixel rate.
[0027] Furthermore, the calculation method of the crack width based on the inscribed circle is as follows:
[0028] (1) Obtain the crack contour from the crack binary image;
[0029] (2) Determine the minimum rectangular area containing the crack;
[0030] (3) construct a pixel grid within the rectangular area and filter out the pixels that fall within the outline;
[0031] (4) Randomly select some pixel points and calculate the corresponding inscribed circle radius;
[0032] (5) Iteratively search the remaining pixels to obtain the maximum inscribed circle radius and the average inscribed circle radius;
[0033] (6) Calculate the crack width. The crack width is calculated as follows:
[0034] W max =2max{R i}
[0035]
[0036] Where: W max is the maximum crack width, W mean is the average crack width, n is the number of inscribed circles, i = 0, 1, 2...n;
[0037] (7) Based on the image pixel rate, calculate the physical size of the crack width.
[0038] Compared with the prior art, the present invention has the following beneficial effects:
[0039] 1. The method for identifying and quantifying road shadow cracks of the present invention uses unpaired shadow crack images and unshadowed crack images for training through the constructed Mask-ShadowGAN shadow removal network, thereby avoiding tedious shadow labeling and compensation one by one, and diversifying the training samples.
[0040] 2. The method for identifying and quantifying road shadow cracks of the present invention constructs an improved UNet network model, which enhances the ability to extract crack feature information based on double convolution, deconvolution, attention mechanism and neural network cognitive module, and improves the accuracy and robustness of crack detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 is a flow chart of the method of the present invention;
[0042] Figure 2 Schematic diagram of the overall Mask-ShadowGAN of the present invention;
[0043] Figure 3 This is a structural diagram of the improved UNet model of the present invention;
[0044] Figure 4 is a structural diagram of the attention module of the present invention;
[0045] Figure 5 This is a structural diagram of the neural network cognitive module of the present invention. DETAILED DESCRIPTION
[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0047] See also Figure 1 In an embodiment of the present invention, a method for identifying and quantifying road shadow cracks is provided, which is implemented based on Mask-ShadowGAN and an improved UNet network model, and specifically includes the following steps:
[0048] Step 1: Obtain crack images collected in real time and create a dataset for training Mask-ShadowGAN and improving the UNet network model; the specific method is as follows:
[0049] (1) Collecting original road image data and normalizing these images;
[0050] (2) Crack images are divided into two categories: those containing shadows and those not containing shadows;
[0051] (3) The crack image is annotated using PS software, with the crack area marked in white and the non-crack area marked in black, to obtain a mask image and save it.
[0052] Step 2: Use the dataset in S1 to train Mask-ShadowGAN and improved UNet network models; the specific method is as follows:
[0053] (1) Use the classified shadow crack images and non-shadow crack images to train the Mask-ShadowGAN network;
[0054] (2) Use the annotated shadow-free crack images to train the improved UNet network model;
[0055] In the above embodiment, the Mask-ShadowGAN model in the present invention is a prior art (see https: / / github.com / xw-hu / Mask-ShadowGAN for details), which will not be described in detail here. The training process of the model is as follows: Figure 2 As shown, it generates shadow masks from shadow images by automatic learning, and then guides shadow generation from shadow-free images and shadow removal for shadow images through a reformulated cycle consistency loss.
[0056] In the above embodiment, the improved UNet model in the present invention is as follows: Figure 3 As shown, it includes an encoder based on double convolution, a decoder based on deconvolution and attention mechanism, and a neural network cognitive module located between the fourth downsampling of the encoder and the first deconvolution of the decoder; wherein the attention mechanism is as shown in Figure 4 As shown in the figure, it is the existing technology (see https: / / github.com / houqb / CoordAttention / blob / main / coordatt.py for details), which will not be described here. The specific operation is: the input feature map is globally averaged pooled in the width and height directions respectively to obtain the features in the width and height directions; then the two parallel features are combined and transposed to the same dimension for stacking, and the features are enhanced by convolution, normalization and activation function; then it is separated into two parallel features, and the attention in the width and height dimensions is obtained by transposition, convolution and sigmoid function, which is multiplied with the original features to form an attention mechanism.
[0057] The neural network cognitive module in the embodiment of the present invention is as follows Figure 5As shown in the figure, it is a self-built module. The specific operation is: the input feature map is divided into two parts through a 1x1 convolution operation. The first part is directly transmitted to the output end without any additional operation. The second part first extracts the detail information of the basic feature map through a convolution; then the extracted detail information is divided into four parts through a 1x1 convolution, and three hole convolutions with different expansion rates are used to simulate the receptive fields of different eccentricities in the human nervous system to learn the detail features. Then, a 1x1 convolution and a 3x3 convolution are used to highlight the importance of the area close to the center and improve the insensitivity to small space changes; finally, it is spliced with the first part without additional operation, and the channel correlation of the two parts is decoupled through a 1x1 convolution to obtain the final output.
[0058] Step 3: Use the trained Mask-ShadowGAN network to remove shadows from the crack image to obtain a crack image without shadows.
[0059] Step 4: Use the trained improved UNet network model to identify cracks in the de-shadowed crack image to obtain a binary image containing crack information;
[0060] Step 5: Measure the crack length and width of the binary image containing crack information; the crack length measurement method is specifically implemented as follows:
[0061] (1) Obtaining the skeleton image of the crack from the binary image of the crack;
[0062] (2) Encode in a counterclockwise direction, traverse all pixels, and obtain the chain code of the crack;
[0063] (3) Calculate the crack chain length. The crack chain length is calculated as follows:
[0064] L=cG a +dG b
[0065] Where: L is the length of the chain code; G a is the number of even chain codes; G b is the number of odd chain codes; c and d are the corresponding weight parameters, usually
[0066] (4) Calculate the physical size of the crack length based on the image pixel rate;
[0067] The crack width measurement method is specifically implemented as follows:
[0068] (1) Obtain the crack contour from the crack binary image;
[0069] (2) Determine the smallest rectangular area that can contain the crack;
[0070] (3) construct a pixel grid within the rectangular area and filter out the pixels that fall within the outline;
[0071] (4) Randomly select some pixel points and calculate the corresponding inscribed circle radius;
[0072] (5) Iteratively search the remaining pixels to obtain the maximum inscribed circle radius and the average inscribed circle radius;
[0073] (6) Calculate the crack width. The crack width is calculated as follows:
[0074] W max =2max{R i}
[0075]
[0076] Where: W max is the maximum crack width, W mean is the average crack width, n is the number of inscribed circles, i = 0, 1, 2...n;
[0077] (7) Based on the image pixel rate, calculate the physical size of the crack width.
[0078] In summary: A method for identifying and quantifying road shadow cracks provided by the present invention is implemented based on Mask-ShadowGAN and an improved UNet network model, including creating a data set, training Mask-ShadowGAN and an improved UNet network model, removing the shadow of the crack image using the trained Mask-ShadowGAN network model, obtaining a binary image containing crack information using the improved UNet network, and measuring the crack length and width of the binary image containing crack information, thereby realizing a high-precision crack identification and quantification method in a shadow environment. At the same time, the operation is simple and has good robustness, which can well solve the problems of missed detection and inaccurate positioning and quantification of road cracks in a shadow environment.
[0079] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.
Claims
1. A method for identifying and quantifying road shadow cracks, characterized in that: The following steps are involved: S1: Obtain crack images collected in real time and create datasets for training Mask-ShadowGAN and improving UNet network models; S2: Use the dataset in S1 to train Mask-ShadowGAN and improved UNet network models; S3: Use the Mask-ShadowGAN network trained in S2 to remove shadows from the crack image to obtain a crack image without shadows; S4: Using the improved UNet network model trained in S2 to perform crack recognition on the de-shadowed crack image, a binary image containing crack information is obtained; S5: Measure the crack length and width of the binary image containing crack information in S4.
2. A method for identifying and quantifying road shadow cracks as claimed in claim 1, characterized in that: The dataset for training the Mask-ShadowGAN network model in S1 is unpaired shadow crack images and unshadowed crack images with similar backgrounds.
3. The method for identifying and quantifying road shadow cracks according to claim 1, characterized in that: The data set for training the improved UNet network model in S1 is the shadow-free crack image and the corresponding binary mask image. The binary mask image is obtained by annotating the shadow-free crack image using PS software.
4. The method for identifying and quantifying road shadow cracks according to claim 1, characterized in that: The improved UNet network model in S1 includes an encoder built based on double convolution, a decoder built based on deconvolution and attention mechanism, and a neural network cognitive module located between the fourth downsampling of the encoder and the first deconvolution of the decoder.
5. The method for identifying and quantifying road shadow cracks according to claim 1, characterized in that: In S5, the crack length measurement is achieved through a calculation method based on chain code, and the crack width measurement is achieved through a method based on inscribed circle.
6. A method for identifying and quantifying road shadow cracks as claimed in claim 4, characterized in that: The construction method of the attention mechanism is as follows: S101: Perform global average pooling on the input feature map in the width and height directions respectively; S102: Combine the two parallel features of width and height and transpose them to the same dimension for stacking, and use convolution, normalization and activation functions to enhance the features; S103: Separate the width and height into two parallel features, obtain the attention status in the width and height dimensions through transposition, convolution and use of sigmoid function, and finally multiply them with the original features to form an attention mechanism.
7. A method for identifying and quantifying road shadow cracks as claimed in claim 4, characterized in that: The neural network cognitive module divides the input feature map into two parts through a 1x1 convolution operation. The first part is directly transmitted to the output end without any additional operation. The second part first extracts the detail information of the basic feature map through a 3×3 convolution; then the extracted detail information is divided into four parts through a 1x1 convolution, and three hole convolutions with different expansion rates are used to simulate the receptive fields of different eccentricities in the human nervous system to learn the detail features. Then, a 1x1 convolution and a 3x3 convolution are used to highlight the importance of the area close to the center, thereby improving the insensitivity to small spatial changes; finally, it is spliced with the first part without any additional operation, and the channel correlation of the two parts is decoupled through a 1x1 convolution to obtain the final output.
8. The method for identifying and quantifying road shadow cracks according to claim 5, characterized in that: The method for calculating crack length based on chain code is as follows: (1) Obtaining the skeleton image of the crack from the binary image of the crack; (2) Encode in a counterclockwise direction, traverse all pixels, and obtain the chain code of the crack; (3) Calculate the crack chain length. The crack chain length is calculated as follows: L=cG a +dG b Where: L is the length of the chain code; G a is the number of even chain codes; G b is the number of odd chain codes; c, d are the corresponding weight parameters, c = 1, (4) Calculate the physical size of the crack length based on the image pixel rate.
9. The method for identifying and quantifying road shadow cracks according to claim 5, characterized in that: The calculation method of crack width based on the inscribed circle is as follows: (1) Obtain the crack contour from the crack binary image; (2) Determine the minimum rectangular area containing the crack; (3) construct a pixel grid within the rectangular area and filter out the pixels that fall within the outline; (4) Randomly select some pixel points and calculate the corresponding inscribed circle radius; (5) Iteratively search the remaining pixels to obtain the maximum inscribed circle radius and the average inscribed circle radius; (6) Calculate the crack width. The crack width is calculated as follows: W max =2max{R i } Where: W max is the maximum crack width, W mean is the average crack width, n is the number of inscribed circles, i = 0, 1, 2...n; (7) Based on the image pixel rate, calculate the physical size of the crack width.
Citation Information
Patent Citations
Automatic tunnel crack detection method based on convolutional network
CN115761613A
Pavement crack classification and identification method under shadow interference
CN116206156A
Pavement crack segmentation detection method and device
CN116862866A
UNet network-based pavement crack disease identification and segmentation method and system
CN117315261A
Road crack detection method and system based on improved UNet
CN117975427A
