A method of crack detection

Abstract

The application belongs to the technical field of concrete structure crack detection, and discloses a crack detection method. Through obtaining a crack image, the crack image is proportionally divided into a training set, a verification set and a test set; a DeepLabv3+(N-S) semantic segmentation model is built based on a DeepLabv3+ network model; the crack image in the training set and the verification set is used to train the DeepLabv3+(N-S) semantic segmentation model, the model parameters are adjusted, and the trained DeepLabv3+(N-S) semantic segmentation model is obtained; the crack image in the test set is input into the trained DeepLabv3+(N-S) semantic segmentation model to obtain a binary mask image, the crack parameter calculation module is input, the crack parameter is calculated, and the crack severity is judged according to the crack parameter. The training parameters of the constructed semantic segmentation model are few, the crack segmentation precision is improved, and the crack parameter calculation error is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of concrete structure crack detection, and in particular to a crack detection method. BACKGROUND

[0002] Timely identification and detection of concrete, calculation of various parameters of cracks, and judgment of whether the crack parameters have reached the dangerous value of engineering and building hazards can effectively reduce the occurrence of engineering disasters and achieve scientific and effective engineering and building maintenance, which has important practical significance for the safe operation of engineering and building. For crack detection, traditional building concrete detection mainly uses indirect measurement and manual reading of crack width through contact-type dial extensometer, inclinometer, automatic reading microscope, crack measurement card, etc. The error problem of manual reading always exists, and it is difficult to ensure the accuracy of detection due to the influence of environmental factors such as airflow and temperature during detection. With the vigorous development of artificial intelligence and image processing technology and the upgrading of hardware equipment, the automation level of concrete crack detection is continuously improved. A large number of domestic and foreign researchers have carried out research on crack identification technology based on deep learning semantic segmentation models, and research on the application of semantic segmentation models such as FCN, U-net, DeepLabv3+ in crack segmentation has emerged. The current crack segmentation model based on deep neural network has the problems of many training parameters, rough crack edge segmentation, insufficient segmentation accuracy, and further large error in crack parameter calculation caused by insufficient segmentation accuracy. SUMMARY

[0003] Therefore, the embodiments of the present application provide a crack detection method, the constructed crack semantic segmentation model has fewer training parameters, improves long-distance dependence, has higher crack boundary segmentation accuracy, and further reduces crack parameter calculation error. The specific technical solution contents are as follows:

[0004] Obtain a crack image, and divide the crack image into a training set, a validation set and a test set according to a proportion;

[0005] A DeepLabv3+(N-S) semantic segmentation model is built based on a DeepLabv3+ network model, wherein the DeepLabv3+(N-S) semantic segmentation model comprises an encoder and a decoder, the encoder comprises an improved backbone network ResNet101-S, a spatial pyramid pooling module ASPP and a Non-local block attention mechanism module, the output end of the encoder is connected to the input end of the decoder, the output end of the improved backbone network ResNet101-S is connected to the input end of the Non-local block attention mechanism module, and the output end of the Non-local block attention mechanism module is connected to the input end of the spatial pyramid pooling module ASPP;

[0006] training the DeepLabv3+(N-S) semantic segmentation model using the crack images in the training set and the validation set, adjusting the model network parameters, and obtaining a trained DeepLabv3+(N-S) semantic segmentation model;

[0007] inputting the crack images in the test set into the trained DeepLabv3+(N-S) semantic segmentation model, outputting a binary mask image of the crack images, inputting the binary mask image into a crack parameter calculation module to calculate crack parameters, and judging the severity of the crack according to the crack parameters.

[0008] In a preferred example of the present application, the step of inputting the binary mask image into the crack parameter calculation module to calculate the crack parameters can further be provided as:

[0009] extracting the center axis of the crack in the binary mask image using a skeleton extraction algorithm, selecting a first pixel point on the center axis and a second pixel point on the crack edge corresponding to the first pixel point, calculating the Euclidean distance between the first pixel point and the second pixel point, and obtaining the crack pixel width of the crack at the first pixel point.

[0010] In a preferred example of the present application, the step of building the DeepLabv3+(N-S) deep semantic segmentation model based on the DeepLabv3+ network model can further be provided as:

[0011] modifying the front-end convolution kernel of the backbone network ResNet101 of the DeepLabv3+ network model and modifying the residual structure of the Bottleneck bottleneck unit to obtain an improved backbone network ResNet101-S, constructing a DeepLabv3+ network model of the improved backbone network, embedding a Non-local block attention mechanism module in the DeepLabv3+ network model of the improved backbone network, and obtaining a DeepLabv3+(N-S) semantic segmentation model.

[0012] In a preferred example of the present application, the step of modifying the front-end convolution kernel of the backbone network ResNet101 of the DeepLabv3+ network model can further be provided as:

[0013] using three 3x3 convolution kernels to replace the 7x7 front-end convolution kernel of the backbone network ResNet101, and inserting a ReLU activation function between the first 3x3 convolution kernel and the second 3x3 convolution kernel and between the second 3x3 convolution kernel and the third 3x3 convolution kernel.

[0014] In a preferred example of the present application, the step of modifying the residual structure of the Bottleneck unit can further be configured to include:

[0015] replacing the 3x3 convolution structure in the residual structure of the Bottleneck unit with a 1x3 and a 3x1 convolution structure.

[0016] In a preferred example of the present application, the step of embedding the Non-local block attention mechanism module in the improved DeepLabv3+ network model of the backbone network can further be configured to include embedding the Non-local block attention mechanism module between the output end of the improved backbone network ResNet101-S and the input end of the spatial pyramid pooling module ASPP module, and between the output end of the backbone network ResNet101-S and the input end of the decoder.

[0017] In a preferred example of the present application, the step of obtaining a crack image and dividing the crack image into a training set, a validation set and a test set in proportion can further include:

[0018] performing contour labeling on each crack image in the training set, the validation set and the test set to obtain a labeled value of the crack in each crack image, which is used to train the DeepLabv3+(N-S) semantic segmentation model.

[0019] In a preferred example of the present application, the step of obtaining a crack image and dividing the crack image into a training set, a validation set and a test set in proportion can further include:

[0020] inputting the crack images in the training set into the DeepLabv3+(N-S) semantic segmentation model for training to obtain a predicted value of the crack; calculating a loss value according to the predicted value and the labeled value, adjusting the network parameters of the DeepLabv3+(N-S) semantic segmentation model according to the loss value, until the loss value accuracy after validation of the validation set reaches a preset condition, fixing the network parameters of the DeepLabv3+(N-S) semantic segmentation model, and obtaining a trained DeepLabv3+(N-S) semantic segmentation model.

[0021] In a preferred example of the present application, the step of calculating a loss value according to the predicted value and the labeled value can further include:

[0022] calculating the loss value using a cross-entropy loss function with an introduced L2 regularization term, wherein the formula of the cross-entropy loss function is:

[0023]

[0024] wherein loss represents a loss value, y represents a label value, represents a predicted value, Wi represents a network parameter to be trained, n represents a number of network parameters, and l represents a weight decay coefficient.

[0025] In a preferred example of the present application, the step of adjusting the network parameters of the DeepLabv3+(N-S) semantic segmentation model according to the loss value can be further set as follows: the parameter updating strategy is a stochastic gradient descent method, the learning rate is set to 0.01, the momentum parameter is set to 0.9, and the learning rate decay value is set to 0.0005.

[0026] In summary, compared with the prior art, the technical scheme provided by the embodiments of the present application has at least the following beneficial effects:

[0027] By modifying the front-end convolution kernel of the backbone network ResNet101 of the DeepLabv3+ model and the residual structure of the bottleneck unit, the number of network layers is increased, the non-linear expression ability of the model is improved, and the improved model has fewer parameters, reducing the occurrence of fitting phenomenon; further embedding the Non-local block attention mechanism module improves the DeepLabv3+(N-S) semantic segmentation model, strengthens the spatial distance information dependence, and strengthens the connection between the pixels in the main body of the crack during crack segmentation. The DeepLabv3+(N-S) semantic segmentation model has better segmentation performance, reduces the error of crack parameter calculation, and improves the accuracy of crack parameter calculation. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 A flowchart of a crack detection and parameter calculation method is provided for an embodiment of the present application;

[0029] Figure 2 A schematic diagram of the front-end convolution structure of the backbone network ResNet101 before and after modification is provided for an embodiment of the present application;

[0030] Figure 3 A schematic diagram of the bottleneck unit structure of the backbone network ResNet101 before and after modification is provided for an embodiment of the present application;

[0031] Figure 4 A schematic diagram of the structure of the Non-local block attention mechanism module is provided for an embodiment of the present application;

[0032] Figure 5 A schematic diagram of the network structure of the DeepLabv3+(N-S) semantic segmentation model is provided for an embodiment of the present application;

[0033] Figure 6A relative error diagram of the maximum crack width calculated based on different semantic segmentation models is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0034] The specific embodiments are merely illustrative of the present application, and are not intended to limit the present application. Those skilled in the art can make modifications to the embodiments without creative contribution, according to the needs, after reading the present specification, and the modifications are protected by the patent law as long as they are within the scope of the claims of the present application.

[0035] To make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative contribution are within the scope of protection of the present application.

[0036] In addition, the term “and / or” in the present application is merely to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can mean that there are three cases of A alone, A and B together, and B alone. In addition, the character “ / ” in the present application generally represents an “or” relationship between the associated objects, unless otherwise specified.

[0037] The terms “first”, “second”, and the like in the present application are used to distinguish the same items or similar items with basically the same function, and it should be understood that there is no logical or time sequence dependence between “first”, “second”, and “nth”, and the quantity and execution order are not limited.

[0038] The embodiments of the present application will be described in further detail below with reference to the drawings of the specification.

[0039] In an embodiment of the present application, a crack detection method is provided, as shown in Figure 1 The main steps are described as follows.

[0040] S10: Obtain a crack image, and divide the crack image into a training set, a validation set and a test set according to a proportion;

[0041] Preferably, the contour of each crack image in the training set, the validation set and the test set is labeled to obtain a labeled value of the crack in each crack image.

[0042] In this embodiment, the cracked concrete beam specimen after the load test is selected as the self-made crack image dataset. In order to improve the robustness of the network model, images containing different light conditions, shadows, handwriting, backgrounds, sundries, and shooting angles are selected to obtain 48 concrete beam images with a resolution of 3648x2736. Further, the sliding window segmentation method is used to segment the images into 20736 images with a resolution of 152x152. The segmented images have sufficient diversity in length, width, and direction. After data cleaning to remove blurred, distorted, and distorted images, 13194 images are obtained. After further screening, 4568 crack images containing cracks are obtained. The crack images are randomly divided into a training set, a validation set, and a test set according to an 8:1:1 ratio.

[0043] Each crack image in the training set, the validation set, and the test set is labeled using the labelme labeling software. Specifically, the crack pixels and non-crack pixels in the crack image are labeled to obtain the label value of the crack in each crack image. During labeling, the principle of leaving as much crack length and width information as possible is maintained to ensure accuracy in pixel classification.

[0044] S20: building a DeepLabv3+(N-S) semantic segmentation model based on the DeepLabv3+ network model;

[0045] Preferably, the front-end convolution kernel of the backbone network ResNet101 of the DeepLabv3+ network model is modified, and the residual structure of the Bottleneck bottleneck unit is modified to construct an improved backbone network ResNet101-S, and a DeepLabv3+ network model of the improved backbone network is obtained. The Non-local block attention mechanism module is embedded in the DeepLabv3+ network model of the improved backbone network to obtain a DeepLabv3+(N-S) semantic segmentation model.

[0046] The DeepLabv3+ network model includes an encoder and a decoder, and the encoder includes a backbone network ResNet101 and a spatial pyramid pooling module ASPP. The DeepLabv3+(N-S) semantic segmentation model built based on the DeepLabv3+ network model includes an encoder and a decoder, and the encoder includes an improved backbone network ResNet101-S, a spatial pyramid pooling module ASPP, and a Non-local block attention mechanism module. The output end of the encoder is connected to the input end of the decoder; the output end of the improved backbone network ResNet101-S is connected to the input end of the Non-local block attention mechanism module, and the output end of the Non-local block attention mechanism module is connected to the input end of the spatial pyramid pooling module ASPP.

[0047] Preferably, the 7×7 front-end convolutional kernel of the backbone network ResNet101 is replaced with three 3×3 convolutional kernels, and ReLU activation functions are inserted between the first and second 3×3 convolutional kernels and between the second and third 3×3 convolutional kernels.

[0048] It should be noted that DeepLabv3+'s backbone network, ResNet101, is a 101-layer residual network built on the Bottleneck architecture. The network front-end has a 64-channel 7×7 convolutional kernel with a stride of 2. Large convolutional kernels lead to more model parameters and fewer layers, which is detrimental to the network's ability to transmit nonlinear features. For example... Figure 2 As shown, (a) is the front-end convolutional structure of the original backbone network ResNet101, and (b) is the modified front-end convolutional structure. This application replaces the 7×7 front-end convolutional kernels in the backbone network ResNet101 with three 3×3 convolutional kernels, and inserts ReLU activation functions between the output of the first 3×3 convolutional kernel and the input of the second 3×3 convolutional kernel, and between the output of the second 3×3 convolutional kernel and the input of the third 3×3 convolutional kernel. The ReLU activation function is a type of activation function that sets some outputs to 0, resulting in network sparsity, reducing the dependencies between parameters, and thus reducing overfitting. The formula for the ReLU activation function is: f(x) = max(0,x).

[0049] Preferably, the 3×3 convolution structure in the residual structure of the Bottleneck unit is replaced with a 1×3 and a 3×1 convolution structure.

[0050] It should be noted that the Bottleneck unit is a unit designed based on residual structures, mainly used to build deeper residual networks, such as ResNet50, ResNet101, and ResNet152. Unlike shallow ResNet residual units, it adds a 1×1 convolutional layer after the input and before the output of each Bottleneck layer to increase or decrease the number of channels and change the matrix dimensions, thus addressing the issue of different input and output sizes for certain layers. To improve model performance, this application replaces the 3×3 convolutional kernel in the original Bottleneck unit with a convolutional structure consisting of one 1×3 kernel and one 3×1 kernel, resulting in the improved backbone network ResNet101-S, as shown below. Figure 3As shown, w is the number of input channels. The receptive field and output size of a convolution with one 1×3 kernel and one 3×1 kernel are the same as those with one 3×3 kernel. The improved backbone network ResNet101-S has an increased number of network layers, which can improve the nonlinear expressive power of the model. Furthermore, the improved model has fewer parameters, which can reduce the occurrence of overfitting.

[0051] Preferably, the Non-local block attention mechanism module is embedded between the output of the improved backbone network ResNet101-S and the input of the spatial pyramid pooling module ASPP, as well as between the output of the backbone network ResNet101-S and the input of the decoder.

[0052] It's important to note that attention mechanisms applied in image processing essentially mimic the human tendency to focus more on a particular object within the entire visual field. In an image feature matrix, the acquisition of global information about the spatial relationships between each pixel strongly guides the structure of semantic segmentation. Non-local block attention mechanisms effectively address the problem of information transmission over long distances. For example... Figure 4 As shown, from Non-local The network structure diagram of the block attention mechanism module shows that this module performs three different linear mappings on the input matrix C×H×W through C / 2×1×1 convolution operations, namely g(x), θ(x), and φ(x), to obtain three different feature matrices C / 2×H×Wv, C / 2×H×Wk, and C / 2×H×Wq, respectively. These feature matrices are then transformed into three vectors: C / 2×HWv, C / 2×HWk, and C / 2×HWq. The C / 2×HWk output of θ(x) is transposed to obtain HWk×C / 2, which is then multiplied by the C / 2×HWq output of φ(x) to obtain the similarity matrix HWq×HWk. The similarity matrix HWq×HWk is then multiplied by the softmax (normalized exponential function) operation and the matrix C / 2×HWv output of g(x) to obtain the feature matrix of dimension C / 2×H×W. Finally, the dimension is restored through a C×1×1 convolution operation to obtain the non-local feature matrix. The block attention mechanism module enhances the feature matrix that depends on spatial distance information; finally, it is summed with the input matrix C×H×W to obtain the final output feature map z.

[0053] This application embeds a non-local block attention mechanism module between the output of the improved backbone network ResNet101-S and the input of the spatial pyramid pooling module ASPP, as well as between the output of the improved backbone network ResNet101-S and the decoder, to obtain the DeepLabv3+(NS) semantic segmentation model, such as... Figure 5The Non-local block attention mechanism module strengthens the spatial distance information dependence and the connection between the pixels in the crack main body during crack segmentation.

[0054] Specifically, the main body of the DeepLabv3+(N-S) semantic segmentation model is composed of an encoder Encoder and a decoder Decoder. The encoder Encoder includes an improved backbone network ResNet101-S, a Non-local block attention mechanism module, and a spatial pyramid pooling module ASPP. In the encoder Encoder module, the feature matrix of the input crack image is subjected to the improved backbone network ResNet101-S to obtain a high-level feature map, the long-distance information dependence is improved by the Non-local block attention mechanism module, the spatial pyramid pooling module ASPP is used to sequentially perform 1x1 convolution operation, 3x3 dilated convolution operation with expansion rates of 6, 12, and 18, and pooling operation on the high-level feature map, and finally the semantic information is extracted by 1x1 convolution operation; in the decoder Decoder module, the low-level feature map output by the improved backbone network ResNet101-S of the encoder Encoder module is subjected to the Non-local block attention mechanism module and then subjected to 1x1 convolution operation to obtain crack boundary detail information, which is then spliced with the feature map obtained by bilinear upsampling by 4 times from the upper layer, and then subjected to 3x3 convolution operation and then subjected to bilinear upsampling by 4 times to obtain the feature value of the crack image.

[0055] S30: training the DeepLabv3+(N-S) semantic segmentation model using the crack images in the training set and the validation set, adjusting the model network parameters, and obtaining the trained DeepLabv3+(N-S) semantic segmentation model;

[0056] The network parameters are parameters used to determine the model, including learning rate, iteration number, layer number, number of neurons in each layer, batch size, and weight of each part in the loss function.

[0057] Preferably, the crack images in the training set are input into the DeepLabv3+(N-S) semantic segmentation model for training to obtain the predicted value of the crack in the crack image; the loss value is calculated according to the predicted value and the labeled value, the network parameters of the DeepLabv3+(N-S) semantic segmentation model are adjusted according to the loss value, until the loss value meets the preset condition after verification of the validation set, the network parameters of the DeepLabv3+(N-S) semantic segmentation model are fixed, and the trained DeepLabv3+(N-S) semantic segmentation model is obtained.

[0058] Preferably, the parameter update strategy is the stochastic gradient descent method, the learning rate is set to 0.01, the momentum parameter is set to 0.9, and the learning rate decay value is set to 0.0005.

[0059] To prevent the DeepLabv3+(N-S) semantic segmentation model from falling into a local optimum during training, the optimization strategy uses the SGD (Stochastic Gradient Descent) random gradient descent method. The random gradient descent method is mainly used to update the model network parameters based on the loss value and the learning rate. The update of the network parameters is based on the gradient descent method, and only one sample is randomly selected to calculate the gradient in each iteration, which can reduce the amount of calculation and jump out of a poor solution. The learning rate is set to 0.01, the momentum parameter is set to 0.9, the learning rate decay value is set to 0.0005, the network is set to 450 training times (epochs), the training batch size is set to 8, and a checkpoint is recorded once every 10 epochs. The learning strategy uses the Poly learning strategy, which is an exponential strategy for adjusting the learning rate during training. The learning rate can be dynamically adjusted according to the set power index in each iteration, which makes the network model converge better. The power index is set to 0.9, and the lower limit of the learning rate is set to 0.0001.

[0060] Preferably, the model training uses a cross-entropy loss function with an L2 regularization term to calculate the loss value. The introduction of the L2 regularization term can prevent overfitting and accelerate model convergence, avoiding the phenomenon of gradient dispersion. The formula of the cross-entropy loss function with the L2 regularization term is:

[0061]

[0062] where loss represents the loss value, y represents the label value, represents the predicted value, Wi is the network parameter to be trained, n is the number of network parameters, and λ is the weight decay coefficient.

[0063] Specifically, after setting up various training network parameters, the terminal inputs the instruction of running the train.py program, the program starts to input the crack images in the training set into the DeepLabv3+(N-S) semantic segmentation model network, and starts to conduct the forward conduction in each module of the semantic segmentation model network, finally obtains the predicted value of the crack image, then according to the predicted value of the crack image and the annotation value of the crack image annotated in advance, the loss value is calculated by using the cross entropy loss function, the gradient calculated according to the loss value and the learning rate adjustment update the network parameters of the model, when the crack images in the training set are trained for one time, the training times epoch is completed, then the crack images in the verification set are input into the DeepLabv3+(N-S) semantic segmentation model network to obtain the predicted value, and the loss value is calculated; when 450 epochs are performed, the training program will stop, the program will select the model with the loss value precision reaching the preset condition after the verification set verification as the final result, fix the network parameters of the DeepLabv3+(N-S) semantic segmentation model, and obtain the trained DeepLabv3+(N-S) semantic segmentation model.

[0064] S40: inputting the crack images in the test set into the trained DeepLabv3+(N-S) semantic segmentation model, outputting the binary mask image of the crack image, inputting the binary mask image into the crack parameter calculation module to calculate the crack parameter, and judging the crack severity according to the crack parameter.

[0065] The crack images in the test set are input into the trained DeepLabv3+(N-S) semantic segmentation model to obtain the binary mask image corresponding to the crack image, and the binary mask image is input into the crack parameter calculation module. The crack parameter calculation module calculates the length and width parameters of the crack based on the skeleton extraction algorithm. The skeleton extraction algorithm is also called binary image thinning method. The algorithm can thin a connected region to a pixel width, extract the center pixel contour of the target on the image, and is used for feature extraction and target topology representation.

[0066] Specifically, the medial_axis function in the skimage library is used to extract the crack skeleton in the binary mask image to grab the center axis of the crack in the binary mask image, select a first pixel point P i (x,y) on the center axis, and extract a second pixel point P j (x,y) corresponding to the first pixel point P j (x,y) on the crack edge. The second pixel point P i (x,y) is the pixel point corresponding to the first pixel point P i (x,y) for calculation. The first pixel point P jEuclidean distance of (x, y) from the first pixel point P i (x, y) and the second pixel point P j (x, y) and the second pixel point P i (x, y) and the second pixel point P

[0067] The calculation formula of S is:

[0068]

[0069] Similarly, the crack pixel width of other pixel points on the crack center line can be calculated according to the above method; the crack pixel width of all pixel points on the crack center line is obtained, that is, the maximum crack pixel width W max , the minimum crack pixel width W min and the average crack pixel width W m .

[0070] The pixel area S of the crack can be obtained by summing the pixel width of all pixel points on the crack center line,

[0071] The calculation formula of S is:

[0072]

[0073] According to the obtained crack parameters such as width, length and area, the severity of the crack can be judged.

[0074] Further, the pixel parameters of the crack in the crack image obtained by the above calculation are needed to calculate the actual size L r of the pixel point according to the camera parameters of the crack image to obtain the actual parameters of the crack. r The calculation formula of L

[0075] Where p is the pixel size, f is the lens focal length, p and f are fixed parameters of the camera for shooting the crack image, and ΔD is the real distance between the camera lens for shooting the crack image and the shooting object.

[0076] The actual size multiplied by the pixel parameter can obtain the actual parameters of the crack.

[0077] The crack measurement performance of the DeepLabv3+(N-S) semantic segmentation model constructed based on the present application will be compared with the traditional solution in the following by combining a specific embodiment, and the model operation speed, model parameter amount and accuracy will be analyzed. The model operation speed is mainly measured by the execution time Et of predicting a single resolution of 152x152 picture; the accuracy is mainly measured by the pixel accuracy PA, the class pixel accuracy CPA and the mean pixel accuracy mPA as comprehensive evaluation indexes, and the intersection over union IoU is a classic evaluation index in the field of semantic segmentation, and the mean intersection over union mIoU is used as a pixel classification performance evaluation index.

[0078] The pixel accuracy PA represents the proportion of the number of correctly classified pixels in all pixels, the class pixel accuracy CPA needs to calculate the proportion of the number of correctly classified pixels of a class in all pixels of the class, and the mean pixel accuracy mPA is the mean value of the pixel accuracy of each class. The closer the above indexes are to 1, the higher the model classification accuracy is. The intersection over union IoU refers to the ratio of the intersection and the union of the predicted value and the real labeled value of a class of pixels, and the mean intersection over union mIoU is obtained by calculating the intersection over union of each class and then averaging. The closer to 1, the better the model classification performance is.

[0079] The calculation formula of the pixel accuracy PA is:

[0080]

[0081] The calculation formula of the class pixel accuracy CPA is:

[0082]

[0083] The calculation formula of the mean pixel accuracy mPA is:

[0084]

[0085] wherein, P ii represents the total number of pixels that are actually i and predicted to be i, i.e. true positive (TP); P jj represents the total number of pixels that are actually j but predicted to be i, i.e. false positive (FP); CPA i represents the class pixel accuracy of the i-th class.

[0086] The calculation formula of the intersection over union IoU is:

[0087]

[0088] The calculation formula of the average intersection over union mIoU is:

[0089]

[0090] wherein, P ji represents the total number of pixel points that are true for the j-th class and are predicted to be the j-th class, i.e., true negative (TN); IoU i represents the intersection over union of the i-th class.

[0091] As shown in Table 1, the comparison and analysis of the crack segmentation performance of the DeepLabv3+(N-S) semantic segmentation model constructed in the present application and other semantic segmentation models under the same data set and training conditions are given. The DeepLabv3+(N-S) semantic segmentation model is obviously improved in each performance compared with the original DeepLabv3+(ResNet101) semantic segmentation model, wherein the average pixel accuracy mPA is improved by 2.12%, the average intersection over union mIoU is improved by 2.54%, the intersection over union IoU reaches 63.65%, which is improved by 2.81% compared with the original DeepLabv3+(ResNet101) semantic segmentation model. Compared with the DeepLabv3+ based on other backbone networks, such as DeepLabv3+(MobileNetv3), DeepLabv3+(MobileNetv2), DeepLabv3+(HRNet48), DeepLabv3+(ResNet50), the advantages are more obvious. This shows that the embedded Non-local block attention mechanism module strengthens the long-distance dependence between crack pixels, thereby improving the accuracy of crack segmentation.

[0092] From the number of model parameters, the improved backbone network model DeepLabv3+(ResNet101-S) has fewer parameters than the original model DeepLabv3+(ResNet101), which indicates that the improved backbone network has fewer parameters and can improve the nonlinear expression ability of the model. Although the parameter amount of the DeepLabv3+(N-S) model after adding the Non-local block attention mechanism module is increased compared with the DeepLabv3+(ResNet101-S) model, it is still lower than that of the original DeepLabv3+(ResNet101) model.

[0093] Table 1

[0094]

[0095]

[0096] The effectiveness of the crack parameter calculation method in the present application is verified by taking the calculation of the maximum crack width as an example. As shown in Table 2, the crack parameter evaluation results calculated based on the DeepLabv3+(N-S) model are given. The absolute error of the crack parameter calculation of 10 crack samples is within 0.4 mm, and the relative error is within 13% when the real distance between the camera lens for shooting the crack image and the shooting object is 50 cm. Figure 6 As shown in Tables 3 and

[0097] Table 2

[0098]

[0099] Table 3

[0100]

[0101] In the present application, multiple small convolution kernels are used to replace large convolution kernels, and asymmetric convolution kernels are used to replace symmetric convolution kernels in the front-end convolution kernel of the backbone network ResNet101 of the DeepLabv3+ model and the residual structure of the Bottleneck bottleneck unit. The number of network layers of the backbone network ResNet101 is increased, the nonlinear expression ability of the model is increased, the Non-local block attention mechanism module is further embedded, the spatial distance information dependence is strengthened, the connection between the pixels in the crack main body during crack segmentation is strengthened, the DeepLabv3+(N-S) semantic segmentation model has better segmentation performance, the error of crack parameter calculation is reduced, and the precision of crack parameter calculation is improved.

[0102] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is exemplified, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the system described in the present application is divided into different functional units or modules to complete all or part of the functions described above.

Claims

1. A crack detection method characterized by, The method comprises: acquiring a crack image, and dividing the crack image into a training set, a verification set and a test set in proportion; building a DeepLabv3+(N-S) semantic segmentation model based on a DeepLabv3+ network model, wherein the DeepLabv3+(N-S) semantic segmentation model comprises an encoder and a decoder, the encoder comprises an improved backbone network ResNet101-S, a spatial pyramid pooling module ASPP and a Non-local block attention mechanism module, wherein the output end of the encoder is connected to the input end of the decoder, the output end of the improved backbone network ResNet101-S is connected to the input end of the Non-local block attention mechanism module, and the output end of the Non-local block attention mechanism module is connected to the input end of the spatial pyramid pooling module ASPP; training the DeepLabv3+(N-S) semantic segmentation model by using crack images in the training set and the verification set, adjusting model network parameters, and obtaining a trained DeepLabv3+(N-S) semantic segmentation model; inputting a crack image in the test set into the trained DeepLabv3+(N-S) semantic segmentation model, outputting a binary mask image of the crack image, inputting the binary mask image into a crack parameter calculation module to calculate crack parameters, and judging the severity of a crack according to the crack parameters.

2. The crack detection method according to claim 1, characterized by, The step of inputting the binary mask image into the crack parameter calculation module to calculate crack parameters comprises: extracting a center axis of a crack in the binary mask image by using a skeleton extraction algorithm, selecting a first pixel point on the center axis and a second pixel point corresponding to the first pixel point on a crack edge, calculating the Euclidean distance between the first pixel point and the second pixel point, and obtaining the crack pixel width of the crack at the first pixel point.

3. The crack detection method according to claim 1, characterized by, The step of building a DeepLabv3+(N-S) deep semantic segmentation model based on a DeepLabv3+ network model comprises: modifying the front-end convolution kernel of a backbone network ResNet101 of the DeepLabv3+ network model and modifying the residual structure of a Bottleneck bottleneck unit to obtain an improved backbone network ResNet101-S, constructing a DeepLabv3+ network model of the improved backbone network, and embedding a Non-local block attention mechanism module in the DeepLabv3+ network model of the improved backbone network to obtain a DeepLabv3+(N-S) semantic segmentation model.

4. The crack detection method according to claim 3, characterized by, The step of modifying the front-end convolution kernel of the backbone network ResNet101 of the DeepLabv3+ network model comprises: replacing the 7×7 front-end convolution kernel of the backbone network ResNet101 with three 3×3 convolution kernels, and inserting a ReLU activation function between the first 3×3 convolution kernel and the second 3×3 convolution kernel and between the second 3×3 convolution kernel and the third 3×3 convolution kernel.

5. The crack detection method according to claim 3, characterized by, The step of modifying the residual structure of the Bottleneck unit comprises: Replace the 3x3 convolution structure in the residual structure of the Bottleneck unit with a 1x3 and a 3x1 convolution structure.

6. The crack detection method according to claim 3, characterized by, The step of embedding the Non-local block attention mechanism module in the improved DeepLabv3+ network model of the main network comprises: Embed the Non-local block attention mechanism module between the output end of the improved main network ResNet101-S and the input end of the spatial pyramid pooling module ASPP module, and between the output end of the main network ResNet101-S and the input end of the decoder.

7. The crack detection method of claim 1, wherein The step of obtaining a crack image and proportionally dividing the crack image into a training set, a validation set and a test set further comprises: Contour labeling is performed on each crack image in the training set, the validation set and the test set to obtain the labeled value of the crack in each crack image, which is used to train the DeepLabv3+(N-S) semantic segmentation model.

8. The crack detection method according to claim 7, characterized by, Further comprising: The crack image in the training set is input into the DeepLabv3+(N-S) semantic segmentation model for training to obtain a predicted value of the crack; a loss value is calculated according to the predicted value and the labeled value, the network parameters of the DeepLabv3+(N-S) semantic segmentation model are adjusted according to the loss value, until the loss value accuracy after validation of the validation set reaches a preset condition, the network parameters of the DeepLabv3+(N-S) semantic segmentation model are fixed, and a trained DeepLabv3+(N-S) semantic segmentation model is obtained.

9. The crack detection method according to claim 8, characterized by, The step of calculating a loss value according to the predicted value and the labeled value comprises: The loss value is calculated using a cross-entropy loss function with an introduced L2 regularization term, wherein the formula of the cross-entropy loss function is: wherein loss represents a loss value, y represents a label value, represents a predicted value, Wi represents a network parameter to be trained, n represents a number of network parameters, and λ represents a weight decay coefficient.

10. The crack detection method of claim 8, wherein, The step of adjusting the network parameters of the DeepLabv3+(N-S) semantic segmentation model according to the loss value comprises: The network parameter adjustment strategy is a stochastic gradient descent method, the learning rate is set to 0.01, the momentum parameter is set to 0.9, and the learning rate decay value is set to 0.0005.

Images