A concrete crack detection method and system based on dynamic coordinate convolution

Abstract

The application provides a concrete crack detection method and system based on dynamic coordinate convolution, relates to the technical field of target detection, and specifically includes the following steps: acquiring a concrete surface crack image set, pre-processing the images to obtain a data set; based on a small-batch stochastic gradient descent optimization algorithm, the data set is used to train a constructed crack detection model; a concrete surface crack image to be detected is input into the trained crack detection model for inference to obtain a crack detection result; wherein the crack detection model is based on YOLOv5, a bottleneck residual structure based on the dynamic coordinate convolution method is added, crack features are extracted, and concrete surface cracks are identified and classified; the application fuses the dynamic coordinate convolution with the YOLOv5 framework system, and greatly improves the concrete crack detection precision under the condition that the number of model parameters is almost unchanged.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of target detection, and particularly relates to a concrete crack detection method and system based on dynamic coordinate convolution. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute prior art.

[0003] Most of the bridges built in the world are concrete bridges. During the service process of the bridges, they need to be regularly detected to facilitate the development of corresponding maintenance countermeasures. Cracks are one of the main diseases of concrete bridges and have become an important content of concrete bridge detection and maintenance. The traditional manual detection method has the disadvantages of inaccuracy and low efficiency. Once the bridge cracks are not detected in time, the collapse of the bridge will cause great property loss.

[0004] At present, some installed concrete crack detection systems have successfully detected and repaired a plurality of bridges by manual on-site repair, preventing the collapse problem caused by the cracking of the bridge deck or pavement, but the existing overall detection system still has many problems in the application process:

[0005] (1) Limited crack feature recognition: existing target detection algorithms mainly focus on object recognition and classification, which may not fully capture the small features and details of cracks, which may lead to poor detection results and be challenging for accurate recognition and positioning of cracks.

[0006] (2) Insufficient adaptability to complex backgrounds: In actual applications, cracks may appear in various complex backgrounds, such as different textures, lighting conditions, etc. Existing target detection algorithms may not be well adapted to these complex backgrounds, thereby affecting the accuracy of crack detection.

[0007] (3) Real-time and computational efficiency: In crack detection, real-time and computational efficiency are crucial, especially in large-scale infrastructure detection. However, many existing target detection algorithms require a large amount of computational resources and time, which may not meet the real-time requirements.

[0008] (4) Robustness and generalization ability: The shape, size, and direction of cracks may vary greatly, and current target detection algorithms may not exhibit good robustness and generalization ability under these different conditions, resulting in limited recognition results. SUMMARY

[0009] To overcome the shortcomings of the prior art, the present application provides a concrete crack detection method and system based on dynamic coordinate convolution, which integrates dynamic coordinate convolution with the YOLOv5 framework system, greatly improves the concrete crack detection accuracy with almost unchanged number of model parameters.

[0010] To achieve the above object, one or more embodiments of the present application provide the following technical solutions:

[0011] The first aspect of the present application provides a concrete crack detection method based on dynamic coordinate convolution.

[0012] A concrete crack detection method based on dynamic coordinate convolution comprises:

[0013] Obtain a concrete surface crack image set, preprocess the images to obtain a data set;

[0014] Based on the small batch stochastic gradient descent optimization algorithm, the data set is used to train the constructed crack detection model;

[0015] The concrete surface crack image to be detected is input into the trained crack detection model for inference to obtain a crack detection result;

[0016] The crack detection model is based on YOLOv5, increases the bottleneck residual structure based on dynamic coordinate convolution method, extracts crack features, and identifies and classifies concrete surface cracks.

[0017] Further, the preprocessing comprises:

[0018] (1) using the K-means method to cluster the image set to generate prior boxes;

[0019] (2) based on the positive and negative sample balance principle, data augmentation is performed on the image set.

[0020] Further, the crack detection model comprises a backbone, a neck and a head;

[0021] The backbone is used to extract image features;

[0022] The neck is used for feature fusion;

[0023] The head is used for decoding and outputting the predicted box coordinate information and classification information of the crack.

[0024] Further, the backbone is composed of Conv, C3, SPPF and bottleneck residual structure, and the specific connection order is: Conv, Conv, C3, Conv, C3, Conv, bottleneck residual structure, Conv, bottleneck residual structure, SPPF.

[0025] Further, the bottleneck residual structure comprises Conv and a residual module;

[0026] The input feature is processed through a Conv and a residual module, and the residual term obtained through the Conv is fused with the feature in the channel dimension to obtain a bottleneck residual image feature.

[0027] Further, the residual module comprises a Conv and a dynamic coordinate convolution module.

[0028] The input feature is processed through a Conv to generate a hidden layer output, and the hidden layer output is input into the dynamic coordinate convolution module to obtain an image feature with coordinate information.

[0029] Further, the dynamic coordinate convolution module comprises a coordinate convolution, a Conv and a dynamic convolution connected in sequence.

[0030] The coordinate convolution adds two coordinate channels to the input feature, and the coordinate feature is extracted through a Concat layer and the input feature.

[0031] The dynamic convolution calculates attention for the input feature, dynamically integrates multiple parallel convolution kernels into a dynamic kernel according to the attention, and extracts dynamic features.

[0032] The second aspect of the present application provides a concrete crack detection system based on dynamic coordinate convolution.

[0033] A concrete crack detection system based on dynamic coordinate convolution comprises a data set construction module, a model training module and a crack reasoning module.

[0034] The data set construction module is configured to obtain a concrete surface crack image set, pre-process the images to obtain a data set.

[0035] The model training module is configured to train the constructed crack detection model using the data set based on a small batch random gradient descent optimization algorithm.

[0036] The crack reasoning module is configured to input the concrete surface crack image to be detected into the trained crack detection model for reasoning to obtain a crack detection result.

[0037] The crack detection model is based on YOLOv5, increases a bottleneck residual structure based on the dynamic coordinate convolution method, extracts crack features, and identifies and classifies concrete surface cracks.

[0038] The third aspect of the present application provides a computer readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of the concrete crack detection method based on dynamic coordinate convolution according to the first aspect of the present application.

[0039] The fourth aspect of the present application provides an electronic device, comprising a memory, a processor and a program stored in the memory and executable on the processor, wherein the processor implements the steps of the dynamic coordinate convolution-based concrete crack detection method according to the first aspect of the present application when executing the program.

[0040] The above one or more technical solutions have the following beneficial effects:

[0041] 1. Based on the deep learning algorithm, a dynamic coordinate convolution CRConv method is introduced to construct a crack detection model YOLOV5-CR, which effectively enhances the detection performance under different crack shapes, sizes and directions, enhances the recognition ability of the model to crack features, improves the recognition and detection accuracy of concrete cracks, and can timely detect the cracks caused by early bridge deck cracking of bridge concrete cracks. The improved system can quickly respond to the detection results, and has an important role in preventing bridge collapse and other measures.

[0042] 2. A crack detection model suitable for the scene is constructed, forming a complete and demand-adapted database model to improve the learning ability of coordinate convolution and dynamic convolution.

[0043] 3. Oriented to the demand, the above network model algorithm is adopted, a large number of data sets are trained to realize more accurate crack image and crack prediction category, and the concrete crack recognition accuracy is greatly improved; the generalization ability of the model is improved by gradually increasing the sample set, and the robustness and stability of the detection algorithm are further optimized.

[0044] The advantages of the additional aspects of the present application will be partially given in the following description, partially will become obvious from the following description, or will be known by the practice of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0045] The drawings accompanying the specification of the present application form a part thereof and serve to provide further understanding of the present application, the illustrative embodiments of the present application and its description serve to explain the present application, and do not constitute improper limitations on the present application.

[0046] Figure 1 The method flowchart of the first embodiment.

[0047] Figure 2 The model structure diagram of the crack detection model YOLOV5-CR of the first embodiment.

[0048] Figure 3 The structure diagram of the dynamic coordinate convolution (CRConv) of the first embodiment.

[0049] Figure 4 The structure diagram of the residual module (CRBottleneck) of the first embodiment.

[0050] Figure 5 Structure diagram of the first embodiment bottleneck residual structure (CRDCSP).

[0051] Figure 6 Structure diagram of the second embodiment. DETAILED DESCRIPTION

[0052] The application will be further described below in conjunction with the accompanying drawings and embodiments.

[0053] Embodiment one

[0054] The embodiment discloses a concrete crack detection method based on dynamic coordinate convolution, realizes a concrete surface crack and category identification process, and mainly includes three steps: data set construction, model training and crack reasoning.

[0055] As shown in Figure 1 A concrete crack detection method based on dynamic coordinate convolution comprises the following steps:

[0056] Step S1: Obtain a concrete surface crack image set, pre-process the images, and obtain a data set.

[0057] First, in order to ensure that the morphology, illumination, size and definition of the crack image have diversity, a large number of concrete surface crack image sets are collected, and then the image set is pre-processed through a k-means algorithm and a positive and negative sample balance principle, specifically as follows:

[0058] (1) The K-means method is used to cluster the image set to generate a prior box.

[0059] In a target detection network (Faster RCNN, SSD, YOLO v2&v3, YOLOv5, etc.), a prior box will be used. The prior box is a box of different sizes and different aspect ratios that is preset in advance on the image.

[0060] In the embodiment, the prior box of the model is obtained by clustering using the K-means method, rather than being artificially designed. Therefore, before carrying out subsequent work, the K-means method is used to obtain a prior box suitable for the concrete surface crack image set for the model.

[0061] In the embodiment, three prior box sizes are clustered on the image set, which are 80*80*128, 40*40*256 and 20*20*512.

[0062] (2) Data augmentation is performed on the concrete surface crack image set based on the positive and negative sample balance principle.

[0063] The original concrete surface crack image set has a sample imbalance problem, that is, the distribution of positive samples and negative samples is uneven, or the number of the two is very different, which will affect the training effect of the model, therefore, based on the positive and negative sample balance principle, the image set is data enhanced to adjust the distribution ratio of positive samples and negative samples, and there are currently three methods:

[0064] (1) Adjusting the value of θ

[0065] According to the positive and negative sample ratio of the image set, the value of θ is adjusted.

[0066] (2) Over-sampling

[0067] The over-sampling is performed on the categories (minority classes) with less number of samples in the image set, and new samples are synthesized to alleviate the class imbalance, for example, the classic over-sampling algorithm SMOTE.

[0068] (3) Under-sampling

[0069] The under-sampling is performed on the categories (majority classes) with more number of samples in the image set, and some samples are discarded to alleviate the class imbalance.

[0070] Step S2: Based on the small batch random gradient descent optimization algorithm, the constructed crack detection model is trained using the data set.

[0071] The following is described from the aspects of construction and training:

[0072] Step S201 of constructing a crack detection model

[0073] The crack detection model YOLOV5-CR constructed by the application is based on the YOLOv5 model, and innovatively adopts a dynamic coordinate convolution method based on deep learning and a bottleneck residual structure, which can accurately and efficiently extract crack features and identify and classify concrete surface cracks.

[0074] Figure 2 is a model structure diagram of the crack detection model YOLOV5-CR, as shown in Figure 2 YOLOV5-CR is divided into three parts: backbone, neck and head. The backbone is responsible for extracting image information features, the neck is responsible for feature fusion, and the head is responsible for decoding and outputting crack prediction box coordinate information and classification information. The size of the prediction box corresponds to the three prior box sizes obtained by clustering in step S1, which are referred to as large, medium and small crack prediction boxes.

[0075] The crack detection model YOLOV5-CR is improved on the basis of the original YOLOv5 model, and specifically, in the backbone of the original YOLOv5 model, the C3 of the sixth layer and the eighth layer is replaced by the bottleneck residual structure (CRDCSP) newly proposed by the application, that is, the backbone of the crack detection model YOLOV5-CR includes Conv, Conv, C3, Conv, C3, Conv, bottleneck residual structure (CRDCSP), Conv, bottleneck residual structure (CRDCSP), and SPPF connected in sequence.

[0076] (1) Conv is composed of convolution, batch normalization and SiLu activation layer, batch normalization has the effect of preventing overfitting and accelerating convergence, SiLu activation layer is a weighted linear combination of Sigmoid, and SiLU function is also called swish function.

[0077] (2) SPPF is improved from SPP, SPP first reduces the input channel by half through a standard convolution, and then performs maximum pooling with a convolution kernel size of 5, 9 and 13; the results of the three maximum poolings and the data without pool operation are concatenated, and the channel number after merging is twice the original.

[0078] SPPF (Spatial Pyramid Pooling-Fast) uses three 5x5 maximum poolings instead of the original 5x5, 9x9 and 13x13 maximum poolings, and multiple small size pooling kernels are cascaded instead of a single large size pooling kernel in the SPP module, thereby further improving the running speed while retaining the original function of fusing feature maps of different receptive fields and enriching the expression ability of the feature maps.

[0079] (3) C3 is a CSP architecture, which includes three standard convolution layers to learn residual features, and its structure has two branches: one branch uses multiple Bottleneck stacks, and the other branch only passes through a basic convolution module, and finally the two branches are concatenated and spliced.

[0080] (4) Bottleneck residual structure (CRDCSP) is based on the structure of C3, and the following improvements are made:

[0081] The Bottleneck in C3 is replaced by the residual module (CRBottleneck) designed by the application, wherein the residual module (CRBottleneck) is constructed based on dynamic coordinate convolution (CRConv).

[0082] Specifically, in order to ensure that the model has high real-time performance, small parameters and calculation amount, and at the same time significantly improve the recognition accuracy, the dynamic coordinate convolution (CRConv) method designed in the application is taken as a basis to construct a bottleneck residual structure (CRDCSP), which will be described in the following order of dynamic coordinate convolution (CRConv), residual module (CRBottleneck) and bottleneck residual structure (CRDCSP).

[0083] Dynamic coordinate convolution (CRConv)

[0084] The dynamic coordinate convolution (CRConv) is obtained by combining the coordinate convolution and the dynamic convolution, Figure 3 The structure diagram of the dynamic coordinate convolution (CRConv) is shown in Figure 3 As shown in the figure, the image features (i.e., feature maps) extracted in advance are input into the dynamic coordinate convolution (CRConv) to obtain features y with dynamic weight coordinates, and the specific processing steps are as follows:

[0085] First, the feature map is input into the coordinate convolution to extract the coordinate feature.

[0086] Specifically, two coordinate channels are added to the feature map to represent the x coordinate and the y coordinate, respectively, and the initial feature map is spliced through the Concat layer. At this time, the feature map has translation dependence, which can be expressed by the formula:

[0087] y1←σ(BN(w1(x)+w x +w y ))

[0088] Where y1 represents the coordinate feature, x represents the input feature map, w1 represents the convolution kernel, w x and w y represent the coordinate convolution kernel, BN represents batch normalization, and sigma represents the ReLU activation function.

[0089] Then, the channel dimension is changed through 1x1 standard convolution, or down-sampling is performed through 3x3 standard convolution, and the coordinate feature is output.

[0090] Finally, the output coordinate feature is input into the dynamic convolution to extract the dynamic feature.

[0091] The basic idea of the dynamic convolution is to adaptively adjust the convolution parameters according to the input feature map, while the corresponding static convolution of the dynamic convolution uses the same convolution kernel to perform the same operation on all input feature maps, and the dynamic convolution adjusts different feature maps with more suitable convolution parameters.

[0092] Specifically, dynamic convolution does not use a single convolutional kernel in each layer. Instead, it dynamically aggregates multiple parallel convolutional kernels based on attention. The attention dynamically adjusts the weights of each convolutional kernel according to the input, thereby generating adaptive dynamic convolutions. The attention is calculated through the ROUTE, expressed by the formula:

[0093] y2←ROUTE(y1)

[0094] y3←σ(BN(w3(COMBINE(y2))))

[0095] y4←Add(y1,y3)

[0096] out←σ(BN(w3(y4)))

[0097] Where y2 represents multi-scale features, y3 represents convolutional features, y4 represents dynamic features, BN represents batch normalization, σ represents ReLU activation function, Concat represents channel-dimensional feature fusion, ROUTE represents attention mechanism, COMBINE represents weighted operation, and Add represents addition operation.

[0098] Residual Module (CRBottleneck)

[0099] Based on dynamic coordinate convolution (CRConv), a residual module (CRBottleneck) is constructed. Figure 4 This is a structural diagram of the residual module (CRBottleneck), as shown below. Figure 4 As shown, during the operation of the residual module, the hidden layer output is first generated through a Conv layer, and then the generated hidden layer output is used as the input of dynamic coordinate convolution (CRConv) for further operation to obtain image features with coordinate information. This approach enhances the detailed information of the feature map and reduces the loss of semantic information during the forward operation.

[0100] Bottleneck Residual Structure (CRDCSP)

[0101] Based on the residual module (CRBottleneck), a bottleneck residual structure (CRDCSP) is constructed. Figure 5 This is a structural diagram of the bottleneck residual structure (CRDCSP), as shown below. Figure 5 As shown, in the bottleneck residual structure, the input features are first processed by the Conv of the backbone branch and the bottleneck residual structure (CRDCSP), and then fused with the residual terms output by the residual block on another branch in the channel dimension to obtain the bottleneck residual image features. This design improves the network depth and performance while reducing the computational cost, as expressed by the formula:

[0102] out←w3(Concat(W(w1(x)),w2(x)))

[0103] wherein w1, w2, w3 represent three layers of Conv convolution kernels respectively, W represents a residual module (CRBottleneck), Concat represents channel dimension feature fusion, out represents a bottleneck residual image feature, and x represents an input feature map.

[0104] Based on the bottleneck residual image feature, the neck further fuses the features, and the head fuses the features to automatically label the cracks in the image, to obtain a crack prediction box. Based on the crack prediction box, a dense connection technology is used to predict the type of the crack, to obtain crack classification information, wherein the type of the crack includes horizontal, vertical, inclined, branch and fissure.

[0105] Step S202 of training a crack detection model

[0106] In the model training stage, the gradient of the model is updated using a small batch stochastic gradient descent optimization algorithm. In order to ensure the accuracy and stability of the model, the data set can be divided into a training set and a validation set according to a predetermined proportion. The validation set is used to calculate the generalization error, and the weight parameters corresponding to the minimum generalization error are saved.

[0107] Step S3: inputting the concrete surface crack image to be detected into the trained crack detection model for inference, to obtain a crack detection result.

[0108] The crack detection model YOLOV5-CR increases the coordinate information through the coordinate convolution in the dynamic coordinate convolution model, realizes more accurate target positioning, automatically generates a convolution kernel according to different input features through the dynamic convolution in the model, and improves the generalization ability and performance of the model. The channel information correlation is realized through the bottleneck residual structure (CRDCSP), and the feature extraction ability of the network is improved. The dynamic coordinate convolution (CRConv) is used to obtain image features with coordinate information, which effectively reduces the demand for computing power of mobile devices, not only overcomes the problems of low efficiency and high false detection rate of manual detection in complex environments, but also greatly improves the accuracy of network model recognition.

[0109] The crack detection model YOLOV5-CR has higher accuracy, lower detection cost and stronger environmental adaptability, realizes real-time identification of concrete surface cracks, and provides convenience for early detection and timely repair of building structure diseases; at the same time, the model has the ability to judge the category of cracks, and according to the category, the damage degree of cracks to the building structure is inferred, which greatly reduces the loss and prevents accidents, and provides strong support for the safety of building structures.

[0110] Embodiment two

[0111] The embodiment discloses a concrete crack detection system based on dynamic coordinate convolution.

[0112] As shown in the figure, a concrete crack detection system based on dynamic coordinate convolution comprises a data set construction module, a model training module and a crack reasoning module: Figure 6

[0113] The data set construction module is configured to: acquire a concrete surface crack image set, pre-process the images to obtain a data set;

[0114] The model training module is configured to: based on a small-batch random gradient descent optimization algorithm, use the data set to train a constructed crack detection model;

[0115] The crack reasoning module is configured to: input a concrete surface crack image to be detected into the trained crack detection model for reasoning to obtain a crack detection result;

[0116] The crack detection model is based on YOLOv5, increases a bottleneck residual structure based on a dynamic coordinate convolution method, extracts crack features, and identifies and classifies concrete surface cracks.

[0117] Embodiment three

[0118] The purpose of the embodiment is to provide a computer-readable storage medium.

[0119] The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the concrete crack detection method based on dynamic coordinate convolution according to Embodiment One of the present disclosure.

[0120] Embodiment four

[0121] The purpose of the embodiment is to provide an electronic device.

[0122] The electronic device comprises a memory, a processor and a program stored on the memory and executable on the processor, and the processor executes the program to implement the steps in the concrete crack detection method based on dynamic coordinate convolution according to Embodiment One of the present disclosure.

[0123] The above is only the preferred embodiment of the present application and is not used to limit the present application. For those skilled in the art, the present application can have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.​

Claims

1. A method for detecting a crack in concrete based on dynamic coordinate convolution, characterized by, include: Obtain a set of images of cracks on the concrete surface, preprocess the images, and obtain the dataset; The crack detection model is trained using a dataset based on the mini-batch stochastic gradient descent optimization algorithm. The image of the concrete surface crack to be detected is input into the trained crack detection model for inference to obtain the crack detection result; The crack detection model, based on YOLOv5, adds a bottleneck residual structure based on dynamic coordinate convolution to extract crack features and identify and classify cracks on the concrete surface. The bottleneck residual structure includes Conv and a residual module; The input features are processed by the Conv and residual modules, and then fused with the residual terms obtained from Conv in the channel dimension to obtain the bottleneck residual image features. The residual module includes Conv and a dynamic coordinate convolution module; The input features are processed by Conv to generate hidden layer outputs, which are then fed into the dynamic coordinate convolution module for computation to obtain image features with coordinate information. The dynamic coordinate convolution module includes coordinate convolution, Conv, and dynamic convolution connected in sequence; The coordinate convolution adds two coordinate channels to the input features, which are then concatenated with the input features through a Concat layer to extract coordinate features. The dynamic convolution calculates attention for the input features, and dynamically integrates multiple parallel convolution kernels into a dynamic kernel based on the attention to extract dynamic features.

2. The method of claim 1, wherein the method is based on dynamic coordinate convolution for concrete crack detection. The preprocessing includes: (1) Cluster the image set using the K-means method to generate prior bounding boxes; (2) Data augmentation of the image set based on the principle of positive and negative sample balance.

3. The method of claim 1, wherein the method is based on dynamic coordinate convolution for concrete crack detection. The crack detection model consists of three parts: the backbone, the neck, and the head. The backbone is used to extract image features; The neck is used for feature fusion; The header is used to decode and output the predicted bounding box coordinates and classification information of the crack.

4. The method of claim 3, wherein the dynamic coordinate convolution-based concrete crack detection method is characterized by, The backbone consists of Conv, C3, SPPF and bottleneck residual structure, with the specific connection order as follows: Conv, Conv, C3, Conv, C3, Conv, bottleneck residual structure, Conv, bottleneck residual structure, SPPF.

5. A dynamic coordinate convolution based concrete crack detection system, characterized in that, A concrete crack detection method based on dynamic coordinate convolution as described in any one of claims 1-4 is adopted, comprising a dataset construction module, a model training module, and a crack inference module: The dataset building module is configured to: acquire a set of images of cracks on concrete surfaces, preprocess the images, and obtain the dataset; The model training module is configured to train the constructed crack detection model using a dataset based on a mini-batch stochastic gradient descent optimization algorithm. The crack inference module is configured to input the image of the concrete surface crack to be detected into the trained crack detection model for inference, and obtain the crack detection result. The crack detection model, based on YOLOv5, adds a bottleneck residual structure based on dynamic coordinate convolution to extract crack features and identify and classify cracks on the concrete surface.

6. An electronic device, characterized in that it comprises: Memory is used to store computer-readable instructions in a non-transitory manner. as well as a processor configured to execute the computer readable instructions, wherein the computer readable instructions, when executed by the processor, perform the method of any of claims 1-4.

7. A storage medium characterized by non-transitorily storing computer readable instructions, wherein, instructions that, when executed by a computer, perform the method of any of claims 1-4.

Images