Method and system for detecting cracks in concrete surfaces
By employing deep learning algorithms that utilize cross-local dense connection convolution and bottleneck residual structures, the accuracy and robustness of concrete surface crack detection are improved. This addresses the issues of low detection accuracy and high computational cost in existing technologies and is suitable for low-computing environments on mobile devices.
Patent Information
- Application Number
- CN202310443711.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-19
- Publication Date
- 2025-12-05
- Estimated Expiration
- 2043-04-19
AI Technical Summary
In existing technologies, methods for detecting cracks on concrete building surfaces have low accuracy and high computational cost. While lightweight networks reduce computational load, their performance is insufficient, and the lack of feature information flow leads to poor detection performance.
By employing a cross-local dense connection convolution method and a bottleneck residual structure, combined with feature fusion and channel correlation, and using a deep learning object detection algorithm, we reduce memory usage and computational overhead, and improve detection accuracy.
It enables efficient and low-cost detection of concrete surface cracks on mobile devices, with real-time performance and environmental adaptability, improving detection accuracy and robustness, and enabling early detection of structural defects in buildings.
Smart Images
Figure CN116485750B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of building structural defect detection technology, specifically to a method and system for detecting cracks on concrete surfaces. Background Technology
[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.
[0003] Concrete structures are prone to defects over time due to natural factors and human damage, with cracks being the most common defect. Existing artificial early crack detection methods identify cracks on the concrete surface by acquiring image information of the concrete structure, but these methods have low accuracy and poor efficiency.
[0004] To address the aforementioned issues, crack detection based on convolutional neural networks and deep learning target detection algorithms is a widely used technique in the field. However, such algorithms can hardly achieve a balance between accuracy and computational cost. Although they have relatively more accurate recognition results, their computational cost is high.
[0005] While some existing lightweight networks require less computation and effectively reduce the space occupied by the network model, they still have many problems. For example:
[0006] To reduce storage space and computational cost on mobile devices in the later stages of detection model development, many lightweight algorithms, while significantly reducing computational overhead and memory consumption, often sacrifice detection performance. Secondly, many existing lightweight networks are based on depth-wise convolutions, extracting features only in the spatial dimension while ignoring the inter-channel relationships. Due to the lack of or limited flow of feature information between channels, this problem can be amplified during the forward computation of the neural network, leading to suboptimal network performance. Summary of the Invention
[0007] To address the technical problems mentioned above, this invention provides a method and system for detecting cracks on concrete surfaces, which is suitable for lightweight networks for detecting cracks on concrete surfaces running on mobile devices, effectively reducing memory usage and computational overhead.
[0008] To achieve the above objectives, the present invention adopts the following technical solution:
[0009] The first aspect of the present invention provides a method for detecting cracks on concrete surfaces, comprising the following steps:
[0010] Acquire concrete surface image information and extract crack feature information based on the obtained hidden layer output;
[0011] The obtained crack feature information is added to the hidden layer output and channel correlation information, and after fusion and random ordering, the output features are obtained.
[0012] Based on the output features, a prediction box corresponding to the crack feature information is obtained. Prediction boxes that match the crack size are selected and redundant prediction boxes are removed. The coordinate information of the prediction box where the crack is located is decoded and output to obtain the location of the crack.
[0013] The obtained crack feature information is added to the hidden layer output and channel association information, and after fusion and random shuffling, the output features are obtained. Specifically, the features generated by the image after standard convolution are passed through the first residual convolution layer to obtain the hidden layer output, and then through the second residual convolution layer to generate the output, and then added to the input features point by point; after point-to-point addition, it is fused with the feature information processed by the standard convolution layer.
[0014] By obtaining a dataset of concrete surface cracks, we divide it into a training set for training and a validation set for validation.
[0015] On the training set, cracks are classified according to their morphology. The probability of crack category is predicted by using activation function regression. The category with the highest probability value is taken as the predicted category information of crack. After training, the system is equipped with the ability to determine crack category.
[0016] During training, the predicted box with the highest confidence value is selected. The intersection-union ratio (IUR) of this predicted box and all other predicted boxes is calculated. When the IUR exceeds a set value, the predicted box is considered an overlapping redundant box and is removed.
[0017] During training, after obtaining the predicted bounding boxes corresponding to the dimensions of the crack feature information, the cross-union ratio (CUI) between the predicted bounding boxes and the actual crack bounding boxes in the training set is determined, and the predicted bounding box with the largest CUI with the actual bounding box is selected as the predicted bounding box of the crack.
[0018] After training, the predicted bounding box coordinates and crack category of the crack are output based on the acquired concrete image information.
[0019] A second aspect of the present invention provides a system for implementing the above-described method, comprising:
[0020] The image acquisition module is configured to: acquire image information of the concrete surface and extract crack feature information based on the obtained hidden layer output;
[0021] The feature fusion module is configured to add the obtained crack feature information to the hidden layer output and channel association information, and then obtain the output features after fusion and random ordering.
[0022] The crack recognition module is configured to: obtain a prediction box corresponding to the dimension of the crack feature information based on the output features; select the prediction box that matches the crack size and remove redundant prediction boxes; and decode and output the coordinate information of the prediction box where the crack is located to obtain the location of the crack.
[0023] A third aspect of the present invention provides a computer-readable storage medium.
[0024] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the concrete surface crack detection method described above.
[0025] A fourth aspect of the present invention provides a computer device.
[0026] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the concrete surface crack detection method described above.
[0027] Compared with existing technologies, one or more of the above technical solutions have the following beneficial effects:
[0028] 1. By improving the accuracy of detection and reducing dimensionality through feature fusion, feature reuse, and information correlation between channels, the dimensionality is reduced, thereby effectively reducing storage space consumption and the computational power requirements of the network model on mobile devices. This overcomes the problems of low detection efficiency and high false positive rates caused by the susceptibility of manual detection to environmental factors such as weather, visibility, water stains, shadows, and the user's own condition. Simultaneously, it effectively reduces the memory consumption and floating-point computation of the network model, resulting in higher accuracy, lower detection costs, and stronger environmental adaptability. It achieves real-time identification of cracks on concrete surfaces, facilitating the early detection and timely repair of structural defects in buildings.
[0029] 2. After training on a large dataset, the algorithm achieves more accurate crack images and crack category prediction, while effectively reducing the space and computing power requirements of existing detection algorithms. Furthermore, by gradually increasing the number of dataset samples, the generalization ability of the network model can be improved, further optimizing the robustness and stability of the detection algorithm.
[0030] 3. On the training set, cracks are classified according to their morphology, and the probability of each crack category is predicted by regression. The category with the highest probability value is taken as the predicted category information of the crack, so that the network can determine the crack category after training. Attached Figure Description
[0031] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.
[0032] Figure 1 This is a schematic diagram of a cross-local dense connection convolution structure in the process of detecting cracks on concrete surfaces provided by one or more embodiments of the present invention;
[0033] Figure 2 This is a schematic diagram of the calculation process of the residual block (IFDBottleneck) in the concrete surface crack detection process provided by one or more embodiments of the present invention;
[0034] Figure 3 This is a schematic diagram of the bottleneck residual structure (IFDCSP) in the concrete surface crack detection network provided by one or more embodiments of the present invention.
[0035] Figure 4 This is a schematic diagram of the architecture of a concrete surface crack detection network provided in one or more embodiments of the present invention. Detailed Implementation
[0036] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0037] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.
[0038] The following embodiments provide a method and system for detecting cracks on concrete surfaces. After extracting image feature information and downsampling, a bottleneck residual module based on the convolution method is introduced for information acquisition and feature fusion. This method is applicable to lightweight networks for detecting cracks on concrete surfaces running on mobile devices, effectively reducing memory usage and computational overhead.
[0039] Example 1:
[0040] like Figure 1-4 As shown, the method for detecting cracks on concrete surfaces includes the following steps:
[0041] Acquire concrete surface image information and extract crack feature information based on the obtained hidden layer output;
[0042] The obtained crack feature information is added to the hidden layer output and channel correlation information, and after fusion and random ordering, the output features are obtained.
[0043] Based on the output features, a prediction box corresponding to the crack feature information is obtained. Prediction boxes that match the crack size are selected and redundant prediction boxes are removed. The coordinate information of the prediction box where the crack is located is decoded and output to obtain the location of the crack.
[0044] Specifically:
[0045] To enhance the correlation between feature map channels, the detection network in this embodiment is based on a lightweight convolutional method across local dense connections (IFDConv), and its structure is as follows: Figure 1 As shown in Table 1, the method combines feature fusion, random channel shuffling, and depth-wise convolution. The image is first processed by a standard convolution with a 1x1 kernel to change the channel dimension, or by downsampling with a 3x3 standard convolution to generate the hidden layer output. This output is then fed into a depth-wise convolution to extract crack feature information in a cost-effective manner in the spatial dimension. Previous feature information and channel association information are then added together. A locally dense connection method is used to perform multi-number feature fusion in the channel dimension, improving feature reuse capabilities. Randomly shuffling the channel order promotes the flow of feature information in the channel dimension. Finally, a 1x1 standard convolution reduces the channel dimension to decrease floating-point computation. Each standard convolution (Conv) and depth convolution (DWConv), in addition to representing the convolution algorithm, includes batch normalization and a swish activation function. After batch normalization and the swish activation function, crack features are fused in the channel dimension.
[0046] Table 1 Lightweight Convolution Algorithm Across Local Dense Connections
[0047]
[0048]
[0049] Where x is the input and y1 is the output. Figure 1 The features after standard convolution (Conv), where y2 is the result of... Figure 1 Features derived from medium-depth convolution (DWConv) are summed from the two features.
[0050] To increase the depth of neural networks and effectively reduce floating-point computation, this embodiment introduces a residual module based on cross-local dense connection convolution (IFDBottleneck) and a bottleneck residual structure based on this module (IFDCSP, as shown in Table 2). Its structure is as follows: Figure 2 and Figure 3As shown. During residual block computation, the input features are processed by the first IFDConv (residual convolution) to obtain the hidden layer output, and then processed by the second IFDConv (residual convolution) to produce the output. This output is then added point-to-point with the input features to enrich the feature information and reduce the loss of semantic information during the forward computation. Here, the input features refer to... Figure 4 The output of the next layer in the network architecture.
[0051] In this embodiment, the features generated by the standard convolutional layer are processed by the first IFDConv (residual convolution) to obtain the hidden layer output, and then processed by the second IFDConv (residual convolution) to generate the output, which is then added point-to-point with the input feature points.
[0052] After point-to-point addition, the features are fused with the features processed by the standard convolutional layer (Conv). That is, in the bottleneck residual structure, the input features are processed by two standard convolutional layers (Conv). The output of one standard convolutional layer (Conv) is processed by IFDBottleneck and then fused with the other standard convolutional layer (Conv) in the channel dimension. The fused feature is then output by the standard convolutional layer (Conv).
[0053] y = w2(w1(x)) + x (1)
[0054] y = w2(w1(x)) (2)
[0055] Equations (1) and (2) are the calculation methods for IFDBottleneck when the number of input and output channels are equal and unequal, respectively, where x is the feature input, w1 and w2 are two hidden convolutional kernels, and y is the feature output.
[0056] Table 2 Bottleneck Residual Structure Algorithm
[0057]
[0058] The detection network in this embodiment uses YOLOv5 as the baseline model and employs a self-built deep learning-based cross-local dense connection convolution method and its bottleneck residual structure. The network architecture is as follows: Figure 4 As shown, the system is divided into three parts: the backbone, the neck, and the head. The backbone is responsible for image feature extraction, the neck for feature fusion, and the head has three output layers that decode and output the coordinate information of the predicted bounding boxes for large, medium, and small objects, respectively. After obtaining the predicted bounding boxes corresponding to the dimensions of the crack feature information, the intersection-over-union (IoU) with the ground truth bounding boxes is calculated during training, and the predicted bounding box with the largest IoU is selected for coordinate information adjustment.
[0059] This detection network improves detection accuracy through extensive feature fusion, feature reuse, and inter-channel information correlation. It effectively reduces storage space requirements and the computational demands on mobile devices by reducing dimensionality and utilizing depth-wise convolution to inexpensively obtain image feature information. It overcomes the problems of low detection efficiency and high false positive rates caused by manual detection being susceptible to environmental factors such as weather, visibility, water stains, shadows, and the network's own condition. Simultaneously, it effectively reduces the network model's memory usage and floating-point computation, resulting in higher accuracy, lower detection costs, and stronger environmental adaptability. It achieves real-time identification of cracks on concrete surfaces, facilitating early detection and timely repair of structural defects in buildings.
[0060] In addition, this network has the ability to determine the type of crack and infer the degree of damage to the building structure based on the type of crack, so as to buy time for early preparation of repair measures, thereby greatly reducing losses and even preventing accidents in advance, and providing a strong guarantee for ensuring the safety of building structures.
[0061] In this embodiment, the process of determining the crack category is as follows: On the training set, cracks are classified into five categories based on their morphology: branching, transverse, longitudinal, inclined, and crazing. Softmax regression is used to predict the probability of each crack category, and the category with the highest probability value is taken as the predicted crack category information. This enables the network to determine the crack category after training.
[0062] The process of identifying concrete surface cracks and their categories can be logically divided into three parts: model training, network loading, and crack inference.
[0063] A large dataset was collected to ensure the diversity of crack image morphology, brightness, noise, and sharpness. The dataset was filtered and preprocessed using clustering algorithms and sample balance principles. The network model was updated with parameters using a mini-batch stochastic gradient descent optimization algorithm (as shown in Table 3) on a large dataset. Its generalization error was calculated using a large number of validation sets. The weight parameters corresponding to the minimum generalization error were saved to ensure the accuracy and stability of the network model.
[0064] Table 3 Mini-batch Stochastic Gradient Descent Algorithm
[0065]
[0066] After training and validation, the weight parameters and network architecture are loaded, and the image information is fed into the network for feature extraction, encoding, decoding, and feature fusion. Crack inference is based on the non-maximum suppression algorithm, which selects the most suitable prediction box and removes redundant boxes. The prediction box is then decoded to output the coordinate information of the crack prediction box and the crack category. During the prediction process, the non-maximum suppression algorithm selects the prediction box with the highest confidence value. This prediction box is then compared with all other prediction boxes to calculate the intersection-union ratio (IUR). If the IUR exceeds a set value, the prediction box is considered an overlapping redundant box and is removed.
[0067] Based on deep learning algorithms, this embodiment introduces a lightweight convolution method with dense cross-local connections, which can effectively enhance the information flow between feature map channels and improve the reliability and accuracy of concrete surface crack detection tasks.
[0068] This embodiment is demand-driven, employing the lightweight convolutional method described above to construct a concrete surface crack detection network. Training with a large dataset achieves more accurate crack images and crack category predictions. This effectively reduces the space and computational requirements of existing detection algorithms. By gradually increasing the number of dataset samples, the generalization ability of the network model is improved, further optimizing the robustness and stability of the detection algorithm.
[0069] This embodiment constructs a network model suitable for running on mobile devices, forming a stable and lightweight detection algorithm that adapts to the needs, so as to achieve a balance between computing power requirements and prediction accuracy, making it more suitable for mobile devices with low computing power, and providing technical support for on-site concrete surface crack detection tasks.
[0070] Example 2:
[0071] A system for implementing the above method includes:
[0072] The image acquisition module is configured to: acquire image information of the concrete surface and extract crack feature information based on the obtained hidden layer output;
[0073] The feature fusion module is configured to add the obtained crack feature information to the hidden layer output and channel association information, and then obtain the output features after fusion and random ordering.
[0074] The crack recognition module is configured to: obtain a prediction box corresponding to the dimension of the crack feature information based on the output features; select the prediction box that matches the crack size and remove redundant prediction boxes; and decode and output the coordinate information of the prediction box where the crack is located to obtain the location of the crack.
[0075] Example 3:
[0076] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the concrete surface crack detection method described in Embodiment 1 above.
[0077] Example 4:
[0078] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the concrete surface crack detection method described in Embodiment 1 above.
[0079] The steps or networks involved in Embodiments 2 to 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.
[0080] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for detecting cracks in a concrete surface, characterized by, The method comprises the following steps: Obtaining concrete surface image information, respectively processing the information through two standard convolution layers, wherein the output of one of the standard convolution layers is processed through a residual module IFDBottleneck, and then the output is fused with the other standard convolution layer in the channel dimension, and the fused output is output through a standard convolution layer; Based on the output features, a prediction box corresponding to the crack feature information is obtained, a prediction box matching the crack size is selected and redundant prediction boxes are removed, the coordinate information of the prediction box where the crack is located is output through decoding, and the position where the crack is located is obtained; The features input into the residual module IFDBottleneck are processed through a first residual convolution IFDConv to obtain hidden layer output, and then the output is processed through a second residual convolution IFDConv to generate output, and then the output is added to the input features point by point; In the residual convolution IFDConv, input features are obtained through standard convolution to obtain output , are obtained through a deep convolution layer to obtain output , and are obtained through feature addition processing to obtain added features ; the output of the standard convolution , the output of the deep convolution layer and the added features are fused in the channel dimension to obtain , which is output after standard convolution; wherein , is channel dimension feature fusion, is random shuffling of the channel order.
2. The method of claim 1, wherein the step of detecting the crack in the concrete surface is performed by using a crack detection device. Through obtaining a concrete surface crack data set, a training set is divided to realize training, and a validation set is divided for verification.
3. The method of claim 2, wherein the step of detecting the crack in the concrete surface is performed by using a crack detection device. On the training set, the classes of cracks are divided according to the crack morphology, the class probability of the crack is predicted by using an activation function, the class with the maximum probability value is taken as the predicted class information of the crack, and after the training is completed, the ability to determine the crack class is obtained.
4. The method of claim 1, wherein the step of detecting the crack in the concrete surface is performed by using a crack detection device. During the training process, after obtaining the prediction box corresponding to the crack feature information, the intersection-over-union size between the prediction box and the labeled box where the real crack is located in the training set is determined, and the prediction box with the maximum intersection-over-union with the real box is selected as the prediction box where the crack is located.
5. The method of claim 2, wherein the step of detecting the crack in the concrete surface is performed by using a crack detection device. During the training process, the prediction box with the maximum confidence value is selected, the intersection-over-union between the prediction box and all other prediction boxes is calculated, and when the intersection-over-union exceeds a set value, the prediction box is regarded as an overlapping redundant box, and is removed.
6. The method of claim 2, wherein the step of detecting the crack in the concrete surface is performed by using a crack detector. After the training is completed, the prediction box coordinate information and the crack class of the crack are output based on the obtained concrete image information.
7. A concrete surface crack detection system for implementing the concrete surface crack detection method according to any one of claims 1 to 6, characterized by, It comprises: An image acquisition module configured to obtain concrete surface image information; A feature fusion module configured to obtain the concrete surface image information, respectively process the information through two standard convolution layers, wherein the output of one of the standard convolution layers is processed through a residual module IFDBottleneck, and then the output is fused with the other standard convolution layer in the channel dimension, and the fused output is output through a standard convolution layer; A crack identification module configured to obtain a prediction box corresponding to crack feature information based on the output features, select a prediction box matching the crack size and remove redundant prediction boxes, output the coordinate information of the prediction box where the crack is located through decoding, and obtain the position where the crack is located. 8.A computer readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the steps of the concrete surface crack detection method according to any one of claims 1-6. 9.A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the concrete surface crack detection method according to any one of claims 1-6 when executing the program.