A concrete surface crack identification method based on a rotating target detection technology
By constructing a Crack-RRPN network and utilizing rotating target detection technology and data augmentation methods, the problem of inaccurate crack identification in existing technologies is solved, achieving efficient and accurate identification of slender cracks.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TONGJI UNIV
- Filing Date
- 2022-09-19
- Publication Date
- 2026-05-29
AI Technical Summary
Existing technologies struggle to accurately identify slender cracks in concrete structures, and the bounding boxes output by existing target detection networks contain a large amount of background, resulting in poor segmentation performance. Furthermore, convolutional features lack rotation invariance, affecting detection accuracy.
A rotating target detection technique is employed. By establishing a dataset targeting crack targets, data augmentation and K-means clustering analysis are performed to construct a Crack-RRPN network. A modified FPN scale allocation formula and a loss function with a SkewIOU factor are used for transfer learning and hyperparameter tuning to output a rotated bounding box, thereby improving detection accuracy.
It achieves more precise positioning of crack targets, reduces background interference, improves the accuracy and precision of crack segmentation, and ensures detection performance in complex scenarios.
Smart Images
Figure CN115511817B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of civil engineering technology, and in particular to a method for identifying cracks on concrete surfaces based on rotating target detection technology. Background Technology
[0002] Cracks are an external manifestation of the condition of concrete structures, and structural diagnosis based on cracking is an important part of the evaluation and assessment of existing structures. However, obtaining detailed crack information requires significant time and manpower, and manually collected crack information often contains omissions and errors, leading to misjudgments of the condition of existing structures and posing safety hazards for continued service. Using computer vision technology to interpret image data and calculate crack feature quantities can significantly improve the efficiency of crack information acquisition, and its effectiveness has been demonstrated in numerous related studies.
[0003] Object detection networks can locate cracks in images, eliminate large areas of background, and reduce the difficulty of crack edge extraction, making them a key algorithm for crack recognition in complex engineering scenarios. Current object detection networks, such as Faster-RCNN and YOLOv3, can only output rectangular boxes. However, cracks are elongated targets, and this characteristic means that the output rectangular boxes often contain a large amount of background, affecting subsequent segmentation results and easily causing large-area overlap of bounding boxes for dense cracks, leading to missed detections. Furthermore, crack propagation directions are variable, but convolutional features lack rotation invariance, making it difficult for existing methods to guarantee detection accuracy. A search of Chinese patent CN114359542A reveals a method for identifying cracks in concrete structures based on computer vision and deep learning. The method includes: S0 training a concrete structure crack detection model; S1 acquiring a visually captured image of the concrete target; and S2 inputting the acquired image into the trained model to obtain the corresponding crack identification result. However, this method fails to detect the minimum bounding box of the target, suffers from high background noise, and has low accuracy in subsequent segmentation and quantization. The loss functions used are the commonly used cross-entropy and smooth-L1 loss, resulting in low precision and completeness for crack detection. These problems can be addressed by detecting the minimum bounding box of the crack target, i.e., using a rotated target detection network. Although such network architectures are widely used in remote sensing image analysis, there is still a lack of targeted algorithm design for targets like cracks with extremely large length and width differences. Summary of the Invention
[0004] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a method for identifying cracks on concrete surfaces based on rotating target detection technology. It is specifically designed for targets with large differences in length and width, such as cracks, and can ensure that it is more accurate and reliable than other rotating target detection networks in crack identification tasks.
[0005] The objective of this invention can be achieved through the following technical solutions:
[0006] A method for identifying cracks on concrete surfaces based on rotating target detection technology includes the following steps:
[0007] S1 establishes a rotating target detection dataset for cracked targets and annotates the crack region;
[0008] S2 performs data augmentation operations on the dataset, including rotation, blurring, flipping, brightening, and background replacement.
[0009] S3 performs K-means clustering analysis on the labeled information in the dataset to determine the anchor box parameters;
[0010] S4 randomly divides the dataset according to a set ratio to form a training set and a test set;
[0011] S5 is based on the Crack-RRPN network to build a deep learning model for crack detection;
[0012] S6 performs transfer learning training on the Crack-RRPN network;
[0013] S7 uses a test set to test the trained network and optimizes the hyperparameters based on the test results.
[0014] S8 uses the Crack-RRPN network, which performed best in the tests, to detect surface cracks in structures.
[0015] Furthermore, the specific method for establishing the dataset in step S1 is as follows:
[0016] S11 collects crack images in different scenarios and annotates the data using manual annotation to obtain a rotating frame;
[0017] S12 uses existing crack semantic segmentation dataset data annotations to automatically calculate the minimum bounding box parameters of cracks and generate multiple rotated box annotations.
[0018] Furthermore, the specific steps of the background replacement operation in step S2 are as follows:
[0019] S21 creates a mask based on the annotation results to determine the location of the crack region in the original image;
[0020] S22 uses a mask to perform Boolean operations on the image, preserving the crack area in the original image and removing the parts of other background images that are located at the mask-indicated positions;
[0021] S23 uses image addition to merge the two to achieve background replacement.
[0022] Furthermore, the specific process of determining the anchor frame parameters in step S3 is as follows:
[0023] S31 uses different numbers of clusters to perform K-means clustering on the height and width of the rotated box annotation;
[0024] S32 calculates the average deviation between the rotated bounding box labels and the cluster centers, and selects the number of clusters based on the deviation;
[0025] S33 determines the anchor frame parameters according to the principle that the anchor frame parameters should be as close as possible to the cluster centers.
[0026] Furthermore, the specific process of constructing the Crack-RRPN network in step S5 is as follows:
[0027] S51 feeds the image into a deep residual neural network to extract image features, and combines it with a feature pyramid network to output feature maps at five different scales.
[0028] S52 inputs the feature map output in step S51 into the RPN network. The RPN network first performs convolution and pooling operations on the feature map, and then uses the rotating anchor box strategy to generate K anchor boxes of different sizes, aspect ratios and angles at each pixel of the feature map, where K is the product of the number of predefined sizes, aspect ratios and angles.
[0029] S53 divides the entire region proposal network into two branches. The first branch is responsible for determining whether the anchor box contains the crack target and calculating the confidence value of each anchor box, i.e. the probability that it is classified as a crack. The second branch is responsible for performing the initial regression from the anchor box to the label box, and then performing non-maximum suppression on the anchor boxes after the location regression. Next, the remaining anchor boxes are sorted according to the confidence value, and 256 of the top-ranked anchor boxes are extracted and output as candidate regions.
[0030] S54 assigns the candidate regions to the feature map based on the candidate region size obtained in step S53 and performs ROI pooling to obtain feature vectors of fixed size. Considering the significant difference between the crack length and width, the scale allocation formula used is as follows:
[0031]
[0032] Where w and h represent the width and height of the candidate region, respectively, k is the appropriate feature layer level for the candidate region, k0 is the feature layer level, and k0 = 4;
[0033] S55 inputs the feature vector obtained in step S54 into the fully connected network to perform class confidence prediction and exact regression of the bounding box position, and outputs the class confidence. With the detection box position parameter v pred ;
[0034] S46 uses a non-maximum suppression filter to output the detection frame, thus obtaining the final detection result.
[0035] Furthermore, the specific process of network training in step S6 is as follows:
[0036] S61 establishes the loss functions for the RPN network and the R-CNN stage, and calculates the corresponding loss terms:
[0037] The loss function of the RPN network includes binary classification loss and candidate region location regression loss. The binary classification loss uses cross-entropy loss, and the candidate region location regression loss uses Smooth-L1 loss. The R-CNN stage uses a loss function with a SkewIOU factor for model training, which also includes the classification loss L1. cls With regression loss L reg Two parts, including classification loss L cls Using cross-entropy loss, regression loss L reg Using Smooth-L1 loss with a SkewIOU factor, the expression for the loss function in this stage is:
[0038]
[0039]
[0040]
[0041] in, For class confidence based on network output, v pred For the detection box position parameters based on the network output, {P crack ,P background} represents the category confidence score of the annotation, and v represents the location parameter of the bounding box of the annotation;
[0042] S62 employs an approximate joint training method for end-to-end learning, which involves summing the loss terms calculated in step S61 to obtain the final loss, and then using this sum for backpropagation to update the parameters.
[0043] Furthermore, the specific steps for adjusting the hyperparameters in step S7 are as follows:
[0044] S71 inputs the test set into the trained network and calculates the average accuracy of the output by combining the network output with the test set annotations.
[0045] S72 Adjust the positive and negative sample thresholds, train the model according to the method in step S6, compare the average accuracy values of the results, and select the hyperparameters of the model with the highest average accuracy value as the optimal hyperparameters.
[0046] Compared with the prior art, the present invention has the following beneficial effects:
[0047] 1. Unlike traditional target detection technologies that can only output rectangular boxes without angular information, this invention uses rotating target detection technology to output a rotating box, achieving more accurate positioning of crack targets. This allows crack segmentation to be performed in a smaller area, avoiding the impact of background interference factors on the accuracy of segmentation and crack quantification.
[0048] 2. In the modeling process, this invention employs RROI pooling and dataset rotation enhancement operations to obtain feature vectors of fixed size, which further improves the rotation invariance of convolutional features and ensures good results even when the crack orientation is random.
[0049] 3. The network architecture and modeling method proposed in this invention take into account the target characteristics of crack targets with large differences in length and width. It specifically modifies the FPN scale allocation formula and introduces a loss function containing the SkewIOU factor, which can improve the detection accuracy of crack targets. Thus, it ensures that the method of this invention is more accurate and reliable than other rotating target detection networks in crack identification tasks. Attached Figure Description
[0050] Figure 1 This is a schematic diagram of the structure of the present invention;
[0051] Figure 2 This is a diagram of the Crack-RRPN network architecture proposed in this invention;
[0052] Figure 3 A schematic diagram of the modified hierarchical allocation formula in the Crack-RRPN network;
[0053] Figure 4 This is a schematic diagram of the RROI pooling operation in the Crack-RRPN network;
[0054] Figure 5 This is a graph showing the K-means clustering results of the bounding boxes in the dataset when K=5.
[0055] Figure 6(a) shows the PR curve of the RRPN model on the test set.
[0056] Figure 6(b) shows the PR curve of the Crack-RRPN model on the test set.
[0057] Figure 7The image shows the results of crack detection in a real-world scenario using the Crack-RRPN network proposed in this invention. Detailed Implementation
[0058] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.
[0059] Example
[0060] like Figure 1 The diagram shown illustrates the structure of the present invention. A method for identifying cracks on concrete surfaces based on rotating target detection technology includes the following steps:
[0061] S1 establishes a rotating target detection dataset for cracked targets and annotates the crack region;
[0062] S2 performs data augmentation operations on the dataset, including rotation, blurring, flipping, brightening, and background replacement.
[0063] S3 performs K-means clustering analysis on the labeled information in the dataset to determine the anchor box parameters;
[0064] S4 randomly divides the dataset according to a set ratio to form a training set and a test set;
[0065] S5 is based on the Crack-RRPN network to build a deep learning model for crack detection;
[0066] S6 performs transfer learning training on the Crack-RRPN network;
[0067] S7 uses a test set to test the trained network and performs hyperparameter tuning based on the test results;
[0068] S8 uses the Crack-RRPN network, which performed best in the tests, to detect surface cracks in structures.
[0069] like Figure 2 The diagram shown is of the Crack-RRPN network architecture proposed in this invention. The specific operation is as follows:
[0070] The image is fed into a deep residual neural network to extract image features, and combined with a feature pyramid network to output feature maps at five different scales.
[0071] The output feature map is input into the RPN network. The RPN network first performs convolution and pooling operations on the feature map, and then uses the rotating anchor box strategy to generate K anchor boxes of different sizes, aspect ratios and angles at each pixel of the feature map, where K is the product of the number of predefined sizes, aspect ratios and angles.
[0072] The entire region proposal network is divided into two branches. The first branch is responsible for determining whether the anchor box contains the crack target and calculating the confidence value of each anchor box, that is, the probability that it is classified as a crack. The second branch is responsible for performing the initial regression from the anchor box to the label box. Then, non-maximum suppression is applied to the anchor boxes after the location regression. Next, the remaining anchor boxes are sorted according to the confidence value, and 256 of the top-ranked anchor boxes are extracted and output as candidate regions.
[0073] Based on the obtained candidate region size, the candidate regions are assigned to the feature map for ROI pooling to obtain feature vectors of fixed size, such as... Figure 3 As shown, considering the significant difference between the crack length and width, the scale allocation formula used is:
[0074]
[0075] Where w and h represent the width and height of the candidate region, respectively, k is the appropriate feature layer level for the candidate region, k0 is the feature layer level, and k0 = 4;
[0076] The obtained feature vectors are input into a fully connected network to perform class confidence prediction and precise regression of bounding box positions, outputting the class confidence score. With the detection box position parameter v pred The detection frame output is filtered by nonmaximum suppression to obtain the final detection result.
[0077] K-means clustering was performed on the labeled information in the dataset. It was found that K=5 strikes a balance between complexity and accuracy. The clustering results are as follows: Figure 5 As shown in Table 1, the anchor frame parameters are determined based on the obtained cluster centers.
[0078] Table 1 Anchor Frame Parameters
[0079]
[0080] A deep learning network, Crack-RRPN, for crack detection was constructed. After model initialization, the Crack-RRPN network was trained using the training set via transfer learning. A learning rate of 0.001 was used during training, with a warmup method to dynamically adjust the learning rate. The training was performed using a single Nvidia RTX 3070 GPU, and the training time was approximately 24 hours. The model loss stabilized after 20 epochs, indicating that the transfer learning was complete.
[0081] The trained network was tested using a test set, and the AP value of the model output was calculated. Simultaneously, the model hyperparameters were modified, and the AP values of different models were compared to find the optimal hyperparameters. Ultimately, it was found that the optimal model was obtained by setting the positive and negative sample threshold to 0.6 based on the default parameter settings. Figure 6(b) shows the PR curve corresponding to the optimal model. Compared to the PR curve of the RRPN network in Figure 6(a), it is more robust, indicating that the model has a higher AP value, and the optimal model is more suitable for crack detection than the RRPN model.
[0082] To demonstrate the effectiveness of the proposed algorithm improvements, including the network structure and modeling method, ablation experiments were conducted using models with different modifications. The results are shown in Table 2.
[0083] Table 2. Experimental results verifying the effectiveness of the network and modeling method proposed in this invention.
[0084]
[0085] As shown in Table 2, compared with the RRPN model, the AP value of the model proposed in this invention is 29.6% higher on the same test set. Furthermore, the improvement proposed in this invention can also significantly improve the model performance, which proves the effectiveness of the method proposed in this invention.
[0086] Furthermore, the Crack-RRPN network was used to detect cracks in a real-world scenario, and the results are as follows: Figure 7 As shown. This demonstrates that the method of the present invention can be used to detect the rotating bounding box of cracks, achieving the desired effect.
[0087] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.
Claims
1. A method for identifying cracks on concrete surfaces based on rotating target detection technology, characterized in that, Includes the following steps: S1 establishes a rotating target detection dataset for cracked targets and annotates the crack region; S2 performs data augmentation operations on the dataset, including rotation, blurring, flipping, brightening, and background replacement. S3 performs K-means clustering analysis on the labeled information in the dataset to determine the anchor box parameters; S4 randomly divides the dataset according to a set ratio to form a training set and a test set; S5 is based on the Crack-RRPN network to build a deep learning model for crack detection; S6 performs transfer learning training on the Crack-RRPN network; S7 uses a test set to test the trained network and optimizes the hyperparameters based on the test results. S8 uses the Crack-RRPN network, which performed best in the tests, to detect surface cracks in structures. The specific process of constructing the Crack-RRPN network in step S5 is as follows: S51 feeds the image into a deep residual neural network to extract image features, and combines it with a feature pyramid network to output feature maps at five different scales. S52 inputs the feature map output in step S51 into the RPN network. The RPN network first performs convolution and pooling operations on the feature map, and then uses the rotating anchor box strategy to generate K anchor boxes of different sizes, aspect ratios and angles at each pixel of the feature map, where K is the product of the number of predefined sizes, aspect ratios and angles. S53 divides the entire region proposal network into two branches. The first branch is responsible for determining whether the anchor box contains the crack target and calculating the confidence value of each anchor box, i.e. the probability that it is classified as a crack. The second branch is responsible for performing the initial regression from the anchor box to the label box, and then performing non-maximum suppression on the anchor boxes after the location regression. Next, the remaining anchor boxes are sorted according to the confidence value, and 256 of the top-ranked anchor boxes are extracted and output as candidate regions. S54 Based on the candidate region size obtained in step S53, the candidate regions are assigned to the feature map using a scale allocation formula for ROI pooling, resulting in feature vectors of fixed size. The scale allocation formula is as follows: in, , These represent the width and height of the candidate region, respectively. k It is a suitable feature layer level for the candidate region. For feature layer level, ; S55 inputs the feature vector obtained in step S54 into the fully connected network to perform class confidence prediction and exact regression of the bounding box position, and outputs the class confidence. With the detection frame position parameters ; The S56 uses a non-maximum suppression filter to output the detection frame, thus obtaining the final detection result.
2. The method for identifying concrete surface cracks based on rotating target detection technology according to claim 1, characterized in that, The specific method for establishing the dataset in step S1 is as follows: S11 collects crack images in different scenarios and annotates the data using manual annotation to obtain a rotating frame; S12 uses existing crack semantic segmentation dataset data annotations to automatically calculate the minimum bounding box parameters of cracks and generate multiple rotated box annotations.
3. The method for identifying concrete surface cracks based on rotating target detection technology according to claim 1, characterized in that, The specific steps of the background replacement operation in step S2 are as follows: S21 creates a mask based on the annotation results to determine the location of the crack region in the original image; S22 uses a mask to perform Boolean operations on the image, preserving the crack area in the original image and removing the parts of other background images that are located at the mask-indicated positions; S23 uses image addition to merge the two to achieve background replacement.
4. The method for identifying concrete surface cracks based on rotating target detection technology according to claim 1, characterized in that, The specific process for determining the anchor frame parameters in step S3 is as follows: S31 uses different numbers of clusters to perform K-means clustering on the height and width of the rotated box annotation; S32 calculates the average deviation between the rotated bounding box labels and the cluster centers, and selects the number of clusters based on the deviation; S33 determines the anchor frame parameters according to the principle that the anchor frame parameters should be as close as possible to the cluster centers.
5. The method for identifying concrete surface cracks based on rotating target detection technology according to claim 1, characterized in that, The specific process of network training in step S6 is as follows: S61 establishes the loss functions for the RPN network and the R-CNN stage, and calculates the corresponding loss terms; S62 employs an approximate joint training method for end-to-end learning, which involves summing the loss terms calculated in step S61 to obtain the final loss, and then using this sum for backpropagation to update the parameters.
6. The method for identifying concrete surface cracks based on rotating target detection technology according to claim 1, characterized in that, The specific steps for adjusting the hyperparameters in step S7 are as follows: S71 inputs the test set into the trained network and calculates the average accuracy of the output by combining the network output with the test set annotations. S72 Adjust the positive and negative sample thresholds, train the model according to the method in step S6, compare the average accuracy values of the results, and select the hyperparameters of the model with the highest average accuracy value as the optimal hyperparameters.
7. The method for identifying concrete surface cracks based on rotating target detection technology according to claim 5, characterized in that, The loss function of the RPN network includes binary classification loss and candidate region location regression loss, wherein the classification loss adopts cross-entropy loss and the regression loss adopts Smooth-L1 loss.
8. A method for identifying concrete surface cracks based on rotating target detection technology according to claim 5, characterized in that, The R-CNN stage described above uses a loss function with a SkewIOU factor for model training, which also includes classification loss. Regression loss The loss consists of two parts: classification loss uses cross-entropy loss, and regression loss uses Smooth-L1 loss with a SkewIOU factor. The expression for the loss function in this stage is: in, The confidence level of the labeled category. The location parameters of the labeled detection box.