A Concrete Surface Crack Detection and Quantification Method Based on a Rotated Bounding Box Object Detection Algorithm

Abstract

The present invention discloses a method for detecting and quantifying concrete surface cracks based on a rotated bounding box object detection algorithm, including the quantification of crack rotation angles and the number of cracks of corresponding categories, and is applicable to the intelligent detection of concrete surface diseases. The steps are as follows: collect concrete surface crack images; formulate marking criteria for concrete surface cracks based on rotated bounding boxes; construct a concrete crack database; optimize the horizontal bounding box object detection method Faster R-CNN, and construct a detection and quantification model for concrete surface cracks of the rotated bounding box object detection method; set hyperparameters, initialize the model and train it; until the model loss function converges, save the network weight parameters according to the best accuracy evaluation index. The method of the present invention not only has good robustness, but also can more finely perform automatic detection of concrete cracks, identification of crack rotation angles, classification, and statistics of the number of cracks of corresponding categories, effectively improving the fineness and efficiency of detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent detection of concrete surface diseases, and particularly relates to the detection of concrete surface cracks based on a deep learning rotated bounding box object detection algorithm, as well as the quantification of the rotation angle of the cracks and the number of cracks of corresponding categories. Background Art

[0002] Concrete materials have good compressive, fire-resistant and durable properties, and are one of the most widely used materials in civil engineering infrastructure structures. They are widely used in civil engineering infrastructure buildings, such as bridges, houses and tunnels. However, during the construction and service processes of concrete materials, due to factors such as construction, harsh service environments and overloading, various diseases are generated, among which the generation of concrete surface crack diseases is particularly frequent. Secondary diseases caused by concrete surface cracks, such as internal steel bar corrosion caused by the cracking of the concrete cover layer, have a relatively serious impact on the safety of concrete. Therefore, the detection of concrete cracks is one of the important detection tasks in its construction stage, service stage and maintenance stage. According to industry specifications, achieving rapid, accurate and quantitative automated detection and data quantification can greatly improve the efficiency of concrete surface crack detection, thereby enhancing the safety performance of the entire life cycle.

[0003] At present, there are mainly four methods for the detection task of concrete surface cracks: (1) Manual detection method, which is the most commonly used method in actual working condition detection. Professional inspection personnel discover and record cracks through visual inspection. The manual detection method is time-consuming and laborious, and has high requirements for the safety and concentration of inspection personnel. (2) Traditional image processing method, whose robustness and accuracy of detection results are unstable in complex environments and cannot adapt to efficient detection under different working conditions; (3) Image recognition method based on machine learning, which overcomes the deficiencies in (1) and (2) to a certain extent, but its generalization ability and anti-noise ability still need to be improved under the actual working conditions of a large number of detection targets and complex environments. (4) Image recognition method based on deep learning, which has good robustness and has good detection accuracy for the detection of a large number of detection targets and complex environments. In the field of intelligent detection technology for concrete surface diseases, as described in documents such as "Review of Crack Detection in Civil Infrastructure Based on Deep Learning" and "Computer vision framework for crack detection of civil infrastructure - A review", the object detection horizontal bounding box algorithm and segmentation algorithm based on deep learning are mainly applied. However, according to the quantitative descriptions of the three distribution categories of horizontal cracks, vertical cracks and inclined cracks in the quantitative standard of relevant industry specifications (such as, "Highway Technical Condition Evaluation Standard JTG / T H21—2011"), the above two commonly used deep learning algorithms cannot achieve refined and automated detection based on the crack rotation angle under the requirements of different detection working conditions. Summary of the Invention

[0004] Aiming at the deficiencies of the prior art, the present invention provides a method for detecting and quantifying concrete surface cracks based on a deep learning rotated bounding box object detection algorithm. According to relevant industry specifications, this method can carry out refined classification based on the crack rotation angle and automated detection of categories and quantification of corresponding quantities for the three categories of concrete surface crack distributions (horizontal cracks, vertical cracks and inclined cracks), or custom categories according to the rotation angle range under different working condition standards, greatly saving labor costs and improving the efficiency and accuracy of detection.

[0005] The technical solution of the present invention is as follows:

[0006] A method for detecting and quantifying concrete surface cracks based on a rotated bounding box object detection algorithm, the steps are as follows:

[0007] S1. Collect concrete surface crack images through an open-source concrete surface dataset;

[0008] S2. Based on the prediction parameters of the rotated bounding box and the crack distribution complexity (the number of single cracks), formulate the concrete surface crack marking criteria based on the rotated bounding box;

[0009] Specifically, the concrete surface crack marking criteria based on the rotated bounding box:

[0010] S2.1. The prediction parameters refer to the x-axis, y-axis coordinates, width, height, and rotation angle of the center point of the rotated bounding box.

[0011] S2.1.1. Specifically, the x-axis and y-axis coordinates of the center point, the width, and the height are defined in the same way as the horizontal bounding box of Faster-RCNN;

[0012] S2.1.2. Specifically, the rotation angle refers to taking the center point coordinates as the rotation center, and taking the long side of the horizontal bounding box along the x-axis direction (in this invention, the x-axis is horizontal to the right and the y-axis is vertical downward) as the rotation starting point, that is, at this time the rotation angle is 0°; after the horizontal bounding box is rotated, taking the long side of the rotated bounding box as the end point, its rotation direction is positive in the clockwise direction (controlled within 0° to 90°) and negative in the counterclockwise direction (controlled within 0° to -90°), and this rotation angle is defined as the rotation angle of the rotated bounding box. Subsequently, this method is simply referred to as the Long Edge (LE) method.

[0013] S2.2. The crack distribution complexity regards the cracks with complex distribution as composed of multiple single cracks, and measures the complexity of the crack distribution on the surface of the same concrete member by the statistical quantity range of the single cracks (this range can be customized according to the actual working conditions). For complex cracks, the intersection points between single cracks are used as the starting and ending points to ensure that the single cracks are enveloped by the rotated bounding box with the smallest area.

[0014] S3. Perform data preprocessing on the collected concrete surface crack images, and construct a concrete surface crack database for training the object detection algorithm based on the rotated bounding box according to the concrete surface crack marking criteria based on the rotated bounding box in step S2;

[0015] S3.1. Perform data augmentation on the contrast and brightness of the concrete surface crack images collected in step S1 to expand the training set data;

[0016] S3.2. Uniformly scale the images after data augmentation in step S3.1 to 1024×1024 pixels for preprocessing;

[0017] S3.3. For the concrete surface crack images obtained in step S3.2, according to the concrete surface crack marking criteria based on the rotated bounding box formulated in step S2, use the roLabelImg software to mark the concrete surface cracks;

[0018] S3.4. Randomly divide the concrete surface crack images marked according to the rotation bounding box marking criterion obtained in step S3.3 into a training set and a validation set at a ratio of 9:1; the training set is used for the training of the rotation bounding box crack detection method, and the validation set is used for the robustness test of the training process of the rotation bounding box crack detection method;

[0019] S4. Based on the horizontal bounding box object detection algorithm Faster-RCNN and related detection specifications, and according to the method for defining the prediction parameters of the rotation bounding box in step S2.1 (where the x-axis and y-axis coordinates of the center point of the prediction parameters, the width and height are as described in step S2.1.1, and the rotation angle prediction parameter θ is added as described in step S2.1.2) and the post-processing classification strategy established based on the reference detection specifications, construct a Rotated Faster-RCNN with post-processing strategy (RFR-P) deep learning model for the concrete surface crack detection and quantification method based on the rotation bounding box object detection algorithm. In particular, the quantification method is the quantification of the crack rotation angle (the rotation angle is defined as described in step S2.1.2, and according to the description of horizontal cracks, inclined cracks and vertical cracks in the specifications, the present invention assigns quantitative classification meanings to the above three types of cracks based on the degree of the rotation angle. The specific ranges of the rotation angles corresponding to the above three different types of descriptions can be customized according to the actual working conditions) and the quantification of the total number of cracks (as described in step S2.2). This RFR-P deep learning model includes a backbone structure, a neck structure, a head structure and a post-processing strategy structure;

[0020] S4.1. The backbone structure adopts a ResNet50 structure containing 5 convolutional layers, which can comprehensively consider the depth of feature extraction and the number of model parameters to achieve feature extraction at different levels;

[0021] S4.2. The neck structure adopts a Feature Pyramid Network (FPN) to fuse the feature maps of different levels of semantic information extracted by the backbone structure from top to bottom;

[0022] S4.3. The head structure is divided into a Region Proposal Network (RPN) and a Rotated Region of Interest (R-RoI). The RPN generates candidate boxes with different aspect ratios and different feature map sizes based on the convolution results of different levels of the Feature Pyramid Network (FPN). To remove multiple candidate boxes for the same target, non-maximum suppression (NMS) is performed on the generated candidate boxes to remove redundant candidate boxes. The R-RoI first sends the horizontal bounding box parameters output by the RPN into the Region of Interest Alignment (RoIAlign) module to detect the mapping of the crack center point to the original image. Then, the Rotated Bounding Box Regression module, which has been enhanced with rotation angle prediction, is used to predict the rotated bounding box parameters in step S2.1, and the Bounding Box Classification module is used to identify cracks.

[0023] S4.4. To ensure the uniqueness of the conversion from a horizontal bounding box to a rotated bounding box and overcome the periodicity of the rotation angle, rotation is performed based on the long edge (LE) method described in step S2.1.2 as a reference to eliminate the periodicity of the rotation angle.

[0024] S4.5. To classify the crack morphology based on the rotation angle of the rotated bounding box, according to the crack morphology quantization standard in the "Highway Technical Condition Assessment Standard JTG / T H21—2011", it is divided into transverse cracks, inclined cracks, and longitudinal cracks.

[0025] S5. Adjust the hyperparameters, apply the transfer learning method, and bring the above-established concrete surface crack dataset based on the rotated bounding box into the RFR-P deep learning model for training.

[0026] S5.1. Set the hyperparameters during the training process, including the initial learning rate, number of training epochs, batch size, momentum, and weight decay. Apply the transfer learning method, and bring the above-established concrete surface crack dataset based on the rotated bounding box into the RFR-P deep learning model for training.

[0027] S5.2. Determine whether the loss function converges as the iteration increases during the training process. If it converges, proceed to step S5.3. If it does not converge, return to S5.1, adjust the parameters, and continue training.

[0028] S5.3. Check whether the results and accuracy metrics of the validation set meet the requirements. If the predicted result image shows a close resemblance to the marked image and the accuracy evaluation metrics meet the requirements, proceed to step S6. If not, return to S5.1, adjust the parameters, and continue training.

[0029] S6. Complete the training of the RFR-P deep learning model and save the training weight parameters of the best test results; for the training model that meets step S5.3, save the weight parameters of the best evaluation metrics.

[0030] Advantages of the present invention: The concrete surface crack detection and quantification method based on the object detection rotated bounding box algorithm of the present invention is based on the horizontal bounding box Faster RCNN, introduces the prediction of the rotation angle parameter, and realizes the RFR-P method of the rotated bounding box. In order to realize the rotated bounding box detection applicable to cracks, the present invention formulates a labeling criterion based on the rotated bounding box and a post-processing method for crack morphology classification based on relevant specifications. In terms of the network structure, the backbone structure ResNet50 is used to comprehensively balance the network depth and the number of parameters; the neck structure FPN fuses the features of each layer to improve the detection accuracy; the head structure RPN provides an anchor box suggestion method and the R-RoI method to realize the feature extraction of the rotated bounding box; the NMS method is applied to eliminate redundant bounding boxes and further improve the accuracy of the detection results. Based on the above, the RFR-P method, the rotated bounding box labeling criterion and the post-processing classification strategy established by the reference specifications proposed by the present invention can provide a guiding basis for crack detection based on the rotated bounding box, and it is expected to apply this invention to crack detection, refined classification and quantification in engineering practice. Description of the Drawings

[0031] Figure 1 is a schematic flow chart of the concrete surface crack detection and quantification method based on the object detection rotated bounding box algorithm of the invention.

[0032] Figure 2 is a schematic diagram of the concrete surface crack labeling criterion based on the rotated bounding box, where (a) is a single crack and (b) is a complex crack.

[0033] Figure 3 is a schematic diagram of converting a horizontal bounding box into a rotated bounding box. That is, the process schematic diagram of converting (C x , C y , w H , h H ) into (C x , C y , w R , h R , θ).

[0034] Figure 4 is a network structure diagram of the concrete surface crack detection and quantification method RFR-P based on the object detection rotated bounding box algorithm.

[0035] Figure 5It is a schematic diagram of the uniqueness criterion for the rotated bounding box. Rotation is performed based on the long edge (LE) method described in step S2.1.2 to eliminate the periodicity of the rotation angle.

[0036] Figure 6 It is a schematic diagram of the quantization classification of the rotation angle. Among them, TC: Transverse Cracks, transverse cracks; DC: Diagonal Cracks, diagonal cracks; LC: Longitudinal Cracks, longitudinal cracks. The quantization classification method of the rotation angle is defined by the range of the rotation angle in the figure. Specific implementation manner

[0037] The following further illustrates the specific implementation manner of the present invention in combination with the attached drawings and technical solutions. The attached drawings give a preferred implementation case of the present invention, and its purpose is to more thoroughly illustrate the disclosed content of the present invention.

[0038] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs. The terms used in the description of the present invention in this specification are only for the purpose of describing specific real-time cases and are not intended to limit the present invention.

[0039] As Figure 1 shown, the present invention provides a method for detecting and quantifying concrete surface cracks based on a rotated bounding box target detection algorithm, which includes the following steps:

[0040] S1. Collect images of concrete surface cracks;

[0041] S2. Develop a criterion for marking concrete surface cracks based on a rotated bounding box according to the prediction parameters of the rotated bounding box and the complexity of crack distribution;

[0042] S3. Perform preprocessing such as data enhancement and cropping on the collected images, and construct a database of concrete surface cracks for training a deep learning neural network based on the above marking criterion;

[0043] S4. Based on the prediction parameters C Hx , C Hy , w H , h H of the horizontal bounding box target detection algorithm Faster - RCNN (which respectively represent the x - axis and y - axis center point coordinates of the horizontal bounding box, the width and height of the horizontal bounding box) and related detection specifications, add the rotation angle prediction parameter θ, that is, C Rx , C Ry , w R , h R, θ (representing the x-axis and y-axis center point coordinates of the rotated bounding box, the width and height of the rotated bounding box, and the rotation angle respectively). Based on the rotation angle and the post-processing classification strategy established by the reference specification, a deep learning model of Rotated Faster-RCNN with post-processing strategy (RFR-P) based on the rotated bounding box object detection algorithm is constructed;

[0044] S5. Adjust the hyperparameters, apply the transfer learning method, and bring the concrete surface crack dataset based on the rotated bounding box established above into the RFR-P model for training;

[0045] S6. Complete the training of the RFR-P model and save the training weight parameters of the best test results.

[0046] Furthermore, the source of the concrete surface images collected in S1 is open-source concrete surface images, images taken by mobile cameras and drones;

[0047] Furthermore, as Figure 2 shown, the marking criterion of the concrete surface cracks based on the rotated bounding box in S2 takes the intersection point between the cracks in the image as the starting (ending) point, and decomposes the complex cracks into multiple single cracks according to the principle that single cracks form medium-complex distributed cracks, and medium-complex distributed cracks form cracked cracks, so that the distribution form of the complex cracks can be represented by the rotated bounding box;

[0048] Furthermore, the specific steps of S3 include:

[0049] S3.1. Perform data enhancement on the contrast and brightness of the data collected in step S1 to expand the training set data;

[0050] S3.2. Uniformly scale the images after data enhancement in step S3.1 to 1024×1024 pixels for preprocessing;

[0051] S3.3. Use the roLabelImg software to mark the concrete surface cracks based on the rotated bounding box marking criterion for the concrete surface crack images in step S3.2 according to the marking criterion in S2;

[0052] S3.4. Divide the concrete surface cracks based on the rotated bounding box marking criterion in step S3.3 according to 90% for the training set and 10% for the validation set; the training set is used for the training of the rotated bounding box crack detection method, and the validation set is used for the robustness test during the training process of the rotated bounding box crack detection method.

[0053] S4. As Figure 3As shown, the prediction parameter C of the horizontal bounding box object detection algorithm Faster-RCNN of S4 Hx , C Hy , w H , h H (representing the x-axis and y-axis center point coordinates of the horizontal bounding box, the width and height of the horizontal bounding box respectively) and related detection specifications, add the rotation angle prediction parameter θ, that is, C Rx , C Ry , w R , h R , θ (representing the x-axis and y-axis center point coordinates of the rotated bounding box, the width and height of the rotated bounding box, and the rotation angle respectively) conversion principle.

[0054] Furthermore, as Figure 4 shown, in step S4, in the RotatedFaster-RCNN with post-processing strategy (RFR-P) deep learning method based on the rotated bounding box object detection algorithm, it includes a backbone structure, a neck structure, a head structure, and a post-processing strategy structure.

[0055] Specifically, S4.1, the backbone structure adopts a ResNet50 structure containing 5 convolutional layers. This structure can comprehensively consider the depth of feature extraction and the number of model parameters to achieve feature extraction at different levels;

[0056] Specifically, S4.2, the neck structure adopts a Feature Pyramid Network (FPN) to fuse the feature maps of different levels of semantic information extracted by the backbone structure from top to bottom, enabling the network structure to have a comprehensive recognition ability for feature maps of different depths, thereby improving the detection accuracy;

[0057] Specifically, in S4.3, the head structure is divided into a Region Proposal Network (RPN) and a Rotating Region of Interest (R-RoI). The former generates candidate box regions with different aspect ratios (1:1, 1:2, 2:1) and different feature map sizes (16×16, 32×32, 64×64, 128×128, 256×256) based on the convolution results of different levels (5 levels) of the FPN. Subsequently, in order to remove the phenomenon of multiple candidate boxes for the same target, the present invention performs Non-Maximum Suppression (NMS) processing on the generated candidate boxes to remove redundant candidate boxes. The latter first sends the horizontal bounding box parameters output by the RPN into RoIAlign to detect the mapping of the crack center point to the original image, and then respectively performs regression analysis through the Bounding Box Regression module with an increased rotation angle to realize the prediction of the rotating bounding box parameters; and performs crack recognition through the Bounding Box Classification module.

[0058] Specifically, the network structure modules of the above S4.1, S4.2, and S4.3 are as described in the following table:

[0059] Taking the Conv2_3 module as an example, Conv2_3 represents that the Conv2 module is repeated 3 times. The input channel numbers 64 / 256 / 256 represent the channel numbers for each repetition. If the channel number is only 64, it means that the input channel number for each repetition is 64; the representation method of the stride is the same as above; C6 represents the corresponding channel numbers of different output layers of the backbone structure, from bottom to top (from deep to shallow network depth) are 2048, 1024, 512, 256; BN represents Batch Normalization; ReLU represents the Rectified Linear Unit non-linear activation function; Maxpooling represents the maximum pooling layer; Upsample represents the upsampling layer; RPN_Conv represents the convolutional layer before the RPN; Regression_Conv and Classification_Conv represent the regression and classification convolutional layers in the RPN; Shared_FC1 and Shared_FC2 are the shared fully connected layers in the R-RoI; Regression_FC and Classification_FC are the regression and classification fully connected layers in the R-RoI.

[0060]

[0061]

[0062] a C6 = 2048, 1024, 512, 256

[0063] Further, S4.4, as Figure 5 shown, to ensure the uniqueness of the conversion of the horizontal bounding box of S4 into a rotated bounding box and overcome the periodicity of the rotation angle, that is, to ensure that the rotation angle is between -π / 2 ≤ θ < π / 2. The present invention rotates with the long edge (Long Edge, LE) as the reference to eliminate the periodicity of the rotation angle.

[0064] Specifically, the calculation methods for the uniqueness of the rotation angle and the long-edge reference are as follows:

[0065] t i,θ = θ i -θ i,a -kπ / 2

[0066] where t i,θ represents the predicted angle θ of the i-th rotated bounding box i minus the predicted angle θ of the anchor box proposed by the RPN of the i-th rotated bounding box i,a , and -kπ / 2 is the key to eliminating the periodicity of the rotation angle;

[0067] Specifically, if k is odd, the original (C Rx , C Ry , w R , h R , θ) has w R , h R swapped, that is, (C Rx , C Ry , w′ R , h′ R , θ), where w′ R = h R , h′ R = w R ; if k is even, the original (C Rx , C Ry , w R , h R , θ) remains unchanged.

[0068] Further, S4.5, as Figure 6As shown in the figure, in order to implement the post - processing method for crack morphology classification based on the rotation angle of the rotated bounding box, according to the crack morphology quantization standard in "Highway Technical Condition Evaluation Standard JTG / T H21—2011", transverse cracks (TC: Transverse Cracks, set - 20°≤θ<20°), diagonal cracks (DC: Diagonal Cracks, set - 70°≤θ<-20° and 20°≤θ<70°); and longitudinal cracks (LC: Longitudinal Cracks, set - 90°≤θ<-70° and 70°≤θ<90°).

[0069] S5. The training process of the method of the present invention includes the forward propagation and backward propagation processes, and the completion of the training needs to meet the requirements of training and testing;

[0070] Specifically, in step S5.1, the hyperparameters are specifically set as follows: the initial learning rate is 0.005, the number of training epochs is 200, the batch size is 2, momentum is 0.9, and weight decay is 0.0001; the transfer learning method is to apply the model parameters of the rotated bounding box in the remote sensing field as the pre - training parameters of the present invention to improve the training accuracy and convergence speed; in the forward propagation process, a fixed number of images are extracted for each round of training and input into the model. After feature extraction by the backbone structure, feature fusion by the neck structure, and classification and regression by the head structure, the predicted data is obtained. The difference between the predicted data and the true label data is compared through the loss function. Among them, the loss functions of RPN and R - RoI are respectively:

[0071]

[0072] Among them, L cls is the average binary cross - entropy loss function containing the sigmoid function, which is used to judge the category of the bounding box proposed by RPN; L reg is the smoothL1Loss loss function, which is used to perform regression calculations on the center point coordinates, width, and height of the bounding box proposed by RPN; λ = 1; p i ∈(0,1), representing the predicted category; represents the true category; t i,j , represents the predicted and true bounding box center point coordinates, width, and height.

[0073]

[0074] Among them, L′ cls is the cross - entropy loss function, which is used to judge the category of R - RoI; L′ regIt is the smoothL1Loss function, which is used to perform regression calculations on the center coordinates, width, height, and rotation angle of the bounding boxes proposed by the RPN; λ′ = 1; p′ i ∈(0,1), representing the predicted category; represents the true category; t′ i,j , represent the predicted and true center coordinates, width, height, and rotation angle of the bounding boxes.

[0075] Specifically, in step S5.2, this loss function is backpropagated, and the weights are updated using the Stochastic Gradient Descent (SGD) method. The error of the loss function is passed forward layer by layer from the last layer, and then the parameters of each layer are updated with the goal of minimizing the loss function. The number of iterations is set to 200 rounds. When the model reaches the maximum number of iterations and the loss function decreases to the convergence process, the training stops;

[0076] Specifically, in step S5.3, in order to test the robustness of the prediction results for images not participating in the training during the training process, the accuracy of brand-new images is verified through the validation set. If the prediction accuracy index reaches more than 85%, it is considered that the process and results of S5 training meet the applicable standards.

[0077] Specifically, the F1 index is used as the accuracy index. This index comprehensively considers the accuracy and recall rate of the prediction results and can more comprehensively reflect the precision and recall of the prediction. The specific formula is as follows:

[0078]

[0079] Among them, P is the accuracy of the validation set, and R is the recall rate of the validation set.

[0080] S6. After meeting the requirements of S5.2 for the convergence of the loss function and S5.3 for the accuracy evaluation index of the validation set, save the model training weight parameters for the detection, classification, and quantification of concrete surface cracks.

[0081] Specifically, after verification, the F1 value of the accuracy evaluation index of the present invention is 90.3%, indicating that the method for detecting and quantifying concrete surface cracks based on the rotated bounding box object detection algorithm provided by the present invention is feasible.

Claims

1. A method for detecting and quantifying concrete surface cracks based on a rotating bounding box object detection algorithm, characterized in that The steps are as follows: S1. Collect concrete surface crack images through an open-source concrete surface dataset; S2. Develop a concrete surface crack marking criterion based on a rotated bounding box according to the prediction parameters of the rotated bounding box and the complexity of crack distribution; Concrete surface crack marking criterion based on a rotated bounding box: S2.

1. The prediction parameters are the x-axis coordinate, y-axis coordinate, width, height, and rotation angle of the center point of the rotated bounding box; S2.1.

1. The x-axis coordinate, y-axis coordinate, width, and height of the center point of the rotated bounding box are defined in the same way as the horizontal bounding box of Faster-RCNN; S2.1.

2. The rotation angle takes the center point coordinates of the rotated bounding box as the rotation center, with the x-axis horizontally to the right and the y-axis vertically downward. The long side of the horizontal bounding box along the x-axis direction is taken as the starting point of rotation, that is, the rotation angle is 0° at this time; after the horizontal bounding box is rotated, the long side of the rotated bounding box is taken as the end point, and its rotation direction is positive in the clockwise direction, controlled within 0° to 90°; negative in the counterclockwise direction, controlled within 0° to -90°. This rotation angle is defined as the rotation angle of the rotated bounding box; this method is called the long side LE method; S2.

2. The complexity of crack distribution regards the cracks with complex distribution as composed of multiple single cracks, and measures the complexity of crack distribution on the surface of the same concrete component by the total number range of single cracks; in complex cracks, the intersection points between single cracks are used as the starting and ending points to ensure that the single cracks are enveloped by the rotated bounding box with the smallest area; S3. Preprocess the collected concrete surface crack images, and construct a concrete surface crack database for training the object detection algorithm based on the rotated bounding box according to the concrete surface crack marking criterion based on the rotated bounding box in step S2; S3.

1. Perform data augmentation on the contrast and brightness of the concrete surface crack images collected in step S1 to expand the training set data; S3.

2. Uniformly scale the images after data augmentation in step S3.1 to 1024×1024 pixels for preprocessing; S3.

3. Use the roLabelImg software to mark the concrete surface cracks for the concrete surface crack images obtained in step S3.2 according to the concrete surface crack marking criterion based on the rotated bounding box formulated in step S2; S3.

4. Randomly divide the concrete surface crack images marked according to the rotated bounding box marking criterion obtained in step S3.3 into a training set and a validation set at a ratio of 9:1; the training set is used for the training of the rotated bounding box crack detection method, and the validation set is used for the robustness test of the training process of the rotated bounding box crack detection method; S4. Based on the horizontal bounding box object detection algorithm Faster-RCNN and related detection specifications, and a post-processing classification strategy established according to the prediction parameters defined in step S2.1 and the reference detection specifications, construct an RFR-P deep learning model for the concrete surface crack detection and quantification method based on the rotated bounding box object detection algorithm; Among them, the quantization method is the quantization of the crack rotation angle and the quantization of the total number of cracks; the RFR-P deep learning model includes a backbone structure, a neck structure, a head structure, and a post-processing strategy structure; S4.

1. The backbone structure adopts a ResNet50 structure containing 5 convolutional layers, which is used to comprehensively consider the depth of feature extraction and the number of model parameters to achieve feature extraction at different levels; S4.

2. The neck structure adopts a Feature Pyramid Network (FPN) to fuse the feature maps of different-level semantic information extracted by the backbone structure from top to bottom; S4.

3. The head structure is divided into a Region Proposal Network (RPN) and a Rotated Region of Interest (R-RoI); the Region Proposal Network (RPN) generates candidate boxes with different aspect ratios and different feature map sizes according to the convolutional results of different levels of the Feature Pyramid Network (FPN); in order to remove multiple candidate boxes for the same object, non-maximum suppression (NMS) is performed on the generated candidate boxes to remove redundant candidate boxes; the Rotated Region of Interest (R-RoI) first sends the horizontal bounding box parameters output by the Region Proposal Network (RPN) into the Region of Interest Align (RoIAlign) module to detect the mapping of the crack center point to the original image; then, the predicted parameters of the rotated bounding box in step S2.1 are realized through the Bounding Box Regression module that adds the prediction of the rotation angle, and the crack is identified through the Bounding Box Classification module; S4.

4. To ensure the uniqueness of the conversion of the horizontal bounding box to the rotated bounding box and overcome the periodicity of the rotation angle, rotation is performed based on the long-edge (LE) method described in step S2.1.2 as a benchmark to eliminate the periodicity of the rotation angle; S4.

5. In order to realize the classification of crack morphology based on the rotation angle of the rotated bounding box, according to the morphology quantization standard for cracks in the "Technical Condition Assessment Standard for Highway Bridges JTG / T H21—2011", it is divided into transverse cracks, inclined cracks, and longitudinal cracks; S5. Adjust the hyperparameters, apply the transfer learning method, and bring the above-established concrete surface crack dataset based on the rotated bounding box into the RFR-P deep learning model for training; S5.

1. The setting of hyperparameters during the training process includes the initial learning rate, the number of training epochs, the batch size, momentum, and weight decay; Apply the transfer learning method, and bring the above-established concrete surface crack dataset based on the rotated bounding box into the RFR-P deep learning model for training; S5.

2. Determine whether the loss function converges with the increase of iterations during the training process; if it converges, enter step S5.3; If it does not converge, return to S5.1, adjust the parameters and continue training; S5.

3. Whether the results and accuracy indicators of the validation set meet the requirements; if the predicted result image display is close to the label and the accuracy evaluation indicators meet the requirements, enter step S6; if not, return to S5.1, adjust the parameters and continue training; S6. Complete the training of the RFR-P deep learning model and save the training weight parameters of the best test results; for the training model that meets the requirements of step S5.3, save the weight parameters of the best evaluation metrics.

Images