Bridge crack automatic classification and detection method based on two-step sequence convolutional neural network

Abstract

This invention discloses an automatic classification and detection method for bridge cracks based on a two-step sequential convolutional neural network, comprising the following steps: S1, collecting bridge crack data and constructing a basic dataset; S2, constructing an image filtering network based on an attention mechanism to filter image data containing cracks; S3, constructing high-quality basic data based on the crack detection dataset; S4, constructing a crack detection network to detect bridge cracks. The advantages are: the overall image hierarchical network structure constructed in this invention reduces time latency while preserving high-dimensional features, and reduces computational load while improving performance. It can accurately reflect whether the corresponding part of the input image has cracks, achieving the purpose of rapid filtering. The crack detection network constructed in this invention achieves compatibility with multiple scale features in its overall structure, including feature fusion and loss function parts, and the network has better adaptability to the ground truth bounding boxes of cracks.

Description

Technical Field

[0001] This invention relates to the field of bridge health monitoring technology, and in particular to an automatic classification and detection method for bridge cracks based on a two-step sequential convolutional neural network. Background Technology

[0002] The appearance of cracks in bridges reduces their structural load-bearing capacity and leads to various bridge defects. Therefore, bridge crack detection has become one of the important research topics in bridge maintenance. In recent years, many excellent target detection models have achieved good results in practical applications. However, the complex and diverse textures and irregular distribution of bridge crack images lead to serious missed detections and false detections. Furthermore, the large number of parameters in these models makes them difficult to deploy on mobile devices, which is not conducive to practical engineering applications.

[0003] The advantages of deep learning technology lie in its ability to learn data-driven, adaptive features, low dependence on specific mathematical models, and transfer learning capabilities from the source domain. Therefore, many scholars have conducted extensive research on crack image and background image classification methods based on deep learning. Torres et al. compared neural network (NN) classifiers, Bayesian classifiers, and traditional support vector machine (SVM) classifiers, summarizing the potential of CNNs in bridge deck defect classification; Cha et al. used CNNs to detect concrete cracks and used a sliding window technique to scan crack images of arbitrary sizes, verifying the reliable performance of CNNs in distinguishing a large number of categories, although this method is time-consuming; Liang et al. used a Faster R-CNN algorithm to automatically detect structural components of reinforced concrete bridge systems using bounding boxes; Joseph Redmon et al. proposed the YOLO model in 2016, employing a regression concept, removing the candidate frame stage, using the Anchor-Free concept to achieve first-level detection, and immediately outputting category and location information through the network, achieving end-to-end learning, significantly improving detection speed, and enabling object detection networks to truly detect in real time; YAN et al. utilized Alex based on the Caffe framework... The Net network enables the identification of crack images; CVDung et al. used the VGG16 network based on transfer learning to identify cracks in steel bridges, and the robustness and accuracy were improved compared with the original VGG16 network.

[0004] Although existing deep learning-based detection methods have high accuracy in crack identification, the sparse distribution of bridge cracks and the large amount of image data acquired by existing detection technologies make it an important issue to quickly filter image data containing cracks to improve detection efficiency. Summary of the Invention

[0005] The purpose of this invention is to provide an automatic classification and detection method for bridge cracks based on a two-step sequential convolutional neural network, thereby solving the aforementioned problems existing in the prior art.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] An automatic classification and detection method for bridge cracks based on a two-step sequential convolutional neural network includes the following steps:

[0008] S1. Collect bridge crack data and construct a basic dataset;

[0009] S2. Construct an image filtering network based on an attention mechanism to filter image data containing cracks;

[0010] A classification dataset containing an equal number of cracked and crack-free images is constructed based on the base dataset, and the classification dataset is divided into training and test sets proportionally. An attention-based image filtering network is trained using the training set, and the network model parameters are updated using BCE With Logits Loss as the loss function. The network is then tested using the test set to obtain the trained attention-based image filtering network. The trained attention-based image filtering network is then used to filter images containing cracks in the base dataset to construct a crack detection dataset.

[0011] S3. Construct high-quality basic data based on the crack detection dataset;

[0012] The boundary boxes of the marked cracks are checked multiple times based on the average height-width of the circumferential and longitudinal cracks to obtain high-quality basic data;

[0013] S4. Construct a crack detection network to detect cracks in the bridge;

[0014] A crack detection network, YOLOv3-spp, was constructed based on the YOLOv3 architecture. A high-quality basic dataset was proportionally divided into training, validation, and test sets. The crack detection network was trained using the training set, and the network model parameters were updated using a total loss function consisting of a classification loss function, a confidence loss function, and a bounding box localization loss function. The network was then validated and tested using the validation and test sets to obtain a well-trained crack detection network. The trained crack detection network was then used to classify and locate cracks in selected images, and the crack recognition results were output.

[0015] Preferably, the main module of the attention-based image filtering network is B-neck, which combines the depthwise separable convolution of MobileNetV1 and the linear bottleneck inverted residual of MobileNetV2. The attention-based image filtering network adopts the SE attention mechanism and the H-Swish function. The attention mechanism is used to adjust the weight of each channel, and the H-Swish function is used to reduce the amount of computation and improve performance. A convolutional layer of size 1 is used after the pooling layer, and the last two convolutional layers perform two convolutions of size 1 to obtain a feature map of size 5×3×1. These 15 pixels reflect whether there are cracks in the corresponding part of the input image. The maximum set method is used to determine whether the entire image contains cracks.

[0016] Preferably, in step S1, before training and testing the attention-based image filtering network using the training and test sets, the image sizes of the training and test sets are adjusted to a preset size.

[0017] Preferably, step S3 further includes performing data augmentation processing on all image data in the crack detection dataset, wherein the data augmentation processing includes three methods: geometric transformation, stitching, and Mixup data augmentation.

[0018] Preferably, the crack detection network YOLOv3-spp uses Darknet-53 as the backbone network and employs multi-scale detection. It divides an image into blocks of multiple scales, extracts features from each block, and fuses them to achieve compatibility with features at multiple scales. This part consists of CBS, CBS2, and SPPF modules, and completes feature fusion through channel-based concatenation and 1x1 convolution. It includes three detection heads of different scales, corresponding to downsampling of 5x, 4x, and 3x respectively.

[0019] Preferably, during the training of the crack detection network, anchor boxes are aggregated, and preliminary output is obtained through the generated anchor boxes and the designed network. Redundant prediction boxes are filtered through a non-maximum suppression algorithm to obtain the final prediction boxes. For high-quality basic datasets, K-Means clustering algorithm is used to adaptively generate anchor boxes, and genetic algorithm is used to overcome the limitations of local optimization.

[0020] Preferably, the total loss function is calculated as follows:

[0021] loss 总 =l cls +l obj +l box

[0022]

[0023]

[0024]

[0025]

[0026]

[0027]

[0028] Among them, l cls l obj l box These are the classification loss function, confidence loss function, and bounding box localization loss function, respectively; λ cls λ represents the weight coefficients of the classification loss function. obj ,λno obj λ represents the weighting coefficients of the confidence loss function. box The weight coefficients of the loss function for bounding box localization; S1×S2 is the dimension of the feature map; B is the number of anchor boxes used for dense prediction; This indicates selecting a high-quality prediction box; p i (c) represents the probability of class c. This is the corresponding predicted value; c i Output the model for the i-th sample; This represents the true value of the category corresponding to the i-th sample; To select poor-quality prediction boxes; IoU is the intersection ratio between the prediction box and the ground reality; ρ 2 (A,B) is the Euclidean distance between two boxes A and B; k is the diagonal distance of the smallest enclosed region that can simultaneously contain the predicted box and the ground truth; υ is a measure of the consistency of the relative proportions of the two rectangles; α is a weighting coefficient; w gt and h gt , where w and h are the width and height of the ground reality, respectively; w and h are the width and height of the prediction bounding box, respectively.

[0029] Preferably, to accelerate convergence and make the network more usable with the locations of the bounding boxes, the coordinates of the bounding boxes need to be transformed, as shown in the formula.

[0030]

[0031] b x =2σ(t) x -0.5+C x

[0032] b y =2σ(t) y -0.5+C y

[0033] bw =P w (2σ(t w )) 2

[0034] b h =P h (2σ(t h )) 2

[0035] Among them, (b) x ,b y ,b w ,b h ) represents the coordinates of the bounding box; (t) represents the coordinates of the bounding box. x ,t y ,t w ,t h (C) represents the data in the network output tensor corresponding to the bounding box coordinates; x C y ) represents the coordinates of the grid center; (P) w ,P h ) represents the width and height of the anchor box.

[0036] Preferably, step S1 specifically involves using intelligent detection equipment to collect bridge crack image data and construct a basic dataset; the intelligent detection equipment integrates sensors including a high-speed array camera and an odometer through a synchronous control system, constructs a spatiotemporal reference and synchronous control relationship between multiple sensors, and realizes the collaborative work between the sensors.

[0037] The beneficial effects of this invention are: 1. This invention provides a two-stage sequential crack identification method to address the sparsity of crack distribution, effectively accelerating detection efficiency. 2. The image hierarchical overall network structure constructed in this invention reduces time delay while preserving high-dimensional features, and reduces computational load while improving performance. It can more accurately reflect whether there are cracks in the corresponding parts of the input image, achieving the purpose of rapid screening. 3. The crack detection network constructed in this invention achieves compatibility with multiple scale features in its overall structure, including feature fusion and loss function parts. In addition, the network has better adaptability to the ground truth bounding boxes of cracks. Attached Figure Description

[0038] Figure 1 This is a flowchart of the method in an embodiment of the present invention;

[0039] Figure 2 This is a structural diagram of the crack detection network in an embodiment of the present invention;

[0040] Figure 3 This is a structural diagram of some modules of the multi-scale detection network for crack detection in an embodiment of the present invention;

[0041] Figure 4 This is a schematic diagram of adjacent positive and negative samples in an embodiment of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0043] like Figure 1 As shown, in this embodiment, this invention addresses the sparsity of cracks by proposing an automatic bridge crack classification and detection method based on a two-step sequence convolutional neural network, thereby accelerating crack detection efficiency. Image data containing bridges is obtained through a self-developed professional acquisition system. The two-step sequence means: First, an image classification network quickly filters out images without cracks, achieving initial data screening. Further data augmentation is then performed. Due to the highly complex background of bridge images, including linear interference such as spider webs and seams, to ensure the accuracy of crack identification and bounding box localization, the average height-width ratio of the bounding boxes of all cracks is statistically analyzed and adjusted through multiple rounds of checks to obtain a high-quality detection dataset. Second, after filtering out images containing cracks, a crack detection network classifies and locates the cracks in the selected images, finally outputting the crack location. Simultaneously, the trade-off between accuracy and efficiency in crack identification is studied, and two novel network structures are designed for the image screening and crack detection steps respectively. While ensuring accuracy, the efficiency of the entire process is significantly improved. This method specifically includes the following four parts:

[0044] I. Collect bridge crack data and construct a basic dataset

[0045] Data acquisition is performed using intelligent detection equipment. This equipment integrates sensors such as a high-speed array camera and odometer through a synchronous control system, establishing a spatiotemporal reference and synchronous control relationships between multiple sensors, enabling collaborative operation among them. The detection equipment achieves a high-definition image data resolution of 1mm, with horizontal and vertical accuracy of 0.50mm. The camera has a horizontal resolution of 4,096 pixels and a vertical resolution of 2,168 pixels. Both the horizontal and vertical pixel sizes are 3.45 micrometers. The camera lens has a focal length of 16.00 mm, and the shooting surface is 2.30 meters away from the camera. The camera's shooting range is 2.03 meters horizontally and 1.08 meters vertically.

[0046] II. Construct an attention-based image filtering network to filter image data containing cracks.

[0047] A classification dataset containing an equal number of cracked and crack-free images is constructed based on the base dataset, and the classification dataset is divided into training and test sets proportionally. An attention-based image filtering network is trained using the training set, and the network model parameters are updated using BCE With Logits Loss as the loss function. The network is then tested using the test set to obtain a well-trained attention-based image filtering network. Finally, the trained attention-based image filtering network is used to filter images containing cracks from the base dataset, thus constructing a crack detection dataset. The specific steps include:

[0048] 2.1 Based on the acquired basic dataset, a classification dataset of 20,000 images is constructed, including 10,000 images with cracks and 10,000 images without cracks. The training set and test set are divided into a 6:4 ratio, that is, 12,000 images are used for training and 8,000 images are used for testing.

[0049] 2.2 Since the aspect ratio of the input image does not meet the requirements of the network input, the image size is adjusted to 1,120×672. This compresses the image while retaining most of the information.

[0050] 2.3 Training the image filtering network with an attention mechanism, and using the maximum set method to filter images with cracks: After the network performs two convolutions of size 1, it obtains a feature map of size 5×3×1. These 15 pixels reflect whether the corresponding part of the input image has cracks, and the maximum set method is used to determine whether the entire image contains cracks.

[0051] The attention-based image filtering network is an improvement on the MobileNetV3_Small network (its specifications are shown in Table 1). The main module is B-neck, which combines the depthwise separable convolutions of MobileNetV1 and the linear bottleneck inverted residuals of MobileNetV2. The attention-based image filtering network employs the SE attention mechanism and the H-Swish function. The attention mechanism adjusts the weights of each channel, and the H-Swish function replaces the Swish function to reduce computation and improve performance. A convolutional layer of size 1 is used after the pooling layer to reduce time latency while preserving high-dimensional features. The last two convolutional layers perform two convolutions of size 1 to obtain a 5×3×1 feature map. These 15 pixels reflect whether the corresponding part of the input image has cracks. The maximum set method is used to determine whether the entire image contains cracks.

[0052] Table 1 Specifications of MobileNetV3_Small

[0053]

[0054] Compared to MobileNetV2, the image filtering network with attention mechanism moves the convolutional layer of size 1 after the pooling layer and removes the bottleneck layer and deep convolutional layer, which reduces time delay while preserving high-dimensional features. At the same time, it removes BN (batch normalization) from the last two convolutional layers.

[0055] 2.4 Adjusting model parameters using loss function: The BCEWithLogits Loss was used as the network loss function, SGD was used as the optimizer, the learning rate was set to 0.001, the momentum to 0.9, the batch size to 1, and the training was conducted for a total of 5 epochs, i.e. 60,000 iterations.

[0056] In this embodiment, the filtered crack data is used to construct a crack detection dataset containing 6,223 images. Based on different extension directions, the cracks are divided into longitudinal and circumferential cracks. Longitudinal cracks refer to cracks perpendicular to the tunnel cross-section, while circumferential cracks refer to cracks parallel to the tunnel cross-section. The dataset contains 7,542 longitudinal cracks and 4,856 circumferential cracks.

[0057] III. Constructing High-Quality Basic Data Based on Crack Detection Dataset

[0058] The boundary boxes of the marked cracks are checked multiple times based on the average height-width of the circumferential and longitudinal cracks to obtain high-quality basic data.

[0059] In this embodiment, data augmentation is employed using three data augmentation techniques: geometric transformation, splicing, and Mixup data augmentation. This reduces the dependence on large batch sizes and the inadequacy of predicting data outside of the training samples. Meanwhile, to ensure the accuracy of crack identification and bounding box localization, the average height-width ratio of the bounding boxes of all cracks is statistically analyzed and adjusted through multiple rounds of checks to obtain a high-quality detection dataset.

[0060] In this embodiment, the marked crack boundary boxes are checked multiple times based on the average height-width ratio of circumferential and longitudinal cracks to obtain high-quality basic data. Statistical results of the crack boundary boxes show that the average height-width ratio of the longitudinal crack boundary boxes is 3.83, while that of the circumferential cracks is 2.98. Longitudinal cracks are straighter than circumferential cracks.

[0061] IV. Construct a crack detection network to detect bridge cracks.

[0062] A crack detection network, YOLOv3-spp, is constructed based on the YOLOv3 architecture. A high-quality base dataset is proportionally divided into training, validation, and test sets. The crack detection network is trained using the training set, and its parameters are updated using a total loss function consisting of classification loss, confidence loss, and bounding box localization loss. The network is then validated and tested using the validation and test sets to obtain a well-trained crack detection network. This network is then used to classify and locate cracks in selected images, outputting the crack recognition results. The specific steps include the following:

[0063] 4.1 The high-quality basic dataset was divided into training, validation, and test sets in a ratio of 0.64:0.16:0.20. Specifically, 3,982 images were used to train the crack detection network, 996 images were used for hyperparameter selection, and 1,245 images were used to test generalization performance.

[0064] 4.2 Crack detection can be performed using the YOLOv3-spp crack detection network. Based on the YOLOv3 structure, improvements are made. First, the YOLOv3-spp network is trained. The initial output is obtained through the generated anchor boxes and the designed network. The NMS (non-maximum suppression) algorithm is used to filter redundant prediction boxes. For the anchor box selection part, the K-Means clustering algorithm is used to adaptively generate anchor boxes to maximize the optimal recall rate (BPR). Considering that the final result of the K-Means algorithm is related to the selection of the initial point and is prone to getting trapped in local optimization, a genetic algorithm is used to overcome the limitation of local optimization.

[0065] Anchor boxes are clustered, and the initial output is obtained through the generated anchor boxes and the designed network. A non-maximum suppression algorithm is used to filter redundant predicted boxes, yielding the final predicted boxes. For high-quality base datasets, the K-Means clustering algorithm is used to adaptively generate anchor boxes, maximizing the best recall rate (BPR). Considering that the final result of the K-Means algorithm is related to the choice of initial points and is prone to getting trapped in local optimization, a genetic algorithm is used to overcome the limitation of local optimization. Specifically, the number of anchor boxes is set to 9, the threshold to 4.0, and the number of generations to 1,000. Based on the size of the receptive field, the obtained 9 anchors are assigned to feature maps of three different scales, as shown in Table 2. The BPR is 98.7%, meaning that up to 98.7% of positive samples can theoretically be identified.

[0066] Table 2 Anchor Frame Dimensions

[0067]

[0068] like Figure 2 and Figure 3As shown, the crack detection network YOLOv3-spp uses Darknet-53 as its backbone network and employs multi-scale detection. It divides an image into blocks of multiple scales, extracts features from each block, and fuses them to achieve compatibility with features at multiple scales. This part consists of CBS, CBS2, and SPPF modules, which complete feature fusion through channel-based concatenation and 1x1 convolution. It includes three detection heads of different scales, corresponding to downsampling of 5x, 4x, and 3x respectively.

[0069] 4.3 The total loss function consists of three parts: classification loss function l cls Confidence loss function l obj and the bounding box localization loss function l box The total loss function is calculated as follows:

[0070] loss 总 =l cls +l obj +l box

[0071] The three types of partial losses mentioned above have different impacts on performance, therefore different weighting coefficients are used, and they are named λ respectively. cls , λ obj and λ box The classification loss is the sum of the calculation results of the feature maps at three scales. The feature map has dimensions S1×S2 and contains B anchor boxes for dense prediction.

[0072] λ cls The BCE With Logits Loss method is used, which combines the sigmoid and binary cross-entropy loss functions. The formula is as follows.

[0073]

[0074] This is a conditional judgment, namely, selecting the prediction box with better quality (IOU ≥ 0.5 between the predicted box and the ground truth box); p i (c) represents the probability of class c. These are the corresponding predicted values. Meanwhile, focus loss is used to further address the imbalance between positive and negative samples, with γ set to 1.5.

[0075] l obj It is also calculated using BCE With Logits Loss. The samples are divided into two classes (positive and negative), with weighting coefficients λ. obj and λ noobj The formula used to balance positive and negative samples is as follows:

[0076]

[0077] c i Output the model for the i-th sample; This represents the true value of the category corresponding to the i-th sample; This is a conditional decision, selecting predicted boxes with lower quality (IoU < 0.5 between predicted and ground truth boxes); IoU smooths out the information of positive samples. To further balance positive and negative samples, the two grid cells closest to the center point of the bounding box on the feature map are also predicted as positive samples, such as... Figure 4 As shown, similarly, γ is set to 1.5.

[0078] l box The CIoU loss function is used for calculation, and the formula is as follows:

[0079]

[0080]

[0081]

[0082]

[0083] IoU is the intersection ratio between the predicted bounding box and the actual ground view; ρ 2 (A,B) is the Euclidean distance between two boxes A and B; k is the diagonal distance of the smallest enclosed region that can simultaneously contain the predicted box and the ground truth; υ is a measure of the consistency of the relative proportions of the two rectangles; α is a weighting coefficient; w gt and h gt , where w and h are the width and height of the ground reality, respectively; w and h are the width and height of the prediction bounding box, respectively.

[0084] In this embodiment, to accelerate convergence and make the network more usable with the locations of the connection boxes, the coordinates of the boundary boxes need to be transformed, as shown in the formula:

[0085]

[0086] b x =2σ(t) x -0.5+C x

[0087] b y =2σ(t) y -0.5+C y

[0088] b w =P w (2σ(t w )) 2

[0089] bh =P h (2σ(t h )) 2

[0090] Among them, (b) x ,b y ,b w ,b h ) represents the coordinates of the bounding box; (t) represents the coordinates of the bounding box. x ,t y ,t w ,t h (C) represents the data in the network output tensor corresponding to the bounding box coordinates; x C y ) represents the coordinates of the grid center; (P) w ,P h ) represents the width and height of the anchor box.

[0091] In this embodiment,

[0092] By adopting the above-disclosed technical solution of this invention, the following beneficial effects are obtained:

[0093] This invention provides an automatic classification and detection method for bridge cracks based on a two-step sequential convolutional neural network. The image-level overall network structure constructed in this invention reduces time latency while preserving high-dimensional features, and reduces computational load while improving performance. It can accurately reflect whether there are cracks in the corresponding parts of the input image, achieving the purpose of rapid screening. The crack detection network constructed in this invention achieves compatibility with features at multiple scales in its overall structure, including feature fusion and loss function parts. In addition, the network has better adaptability to the ground truth bounding boxes of cracks.

[0094] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for automatic classification and detection of bridge cracks based on a two-step sequential convolutional neural network, characterized in that: Includes the following steps, S1. Collect bridge crack data and construct a basic dataset; S2. Construct an image filtering network based on an attention mechanism to filter image data containing cracks; A classification dataset containing the same number of cracked and crack-free images was constructed based on the basic dataset, and the classification dataset was divided into training and test sets according to the proportions. An attention-based image filtering network was trained using the training set, and the network model parameters were updated using BCE With Logits Loss as the loss function. The network was then tested using the test set to obtain the trained attention-based image filtering network. The trained attention-based image filtering network was then used to filter images containing cracks in the base dataset to construct a crack detection dataset. S3. Construct high-quality basic data based on the crack detection dataset; The boundary boxes of the marked cracks are checked multiple times based on the average height-width of the circumferential and longitudinal cracks to obtain high-quality basic data; S4. Construct a crack detection network to detect cracks in the bridge; A crack detection network, YOLOv3-spp, was constructed based on the YOLOv3 architecture. A high-quality basic dataset was proportionally divided into training, validation, and test sets. The crack detection network was trained using the training set, and the network model parameters were updated using a total loss function consisting of a classification loss function, a confidence loss function, and a bounding box localization loss function. The network was then validated and tested using the validation and test sets to obtain a well-trained crack detection network. The trained crack detection network was then used to classify and locate cracks in selected images, and the crack recognition results were output.

2. The automatic classification and detection method for bridge cracks based on a two-step sequential convolutional neural network according to claim 1, characterized in that: The main module of the attention-based image filtering network is B-neck, which combines the depthwise separable convolution of MobileNetV1 and the linear bottleneck inverted residual of MobileNetV2. The attention-based image filtering network adopts the SE attention mechanism and the H-Swish function. The attention mechanism is used to adjust the weights of each channel, and the H-Swish function is used to reduce the amount of computation and improve performance. A convolutional layer of size 1 is used after the pooling layer, and the last two convolutional layers perform two convolutions of size 1 to obtain a feature map of size 5×3×1. These 15 pixels reflect whether there are cracks in the corresponding part of the input image. The maximum set method is used to determine whether the entire image contains cracks.

3. The automatic classification and detection method for bridge cracks based on a two-step sequential convolutional neural network according to claim 1, characterized in that: In step S1, before training and testing the attention-based image filtering network using the training and test sets, the image sizes of the training and test sets are adjusted to a preset size.

4. The automatic classification and detection method for bridge cracks based on a two-step sequential convolutional neural network according to claim 1, characterized in that: Step S3 also includes performing data augmentation on all image data in the crack detection dataset, which includes three methods: geometric transformation, stitching, and Mixup data augmentation.

5. The automatic classification and detection method for bridge cracks based on a two-step sequential convolutional neural network according to claim 1, characterized in that: The crack detection network YOLOv3-spp uses Darknet-53 as its backbone network and employs multi-scale detection. It divides an image into blocks of multiple scales, extracts features from each block, and fuses them to achieve compatibility with features at multiple scales. The feature fusion component that enables multi-scale feature compatibility consists of CBS, CBS2, and SPPF modules. Feature fusion is achieved through channel-based concatenation and 1x1 convolution. It includes three detection heads of different scales, corresponding to downsampling of 5x, 4x, and 3x respectively.

6. The automatic classification and detection method for bridge cracks based on a two-step sequential convolutional neural network according to claim 1, characterized in that: During the training of the crack detection network, anchor boxes are aggregated, and preliminary output is obtained through the generated anchor boxes and the designed network. Redundant prediction boxes are filtered through a non-maximum suppression algorithm to obtain the final prediction boxes. For high-quality basic datasets, K-Means clustering algorithm is used to adaptively generate anchor boxes, and a genetic algorithm is used to overcome the limitations of local optimization.

7. The automatic classification and detection method for bridge cracks based on a two-step sequential convolutional neural network according to claim 1, characterized in that: The total loss function is calculated as follows: = ； ； ； ； ； ； ； in, , , These are the classification loss function, the confidence loss function, and the bounding box localization loss function, respectively. These are the weight coefficients of the classification loss function. , These are the weighting coefficients of the confidence loss function. Weighting coefficients for the loss function to locate the bounding box; B is the dimension of the feature map; B is the number of anchor boxes used for dense prediction. This indicates that a high-quality prediction box has been selected. for The probability of a class These are the corresponding predicted values; For the first The model output corresponding to each sample; For the first The true value of the category corresponding to each sample; To select prediction boxes of poor quality; The intersection ratio between the predicted bounding box and the actual ground view; for The Euclidean distance between the two boxes; The diagonal distance of the smallest enclosed region that can simultaneously contain both the prediction box and the ground truth; To measure the consistency of the relative proportions of two rectangles; These are the weighting coefficients; and These represent the width and height of the ground-level scene, respectively. and These represent the width and height of the prediction box, respectively.

8. The automatic classification and detection method for bridge cracks based on a two-step sequential convolutional neural network according to claim 7, characterized in that: To accelerate convergence and make the network more usable with the locations of the bounding boxes, the coordinates of the bounding boxes need to be transformed, as shown in the formula. ； ； ； ； ； in,( , , , ) represents the coordinates of the bounding box; , , , ) represents the data in the network output tensor corresponding to the bounding box coordinates; , ) represents the coordinates of the grid center; , ) represents the width and height of the anchor box.

9. The automatic classification and detection method for bridge cracks based on a two-step sequential convolutional neural network according to claim 1, characterized in that: Step S1 specifically involves using intelligent detection equipment to collect image data of bridge cracks and constructing a basic dataset. The intelligent detection equipment integrates sensors, including a high-speed array camera and an odometer, through a synchronous control system, constructing a spatiotemporal reference and synchronous control relationships between multiple sensors to achieve collaborative work among the sensors.

Images