Pavement crack detection method based on YOLO v4-tiny

Abstract

The invention provides a pavement crack detection method based on YOLO v4-tiny, and the method comprises the following steps: S1, carrying out the labeling of a pavement data set, and obtaining a labeling file of the data set; s2, establishing a model training set, a verification set and a test set; s3, introducing a channel semantic extraction module, adding angle prediction information to optimize the YOLO v4-tiny model, and constructing an improved YOLO v4-tiny model; s4, the marked training set pictures in the step S2 are sent to the improved YOLO v4-tiny model constructed in the step S3 for training; and S5, using the optimal model trained in the step S4 to detect pavement cracks. The method can effectively solve the problems that the crack detection precision is low, the proportion of crack pixels in a detection frame is small, and missing detection is prone to occurring.

Description

Technical Field

[0001] This invention relates to the field of pavement defect detection, and more particularly to a pavement crack detection method based on YOLO v4-tiny. Background Technology

[0002] Road surface cracks are a common type of road defect, severely impacting the service life of highways and vehicle safety. Therefore, crack detection is of significant research importance. With the development of artificial intelligence technology in recent years, deep learning has achieved considerable success in intelligent crack detection. Compared to traditional crack detection methods, deep learning technology offers superior model generalization ability and robustness, along with higher detection accuracy, making it suitable for complex and diverse road surface conditions. Summary of the Invention

[0003] This invention aims to at least solve the technical problems existing in the prior art, and in particular, it innovatively proposes a road surface crack detection method based on YOLO v4-tiny.

[0004] To achieve the above-mentioned objectives of this invention, this invention provides a method for detecting pavement cracks based on YOLO v4-tiny, comprising the following steps:

[0005] S1, Label the road surface dataset to obtain the dataset's label file;

[0006] S2, establish the model training set, validation set, and test set;

[0007] S3 introduces a channel semantic extraction module and adds angle prediction information to optimize the YOLO v4-tiny model, thus constructing an improved YOLO v4-tiny model;

[0008] S4. The labeled training set images from step S2 are fed into the improved YOLO v4-tiny model constructed in step S3 for training.

[0009] S5. Use the optimal model trained in step S4 to detect road surface cracks.

[0010] Furthermore, the label includes the type of crack, which includes longitudinal cracks, transverse cracks, and diagonal cracks.

[0011] Furthermore, the Neck module of the improved YOLO v4-tiny network includes a channel semantic extraction module, namely the CSE module; the channel semantic extraction module includes:

[0012] For the input image, firstly, global average pooling is used to compress the feature map of each channel, and then the average of all feature values ​​in each channel is calculated to obtain the global feature representation z of the channel. c ;

[0013] Then, by using max pooling to extract the maximum value feature of each channel, the texture features of the image can be extracted more effectively.

[0014] Finally, the weights of each channel, namely channel attention weights, are obtained through a fully connected layer. These weights are used to weight the feature maps of each channel, so that important feature channels receive greater weights and unimportant feature channels receive smaller weights, thereby achieving feature enhancement and suppression.

[0015] Adding a CSE module to the YOLOv4-tiny network can help the network better distinguish different targets, improving the accuracy and efficiency of target detection.

[0016] Furthermore, the channel semantic extraction module also includes:

[0017] For the original feature map X∈C×H×W, keeping the number of channels unchanged, calculate the average and maximum values ​​for each channel to obtain two feature maps Z of size C×1×1:

[0018] Z = MaxPool(X) + AvgPool(X)

[0019]

[0020] Where, x c,i,j These are the feature values ​​in the original feature map X;

[0021] c, i, j are the coordinates of the pixels in the feature map;

[0022] C, H, and W represent the number of channels, height, and width of the feature map, respectively;

[0023] z c This represents the channel features extracted after performing max pooling and global average pooling on the feature map.

[0024] The fully connected layer is used to learn the channel weights, which enables the model to better extract semantic information between channels. The specific process is as follows:

[0025]

[0026]

[0027] Z is used as the input to two fully connected layers to learn the weight relationship between channels;

[0028] The first fully connected layer has weights W1, bias b1, and activation function ReLU. Its output is...

[0029] The weights of the second fully connected layer are W2, the bias is b2, and σ represents the weight of a convolutional kernel. The convolution operation, s c This represents the channel attention weight coefficient;

[0030] Finally, each channel in X is multiplied by its corresponding weight s. c This process is equivalent to assigning weights to each channel to weight its contribution to the final feature representation, resulting in the weighted feature map shown below:

[0031]

[0032] This represents the eigenvalues ​​in the weighted feature map;

[0033] c and j represent the channel dimension and x-coordinate in the feature map, respectively.

[0034] Furthermore, to reduce the computational cost of the convolution process in the feature pyramid network within the FPN feature extraction network (Neck module), the convolution operation is a grouped convolution. In the feature map, the length and width are compared with... Same convolution kernel Instead of performing convolution, the convolution process is as follows:

[0035]

[0036] in It is the nth feature map in the g-th group;

[0037] It is the nth convolution kernel in the g-th group.

[0038] By using grouped convolutions instead of standard convolutions in modules, network parameters and computational load can be reduced while ensuring the results, thus accelerating the training process.

[0039] Furthermore, the improved YOLO v4-tiny model uses a rotating bounding box to detect cracks and uses CircleSmooth Labels to classify the angles of the rotating bounding box, and selects a Gaussian function to classify the angles and generate Gaussian labels. Replacing angle regression with angle classification requires the detection head to output more coordinate information, such as the center coordinates, length, and width of the detection box, as well as the angle labels. Gaussian, impulse, rectangular, and trigonometric functions are all suitable for the four conditions of the CSL window function. However, the Gaussian function, as the window function for circular smooth labels, has advantages such as high smoothness, good adjustability, no edge effects, and good mathematical properties.

[0040] Furthermore, the loss function of the improved YOLO v4-tiny network includes Gaussian loss, crack category loss, confidence loss, and bounding box regression loss;

[0041] The Gaussian loss is expressed as:

[0042]

[0043] Where |X|=(T / ΔT) is the number of angle categories, where y∈{0,1} represents the CSL label of each category, and x∈[0,1] is the network's estimated probability for each category;

[0044] log(x t The cross-entropy part is x, which is only true when y = 1. t =x; otherwise, x t =1-x;

[0045] (1-y) β It is the Gaussian weighted part, applied to the negative sample location;

[0046] Modulation factor part (1-x) t ) γ Used to balance the number of angle categories for positive and negative samples;

[0047] Gaussian loss is a suitable loss function for rotating bounding box target detection, which can improve the accuracy and efficiency of network crack detection.

[0048] The crack category loss, confidence loss, and bounding box regression loss are expressed as follows:

[0049]

[0050] Among them, L obj It is the bounding box regression loss. It is the regression loss of the nth bounding box;

[0051] N represents the number of anchor points;

[0052] x,y,w,h,θ reg These represent the center coordinates, width, height, and angle of the rotating frame, respectively.

[0053] L reg (v′ nj ,v nj The value represents the confidence loss between the predicted offset of the independent variable and the true value of the target, which is the loss between the detection box and the target box output by the detection head.

[0054] v′ nj This represents the predicted offset;

[0055] v nj Indicates the true value of the predicted target;

[0056] L cls (p n ,t n ) represents the category label of the target object and the crack classification loss calculated by the sigmoid function, which is the loss between the detection box and the target box output by the detection head.

[0057] t n It is the category label of the target;

[0058] p n The probability distribution of each category is calculated using the sigmoid function;

[0059] θ n , θ′ n This represents both the realistic perspective and the predictive perspective.

[0060] L cls It is the crack classification loss.

[0061] L reg It is a confidence loss.

[0062] In summary, by adopting the above technical solutions, the present invention can effectively solve the problems of low crack detection accuracy, small proportion of crack pixels in the detection frame, and easy missed detection.

[0063] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0064] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0065] Figure 1 This is a schematic diagram of the logical structure of the present invention.

[0066] Figure 2 This is a schematic diagram of the improved YOLO v4-tiny network structure of the present invention.

[0067] Figure 3 This is a structural diagram of the Channel Attention (CSE) module introduced in this invention.

[0068] Figure 4 This is a comparison chart of the detection results for horizontal and rotated bounding boxes.

[0069] Figure 5 This is a schematic diagram of the IOU loss caused by the angle loss in this invention.

[0070] Figure 6 This is a comparison chart showing the detection results of YOLO v4-tiny and YOLO v4-tiny+CSE on a three-minute road surface video, respectively. Figure 6 (a) shows the detection results of YOLO v4-tiny; Figure 6 (b) shows the detection results of YOLOv4-tiny+CSE.

[0071] Figure 7 These are comparison images showing the performance of YOLO v4-tiny+CSE in detecting horizontal bounding boxes and rotated bounding boxes, respectively, according to this invention. Figure 7 (a) shows the horizontal bounding box detection result. Figure 7 (b) shows the effect of rotating frame detection. Detailed Implementation

[0072] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0073] To address the aforementioned issues, this invention discloses a road surface crack detection method based on YOLO v4-tiny. This method introduces a channel semantic extraction module into YOLO v4-tiny, using grouped convolution instead of standard convolution to reduce network parameters and computational load, thus improving the accuracy and efficiency of crack detection to some extent. Furthermore, angle information prediction is added to the network's detection head, using a rotated bounding box for crack detection and angle classification instead of angle regression. This effectively solves the problems of horizontal bounding boxes not accurately reflecting crack shape information and the small proportion of crack pixels within the detection box, further enhancing the accuracy of crack detection.

[0074] This invention provides a method for detecting pavement cracks based on YOLO v4-tiny, such as... Figure 1As shown, it includes the following steps:

[0075] Step 1: Label the pavement crack dataset to obtain the dataset's label file;

[0076] Step 2: Establish the model training set, validation set, and test set;

[0077] Step 3: Introduce a channel semantic extraction module and add angle prediction information to optimize the YOLO v4-tiny model, constructing an improved YOLO v4-tiny model, such as... Figure 2 As shown;

[0078] Step 4: Feed the labeled training set images from Step 2 into the improved YOLO v4-tiny model constructed in Step 3 for training;

[0079] Step 5: Use the optimal model trained in step S4 to detect road surface cracks.

[0080] in, Figure 1 This section demonstrates the overall technical framework and steps of the invention. This chapter utilizes the optimized Backbone and Neck modules in the YOLO v4-tiny network for image feature extraction, reducing redundant feature channels and giving the extracted crack features a global advantage, thereby improving the accuracy and efficiency of crack detection; in the detection head (such as...) Figure 1 Adding angle prediction information to the "Setting network prediction parameters and true labels" section and encoding the regression components of the rotating frame can more accurately reflect the crack outline and further improve the accuracy of crack classification.

[0081] Figure 2 This paper demonstrates the improved YOLO v4-tiny model proposed in this invention. Compared with the original network, the improved network introduces a channel semantic extraction module in the Neck module, uses grouped convolution instead of standard convolution, introduces a rotated bounding box and replaces angle regression of the rotated bounding box with angle classification, and proposes a loss function suitable for rotating bounding box target detection, thereby improving the accuracy and efficiency of the network in detecting cracks.

[0082] 1) Introduce a Channel Semantic Extraction (CSE) module, and use grouped convolution instead of standard convolution.

[0083] While the YOLO v4-tiny network boasts a simplified and lightweight structure, it also suffers from low detection accuracy. Cracks are multi-scale targets, and images often capture cracks that traverse the road surface. The YOLO v4-tiny network struggles to detect longitudinal and transverse cracks with large aspect ratios or large-area network cracks completely, resulting in significant missed detections. To address this issue, this paper introduces a CSE module into the Neck module, using grouped convolutions instead of standard convolutions to improve the accuracy and efficiency of crack target detection.

[0084] The role of the CSE module is to add a channel attention mechanism to the object detection network. The channel attention mechanism can enhance the network's ability to express and distinguish target features by compressing or stimulating the network to learn the feature channels of each convolutional block. It is currently widely used to improve the performance of deep convolutional neural networks.

[0085] The CSE module includes global average pooling, max pooling, and fully connected layers. For the input image, global average pooling is first used to compress the feature map of each channel. Then, the average of all feature values ​​in each channel is calculated to obtain the global feature representation z of that channel. c Then, max pooling is used to extract the maximum value feature of each channel, which can better extract the texture features of the image. Finally, the weight of each channel is obtained through a fully connected layer, namely the channel attention weight, which is used to weight the feature map of each channel, so that important feature channels receive greater weight and unimportant feature channels receive smaller weight, thereby achieving feature enhancement and suppression.

[0086] Adding a CSE module to the YOLOv4-tiny network can help the network better distinguish different targets, improving the accuracy and efficiency of target detection.

[0087] In this module, we use max pooling and global average pooling to extract channel features. Global max pooling is used to extract the maximum value feature of each channel, which can better extract the texture features of the image. Global average pooling reduces the number of parameters in the last layer weight matrix by calculating the mean of channel features, effectively suppressing model overfitting. The structure diagram of the CSE module is shown below. Figure 3 As shown.

[0088] In the channel semantic extraction module, for the original feature map X∈C×H×W, the number of channels remains unchanged, and the average and maximum values ​​of each channel are calculated to obtain two feature maps Z of size C×1×1.

[0089] Z = MaxPool(X) + AvgPool(X)

[0090]

[0091] Where Z is the feature map extracted after max pooling and global average pooling of the feature map;

[0092] x c,i,j These are the feature values ​​in the original feature map X;

[0093] c, i, j are the coordinates of the pixels in the feature map;

[0094] C, H, and W represent the number of channels, height, and width of the feature map, respectively;

[0095] z c z represents the channel features extracted after max pooling and global average pooling of the feature map. c ∈Z.

[0096] The fully connected layer plays a role in learning channel weights in the channel semantic extraction module, thereby enabling the model to better extract semantic information between channels. The specific process is as follows.

[0097]

[0098]

[0099] Z is used as input to two fully connected layers to learn the weight relationship between channels. The first fully connected layer has weights W1, bias b1, and ReLU activation function, with the output being... The weights of the second fully connected layer are W2, the bias is b2, and σ represents the weight of a convolutional kernel. The convolution operation finally yields the channel attention weight coefficients s. c Finally, each channel in X is multiplied by its corresponding weight s. c This process is equivalent to assigning weights to each channel to weight its contribution to the final feature representation, resulting in the weighted feature map shown below.

[0100]

[0101] Furthermore, to reduce the computational cost of convolution in the feature pyramid network within the FPN feature extraction network (Neck module), this paper uses grouped convolution instead of standard convolution in the module. This reduces network parameters and computational cost while ensuring optimal results, thus accelerating the training process. In this example, the number of groups, g, is set to 3, with values ​​of 1, 2, and 3. The feature of the g-th group in Z is G, where G is the number of groups; It is the nth feature map in the g-th group. Convolutional operations have local awareness capabilities, effectively capturing local feature information of an image. This allows convolutional networks to better handle the spatial locality of images, such as edge and texture information. In the feature map, the length and width are used in conjunction with... Same convolution kernel Instead of performing convolution, the convolution process is as follows.

[0102]

[0103] in It is the nth convolutional kernel in the g-th group;

[0104] It is the nth feature map in the gth group.

[0105] 2) Introduce a rotating bounding box and replace angle regression with angle classification, proposing a loss function suitable for rotating bounding box target detection.

[0106] Most object detection networks use horizontal bounding boxes to represent objects, and many current object detection tasks are based on neural networks using horizontal bounding boxes. Experiments have shown that horizontal bounding boxes have significant drawbacks. For objects with large aspect ratios, horizontal bounding boxes cannot accurately reflect the physical contours; often, the actual pixel area of ​​the object occupies less than half of the labeled area. When objects are densely packed, multiple pre-selected boxes may overlap. To find the optimal object bounding box and eliminate redundant pre-selected boxes, non-maximum suppression algorithms may mistakenly eliminate correct pre-selected boxes of neighboring objects, leading to missed detections.

[0107] Compared to horizontal frame inspection, using a rotated frame to inspect cracks has the following advantages:

[0108] (1) When cracks are dense or cover a large area, a horizontal bounding box is unlikely to detect all cracks, while a rotated bounding box can accurately locate the optimal bounding box. For example... Figure 4 As shown in (a).

[0109] (2) The aspect ratio of the horizontal border cannot clearly show the shape of the crack, while the rotated border can show the width and angle of the crack. For example... Figure 4 As shown in (b).

[0110] (3) When detecting images where cracks penetrate the road surface, the actual pixel proportion of the crack in the output detection box is relatively small, while a rotated box can distinguish the background and the crack target. For example... Figure 4 As shown in (c).

[0111] Therefore, angle prediction information is added to the detection head of the YOLO v4-tiny network, and cracks are detected using a rotating box. A prior box regression formula suitable for rotating box target detection is proposed.

[0112] This invention uses angular classification instead of angular regression to avoid the problems of discontinuous loss values ​​and inconsistent regression.

[0113] In object detection tasks, there are many methods for representing the angle of the detection box, and different angle representation methods have a significant impact on the model's learning of the angle. Among them, the five-parameter definition method (x, y, h, w, θ) is the most common for representing the target angle. (x, y) is the center of the smallest bounding rectangle of the target to be detected, h is the height of the bounding rectangle, w is the width of the bounding rectangle, and θ is the angle between the height of the bounding rectangle and the x-axis. However, the five-parameter method has the problem of angle ambiguity in representing the angle of the rotated box. That is, when two rotated boxes A and B have different angles, they can both be regarded as the detection boxes of the same target. Furthermore, the periodicity of the angle may cause the angle to jump. When the angle reaches the boundary value of 360 degrees, it may jump to 0 degrees, resulting in a sudden increase in the loss value and inconsistency in regression. Even a small deviation in the angle regression result can have a significant impact on the output of the target box. This is because the angle change is periodic, and the difference in the choice of plane will affect the direction of the angle definition.

[0114] While the eight-parameter method can solve the problem of angle ambiguity, the large number of parameters leads to a surge in computation, which significantly reduces the detection speed on platforms with weak computing power, and the problem of discontinuous loss values ​​still exists.

[0115] Therefore, this paper uses angle classification instead of angle regression to avoid the problems of discontinuous loss values ​​and inconsistent regression. The rotation box angle θ∈[-90,90) is set, and the angle is divided into 180 classes, each representing one degree. The loss caused by the inability of the pre-selected box to learn an error smaller than 1 degree is represented as the maximum angle loss and the average angle loss. The calculation process is shown in the following formula:

[0116]

[0117]

[0118] in, It's the angle, when At that time, the maximum angle loss was 0.5, and the average angle loss was 0.25. The maximum and average angle losses were used to determine whether the angle loss resulting from angle classification was acceptable.

[0119] The loss in angle also leads to a loss in the Intersection over Union (IOU) of the preselected bounding boxes. IOU can be used to calculate the degree of overlap between the preselected boxes and the ground truth boxes in object detection. As one of the metrics for evaluating object detection, IOU can be used to compare the performance of different models.

[0120] Max(loss) represents the maximum angle loss;

[0121] loss represents the loss incurred by the preselected box because it cannot learn an error smaller than 1 degree;

[0122] E(loss) represents the average angle loss;

[0123] a and b represent the integration range;

[0124] If the aspect ratio of the detected target is large, the IOU loss will be greater. Taking a rectangle S1 with an aspect ratio of 9:1 as an example, S1 is rotated around its center by a certain angle to obtain rectangle S1′. A schematic diagram for calculating the IOU loss is shown below. Figure 5 As shown, this is used to characterize the IOU loss caused by angle loss in angle prediction.

[0125] The IOU loss calculation process is shown in the following formula. First, the Intersection part, i.e., the intersecting part, is calculated:

[0126] x i1 =max(x1,x1′),y i1 =max(y1,y1′)

[0127] x i2 =min(x3,x3′),y i2 =min(y3,y3′)

[0128] intersection = max(x) i2 -x i1 +1.0,0)·max(y i2 -y i1 +1.0,0)

[0129] In the above formula, adding 1 is to eliminate the interference of overlapping pixels between the two rotated frames on the area. To avoid negative numbers, the x-coordinate is obtained by taking the maximum value of the horizontal and vertical coordinates in the above formula. i1 y i1 x i2 y i2 .

[0130] Then calculate the area of ​​the union part, i.e., the area of ​​the comparison part, where S1 and S1′ represent the areas of the two rotated frames, respectively:

[0131] S1 = (x2 - x1 + 1.0) * (y2 - y1 + 1.0)

[0132] S1′=(x2′-x1′+1.0)·(y2′-y1′+1.0)

[0133] union = S1 + S1' - intersection

[0134]

[0135] Calculated from the above formula, the IOU loss is 0.05 when the angle θ is 0.5 and 0.02 when the angle θ is 0.25. This level of loss is acceptable for the model.

[0136] Currently, there are two commonly used classification methods in the field of deep learning: One-hot encoding and Label Smoothing. Traditional One-hot encoding maps a class variable to a vector of 0s and 1s, with the length of the vector equal to the number of classes. For a given sample, its class label is marked as 1 at the corresponding position in the vector, and 0 at the remaining positions.

[0137] Label Smoothing is a technique used to reduce overfitting and improve model generalization. It adjusts the label distribution by assigning a certain probability mass between the true label and other class labels. In other words, LabelSmoothing adds random noise to each dimension of the one-hot classification result.

[0138] Transforming a continuous problem into a discrete problem requires a reasonable measurement of the angular distance between the classification prediction result and the label. If the angular distance between two preselected boxes is close, the loss value given by the model should also be close. Conversely, if the angular distance between the preselected boxes is large, the loss values ​​should also differ greatly. The one-hot classification method obviously cannot meet this requirement. Moreover, the vector dimension of one-hot classification for angular classification is very high, which increases the complexity of the model and makes training more difficult.

[0139] Using Circle Smooth Labels (CSL) for angular classification is a special LabelSmoothing method designed for periodic data. Circle Smooth Labels assume that the first and last values ​​of periodic data are continuous, and adjust the label value by adding it to the average of the preceding and following label values. CSL has circular symmetry, which more reasonably measures the distance between labels compared to One-hot labels. This paper uses CSL to divide [90, 90) into 180 classes. The specific expression of the CSL (Circular Smooth Labels) used is shown in the following formula:

[0140]

[0141] Where g(x) is the window function and r is the window radius.

[0142] θ is the rotation angle of the frame;

[0143] x is the current angle;

[0144] When using CSL (Common Squared Labels) for label representation, CSL labels for adjacent angles are also adjacent. This allows for a reasonable measurement of the distance between labels, ensuring that the loss values ​​are equal when the deviation degrees are equal. CSL also addresses the regression loss caused by angular periodicity, enabling the model to learn the target's angular information when calculating the angular loss between the preselected bounding box and the actual detection box.

[0145] Gaussian, impulse, rectangular, and trigonometric functions are all applicable to the four conditions of CSL window functions. However, the Gaussian function, as a window function for circular smooth labels, has advantages such as high smoothness, good adjustability, no edge effects, and good mathematical properties. Therefore, this paper chooses the Gaussian function to classify angles and generate angle Gaussian labels.

[0146] The detection head in the YOLO v4-tiny network is used to detect and classify features extracted by the Backbone and Neck networks to obtain the final target bounding boxes. When rotating bounding boxes are introduced into the YOLOv4-tiny network, the detection head changes. When using horizontal bounding boxes, the detection head only needs to output the target's position, size, and category information. However, with rotating bounding boxes, the detection head needs to output more coordinate information, such as the center coordinates, length, width, and angle label of the bounding box. The Head needs to add an angle prediction branch (Angle Yolo Head) to the Neck network to output the detection of rotated targets at arbitrary angles. The angle prediction branch has a total of 180 channels.

[0147] The image passes through the Backbone and Neck modules before reaching the Head detection layer. The detection layer has a channel dimension of 3×(C+5), where C is the number of categories, 3 indicates that three pre-selected bounding boxes with different aspect ratios are preset, and 5 refers to the five rotation box regression parameters. Each pre-selected bounding box predicts C categories and bounding box parameter information. The crack category prediction channel is learned through a loss function.

[0148] Finally, the loss function is defined as consisting of a Gaussian loss function, a crack category loss, a confidence loss, and a bounding box regression loss. The Gaussian loss function is calculated as follows:

[0149]

[0150] Where |X|=(T / ΔT) is the number of angle categories, where y∈{0,1} represents the CSL label of each category, and x∈[0,1] is the network's estimated probability for each category.

[0151] log(x t The cross-entropy part is x, which is only true when y = 1. t =x; otherwise, x t =1-x.

[0152] (1-y) β It is the Gaussian weighted part, applied to the negative sample positions. During training, the loss weights adaptively decrease as the distance from the target class decreases, ensuring that the network has a certain degree of fault tolerance for predictions close to the positive class.

[0153] Modulation factor part (1-x) t ) γ Used to balance the number of angle categories for positive and negative samples. It can reshape the loss function to reduce the weight of the negative category, thus focusing training on positive angle positions.

[0154] To handle adjacent and cross-boundary angle categories, the Gaussian loss function uses a dynamic weighting method on all negative angle categories, which helps the network learn a more polarized and accurate angle distribution while avoiding sudden increases in loss values ​​caused by boundary discontinuities.

[0155] The crack category loss, confidence loss, and bounding box regression loss can be expressed by the following formulas:

[0156]

[0157] Among them, L obj It is the bounding box regression loss. It is the regression loss of the nth bounding box;

[0158] N represents the number of anchor points;

[0159] x,y,w,h,θ reg These represent the center coordinates, width, height, and angle of the rotating frame, respectively.

[0160] L reg (v′ nj ,v nj The value represents the confidence loss between the predicted offset of the independent variable and the true value of the target, which is the loss between the detection box and the target box output by the detection head.

[0161] v′ nj This represents the predicted offset;

[0162] v nj Indicates the true value of the predicted target;

[0163] L cls (p n ,t n ) represents the category label of the target object and the crack classification loss calculated by the sigmoid function, which is the loss between the detection box and the target box output by the detection head.

[0164] t nIt is the category label of the target;

[0165] p n The probability distribution of each category is calculated using the sigmoid function;

[0166] θ n , θ′ n This represents both the realistic perspective and the predictive perspective.

[0167] L cls It is the crack classification loss.

[0168] L reg It is a confidence loss.

[0169] Figure 6 This image compares the detection performance of YOLO v4-tiny and YOLO v4-tiny+CSE. The selected area represents the missed detection region. This missed detection occurs because cracks have a large aspect ratio and their pixel count is close to the background pixel count. YOLO v4-tiny's relatively simplified network structure struggles to completely detect crack areas in the image. This paper introduces the CSE module into YOLO v4-tiny, enhancing the feature output of effective feature channels to improve the model's accuracy in detecting road surface cracks, thus enabling the complete detection of missed areas.

[0170] Figure 7 This is a comparison chart showing the performance of YOLO v4-tiny+CSE in detecting horizontal bounding boxes and rotating bounding boxes on a three-minute road surface video. Figure 7 The bounding box in (a) represents the missed detection area. The YOLO v4-tiny+CSE network exhibited missed detections when detecting cracks. Adding angle prediction to the YOLO v4-tiny+CSE network and using a rotated bounding box to detect cracks on the same image yielded the following results: Figure 7 As shown in (b), the network's detection performance is significantly improved. The rotating frame can completely detect crack targets with large aspect ratios and can also more accurately represent the shape information of the cracks.

[0171] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for detecting road cracks based on YOLO v4-tiny, characterized in that, Includes the following steps: S1, Label the road surface dataset to obtain the dataset's label file; S2, establish the model training set, validation set, and test set; S3 introduces a channel semantic extraction module and adds angle prediction information to optimize the YOLO v4-tiny model, thus constructing an improved YOLO v4-tiny model; S4. The labeled training set images from step S2 are fed into the improved YOLO v4-tiny model constructed in step S3 for training. S5. Use the optimal model trained in step S4 to detect road surface cracks.

2. The road crack detection method based on YOLO v4-tiny according to claim 1, characterized in that, The label includes the type of crack, which includes longitudinal cracks, transverse cracks, and diagonal cracks.

3. The road crack detection method based on YOLO v4-tiny according to claim 1, characterized in that, The Neck module of the improved YOLO v4-tiny network includes a channel semantic extraction module, namely the CSE module; the channel semantic extraction module includes: For the input image, firstly, global average pooling is used to compress the feature map of each channel, and then the average of all feature values ​​in each channel is calculated to obtain the global feature representation z of the channel. c ; Then, by using max pooling to extract the maximum value feature of each channel, the texture features of the image can be extracted more effectively. Finally, the weights of each channel, namely channel attention weights, are obtained through a fully connected layer. These weights are used to weight the feature maps of each channel, so that important feature channels receive greater weights and unimportant feature channels receive smaller weights, thereby achieving feature enhancement and suppression.

4. The road crack detection method based on YOLO v4-tiny according to claim 3, characterized in that, The channel semantic extraction module also includes: For the original feature map X∈C×H×W, keeping the number of channels unchanged, calculate the average and maximum values ​​for each channel to obtain two feature maps Z of size C×1×1: Z = MaxPool(X) + AvgPool(X) wherein x c,i,j is a feature value in the original feature map X; c, i, j are the coordinates of the pixels in the feature map; C, H, and W represent the number of channels, height, and width of the feature map, respectively; z c represents the channel features extracted after maximum pooling and global average pooling of the feature map; The fully connected layer is used to learn the channel weights, which enables the model to better extract semantic information between channels. The specific process is as follows: Z is used as the input to two fully connected layers to learn the weight relationship between channels; The weights of the first fully connected layer are W1, the bias is b1, the activation function is ReLU, and the output is The weights of the second fully connected layer are W2, and the bias is b2. σ represents a convolution operation with a kernel of ​ s c denotes a channel attention weight coefficient; Finally, each channel in X is multiplied by the corresponding weight s c This process is equivalent to assigning a weight to each channel for weighting its contribution in the final feature representation, resulting in a weighted feature map as follows: denotes a feature value in the weighted feature map; c and j represent the channel dimension and x-coordinate in the feature map, respectively.

5. The method for detecting pavement cracks based on YOLO v4-tiny according to claim 4, characterized in that, To reduce the computational cost of convolution in the feature pyramid network within the FPN feature extraction network (Neck module), the convolution operation is performed as grouped convolution. In the feature map, the dimensions of the convolutional layers are compared with... Same convolution kernel Instead of performing convolution, the convolution process is as follows: in It is the nth feature map in the g-th group; It is the nth convolution kernel in the g-th group.

6. The method for detecting pavement cracks based on YOLO v4-tiny according to claim 1, characterized in that, The improved YOLO v4-tiny model uses a rotating frame to detect cracks, uses Circle Smooth Label to classify the angles of the rotating frame, and selects a Gaussian function to classify the angles and generate Gaussian labels for the angles.

7. The method for detecting pavement cracks based on YOLO v4-tiny according to claim 1, characterized in that, The loss function of the improved YOLO v4-tiny network includes Gaussian loss, crack category loss, confidence loss, and bounding box regression loss; The Gaussian loss is expressed as: Where |X|=(T / ΔT) is the number of angle categories, where y∈{0,1} represents the CSL label of each category, and x∈[0,1] is the network's estimated probability for each category; log(x t ) is the cross-entropy part, x t = x; otherwise, x t = 1 - x; (1 - y) β is the Gaussian reweighting part applied to the negative sample locations; modulation factor part (1 - x t ) γ number of angle classes for balancing positive and negative samples; The crack category loss, confidence loss, and bounding box regression loss are expressed as follows: Among them, L obj It is the bounding box regression loss. It is the regression loss of the nth bounding box; N represents the number of anchor points; x, y, w, h, θ reg respectively represent the center coordinates, width, height and angle of the rotating frame, L reg (v′ nj ,v nj ) represents the confidence loss between the predicted bias of the independent variable and the true value of the target; v′ nj represents a predicted offset; v nj represents the predicted target true value; L cls (p n ,t n ) represents the class label of the argument target and the crack classification loss calculated by the sigmod function to obtain the probability distribution of each class. t n is the class label of the target; p n is calculated by sigmod function for probability distribution of various categories; θ n , θ′ n This represents both the realistic perspective and the predictive perspective. L cls is a crack classification loss, L reg is a confidence loss.

Images