Road crack detection method, medium, and product

Abstract

The present invention provides a road crack detection method, a medium, and a product. The method comprises: acquiring a road crack image and inputting same into a pre-trained lightweight YOLO-MCS road crack detection model to obtain a road crack detection result, wherein the lightweight YOLO-MCS road crack detection model is constructed on the basis of an improved yolov8 network. A yolov8 network improvement method comprises: replacing a feature extraction network backbone with a lightweight convolutional neural network MobileNet V3, and embedding a coordinate attention (CA) mechanism module in the lightweight convolutional neural network MobileNet V3; adding a small target detection layer and a squeeze-and-excitation (SE) module at a neck end; and introducing a power IoU loss function to a head prediction structure. The low detection accuracy of existing detection algorithms for fine cracks and the difficulty in applying existing detection algorithms to edge devices having limited computing resources are overcome.

Description

A method, medium, and product for detecting road cracks Technical Field

[0001] This invention relates to a method, medium, and product for detecting road cracks, belonging to the field of computer vision technology. Background Technology

[0002] The operational efficiency, reliability, and safety of public roads and transportation systems are crucial supports for social development. Currently, my country's highways face severe challenges such as population growth, infrastructure degradation, and rapidly rising construction and maintenance costs. Cracks are a common pavement defect; if left unattended, small cracks can develop into larger ones. For example, cracks on public roads, if not detected promptly, will widen further in rainy weather, posing a significant threat to driving safety. Therefore, timely detection and repair of cracks are of paramount importance.

[0003] With over 5.35 million kilometers of highways in China, manual inspection remains a complex process with a high rate of missed detections and poor timeliness. Automated inspection technology, however, can reduce costs while maintaining accuracy and offering greater real-time performance. Furthermore, repairing small cracks is relatively easy and inexpensive, making the ability to detect even minute cracks a crucial requirement for automated inspection methods. Simultaneously, the continuity of cracks is affected by road surface texture and environmental factors, such as lighting, shadows, and obstructions caused by rain or fog. In these conditions, the contrast between the crack and the background decreases, making identification more difficult. Therefore, automated inspection methods must also possess strong noise resistance.

[0004] Traditional image processing techniques typically use manually selected features, such as color, texture, and geometric features, to segment road surface defects. Machine learning algorithms are then used for classification and matching to detect road cracks. However, due to the complexity of the road environment, traditional image processing methods cannot meet the generalization and robustness requirements of practical engineering models through manually designed feature extraction. Compared to traditional image processing techniques, deep learning-based image processing techniques eliminate the need for manual feature extraction and offer higher accuracy, faster speed, better noise resistance, and embeddability, leading to their widespread application in road surface defect detection.

[0005] Currently, deep learning-based object detection algorithms are mainly divided into two categories. The first category is based on classification after generating candidate regions, such as R-CNN and Fast R-CNN. The second category is single-stage detection methods that directly predict the category and bounding box of all objects in the image, such as the YOLO series and SSD. Compared with other single-stage detection methods, the YOLOv8 network stands out with its advantages of high speed, simplicity, global receptive field, and multi-scale fusion, making it particularly suitable for scenarios requiring high efficiency and real-time performance. However, the YOLOv8 network still faces problems such as low detection accuracy for small objects like tiny cracks, large number of training parameters, and difficulty in convergence, which require further optimization.

[0006] The information disclosed in this background section is intended only to enhance the understanding of the overall background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0007] The technical problem to be solved by this invention is: how to overcome the problem that current detection algorithms have low detection accuracy for small cracks and are difficult to apply to edge devices with limited computing resources.

[0008] To solve the above-mentioned technical problems, the present invention is implemented using the following technical solution.

[0009] In a first aspect, the present invention provides a method for detecting road cracks, comprising:

[0010] Acquire images of road cracks to be detected;

[0011] The road crack image is input into a pre-trained lightweight YOLO-MCS road crack detection model to obtain the road crack detection results.

[0012] The lightweight YOLO-MCS road crack detection model is constructed based on an improved YOLOv8 network.

[0013] The improvements to the YOLOv8 network include:

[0014] The feature extraction network backbone of the YOLOv8 network is replaced with the lightweight convolutional neural network MobileNetV3, and a coordinate attention mechanism (CA) module is embedded in the lightweight convolutional neural network MobileNetV3.

[0015] Add a small target detection layer and a compression and excitation SE module to the neck end of the YOLOv8 network;

[0016] In the head prediction structure of the YOLOv8 network, the power IoU loss function is introduced to replace the bounding box regression loss function bbox_loss.

[0017] Furthermore, before inputting the road crack dataset into the pre-trained lightweight YOLO-MCS road crack detection model, data preprocessing is required on the road crack dataset.

[0018] Furthermore, the specific steps for preprocessing the road crack dataset include:

[0019] Filter the road crack images in the road crack dataset to obtain an labeled image dataset containing four different types of road cracks;

[0020] By reducing the application of default parameters in image enhancement methods such as scale, mosaic, mixup, and paste_in, and introducing shear and perspective image enhancement methods, we can enhance the labeled image dataset containing four different types of road cracks to obtain the image-enhanced road crack dataset.

[0021] The road crack dataset after image enhancement is normalized and standardized to obtain a road crack dataset after data scaling.

[0022] The road crack dataset, after being scaled, was randomly divided into a training set and a validation set.

[0023] Furthermore, a coordinate attention mechanism (CA) module is embedded in the lightweight convolutional neural network MobileNetV3. With the help of the coordinate attention mechanism (CA) module, the input feature map is expanded to obtain an output feature map with expanded spatial information.

[0024] Furthermore, a method for expanding the input feature map using the coordinate attention mechanism (CA) module to obtain an expanded spatial information output feature map includes:

[0025] (1) Pooling is performed in the horizontal and vertical directions of the input feature map to obtain one-dimensional perceptual attention feature maps in the X and Y directions;

[0026] (2) The perception attention feature maps in the X and Y directions are concatenated and then fed into a 1×1 convolution F for transformation to generate an intermediate feature map o containing spatial information in the horizontal and vertical directions.

[0027] (3) Decompose the intermediate feature map o into tensors o along the spatial dimension. h ∈R C / r×H×1 With tensor o w ∈RC / r×1×W Then use two 1×1 convolutions F h and F w Tensor o h With tensor o w Transform it into a tensor with the same number of channels as the input features to obtain a two-dimensional attention map;

[0028] (4) Expand the two-dimensional attention map to obtain the output features after expanding the spatial information.

[0029] Furthermore, the output features after expanding spatial information are represented as:

[0030] Among them, y c(i,j) This represents the value of the output feature map at channel c and position (i,j), x c (i,j) represents the value of the input feature map at position (i,j) in channel c. Represents the attention vector g h The attention weight at horizontal position i on channel c, Represents the attention vector g w Attention weight at vertical position j on channel c.

[0031] Furthermore, a method for adding a small target detection layer and a compression and excitation SE module to the neck of the YOLOv8 network includes:

[0032] (1) Retain the PAN-FPN structure at the neck of the YOLOv8 network, and add a compression and excitation SE module after the upsample structure at the neck, in conjunction with the added small target detection layer.

[0033] (2) Accumulate the 80×80 scale feature layer at the Backbone end of the feature extraction network and the upsampled feature layer at the Neck end to obtain a fused feature map of feature information at different scales and levels;

[0034] (3) The fused feature map of the feature information of different scales and different levels is processed by the convolution module C2f, the compression and excitation SE module and the upsampling Upsample operation to obtain a deep semantic feature layer containing small target feature information;

[0035] (4) Accumulate the deep semantic feature layer containing small target feature information and the shallow position feature layer in the feature extraction network Backbone to obtain a feature representation containing rich semantic information and precise position information;

[0036] (5) Add an extra decoupling head to the head end of the yolov8 network as a small target detection layer to detect small targets.

[0037] Furthermore, the power IoU loss function introduces the area of ​​the minimum bounding rectangle of the ground truth box and the predicted box as a parameter, and unifies the existing loss functions in YOLOv8 into a power function. The power IoU loss function includes the improved IoU loss function α-IoU, the improved GIoU loss function α-GIoU, the improved DIoU loss function α-DIoU, and the improved CIoU loss function α-CIoU.

[0038] In a second aspect, the present invention provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0039] Thirdly, the present invention provides a computer program product, including a computer program / instruction, characterized in that the computer program / instruction, when executed by a processor, implements the steps of the above-described method.

[0040] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0041] The lightweight YOLO-MCS road crack detection model proposed in this invention significantly reduces the amount of computation and network parameters required for detecting road cracks, and enhances the ability to extract features of small-target road cracks. This enables efficient and accurate detection and identification of road cracks, overcoming the problems of diverse road crack morphologies, low distinguishability from road surface texture, and the difficulty of applying current detection algorithms to edge devices with limited computing resources. Attached Figure Description

[0042] Figure 1 is a schematic diagram of the construction method of the lightweight YOLO-MCS road crack detection model provided in the embodiment of the present invention;

[0043] Figure 2 is a schematic diagram of the network architecture of the lightweight YOLO-MCS road crack detection model provided in an embodiment of the present invention.

[0044] Figure 3 is a schematic diagram of the network structure of embedding the coordinate attention mechanism (CA) module in the lightweight convolutional neural network MobileNetV3 according to an embodiment of the present invention. Detailed Implementation

[0045] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0046] Example 1

[0047] This embodiment describes a method for detecting road cracks, including the following steps:

[0048] Acquire images of road cracks to be detected;

[0049] The road crack image is input into a pre-trained lightweight YOLO-MCS road crack detection model to obtain the road crack detection results.

[0050] The lightweight YOLO-MCS road crack detection model is constructed based on an improved YOLOv8 network.

[0051] The improvements to the YOLOv8 network include:

[0052] The feature extraction network backbone of the YOLOv8 network is replaced with the lightweight convolutional neural network MobileNetV3, and a coordinate attention mechanism (CA) module is embedded in the lightweight convolutional neural network MobileNetV3.

[0053] Add a small target detection layer and a compression and excitation SE module to the neck end of the YOLOv8 network;

[0054] In the head prediction structure of the YOLOv8 network, the power IoU loss function is introduced to replace the bounding box regression loss function bbox_loss.

[0055] This invention replaces the YOLOv8 feature extraction backbone network with an improved lightweight convolutional neural network, MobileNetV3, reducing the number of parameters and computational cost while minimizing the loss of low-dimensional feature information. Furthermore, a coordinate attention (CA) mechanism containing precise location information is embedded into the original compression and excitation (SE) modules of MobileNetV3, forming an attention mechanism module that integrates precise spatial information. This helps the lightweight YOLO-MCS road crack detection model extract more semantic information from road crack images while reducing unnecessary computational complexity, thereby improving the efficiency of the detection algorithm.

[0056] The lightweight YOLO-MCS road crack detection model provided by this invention mainly addresses the problem that road crack images often have a low proportion of cracks in the image due to the long shooting distance and the small size of the cracks. Since the original YOLOv8 network cannot accurately identify feature information when the height and width of the crack are both less than 8 pixels, a small target detection layer is added to improve the detection performance. This allows the lightweight YOLO-MCS road crack detection model to focus more on detecting small road cracks. Furthermore, a compression and excitation SE module is added after the upsampling structure, which, together with the added small target detection layer, makes the lightweight YOLO-MCS road crack detection model more efficient and faster during the detection process.

[0057] In addition, this invention replaces the loss function of the original YOLOv8 network with the power IoU loss function, which can improve the training effect of bounding box regression and increase the convergence speed and regression accuracy.

[0058] Example 2

[0059] This embodiment describes a method for detecting road cracks, including:

[0060] Road crack images containing transverse cracks, longitudinal cracks, alligator cracks, and potholes are input into a pre-trained lightweight YOLO-MCS road crack detection model to obtain road crack detection results.

[0061] As shown in Figure 1, the method for constructing a lightweight YOLO-MCS road crack detection model includes:

[0062] Step 1: Obtain road crack images containing one or more types of road cracks, including transverse cracks, longitudinal cracks, alligator cracks, and potholes, to construct a road crack dataset.

[0063] Road crack datasets can be constructed by collecting images of road cracks, including transverse cracks, longitudinal cracks, alligator cracks, and potholes, using smartphone cameras or drones, or by using open-source road crack datasets.

[0064] This embodiment uses the publicly available RDD2022 road damage dataset as the road crack dataset.

[0065] The RDD2022 road damage dataset contains training and testing road damage images from six countries: Japan, India, Czech Republic, Norway, the United States, and China. It has 34,702 ground truth labels, which include bounding boxes and damage types.

[0066] In this embodiment, a computer is selected as the image recognition processing terminal, with an i9-12900H CPU processor, a 64-bit Windows 11 operating system, and an NVIDIA RTX3060Ti GPU.

[0067] Step 2: Preprocess the road crack dataset by dividing it into training and validation sets.

[0068] The specific steps for preprocessing the road crack dataset include:

[0069] Step 2.1: Filter the road crack images in the road crack dataset to obtain an labeled image dataset containing four different types of road cracks.

[0070] Due to the severe imbalance in the number of data points across different categories in the dataset selected in this embodiment, 4378 training and testing images from China were chosen from the RDD2022 road damage dataset.

[0071] For unlabeled road crack images, use the labeling tool to label the crack categories, remove label data that is not related to road cracks, and also remove labels with very few labels: longitudinal splice joint (D01), transverse splice joint (D11), labels that are not related to road cracks (D43), blurred white lines (D44), and manhole covers (D50), etc. Only four crack categories are retained: transverse cracks, longitudinal cracks, alligator cracks, and road potholes.

[0072] Finally, four different types of road cracks were selected and labeled with images: longitudinal cracks (D00), transverse cracks (D10), alligator cracks (D20), and potholes (D40).

[0073] Step 2.2: Perform image augmentation on the labeled image dataset containing four different types of road cracks to obtain the image-augmented road crack dataset.

[0074] To optimize the performance of the lightweight YOLO-MCS road crack detection model in road damage detection tasks, we first performed fine image augmentation on a dataset of labeled images containing four different types of road cracks.

[0075] In the process of image enhancement, reduce the application of default parameters of traditional image enhancement methods such as scaling, mosaic, mixup, and paste_in;

[0076] Two image enhancement methods, shear and perspective, are introduced to perform image enhancement processing on a labeled image dataset containing four different types of road cracks.

[0077] By reducing the application of default parameters in traditional image enhancement methods such as scaling, mosaicking, mixup, and paste-in, this approach avoids the potential for changes in the size and shape of road cracks, the introduction of unnatural boundaries and textures, and the disruption of continuous information on the road surface that can result from these operations. Simultaneously, it ensures that the enhanced image still accurately reflects the actual road damage, providing valuable information for the subsequent learning of the lightweight YOLO-MCS road crack detection model.

[0078] The introduction of shear and perspective image enhancement methods to perform image enhancement processing on an labeled image dataset containing four different types of road cracks aims to further improve the adaptability of the lightweight YOLO-MCS road crack detection model to complex road conditions and perspective changes. The lightweight YOLO-MCS road crack detection model can simulate image perspective changes or occlusion caused by changes in vehicle speed, road conditions, and lighting conditions, making the training data closer to the complex scenarios in actual road detection.

[0079] Step 2.3: Normalize and standardize the image-enhanced road crack dataset to obtain a scaled road crack dataset.

[0080] Normalize the image-enhanced road crack dataset:

[0081] Traverse the entire image-enhanced road crack dataset, calculate the maximum or minimum value of each pixel channel for each image in the road crack dataset, normalize each pixel value using the maximum or minimum value, and replace the original pixel value with the normalized pixel value to obtain the normalized road crack dataset.

[0082] The normalized road crack dataset is standardized by iterating through the entire image-enhanced road crack dataset, calculating the mean and standard deviation of each pixel channel of each image in the road crack dataset, and standardizing each pixel value by subtracting the mean and dividing by the standard deviation. The standardized pixel values ​​replace the original pixel values ​​to obtain the road crack dataset after data scaling.

[0083] Normalization reduces the absolute differences in pixel values, while standardization further smooths the distribution of data, helping the lightweight YOLO-MCS road crack detection model to better learn image features.

[0084] Step 2.4: Divide the road crack dataset after data scaling.

[0085] In this embodiment, the road crack dataset after data scaling is randomly divided into a training set and a validation set at a ratio of 9:1.

[0086] Step 3: Construct a lightweight YOLO-MCS road crack detection model.

[0087] The lightweight YOLO-MCS road crack detection model provided by this invention is constructed based on an improved YOLOv8 network. The improvement method for the YOLOv8 network includes:

[0088] Step 3.1: Replace the feature extraction network backbone of the yolov8 network with the lightweight convolutional neural network MobileNetV3.

[0089] The lightweight convolutional neural network MobileNetV3 uses depthwise separable convolution to replace conventional convolution operations, reducing the number of parameters and computational cost. At the same time, it uses a linear bottleneck structure with embedded inverse residual structure for feature extraction, which can reduce the loss of low-dimensional feature information. It also embeds a channel attention mechanism, which can enhance the channel feature selection capability.

[0090] Depthwise separable convolution divides conventional convolution operations into depthwise convolution and pointwise convolution. It applies a single convolution kernel to each input channel to obtain features that are not correlated between channels. Then, pointwise convolution is used to correlate the features of each channel output by depthwise convolution.

[0091] In depthwise convolution, each input channel undergoes a convolution operation separately, instead of being combined into a single input channel as in standard convolution. If the input has C channels, depthwise convolution generates C distinct feature maps, each corresponding to one input channel. Because the weights of each channel are only related to themselves, rather than to all channels, the number of parameters is reduced.

[0092] Pointwise convolution, also known as 1×1 convolution, is a special type of convolution with a kernel size of 1×1.

[0093] In pointwise convolution, the T feature maps generated by depthwise convolution are combined together, and the depth of the lightweight convolutional neural network MobileNetV3 is increased by pointwise multiplication. Pointwise convolution allows the lightweight convolutional neural network MobileNetV3 to increase the network capacity without changing the width and height of the feature maps.

[0094] Assume the dimension of the input features is D. f ×D f ×M, kernel size is D k×D k The dimension of the output feature is D f ×D f ×N, where M and N represent the number of input channels and the number of output channels, respectively, and D f D represents the spatial dimensions of the input and output feature maps, f represents the feature map height and width, and D represents the feature map height and width. k This indicates the size of the convolution kernel, where k represents the kernel height and width.

[0095] The ratio of the computational cost of depthwise separable convolution to that of standard convolution is expressed as:

[0096] Where C1 represents the computational cost of standard convolution, C2 represents the computational cost of depthwise separable convolution, α represents the width factor controlling the number of feature channels, typically 0.25, 0.5, 0.75 or 1; β represents the resolution factor controlling the feature resolution, typically 4 / 7, 5 / 7, 6 / 7 or 1.

[0097] As can be seen from the expression for the computational cost ratio between depthwise separable convolution and standard convolution, a single depthwise separable convolution can save [amount missing] compared to standard convolution. The computational cost is reduced, and by adjusting the width and resolution factors, the number of parameters and computational cost of the lightweight convolutional neural network MobileNetV3 can be further reduced.

[0098] The linear bottleneck structure first uses pointwise convolution to increase dimensionality, then uses depthwise convolution to extract features, then uses a channel attention mechanism to accurately model the relationship between each channel of the convolutional features, then uses pointwise convolution to reduce dimensionality, and finally uses a linear activation function to reduce feature loss. The inverse residual structure is also applied to the linear bottleneck structure. With almost no increase in parameters and computation, it can significantly improve the search capability of the lightweight convolutional neural network MobileNetV3.

[0099] Step 3.2: Embed the coordinate attention mechanism (CA) module in the lightweight convolutional neural network MobileNetV3, as shown in Figure 3.

[0100] The coordinate attention mechanism (CA) module is embedded in the lightweight convolutional neural network MobileNetV3. With the help of the coordinate attention mechanism (CA) module, the input feature map is expanded to obtain the output feature map after expanding spatial information.

[0101] In the lightweight convolutional neural network MobileNetV3, a coordinate attention mechanism (CA) module is embedded to fuse precise spatial information. This approach does not directly improve the spatial scale of the original compression and excitation (SE) modules of MobileNetV3, but instead proposes a novel attention mechanism to enhance the spatial awareness capability of MobileNetV3.

[0102] By using the coordinate attention mechanism (CA) module, the input feature map is expanded to obtain the output features with expanded spatial information. The specific steps include:

[0103] (1) Pooling is performed in the horizontal and vertical directions of the input feature map to obtain one-dimensional perceptual attention feature maps in the X and Y directions.

[0104] Let C, H, and W be the number of channels, height, and width of the input feature map, respectively.

[0105] The global average pooling formula of the original compression and activation SE module of the lightweight convolutional neural network MobileNetV3 is decomposed to obtain the features in the X and Y directions. The features in the X and Y directions are used to generate two one-dimensional perceptual attention feature maps, which represent the feature importance in the horizontal and vertical directions, respectively.

[0106] The global average pooling expression is as follows:

[0107] Among them, Z c x represents the output of the c-th channel. c Let i represent the two-dimensional feature set of the c-th channel, where i and j represent the x-coordinate and y-coordinate of the output feature, respectively.

[0108] The expression for the one-dimensional perceptual attention feature map in the Y direction is:

[0109] in, x represents the output of the c-th channel at height h. c It represents the two-dimensional feature set of channel c, where i represents the horizontal coordinate of the output feature and h represents the specific coordinate of the c-th channel in the height direction.

[0110] The expression for the one-dimensional perceptual attention feature map in the X direction is:

[0111] in, This represents the output of the c-th channel with width w, x c is a two-dimensional feature set representing channel c, j represents the ordinate of the output feature, and w represents the specific coordinate of the c-th channel in the width direction.

[0112] (2) The perception attention feature maps in the X and Y directions are concatenated and then fed into a 1×1 convolution F for transformation to generate an intermediate feature map o containing spatial information in the horizontal and vertical directions.

[0113] Where [] represents the spatial dimension concatenation operation, Let z denote a non-linear activation function, F denote a 1×1 convolution operation, and z denote a non-linear activation function. h This represents a one-dimensional feature map along the height direction, z. w Let o represent a one-dimensional feature map along the width direction, and o represent an intermediate feature map containing spatial information in the horizontal and vertical directions, where o∈R C / r×(H+W)×1 Where C represents the number of channels and r represents the hyperparameter of the control module size.

[0114] (3) Decompose the intermediate feature map o into tensors o along the spatial dimension. h ∈R C / r×H×1 With tensor o w ∈R C / r×1×W Then use two 1×1 convolutions F h and F w Tensor o h With tensor o w Transform it into a tensor with the same number of channels as the input features to obtain a two-dimensional attention map. h =σ(F h (o h )) (6); g w =σ(F w (o w )) (7);

[0115] Where σ is the Sigmoid function, g h Represents tensor o h Perform 1×1 convolution F h The attention vector obtained by the transformation, g h ∈R C×H×1 g w Represents tensor o w Perform 1×1 convolution F w The attention vector obtained by the transformation, g w ∈R C×1×W .

[0116] The generated two-dimensional attention map takes into account both feature recalibration between channels and incorporates precise spatial location information.

[0117] (4) Expand the two-dimensional attention map to obtain the output features after expanding the spatial information:

[0118] Among them, y c(i,j) This represents the value of the output feature map at channel c and position (i,j), x c (i,j) represents the value of the input feature map at position (i,j) in channel c. Represents the attention vector g h The attention weight at horizontal position i on channel c, Represents the attention vector g w Attention weight at vertical position j on channel c.

[0119] Through the above steps, not only is a new attention mechanism introduced into the lightweight convolutional neural network MobileNetV3, but also the accurate capture and fusion of spatial information is achieved.

[0120] Step 3.3: Add a small target detection layer and a compression and excitation SE module to the neck of the YOLOv8 network.

[0121] This embodiment accepts input images of 640×640 pixels.

[0122] However, the unmodified YOLOv8 network cannot accurately identify the feature information of targets whose height and width are both less than 8 pixels. Therefore, this invention adds a small target detection layer, that is, adds a detection feature map with a pixel size of 160×160 to the unmodified YOLOv8 network, which allows the improved YOLOv8 network to pay more attention to the detection of small targets such as road cracks, thereby improving the detection effect.

[0123] The specific steps for adding a small target detection layer and a compression and excitation SE module at the neck end of the YOLOv8 network include:

[0124] (1) Retain the PAN-FPN structure at the neck of the YOLOv8 network, and add a compression and excitation SE module after the upsample structure at the neck, in conjunction with the added small target detection layer.

[0125] (2) Accumulate the 80×80 scale feature layer at the Backbone end of the feature extraction network and the upsampled feature layer at the Neck end to obtain a fused feature map of feature information at different scales and levels;

[0126] (3) The fused feature map of the feature information of different scales and different levels is processed by the convolution module C2f, the compression and excitation SE module and the upsampling Upsample operation to obtain a deep semantic feature layer containing small target feature information;

[0127] (4) Accumulate the deep semantic feature layer containing small target feature information and the shallow position feature layer in the feature extraction network Backbone to obtain a feature representation containing rich semantic information and precise position information;

[0128] (5) Add an extra decoupling head to the head end of the yolov8 network as a small target detection layer to detect small targets.

[0129] In use, the feature representation containing rich semantics and precise location information is sent to the small target detection layer for detection.

[0130] Step 3.3: In the head prediction structure of the YOLOv8 network, the power IoU loss function is introduced to replace the bounding box regression loss function bbox_loss.

[0131] In the original YOLOv8 network's head prediction structure, several different loss functions are used to calculate the bounding box loss:

[0132] While IoU, GIoU, CIoU, and CIoU loss functions are significantly superior to using the traditional IoU loss function alone in many aspects, they still have some drawbacks and limitations.

[0133] In this embodiment, the power IoU loss function is introduced to replace the bounding box regression loss function bbox_loss in the head prediction structure of the YOLOv8 network. Here, α = 3. By introducing the parameter α, the power IoU loss function unifies the existing loss function in YOLOv8, adjusts the sensitivity of the bounding box to different degrees of overlap, and has stronger robustness.

[0134] The power IoU loss function includes the improved IoU loss function α-IoU, the improved GIoU loss function α-GIoU, the improved DIoU loss function α-DIoU, and the improved CIoU loss function α-CIoU.

[0135] The IoU loss function is calculated as follows: Loss IOU =1-IOU (9);

[0136] Among them, Loss IOU This represents the loss function of IoU, where IoU is the area of ​​intersection between the predicted box and the ground truth box divided by the area of ​​union.

[0137] The improved IoU loss function α-IoU is calculated as follows: Loss α-IOU =1-IOU α (10);

[0138] Among them, Loss α-IOU Let α-IoU represent the improved IoU loss function, where α represents the area of ​​the minimum bounding rectangle between the ground truth box and the predicted box, and IoU represents the area of ​​the intersection between the predicted box and the ground truth box divided by the area of ​​their union.

[0139] The GIoU loss function is calculated as follows:

[0140] Among them, Loss GIOU Let α represent the area of ​​the minimum bounding rectangle of the ground truth box and the predicted box, u represent the area of ​​the union of the ground truth box and the predicted box, and IoU represent the area of ​​the intersection of the predicted box and the ground truth box divided by the area of ​​the union.

[0141] The improved GIoU loss function α-GIoU is calculated as follows:

[0142] Among them, Loss α-GIOU Let α-DIoU represent the loss of the improved DIoU loss function, where α represents the area of ​​the minimum bounding rectangle of the ground truth box and the predicted box, u represents the area of ​​the union of the ground truth box and the predicted box, and IoU represents the area of ​​the intersection of the predicted box and the ground truth box divided by the area of ​​the union.

[0143] The DIoU loss function is calculated as follows:

[0144] Among them, Loss DIOU This represents the loss function DIoU, where IoU represents the area of ​​intersection between the predicted and ground truth boxes divided by the area of ​​union. b and b gt ρ represents the center coordinates of the predicted bounding box and the ground truth bounding box, respectively. 2 (b,b gt ) represents the Euclidean distance between the two center points, and l represents the diagonal length of the smallest bounding rectangle of the two frames.

[0145] The improved DIoU loss function α-DIoU is calculated as follows:

[0146] Among them, Loss α-DIOU This represents the improved DIoU loss function α-DIoU, where α represents the area of ​​the minimum bounding rectangle between the ground truth box and the predicted box, IoU represents the area of ​​the intersection of the predicted box and the ground truth box divided by the area of ​​their union, and b and b gt ρ represents the center coordinates of the predicted bounding box and the ground truth bounding box, respectively. 2 (b,b gt) represents the Euclidean distance between the two center points (the distance between two points in Euclidean space), and l represents the diagonal length of the smallest bounding rectangle of the two boxes.

[0147] The CIoU loss function is calculated as follows:

[0148] Among them, Loss CIOU This represents the loss function of the improved CIoU loss function, where IoU represents the area of ​​intersection between the predicted box and the ground truth box divided by the area of ​​union, and b and b gt ρ represents the center coordinates of the predicted bounding box and the ground truth bounding box, respectively. 2 (b,b gt ) represents the Euclidean distance between the two center points (the distance between two points in Euclidean space), l represents the diagonal length of the minimum bounding rectangle of the two boxes, β represents the weight function, and ν represents the similarity of aspect ratio.

[0149] The improved CIoU loss function α-CIoU is calculated as follows:

[0150] Among them, Loss α-CIOU This represents the improved CIoU loss function α-CIoU, where α represents the area of ​​the minimum bounding rectangle between the ground truth box and the predicted box, IoU represents the area of ​​the intersection of the predicted box and the ground truth box divided by the area of ​​their union, and b and b gt ρ represents the center coordinates of the predicted bounding box and the ground truth bounding box, respectively. 2 (b,b gt ) represents the Euclidean distance between the two center points, l represents the diagonal length of the minimum bounding rectangle of the two boxes, β represents the weight function, and ν represents the similarity of aspect ratio.

[0151] Where β represents the weight function, and the formula for calculating the weight function β is as follows:

[0152] Where ν represents the similarity of aspect ratio, and the formula for calculating the similarity of aspect ratio ν is as follows:

[0153] Among them, width gt height gt `width` and `height` represent the width and height of the ground truth bounding box and the predicted bounding box, respectively.

[0154] The network architecture of the constructed lightweight YOLO-MCS road crack detection model is shown in Figure 2.

[0155] Step 4: Train a lightweight YOLO-MCS road crack detection model using the preprocessed training set.

[0156] The training steps for the lightweight YOLO-MCS road crack detection model include:

[0157] (1) Set training hyperparameters:

[0158] The optimizer is set based on stochastic gradient descent (SGD): Momentum is set to 0.9, initial learning rate is set to 0.01, batch size is set to 32, and the number of training epochs is 200.

[0159] (2) Train the lightweight YOLO-MCS road crack detection model using the training set. In each training iteration, calculate the loss and update the weights of the lightweight YOLO-MCS road crack detection model.

[0160] (3) After each training cycle, the performance of the lightweight YOLO-MCS road crack detection model is evaluated using the preprocessed validation set. The loss and accuracy on the validation set are monitored to stop training in time to prevent overfitting.

[0161] Step 5: Use the preprocessed validation set to evaluate and optimize the trained lightweight YOLO-MCS road crack detection model.

[0162] This embodiment uses two indicators, parameter quantity and computational quantity, to evaluate the lightweight YOLO-MCS road crack detection model.

[0163] The number of parameters determines the storage space required for the lightweight YOLO-MCS road crack detection model, measured in MB.

[0164] The computational load is described using floating-point numbers (FLOPs). Given a fixed hardware configuration, the computational load determines the inference speed of the lightweight YOLO-MCS road crack detection model, measured in megabytes (M).

[0165] This embodiment uses accuracy, precision, recall, and F1 score to evaluate the performance of the lightweight YOLO-MCS road crack detection model.

[0166] The formula for calculating the accuracy is:

[0167] The expression for calculating the precision is:

[0168] The formula for calculating Recall is:

[0169] The expression for calculating the F1 score is as follows:

[0170] Where A represents the proportion of correctly predicted cracks in all predictions, P represents the proportion of samples predicted as cracks that were actually cracks, R represents the proportion of samples that were actually cracks that were correctly predicted as cracks by the model, F1 represents the harmonic mean of precision and recall, used to balance precision and recall, TP represents the number of positive samples predicted as road cracks, FP represents the number of positive samples predicted as no cracks, FN represents the number of negative samples predicted as no cracks, and TN represents the number of positive samples predicted as road cracks.

[0171] When the performance of the lightweight YOLO-MCS road crack detection model fails to meet expectations, its performance can be improved by adjusting hyperparameters such as learning rate, batch size, and number of training iterations; data augmentation methods can be added to increase data diversity; a large number of training samples can be collected to improve the generalization ability of the lightweight YOLO-MCS road crack detection model; and different weight initialization strategies can be adopted to prevent getting trapped in local optima.

[0172] Based on this, the lightweight YOLO-MCS road crack detection model is retrained and evaluated. This process is repeated until the lightweight YOLO-MCS road crack detection model achieves a satisfactory performance level.

[0173] Step 6: Input the road crack images containing transverse cracks, longitudinal cracks, alligator cracks, and potholes into the optimized lightweight YOLO-MCS road crack detection model to obtain the road crack detection results.

[0174] Example 3

[0175] A computer-readable storage medium storing computer instructions thereon, characterized in that the computer instructions, when executed by a processor, implement the steps of the method described in Embodiment 1 or 2 above.

[0176] Example 4

[0177] The present invention also provides a computer program product, including computer instructions, characterized in that the computer instructions, when executed by a processor, implement the steps of the methods described in Embodiment 1 or 2 above.

[0178] In summary, the lightweight YOLO-MCS road crack detection model proposed in this invention significantly reduces the computational and network parameter requirements for detecting road cracks, and enhances the ability to extract features of small-target road cracks. This enables efficient and accurate detection and identification of road cracks, overcoming the problems of diverse road crack morphologies, low differentiation from road surface textures, and the difficulty of applying current detection algorithms to edge devices with limited computing resources.

[0179] This invention addresses the challenges of diverse road crack morphologies, low indistinguishability from road surface textures, and the difficulty of applying current detection algorithms to resource-constrained edge devices. It proposes a lightweight YOLO-MCS road crack detection model. Based on the YOLOv8 algorithm, it employs an improved lightweight network, MobileNetV3, as the backbone to extract image features. This is combined with depthwise separable convolution, inverse residual structures, and a coordinate attention (CA) module to reduce parameter and computational costs while maintaining accuracy. A small target detection layer is added to the network framework, along with compression and excitation (SE) modules and an α-IoU loss function, further enhancing the detection accuracy for minute targets such as small cracks. This road crack detection method significantly improves the robustness of road crack detection and enhances the model's ability to extract features from small road cracks, thus enabling efficient and accurate detection and identification. Furthermore, this method has low computational and parameter requirements, making it widely applicable to most edge devices for road crack detection.

[0180] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0181] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0182] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0183] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0184] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A method of detecting a road crack, characterized by, The method comprises the following steps: acquiring a road crack image to be detected; inputting the road crack image into a pre-trained lightweight YOLO-MCS road crack detection model to obtain a road crack detection result; wherein the lightweight YOLO-MCS road crack detection model is constructed based on an improved yolov8 network; the improvement of the yolov8 network comprises: replacing the feature extraction network backbone of the yolov8 network with a lightweight convolutional neural network MobileNetV3, and embedding a coordinate attention mechanism CA module in the lightweight convolutional neural network MobileNetV3; adding a small target detection layer and a compression and excitation SE module to the neck end of the yolov8 network; in the head prediction structure of the yolov8 network, a power IoU loss function is introduced to replace the bounding box regression loss function bbox_loss.

2. The road crack detection method according to claim 1, characterized by, Before inputting the road crack dataset into the pre-trained lightweight YOLO-MCS road crack detection model, a data preprocessing operation needs to be performed on the road crack dataset.

3. The road crack detection method according to claim 2, characterized by, The specific steps of the data preprocessing operation of the road crack dataset comprise: screening road crack pictures in the road crack dataset to obtain a labeled picture dataset containing four different types of road cracks; reducing the application of default parameters of scale, mosaic, mixup and paste_in image enhancement methods, introducing shear and perspective image enhancement methods to perform image enhancement on the labeled picture dataset containing four different types of road cracks, and obtaining an image-enhanced road crack dataset; performing normalization and standardization processing on the image-enhanced road crack dataset to obtain a data-scaled road crack dataset; randomly dividing the data-scaled road crack dataset into a training set and a validation set.

4. The road crack detection method according to claim 1, characterized by, Embedding the coordinate attention mechanism CA module in the lightweight convolutional neural network MobileNetV3, and expanding the input feature map by means of the coordinate attention mechanism CA module to obtain an output feature map with expanded spatial information.

5. The road crack detection method according to claim 1, characterized by, The method for expanding the input feature map by means of the coordinate attention mechanism CA module to obtain an output feature map with expanded spatial information comprises: (1) performing pooling in the horizontal and vertical directions of the input feature map to obtain one-dimensional perception attention feature maps in the X and Y directions; (2) concatenating the perception attention feature maps in the X and Y directions and then inputting them into a 1×1 convolution F for transformation to generate an intermediate feature map o containing horizontal and vertical spatial information; (3) Decompose the intermediate feature map o into tensors o along the spatial dimension. h ∈R C / r×H×1 With tensor o w ∈R C / r×1×W Then use two 1×1 convolutions F h and F w Tensor o h With tensor o w Transform it into a tensor with the same number of channels as the input features to obtain a two-dimensional attention map. (4) expanding the two-dimensional attention map to obtain an output feature with expanded spatial information.

6. The road crack detection method according to claim 5, characterized by, The output feature after the extended spatial information is expressed as: where y c(i,j) represents the value of the output feature map at channel c, location (i,j), x c (i,j) represents the value of the input feature map at channel c, location (i,j), attention vector g h attention weight at horizontal position i on channel c, attention vector g w attention weight at vertical position j on channel c.

7. The road crack detection method according to claim 1, characterized by, The method for adding a small target detection layer and a compression and excitation SE module to the neck end of the yolov8 network comprises: (1) Retain the PAN-FPN structure at the neck end of the yolov8 network, add a compression and excitation SE module after the upsample structure at the neck end, and increase a small target detection layer. (2) Accumulate the 80x80 scale feature layer at the end of the backbone network and the upsample feature layer at the end of the neck network to obtain a fusion feature map of different scales and different levels of feature information; (3) The fusion feature map of different scales and different levels of feature information is obtained by convolution module C2f, compression and excitation SE module and upsample operation, and a deep semantic feature layer containing small target feature information is obtained; (4) Accumulate the deep semantic feature layer containing small target feature information and the shallow position feature layer in the feature extraction network Backbone to obtain a feature representation containing semantic information and position information; (5) An additional decoupling head is added as a small target detection layer in the head of the yolov8 network for detecting small targets.

8. The road crack detection method according to claim 1, characterized by, The power IoU loss function unifies the existing loss function in yolov8 by introducing the area of the minimum bounding rectangle of the real box and the predicted box as a parameter, and the power IoU loss function includes an improved IoU loss function α-IoU, an improved GIoU loss function α-GIoU, an improved DIoU loss function α-DIoU and an improved CIoU loss function α-CIoU.

9. A computer readable storage medium having stored thereon computer programs / instructions, characterized in that, The computer program / instructions are executed by the processor to implement the steps of the method of any one of claims 1-8.

10. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instructions are executed by the processor to implement the steps of the method of any one of claims 1-8.

Images