Road crack detection method, medium and product
Through the improved lightweight YOLO-MCS model, the problems of low detection accuracy of small cracks and limited computing resources are solved, and efficient and accurate road crack detection is achieved, suitable for complex road environments and edge equipment.
Patent Information
- Application Number
- CN202410987176.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-12
AI Technical Summary
The existing road crack detection algorithm has low accuracy for fine crack detection and is difficult to apply to edge devices with limited computing resources, and traditional methods cannot meet the requirements of road environment complexity and noise resistance.
Using the improved lightweight YOLO-MCS road crack detection model, by replacing the feature extraction network of the yolov8 network with MobileNetV3, and embedding the coordinate attention mechanism CA module in it, a small object detection layer and a compression and excitation SE module were added, and a small object detection layer was added, and a compression and excitation SE module was introduced. power IoU loss function to improve detection accuracy and adaptability.
It significantly reduces the detection calculation amount and network parameters, enhances the ability to extract crack features of small-target roads, improves detection efficiency and accuracy, and is suitable for edge devices with limited computing resources.
Smart Images

Figure CN118941526A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a road crack detection method, a medium and a product, and belongs to the technical field of computer vision. Background Art
[0002] The operating efficiency, reliability and safety of public roads and transportation systems are important supports for social development. However, my country's roads are currently facing severe challenges such as population growth, infrastructure degradation, and rapid increases in construction and maintenance costs. Cracks are a common road disease. If left unchecked, small cracks will evolve into larger cracks. For example, cracks on public roads will further expand in rainy weather if they are not detected in time, posing a great threat to driving safety. Therefore, it is particularly important to detect and repair cracks in a timely manner.
[0003] China's highway mileage has exceeded 5.35 million kilometers. If manual inspection is still used, the test process is complicated and the missed detection rate is high, and the timely effect is poor. The use of automatic detection technology can reduce costs while ensuring test accuracy and is more real-time. In addition, the repair of small cracks is relatively easy and low-cost, so the ability to detect tiny cracks is one of the important requirements of automatic detection methods. At the same time, the continuity of cracks will be affected by road texture, natural environment, etc., such as light shadows, rainy days and foggy days. In this case, the contrast between the cracks and the background is reduced, and identification will be more difficult. Therefore, the automatic detection method must also have good noise resistance.
[0004] Traditional image processing technology usually uses manually selected features such as color, texture and geometric features to segment road defects, and then uses machine learning algorithms for classification and matching to achieve road crack detection. However, due to the complexity of the road environment, traditional image processing methods cannot achieve the requirements for model generalization and robustness in actual engineering through manually designed feature extraction. Compared with traditional image processing technology, image processing technology based on deep learning theory does not require manual feature extraction, and has higher accuracy, faster speed, better noise resistance and embeddability, and has been widely used in road defect detection.
[0005] At present, the target detection algorithms based on deep learning are mainly divided into two categories. The first category is based on the classification method after generating candidate regions, such as R-CNN, Fast R-CNN, etc. The second category is a single-stage detection method that directly predicts the categories and bounding boxes of all targets in the image, such as the YOLO series, SSD, etc. Compared with other single-stage detection methods, the yolov8 network stands out with its advantages of high speed, simplicity, global receptive field and multi-scale fusion, which is especially suitable for scenarios that require high efficiency and real-time performance. However, the yolov8 network still faces problems such as low detection accuracy for small objects such as fine cracks, large number of training parameters, and difficulty in convergence, which need further optimization.
[0006] The information disclosed in this background technology section is only intended to enhance the understanding of the overall background of the invention and should not be regarded as an acknowledgement or any form of suggestion that the information constitutes the prior art already known to ordinary technicians in this field. Summary of the invention
[0007] The technical problem to be solved by the present invention is: how to overcome the problem that the current detection algorithm has low detection accuracy for small cracks and is difficult to apply to edge devices with limited computing resources.
[0008] In order to solve the above technical problems, the present invention is implemented by adopting the following technical solutions.
[0009] In a first aspect, the present invention provides a road crack detection method, comprising:
[0010] Acquire a road crack image to be detected;
[0011] Input the road crack image into a pre-trained lightweight YOLO-MCS road crack detection model to obtain a road crack detection result;
[0012] Among them, the lightweight YOLO-MCS road crack detection model is constructed based on the improved yolov8 network;
[0013] Improvements to the yolov8 network include:
[0014] The feature extraction network backbone of the yolov8 network is replaced with a lightweight convolutional neural network MobileNetV3, and a coordinate attention mechanism CA module is embedded in the lightweight convolutional neural network MobileNetV3;
[0015] Add a small target detection layer and a compression and excitation SE module to the neck end of the yolov8 network;
[0016] In the head prediction structure of the yolov8 network, the power IoU loss function is introduced to replace the bounding box regression loss function bbox_loss.
[0017] Furthermore, before inputting the road crack dataset into the pre-trained lightweight YOLO-MCS road crack detection model, it is also necessary to perform data preprocessing operations on the road crack dataset.
[0018] Furthermore, the specific steps of performing data preprocessing operations on the road crack dataset include:
[0019] Filter the road crack images in the road crack dataset to obtain a dataset of labeled images containing four different types of road cracks;
[0020] Reduce the default parameters of scale, mosaic, mixup and paste_in image enhancement methods, introduce shear and perspective image enhancement methods to enhance the labeled image dataset containing four different types of road cracks, and obtain the enhanced road crack dataset;
[0021] The road crack dataset after image enhancement is normalized and standardized to obtain a road crack dataset after data scaling;
[0022] The road crack dataset after data scaling is randomly divided into training set and validation set.
[0023] Furthermore, a coordinate attention mechanism CA module is embedded in the lightweight convolutional neural network MobileNetV3. With the help of the coordinate attention mechanism CA module, the input feature map is expanded to obtain an output feature map with expanded spatial information.
[0024] Furthermore, by using the coordinate attention mechanism CA module, the input feature map is expanded to obtain the output feature map after the expanded spatial information, including:
[0025] (1) Pooling is performed in the horizontal and vertical directions of the input feature map to obtain one-dimensional perceptual attention feature maps in the X and Y directions;
[0026] (2) The perceptual attention feature maps in the X and Y directions are concatenated and fed into a 1×1 convolution Transform to generate an intermediate feature map containing horizontal and vertical spatial information ;
[0027] (3) Decompose the intermediate feature map o into tensors along the spatial dimension With tensor , and then use two 1×1 convolutions and Separate tensors With tensor Transform to a tensor with the same number of channels as the input feature to obtain a two-dimensional attention map;
[0028] (4) Expand the two-dimensional attention map to obtain the output features after expanding the spatial information.
[0029] Furthermore, the output features after expanding the spatial information are expressed as:
[0030] ;
[0031] in, Indicates that the output feature map is in the channel , location is ( , ), Indicates that the input feature map is in the channel , location is ( , ), Represents the attention vector In the channel Upper horizontal position The attention weight, Represents the attention vector In the channel Upper vertical position The attention weight.
[0032] Furthermore, a method of adding a small target detection layer and a compression and excitation SE module to the neck end of the yolov8 network includes:
[0033] (1) Keep the PAN-FPN structure at the neck end of the yolov8 network, add a compression and excitation SE module after the upsample structure at the neck end, and add a small target detection layer;
[0034] (2) Accumulate the 80×80 scale feature layer of the feature extraction network Backbone end and the upsampling feature layer of the Neck end to obtain a fusion feature map of feature information of different scales and levels;
[0035] (3) The fused feature map of feature information of different scales and different levels is subjected to a convolution module C2f, a compression and excitation SE module, and an upsampling Upsample operation to obtain a deep semantic feature layer containing small target feature information;
[0036] (4) Accumulating the deep semantic feature layer containing the small target feature information and the shallow position feature layer in the feature extraction network Backbone to obtain a feature representation containing rich semantic information and precise position information;
[0037] (5) Add an additional decoupling head in the head end of the yolov8 network as a small target detection layer to detect small targets.
[0038] Furthermore, the power IoU loss function unifies the existing loss function in yolov8 by introducing the area of the minimum circumscribed rectangle of the real box and the predicted box as a parameter. The power IoU loss function includes an improved IoU loss function , Improved GIoU loss function , Improved DIoU loss function And improved CIoU loss function .
[0039] In a second aspect, the present invention provides a computer-readable storage medium having a computer program / instruction stored thereon, which implements the steps of the above method when the computer program / instruction is executed by a processor.
[0040] In a third aspect, the present invention provides a computer program product, comprising a computer program / instruction, characterized in that the computer program / instruction implements the steps of the above method when executed by a processor.
[0041] Compared with the prior art, the present invention has the following beneficial effects:
[0042] The lightweight YOLO-MCS road crack detection model proposed in the present invention significantly reduces the amount of calculation and network parameters required for detecting road cracks, and enhances the ability to extract small target road crack features, so that road cracks can be detected and identified efficiently and accurately, overcoming the current problems of diverse road crack morphology, low distinction from road surface texture, and the difficulty of current detection algorithms to apply to edge devices with limited computing resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 It is a schematic flow chart of a method for constructing a lightweight YOLO-MCS road crack detection model provided by an embodiment of the present invention;
[0044] Figure 2 is a schematic diagram of the network architecture of the lightweight YOLO-MCS road crack detection model provided by an embodiment of the present invention;
[0045] Figure 3It is a schematic diagram of the network structure of embedding a coordinate attention mechanism CA module in a lightweight convolutional neural network MobileNetV3 provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0046] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. The embodiments of the present invention and the technical features in the embodiments may be combined with each other unless there is a conflict.
[0047] Example 1
[0048] This embodiment introduces a road crack detection method, which includes the following steps:
[0049] Acquire a road crack image to be detected;
[0050] Input the road crack image into a pre-trained lightweight YOLO-MCS road crack detection model to obtain a road crack detection result;
[0051] Among them, the lightweight YOLO-MCS road crack detection model is constructed based on the improved yolov8 network;
[0052] Improvements to the yolov8 network include:
[0053] The feature extraction network backbone of the yolov8 network is replaced with a lightweight convolutional neural network MobileNetV3, and a coordinate attention mechanism CA module is embedded in the lightweight convolutional neural network MobileNetV3;
[0054] Add a small target detection layer and a compression and excitation SE module to the neck end of the yolov8 network;
[0055] In the head prediction structure of the yolov8 network, the power IoU loss function is introduced to replace the bounding box regression loss function bbox_loss.
[0056] The present invention replaces the feature extraction backbone network of yolov8 with an improved lightweight convolutional neural network MobileNetV3, which reduces the number of parameters and calculations, and can also reduce the loss of low-dimensional feature information. In addition, a coordinate attention CA mechanism containing precise location information is embedded in the original compression and excitation SE module of MobileNetV3 to form an attention mechanism module that integrates precise spatial information, helping the lightweight YOLO-MCS road crack detection model to extract more semantic information from road crack images, while reducing unnecessary computational complexity, thereby improving the efficiency of the detection algorithm.
[0057] The lightweight YOLO-MCS road crack detection model provided by the present invention mainly solves the problem that the road crack image has a low proportion of crack targets in the image due to the long shooting distance and small cracks. Since the original yolov8 network cannot accurately identify feature information when the height and width of the target are both less than 8 pixels, a small target detection layer is added on this basis, which allows the lightweight YOLO-MCS road crack detection model to pay more attention to the detection of small target road cracks and improve the detection effect. At the same time, a compression and excitation SE module is added after the upsample structure of the upsampling, and the small target detection layer is added to make the lightweight YOLO-MCS road crack detection model more efficient and fast in the detection process.
[0058] In addition, the present invention also replaces the loss function of the original yolov8 network with the power IoU loss function, which can improve the training effect of bounding box regression and increase the convergence speed and regression accuracy.
[0059] Example 2
[0060] This embodiment introduces a road crack detection method, including:
[0061] The road crack images including transverse cracks, longitudinal cracks, alligator cracks, and potholes are input into the pre-trained lightweight YOLO-MCS road crack detection model to obtain the road crack detection results.
[0062] like Figure 1 As shown, the method for constructing a lightweight YOLO-MCS road crack detection model includes:
[0063] Step 1: Acquire road crack images including one or more road cracks including transverse cracks, longitudinal cracks, alligator cracks, and road potholes to construct a road crack dataset.
[0064] The road crack dataset can be constructed by using a smartphone camera or a drone to capture road cracks and collect road crack images including transverse cracks, longitudinal cracks, alligator cracks, and potholes, or by using an open source road crack dataset.
[0065] This embodiment uses the public RDD2022 road damage dataset as the road crack dataset.
[0066] The RDD2022 Road Damage Dataset contains training and testing road damage images from six countries / regions: Japan, India, Czech Republic, Norway, the United States, and China, with 34,702 ground truth labels including bounding boxes and damage types.
[0067] In this embodiment, a computer is selected as the image recognition processing end, the processor is configured as i9-12900H CPU, the operating system is configured as 64-bit Windows 11, and the graphics processing unit GPU is configured as NVIDIA RTX3060Ti.
[0068] Step 2: Preprocess the road crack dataset and divide it into training set and validation set.
[0069] The specific steps of data preprocessing for the road crack dataset include:
[0070] Step 2.1: Filter the road crack images in the road crack dataset to obtain a dataset of labeled images containing four different road crack types.
[0071] Since there is a serious imbalance in the number of different categories of the data sets selected in this embodiment, 4378 training and test data from China in the RDD2022 road damage dataset are selected.
[0072] The unlabeled road crack images were labeled with the crack categories using the labeling tool, and the labeled data that were not related to road cracks were removed. At the same time, the annotations with very few labels were removed: longitudinal joints (D01), transverse joints (D11), annotations that were not related to road cracks (D43), blurred white lines (D44), and manhole covers (D50). Only four crack categories, namely transverse cracks, longitudinal cracks, crocodile cracks, and potholes, were retained.
[0073] Finally, four annotated images of different road crack types were screened out: longitudinal cracks (D00), transverse cracks (D10), crocodile cracks (D20), and potholes (D40).
[0074] Step 2.2: Perform image enhancement on the labeled image dataset containing four different types of road cracks to obtain an image-enhanced road crack dataset.
[0075] In order to optimize the performance of the lightweight YOLO-MCS road crack detection model in handling road damage detection tasks, we first perform fine image enhancement on a dataset of annotated images containing four different road crack types.
[0076] In the image enhancement process, reduce the application of default parameters of traditional image enhancement methods such as scaling, mosaic, mixup and paste_in;
[0077] Two image enhancement methods, shear and perspective, are introduced to perform image enhancement on a dataset of labeled images containing four different types of road cracks.
[0078] The default parameters of traditional image enhancement methods such as scaling, mosaic, mixup and paste_in are reduced, which avoids the changes in the size and shape of road cracks, the introduction of unnatural boundaries and textures, and the destruction of continuous information on the road surface that may be caused by scaling, mosaic, mixup and paste_in operations. At the same time, it also ensures that the enhanced image can still truly reflect the actual situation of road damage, providing valuable information for the subsequent lightweight YOLO-MCS road crack detection model learning.
[0079] The introduction of shear and perspective image enhancement methods to perform image enhancement processing on a dataset of annotated images containing four different types of road cracks is to further improve the adaptability of the lightweight YOLO-MCS road crack detection model to complex road conditions and perspective changes. The lightweight YOLO-MCS road crack detection model can simulate the image perspective changes or occlusions caused by changes in vehicle speed, road conditions and light conditions, making the training data closer to the complex scenes in actual road detection.
[0080] Step 2.3: Normalize and standardize the road crack dataset after image enhancement to obtain a road crack dataset after data scaling.
[0081] Normalize the road crack dataset after image enhancement:
[0082] The entire image-enhanced road crack dataset is traversed, the maximum or minimum value of each pixel channel of each image in the road crack dataset is calculated, each pixel value is normalized by the maximum or minimum value, and the normalized pixel value replaces the original pixel value to obtain the normalized road crack dataset.
[0083] The normalized road crack dataset is standardized: the entire image-enhanced road crack dataset is traversed, the mean and standard deviation of each pixel channel of each image in the road crack dataset are calculated, each pixel value is standardized by subtracting the mean and dividing by the standard deviation, and the standardized pixel value replaces the original pixel value to obtain the road crack dataset after data scaling.
[0084] Normalization reduces the absolute difference in pixel values, and standardization further smoothes the distribution of the data, which helps the lightweight YOLO-MCS road crack detection model to better learn image features.
[0085] Step 2.4: Divide the data The road crack dataset is scaled and processed.
[0086] In this embodiment, the road crack data set after data scaling is randomly divided into a training set and a validation set in a ratio of 9:1.
[0087] Step 3: Build a lightweight YOLO-MCS road crack detection model.
[0088] The lightweight YOLO-MCS road crack detection model provided by the present invention is constructed based on the improved yolov8 network. The improvement method of the yolov8 network includes:
[0089] Step 3.1: Replace the feature extraction network backbone of the yolov8 network with the lightweight convolutional neural network MobileNetV3.
[0090] The lightweight convolutional neural network MobileNetV3 uses deep separable convolution to replace conventional convolution operations, reducing the number of parameters and computational complexity. It also uses a linear bottleneck structure embedded in an inverse residual structure for feature extraction, which can reduce the loss of low-dimensional feature information. It also embeds a channel attention mechanism to enhance channel feature selection capabilities.
[0091] Depthwise separable convolution divides the conventional convolution operation into depthwise convolution and pointwise convolution. A single convolution kernel is applied to each input channel for convolution to obtain unrelated features of each channel, and then pointwise convolution is used to associate the channel features of the depthwise convolution output.
[0092] In depthwise convolution, each input channel is convolved separately, rather than mixing all input channels together as in standard convolution. channels, then the depthwise convolution will generate Different feature maps, each corresponding to an input channel. Because the weight of each channel is only related to itself, rather than to all channels, the number of parameters is reduced.
[0093] Point-by-point convolution, also known as 1×1 convolution, is a special type of convolution whose convolution kernel size is 1×1. In point-by-point convolution, the depth convolution generates The feature maps are combined together and the depth of the lightweight convolutional neural network MobileNetV3 is increased by point-by-point multiplication. Point-by-point convolution allows the lightweight convolutional neural network MobileNetV3 to increase the capacity of the network without changing the width and height of the feature map.
[0094] Assume that the dimension of the input feature is , the convolution kernel size is , the dimension of the output feature is ,in, and Respectively represent the number of input channels and output channels, Represents the spatial dimensions of the input and output feature maps, f represents the feature map height value and feature map width value, Represents the size of the convolution kernel, and k represents the height and width of the convolution kernel.
[0095] The computational complexity ratio of depth-wise separable convolution to standard convolution is expressed as:
[0096] (1);
[0097] in, represents the computational effort of standard convolution, represents the computational effort of depthwise separable convolution, Indicates the width factor that controls the number of feature channels, usually 0.25, 0.5, 0.75 or 1; Represents the resolution factor that controls feature resolution, usually 4 / 7, 5 / 7, 6 / 7, or 1.
[0098] From the expression of the computational amount ratio of depthwise separable convolution to standard convolution, we can see that compared with standard convolution, one depthwise separable convolution can save And by adjusting the width factor and resolution factor, the number of parameters and the amount of calculation of the lightweight convolutional neural network MobileNetV3 can be further reduced.
[0099] The linear bottleneck structure first uses point-by-point convolution to increase the dimension, then uses deep convolution to extract features, and then uses the channel attention mechanism module to accurately model the relationship between the channels of the convolution features. After that, point-by-point convolution reduces the dimension, and finally uses a linear activation function to reduce feature loss. The inverse residual structure is applied to the linear bottleneck structure, which can fully enhance the search capability of the lightweight convolutional neural network MobileNetV3 with almost no increase in parameters and computational complexity.
[0100] Step 3.2: Embed the coordinate attention mechanism CA module in the lightweight convolutional neural network MobileNetV3, such as Figure 3 shown.
[0101] A coordinate attention mechanism CA module is embedded in the lightweight convolutional neural network MobileNetV3. With the help of the coordinate attention mechanism CA module, the input feature map is expanded to obtain the output feature map with expanded spatial information.
[0102] In the lightweight convolutional neural network MobileNetV3, a coordinate attention mechanism CA module is embedded to integrate precise spatial information. This method does not directly improve the original compression and excitation SE module of the lightweight convolutional neural network MobileNetV3 in terms of spatial scale, but proposes a new attention mechanism to enhance the spatial perception ability of the lightweight convolutional neural network MobileNetV3.
[0103] With the help of the coordinate attention mechanism CA module, the input feature map is expanded to obtain the output features after the expanded spatial information. The specific steps include:
[0104] (1) Pooling is performed in the horizontal and vertical directions of the input feature map to obtain one-dimensional perceptual attention feature maps in the X and Y directions.
[0105] Assumptions are the number of channels, height and width of the input feature map respectively.
[0106] Decompose the global average pooling formula of the original compression and excitation SE module of the lightweight convolutional neural network MobileNetV3 to obtain the features in the X and Y directions. Use the features in the X and Y directions to generate two one-dimensional perceptual attention feature maps, which represent the importance of features in the horizontal and vertical directions respectively.
[0107] The global average pooling expression is expressed as:
[0108] (2);
[0109] in, Indicates The output of the channel, Indicates The two-dimensional feature set of the channel, and Represent the horizontal and vertical coordinates of the output features respectively.
[0110] The one-dimensional perceptual attention feature map expression in the Y direction is:
[0111] (3);
[0112] in, The height h represents the The output of the channel, Yes means The two-dimensional feature set of the channel, represents the horizontal coordinate on the output feature, Indicates The specific coordinates of the channel in the height direction.
[0113] The one-dimensional perceptual attention feature map expression in the X direction is:
[0114] (4);
[0115] in, Indicates the width is No. The output of the channel, Yes means The two-dimensional feature set of the channel, represents the ordinate on the output feature, Indicates The specific coordinates of the channel in the width direction.
[0116] (2) The perceptual attention feature maps in the X and Y directions are concatenated and fed into a 1×1 convolution Transform to generate an intermediate feature map containing horizontal and vertical spatial information .
[0117] o =∂ ( F ([ z h , z w ])) (5);
[0118] Among them, [] represents the spatial dimension splicing operation, represents a nonlinear activation function, represents a 1×1 convolution operation, represents a one-dimensional feature map in the height direction, represents a one-dimensional feature map in the width direction, o represents an intermediate feature map containing spatial information in the horizontal and vertical directions, ; Where C represents the number of channels and r represents the hyperparameter controlling the module size.
[0119] (3) Decompose the intermediate feature map o into tensors along the spatial dimension With tensor , and then use two 1×1 convolutions and Separate tensors With tensor Transform to a tensor with the same number of channels as the input feature to obtain a two-dimensional attention map.
[0120] (6);
[0121] (7);
[0122] in, is the Sigmoid function, Represents a tensor Perform 1×1 convolution The transformed attention vector is , Represents a tensor Perform 1×1 convolution The transformed attention vector is .
[0123] The generated two-dimensional attention map not only takes into account the feature recalibration between channels, but also incorporates precise spatial location information.
[0124] (4) Expand the two-dimensional attention map to obtain the output features after expanding the spatial information:
[0125] (8);
[0126] in, Indicates that the output feature map is in the channel , location is ( , ), Indicates that the input feature map is in the channel , location is ( , ), Represents the attention vector In the channel Upper horizontal position The attention weight, Represents the attention vector In the channel Upper vertical position The attention weight.
[0127] Through the above steps, not only a new attention mechanism is introduced into the lightweight convolutional neural network MobileNetV3, but also the accurate capture and fusion of spatial information is achieved.
[0128] Step 3.3: Add a small target detection layer and a compression and excitation SE module to the neck end of the yolov8 network.
[0129] This embodiment accepts an input image of 640×640 pixels.
[0130] However, the unmodified yolov8 network cannot accurately identify the feature information of targets with a height and width less than 8 pixels. Therefore, by adding a small target detection layer, that is, adding a detection feature map with a pixel size of 160×160 in the unmodified yolov8 network, the improved yolov8 network can pay more attention to the detection of small target road cracks and improve the detection effect.
[0131] The specific steps of adding a small target detection layer and a compression and excitation SE module to the neck end of the yolov8 network include:
[0132] (1) Keep the PAN-FPN structure at the neck end of the yolov8 network, add a compression and excitation SE module after the upsample structure at the neck end, and add a small target detection layer.
[0133] (2) Accumulate the 80×80 scale feature layer of the feature extraction network Backbone end and the upsampling feature layer of the Neck end to obtain a fusion feature map of feature information of different scales and levels;
[0134] (3) The fused feature map of feature information of different scales and different levels is subjected to a convolution module C2f, a compression and excitation SE module, and an upsampling Upsample operation to obtain a deep semantic feature layer containing small target feature information;
[0135] (4) Accumulating the deep semantic feature layer containing the small target feature information and the shallow position feature layer in the feature extraction network Backbone to obtain a feature representation containing rich semantic information and precise position information;
[0136] (5) Add an additional decoupling head in the head end of the yolov8 network as a small target detection layer to detect small targets.
[0137] When in use, the feature representation containing rich semantics and precise position information is sent to the small target detection layer for detection.
[0138] Step 3.3: In the head prediction structure of the yolov8 network, the power IoU loss function is introduced to replace the bounding box regression loss function bbox_loss.
[0139] In the head prediction structure of the original yolov8 network, several different loss functions are used to calculate the bounding box loss: IoU loss function, GIoU loss function, CIoU loss function and CIoU loss function. Although they are obviously better than the traditional IoU loss function in many aspects, they still have some shortcomings and limitations.
[0140] In this embodiment, the power IoU loss function is introduced to replace the bounding box regression loss function bbox_loss in the head prediction structure of the yolov8 network, where: = 3, the power IoU loss function is introduced by parameter , the existing loss functions in yolov8 are unified and exponentiated, and the sensitivity of the bounding box to different degrees of overlap is adjusted, making it more robust.
[0141] The power IoU loss function includes the improved IoU Loss Function , improved GIoU Loss Function , improved DIoU Loss Function and improved CIo Loss Function .
[0142] The calculation method of the IoU loss function is expressed as:
[0143] (9);
[0144] in, represents the loss of the IoU loss function, IoU It represents the area of intersection between the predicted box and the true box divided by the area of union.
[0145] Improved IoU loss function The calculation method is expressed as:
[0146] (10);
[0147] in, Represents the improved IoU loss function of loss, Represents the area of the minimum enclosing rectangle of the real box and the predicted box, IoU It represents the area of intersection between the predicted box and the true box divided by the area of union.
[0148] The calculation method of GIoU loss function is expressed as:
[0149] (11);
[0150] in, represents the loss of the GIoU loss function, Represents the area of the minimum enclosing rectangle of the real box and the predicted box, u Represents the union area of the real box and the predicted box, IoU It represents the area of intersection between the predicted box and the true box divided by the area of union.
[0151] Improved GIoU loss function The calculation method is expressed as:
[0152] (12);
[0153] in, Represents the improved DIoU loss function of loss, Represents the area of the minimum enclosing rectangle of the real box and the predicted box, u Represents the union area of the real box and the predicted box, IoU It represents the area of intersection between the predicted box and the true box divided by the area of union.
[0154] The calculation method of DIoU loss function is expressed as:
[0155] (13);
[0156] in, represents the loss of DIoU loss function, IoU It represents the area of intersection between the predicted box and the real box divided by the area of union. b and Represent the center point coordinates of the predicted box and the real box respectively, represents the Euclidean distance between two center points, l Indicates the diagonal length of the minimum circumscribed rectangle of the two boxes.
[0157] Improved DIoU loss function The calculation method is expressed as:
[0158] (14);
[0159] in, Represents the improved DIoU loss function of loss, Represents the area of the minimum enclosing rectangle of the real box and the predicted box, IoU It represents the area of intersection between the predicted box and the real box divided by the area of union. b and Represent the center point coordinates of the predicted box and the real box respectively, Represents the Euclidean distance between two center points (the distance between two points in Euclidean space). l Indicates the diagonal length of the minimum circumscribed rectangle of the two boxes.
[0160] The calculation method of CIoU loss function is expressed as:
[0161] (15);
[0162] in, represents the loss of the improved CIoU loss function, IoU It represents the area of intersection between the predicted box and the real box divided by the area of union. b and Represent the center point coordinates of the predicted box and the real box respectively, Represents the Euclidean distance between two center points (the distance between two points in Euclidean space). l Indicates the diagonal length of the minimum circumscribed rectangle of the two boxes, represents the weight function, Indicates similarity in aspect ratio.
[0163] Improved CIoU loss function The calculation method is expressed as:
[0164] (16);
[0165] in, Represents the improved CIoU loss function of loss, Represents the area of the minimum enclosing rectangle of the real box and the predicted box, IoU It represents the area of intersection between the predicted box and the real box divided by the area of union. b and Represent the center point coordinates of the predicted box and the real box respectively, represents the Euclidean distance between two center points, l Indicates the diagonal length of the minimum circumscribed rectangle of the two boxes, represents the weight function, Indicates similarity in aspect ratio.
[0166] in, represents the weight function, weight function The calculation formula is expressed as:
[0167] (17);
[0168] in, Indicates the similarity of aspect ratios, similarity of aspect ratios The calculation formula is expressed as:
[0169] (18);
[0170] in, , and , Represent the width and height of the real box and the predicted box respectively.
[0171] The network architecture of the constructed lightweight YOLO-MCS road crack detection model is shown in Figure 2.
[0172] Step 4: Use the preprocessed training set to train a lightweight YOLO-MCS road crack detection model.
[0173] The training steps of the lightweight YOLO-MCS road crack detection model include:
[0174] (1) Set training hyperparameters:
[0175] The optimizer is set based on stochastic gradient descent (SGD): the momentum is set to 0.9, the initial learning rate is set to 0.01, the batch size is set to 32, and the number of training rounds is 200 epochs.
[0176] (2) Use the training set to train a lightweight YOLO-MCS road crack detection model. In each training iteration, calculate the loss and update the weights of the lightweight YOLO-MCS road crack detection model.
[0177] (3) After each training cycle, the preprocessed validation set is used to evaluate the performance of the lightweight YOLO-MCS road crack detection model. Indicators such as loss and accuracy on the validation set are monitored so that training can be stopped in time to prevent overfitting.
[0178] Step 5: Use the preprocessed validation set to evaluate and optimize the trained lightweight YOLO-MCS road crack detection model.
[0179] This embodiment uses two indicators, parameter quantity and computational complexity, to evaluate the lightweight YOLO-MCS road crack detection model.
[0180] The number of parameters determines the storage space required by the lightweight YOLO-MCS road crack detection model, and the unit is M.
[0181] The amount of computation is described by floating-point operations FLOPs. When the hardware device is determined, the amount of computation determines the inference speed of the lightweight YOLO-MCS road crack detection model, and the unit is M.
[0182] This embodiment uses accuracy, precision, recall, and F1-score to evaluate the performance of the lightweight YOLO-MCS road crack detection model.
[0183] The calculation expression of the accuracy is:
[0184] (19);
[0185] The calculation expression of the precision Precision is:
[0186] (20);
[0187] The calculation expression of the recall rate Recall is:
[0188] (twenty one);
[0189] The calculation expression of the F1 score F1-score is:
[0190] (twenty two);
[0191] in, A represents the proportion of correctly predicted cracks among all predictions, P It represents the proportion of samples that are actually cracks among all samples predicted to be cracks. R It represents the proportion of samples that are actually cracks that are correctly predicted by the model to be cracks. F1 It represents the harmonic mean of precision and recall, and is used to weigh the precision and recall. TP represents the number of positive samples predicted to be road cracks, FP represents the number of positive samples predicted to be crack-free, FN represents the number of negative samples without cracks, TN Represents the number of positive samples predicted to be road cracks.
[0192] When the performance of the lightweight YOLO-MCS road crack detection model does not meet expectations, the performance of the lightweight YOLO-MCS road crack detection model can be improved by adjusting hyperparameters such as learning rate, batch size, and number of training times; some data enhancement methods are added to improve data diversity; the generalization ability of the lightweight YOLO-MCS road crack detection model is improved by collecting a large number of training samples; in order to prevent falling into local optimality, different weight initialization strategies can also be used.
[0193] On this basis, the lightweight YOLO-MCS road crack detection model is retrained and evaluated. The above process is repeated until the lightweight YOLO-MCS road crack detection model reaches a satisfactory performance level.
[0194] Step 6: Input the road crack images including transverse cracks, longitudinal cracks, alligator cracks, and potholes into the optimized lightweight YOLO-MCS road crack detection model to obtain the road crack detection results.
[0195] Example 3
[0196] A computer-readable storage medium having computer instructions stored thereon, characterized in that when the computer instructions are executed by a processor, the steps of the method of embodiment 1 or 2 above are implemented.
[0197] Example 4
[0198] The present invention also provides a computer program product, including computer instructions, characterized in that when the computer instructions are executed by a processor, the steps of the method in the above-mentioned embodiment 1 or 2 are implemented.
[0199] In summary, the lightweight YOLO-MCS road crack detection model proposed in the present invention significantly reduces the amount of calculation and network parameters required for detecting road cracks, and enhances the ability to extract small target road crack features, so that road cracks can be detected and identified efficiently and accurately, overcoming the current problems of diverse road crack morphology, low distinction from road surface texture, and the difficulty of current detection algorithms to apply to edge devices with limited computing resources.
[0200] In view of the problems that road cracks have diverse morphologies, low differentiation from road surface textures, and current detection algorithms are difficult to apply to edge devices with limited computing resources, this paper proposes a lightweight YOLO-MCS road crack detection model. Based on the yolov8 algorithm, an improved lightweight network MobileNetV3 is used as the backbone network to extract image features, combined with deep separable convolution, reverse residual structure and coordinate attention mechanism CA module, while ensuring accuracy, the number of parameters and calculations are reduced. A small target detection layer is added to the network framework, and the compression and excitation SE module and The loss function further improves the detection accuracy of small targets such as fine cracks. The road crack detection method provided by the present invention can significantly improve the detection robustness of road cracks and enhance the model's ability to extract small target road crack features, thereby efficiently and accurately detecting and identifying road cracks. At the same time, the method has a small amount of calculation and parameters, has wide applicability, and can be applied to most edge devices for road crack detection.
[0201] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0202] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0203] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0204] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0205] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.
Claims
1. A road crack detection method, characterized in that: include: Acquire a road crack image to be detected; Input the road crack image into a pre-trained lightweight YOLO-MCS road crack detection model to obtain a road crack detection result; Among them, the lightweight YOLO-MCS road crack detection model is constructed based on the improved yolov8 network; Improvements to the yolov8 network include: The feature extraction network backbone of the yolov8 network is replaced with a lightweight convolutional neural network MobileNetV3, and a coordinate attention mechanism CA module is embedded in the lightweight convolutional neural network MobileNetV3; Add a small target detection layer and a compression and excitation SE module to the neck end of the yolov8 network; In the head prediction structure of the yolov8 network, the power IoU loss function is introduced to replace the bounding box regression loss function bbox_loss.
2. The road crack detection method according to claim 1, characterized in that: Before inputting the road crack dataset into the pre-trained lightweight YOLO-MCS road crack detection model, it is also necessary to perform data preprocessing operations on the road crack dataset.
3. The road crack detection method according to claim 2, characterized in that: The specific steps of performing data preprocessing operations on the road crack dataset include: Filter the road crack images in the road crack dataset to obtain a dataset of labeled images containing four different types of road cracks; Reduce the default parameters of scale, mosaic, mixup and paste_in image enhancement methods, introduce shear and perspective image enhancement methods to enhance the labeled image dataset containing four different types of road cracks, and obtain the enhanced road crack dataset; The road crack dataset after image enhancement is normalized and standardized to obtain a road crack dataset after data scaling; The road crack dataset after data scaling is randomly divided into training set and validation set.
4. The road crack detection method according to claim 1, characterized in that: A coordinate attention mechanism CA module is embedded in the lightweight convolutional neural network MobileNetV3. With the help of the coordinate attention mechanism CA module, the input feature map is expanded to obtain the output feature map with expanded spatial information.
5. The road crack detection method according to claim 1, characterized in that: The method of expanding the input feature map by using the coordinate attention mechanism CA module to obtain the output feature map after the expanded spatial information includes: (1) Pooling is performed in the horizontal and vertical directions of the input feature map to obtain one-dimensional perceptual attention feature maps in the X and Y directions; (2) The perceptual attention feature maps in the X and Y directions are concatenated and fed into a 1×1 convolution Transform to generate an intermediate feature map containing horizontal and vertical spatial information ; (3) Decompose the intermediate feature map o into tensors along the spatial dimension With tensor , and then use two 1×1 convolutions and Separate tensors With tensor Transform to a tensor with the same number of channels as the input feature to obtain a two-dimensional attention map; (4) Expand the two-dimensional attention map to obtain the output features after expanding the spatial information.
6. The road crack detection method according to claim 5, characterized in that: The output features after expanding the spatial information are expressed as: ; in, Indicates that the output feature map is in the channel , location is ( , ), Indicates that the input feature map is in the channel , location is ( , ), Represents the attention vector In the channel Upper horizontal position The attention weight, Represents the attention vector In the channel Upper vertical position The attention weight.
7. The road crack detection method according to claim 1, characterized in that: The method of adding a small target detection layer and a compression and excitation SE module to the neck end of the yolov8 network includes: (1) Keep the PAN-FPN structure at the neck end of the yolov8 network, add a compression and excitation SE module after the upsample structure at the neck end, and add a small target detection layer; (2) Accumulate the 80×80 scale feature layer of the feature extraction network Backbone end and the upsampling feature layer of the Neck end to obtain a fusion feature map of feature information of different scales and levels; (3) The fused feature map of feature information of different scales and different levels is subjected to a convolution module C2f, a compression and excitation SE module, and an upsampling Upsample operation to obtain a deep semantic feature layer containing small target feature information; (4) Accumulating the deep semantic feature layer containing the small target feature information and the shallow position feature layer in the feature extraction network Backbone to obtain a feature representation containing semantic information and position information; (5) Add an additional decoupling head in the head end of the yolov8 network as a small target detection layer to detect small targets.
8. The road crack detection method according to claim 1, characterized in that: The power IoU loss function unifies the existing loss function in yolov8 by introducing the area of the minimum circumscribed rectangle of the real box and the predicted box as a parameter. The power IoU loss function includes an improved IoU loss function. , Improved GIoU loss function , Improved DIoU loss function And improved CIoU loss function .
9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.
10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.
Citation Information
Cited By
Bridge crack segmentation method based on YOLOv8n-seg
CN119273925A
Illumination adaptive road surface detection method and system considering small target area recognition
CN119559604A
Method for automatically detecting inconsistency of surface topological relation
CN119579871A
An automatic detection method for inconsistency in surface topology relationships
CN119579871B
Model and method for detecting pavement defects
CN119649017A
