Road damage detection method based on EDG-YOLO

Abstract

The invention belongs to the technical field of road damage detection, and provides a road damage detection method based on EDG-YOLO, which comprises the following steps: acquiring a road damage image data set, and preprocessing to obtain a preprocessed road damage image data set; the method comprises the following steps: constructing an EDG-YOLO detection model by fusing EIEStem, Dysample and GNSCDH on the basis of a YOLOv8 network model, and training the EDG-YOLO detection model to obtain a trained EDG-YOLO detection model; and performing road damage detection on the preprocessed road damage image data set by using the trained EDG-YOLO detection model to obtain a plurality of road damage types. According to the method, richer image feature representation can be learned, and the model detection performance is improved; road damage features can be better restored, and the perception ability of the model to details is improved; the number of parameters can be effectively reduced, and meanwhile the detection head has higher multi-scale sensing capacity.

Description

Technical Field

[0001] The present invention relates to the technical field of road damage detection, and in particular to a road damage detection method based on EDG-YOLO. Background Art

[0002] Highways are one of the most important civil facilities. Under the influence of natural environment and vehicle load, paved roads will be damaged to varying degrees. Common road surface damage includes cracks, fissures, potholes, etc. If road damage is not repaired in time, it will reduce the service life of the road and cause damage to passing vehicles. Therefore, it is of great significance to discover and repair road damage in time.

[0003] The detection and identification of road damage is an important technical means to realize the automation and intelligence of road maintenance. It can provide accurate and comprehensive information support for highway maintenance decision-making and the implementation of maintenance projects, and is of great significance for the timely and efficient repair of road damage. Automatic road inspection uses advanced technologies and methods to detect and monitor road damage. By quickly and accurately detecting potential hazards, automatic road inspection can enhance road safety and reduce the cost and time associated with manual inspection. With the rapid development of artificial intelligence algorithms, road damage detection based on deep learning models has become a research focus. YOLOv8 is a major update to the YOLO target detection algorithm family, which can quickly and effectively locate and identify targets in images. It includes five different versions: YOLOv8n, YOLOv8s, YOLOv8m, YOLOv8l and YOLOv8x. YOLOv8n not only supports target detection tasks, but also supports image classification and instance segmentation tasks. YOLOv8n consists of 4 parts, including the input end (Input), the backbone network (Backbone), the Neck module and the Head output end, which not only supports target detection tasks, but also supports image classification and instance segmentation tasks. YOLOv8 loads images through the input module. At this stage, a variety of methods are usually used to enhance the data of the input images. The main methods of data enhancement are HSV (Hue SaturationValue) transformation, Mosaic data enhancement, Copy paste data enhancement, Random Affine affine transformation, MixUp data enhancement and other methods; YOLOv8 uses CSPDarknet53 as its backbone network, whose main function is to aggregate image information according to different types of images to propose image features and finally output feature maps containing target location, category and other information; the backbone network is mainly composed of CBS, C2f and SPPF modules, among which the CBS module is mainly responsible for extracting image feature information through convolution, which includes two-dimensional convolution (conv2d), batch normalization (BN) and SiLU activation function. The C2f module is a simplified CSP residual block, which divides the input feature map into multiple branches. After a branch performs a convolution operation, it is output through multiple Bottleneck blocks, and the other branches are directly spliced ​​without any operation, which not only reduces the calculation difficulty of the model, but also realizes a richer fusion of features. The SPPF module extracts and fuses high-level features by performing multiple maximum pooling operations on the feature map, thereby obtaining feature information at different scales.

[0004] Currently, in the process of using deep learning models for road damage detection, the detection algorithm is prone to false detection and missed detection due to the interference of complex scenes such as pedestrians, vehicles, uneven lighting, and tree branches in the actual road environment. The shapes and sizes of road diseases vary greatly, and it is difficult for the algorithm to effectively extract the characteristics of the disease. At the same time, when improving the detection accuracy of the model, its computational cost cannot be taken into account. The detection model is large in size and difficult to deploy on edge devices such as embedded devices.

[0005] Therefore, it is necessary to provide a road damage detection method based on EDG-YOLO. Summary of the invention

[0006] The present invention provides a road damage detection method based on EDG-YOLO. By constructing the ElEStem front-end module, richer image feature representations can be learned to improve the performance of model detection. The introduction of dynamic upsampling DySample can better restore the characteristics of road damage and improve the model's ability to perceive details. Replacing the GNCSDH detection head can effectively reduce the number of parameters and enable the detection head to have stronger multi-scale perception capabilities so that it can be deployed on resource-constrained devices.

[0007] The present invention provides a road damage detection method based on EDG-YOLO, comprising:

[0008] Acquire a road damage image dataset, and perform preprocessing to obtain a preprocessed road damage image dataset;

[0009] Based on the YOLOv8 network model, EIEStem, Dysample and GNSCDH are integrated to build the EDG-YOLO detection model, and the EDG-YOLO detection model is trained to obtain the trained EDG-YOLO detection model;

[0010] The trained EDG-YOLO detection model is used to perform road damage detection on the preprocessed road damage image dataset to obtain multiple road damage types.

[0011] Further, a road damage image dataset is obtained and preprocessed to obtain a preprocessed road damage image dataset, including:

[0012] Based on the RDD_China part of the public RDD2022 dataset, a road damage image dataset is obtained, and the road damage image dataset is divided into a training sub-dataset, a validation sub-dataset, and a test sub-dataset according to a division ratio of 8:1:1;

[0013] The training sub-dataset, the validation sub-dataset and the test sub-dataset are standardized or normalized to obtain a preprocessed road damage image dataset.

[0014] Furthermore, based on the YOLOv8 network model, EIEStem, Dysample and GNSCDH are integrated to build the EDG-YOLO detection model, including:

[0015] Get the original YOLOv8 network model, replace the first layer of the YOLOv8 network model with the EIEStem module, introduce dynamic upsampling DySample in the neck network of the YOLOv8 network model, replace the detection head of the YOLOv8 network model with GNSCDH, and complete the construction of the EDG-YOLO detection model.

[0016] Furthermore, the EDG-YOLO detection model is trained, including:

[0017] Configure the training hyperparameters, set the batch size to 8 and the number of iterations to 300 rounds; train the EDG-YOLO detection model, use the training sub-dataset, calculate the loss and update the EDG-YOLO detection model weights in each training iteration, and use the validation sub-dataset to evaluate the performance of the EDG-YOLO detection model after each training cycle. Monitor the loss and accuracy on the validation sub-dataset so that training can be stopped in time to prevent overfitting. At the same time, turn on Mosaic for data enhancement during EDG-YOLO detection model training, and turn off Mosaic in the last 10 rounds of training to improve the generalization ability of the EDG-YOLO detection model.

[0018] Furthermore, the EDG-YOLO detection model is trained, including:

[0019] Use the test sub-dataset to evaluate the performance of the EDG-YOLO detection model, calculate the average precision mean and F1 score of the EDG-YOLO detection model on the test sub-dataset, and draw a visual evaluation result confusion matrix based on the average precision mean and F1 score to determine its performance in real situations.

[0020] Furthermore, the input of the YOLOv8 network model is replaced with the EIEStem module, including:

[0021] The input end of the YOLOv8 network model is replaced with the Sobel convolution branch and the maximum pooling branch. The Sobel convolution branch is used to extract edge information, and the maximum pooling branch is used to extract spatial information.

[0022] Furthermore, dynamic upsampling DySample is introduced into the neck network of the YOLOv8 network model, including:

[0023] Given an input feature map X, the size is C×H×W;

[0024] Use a linear layer to map the input feature map X to an offset matrix O. The number of output channels of the linear layer is 2s. 2 , where s is the upsampling factor and the size of the offset matrix O is 2s 2 ×H×W;

[0025] Reorganize the pixels of the offset matrix O and reshape it into a shape of 2×sH×sW to accommodate subsequent sampling operations;

[0026] Add the reshaped offset matrix O to the original sampling grid G ​​to obtain the sampling set S;

[0027] Use the Grid sample function in deep learning PyTorch to perform grid sampling on the input feature map X based on the sampling set S;

[0028] During the grid sampling process, bilinear interpolation is used to sample the value of the input feature map at a specific location to generate the final upsampled feature map The calculation process is as follows:

[0029] O=linear(X)

[0030] S=G+O

[0031]

[0032] In the above formula, O is the offset matrix, linear is the linear interpolation function, S is the sampling set, and G is the sampling grid. is the sampling feature map, and Grid sample is the bilinear interpolation function.

[0033] Furthermore, the detection head of the YOLOv8 network model is replaced with GNSCDH, including: introducing Group Normalization (GN) to create a group normalized shared convolutional detection head (GNSCDH), specifically: each detection layer first extracts features through GN Conv, and then processes the extracted features through a shared convolutional layer, and introduces learnable scaling parameters to allow the detection head to better handle input data of different scales and complexities.

[0034] Furthermore, it also includes cluster analysis of road damage types to obtain road damage maintenance strategies, specifically:

[0035] Based on different road damage types, according to the set road damage range and road damage degree, formulate road damage clustering rules; based on the clustering rules, use clustering algorithms to perform cluster analysis on each type of road damage to obtain clustering results;

[0036] According to the clustering results, the data of road damage items to be maintained that meet the set clustering requirements and the data of road damage items to be determined that do not meet the set clustering requirements are obtained;

[0037] Based on the neural network probability prediction model, the road damage maintenance success rate is predicted for the road damage category data to be maintained, and the prediction result is obtained; if the success rate in the prediction result is greater than the set success rate threshold, a first road damage maintenance strategy for routine maintenance of the road is formulated; if the success rate in the prediction result is less than the set success rate threshold, a second road damage maintenance strategy for thorough renovation of the road is formulated;

[0038] Based on the long short-term memory network model, for the road damage item data to be determined that do not meet the set clustering requirements, the development trend of the road damage item data is predicted to obtain the road damage development trend prediction result; according to the road damage development trend prediction result, the priority of road damage maintenance monitoring is formulated; according to the priority, the road damage item data to be determined are monitored in a hierarchical manner. If the similarity between the road damage development trend prediction result and the road damage item data to be maintained is greater than the set similarity threshold, the road damage maintenance success rate is predicted for the road damage item data to be determined that do not meet the set clustering requirements, so as to formulate the first road damage maintenance strategy or the second road damage maintenance strategy.

[0039] Furthermore, it also includes calculating the maintenance workload according to the first road damage maintenance strategy or the second road damage maintenance strategy, specifically:

[0040] According to the first road damage maintenance strategy or the second road damage maintenance strategy, the difficulty value of implementing the first road damage maintenance strategy or implementing the second road damage maintenance strategy is estimated and calculated to obtain an estimated value of the difficulty of the work;

[0041] Estimating and calculating the working cycle of implementing the first road damage maintenance strategy or implementing the second road damage maintenance strategy to obtain an estimated working cycle value;

[0042] The binomial function is used to construct the objective function of the maintenance workload, and the objective function is solved to obtain the maintenance workload measurement value; wherein the work difficulty estimation measurement value and the work cycle estimation measurement value are the binomial coefficients, and the objective function value is the maintenance workload measurement value;

[0043] The feasibility of implementing the calculated value of the maintenance workload is evaluated. If the evaluation result meets the set evaluation value, the maintenance work is arranged according to the calculated value of the maintenance workload. If the evaluation result does not meet the set evaluation value, the working cycle or maintenance workload is adjusted based on the correspondence library between the estimated value of the working cycle and the calculated value of the maintenance workload, so that the evaluation result meets the set evaluation value.

[0044] Compared with the prior art, the present invention has the following advantages and beneficial effects: by constructing the ElEStem front-end module, richer image feature representations can be learned to improve the performance of model detection; the introduction of dynamic upsampling DySample can better restore the characteristics of road damage and improve the model's ability to perceive details; replacing the GNCSDH detection head can effectively reduce the number of parameters, and at the same time enable the detection head to have stronger multi-scale perception capabilities so that it can be deployed on resource-constrained devices.

[0045] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0046] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0048] Figure 1 A schematic diagram of the steps of a road damage detection method based on EDG-YOLO;

[0049] Figure 2 is a road damage image type map;

[0050] Figure 3 This is a schematic diagram of the structure of the constructed ElEStem front-end module;

[0051] Figure 4 Schematic diagram of the structure of the replaced GNCSDH detection head. DETAILED DESCRIPTION

[0052] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0053] The present invention provides a road damage detection method based on EDG-YOLO, such as Figure 1 As shown, including:

[0054] Acquire a road damage image dataset, and perform preprocessing to obtain a preprocessed road damage image dataset;

[0055] Based on the YOLOv8 network model, EIEStem, Dysample and GNSCDH are integrated to build the EDG-YOLO detection model, and the EDG-YOLO detection model is trained to obtain the trained EDG-YOLO detection model;

[0056] The trained EDG-YOLO detection model is used to perform road damage detection on the preprocessed road damage image dataset to obtain multiple road damage types.

[0057] The working principle of the above technical solution is: in order to realize the road damage detection method based on EDG-YOLO, the present invention first obtains a road damage image dataset and performs preprocessing to obtain a preprocessed road damage image dataset; then, based on the YOLOv8 network model, EIEStem, Dysample and GNSCDH are integrated to construct an EDG-YOLO detection model, and the EDG-YOLO detection model is trained to obtain a trained EDG-YOLO detection model; finally, the trained EDG-YOLO detection model is used to perform road damage detection on the preprocessed road damage image dataset to obtain multiple road damage types.

[0058] The beneficial effects of the above technical solution are: by adopting the solution provided in this embodiment, by constructing the ElEStem front-end module, richer image feature representations can be learned to improve the performance of model detection; the introduction of dynamic upsampling DySample can better restore the characteristics of road damage and improve the model's perception of details; replacing the GNCSDH detection head can effectively reduce the number of parameters, and at the same time enable the detection head to have stronger multi-scale perception capabilities so that it can be deployed on resource-constrained devices.

[0059] In one embodiment, a road damage image dataset is obtained and preprocessed to obtain a preprocessed road damage image dataset, including:

[0060] Based on the RDD_China part of the public RDD2022 dataset, obtain the road damage image dataset, such as Figure 2 As shown, the road damage image dataset is divided into a training sub-dataset, a validation sub-dataset and a test sub-dataset according to a division ratio of 8:1:1;

[0061] The training sub-dataset, the validation sub-dataset and the test sub-dataset are standardized or normalized to obtain a preprocessed road damage image dataset.

[0062] The working principle of the above technical solution is: in order to realize the preprocessing of road damage images, the present invention first obtains a road damage image dataset based on the RDD_China part in the public RDD2022 dataset, and divides the road damage image dataset into a training sub-dataset, a validation sub-dataset and a test sub-dataset according to a division ratio of 8:1:1; then the training sub-dataset, the validation sub-dataset and the test sub-dataset are standardized or normalized to obtain the preprocessed road damage image dataset.

[0063] The beneficial effect of the above technical solution is: by adopting the solution provided by this embodiment, by preprocessing the road damage image, it is possible to ensure that an accurate preprocessed road damage image data set is obtained.

[0064] In one embodiment, based on the YOLOv8 network model, the EDG-YOLO detection model is constructed by integrating EIEStem, Dysample and GNSCDH, including:

[0065] Get the original YOLOv8 network model, replace the first layer of the YOLOv8 network model with the EIEStem module, introduce dynamic upsampling DySample in the neck network of the YOLOv8 network model, replace the detection head of the YOLOv8 network model with GNSCDH, and complete the construction of the EDG-YOLO detection model.

[0066] The working principle of the above technical solution is: in order to realize the construction of the EDG-YOLO detection model, the present invention first obtains the original YOLOv8 network model, then replaces the first layer of the YOLOv8 network model with the EIEStem module, introduces dynamic upsampling DySample in the neck network of the YOLOv8 network model, replaces the detection head of the YOLOv8 network model with GNSCDH, and completes the construction of the EDG-YOLO detection model.

[0067] The beneficial effect of the above technical solution is: by adopting the solution provided in this embodiment, by constructing the EDG-YOLO detection model, a basis is provided for subsequent road damage detection.

[0068] In one embodiment, training the EDG-YOLO detection model includes:

[0069] Configure the training hyperparameters, set the batch size to 8 and the number of iterations to 300 rounds; train the EDG-YOLO detection model, use the training sub-dataset, calculate the loss and update the EDG-YOLO detection model weights in each training iteration, and use the validation sub-dataset to evaluate the performance of the EDG-YOLO detection model after each training cycle. Monitor the loss and accuracy on the validation sub-dataset so that training can be stopped in time to prevent overfitting. At the same time, turn on Mosaic for data enhancement during EDG-YOLO detection model training, and turn off Mosaic in the last 10 rounds of training to improve the generalization ability of the EDG-YOLO detection model.

[0070] The working principle of the above technical solution is as follows: in order to implement the training of the EDG-YOLO detection model, the present invention first configures the training hyperparameters, sets the batch size to 8 and the number of iterations to 300 rounds; then trains the EDG-YOLO detection model, uses the training sub-dataset, calculates the loss and updates the EDG-YOLO detection model weight in each training iteration, and after each training cycle, uses the validation sub-dataset to evaluate the performance of the EDG-YOLO detection model, monitors the loss and accuracy on the validation sub-dataset, so as to stop the training in time to prevent overfitting; at the same time, turns on Mosaic for data enhancement during the EDG-YOLO detection model training, and turns off Mosaic in the last 10 rounds of training to improve the generalization ability of the EDG-YOLO detection model.

[0071] The beneficial effect of the above technical solution is: by adopting the solution provided in this embodiment, by training the model, the performance of the model can be guaranteed and the generalization ability of the model can be improved.

[0072] In one embodiment, training the EDG-YOLO detection model further includes:

[0073] Use the test sub-dataset to evaluate the performance of the EDG-YOLO detection model, calculate the average precision mean and F1 score of the EDG-YOLO detection model on the test sub-dataset, and draw a visual evaluation result confusion matrix based on the average precision mean and F1 score to determine its performance in real situations.

[0074] The working principle of the above technical solution is as follows: the indicators used to evaluate the model are F1 score, mean average precision (mAP), number of parameters (Params), and floating point operations per second (GFLOPs); among them, accuracy and recall are used as basic indicators, and the F1 score and mean average precision calculated based on the accuracy and recall are used as final evaluation indicators to measure the recognition accuracy of the model; GFLOPs is used to measure the complexity of the model or algorithm, and Params indicates the size of the model; generally, the smaller the Params and GFLOPs, the smaller the computing power required for the model, the lower the performance requirements for the hardware, and the easier it is to build in low-end devices; in the maintenance work of road damage detection, different types of damage have different repair methods, and the accuracy is the proportion of actual road damage in the road damage detected by the model, among which the mean average precision is an important indicator for measuring the accuracy of multi-category target detection, and the F1 score is the weighted average of accuracy and recall, which is used to measure the comprehensive performance and stability of the model.

[0075] The beneficial effect of the above technical solution is: adopting the solution provided in this embodiment is conducive to the role of the model through evaluation of the model.

[0076] In one embodiment, the input end of the YOLOv8 network model is replaced with an EIEStem module, including:

[0077] The input end of the YOLOv8 network model is replaced with the Sobel convolution branch and the maximum pooling branch. The Sobel convolution branch is used to extract edge information, and the maximum pooling branch is used to extract spatial information.

[0078] The working principle of the above technical solution is as follows: image edge information is very helpful for image recognition and computer analysis. The edge can outline the target object and contains rich internal information. It is one of the important image features in image recognition. The edge detection algorithm greatly affects the effect and accuracy of recognition and detection. For the road damage detection task, the target grayscale changes greatly, and the edge detection method can be used to identify road damage; the ElEStem module proposed in the present invention combines the Sobel Conv branch for extracting edge information and the convolution branch for extracting spatial information, and can learn richer image feature representations. The network structure is as follows Figure 3 As shown in the figure, the ElEStem module efficiently implements the Sobel operator through 3D group convolution; it extracts spatial information through pooling branches and retains important spatial information.

[0079] The beneficial effect of the above technical solution is: adopting the solution provided by this embodiment, combining edge information and spatial information, can help the model better understand the image content, and fuse the features extracted from the Sobel Conv branch and conv_branch, so that the learned feature representation contains both rich edge information and spatial information, which can better improve the performance of model detection.

[0080] In one embodiment, a dynamic upsampling DySample is introduced into the neck network of the YOLOv8 network model, including:

[0081] Given an input feature map X, the size is C×H×W;

[0082] Use a linear layer to map the input feature map X to an offset matrix O. The number of output channels of the linear layer is 2s. 2 , where s is the upsampling factor and the size of the offset matrix O is 2s 2 ×H×W;

[0083] Reorganize the pixels of the offset matrix O and reshape it into a shape of 2×sH×sW to accommodate subsequent sampling operations;

[0084] Add the reshaped offset matrix O to the original sampling grid G ​​to obtain the sampling set S;

[0085] Use the Grid sample function in deep learning PyTorch to perform grid sampling on the input feature map X based on the sampling set S;

[0086] During the grid sampling process, bilinear interpolation is used to sample the value of the input feature map at a specific location to generate the final upsampled feature map The calculation process is as follows:

[0087] O=linear(X)

[0088] S=G+O

[0089]

[0090] In the above formula, O is the offset matrix, linear is the linear interpolation function, S is the sampling set, and G is the sampling grid. is the sampling feature map, and Grid sample is the bilinear interpolation function.

[0091] The working principle of the above technical solution is as follows: the upsampling operation in the neck network of the YOLOv8 network model is usually used to restore the feature image resolution in order to better understand the detail information in the image; YOLOV8 uses the nearest neighbor interpolation method for upsampling. This method uses the nearest distance between pixels to represent the upsampling, which is simple and intuitive, but only considers the relationship between adjacent pixels and ignores the contextual semantic information of the feature map, which will cause the upsampled image to have jagged edges and blurred textures, resulting in loss of image detail information; and road damage is often small and scattered, and traditional interpolation methods may not be able to accurately capture these details, making it difficult to distinguish road damage from the background; specifically: given an input feature map X, the size is C×H×W; a linear layer is used to map the input feature map X to an offset matrix O, and the number of output channels of the linear layer is 2s 2 , where s is the upsampling factor and the size of the offset matrix O is 2s 2 ×H×W; reorganize the pixels of the offset matrix O and reshape it into a shape of 2×sH×sW to adapt to subsequent sampling operations; add the reshaped offset matrix O to the original sampling grid G ​​to obtain the sampling set S; use the Grid sample function in deep learning PyTorch to perform grid sampling on the input feature map X based on the sampling set S; in the grid sampling process, use bilinear interpolation to sample the value of the input feature map at a specific position to generate the final upsampled feature map The calculation process is as follows:

[0092] O=linear(X)

[0093] S=G+O

[0094]

[0095] In the above formula, O is the offset matrix, linear is the linear interpolation function, S is the sampling set, and G is the sampling grid. is the sampling feature map, and Grid sample is the bilinear interpolation function.

[0096] The beneficial effect of the above technical solution is: by adopting the solution provided in this embodiment, a dynamic upsampling DySample method is introduced to better restore the characteristics of road damage while improving the resolution of feature maps, thereby improving the model's ability to perceive details.

[0097] In one embodiment, the detection head of the YOLOv8 network model is replaced with GNSCDH, including: introducing Group Normalization (GN) to create a group normalized shared convolutional detection head (GNSCDH), specifically: each detection layer first extracts features through GN Conv, and then processes the extracted features through a shared convolutional layer, and introduces learnable scaling parameters to allow the detection head to better handle input data of different scales and complexities.

[0098] The working principle of the above technical solution is as follows: DySample dynamically adjusts each upsampled point by learning the offset to better adapt to the pixel information and content characteristics of the feature map; in the road damage detection task, although the original YOLOv8 detector head has a high detection efficiency, it still has certain limitations; first, the three detector heads need to extract image information through two 3×3 convolutions and one 1×1 convolution respectively. This structure leads to a significant increase in the number of algorithm parameters, accounting for one-fifth of the total algorithm calculation; secondly, the traditional single-scale prediction structure used by the original algorithm cannot handle multi-scale targets well. It only predicts from one scale of the feature map, ignoring the contribution of other scale features to the detection, which leads to low utilization of model parameters; because for different feature levels, the features of the objects detected should be similar at relative scales, which is also confirmed by the detection heads with shared parameters used by detectors such as Retina-Ne and FCOS; different from the standard Retina-Net, FCOS uses Group Normalization in the newly added convolutional layers except for the last prediction layer to make its training more stable; referring to its approach, the present invention introduces Group Normalization (GN) to create a group normalized shared convolutional detection head (GNSCDH), and the improved detection head structure is as follows: Figure 4 As shown in the figure, the core idea of ​​GNSCDH design is that each detection layer first extracts features through GN Conv. The extracted features are further processed through shared convolutional layers, which reduces the number of parameters and improves the efficiency of feature extraction. In addition, in order to deal with the problem of inconsistent target scales detected by each detection head, a learnable scaling parameter is introduced to allow the detection head to better handle input data of different scales and complexities. Through the above structure, the improved detection head effectively reduces the number of model parameters and computational complexity, and the detection head has higher multi-scale perception capabilities so that it can be deployed on resource-constrained devices.

[0099] The beneficial effect of the above technical solution is: adopting the solution provided by this embodiment can avoid the problem that the road damage feature information may be weakened during the upsampling process, improve the model's ability to perceive details, and can more accurately locate and identify road damage.

[0100] In one embodiment, the method further includes performing a cluster analysis on the road damage types to obtain a road damage maintenance strategy, specifically:

[0101] Based on different road damage types, according to the set road damage range and road damage degree, formulate road damage clustering rules; based on the clustering rules, use clustering algorithms to perform cluster analysis on each type of road damage to obtain clustering results;

[0102] According to the clustering results, the data of road damage items to be maintained that meet the set clustering requirements and the data of road damage items to be determined that do not meet the set clustering requirements are obtained;

[0103] Based on the neural network probability prediction model, the road damage maintenance success rate is predicted for the road damage category data to be maintained, and the prediction result is obtained; if the success rate in the prediction result is greater than the set success rate threshold, a first road damage maintenance strategy for routine maintenance of the road is formulated; if the success rate in the prediction result is less than the set success rate threshold, a second road damage maintenance strategy for thorough renovation of the road is formulated;

[0104] Based on the long short-term memory network model, for the road damage item data to be determined that do not meet the set clustering requirements, the development trend of the road damage item data is predicted to obtain the road damage development trend prediction result; according to the road damage development trend prediction result, the priority of road damage maintenance monitoring is formulated; according to the priority, the road damage item data to be determined are monitored in a hierarchical manner. If the similarity between the road damage development trend prediction result and the road damage item data to be maintained is greater than the set similarity threshold, the road damage maintenance success rate is predicted for the road damage item data to be determined that do not meet the set clustering requirements, so as to formulate the first road damage maintenance strategy or the second road damage maintenance strategy.

[0105] The working principle of the above technical solution is as follows: in order to realize the formulation of road damage maintenance strategy, the present invention also includes clustering analysis of road damage types to obtain road damage maintenance strategy, specifically: based on different road damage types, according to the set road damage range and road damage degree, formulate road damage clustering rules; based on the clustering rules, use the clustering algorithm to perform clustering analysis on each type of road damage situation to obtain clustering results; according to the clustering results, obtain the road damage category data to be maintained that meets the set clustering requirements, and the road damage category data to be determined that does not meet the set clustering requirements; based on the neural network probability prediction model, predict the road damage maintenance success rate of the road damage category data to be maintained to obtain the prediction result; if the success rate in the prediction result is greater than the set success rate threshold, formulate a regular maintenance of the road a first road damage maintenance strategy for road protection; if the success rate in the prediction result is less than the set success rate threshold, a second road damage maintenance strategy for completely renovating the road is formulated; based on the long short-term memory network model, for the road damage item data to be determined that does not meet the set clustering requirements, the road damage item data development trend is predicted to obtain the road damage development trend prediction result; according to the road damage development trend prediction result, the priority of road damage maintenance monitoring is formulated; according to the priority, the road damage item data to be determined are monitored in a hierarchical manner, and if the similarity between the road damage development trend prediction result and the road damage item data to be maintained is greater than the set similarity threshold, the road damage maintenance success rate is predicted for the road damage item data to be determined that does not meet the set clustering requirements, so as to formulate the first road damage maintenance strategy or the second road damage maintenance strategy.

[0106] The beneficial effect of the above technical solution is: by adopting the solution provided by this embodiment, a road damage maintenance strategy is obtained by clustering analysis of road damage types, which can provide a basis for subsequent road damage maintenance.

[0107] In one embodiment, the method further includes calculating the maintenance workload according to the first road damage maintenance strategy or the second road damage maintenance strategy, specifically:

[0108] According to the first road damage maintenance strategy or the second road damage maintenance strategy, the difficulty value of implementing the first road damage maintenance strategy or implementing the second road damage maintenance strategy is estimated and calculated to obtain an estimated value of the difficulty of the work;

[0109] Estimating and calculating the working cycle of implementing the first road damage maintenance strategy or implementing the second road damage maintenance strategy to obtain an estimated working cycle value;

[0110] The binomial function is used to construct the objective function of the maintenance workload, and the objective function is solved to obtain the maintenance workload measurement value; wherein the work difficulty estimation measurement value and the work cycle estimation measurement value are the binomial coefficients, and the objective function value is the maintenance workload measurement value;

[0111] The feasibility of implementing the calculated value of the maintenance workload is evaluated. If the evaluation result meets the set evaluation value, the maintenance work is arranged according to the calculated value of the maintenance workload. If the evaluation result does not meet the set evaluation value, the working cycle or maintenance workload is adjusted based on the correspondence library between the estimated value of the working cycle and the calculated value of the maintenance workload, so that the evaluation result meets the set evaluation value.

[0112] The working principle of the above technical solution is as follows: in order to calculate the maintenance workload, the present invention estimates and calculates the work difficulty value of implementing the first road damage maintenance strategy or the second road damage maintenance strategy according to the first road damage maintenance strategy or the second road damage maintenance strategy, and obtains the estimated work difficulty value; estimates and calculates the work cycle of implementing the first road damage maintenance strategy or the second road damage maintenance strategy, and obtains the estimated work cycle value; constructs an objective function about the maintenance workload using a binomial function, and solves the objective function to obtain the maintenance workload value; wherein, the estimated work difficulty value and the estimated work cycle value are binomial coefficients, and the objective function value is the maintenance workload value; evaluates the feasibility of the maintenance workload value, and if the evaluation result meets the set evaluation value, arranges the maintenance work according to the maintenance workload value; if the evaluation result does not meet the set evaluation value, adjusts the work cycle or maintenance workload according to the correspondence library between the estimated work cycle value and the maintenance workload value, so that the evaluation result meets the set evaluation value.

[0113] The beneficial effect of the above technical solution is: by adopting the solution provided in this embodiment, in order to calculate the maintenance workload, the work difficulty value and work cycle of implementing the first road damage maintenance strategy or implementing the second road damage maintenance strategy are estimated and calculated according to the first road damage maintenance strategy or the second road damage maintenance strategy, and the maintenance workload is obtained by solving the objective function, which is convenient for accurately determining the maintenance workload to ensure the efficient implementation of road damage maintenance.

[0114] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A road damage detection method based on EDG-YOLO, characterized in that: include: Acquire a road damage image dataset, and perform preprocessing to obtain a preprocessed road damage image dataset; Based on the YOLOv8 network model, EIEStem, Dysample and GNSCDH are integrated to build the EDG-YOLO detection model, and the EDG-YOLO detection model is trained to obtain the trained EDG-YOLO detection model; The trained EDG-YOLO detection model is used to perform road damage detection on the preprocessed road damage image dataset to obtain multiple road damage types.

2. The road damage detection method based on EDG-YOLO according to claim 1, characterized in that: Obtain a road damage image dataset and perform preprocessing to obtain a preprocessed road damage image dataset, including: Based on the RDD_China part of the public RDD2022 dataset, a road damage image dataset is obtained, and the road damage image dataset is divided into a training sub-dataset, a validation sub-dataset, and a test sub-dataset according to a division ratio of 8:1:1; The training sub-dataset, the validation sub-dataset and the test sub-dataset are standardized or normalized to obtain a preprocessed road damage image dataset.

3. The road damage detection method based on EDG-YOLO according to claim 2, characterized in that: Based on the YOLOv8 network model, the EDG-YOLO detection model is constructed by integrating EIEStem, Dysample and GNSCDH, including: Get the original YOLOv8 network model, replace the first layer of the YOLOv8 network model with the EIEStem module, introduce dynamic upsampling DySample in the neck network of the YOLOv8 network model, replace the detection head of the YOLOv8 network model with GNSCDH, and complete the construction of the EDG-YOLO detection model.

4. The road damage detection method based on EDG-YOLO according to claim 2, characterized in that: Training the EDG-YOLO detection model includes: Configure the training hyperparameters, set the batch size to 8 and the number of iterations to 300 rounds; train the EDG-YOLO detection model, use the training sub-dataset, calculate the loss and update the EDG-YOLO detection model weights in each training iteration, and use the validation sub-dataset to evaluate the performance of the EDG-YOLO detection model after each training cycle. Monitor the loss and accuracy on the validation sub-dataset so that training can be stopped in time to prevent overfitting. At the same time, turn on Mosaic for data enhancement during EDG-YOLO detection model training, and turn off Mosaic in the last 10 rounds of training to improve the generalization ability of the EDG-YOLO detection model.

5. The road damage detection method based on EDG-YOLO according to claim 1, characterized in that: Training the EDG-YOLO detection model also includes: Use the test sub-dataset to evaluate the performance of the EDG-YOLO detection model, calculate the average precision mean and F1 score of the EDG-YOLO detection model on the test sub-dataset, and draw a visual evaluation result confusion matrix based on the average precision mean and F1 score to determine its performance in real situations.

6. The road damage detection method based on EDG-YOLO according to claim 3, characterized in that: Replace the input of the YOLOv8 network model with the EIEStem module, including: The input end of the YOLOv8 network model is replaced with the Sobel convolution branch and the maximum pooling branch. The Sobel convolution branch is used to extract edge information, and the maximum pooling branch is used to extract spatial information.

7. The road damage detection method based on EDG-YOLO according to claim 3, characterized in that: Introduce dynamic upsampling DySample in the neck network of the YOLOv8 network model, including: Given an input feature map X, the size is C×H×W; Use a linear layer to map the input feature map X to an offset matrix O. The number of output channels of the linear layer is 2s. 2 , where s is the upsampling factor and the size of the offset matrix O is 2s 2 ×H×W; Reorganize the pixels of the offset matrix O and reshape it into a shape of 2×sH×sW to accommodate subsequent sampling operations; Add the reshaped offset matrix O to the original sampling grid G ​​to obtain the sampling set S; Use the Grid sample function in deep learning PyTorch to perform grid sampling on the input feature map X based on the sampling set S; During the grid sampling process, bilinear interpolation is used to sample the value of the input feature map at a specific location to generate the final upsampled feature map The calculation process is as follows: O=linear(X) S=G+O In the above formula, O is the offset matrix, linear is the linear interpolation function, S is the sampling set, and G is the sampling grid. is the sampling feature map, and Grid sample is the bilinear interpolation function.

8. The road damage detection method based on EDG-YOLO according to claim 3, characterized in that: The detection head of the YOLOv8 network model is replaced with GNSCDH, including: introducing Group Normalization (GN) to create a group normalized shared convolutional detection head (GNSCDH). Specifically, each detection layer first extracts features through GN Conv, and then processes the extracted features through a shared convolutional layer, and introduces learnable scaling parameters to allow the detection head to better handle input data of different scales and complexities.

9. The road damage detection method based on EDG-YOLO according to claim 1, characterized in that: It also includes cluster analysis of road damage types to obtain road damage maintenance strategies, specifically: Based on different road damage types, according to the set road damage range and road damage degree, formulate road damage clustering rules; based on the clustering rules, use clustering algorithms to perform cluster analysis on each type of road damage to obtain clustering results; According to the clustering results, the data of road damage items to be maintained that meet the set clustering requirements and the data of road damage items to be determined that do not meet the set clustering requirements are obtained; Based on the neural network probability prediction model, the road damage maintenance success rate is predicted for the road damage category data to be maintained, and the prediction result is obtained; if the success rate in the prediction result is greater than the set success rate threshold, a first road damage maintenance strategy for routine maintenance of the road is formulated; if the success rate in the prediction result is less than the set success rate threshold, a second road damage maintenance strategy for thorough renovation of the road is formulated; Based on the long short-term memory network model, the road damage category data to be determined that does not meet the set clustering requirements are predicted to obtain the road damage development trend prediction results; According to the prediction results of road damage development trend, the priority of road damage maintenance monitoring is formulated; according to the priority, the data of road damage items to be determined are monitored in a graded manner. If the similarity between the prediction results of road damage development trend and the data of road damage items to be maintained is greater than the set similarity threshold, the road damage maintenance success rate is predicted for the data of road damage items to be determined that do not meet the set clustering requirements, so as to formulate the first road damage maintenance strategy or the second road damage maintenance strategy.

10. The road damage detection method based on EDG-YOLO according to claim 9, characterized in that: It also includes calculating the maintenance workload according to the first road damage maintenance strategy or the second road damage maintenance strategy, specifically: According to the first road damage maintenance strategy or the second road damage maintenance strategy, the difficulty value of implementing the first road damage maintenance strategy or implementing the second road damage maintenance strategy is estimated and calculated to obtain an estimated value of the difficulty of the work; Estimating and calculating the working cycle of implementing the first road damage maintenance strategy or implementing the second road damage maintenance strategy to obtain an estimated working cycle value; The binomial function is used to construct the objective function of the maintenance workload, and the objective function is solved to obtain the maintenance workload measurement value; wherein the work difficulty estimation measurement value and the work cycle estimation measurement value are the binomial coefficients, and the objective function value is the maintenance workload measurement value; The feasibility of implementing the calculated value of the maintenance workload is evaluated. If the evaluation result meets the set evaluation value, the maintenance work is arranged according to the calculated value of the maintenance workload. If the evaluation result does not meet the set evaluation value, the working cycle or maintenance workload is adjusted based on the correspondence library between the estimated value of the working cycle and the calculated value of the maintenance workload, so that the evaluation result meets the set evaluation value.

Images