Lightweight road defect detection method and related device
By adding downsampling convolution modules and feature extraction modules to the basic YOLOv5 network model, and introducing RGConv and GRGConv, the problems of insufficient real-time, limited application range, high computational complexity and slow detection in the prior art are solved, and more efficient and accurate road defect detection is achieved.
Patent Information
- Application Number
- CN202510212508.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-25
- Publication Date
- 2025-05-30
AI Technical Summary
The existing road defect detection methods based on deep learning technology have problems such as insufficient real-time, limited application range, high computational complexity and slow detection speed.
The basic YOLOv5 network model is adopted as the basic framework of the road defect detection model, where the downsampling convolution module and feature extraction module are added, and RGConv and GRGConv are introduced to improve the accuracy and efficiency of detection.
Through the improved model, more accurate road defect detection results can be obtained with fewer computing resources, improving the accuracy and efficiency of detection, especially with practical value on end devices.
Smart Images
Figure CN120071090A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of computer vision, and particularly relates to a lightweight road defect detection method and related devices. Background Art
[0002] Traditional road defect detection mainly relies on manual inspections or semi-automatic detection equipment; manual inspections are limited by the professional level and experience of inspectors, and it is difficult to accurately identify all defects, especially in complex conditions such as at night or in bad weather; although semi-automatic detection equipment can automatically detect some defects, manual intervention is still required, and its detection ability for special defects such as fatigue cracks is limited.
[0003] With the development of deep learning technology, especially the progress of neural networks, some researchers have used methods such as convolutional neural networks to analyze road images to identify and locate potential road defects; existing road defect detection methods based on deep learning technology usually include: first, using semi-automatic detection equipment to collect data and conduct manual reviews; second, object recognition technologies relying on key area feature extraction; finally, object detection methods based on deep neural networks; since deep neural networks exhibit powerful image recognition and analysis capabilities, they can significantly improve the accuracy of road defect detection; however, the above-mentioned road defect detection methods based on deep learning technology are semi-automatic recognition methods, which have problems of insufficient real-time performance and high maintenance costs in the later automatic processing of images; secondly, the methods based on key area feature extraction have limited application scope and high computational complexity; in addition, although the two-stage detection method has high accuracy, its detection speed is slow. Summary of the Invention
[0004] Aiming at the technical problems existing in the prior art, the present invention provides a lightweight road defect detection method and related devices to solve the technical problems of insufficient real-time performance, limited application scope, high computational complexity and slow detection speed existing in the existing road defect detection methods based on deep learning technology.
[0005] To achieve the above object, the technical solution adopted by the present invention is as follows: The present invention provides a lightweight road defect detection method, including: Obtain the original image of the road to be detected; Input the original image of the road to be detected into a pre-constructed road defect detection model, and output the defect detection result of the road to be detected; wherein, the pre-constructed road defect detection model is obtained by improving the basic YOLOv5 network model; The improvement of the basic YOLOv5 network model includes: In the basic YOLOv5 network model, a downsampling convolutional module and a feature extraction module are added, and RGConv and GRGConv are introduced; among them, the GRGConv is the RGConv with Gaussian blur introduced.
[0006] Furthermore, the RGConv is a Ghost convolutional module with RepDW convolution introduced; among them, the RepDW convolution includes two depthwise separable convolutional modules with different kernel sizes, and the RepDW is used to replace the depthwise separable convolutional module in the Ghost convolutional module.
[0007] Furthermore, in the GRGConv, the introduced Gaussian blur is used to perform Gaussian blur operation on the feature map before the feature map is input into the RepDW convolution.
[0008] Furthermore, in the process of adding a downsampling convolutional module and a feature extraction module to the basic YOLOv5 network model and introducing RGConv and GRGConv, it includes: After the first downsampling convolutional module in the backbone network of the basic YOLOv5 network model, a downsampling convolutional module and a feature extraction module are sequentially added; Among them, the added downsampling convolutional module is implemented by the CBS module, and the added feature extraction module is implemented by the C3GRG module; the C3GRG module is obtained by modifying the bottleneck of the C3 module in the basic YOLOv5 network model using GRGConv.
[0009] Furthermore, in the process of adding a downsampling convolutional module and a feature extraction module to the basic YOLOv5 network model and introducing RGConv and GRGConv, it also includes: In the backbone network of the basic YOLOv5 network model, the last two C3 modules are replaced by C3RG modules; In the neck network of the basic YOLOv5 network model, the last two C3 modules are replaced by C3RG modules; Among them, the C3RG module is obtained by modifying the bottleneck of the C3 module in the basic YOLOv5 network model using RGConv.
[0010] Furthermore, the C3GRG module is used to perform feature extraction on the input feature map through GRGConv, then perform convolution processing with a kernel size of 1 and add it to the input feature map; the C3RG module is used to perform feature extraction on the input feature map through RGConv, then perform convolution processing with a kernel size of 1 and add it to the input feature map.
[0011] The present invention also provides a lightweight road defect detection system, including: An image acquisition module, configured to acquire the original image of the road to be detected; A defect detection module, configured to input the original image of the road to be detected into a pre-constructed road defect detection model, and output the defect detection result of the road to be detected; wherein, the pre-constructed road defect detection model is obtained by improving the basic YOLOv5 network model; The improvement of the basic YOLOv5 network model includes: Adding a downsampling convolutional module to the basic YOLOv5 network model, and introducing RGConv and GRGConv; wherein, the GRGConv is an RGConv introduced with Gaussian blur.
[0012] The present invention also provides a lightweight road defect detection device, including: A processor, suitable for executing a computer program; A computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by the processor, it executes the lightweight road defect detection method described above.
[0013] The present invention also provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, it implements the lightweight road defect detection method described above.
[0014] The present invention also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the lightweight road defect detection method described above.
[0015] Compared with the prior art, the beneficial effects of the present invention are: The lightweight road defect detection method provided by the present invention uses the basic YOLOv5 network model as the basic framework of the road defect detection model, adds a downsampling convolution module and a feature extraction module to the basic YOLOv5 network model, and introduces RGConv and GRGConv, which can effectively improve the accuracy of road defect detection and achieve more accurate results with fewer computing resources. Among them, by adding a downsampling convolution module, more detailed feature extraction can be effectively carried out to obtain better road defect detection effects. By introducing RGConv, the number of parameters and the amount of calculation can be significantly reduced without affecting the accuracy of target detection, which has great practical value for terminal devices that cannot use GPUs. By introducing GRGConv, through Gaussian blur operation, the pixel ratio of cracks in the original image in the detection frame is expanded and the abnormal parts of abnormal objects are diffused and enhanced, making them easier to be recognized, and further improving the accuracy of target detection.
[0016] The lightweight road defect detection system, lightweight road defect detection device, computer-readable storage medium and computer program product provided by the present invention have all the advantages of the above lightweight road defect detection method. Description of the Drawings
[0017] Figure 1 It is a flowchart of the lightweight road defect detection method provided for Example 1; Figure 2 It is a structural block diagram of the RepDW-Ghost convolution module in Example 1; Figure 3 It is a structural block diagram of the RepDW-Ghost convolution module with Gaussian blur introduced in Example 1; Figure 4 It is a structural block diagram of the bottleneck improved by using RGConv or GRGConv in Example 1; Figure 5 It is a structural block diagram of C3RG or C3GRG in Example 1; Figure 6 It is a structural block diagram of the pre-built road defect detection model in Example 1; Figure 7 It is a comparison schematic diagram of the road defect detection effects of YOLOv5m and the pre-built road defect detection model in Example 1. Among them, Figure 7 The four left pictures are the road defect detection effect pictures of YOLOv5m, Figure 7 The four right pictures are the road defect detection effect pictures of the pre-built road defect detection model in Example 1; Figure 8It is the structural block diagram of the lightweight road defect detection system provided in Embodiment 2; Figure 9 It is the structural block diagram of the lightweight road defect detection device provided in Embodiment 3. Specific Embodiments
[0018] In order to make the technical problems, technical solutions and beneficial effects solved by the present invention clearer, the following specific embodiments are used to further elaborate on the present invention in detail. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0019] Embodiment 1 As shown in the appendix Figure 1 This Embodiment 1 provides a lightweight road defect detection method, including the following steps: Step 1: Obtain the original image of the road to be detected.
[0020] Step 2: Input the original image of the road to be detected into a pre-constructed road defect detection model, and output the defect detection result of the road to be detected; wherein, the pre-constructed road defect detection model is obtained by improving the basic YOLOv5 network model.
[0021] In this Embodiment 1, the process of obtaining the pre-constructed road defect detection model by improving the basic YOLOv5 network model includes: adding a downsampling convolution module and a feature extraction module to the basic YOLOv5 network model, and introducing RGConv and GRGConv; wherein, the process of adding a downsampling convolution module and a feature extraction module to the basic YOLOv5 network model includes: adding a downsampling convolution module and a feature extraction module to the backbone network of the basic YOLOv5 network model, and the added downsampling convolution module and feature extraction module are sequentially arranged after the first downsampling convolution module of the backbone network.
[0022] It should be noted that in YOLOv5, a higher input image resolution often leads to better object detection effects; however, due to the two-dimensional nature of images, increasing the image resolution will cause the computational complexity of the model to increase quadratically; in this Embodiment 1, by adding a downsampling convolution module and a feature extraction module after the first downsampling convolution module of the backbone network, it is possible to make the original image input into the pre-constructed road defect detection model increase only the computational complexity corresponding to preprocessing, downsampling convolution, and feature extraction under the condition of doubling the resolution, avoiding the quadratic growth of the computational complexity, so as to effectively perform more detailed feature extraction and thus obtain better road defect detection effects.
[0023] Preferably, the added downsampling convolution module is implemented by adding a new CBS module, so as to increase the resolution of the input original image without significantly increasing the computational load; the added feature extraction module is implemented by the C3GRG module; the C3GRG module is obtained by modifying the bottleneck of the C3 module in the basic YOLOv5 network model using GRGConv; wherein, the basic YOLOv5 network model with the added downsampling convolution module and feature extraction module is denoted as the extended network structure, and is simply referred to as the YOLOv5m-long model.
[0024] In Embodiment 1, the RGConv, namely the RepDW-Ghost convolution module, is specifically: a Ghost convolution module introducing RepDW convolution; wherein, the RepDW is used to replace the depthwise separable convolution module in the Ghost convolution module, and the RepDW convolution includes two depthwise separable convolution modules with different kernel sizes; specifically, the RGConv is based on the Ghost convolution module, and the depthwise separable convolution module in the Ghost convolution module is modified into a RepDW convolution composed of two depthwise separable convolutions with different kernel sizes, as shown in the appendix Figure 2 shown; as can be seen from the appendix Figure 2 it can be seen that the RGConv includes a CBS module, a RepDW convolution, a SiLU module, and a CONCAT module connected in series in sequence; wherein, the RepDW convolution includes a first DWConv module and a second DWConv module, and the kernel sizes of the first DWConv module and the second DWConv module are different.
[0025] It should be noted that since the road defect detection model often needs to be deployed on terminal devices without a configured GPU to achieve road defect detection; and terminal devices without a configured GPU usually cannot bear too large a number of parameters and computational load; in Embodiment 1, by introducing RGConv, it is possible to perform feature extraction on features with different receptive fields while keeping the accuracy from decreasing significantly and reducing the number of parameters and computational load, so as to more accurately capture the features of the target.
[0026] In Embodiment 1, the GRGConv is an RGConv introducing Gaussian blur, that is, a RepDW-Ghost convolution module introducing Gaussian blur, as shown in the appendix Figure 3 shown, and the GRGConv is used to implement the function of the added feature extraction module; as can be seen from the appendix Figure 3It can be seen that the GRGConv includes a CBS module, a Gaussian blur module, a RepDW convolution, a SiLU module, and a CONCAT module connected in series in sequence; wherein, the RepDW convolution includes a first DWConv module and a second DWConv module, and the kernel sizes of the first DWConv module and the second DWConv module are different.
[0027] It should be noted that in the GRGConv, the introduced Gaussian blur is used to perform a Gaussian blur operation on the feature map before the feature map is input into the RepDW convolution; through the Gaussian blur operation, the pixel proportion of the cracks in the original image in the detection frame is enlarged and the abnormal parts of the abnormal objects are diffused and enhanced, so that they can be more easily recognized; at the same time, the residual structure can preserve the original features and avoid loss due to the Gaussian blur operation, and can achieve better recognition of road cracks and abnormal road objects without affecting the detection of normal road objects.
[0028] It should also be noted that road cracks are one of the main targets of road defect detection; since road cracks are often narrow strip-shaped, bifurcated or reticular structures, they often have a small pixel ratio in the detection frame, that is, most of the pixels in the detection frame are the background; secondly, the shape of road cracks is often a concave polygon, while the targets in conventional object detection tasks are often convex polygons; in addition, for the abnormalities of road objects such as speed bumps, curbs, and manhole covers, traditional feature extraction methods cannot distinguish them from normal road objects; in this Embodiment 1, before the feature map is input into the RepDW convolution, the Gaussian blur module is used to perform Gaussian blur processing on the input feature map, which can effectively improve the recognition effect of cracks and abnormal road objects.
[0029] In this Embodiment 1, the process of introducing RGConv and GRGConv into the basic YOLOv5 network model includes: in the backbone network of the basic YOLOv5 network model, the last two C3 modules are replaced with C3RG modules; in the neck network of the basic YOLOv5 network model, the last two C3 modules are replaced with C3RG modules; the GRGConv is introduced through the added feature extraction module, that is, the added feature extraction module is implemented by the C3GRG module; wherein, the C3GRG module is obtained by modifying the bottleneck of the C3 module in the basic YOLOv5 network model using GRGConv; the C3RG module is obtained by modifying the bottleneck of the C3 module in the basic YOLOv5 network model using RGConv.
[0030] That is to say, by selecting the basic YOLOv5 network model as the basic framework of the road defect detection model, an extended network structure is obtained by adding a downsampling convolutional module and a feature extraction module. By introducing GRGConv into the feature extraction module and introducing RGConv into the extended network structure, RGConv or GRGConv is used to improve the C3 module in the basic YOLOv5 network model.
[0031] Specifically, the improvement of the C3 module includes: Using RGConv or GRGConv to improve the bottleneck in the C3 module, as shown in the appendix Figure 4 shown; then, replace the bottleneck in the original C3 module with the improved bottleneck to complete the improvement of C3 and obtain the C3RG module or C3GRG module, as shown in the appendix Figure 5 shown; where the C3GRG module is used to perform feature extraction on the input feature map through GRGConv, then perform convolutional processing with a kernel size of 1 and add it to the input feature map; the C3RG module is used to perform feature extraction on the input feature map through RGConv, then perform convolutional processing with a kernel size of 1 and add it to the input feature map.
[0032] For the overall structure of the pre-constructed road defect detection model, the basic YOLOv5 network model is used as the benchmark framework. After the first downsampling convolutional module in the backbone network of the basic YOLOv5 network model, a downsampling convolutional module and a feature extraction module are added. The C3 module (i.e., C3GRG) improved by the RepDW-Ghost convolutional module with introduced Gaussian blur is used as the additional feature extraction module to introduce GRGConv; and the C3 module (i.e., C3RG) improved by the RepDW-Ghost convolutional module is used to replace the last two C3 modules in the backbone network and the last two C3 modules in the neck network to introduce RGConv, as shown in the appendix Figure 6 shown; the appendix Figure 6 shows the overall structure of the pre-constructed road defect detection model, including the backbone network, the neck network, and the head network.
[0033] The backbone network includes a first CBS module, a second CBS module, a first C3GRG module, a third CBS module, a first C3 module, a fourth CBS module, a second C3 module, a fifth CBS module, a first C3RG module, a sixth CBS module, a second C3RG module, and an SPPF module connected in series in sequence; where the second CBS module is used to achieve the downsampling effect of the downsampling convolutional module.
[0034] The neck network includes a seventh CBS module, a first upsampling module, a first fusion module, a third C3 module, an eighth CBS module, a second upsampling module, a second fusion module, a fourth C3 module, a ninth CBS module, a third fusion module, a third C3RG module, a tenth CBS module, a fourth fusion module, and a fourth C3RG module.
[0035] The head network includes a first detection head, a second detection head, and a third detection head.
[0036] In the backbone network, the first CBS module is used to generate a first CBS feature based on the original image of the road to be detected; the second CBS module is used to generate a second CBS feature based on the first CBS feature; the first C3GRG module is used to generate a first C3GRG feature based on the second CBS feature; the third CBS module is used to generate a third CBS feature based on the first C3GRG feature; the first C3 module is used to generate a first C3 feature based on the third CBS feature; the fourth CBS module is used to generate a fourth CBS feature based on the first C3 feature; the second C3 module is used to generate a second C3 feature based on the fourth CBS feature; the fifth CBS module is used to generate a fifth CBS feature based on the second C3 feature; the first C3RG module is used to generate a first C3RG feature based on the fifth CBS feature; the sixth CBS module is used to generate a sixth CBS feature based on the first C3RG feature; the second C3RG module is used to generate a second C3RG feature based on the sixth CBS feature; the SPPF module is used to generate a pooled feature based on the second C3RG feature.
[0037] In the neck network, the seventh CBS module is used to generate seventh CBS features based on pooled features; the first upsampling module is used to generate first upsampled features based on the seventh CBS features; the first fusion module is used to fuse and generate first fusion features based on the first upsampled features and the first C3RG features; the third C3 module is used to generate third C3 features based on the first fusion features; the eighth CBS module is used to generate eighth CBS features based on the third C3 features; the second upsampling module is used to generate second upsampled features based on the eighth CBS features; the second fusion module is used to fuse and generate second fusion features based on the second upsampled features and the second C3 features; the fourth C3 module is used to generate fourth C3 features based on the second fusion features; the ninth CBS module is used to generate ninth CBS features based on the fourth C3 features; the third fusion module is used to fuse and generate third fusion features based on the eighth CBS features and the ninth CBS features; the third C3RG module is used to generate third C3RG features based on the third fusion features; the tenth CBS module is used to generate tenth CBS features based on the third C3RG features; the fourth fusion module is used to fuse and generate fourth fusion features based on the seventh CBS features and the tenth CBS features; the fourth C3RG module is used to generate fourth C3RG features based on the fourth fusion features.
[0038] It should be noted that the input of the seventh CBS module is the pooled features output by the SPPF module, and the inputs of the first fusion module include the first upsampled features output by the first upsampling module and the first C3RG features output by the first C3RG module; the inputs of the second fusion module include the second upsampled features output by the second upsampling module and the second C3 features output by the second C3 module; the inputs of the third fusion module include the eighth CBS features output by the eighth CBS module and the ninth CBS features output by the ninth CBS module; the inputs of the fourth fusion module include the seventh CBS features output by the seventh CBS module and the tenth CBS features output by the tenth CBS module.
[0039] In the head network, the first detection head is used to detect defects based on the fourth C3 features, the second detection head is used to detect defects based on the third C3RG features, and the third detection head is used to detect defects based on the fourth C3RG features.
[0040] In this Embodiment 1, the depthwise separable convolution part in Ghost convolution is modified into a RepDW convolution composed of two depthwise separable convolutions with different kernel sizes, so that it can extract features of different receptive fields without significantly increasing the number of parameters and computational complexity, thereby more accurately capturing the features of the target. Before the feature map is input into the RepDW convolution, a Gaussian blur operation is performed on it. Through the Gaussian blur operation, the pixel ratio of cracks in the original image in the detection box is enlarged and the abnormal parts of abnormal objects are diffused and enhanced, making it easier to be recognized. Sampling: After the first downsampling convolution module in the image feature extraction part of the basic YOLOv5 network model, a downsampling convolution and a corresponding feature extraction module are added, so that when the image uses twice the input image resolution, only the preprocessing and the computational complexity corresponding to the added downsampling convolution and feature extraction module are increased, and the quadratic growth of the computational complexity will not occur. Furthermore, more detailed feature extraction can be effectively performed to obtain better road defect detection results.
[0041] Verification experiment description: (1)Comparative experiment Verification experiment parameter description: During the verification experiment, an experimental environment was created with two RTX4090 GPUs and one Intel i9-13900k CPU; the model was trained using a batch size of 64; in order to fully train the model and achieve good convergence, the number of training epochs was set to 120; in terms of the optimizer and data augmentation, SGD was selected as the optimizer, and the mosaic data augmentation method was adopted; in the test stage, a confidence threshold of 0.25 and an IOU (Intersection over Union) threshold of 0.45 were set; among them, the selection of the above parameters is based on experimental experience during training and verification, aiming to achieve the best detection performance; all experiments were completed on the same hardware device, ensuring the fairness and comparability of the experiments.
[0042] Comparative experiments were conducted on the basic YOLOv5 network model and the pre-constructed road defect detection model in this Embodiment 1 using input image resolutions of 640 and 1280 respectively; among them, the basic YOLOv5 network model is YOLOv5m as an example; the comparative experiment results of YOLOv5m and the pre-constructed road defect detection model in this Embodiment 1 are shown in Table 1 below; the schematic diagram of the comparison of the road defect detection effects of YOLOv5m and the pre-constructed road defect detection model in Embodiment 1 is as shown in the appendix Figure 7 as shown.
[0043] Table 1 Comparative experiment results
[0044] As can be seen from Table 1 above, the pre-built road defect detection model described in Embodiment 1 of the present invention can outperform YOLOv5m in terms of both mAP50 and mAP95 while using relatively fewer parameters and computational resources. In particular, it has achieved a 2.3% improvement in mAP50. For terminal devices, the pre-built road defect detection model can obtain more accurate results with fewer computational resources and time.
[0045] As can be seen from the appendix Figure 7 the pre-built road defect detection model described in Embodiment 1 of the present invention has better road defect detection capabilities, with better detection effects than YOLOv5m under both sparse and dense defect conditions; it can clearly detect defects even when different defects overlap; and it can more accurately identify road objects such as signs, reducing misjudgments.
[0046] (2) Ablation experiment To verify the effectiveness of each improvement method in the model, YOLOv5m, YOLOv5m-long, YOLOv5m-longRG, and YOLOv5m-longGRG were trained and tested respectively to verify the effectiveness of the model through ablation experiments. Among them, the input image resolution of YOLOv5m was 640×640, while the input image resolution of the other three models was 1280×1280. Specifically, the results of the ablation experiment are shown in Table 2 below.
[0047] Table 2 Results of the ablation experiment
[0048] It should be noted that YOLOv5m is the basic YOLOv5 network model; YOLOv5m-long is the model after changing the C3GRG module and C3RG module in the pre-built road defect detection model to the C3 module; YOLOv5m-longRG is YOLOv5m-long improved with the RepDW-Ghost convolution module, that is, the model after changing the C3GRG module in the pre-built road defect detection model to the C3RG module; YOLOv5m-longGRG is the pre-built road defect detection model.
[0049] As can be seen from Table 2 above, in the YOLOv5m-longGRG, that is, the pre-built road defect detection model, an extended network structure is formed by adding a downsampling convolutional module and a feature extraction module to effectively improve the accuracy of road defect detection; by introducing RGConv, the number of parameters and the computational complexity can be significantly reduced without affecting the accuracy of object detection, which has great practical value for terminal devices that cannot use GPUs; secondly, by introducing GRGConv in the added feature extraction module, that is, introducing Gaussian blur in the feature extraction stage, the accuracy of object detection can be further improved.
[0050] Example 2 As shown in the appendix Figure 8 As shown in the figure, this Example 2 provides a lightweight road defect detection system, including an image acquisition module and a defect detection module; the image acquisition module is used to acquire the original image of the road to be detected; the defect detection module is used to input the original image of the road to be detected into the pre-built road defect detection model and output the defect detection result of the road to be detected; wherein, the pre-built road defect detection model is obtained by improving the basic YOLOv5 network model.
[0051] In this Example 2, the improvement of the basic YOLOv5 network model includes: adding a downsampling convolutional module and a feature extraction module to the basic YOLOv5 network model, and introducing RGConv and GRGConv; wherein, the GRGConv is an RGConv introduced with Gaussian blur.
[0052] Example 3 As shown in the appendix Figure 9 As shown in the figure, this Example 3 provides a lightweight road defect detection device, including: a memory for storing a computer program; a processor for implementing the steps of the lightweight road defect detection method when executing the computer program.
[0053] Acquire the original image of the road to be detected; input the original image of the road to be detected into the pre-built road defect detection model and output the defect detection result of the road to be detected; wherein, the pre-built road defect detection model is obtained by improving the basic YOLOv5 network model.
[0054] Alternatively, when the processor executes the computer program, it implements the functions of each module in the above-mentioned lightweight road defect detection system, for example: An image acquisition module for acquiring the original image of the road to be detected; a defect detection module for inputting the original image of the road to be detected into a pre-constructed road defect detection model and outputting the defect detection result of the road to be detected; wherein, the pre-constructed road defect detection model is obtained by improving the basic YOLOv5 network model.
[0055] Exemplarily, the computer program can be divided into one or more modules / units, and the one or more modules / units are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of completing preset functions, and the instruction segments are used to describe the execution process of the computer program in the lightweight road defect detection device.
[0056] The lightweight road defect detection device can be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The lightweight road defect detection device may include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the above are examples of the lightweight road defect detection device, and do not constitute a limitation on the lightweight road defect detection device. It may include more components than the above, or combine some components, or different components. For example, the lightweight road defect detection device may further include input / output devices, network access devices, buses, etc.
[0057] The so-called processor may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or the processor may also be any conventional processor, etc. The processor is the control center of the lightweight road defect detection device, and connects various parts of the entire lightweight road defect detection device through various interfaces and lines.
[0058] The memory can be used to store the computer program and / or module. The processor realizes various functions of the lightweight road defect detection device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory.
[0059] The memory may mainly include a program storage area and a data storage area. Among them, the program storage area may store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area may store data created according to the use of the mobile phone (such as audio data, phone book, etc.). In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0060] Embodiment 4 Embodiment 4 of the present invention also provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps of the lightweight road defect detection method are implemented.
[0061] If the modules / units integrated in the lightweight road defect detection system are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.
[0062] Based on such an understanding, all or part of the processes in the lightweight road defect detection method of the present invention can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the lightweight road defect detection method can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or a preset intermediate form, etc.
[0063] The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disc, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.
[0064] It should be noted that the content included in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.
[0065] Embodiment 5 Embodiment 5 provides a computer product. The computer program product includes a computer program which is stored in a computer-readable storage medium. The processor of the lightweight road defect detection device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the lightweight road defect detection device can execute the lightweight road defect detection method described in Embodiment 1, which will not be elaborated here.
[0066] It should be noted that those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods.
[0067] The above embodiments are only one of the implementation manners that can implement the technical solution of the present invention. The scope of protection required by the present invention is not only limited by this embodiment, but also includes any changes, substitutions and other implementation manners that are easily conceivable by those skilled in the art within the technical scope disclosed by the present invention.
Claims
1. A lightweight road defect detection method, characterized in that: include: Obtaining the original image of the road to be detected; Inputting the original image of the road to be detected into a pre-built road defect detection model, and outputting a defect detection result of the road to be detected; wherein the pre-built road defect detection model is obtained by improving the basic YOLOv5 network model; Improvements to the basic YOLOv5 network model include: A downsampling convolution module and a feature extraction module are added to the basic YOLOv5 network model, and RGConv and GRGConv are introduced; wherein the GRGConv is RGConv with Gaussian blur introduced.
2. A lightweight road defect detection method according to claim 1, characterized in that: The RGConv is a Ghost convolution module that introduces the RepDW convolution; wherein the RepDW convolution includes two depth separation convolution modules with kernels of different sizes, and the RepDW is used to replace the depth separation convolution module in the Ghost convolution module.
3. A lightweight road defect detection method according to claim 2, characterized in that: In the GRGConv, the introduced Gaussian blur is used to perform a Gaussian blur operation on the feature map before the feature map is input into the RepDW convolution.
4. A lightweight road defect detection method according to claim 1, characterized in that: The process of adding downsampling convolution modules and feature extraction modules to the basic YOLOv5 network model and introducing RGConv and GRGConv includes: After the first layer of downsampling convolution module in the backbone network of the basic YOLOv5 network model, a downsampling convolution module and a feature extraction module are added in sequence; wherein the added downsampling convolution module is implemented by the CBS module, and the added feature extraction module is implemented by the C3GRG module; the C3GRG module is obtained by modifying the bottleneck of the C3 module in the basic YOLOv5 network model by using GRGConv.
5. A lightweight road defect detection method according to claim 4, characterized in that: The process of adding downsampling convolution modules and feature extraction modules to the basic YOLOv5 network model and introducing RGConv and GRGConv also includes: In the backbone network of the basic YOLOv5 network model, the last two layers of C3 modules are replaced by C3RG modules; In the neck network of the basic YOLOv5 network model, the last two layers of C3 modules are replaced with C3RG modules; The C3RG module is obtained by modifying the bottleneck of the C3 module in the basic YOLOv5 network model using RGConv.
6. A lightweight road defect detection method according to claim 5, characterized in that: The C3GRG module is used to extract features from the input feature map through GRGConv, perform convolution processing with a kernel size of 1, and add the feature map to the input feature map; the C3RG module is used to extract features from the input feature map through RGConv, perform convolution processing with a kernel size of 1, and add the feature map to the input feature map.
7. A lightweight road defect detection system, characterized in that: include: An image acquisition module is used to acquire the original image of the road to be detected; A defect detection module, used for inputting the original image of the road to be detected into a pre-built road defect detection model, and outputting a defect detection result of the road to be detected; wherein the pre-built road defect detection model is obtained by improving the basic YOLOv5 network model; Improvements to the basic YOLOv5 network model include: A downsampling convolution module and a feature extraction module are added to the basic YOLOv5 network model, and RGConv and GRGConv are introduced; wherein the GRGConv is RGConv with Gaussian blur introduced.
8. A lightweight road defect detection device, characterized in that: include: a processor suitable for executing a computer program; A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the lightweight road defect detection method according to any one of claims 1 to 6 is executed.
9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the lightweight road defect detection method as described in any one of claims 1 to 6 is implemented.
10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the lightweight road defect detection method according to any one of claims 1 to 6 is implemented.