Road hidden disease detection method and device, storage medium and program product

Abstract

The invention relates to road engineering, geophysical exploration and computer science and technology, and particularly provides a road hidden disease detection method and device, a storage medium and a program product. According to the method, a virtual image of an actually collected road hidden disease image is obtained through a diffusion model, a mixed data set is generated by using the virtual image and the road hidden disease image, then an optimized YOLO model is trained, verified and tested on the mixed data set, and then an optimal model and an optimal data set are obtained. And the system serves the inspection work of actual road hidden diseases. The hybrid data set on which the YOLO model depends is not completely the actually acquired real image, the hybrid data set can be obtained only by acquiring the small sample real image, and by adopting the scheme of the embodiment of the invention, the manpower and material resource consumption in the real image acquisition process can be greatly reduced, and the serious dependence of the YOLO on the data volume of the road hidden disease can be met.

Description

Technical Field

[0001] The present disclosure relates to road engineering, geophysical exploration and computer science and technology, and in particular to a method, device, storage medium and program product for detecting hidden road defects. Background Art

[0002] Under the coupling of factors such as traffic load and complex environment, road structures are prone to hidden defects such as cracks and voids, which seriously affect the stability of road structures and cause traffic safety hazards. At present, some non-destructive testing equipment has been applied to the distress screening process. Among them, ground penetrating radar is an effective and user-friendly technology. Compared with traditional destructive testing solutions, it has the characteristics of almost no impact on road structures, high detection efficiency and low labor costs. Ground penetrating radar has a wide range of applications in the operation and maintenance of road infrastructure. It extracts the characteristics and specific locations of defects in hidden road structures based on the propagation laws of emitted and received electromagnetic waves in different media, and presents the original data in the form of two-dimensional images.

[0003] In recent years, the further development of digital signal and image processing technology has made it possible to detect objects in images. In particular, in the field of deep learning without the need for manual extraction of target features, various convolutional neural network target detection models represented by YOLOv8 have made rapid progress, and their combined application with vehicle-mounted ground-penetrating radar has gradually realized the automated detection of hidden road hazards.

[0004] However, the practical engineering application of ground penetrating radar still faces the following severe challenges:

[0005] The training process of a series of target detection models represented by YOLOv8 heavily relies on a large amount of high-quality data, while the collection and verification of ground penetrating radar field data is still labor-intensive and time-consuming, with unsatisfactory economic cost and engineering efficiency. Summary of the invention

[0006] The present disclosure is proposed in view of the above problems. The present disclosure provides a method, device, storage medium and program product for detecting hidden road defects.

[0007] According to a first aspect of the present disclosure, a method for detecting hidden road hazards is provided, comprising:

[0008] Acquire a first number of road hidden disease images collected in a target area by an image acquisition device;

[0009] Generate a virtual image of each of the road hidden disease images through a diffusion model, wherein the label of the virtual image is the same as the label of each of the road hidden disease images;

[0010] Based on the virtual image, the road hidden disease image, the label of the virtual image and the label of the road hidden disease image, a second number of mixed data sets are generated; each of the mixed data sets includes a training set, a validation set and a test set, the training set includes images obtained from the road hidden disease image and images obtained from the virtual image, and the images in the validation set and the test set are both obtained from the road hidden disease image;

[0011] In the process of processing each of the mixed data sets using the pre-constructed lightweight YOLO model, obtaining performance data of the lightweight YOLO model on the test set included in each of the mixed data sets;

[0012] Based on the performance data, obtaining an optimal mixed data set from the second number of mixed data sets;

[0013] The lightweight YOLO model and the optimal mixed data set are used to perform disease detection on the roads in the area to be detected.

[0014] In addition, according to the road hidden disease detection method of the first aspect of the present disclosure, a virtual image of each of the road hidden disease images is generated by a diffusion model, including:

[0015] A preset implicit diffusion method is used to diffuse each of the road hidden disease images in latent space to obtain a virtual image of each of the road hidden disease images.

[0016] In addition, according to the method for detecting hidden road damages of the first aspect of the present disclosure, a second number of mixed data sets are generated based on the virtual image, the hidden road damage image, the label of the virtual image and the label of the hidden road damage image, including:

[0017] Dividing the road hidden disease images into the second number of image sets, each image set includes a first training image, a verification image and a test image with a label, the sum of the first training images, the verification images and the test images is the first number, the number of the verification images is the same as the number of the test images, and the number of the first training images, the verification images or the test images in different groups of image sets is different;

[0018] According to a preset mixed image ratio, based on the respective numbers of first training images, verification images, and test images in each of the image sets, a target number of second training images in each of the image sets is calculated;

[0019] The second number of mixed data sets is obtained based on each of the image sets and the target number of second training images obtained from the virtual images with labels.

[0020] In addition, according to the method for detecting hidden road damages in the first aspect of the present disclosure, before obtaining the second number of mixed data sets based on each of the image sets and the target number of second training images obtained from the virtual image with labels, the method further includes:

[0021] The pixel sizes of the first training image and the second training image in each of the image sets are cut to a preset size.

[0022] In addition, according to the method for detecting hidden road damage in the first aspect of the present disclosure, before acquiring a first number of hidden road damage images acquired by the image acquisition device, the method further includes:

[0023] The equipment parameters of the image acquisition equipment are adjusted so that the road hidden disease image meets the requirements of zero point correction, automatic gain and background elimination.

[0024] In addition, according to the road hidden disease detection method of the first aspect of the present disclosure, based on the performance data, obtaining the optimal mixed data set from the second number of mixed data sets includes:

[0025] Based on the average accuracy included in the performance data of the lightweight YOLO model on each mixed data set in the second number of mixed data sets, an average accuracy curve is generated; and based on the total time cost included in the performance data of the lightweight YOLO model on each of the mixed data sets, a total time cost curve is generated; wherein the abscissas of the average accuracy curve and the total time cost curve are both the ratios between the road hidden disease images and the virtual images in the training set included in the mixed data set;

[0026] Obtaining from the average accuracy curve all horizontal coordinate values ​​that make the average accuracy of the average accuracy curve greater than a preset accuracy threshold;

[0027] Obtaining a target horizontal coordinate value from all horizontal coordinate values ​​that makes the total time cost of the total time cost curve less than or equal to a preset time;

[0028] The mixed data set corresponding to the target horizontal coordinate value is used as the optimal mixed data set.

[0029] According to a second aspect of the present disclosure, a device for detecting hidden road hazards is provided, comprising:

[0030] A first acquisition module is used to acquire a first number of road hidden disease images acquired in a target area by an image acquisition device;

[0031] A first acquisition module is used to generate a virtual image of each of the road hidden disease images through a diffusion model, wherein the label of the virtual image is the same as the label of each of the road hidden disease images;

[0032] A generating module is used to generate a second number of mixed data sets based on the virtual image, the road hidden disease image, the label of the virtual image and the label of the road hidden disease image; each of the mixed data sets includes a training set, a validation set and a test set, the training set includes images obtained from the road hidden disease image and images obtained from the virtual image, and the images in the validation set and the test set are both obtained from the road hidden disease image;

[0033] A second acquisition module is used to obtain performance data of the lightweight YOLO model on a test set included in each of the mixed data sets in a process of processing each of the mixed data sets using a pre-constructed lightweight YOLO model;

[0034] A second acquisition module, configured to acquire an optimal mixed data set from the second number of mixed data sets based on the performance data;

[0035] The detection module is used to perform disease detection on roads in a to-be-detected area using the lightweight YOLO model and the optimal mixed data set.

[0036] According to a third aspect of the present disclosure, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the first aspect.

[0037] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program / instruction is stored, and when the computer program / instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0038] According to a fifth aspect of the present disclosure, there is provided a computer program product, comprising a computer program / instruction, which implements the steps of the method described in the first aspect when executed by a processor.

[0039] As will be described in detail below, according to the method for detecting hidden road defects in the embodiment of the present disclosure, a virtual image of the actually collected hidden road defect image can be obtained through the diffusion principle, and a mixed data set is generated using the virtual image and the hidden road defect image, and the mixed data set is processed by the YOLO model. The images in the mixed data set processed by the YOLO model are not completely real images actually collected. Only part of the real images need to be collected to obtain the mixed data set processed by the YOLO model. Compared with the images in the data set processed by the YOLO model in the prior art that are completely real images collected, the solution of the embodiment of the present application can greatly reduce the manpower and material resources consumed in the real image collection process.

[0040] It is to be understood that both the foregoing general description and the following detailed description are exemplary, and are intended to provide further explanation of the technology as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The above and other purposes, features and advantages of the present disclosure will become more apparent by describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0042] Figure 1 is a flowchart illustrating the application of the method for detecting hidden road hazards according to an embodiment of the present disclosure.

[0043] Figure 2 is a schematic diagram illustrating application of zero point correction, background removal, and automatic gain according to an embodiment of the present disclosure.

[0044] Figure 3 It is a diagram comparing the effects of applying a virtual image and a road hidden disease image according to an embodiment of the present disclosure.

[0045] Figure 4 is a schematic diagram illustrating the application of the C2fGhost module according to an embodiment of the present disclosure.

[0046] Figure 5 is a network structure diagram illustrating the application of a lightweight YOLO model with a P6 layer added according to an embodiment of the present disclosure.

[0047] Figure 6 is a schematic diagram illustrating the performance of the YOLO model according to an embodiment of the present disclosure on a test set.

[0048] Figure 7 It is a structural diagram illustrating the application of a device for detecting hidden road defects according to an embodiment of the present disclosure.

[0049] Figure 8 is a hardware block diagram illustrating an electronic device according to an embodiment of the present disclosure.

[0050] Fig. 9 is a schematic diagram illustrating a computer-readable storage medium according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0052] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0053] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.

[0054] In the related technologies, the model training process in the scheme of combining digital signal and image processing technology with vehicle-mounted ground penetrating radar for hidden road disease detection relies heavily on a large amount of high-quality data, while the collection and verification of ground penetrating radar field data is still labor-intensive and time-consuming, and its economic cost and engineering efficiency are not satisfactory.

[0055] In order to alleviate the technical problems existing in the related art, the embodiments of the present disclosure provide a method, device, storage medium and program product for detecting hidden road diseases. The method can obtain a virtual image of the actually collected hidden road disease image through the diffusion principle, and generate a mixed data set using the virtual image and the hidden road disease image, and the mixed data set is processed by the YOLO model. The images in the mixed data set processed by the YOLO model are not completely the real images actually collected. Only part of the real images need to be collected to obtain the mixed data set processed by the YOLO model. Compared with the images in the data set processed by the YOLO model in the prior art that are completely real collected images, the solution of the embodiment of the present application can greatly reduce the manpower and material resources consumed in the real image collection process.

[0056] To facilitate understanding of this embodiment, a road hidden disease detection method disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the road hidden disease detection method provided in the embodiment of the present disclosure is generally an electronic device with certain computing capabilities, and the electronic device includes, for example, a terminal device or a server or other processing device. In some possible implementations, the road hidden disease detection method can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0057] See also Figure 1 FIG. 1 is a flow chart of a method for detecting hidden road hazards provided by an embodiment of the present disclosure, and the method comprises the following steps:

[0058] Step 101: Acquire a first number of road hidden disease images collected in a target area by an image acquisition device.

[0059] In this embodiment, the image acquisition device includes but is not limited to a ground penetrating radar device. In order to meet the requirements of data balance and diversity as much as possible, the first number of road hidden disease images may include road hidden disease images of multiple disease types, where the disease types include but are not limited to cracks and holes. The first number can be set artificially based on experience or according to actual needs. For example, the first number is set to 500 images, and these 500 images include 250 images of holes and 250 images of cracks.

[0060] In actual applications, when the image acquisition device acquires images of hidden road diseases in the target area, the image acquisition device first acquires images of suspected diseases, and then professionals use endoscopes to verify whether there are diseases inside the road. If it is confirmed that there are diseases, the image is used as an image of hidden road diseases.

[0061] In the application, professionals can also cross-validate the screened and collected images to retain images that have been cleaned of disease features.

[0062] In some embodiments, in order to improve the precision and accuracy of images captured by the image acquisition device, before obtaining a first number of road hidden disease images captured by the image acquisition device, the device parameters of the image acquisition device can also be adjusted so that the road hidden disease images meet the zero point correction requirements, automatic gain requirements and background elimination requirements.

[0063] Among them, zero-point correction refers to determining the ground reflection position and eliminating the error caused by the air layer between the antenna of the ground-penetrating radar equipment and the ground, so that the starting positions of multiple reflection waves are unified to the same zero point. Automatic gain refers to amplifying the amplitude of the reflected signal deep in the road structure layer to compensate for the energy attenuation during the propagation of electromagnetic waves and enhance the disease characteristics of deep targets. Background elimination refers to calculating the mean amplitude of all single-channel signal waves, subtracting this mean from the original signal amplitude, eliminating various influences such as direct waves, background noise, DC bias, etc., and improving the signal-to-noise ratio of the target area. For ease of understanding, the following are given. Figure 2 Schematic diagram of zero point correction, background removal and automatic gain shown.

[0064] Step 102: Generate a virtual image of each road hidden disease image through a diffusion model, and the label of the virtual image is the same as the label of each road hidden disease image.

[0065] In this embodiment, the labels of the road hidden disease images can be logged in to a specific website by professionals, and the diseases in the road hidden disease images can be annotated under cross-validation by no less than two people. The cross-validation process requires that the error rate does not exceed 5%.

[0066] In this embodiment, the number of virtual images of each road hidden disease image may be one or more, for example, two.

[0067] In some optional embodiments, generating a virtual image of each road hidden disease image by using a diffusion model may include:

[0068] A preset implicit diffusion method is used to diffuse each road hidden disease image in the latent space to obtain a virtual image of each road hidden disease image.

[0069] In order to improve the efficiency of virtual data (i.e., virtual image) generation, a stable diffusion model is used. Although the stable diffusion model supports both text-image and image-image generation functions, in order to limit the input of subjective information, this embodiment only deploys the img2img module of the model, i.e., the image-image generation module.

[0070] Specifically, the stable diffusion model adopts the implicit diffusion principle and latent space acceleration method:

[0071] The implicit diffusion process is divided into forward and reverse diffusion. Forward diffusion retains part of the information of the input real image, and on this basis, it is first denoised to a completely noisy image. Afterwards, according to the Markov chain principle, reverse diffusion denoising is performed to obtain the generated image. The generated image can retain part of the original image information and produce appropriate texture changes, thereby enhancing data diversity while doubling the amount of data. Finally, the generated images are sequentially numbered to form a generated image database. In this process, the hyperparameter denoising strength (strength) can determine the original image information retained by the generated image to the greatest extent, and the range is controlled within [0-1]. 0 means retaining all the original image information, and 1 means losing all the original image information. After a large number of experiments, in this disclosure, the dynamic adjustment range of this parameter shall not exceed [0.2-0.4], and 0.23 is recommended as the control parameter. Technical personnel can fine-tune it according to the self-built real data set, and ensure that the similarity between the generated image and the real image is achieved during fine-tuning. Figure 3 In addition, for each input image, multiple images can be generated by repeated diffusion, that is, the number of generated virtual images is theoretically unlimited. However, in order to ensure the robustness of the mixed data set, it is recommended to generate only one corresponding virtual image for each input image, and consider generating two if the amount of real data is insufficient.

[0072] Latent space acceleration is to introduce the process of generating images by forward and reverse diffusion into the latent space. Specifically, the input image is first vectorized, the image size is compressed in the latent space, and then a stable diffusion process is performed. Finally, the latent space vector is inversely transformed to output a two-dimensional generated image. This process can increase the speed of the diffusion process by at least ten times. In addition, the hyperparameter batch size has a slight effect on the speed of diffusion, and it is recommended to set it to 8. After testing, the speed of generating a single image under the server configuration and environment shown in Table 1 is about 1.72 seconds, which meets the actual engineering efficiency requirements.

[0073] Please refer to Figure 3 , Figure 3 This is a comparison diagram of the effects of the virtual image and the road hidden disease image given in this embodiment.

[0074] Step 103: Generate a second number of mixed data sets based on the virtual images, the road hidden disease images, the labels of the virtual images and the labels of the road hidden disease images; each mixed data set includes a training set, a validation set and a test set, the training set includes images obtained from the road hidden disease images and images obtained from the virtual images, and the images in the validation set and the test set are both obtained from the road hidden disease images.

[0075] In some embodiments, Python's standard library random can be used to write a dynamic random extraction algorithm for small sample situations, and the dynamic random extraction algorithm is used to generate a mixed data set. The mixed data set generated by the dynamic random extraction algorithm can achieve the following goals:

[0076] Limit the size of the training, validation, and test sets of the mixed dataset to 8:1:1, and allow for fine-tuning based on actual engineering situations.

[0077] Limit the ratio of crack and cavity disease data in the training, validation and test sets in the mixed dataset and strictly control it to 1:1 to meet the data balance requirements;

[0078] Restrict the data types in the validation and test sets (only real data) to ensure that only real data is used to verify and tune the detection effect during model training;

[0079] The ratio of real and generated images in the training set is dynamically adjusted, gradually increasing from 1:0 to 1:5 (as the total size of the training set increases), and then 11 mixed data sets are established.

[0080] The total amount of real data in the mixed data set is limited (about 500 images) to verify the small sample situation.

[0081] It should be understood that the first number of images of hidden road damage are acquired in real time by the image acquisition device, so the first number of images of hidden road damage are the real data in this embodiment, and the virtual images are images virtualized by the diffusion model, so the virtual images are not real data.

[0082] In order to achieve the above objectives, in some embodiments, step 103 may specifically include the following steps:

[0083] Dividing the road hidden disease images into a second number of image sets, each image set includes a first training image, a verification image, and a test image with a label, the sum of the number of the first training images, the verification images, and the test images is the first number, the number of the verification images is the same as the number of the test images, and the number of the first training images, the verification images, or the test images in different groups of image sets is different;

[0084] According to a preset mixed image ratio, based on the respective numbers of first training images, verification images, and test images in each image set, a target number of second training images in each image set is calculated;

[0085] A second number of mixed data sets are obtained based on each image set and a target number of second training images obtained from the virtual images with labels.

[0086] In this embodiment, the second number can be set artificially based on experience or according to actual needs. For example, the second number is set to 11.

[0087] As an example, taking the first number of 500 road hidden disease images as an example, 11 mixed data sets are finally generated as shown in Table 1:

[0088] Table 1

[0089]

[0090] In some embodiments, in order to improve the processing speed of the subsequent YOLO model, before obtaining the second number of mixed data sets based on each image set and the target number of second training images obtained from the virtual image, the pixel sizes of the first training images and the second training images belonging to each image set can also be cropped to a preset size.

[0091] The preset size here can be set manually based on experience or actual needs, such as setting the preset size to 1180×1180 or 1848×1848.

[0092] Step 104: In the process of using the pre-constructed lightweight YOLO model to process each mixed data set, performance data of the lightweight YOLO model on the test set included in each mixed data set is obtained.

[0093] The C2f module in the lightweight YOLO model in this embodiment is replaced with a C2fGhost module with a lower number of model parameters. Figure 4 , Figure 4 The figure is a schematic diagram of the C2fGhost module. The working principle of the C2fGhost module is as follows:

[0094] The features of the previous layer output of the C2fGhost module are input into the C2fGhost module as input features. After a conventional convolution (GhostConv) process, the convolved feature map is split into two branches, one of which passes through two GhostConv modules in succession for feature extraction. The feature map obtained after passing through the two GhostConv modules and the feature map that has not passed through this module are spliced ​​together, and finally the final feature map is output after conventional convolution. The GhostConv module can greatly reduce the total number of model parameters and reduce computational complexity.

[0095] In this embodiment, a P6 detection layer with a lower output feature map resolution is added to the original detection layer of the YOLO model. The improved model includes P3, P4, P5 and P6 detection layers, which can better identify multi-scale features of images. Specifically, the shallow network detection layer (P3, P4, P5) mainly identifies image detail features, which is beneficial for small target detection. The deep network detection layer (P6) is located at a deeper position in the YOLO network structure, which is more conducive to identifying large target diseases. In addition, the P6 output feature map has a lower resolution, which can save computing power and increase the model training speed. The corresponding improved lightweight YOLO model network structure is added as follows: Figure 5 shown.

[0096] In order to verify the performance of the YOLO model in this embodiment, in some embodiments, the following comparative verification implementation scheme is also given.

[0097] According to YOLO's performance indicators average precision (map) and total time cost (TC), the relevant indicators are defined as follows:

[0098] "Positive" and "Negative" are the predicted sample labels, where "Positive" is the disease of the image to be detected (positive sample), and "Negative" is the background of the image to be detected (negative sample). "True" and "False" represent the prediction results, where "True" means the prediction is correct and "False" means the prediction is wrong. Therefore, FP means that the negative sample is wrongly predicted as a positive sample, FN means that the positive sample is wrongly predicted as a negative sample, TN means that the negative sample is correctly predicted as a negative sample, and TP means the correct prediction classification.

[0099] Precision, also known as accuracy, evaluates whether disease detection is accurate, that is, the proportion of actual positive samples among all data predicted to be positive samples.

[0100]

[0101] The recall rate, also known as the recall rate, evaluates whether the disease detection is comprehensive, that is, the proportion of data that are correctly predicted as positive samples among all data that are actually positive samples.

[0102]

[0103] Average precision map, precision and recall are usually not optimized at the same time, resulting in the neglect of comprehensive consideration of the results. Therefore, AP is introduced i For a disease type, the Precision-Recall (PR) curve is obtained with Precision as the vertical axis and Recall as the horizontal axis. The area under the PR curve represents the AP iFurthermore, map represents the average AP of all classes in the entire test set. i .

[0104]

[0105] Where i represents the category (crack or cavity); N i Represents the total number of a certain type of disease in the test set; k is the subscript symbol to distinguish a certain type of disease.

[0106] In addition, the present disclosure defines the total time cost (TC) indicator to quantitatively describe the training cost, and its calculation method is as follows:

[0107] TC=TT+TG (4)

[0108] Among them, TT represents the training time of improving the YOLO model during training; TG represents the time required to complete data augmentation, which is the product of the total number of generated images and the efficiency of single image generation (1.72 seconds / image).

[0109] The improved and baseline models are trained, verified, and tested on a multi-gradient mixed dataset using the control variable method. The optimal lightweight model is obtained by combining their performance indicators on 11 test sets. The results are shown in Table 2:

[0110] Table 2

[0111]

[0112]

[0113] Combined with Table 2, after introducing the C2fGhost module and the P6 detection layer, the improved YOLO model achieves the highest recognition accuracy and the fastest training speed. Compared with the baseline model, the average precision map is improved by 1.54% on average, while the total time cost TC is reduced by 7.2% on average. YOLOv8-Ghost-P6 is determined to be the optimal lightweight model.

[0114] Step 105: Based on the performance data, obtain an optimal mixed data set from the second number of mixed data sets.

[0115] In some embodiments, step 105 may include the following steps:

[0116] Based on the average accuracy included in the performance data of the lightweight YOLO model on each mixed data set in the second number of mixed data sets, an average accuracy curve is generated; and based on the total time cost included in the performance data of the lightweight YOLO model on each mixed data set, a total time cost curve is generated; wherein the abscissas of the average accuracy curve and the total time cost curve are both the ratios between the road hidden disease images in the training set included in the mixed data set and the virtual images;

[0117] Obtaining from the average accuracy curve all horizontal coordinate values ​​that make the average accuracy of the average accuracy curve greater than a preset accuracy threshold;

[0118] Obtaining a target horizontal coordinate value from all horizontal coordinate values ​​that makes the total time cost of the total time cost curve less than or equal to the preset time;

[0119] The mixed data set corresponding to the target horizontal coordinate value is taken as the optimal mixed data set.

[0120] In this embodiment, the accuracy threshold and the preset time can be preset manually according to one's own needs and actual requirements. For example, the accuracy threshold can be set to 90%, and the preset time can be set to 20 minutes.

[0121] See also Figure 6 , Figure 6 Taking the second number of mixed data sets as an example, the YOLO model is trained, verified and tested using the 11 multi-gradient mixed data sets shown in Table 1, and its performance on the test set (average precision (map) and total time cost (TC)).

[0122] analyze Figure 6 The data on the left shows that as the proportion of generated data in the training set increases and the total size of the training set continues to increase, the map first rises rapidly and then slowly to nearly 0.99. When the ratio of generated data to real data reaches 5:1, compared with the original small sample data, the mixed data set brings a significant improvement of nearly 0.13, indicating that the method of using the stable diffusion model to generate data to overcome the small sample problem in actual engineering has achieved remarkable results.

[0123] analyze Figure 6 In the data on the right, as the proportion of generated data in the training set increases and the total size of the training set continues to increase, the total time cost TC gradually increases, and the time cost TC required to run the stable diffusion model gradually dominates. When the ratio of generated data to real data reaches 5:1, TC reaches nearly 5 times that of the original small sample conditions.

[0124] Based on the above analysis, in order to balance the recognition accuracy and total time cost of diseases in small sample conditions, it is recommended that the optimal mixed dataset should meet the following requirements:

[0125] The ratio of generated data to real data in the training set should be controlled at about 1:1 to 3:1. At this time, the small sample ground penetrating radar underground disease target detection system based on stable diffusion and lightweight YOLO can meet the accuracy and efficiency requirements of engineering detection. Figure 6 It can be found that when the ratio of generated data to real data in the training set is controlled at about 1:1 to 3:1, the average accuracy map is greater than 90% and the total time cost TC is greater than 20 minutes.

[0126] Step 106: Use the YOLO model and the optimal mixed data set to perform disease detection on the roads in the area to be detected.

[0127] When performing disease detection on roads in the area to be detected, images of hidden road diseases can be collected from the area to be detected according to the data ratio of the training set, test set and validation set in the optimal mixed data set, and the images of hidden road diseases can be diffused to form a mixed data set for processing the YOLO model.

[0128] In the solution provided in this embodiment, a virtual image of the actually collected road hidden disease image can be obtained through the diffusion principle, and a mixed data set is generated using the virtual image and the road hidden disease image, and the mixed data set is processed by the YOLO model. The images in the mixed data set processed by the YOLO model are not completely real images actually collected. Only part of the real images need to be collected to obtain the mixed data set processed by the YOLO model. Compared with the images in the data set processed by the YOLO model in the prior art that are completely real images collected, the solution of the embodiment of the present application can greatly reduce the manpower and material resources consumed in the real image collection process.

[0129] On this basis, this embodiment also has the following advantages:

[0130] The real collected road hidden disease images are preprocessed through zero point correction, automatic gain and background elimination to enhance the disease features, eliminate errors and signal interference, and provide a high-quality data foundation for subsequent image generation and improvement of the YOLO model training process.

[0131] The present invention discloses a technical method for data amplification using a stable diffusion model, which makes up for the time-consuming and labor-intensive defects of collecting and verifying original real disease data of vehicle-mounted ground penetrating radar. Only a small sample of real disease data is needed to establish a large-volume mixed data set, thus overcoming the small sample problem and improving the robustness and stability of the model.

[0132] The present invention makes lightweight improvements to the YOLO model, introduces the C2fGhost module to replace the C2f module to reduce the number of model parameters; adds the P6 detection layer to the original detection layer of the YOLO model, expands the receptive field of the model, improves the network's perception of global features, reduces the resolution of the output feature map, and reduces the time required for training the model, providing a new model for its rapid deployment and application in actual projects.

[0133] This paper introduces the average precision (MAP) and total time cost (TC) performance indicators to achieve a balance between detection accuracy and efficiency, and then establishes a complete optimal lightweight model and hybrid data set screening system.

[0134] The present disclosure also provides a road hidden disease detection device, which is used to execute the road hidden disease detection method provided in any of the above embodiments. Figure 7 As shown, the device comprises:

[0135] A first acquisition module 71 is used to acquire a first number of road hidden disease images acquired in a target area by an image acquisition device;

[0136] A first obtaining module 72 is used to generate a virtual image of each of the road hidden disease images through a diffusion model, wherein the label of the virtual image is the same as the label of each of the road hidden disease images;

[0137] A generating module 73 is used to generate a second number of mixed data sets based on the virtual image, the road hidden disease image, the label of the virtual image and the label of the road hidden disease image; each of the mixed data sets includes a training set, a validation set and a test set, the training set includes images obtained from the road hidden disease image and images obtained from the virtual image, and the images in the validation set and the test set are both obtained from the road hidden disease image;

[0138] A second obtaining module 74 is used to obtain performance data of the lightweight YOLO model on the test set included in each of the mixed data sets in the process of processing each of the mixed data sets using the pre-constructed lightweight YOLO model;

[0139] A second acquisition module 75, configured to acquire an optimal mixed data set from the second number of mixed data sets based on the performance data;

[0140] The detection module 76 is used to perform disease detection on the roads in the area to be detected using the lightweight YOLO model and the optimal mixed data set.

[0141] In some embodiments, the first obtaining module 72 is used to:

[0142] A preset implicit diffusion method is used to diffuse each of the road hidden disease images in latent space to obtain a virtual image of each of the road hidden disease images.

[0143] In some embodiments, the generation module 73 is used to:

[0144] Dividing the road hidden disease images into the second number of image sets, each image set includes a first training image, a verification image and a test image with a label, the sum of the first training images, the verification images and the test images is the first number, the number of the verification images is the same as the number of the test images, and the number of the first training images, the verification images or the test images in different groups of image sets is different;

[0145] According to a preset mixed image ratio, based on the respective numbers of first training images, verification images, and test images in each of the image sets, a target number of second training images in each of the image sets is calculated;

[0146] The second number of mixed data sets is obtained based on each of the image sets and the target number of second training images obtained from the virtual images with labels.

[0147] In some embodiments, the device is also used to:

[0148] Based on each of the image sets and the target number of second training images obtained from the virtual image with the label, before obtaining the second number of mixed data sets, the pixel sizes of the first training images and the second training images belonging to each of the image sets are cropped to a preset size.

[0149] In some embodiments, the device is also used to:

[0150] Before acquiring a first number of road hidden disease images acquired by an image acquisition device, the device parameters of the image acquisition device are adjusted so that the road hidden disease images meet the zero point correction requirements, automatic gain requirements and background elimination requirements.

[0151] In some embodiments, the second acquisition module 75 is used to:

[0152] Based on the average accuracy included in the performance data of the lightweight YOLO model on each mixed data set in the second number of mixed data sets, an average accuracy curve is generated; and based on the total time cost included in the performance data of the lightweight YOLO model on each of the mixed data sets, a total time cost curve is generated; wherein the abscissas of the average accuracy curve and the total time cost curve are both the ratios between the road hidden disease images and the virtual images in the training set included in the mixed data set;

[0153] Obtaining from the average accuracy curve all horizontal coordinate values ​​that make the average accuracy of the average accuracy curve greater than a preset accuracy threshold;

[0154] Obtaining a target horizontal coordinate value from all horizontal coordinate values ​​that makes the total time cost of the total time cost curve less than or equal to a preset time;

[0155] The mixed data set corresponding to the target horizontal coordinate value is used as the optimal mixed data set.

[0156] The device for detecting hidden road defects provided in the embodiment of the present disclosure and the method for detecting hidden road defects provided in the embodiment of the present disclosure are based on the same inventive concept and have the same beneficial effects as the methods adopted, operated or implemented therein.

[0157] The present disclosure also provides an electronic device to implement the above-mentioned road hidden disease detection method. Figure 8 It shows a schematic diagram of an electronic device provided by some embodiments of the present disclosure. Figure 8 As shown, the electronic device 8 includes: a processor 800, a memory 801, a bus 802 and a communication interface 803, wherein the processor 800, the communication interface 803 and the memory 801 are connected via the bus 802; the memory 801 stores a computer program that can be run on the processor 800, and when the processor 800 runs the computer program, the method for detecting hidden road hazards provided by any of the aforementioned embodiments of the present disclosure is executed.

[0158] The memory 801 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the device network element and at least one other network element is realized through at least one communication interface 803 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0159] The bus 802 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 801 is used to store programs, and the processor 800 executes the programs after receiving execution instructions. The road hidden disease detection method disclosed in any implementation of the above-mentioned embodiment of the present disclosure may be applied to the processor 800, or implemented by the processor 800.

[0160] The processor 800 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 800 or the instruction in the form of software. The above processor 800 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present disclosure can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present disclosure can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 801, and the processor 800 reads the information in the memory 801 and completes the steps of the above method in combination with its hardware.

[0161] The electronic device provided in the embodiment of the present disclosure and the method for detecting hidden road hazards provided in the embodiment of the present disclosure are based on the same inventive concept and have the same beneficial effects as the methods adopted, operated or implemented therein.

[0162] The present disclosure also provides a computer-readable storage medium corresponding to the road hidden disease detection method provided in the above embodiment. Fig. 9 The computer-readable storage medium shown is a CD 30 on which a computer program (ie, a program product) is stored. When the computer program is run by a processor, the method for detecting hidden road hazards provided by any of the aforementioned embodiments will be executed.

[0163] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0164] The computer-readable storage medium provided in the above-mentioned embodiments of the present disclosure and the method for detecting hidden road hazards provided in the embodiments of the present disclosure are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0165] It should be noted that:

[0166] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present disclosure can be practiced without these specific details. In some instances, well-known structures and technologies are not shown in detail so as not to obscure the understanding of this description.

[0167] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present disclosure, the various features of the present disclosure are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be interpreted as reflecting the following schematic diagram: the claimed disclosure requires more features than the features explicitly recited in each claim. More specifically, as reflected in the claims below, the inventive aspects lie in less than all the features of the single embodiment disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present disclosure.

[0168] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of this disclosure and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0169] The above is only a preferred specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with the technical field within the technical scope disclosed in the present disclosure should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A method for detecting hidden road hazards, characterized in that: include: Acquire a first number of road hidden disease images collected in a target area by an image acquisition device; Generate a virtual image of each of the road hidden disease images through a diffusion model, wherein the label of the virtual image is the same as the label of each of the road hidden disease images; Based on the virtual image, the road hidden disease image, the label of the virtual image and the label of the road hidden disease image, a second number of mixed data sets are generated; each of the mixed data sets includes a training set, a validation set and a test set, the training set includes images obtained from the road hidden disease image and images obtained from the virtual image, and the images in the validation set and the test set are both obtained from the road hidden disease image; In the process of processing each of the mixed data sets using the pre-constructed lightweight YOLO model, obtaining performance data of the lightweight YOLO model on the test set included in each of the mixed data sets; Based on the performance data, obtaining an optimal mixed data set from the second number of mixed data sets; The lightweight YOLO model and the optimal mixed data set are used to perform disease detection on the roads in the area to be detected.

2. The method according to claim 1, characterized in that: Generate a virtual image of each of the road hidden disease images through a diffusion model, including: A preset implicit diffusion method is used to diffuse each of the road hidden disease images in latent space to obtain a virtual image of each of the road hidden disease images.

3. The method according to claim 1, characterized in that Generating a second number of mixed data sets based on the virtual image, the road hidden disease image, the label of the virtual image and the label of the road hidden disease image, including: Dividing the road hidden disease images into the second number of image sets, each image set includes a first training image, a verification image and a test image with a label, the sum of the first training images, the verification images and the test images is the first number, the number of the verification images is the same as the number of the test images, and the number of the first training images, the verification images or the test images in different groups of image sets is different; Calculating a target number of second training images in each of the image sets based on the respective numbers of first training images, verification images, and test images in each of the image sets according to a preset mixed image ratio; The second number of mixed data sets is obtained based on each of the image sets and the target number of second training images obtained from the virtual images with labels.

4. The method according to claim 3, characterized in that Before obtaining the second number of mixed data sets based on each of the image sets and the target number of second training images obtained from the virtual image with the label, the method further includes: The pixel sizes of the first training image and the second training image in each of the image sets are cut to a preset size.

5. The method according to claim 1, characterized in that Before acquiring a first number of road hidden disease images acquired by an image acquisition device, the method further includes: The equipment parameters of the image acquisition equipment are adjusted so that the road hidden disease image meets the requirements of zero point correction, automatic gain and background elimination.

6. The method according to claim 1, characterized in that Based on the performance data, obtaining an optimal mixed data set from the second number of mixed data sets includes: Based on the average accuracy included in the performance data of the lightweight YOLO model on each mixed data set in the second number of mixed data sets, an average accuracy curve is generated; and based on the total time cost included in the performance data of the lightweight YOLO model on each of the mixed data sets, a total time cost curve is generated; wherein the abscissas of the average accuracy curve and the total time cost curve are both the ratios between the road hidden disease images and the virtual images in the training set included in the mixed data set; Obtaining from the average accuracy curve all horizontal coordinate values ​​that make the average accuracy of the average accuracy curve greater than a preset accuracy threshold; Obtaining a target horizontal coordinate value from all horizontal coordinate values ​​that makes the total time cost of the total time cost curve less than or equal to a preset time; The mixed data set corresponding to the target horizontal coordinate value is used as the optimal mixed data set.

7. A device for detecting hidden road hazards, characterized in that: include: A first acquisition module is used to acquire a first number of road hidden disease images acquired in a target area by an image acquisition device; A first acquisition module is used to generate a virtual image of each of the road hidden disease images through a diffusion model, wherein the label of the virtual image is the same as the label of each of the road hidden disease images; A generating module is used to generate a second number of mixed data sets based on the virtual image, the road hidden disease image, the label of the virtual image and the label of the road hidden disease image; each of the mixed data sets includes a training set, a validation set and a test set, the training set includes images obtained from the road hidden disease image and images obtained from the virtual image, and the images in the validation set and the test set are both obtained from the road hidden disease image; A second acquisition module is used to obtain performance data of the lightweight YOLO model on a test set included in each of the mixed data sets in a process of processing each of the mixed data sets using a pre-constructed lightweight YOLO model; A second acquisition module, configured to acquire an optimal mixed data set from the second number of mixed data sets based on the performance data; The detection module is used to perform disease detection on roads in a to-be-detected area using the lightweight YOLO model and the optimal mixed data set.

8. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images