Well lid disease detection method based on virtual sample driving

Abstract

The invention discloses a manhole cover disease detection method based on virtual sample driving, and the method comprises the steps: generating a large number of virtual samples simulating real manhole cover diseases through AIGC, i.e., a generative artificial intelligence technology, constructing a diseased manhole cover virtual data set, carrying out the training of a target detection model, and carrying out the discovery and recognition of the manhole cover diseases in a real image through employing the model; a virtual data set is constructed by using a virtual image generated by AIGC, so that the long-tail distribution phenomenon of well lid disease types is effectively inhibited, and well lid disease labels are smoothed; in combination with posterior knowledge, model optimization is guided, that is, after model training, the model is further utilized to carry out index analysis on a real image recognition result, and feature distribution in a targeted smooth virtual data set is carried out from three dimensions of a disease well lid scene easy to recognize by mistake, a disease form easy to recognize by mistake and a disease well lid type easy to recognize by mistake and omission; correcting and optimizing a model prediction result; through the method, the model can identify well lid diseases more accurately, and more reliable disease classification and statistical results are provided.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a manhole cover disease detection method based on virtual sample driving. Background Art

[0002] Manhole cover damage is a stubborn problem faced in the process of urban development, which has gradually attracted the attention of academia and industry in recent years. Using machine learning and deep learning methods instead of traditional manual interpretation to identify the type and degree of damage of manhole covers has become a research hotspot.

[0003] Existing technologies face many challenges when dealing with manhole cover disease detection tasks. First, it is difficult to obtain rare manhole cover disease samples in reality, which leads to an imbalance in the types of data sets and seriously affects the training effect of the model. Secondly, the intra-class features in the data set are significantly different, while the inter-class features are highly similar. These unfavorable factors in features further increase the difficulty of model characterization and classification. Finally, most of the existing research stays at the recognition of manhole cover objects and has not gone deep into the recognition level of manhole cover diseases. The research is not fine-grained enough and has not formed a systematic classification system for manhole cover hidden danger types. These problems have not been fully solved in the current research, and innovative solutions are urgently needed to improve the accuracy and reliability of manhole cover disease detection. Summary of the invention

[0004] In view of the above, the present invention provides a manhole cover disease detection method based on virtual sample drive, further studies the impact of disease entity characteristics, manhole cover characteristics and background characteristics on model performance, realizes low-cost and high-quality manhole cover disease data set expansion, and then alleviates the problems of sample scarcity, type imbalance, high feature similarity, etc., and improves the accuracy and generalization ability of the manhole cover disease target detection model.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A manhole cover disease detection method based on virtual sample drive includes constructing a manhole cover disease generation model, drawing a virtual image based on the manhole cover disease generation model, constructing a manhole cover disease virtual data set based on a human-computer collaborative mode, training a manhole cover target detection model driven by the manhole cover disease virtual data set, analyzing and counting indicators of the manhole cover target detection model, and adaptively generating manhole cover disease parameters to be optimized. The manhole cover disease detection method is as follows:

[0007] (1) Construct a model for generating defective manhole covers. Through manual field photography and Internet image downloading, we collect manhole cover scene images and manhole cover defect images, encode each image, and train a Stable Diffusion model using the LoRA (low-rank adaptive method) to generate virtual defective manhole cover images.

[0008] (2) Drawing a virtual image based on the defective manhole cover generation model, integrating information on the defective manhole cover scene, the defective manhole cover background, the defective manhole cover morphology, and the description of the defective manhole cover type, inputting the defective manhole cover generation model trained in step (1), and obtaining a virtual defective manhole cover image generated by the model;

[0009] (3) Construction of a virtual dataset of defective manhole covers based on the human-machine collaborative mode. Repeat step (2) multiple times to obtain a large number of virtual images containing defective manhole covers. Manually select and generate virtual images with good quality. Generate target detection frames and disease types through the image reconstruction position and prompt words generated in step (2) to obtain data annotations and construct a virtual dataset of defective manhole covers.

[0010] (4) Training of a target detection model for manhole covers driven by a virtual dataset of manhole covers. The virtual dataset of manhole covers obtained in step (3) is used as a training dataset. The YOLOv10 model is selected as the target detection model. The virtual dataset is used to train the YOLOv10 model, so that the YOLOv10 model can effectively identify various types of manhole cover defects.

[0011] (5) Analysis and statistics of indicators of the defective manhole cover target detection model. The key performance indicators of the model are recorded during the training process. The trained model is evaluated using the validation data set. The detection performance of the model on different types of defects is statistically analyzed. The detection performance includes confusion matrix, category loss, and IOU loss. If the indicators of various types of defective manhole covers reach the expected indicators, the training is stopped. If the indicators of various types of defective manhole covers do not reach the expected indicators, the training is continued.

[0012] (6) Adaptively generate the parameters of the manhole cover defects to be optimized. According to the performance evaluation results in step (5), identify the test samples with insufficient performance, extract the background, morphology, and type information of the defective manhole cover as the generation conditions, jump to step (2) to supplement the type data to be optimized, start the iterative optimization of model training, and repeat the statistics and evaluation of the target detection model indicators until the expected indicators are achieved or the stopping conditions are met, and stop the iteration to complete the training.

[0013] Furthermore, in step (1), the manhole cover defect images collected through various channels are subjected to basic classification, and the defect types include "intact", "damaged", "missing", "uncovered" and "damaged manhole ring". Each image is encoded and divided into training set, validation set and test set of the defect manhole cover target detection model according to the ratio of 8:1:1. Defective manhole covers with complex backgrounds and obvious features are selected for LoRA fine-tuning.

[0014] Furthermore, the steps of training a stable diffusion model by the LoRA method include:

[0015] A. Data preparation: Select the images with obvious disease characteristics and retain high-quality images with clear images and obvious features;

[0016] B. Text encoding: Use the pre-trained CLIP (Contrastive Language-Image) pre-trained model to generate text annotations from images, add manhole cover disease codes to the automatically generated annotations, and give the disease code label a larger weight;

[0017] C. LoRA method training: Based on a stable diffusion model pre-trained weight, the LoRA method is used for training, and the low-rank adaptive weights added to the image generator part of the model are updated to obtain a stable diffusion model that can generate manhole cover disease images;

[0018] D. Model screening: After training LoRA, you will see sample images under different epochs. Exclude underfitting and overfitting models and obtain a stable diffusion model that is most suitable for generating virtual diseased manhole cover images as the diseased manhole cover generation model.

[0019] Furthermore, in step (2), a corpus of natural image descriptions is constructed from manhole cover defects, manhole cover scenes, and manhole cover backgrounds, wherein manhole cover defects are organized into five categories: "intact", "damaged", "missing", "uncovered", and "damaged manhole ring", and manhole cover scenes are organized into six categories: "urban road scene", "urban sidewalk scene", "park scene", "urban residential area scene", "industrial area scene", and "rural homestead scene", and manhole cover backgrounds are organized into six categories: "asphalt road", "pedestrian brick road", "green lawn", "cement hardened pavement", "rubber pavement", and "marble pavement". The construction of prompt words is completed based on the classification standards of manhole cover defects, manhole cover scenes, and manhole cover backgrounds; virtual data containing manhole cover defects are generated based on the defect manhole cover generation model through the prompt word text-generated image, prompt word and image multimodal image-generated image mode.

[0020] Furthermore, in step (3), prompt words for each category of the virtual data set of diseased manhole covers are designed according to the manhole cover disease category, background, and scene. The prompt words and background images are used as input data, and step (2) is repeated multiple times to obtain a large number of virtual images containing diseased manhole covers. Virtual images with good quality are manually selected until each disease category reaches the designed threshold N. Image labels are generated by the virtual diseased manhole cover generation model based on the prompt words and disease location information to achieve image annotation, and finally a virtual data set of diseased manhole covers with accurate annotations, sufficient quantity, diverse generalization, balanced categories, and good consistency is constructed, which is completely composed of virtual images.

[0021] Furthermore, in step (4), the virtual dataset of damaged manhole covers generated in step (3) is first used as a training dataset, and then the YOLOv10 model is initialized and configured with appropriate hyperparameters for training, and the model is iteratively optimized using the virtual dataset of damaged manhole covers.

[0022] Furthermore, the hyperparameter settings of the training process are as follows: the optimizer is AdamW, the learning rate is 0.001, the learning rate decay adopts CosineAnnealingLR, the batch size is 2, the IoU threshold of positive and negative samples is set to 0.5, the NMS threshold is set to 0.4, and the number of training rounds is set to 300.

[0023] Furthermore, in step (5), the indicators of the defective manhole cover target detection model are analyzed and counted, and the model indicator analysis is carried out based on the inference results and the true value using the trained model reasoning test set;

[0024] First, we construct a multi-classification confusion matrix for defective manhole covers, calculate the true positive, false negative, false positive, and true negative classes for each defect category, and calculate the precision, recall, F1 score, and AP for each category to evaluate the performance of the object detection model on different target categories.

[0025]

[0026] Among them: i represents category 1, 2, 3, 4, 5 correspond to "intact", "damaged", "missing", "uncovered" and "well ring damaged" respectively. min When it is greater than the threshold M, the confusion matrix analysis result is true, otherwise it is false;

[0027] Secondly, the loss function of the defective manhole cover is analyzed to obtain the average loss of each object category in the test set.

[0028]

[0029] Among them: mCLoss represents the average loss of the category, t represents the number of objects in a certain category of test samples, CLoss k Represents the category cross entropy loss of the k-th object. When the smallest mCLoss of the five categories is greater than the threshold P, the result of the defective manhole cover loss function analysis is true, otherwise it is false;

[0030] Next, perform IoU analysis on the defective manhole cover targets and find the average IoU loss of each object in the test set.

[0031]

[0032] Among them: mIoULoss represents the category IoU loss, IoULoss kIt represents the intersection over union (IoU) loss of the k-th object. When the smallest IoULoss of the five categories is greater than the threshold Q, the IoU loss analysis result of the defective manhole cover is true, otherwise it is false.

[0033] Take the mode of the confusion matrix analysis, defective manhole cover loss function analysis, and defective manhole cover IoU loss analysis results. If it is true, stop training; if it is false, go to step (6).

[0034] Furthermore, in step (6), during the statistical analysis of the indicators of the defective manhole cover target detection model in step (5), the label parameters of the manhole cover defect category to be optimized are determined for the minimum AP, minimum mCLoss and minimum mIoULoss categories respectively, and sent to the defective manhole cover generation model, the virtual data set is expanded in a targeted manner, and the target detection model is iteratively optimized.

[0035] The technical effects and advantages of the present invention are as follows: the present invention uses AIGC (generative artificial intelligence) technology to generate a large number of virtual samples simulating real manhole cover diseases, constructs a virtual data set of diseased manhole covers for target detection model training, and uses the model to discover and identify manhole cover diseases in real images; fully considers the deficiencies of the target detection model in the scene of uneven distribution of manhole cover disease labels and features, uses virtual images generated by AIGC to construct virtual data sets, effectively suppresses the phenomenon of long-tail distribution of manhole cover disease types, smoothes manhole cover disease labels, and combines posterior knowledge to guide model optimization, that is, after model training, further uses the model to carry out index analysis on real image recognition results, from the three dimensions of easy-to-misidentify diseased manhole cover scenes (backgrounds), easy-to-misidentify disease morphology, and easy-to-misidentify diseased manhole cover types, the feature distribution of the virtual data set is targeted and smoothed, and the prediction results of the model are corrected and optimized. Through this method, the model can not only identify manhole cover diseases more accurately, but also provide more reliable disease classification and statistical results. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0037] Figure 1 is a flow chart of the present invention;

[0038] Figure 2 Data annotation based on diseased manhole cover scenes, prompt words and virtual images;

[0039] Figure 3Centralized image display for the virtual data set of diseased manhole covers;

[0040] Figure 4 This is the effect diagram of the target detection model trained based on the manhole cover virtual dataset. DETAILED DESCRIPTION

[0041] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0042] The method for detecting manhole cover defects based on virtual sample driving of the present invention comprises the following steps:

[0043] (1) Construct a model for generating defective manhole covers. Collect images of manhole cover scenes and images of manhole cover defects by means of field photography and Internet image downloading. Encode each image and train a Stable Diffusion model using the LoRA (low-rank adaptive) method to generate virtual images of defective manhole covers.

[0044] (2) Drawing a virtual image based on the defective manhole cover generation model, integrating information such as the defective manhole cover scene (background), the defective manhole cover morphology, and the defective manhole cover type description, inputting the defective manhole cover generation model trained in step (1), and obtaining a virtual defective manhole cover image generated by the model;

[0045] (3) Construction of a virtual dataset of damaged manhole covers based on the human-machine collaborative mode. By repeating step (2) multiple times, a large number of virtual images containing damaged manhole covers can be obtained. Virtual images with better quality are manually selected. The target detection frame and the type of damage are generated by the image reconstruction position, prompt words and other information of the image generated in step (2), thereby obtaining data annotations and constructing a virtual dataset of damaged manhole covers.

[0046] (4) Training of a defective manhole cover target detection model driven by a defective manhole cover virtual data set: using the defective manhole cover virtual data set obtained in step (3) as a training data set, selecting a YOLOv10 model as a target detection model, and using the virtual data set to train the YOLOv10 model so that the model can effectively identify various types of manhole cover defects;

[0047] (5) Analysis and statistics of indicators of the defective manhole cover target detection model. During the training process, the key performance indicators of the model are recorded, and the trained model is evaluated using the validation data set. The detection performance of the model on different types of defects is statistically analyzed, such as confusion matrix, category loss, IOU loss, etc. If the indicators of various types of defective manhole covers meet the expected indicators, the training is stopped. If they do not meet the expected indicators, the training is continued.

[0048] (6) Adaptively generate the parameters of the manhole cover defects to be optimized. According to the performance evaluation results in step (5), identify the test samples with insufficient performance, extract the background, morphology, and type information of the defective manhole cover as the generation conditions, jump to step (2) to supplement the type data to be optimized, start the iterative optimization of model training, and repeat the statistics and evaluation of the target detection model indicators until the expected indicators are achieved or the stopping conditions are met, and stop the iteration to complete the training.

[0049] In step (1), the manhole cover defect images collected through various channels are classified into basic categories, including "intact", "damaged", "missing", "uncovered" and "damaged manhole ring". Each image is encoded and divided into training set, validation set and test set of the defect manhole cover target detection model according to the ratio of 8:1:1. The defect manhole covers with complex background and obvious features are selected for LoRA fine-tuning.

[0050] The steps of the LoRA method to train a stable diffusion model include:

[0051] A. Data preparation: Select the images with obvious disease characteristics and retain high-quality images with clear images and obvious features;

[0052] B. Text encoding: Use the pre-trained CLIP (Contrastive Language-Image) pre-trained model to generate text annotations from images, add manhole cover disease codes to the automatically generated annotations, and give the disease code label a larger weight;

[0053] C. LoRA method training: Based on a stable diffusion model pre-trained weight, the LoRA method is used for training, and the low-rank adaptive weights added to the image generator part of the model are updated to obtain a stable diffusion model that can generate manhole cover disease images;

[0054] D. Model screening: After training LoRA, you will see sample images under different epochs. Exclude underfitting and overfitting models and obtain a stable diffusion model that is most suitable for generating virtual diseased manhole cover images as the diseased manhole cover generation model.

[0055] In step (2), a corpus of natural image descriptions is constructed from manhole cover defects, manhole cover scenes, and manhole cover backgrounds, wherein manhole cover defects are organized into five categories: "intact", "damaged", "missing", "uncovered", and "damaged manhole ring". Manhole cover scenes are organized into six categories: "urban road scenes", "urban sidewalk scenes", "park scenes", "urban residential scenes", "industrial scene", and "rural homestead scenes". Manhole cover backgrounds are organized into six categories: "asphalt road", "pedestrian brick road", "green lawn", "cement hardened pavement", "rubber pavement", and "marble pavement". The construction of prompt words is completed based on the classification standards of manhole cover defects, manhole cover scenes, and manhole cover backgrounds; virtual data containing manhole cover defects are generated based on the defect manhole cover generation model through the prompt word text-generated image, prompt word and image multimodal image-generated image mode.

[0056] In step (3), the prompt words for each category of the virtual dataset of manhole covers with diseases are designed according to the manhole cover disease category, background, and scene. The prompt words and background images are used as input data. Step (2) is repeated many times to obtain a large number of virtual images containing manhole covers with diseases. Virtual images with good quality are manually selected until each disease category reaches the designed threshold N. The virtual dataset N = 1000 is generated for the first time. The details of the real data and virtual data are listed in Table 1.

[0057] Table 1 is the detailed table of manhole cover defects of real data set and virtual data set:

[0058]

[0059] According to the prompt words and the location information of the disease, the image labels generated by the virtual diseased manhole cover generation model are constructed to realize image annotation. The process is as follows: Figure 2 As shown in the figure, a virtual dataset of damaged manhole covers with accurate annotations, sufficient quantity, diverse generalization, balanced categories and good consistency is finally constructed, which is composed entirely of virtual images.

[0060] In step (4), the virtual dataset of damaged manhole covers generated in step (3) is first used as the training dataset, and then the YOLOv10 model is initialized and configured with appropriate hyperparameters for training. The model is iteratively optimized using the virtual dataset of damaged manhole covers. The hyperparameter settings of the training process are as follows: the optimizer is AdamW, the learning rate is 0.001, the learning rate decay uses CosineAnnealingLR, the batch size is 2, the IoU threshold of positive and negative samples is set to 0.5, the NMS threshold is set to 0.4, and the number of training rounds is set to 300. In this way, the model can learn a wider range of features in the virtual scene, thereby improving its generalization ability and detection accuracy in practical applications.

[0061] In step (5), the indicators of the defective manhole cover target detection model are analyzed and counted, using the trained model reasoning test set, and the model indicator analysis is carried out based on the reasoning results and the true value;

[0062] First, we construct a multi-classification confusion matrix for defective manhole covers, calculate the true positive, false negative, false positive, and true negative classes for each defect category, and calculate the precision, recall, and F1 score for each category, as well as the AP for evaluating the performance of the object detection model on different target categories.

[0063]

[0064] Among them: i represents category 1, 2, 3, 4, 5 correspond to "intact", "damaged", "missing", "uncovered" and "well ring damaged" respectively. min When it is greater than the threshold M, the confusion matrix analysis result is true, otherwise it is false. In the experiment, the M value is set to 0.75;

[0065] Secondly, the loss function of the defective manhole cover is analyzed to obtain the average loss of each object category in the test set.

[0066]

[0067] Among them: mCLoss represents the average loss of the category, t represents the number of objects in a certain category of test samples, CLoss k It represents the category cross entropy loss of the k-th object. When the smallest mCLoss of the five categories is greater than the threshold P, the result of the loss function analysis of the defective manhole cover is true, otherwise it is false. The P value is set according to expert experience.

[0068] Next, perform IoU analysis on the defective manhole cover targets and find the average IoU loss of each object in the test set.

[0069]

[0070] Among them: mIoULoss represents the category IoU loss, IoULoss k It represents the intersection over union (IoU) loss of the k-th object. When the smallest IoULoss of the five categories is greater than the threshold Q, the IoU loss analysis result of the defective manhole cover is true, otherwise it is false. The Q value is set according to expert experience.

[0071] Take the mode of the confusion matrix analysis, defective manhole cover loss function analysis, and defective manhole cover IoU loss analysis results. If it is true, stop training; if it is false, go to step (6).

[0072] In step (6), during the statistical analysis of the indicators of the defective manhole cover target detection model in step (5), the label parameters of the manhole cover defect category to be optimized are determined for the minimum AP, minimum mCLoss and minimum mIoULoss categories, respectively, and sent to the defective manhole cover generation model. The virtual data set is expanded in a targeted manner, and the target detection model is iteratively optimized. The model accuracy is listed in Table 2.

[0073] Table 2 is a comparison table of model accuracy:

[0074]

[0075] The above description of the embodiments is to facilitate the understanding and application of the present invention by those skilled in the art. It is obvious that those skilled in the art can easily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without creative work. Therefore, the present invention is not limited to the above embodiments. Improvements and modifications made by those skilled in the art to the present invention based on the disclosure of the present invention should be within the protection scope of the present invention.

Claims

1. A method for detecting manhole cover defects based on virtual sample drive, comprising constructing a defect manhole cover generation model, drawing a virtual image based on the defect manhole cover generation model, constructing a virtual data set of defect manhole covers based on a human-computer collaborative mode, training a defect manhole cover target detection model driven by the virtual data set of defect manhole covers, analyzing and counting indicators of the defect manhole cover target detection model, and adaptively generating manhole cover defect parameters to be optimized, characterized in that: The method for detecting manhole cover defects is as follows: (1) Construct a model for generating defective manhole covers. Through manual field photography and Internet image downloading, we collect manhole cover scene images and manhole cover defect images, encode each image, and train a Stable Diffusion model using the LoRA (low-rank adaptive method) to generate virtual defective manhole cover images. (2) Drawing a virtual image based on the defective manhole cover generation model, integrating information on the defective manhole cover scene, the defective manhole cover background, the defective manhole cover morphology, and the description of the defective manhole cover type, inputting the defective manhole cover generation model trained in step (1), and obtaining a virtual defective manhole cover image generated by the model; (3) Construction of a virtual dataset of defective manhole covers based on the human-machine collaborative mode. Repeat step (2) multiple times to obtain a large number of virtual images containing defective manhole covers. Manually select and generate virtual images with good quality. Generate target detection frames and disease types through the image reconstruction position and prompt words generated in step (2) to obtain data annotations and construct a virtual dataset of defective manhole covers. (4) Training of a target detection model for manhole covers driven by a virtual dataset of manhole covers. The virtual dataset of manhole covers obtained in step (3) is used as a training dataset. The YOLOv10 model is selected as the target detection model. The virtual dataset is used to train the YOLOv10 model, so that the YOLOv10 model can effectively identify various types of manhole cover defects. (5) Analysis and statistics of indicators of the defective manhole cover target detection model. The key performance indicators of the model are recorded during the training process. The trained model is evaluated using the validation data set. The detection performance of the model on different types of defects is statistically analyzed. The detection performance includes confusion matrix, category loss, and IOU loss. If the indicators of various types of defective manhole covers reach the expected indicators, the training is stopped. If the indicators of various types of defective manhole covers do not reach the expected indicators, the training is continued. (6) Adaptively generate the parameters of the manhole cover defects to be optimized. According to the performance evaluation results in step (5), identify the test samples with insufficient performance, extract the background, morphology, and type information of the defective manhole cover as the generation conditions, jump to step (2) to supplement the type data to be optimized, start the iterative optimization of model training, and repeat the statistics and evaluation of the target detection model indicators until the expected indicators are achieved or the stopping conditions are met, and stop the iteration to complete the training.

2. The method for detecting manhole cover defects based on virtual sample driving according to claim 1 is characterized in that: In step (1), the manhole cover defect images collected through various channels are classified into basic categories, including "intact", "damaged", "missing", "uncovered" and "damaged manhole ring". Each image is encoded and divided into the training set, validation set and test set of the defect manhole cover target detection model according to the ratio of 8:1:

1. The defect manhole covers with complex background and obvious features are selected for LoRA fine-tuning.

3. The method for detecting manhole cover defects based on virtual sample driving according to claim 2 is characterized in that: The steps of training a stable diffusion model by the LoRA method include: A. Data preparation: Select the images with obvious disease characteristics and retain high-quality images with clear images and obvious features; B. Text encoding: Use the pre-trained CLIP (Contrastive Language-Image) pre-trained model to generate text annotations from images, add manhole cover disease codes to the automatically generated annotations, and give the disease code label a larger weight; C. LoRA method training: Based on a stable diffusion model pre-trained weight, the LoRA method is used for training, and the low-rank adaptive weights added to the image generator part of the model are updated to obtain a stable diffusion model that can generate manhole cover disease images; D. Model screening: After training LoRA, you will see sample images under different epochs. Exclude underfitting and overfitting models and obtain a stable diffusion model that is most suitable for generating virtual diseased manhole cover images as the diseased manhole cover generation model.

4. The method for detecting manhole cover defects based on virtual sample driving according to claim 1, characterized in that: In step (2), a corpus of natural image descriptions is constructed from manhole cover defects, manhole cover scenes, and manhole cover backgrounds, wherein manhole cover defects are organized into five categories: "intact", "damaged", "missing", "uncovered", and "damaged manhole ring". Manhole cover scenes are organized into six categories: "urban road scenes", "urban sidewalk scenes", "park scenes", "urban residential scene", "industrial scene", and "rural homestead scene". Manhole cover backgrounds are organized into six categories: "asphalt road", "pedestrian brick road", "green lawn", "cement hardened pavement", "rubber pavement", and "marble pavement". The construction of prompt words is completed based on the classification standards of manhole cover defects, manhole cover scenes, and manhole cover backgrounds; virtual data containing manhole cover defects is generated based on the defect manhole cover generation model through the prompt word text generation image, prompt word and image multimodal image generation mode.

5. The method for detecting manhole cover defects based on virtual sample driving according to claim 1 is characterized in that: In step (3), prompt words for each category of the virtual dataset of manhole covers with defects are designed according to the category, background and scene of the manhole covers with defects. The prompt words and background images are used as input data, and step (2) is repeated multiple times to obtain a large number of virtual images containing manhole covers with defects. Virtual images with good quality are manually selected until each defect category reaches the designed threshold value N. Image labels are generated by the virtual manhole cover generation model based on the prompt words and the location information of the defects to achieve image annotation. Finally, a virtual dataset of manhole covers with defects that is completely composed of virtual images and that has accurate annotations, sufficient quantity, diverse generalization, balanced categories and good consistency is constructed.

6. The method for detecting manhole cover defects based on virtual sample driving according to claim 1, characterized in that: In step (4), the virtual dataset of damaged manhole covers generated in step (3) is first used as a training dataset, and then the YOLOv10 model is initialized and configured with appropriate hyperparameters for training, and the model is iteratively optimized using the virtual dataset of damaged manhole covers.

7. The method for detecting manhole cover defects based on virtual sample driving according to claim 6 is characterized in that: Hyperparameter settings of the training process: the optimizer is AdamW, the learning rate is 0.001, the learning rate decay uses CosineAnnealingLR, the batch size is 2, the IoU threshold of positive and negative samples is set to 0.5, the NMS threshold is set to 0.4, and the number of training rounds is set to 300.

8. The method for detecting manhole cover defects based on virtual sample driving according to claim 1, characterized in that: In step (5), the indicators of the defective manhole cover target detection model are analyzed and counted, using the trained model reasoning test set, and the model indicator analysis is carried out based on the reasoning results and the true value; First, we construct a multi-classification confusion matrix for defective manhole covers, calculate the true positive, false negative, false positive, and true negative classes for each defect category, and calculate the precision, recall, F1 score, and AP for each category to evaluate the performance of the object detection model on different target categories. Among them: i represents category 1, 2, 3, 4, 5 correspond to "intact", "damaged", "missing", "uncovered" and "well ring damaged" respectively. min When it is greater than the threshold M, the confusion matrix analysis result is true, otherwise it is false; Secondly, the loss function of the defective manhole cover is analyzed to obtain the average loss of each object category in the test set. Among them: mCLoss represents the average loss of the category, t represents the number of objects in a certain category of test samples, CLoss k Represents the category cross entropy loss of the k-th object. When the smallest mCLoss of the five categories is greater than the threshold P, the result of the defective manhole cover loss function analysis is true, otherwise it is false; Next, perform IoU analysis on the defective manhole cover targets and find the average IoU loss of each object in the test set. Among them: mIoULoss represents the category IoU loss, IoULoss k It represents the intersection over union (IoU) loss of the k-th object. When the smallest IoULoss of the five categories is greater than the threshold Q, the IoU loss analysis result of the defective manhole cover is true, otherwise it is false. Take the mode of the confusion matrix analysis, defective manhole cover loss function analysis, and defective manhole cover IoU loss analysis results. If it is true, stop training; if it is false, go to step (6).

9. The method for detecting manhole cover defects based on virtual sample driving according to claim 1, characterized in that: In step (6), during the statistical analysis of the indicators of the defective manhole cover target detection model in step (5), the label parameters of the manhole cover defect category to be optimized are determined for the minimum AP, minimum mCLoss and minimum mIoULoss categories respectively, and sent to the defective manhole cover generation model, the virtual data set is expanded in a targeted manner, and the target detection model is iteratively optimized.

Images