Well lid hidden danger intelligent identification method and system based on improved KDWC-YOLOv5

Abstract

The invention provides a manhole cover hidden danger intelligent identification and detection method and system based on improved KDWC-YOLOv5, and belongs to the field of target detection. The method comprises the following steps: S1, collecting well lid image data and preprocessing the well lid image data to obtain a training data set and a test data set; s2, constructing a KDWC-YOLOv5 network, and carrying out the training of the KDWC-YOLOv5 network through employing the training data set; and S3, identifying the image information in the data set to be identified by using the KDWC-YOLOv5 network in the step S2. According to the method, the improved KDWC-YOLOv5 target detection algorithm is adopted, and on the basis of a traditional model, through a series of innovative improvements, the accuracy and robustness of manhole cover hidden danger recognition are remarkably improved. According to the method, the detection precision of the hidden danger of the manhole cover in a conventional scene is improved, and the detection performance under the conditions of insufficient data volume, complex hidden danger position and the like is improved.

Description

Technical Field

[0001] The present invention belongs to the field of target detection technology, and specifically relates to an intelligent identification method and system for manhole cover hidden dangers based on an improved KDWC-YOLOv5 (Knowledge Distillation Based YOLOv5 for Well Cover) model, which is suitable for rapid detection and judgment of manhole cover hidden dangers. Background Art

[0002] With the development of smart cities, more and more intelligent technology products are being applied to every corner of the city. Recently, the national standard "Intelligent Manhole Cover" (GB / T41401-2022) was officially released, promoting the rapid and healthy development of the intelligent manhole cover industry.

[0003] Manhole covers are outdoors, exposed to wind, rain and crushing, and therefore may have hidden dangers such as damage, bulges, missing, and manhole ring problems. These manhole covers with hidden dangers not only affect the cleanliness of the city, but also pose a great threat to public safety. Currently, there are some studies on realizing smart manhole covers by installing sensors inside the manhole covers, but such methods have high construction and maintenance costs and cannot be fully promoted. In addition, due to the wide distribution and large number of manhole covers, the number of work orders for repairing hidden danger manhole covers is huge every month, and manual review is inefficient. Based on this, the present invention proposes a method for intelligent identification of hidden dangers in manhole covers, which uses target detection technology to realize intelligent detection of the status of manhole covers for outdoor inspection and audit scenarios of transmission routes, and identifies whether there are hidden dangers in the manhole covers and their specific hidden danger categories, thereby improving the work efficiency and quality of audit management personnel. Summary of the Invention

[0004] In view of the problems existing in the manual manhole cover identification process, such as complex actual scenarios, difficulty in identifying hidden dangers, difficulty in distinguishing categories, and low efficiency, the present invention has developed a high-accuracy intelligent identification method and system for manhole cover hidden dangers. The present invention can accurately identify the status of the manhole cover in the complex environment of actual road conditions, determine whether there are hidden dangers in the manhole cover, and accurately classify the hidden dangers, thereby improving the efficiency and quality of quality inspection while achieving comprehensive coverage.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] The intelligent identification method of manhole cover hidden dangers based on the improved KDWC-YOLOv5 includes the following steps:

[0007] S1, collect manhole cover image data and preprocess it to obtain model training data set and test data set;

[0008] S2. Improve the original YOLOv5 model architecture, build an improved KDWC-YOLOv5 network, and train the KDWC-YOLOv5 network using the training dataset;

[0009] S3. Use the improved KDWC-YOLOv5 network to identify the image information in the test dataset and realize intelligent identification of manhole cover hazards.

[0010] Preferably, in step S1, collecting manhole cover image data specifically includes: collecting public data sets through the Internet to obtain image sample data containing 5 categories of different features (normal manhole cover, missing manhole cover, damaged manhole cover, manhole ring problem and uncovered manhole cover).

[0011] Preferably, in step S1, the preprocessing operations performed on the collected target image sample data specifically include: using Python crawler technology, deep learning diffusion model Stable Diffusion XL (SDXL), image geometric color transformation and field collection to perform data enhancement in four ways, constructing a large-scale data set of up to 14,392 manhole cover images, and using the Labelme tool to perform detailed data annotation after the data set is amplified. Finally, the annotated XML file format is converted into a TXT file format for YOLO model training.

[0012] Preferably, step S2 specifically includes:

[0013] S2.1. Use knowledge distillation to train the KDWC-YOLOv5 teacher-student network.

[0014] S2.2. Using the DM-IoU calculation method based on the shape and size of the bounding box itself, this method more accurately evaluates the degree of match between the predicted bounding box and the true bounding box;

[0015] S2.3. Use a training strategy based on multi-scale input and model fusion to enhance the model. This training strategy averages the output of multiple models to smooth out fluctuations in individual model predictions.

[0016] Preferably, in step S2.1, a KDWC-YOLOv5 teacher-student network based on knowledge distillation is used, that is, two YOLOv5 networks are trained simultaneously. There are certain differences in the parameter amounts and model architectures of the two networks. A more complex network is pre-trained as the teacher network, and the other simpler network is pre-trained as the student network. The key knowledge and information obtained from the teacher network training are extracted to guide the learning process of the student network, so that the student network can achieve better performance and accuracy, thereby effectively improving the recognition accuracy and reducing the volume of the original model.

[0017] Preferably, in step S2.2, the DM-IoU (Distance Matching-IoU) calculation method based on the shape and size of the bounding box itself is as shown in formula (1-6), where b and b gt where w, h, and c represent the predicted and ground-truth bounding boxes, respectively, while w, h, and c represent the width, height, and center of the bounding box, respectively. This new calculation method not only considers the degree of overlap between the predicted and ground-truth bounding boxes, but also fully accounts for the impact of the bounding box's shape and size on the regression results. By incorporating this shape and size information, DM-IoU can more accurately assess the degree of match between the predicted and ground-truth bounding boxes, thereby improving detection performance.

[0018]

[0019] L Shape-IoU =IoU+distance shape +0.5×Ω shape (6)

[0020] Preferably, in step S2.3, a training strategy based on multi-scale input and model fusion is used. Multi-scale input involves randomly selecting an image of different resolution or cropping scale as input within a set size range every set batch during training, enabling the model to simultaneously focus on both global and local information of the image. Model fusion involves averaging the output results of multiple KDWC-YOLOV5 models with different parameter values, combining the prediction results of multiple independent models to fully leverage the strengths of each model and reduce potential bias or error in a single model. By using multi-scale image input, the model can simultaneously consider both global and local information in the image, a strategy that plays a key role in improving the model's predictive ability. Global information in the image helps the model capture overall structure and contextual relationships, while local information is crucial for fine details and feature recognition. Multi-scale input means that the model receives images of different resolutions or cropping scales as input, allowing it to simultaneously utilize global and local information. Furthermore, using model fusion to average the output results of multiple models is an effective method for reducing prediction generalization error. By combining the prediction results of multiple independent models, model fusion can fully leverage the strengths of each model and reduce potential bias or error in a single model. Averaging the output of multiple models can smooth out fluctuations in the predictions of individual models and improve the stability and accuracy of the overall prediction.

[0021] Preferably, in step S3, based on the deep learning target detection model KDWC-YOLOv5, a "Manhole Cover Detective" web page and WeChat applet system are built in combination with Vue2 and Flask technologies. Among them, the back-end system is built using the Flask framework. The routing of the Flask framework is used to encapsulate the interface for front-end and back-end interaction. This interface defines a route for uploading files, receives the manhole cover images uploaded by the user, and then uses the core model to detect the image and return the processing results. The front-end uses the Vue2 framework to create the web page structure of the web page, and then uses the interface provided by the back-end to upload the image to be detected and render and display the detection results. The front-end part of the WeChat applet is built using the Uniapp framework, and the method provided by the framework is used to upload images and render and display data. The overall structure of the two systems is: the Flask back-end uses the model to process the image uploaded by the front-end and returns the detection results to the front-end through the interface; the Vue2 and Uniapp front-ends receive the image uploaded by the user and display the detection results. On the web page, after the user uploads the image, the back-end of the present invention will call the trained target detection model. The model determines whether a manhole cover is abnormal and the type of abnormality, and provides corresponding analysis and suggestions for different abnormality types. If multiple images are uploaded, the images are batch tested and statistical and analytical images of the corresponding results are returned. Specifically, it primarily utilizes Vue2 to design the front-end interface and the Python-based Flask back-end framework to load the model and process the images to be tested. As a typical front-end and back-end separated system, after uploading the image on the front-end, all tests must be completed on the back-end, and the corresponding interface functions must be encapsulated before the test results are uploaded to the front-end. In addition to inheriting the above functions, the WeChat mini-program also adds an additional feature—support for calling the mobile phone camera, making it more real-time and portable.

[0022] The present invention also discloses an intelligent identification system for manhole cover hidden dangers based on improved KDWC-YOLOv5, which is used to execute the above method and includes the following modules:

[0023] Data acquisition module: collects manhole cover image data and preprocesses it to obtain training data sets and test data sets;

[0024] Model training module: builds the KDWC-YOLOv5 network and trains the KDWC-YOLOv5 network using the training dataset;

[0025] Recognition module: Use the KDWC-YOLOv5 network to identify image information in the test dataset.

[0026] The present invention provides an intelligent identification method and system for manhole cover hidden dangers based on improved KDWC-YOLOv5, which has the following significant advantages and beneficial effects:

[0027] The KDWC-YOLOv5 model of this invention achieves a 10.3% improvement in the key mAP metric, reaching 0.948, compared to leading models such as DETR and YOLOv7. Furthermore, the detection time for a single image is reduced by 12.1%, and the model size is only 42.1MB, nearly half the original YOLOv5 model's 76.8MB. This performance optimization makes the KDWC-YOLOv5 model particularly suitable for deployment on mobile devices and embedded systems.

[0028] In order to further reflect the practical value of the present invention, in the preferred embodiment of the present invention, the KDWC-YOLOv5 model is successfully applied to the intelligent identification system of manhole cover hazards on the mini program and web page.

[0029] "Manhole Cover Detective" has achieved an efficient transition from theoretical research to practical application. Through this application, users can quickly, efficiently, and visually detect and identify hidden dangers in manhole covers, providing reliable technical support for the safe management of urban infrastructure.

[0030] In general, the significance of the present invention is:

[0031] 1. Improved efficiency and accuracy of intelligent detection: By improving the model architecture, not only does it improve detection accuracy, but it also reduces detection latency, meeting the needs of efficient processing.

[0032] 2. Enhanced model adaptability and practicality: The lightweight design of the model significantly reduces the hardware burden and is suitable for embedded systems and mobile devices with limited resources.

[0033] 3. Promoted technology implementation and practical application: Successfully integrated cutting-edge algorithms into practical systems, providing innovative solutions for smart city construction and public safety management.

[0034] 4. It has broad application prospects: In addition to detecting hidden dangers in manhole covers, the present invention can also be expanded to the field of intelligent monitoring and identification of other urban infrastructure, further promoting the development of intelligent management.

[0035] In summary, the present invention has not only achieved a breakthrough in technology, but also demonstrated significant social effects and economic value in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0037] Figure 1 This is a flow chart of a method for intelligently identifying hidden dangers of manhole covers based on improved KDWC-YOLOv5 in a preferred embodiment of the present invention;

[0038] Figure 2 is the sample data set diagram collected;

[0039] Figure 3 This is the basic flow chart of Python crawler;

[0040] Figure 4 Automatically expand the image background based on SDXL;

[0041] Figure 5 This is the YOLOv5 model architecture diagram;

[0042] Figure 6 This is the KDWC-YOLOv5 network diagram based on knowledge distillation;

[0043] Figure 7 In the figure, (a) is the traditional IoU calculation method, and (b) is the DW-IoU calculation method proposed by the present invention;

[0044] Figure 8 A training graph based on multi-scale input and model fusion;

[0045] Figure 9 This is a display diagram of the web APP based on the improved network;

[0046] Figure 10 This is a diagram showing the WeChat applet based on the improved network;

[0047] Figure 11 This is a block diagram of an intelligent identification system for manhole cover hazards based on improved KDWC-YOLOv5 in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to enable those skilled in the art to better understand the technical solution of the present invention and to be able to implement it, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention and are not intended to limit the scope of protection of the present invention.

[0049] The present invention belongs to the field of target detection. Figure 1 As shown, this embodiment provides an intelligent identification method for manhole cover hidden dangers based on the improved KDWC-YOLOv5. The specific operations are as follows:

[0050] Step 1: Collect target image sample data and build a sample data set;

[0051] In this step, we collected public datasets from the Internet to obtain image sample data with different features. A total of 1,325 training set sample images were collected (347 with intact manhole covers, 169 with damaged manhole covers, 292 with missing manhole covers, 492 with uncovered (lifted) manhole covers, and 25 with manhole ring problems). 350 test set samples were collected, and the resolution was cropped to 640 pixels × 640 pixels. Figure 2 shown.

[0052] Step 2: Perform data enhancement on the sample data:

[0053] In this embodiment, to address the problems of small training sample data capacity (1325 images) and imbalanced data volume between categories (i.e., the number of manhole cover problems in the category is much larger than that in other categories, with only 25 images), four methods were used for data enhancement: Python crawler technology, deep learning diffusion model Stable Diffusion XL (SDXL), field acquisition, and image geometric color transformation. Ultimately, a large-scale dataset of 14,392 manhole cover images was constructed (of which the number of manhole cover problem data increased from the original 25 to 223). After the dataset was amplified, the Labelme tool was used for detailed data annotation. Finally, the annotated XML file format was converted into a TXT file format for YOLO model training. A more detailed description of this step is as follows:

[0054] Step 2.1: Data augmentation for the well circle problem based on crawlers and SDXL

[0055] This embodiment uses Python crawler technology to perform focused crawling of well circle problems on the two search engines Baidu Pictures and Bing Pictures, where a crawler is a program or script that automatically crawls Internet information. It simulates the behavior of humans requesting websites, automatically requests web pages and crawls data, and then uses certain rules to extract valuable data. The crawler program collects data by requesting URL addresses and parsing the response content. If the response content is HTML, the crawler will analyze its DOM tree structure, perform DOM parsing, or use regular matching; if the response content is XML / JSON data, the crawler can convert the data object and then parse the data object, such as Figure 3As shown. In the specific crawling process, for Baidu images, this embodiment calls the re.findall() function in the Python re module, which searches for all parts that match a given regular expression in an HTML string and returns a list containing all matching items, thereby extracting image links from the response HTML content using regular expressions. For Bing images, an XPath (XML query language used to navigate through elements and attributes in XML documents and find nodes in an XML tree structure) query is used to locate a list of elements containing image information. For each element in the list, the JSON data of the image link is further extracted and parsed to obtain the actual image URL. Ultimately, this method obtains data on 223 well circle problems, greatly increasing the number of well circle categories.

[0056] Stable Diffusion XL (SDXL) is a generative pre-training model based on the diffusion model. It gradually generates high-quality data samples by simulating the data distribution process. Compared with traditional generative models, SDXL has higher generation quality and stability. The core principles of SDXL include the diffusion process and the reverse process. In the diffusion process, the model gradually adds noise to the original data to make it gradually approach a random distribution, while in the reverse process, the model starts from the random distribution, gradually removes the noise, and finally generates high-quality data samples. Based on this, the image size can be changed by cropping, and the function of automatically generating (completing the cropped part) images by the model can be used to automatically expand the cropped image, so that the image content can be more completely presented. Using SDXL, an advanced deep learning algorithm, the retention of specific subjects in the manhole problem image and the reconstruction of the surrounding background are completed. By maintaining the invariance of the main elements of the manhole cover and the manhole ring, the surrounding background parts are cropped, and the algorithm is used to automatically complete the surrounding background, such as Figure 4 shown.

[0057] Step 2.2: Data augmentation based on geometric color transformation

[0058] Each existing image in the sample data is enhanced using seven methods: geometric and color space transformations. These methods include random noise addition, brightness adjustment, cutout (randomly cropping certain areas of the image and filling them with zero pixel values or other fixed values), random rotation and flipping, random cropping, translation, and mirroring. These transformations increase the model's generalization capabilities and enable the generation of more training data using existing training sample data, thus addressing the data shortage issue. This approach not only reduces overfitting but also significantly improves model accuracy.

[0059] Step 2.3: Data enhancement based on field collection

[0060] Through field collection and filming of a large amount of materials about well ring problems, and after careful sorting and screening, 27 pictures of well ring problem data were obtained, providing real and accurate first-hand information for the well ring problem dataset.

[0061] Step 3: Label the sample data to obtain the data to be identified, specifically:

[0062] LabelMe is an image annotation tool developed by the Computer Science and Artificial Intelligence Laboratory (CSAIL) at the Massachusetts Institute of Technology (MIT). Written in Python and PyQT, it can annotate polygons, rectangles, circles, polylines, line segments, points, and other forms of data, making it suitable for tasks such as object detection and image segmentation. LabelMe was selected as the annotation tool for annotating the expanded manhole ring dataset. The primary annotation area focused on the manhole cover and the surrounding manhole ring. After annotation, the resulting XML file format was converted to a txt file format supported by the YOLO model for subsequent data processing, model training, and analysis.

[0063] Step 4: Improve the target detection algorithm YOLOv5 to obtain an improved target detection algorithm YOLOv5;

[0064] First, the principle of YOLOv5 algorithm is introduced as follows:

[0065] The YOLO series (You Only Look Once version) model is a single-stage target detection model proposed based on the progress of deep learning of convolutional neural networks. It has been proven by data that it can provide good classification and detection effects for multi-category and multi-target tasks under limited computing resources. In the field of target detection, although many models have achieved good results on more classic target detection datasets (such as COCO datasets), the complexity of many models will hinder them in many environments with limited computing resources, and training such models also requires huge costs. Therefore, there is a need for an efficient neural network architecture that can minimize the consumption of computing resources (such as floating-point operations, model parameters, etc.) while ensuring excellent output results. Therefore, the present invention finally adopts the YOLOv5 model, that is, a model with a smaller model and higher accuracy. This approach avoids the need for an overly wide input layer or too many hidden layers, and effectively improves the performance of manhole cover detection.

[0066] like Figure 5As shown, the three major components of the YOLOv5 architecture are the backbone network, the PANet module, and the output layer. First, the residual structure is a key component of YOLOv5. The design concept of the residual network (ResNet) is to introduce short-circuit connections, allowing gradients to be directly propagated back to previous layers, effectively addressing the problems of vanishing gradients and model degradation in deep neural network training. In YOLOv5, the use of the residual structure helps the model maintain good performance even with increasing depth and extract richer feature information.

[0067] Secondly, the SPP module (Spatial Pyramid Pooling module) is also a key component in YOLOv5. The role of the SPP module is to implement spatial pyramid pooling and information fusion at different scales. This means that it can fuse feature maps at different levels through pooling operations, thereby fully utilizing feature information at different scales. This design helps improve the accuracy and speed of object detection and enables the model to better adapt to objects of different sizes and shapes. Specifically, the SPP module can pool image features into multiple feature maps of different scales, then concatenate them together and input them into subsequent convolutional layers for further processing. This operation enables the model to focus on both global and local feature information, thereby improving detection performance.

[0068] The present invention improves the target detection algorithm YOLOv5 by the following improvement steps:

[0069] Step 4.1: Incorporate knowledge distillation ideas to build the KDWC-YOLOv5 model

[0070] Based on the in-depth study of the YOLOv5 network architecture, this paper creatively incorporates the idea of knowledge distillation and proposes a new KDWC-YOLOv5 network based on knowledge distillation. Figure 6As shown in Figure 2, the core of this innovative approach lies in the simultaneous training of two YOLOv5 networks with different parameter counts and model architectures. A more complex and high-performance network serves as the teacher network, providing rich knowledge during training; the other, simpler and lightweight network serves as the student network, aiming to improve its own performance by learning from the teacher network's knowledge. During the training process of the KDWC-YOLOv5 network, a pre-trained YOLOv5 network is first used as the teacher network. This network, trained on a large number of datasets to achieve high detection accuracy and stability, is then used to extract key knowledge and information from the teacher network. This information includes not only the location and category of detected objects, but also deeper feature representations and contextual relationships. This knowledge is then appropriately transferred to the student network to guide its learning process. Through knowledge distillation, the student network is able to fully absorb the rich knowledge of the teacher network, significantly improving its own performance and accuracy while maintaining the advantages of a lightweight model. This approach not only helps improve the student network's object recognition accuracy but also effectively reduces the size of the original model, making it more suitable for real-world applications.

[0071] Step 4.2: Introduce the DM-IoU calculation method based on the shape and size of the bounding box itself

[0072] In tasks such as object detection and instance segmentation, IoU (Intersection over Union) is an important indicator to measure the degree of overlap between the predicted result and the true result. Usually, when the IoU value is greater than or equal to a preset threshold (such as 0.5 or 0.7), the prediction result is considered correct. The traditional IoU calculation method is to evaluate the accuracy of the algorithm's prediction by measuring the degree of overlap between the predicted bounding box and the true bounding box, such as Figure 7 (a) in the figure. The intersection area is the area of the overlap between the predicted and true bounding boxes, while the union area is the total area covered by the two bounding boxes, including their overlap and their unique areas. A higher IoU value indicates a closer match between the predicted and true bounding boxes, and a higher detection accuracy.

[0073] However, existing bounding box regression losses typically only consider the relative positional relationship between the true box and the predicted box, while ignoring the impact of the inherent properties of the bounding box (such as its shape and size) on bounding box regression. In daily life and official datasets, it can be observed that the shape and scale of manhole covers in images are somewhat different from the shape and scale of the predicted boxes. In this case, the calculation of traditional IoU and loss functions is not very close to the actual situation. The four corners of the detection box are often more prone to "loss redundancy", which is not conducive to the model's more accurate learning of the target's positional features. In addition, in scenarios where the manhole covers are densely distributed, the predicted boxes are easily removed by the "non-maximum suppression" mechanism in YOLO, which may lead to missed detections and make it impossible to eliminate the safety hazards caused by manhole cover anomalies.

[0074] In order to solve this problem, the present invention makes innovative improvements on the basis of traditional IoU and proposes a new DM-IoU calculation method based on the shape and size of the bounding box itself, such as Figure 7 As shown in (b) and formula (1-6).

[0075]

[0076] L Shape-IoU =IoU+distance shape +0.5×Ω shape (6)

[0077] This new calculation method not only considers the degree of overlap between the predicted and true bounding boxes, but also fully accounts for the impact of the bounding box's shape and size on the regression results. By incorporating this shape and size information, DM-IoU can more accurately assess the degree of match between the predicted and true bounding boxes, thereby improving detection performance.

[0078] Step 4.3: Training strategy based on multi-scale input and model fusion

[0079] Multi-scale training is very effective for dense convolutional neural networks such as YOLO. During training, every certain batch, within a certain size range, an image of a different resolution or a different cropping scale is randomly selected as input, so that the model can focus on the global and local information of the image for training. By training on images of different scales, the model can, to a certain extent, take into account both the global and local information of the image. This strategy plays a key role in improving the predictive ability of the model. The global information of the image helps the model capture the overall structure and contextual relationships, while the local information is crucial for fine details and feature recognition. In addition, using model fusion to average the output results of multiple models is an effective way to reduce prediction generalization errors. Model fusion can fully utilize the advantages of each model and reduce the bias or error that may exist in a single model by combining the prediction results of multiple independent models. Averaging the output results of multiple models can smooth the prediction fluctuations of individual models and improve the stability and accuracy of the overall prediction. The training strategy process based on multi-scale input and model fusion is as follows: Figure 8 shown.

[0080] Step 5: Build a manhole cover target anomaly detection web page and mini-program system based on the improved model

[0081] The present invention builds a manhole cover anomaly detection system based on the deep learning target detection model KDWC-YOLOv5, which can upload the manhole covers that need to be detected on the web page and WeChat applet. On the web page, after the user uploads the image, the algorithm backend will call the trained target detection model. The model determines whether the manhole cover is abnormal and the type of anomaly, and gives corresponding analysis and suggestions for different pairs of anomaly types. If multiple images are uploaded, the images can be batch detected, and then the statistical and analytical images of the corresponding results are returned. Specifically, the system mainly utilizes Vue2 to design the front-end page and the Python-based Flask back-end framework to load the model and process the images to be detected. As a typical front-end and back-end separated system, after uploading the picture on the front end, all detections need to be completed on the back end, as well as the encapsulation of the corresponding interface functions, and then the detection results are uploaded to the front end. In the WeChat applet, in addition to inheriting the above functions, an additional function is added - support for calling the mobile phone camera, making it more real-time and portable. A more specific description is as follows:

[0082] Step 5.1: Build a web-based system for manhole cover target anomaly detection based on the improved model

[0083] Front-end part:

[0084] The front end of the system developed in this embodiment is based on the Vue2 framework, and uses the website rapid prototyping tool ElementUI to build a simple and beautiful user interface. Figure 9The user can upload the image of the manhole cover to be inspected through this interface, and the program will return and display the inspection results after the inspection is completed.

[0085] The system interface layout developed in this embodiment includes:

[0086] (1) The title "Manhole Cover Detective" and a multi-language selector are displayed above.

[0087] (2) The middle area is the image upload area. You can click the button to select an image file to upload. If you choose to upload multiple images, a statistical analysis chart of the batch processing results will be displayed here.

[0088] (3) After successful upload, the model will be automatically used for detection. After the detection is completed, the detection results will be displayed below. The displayed content includes: whether it is abnormal, the type of abnormality, the confidence level, and the detection result suggestions.

[0089] In addition, after the file is uploaded, the system will display a progress bar pop-up box to indicate whether the upload is successful and the model detection progress. After the detection is completed, the user can also click the Re-upload button, select the file to re-detect, and then display the new detection results.

[0090] Backend part:

[0091] The back-end system built with Flask can detect targets in uploaded images, as well as perform statistics and analysis on the recognition results. The routing based on the Flask framework encapsulates the interface for front-end and back-end interaction: upload. This interface defines a route for uploading files, receives images of manhole covers uploaded by users, processes them using the core model, and returns the processing results. This includes the acquisition and verification of image files, and the files will be saved and copied after successful upload. The processing results include whether the manhole cover in the image is abnormal, the type of abnormal result, and the corresponding confidence level. If multiple images are uploaded, the results also include statistics and analysis of the detection results of these multiple images. If the file upload fails, an error status code 500 will be returned.

[0092] The specific implementation of the interface is as follows:

[0093] 1) Use Flask's decorator to define the route " / upload".

[0094] 2) After checking the existence of the file, use the custom function allowed_file() to check whether the file type is allowed to be uploaded. This function can check whether the file extension is in the allowed list.

[0095] 3) Use the custom function os.path.join() to construct the file save path and save the uploaded file to the specified file path.

[0096] 4) Use the custom function core.main.c_main() to call the trained model to perform anomaly detection and classification on images.

[0097] 5) Finally, the backend detection status status is returned (200 for success / 500 for failure). If the detection is successful, the detection result will also be returned.

[0098] Step 5.2: Build a WeChat applet system for manhole cover target anomaly detection based on the improved model

[0099] In the front-end part, this embodiment also innovatively developed a WeChat applet version of the manhole cover detective, providing users with a more convenient and flexible use experience. Figure 10 As shown in the figure, the mini program version not only inherits the core functions of the web version, that is, it can quickly return prediction results and related suggestions after the user uploads an image, but also incorporates the unique advantage of WeChat mini programs on this basis - support for calling the mobile phone camera.

[0100] WeChat mini-programs support accessing mobile phone cameras, a real-time feature that is crucial for manhole cover inspections. First, it significantly improves work efficiency, allowing inspectors to check the condition of manhole covers anytime, anywhere without having to carry additional cameras. Second, real-time capture and instant feedback on prediction results help inspectors quickly identify and address potential safety hazards, reducing the risk of accidents.

[0101] The WeChat Mini Program version also offers enhanced portability and ease of use. Users can easily access the Mini Program by simply searching and opening it within WeChat, without having to download or install additional apps. This is undoubtedly a significant convenience for those who don't frequently use a computer or don't have specific apps installed. Users can quickly access the Mini Program simply by searching within WeChat or scanning a QR code. This out-of-the-box functionality makes it even more convenient to use the camera to capture images, eliminating the need to navigate to other apps or webpages, improving operational efficiency. This real-time nature gives the Mini Program a distinct advantage in scenarios where rapid image acquisition and processing are crucial.

[0102] Step 6: Use the improved KDWC-YOLOv5 algorithm to achieve efficient target detection and recognition of the data to be identified through operations such as target anchor box detection and backbone network feature extraction.

[0103] This step is described in conjunction with comparative experiments. We compared and analyzed the recognition performance of four models (DETR, Faster R-CNN, YOLOv7, and the KDWC-YOLOv5 model of our invention) on a 350-image test set. These four models cover the three major types of object recognition currently available (single-stage, dual-stage, and end-to-end). To comprehensively evaluate the performance differences between the models, we used two main evaluation metrics: mAP (from a model performance perspective) and single-image detection time (from a model volume perspective). The experimental results are shown in Table 1:

[0104] Table 1 Comparative experimental results

[0105] Model Name / Indicator mAP Single image detection time / ms DETR 0.856 11.2 YOLOv7 0.846 9.9 Faster R-CNN 0.844 10.4 KDWC-YOLOv5 0.944 8.7

[0106] Experimental results show that KDWC-YOLOv5 performs exceptionally well in both mAP and single-image detection time, outperforming the other three compared models. As shown in the table, our single-image detection time is at least 12.1% higher than these advanced models, and our mAP is at least 10.3% higher. Our model is only 42.1MB in size, making it easy to deploy on mobile devices and embedded systems, achieving a win-win situation of lightweight and high accuracy.

[0107] In addition, to demonstrate the effectiveness of the knowledge distillation module, DM-IoU strategy, and multi-scale input and model fusion techniques, this example conducted an ablation experiment. As shown in Table 2, the results show that the YOLOv5 model incorporating the above techniques improves mAP and single-image detection time by 11% and 8%, respectively, compared with the original YOLOv5 model. This result strongly demonstrates the practicality and superiority of these techniques introduced in this invention.

[0108] Table 2 Ablation experiment results

[0109] Model Name / Indicator mAP Single image detection time / ms YOLOv5 0.842 9.4 KDWC-YOLOv5 0.944 8.7

[0110] like Figure 11 As shown, this embodiment provides an intelligent identification system for manhole cover hidden dangers based on improved KDWC-YOLOv5, which is used to execute the above method and includes the following modules:

[0111] Data acquisition module: collects manhole cover image data and preprocesses it to obtain training data sets and test data sets;

[0112] Model training module: builds the KDWC-YOLOv5 network and trains the KDWC-YOLOv5 network using the training dataset;

[0113] Recognition module: Use the KDWC-YOLOv5 network to identify image information in the test dataset.

[0114] For other contents of this embodiment, please refer to the above method embodiment.

[0115] In summary, this invention improves upon the YOLOv5 algorithm, employing multiple approaches, including geometric transformations, Python crawlers, diffusion models, and field data collection, to address data shortages and imbalanced manhole data. Furthermore, it introduces knowledge distillation and proposes the novel concept of DM-IoU (Intersection over Union) for detection boxes. Furthermore, multi-scale image training and model fusion strategies are employed in model training to successfully construct the KDWC-YOLOv5 model. Under the experimental environment of this invention, ablation and comparison experiments were conducted on algorithms with both individual and multiple improvements. The KDWC-YOLOv5m model achieved the best performance, achieving a mAP of 0.948, a 10.3% improvement over the state-of-the-art model. The single-image detection time was reduced by 12.1%, and the model size was only 42.1MB, smaller than the 76.8MB of the original YOLOv5 model. Therefore, the improved KDWC-YOLOv5 model significantly improves the accuracy and efficiency of manhole cover hazard identification, making it particularly suitable for deployment on mobile devices and embedded systems, providing an effective technical solution for the intelligent identification of manhole cover hazards.

[0116] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. It will be apparent to those skilled in the art that various changes, modifications, substitutions, and variations of these embodiments, including components, without departing from the principles and spirit of the present invention are still within the scope of protection of the present invention.

Claims

1. An intelligent identification method for manhole cover hidden dangers based on improved KDWC-YOLOv5 is characterized by: The steps include: S1, collect manhole cover image data and preprocess it to obtain training data set and test data set; S2. Build the KDWC-YOLOv5 network and train it using the training dataset. S3. Use the KDWC-YOLOv5 network in step S2 to identify the image information in the test data set.

2. The method for intelligently identifying hidden dangers of manhole covers based on improved YOLOv5 according to claim 1 is characterized in that: In step S1, the manhole cover image data is collected as follows: a public data set is collected through the Internet, and the characteristics of the obtained manhole cover image data include normal manhole cover, missing manhole cover, damaged manhole cover, manhole ring problem and uncovered manhole cover.

3. The method for intelligently identifying hidden dangers of manhole covers based on improved YOLOv5 according to claim 1 is characterized in that: In step S1, the preprocessing is as follows: data enhancement is performed using four methods: Python crawler technology, deep learning diffusion model Stable Diffusion XL, image geometric color transformation, and field collection. A manhole cover image dataset is constructed, and after the enhanced dataset, the Labelme tool is used for data annotation. The annotated XML file format is converted into a TXT file format for model training.

4. The method for intelligently identifying hidden dangers of manhole covers based on improved YOLOv5 according to claim 1 is characterized in that: Step S2 specifically includes: S2.

1. Use knowledge distillation to train the KDWC-YOLOv5 network. S2.

2. Use the DM-IoU calculation method based on the shape and size of the bounding box itself to evaluate the degree of match between the bounding box and the ground-truth bounding box; S2.

3. Enhance the model using a training strategy based on multi-scale input and model fusion.

5. The method for intelligently identifying hidden dangers of manhole covers based on improved YOLOv5 according to claim 4 is characterized in that: The specific training of step S2.1 is as follows: pre-train a complex network as the teacher network and another simple network as the student network, extract the key knowledge and information obtained from the teacher network training, and use it to guide the learning process of the student network.

6. The method for intelligently identifying hidden dangers of manhole covers based on improved YOLOv5 according to claim 4 is characterized in that: In step S2.2, the DM-IoU calculation method based on the shape and size of the bounding box itself is shown in formulas (1)-(6): L Shape-IoU =IoU+distance shape +0.5×Ω shape (6) Among them, IoU: intersection over union, represents the overlap between the predicted box and the real box, b and b gt Represents the predicted bounding box and the true bounding box respectively, ww: the scale factor in the horizontal direction, which is used to measure the overlap between the predicted box and the true box in the horizontal direction, w gt Indicates the width of the target box, h gt Indicates the height of the target box, scale indicates the scaling factor, w, h and c indicate the width, height and center point of the bounding box respectively, and distance shape Represents the shape distance, which is used to calculate the shape difference between the predicted box and the real box, hh is the scale factor in the longitudinal direction, which is used to measure the overlap between the predicted box and the real box in the vertical direction, Ω shape Represents the overall loss term of the shape, L Shape-IoU Represents the final loss function, x c and y c is the center coordinate of the prediction box.

7. The method for intelligently identifying hidden dangers of manhole covers based on improved YOLOv5 according to claim 4 is characterized in that: In step S2.3, the multi-scale input is specifically as follows: during training, every set batch, within the set size range, randomly select an image of different resolution or different cropping scale as input, so that the model can pay attention to both the global and local information of the image; the model fusion is specifically as follows: the output results of multiple KDWC-YOLOV5 models with different parameter values are averaged and combined with the prediction results of multiple independent models.

8. The method for intelligently identifying hidden dangers of manhole covers based on improved YOLOv5 according to claim 1, characterized in that: In step S3, a manhole cover recognition web page and WeChat applet system are built based on the KDWC-YOLOv5 network to identify the image information in the test dataset.

9. An intelligent identification system for hidden dangers of manhole covers based on improved KDWC-YOLOv5, used to execute the method according to any one of claims 1 to 8, characterized in that: Includes the following modules: Data acquisition module: collects manhole cover image data and preprocesses it to obtain training data sets and test data sets; Model training module: builds the KDWC-YOLOv5 network and trains the KDWC-YOLOv5 network using the training dataset; Recognition module: Use the KDWC-YOLOv5 network to identify image information in the test dataset.

Images