Well lid hidden danger detection method

Abstract

The invention belongs to the technical field of computer vision and deep learning, and discloses a manhole cover hidden danger detection method. The method comprises the following steps: firstly, performing rapid target detection on an input image by using a YOLOv8 model, positioning a well lid area and outputting a bounding box; then, inputting a target area corresponding to the bounding box into a ViT classification model, and performing fine-grained classification on manhole cover states (such as damage and loss); and finally, integrating detection and classification results, and outputting structured detection information with coordinates, state category labels and confidence scores. According to the method, the influence of class imbalance is reduced by adopting a combined loss function of Focal Loss and label smooth cross entropy loss, the defect identification capability is improved, and the detection precision is optimized through an OfficientNMSTRT plug-in of TensorRT.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and deep learning technology, and specifically relates to a manhole cover hidden danger detection method based on the fusion of the YOLOv8 (YouOnly Look Once version 8) target detection model and the ViT (Vision Transformer) classification model. Background Art

[0002] With the acceleration of urbanization, manhole covers, as a vital component of municipal infrastructure, have a direct impact on public safety and residents' quality of life. Currently, manhole cover management faces multiple challenges, including inefficient manual inspections, high sensor technology costs, and insufficient accuracy in computer vision algorithms. Traditional manual inspections are time-consuming and labor-intensive, making comprehensive urban coverage difficult. This inefficient model, particularly in rapidly developing regions, is no longer able to meet the demands of modern management. While sensor-based smart manhole cover systems enable remote monitoring through wireless communication, these solutions require large-scale infrastructure upgrades. The high deployment and maintenance costs make them difficult to implement in small and medium-sized cities and rural areas, and they lack an effective linkage mechanism with maintenance resource scheduling.

[0003] In recent years, computer vision-based detection technology has been seen as a key area for overcoming bottlenecks. For example, some studies have used object detection algorithms such as YOLOv8 to locate manhole covers. However, existing methods still have significant drawbacks. First, single-stage detection models lack the accuracy to accurately identify fine-grained defects such as minor damage to manhole covers, leading to frequent false detections and missed detections. Second, the algorithms are also less robust in complex lighting conditions, rainy and snowy conditions, or occlusion scenarios. Therefore, developing a highly accurate and robust method for detecting hidden dangers in manhole covers that overcomes these drawbacks is of great practical significance and application value. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies and provide a method for detecting hidden dangers in manhole covers. This method combines the YOLOv8 object detection model with the ViT classification model, supplemented by model optimization and acceleration strategies, to achieve rapid and accurate identification and classification of manhole cover status.

[0005] The method for detecting hidden dangers of manhole covers provided by the present invention comprises the following steps:

[0006] Step 1: Target detection: Use the YOLOv8 model to quickly scan the input image, locate the manhole cover area, and output the bounding box to achieve efficient target positioning.

[0007] Step 2: Target classification: The image of the manhole cover area detected in step 1 is input into the ViT classification model. Through global feature extraction and fine-grained analysis, the manhole cover status (good, damaged, missing, uncovered, and manhole ring problem) is accurately classified.

[0008] Step 3: Result output and optimization: Integrate the detection and classification results, and output the category label, corresponding bounding box information, and confidence score of each detected target object.

[0009] Furthermore, in step one, the YOLOv8 model uses a dynamic anchor point generation mechanism and multi-scale detection technology to adaptively adjust the step size and density of the detection grid according to the input image size to ensure accurate positioning of manhole covers of different sizes.

[0010] Furthermore, in step 2, the ViT classification model uses transfer learning technology to fine-tune the characteristics of manhole cover hidden dangers to improve its ability to identify subtle defects.

[0011] Furthermore, in step three, the system uses the non-maximum suppression (NMS) algorithm to process the multiple candidate regions output by the target detection model, and integrates the EfficientNMS_TRT plug-in of NVIDIA TensorRT (a software development toolkit for high-performance deep learning inference optimization, referred to as TensorRT) to remove redundant detection boxes and optimize the positioning results.

[0012] Furthermore, in the model training step, data enhancement technology is used to improve the model's adaptability to scene changes.

[0013] Furthermore, the system adopts a combined loss function of label smoothed cross entropy and FocalLoss to effectively alleviate the problem of data category imbalance and improve the ability to identify small sample defects.

[0014] Furthermore, the present invention adopts a model optimization and acceleration strategy, combined with the TensorRT optimization engine, through FP16 (half-precision floating point) precision reasoning and memory optimization technology to shorten the model reasoning time and reduce computing resource consumption.

[0015] The beneficial effects of the present invention are:

[0016] 1. This paper adopts the YOLOv8 model to achieve rapid target detection, and combines it with the ViT classification model for fine-grained classification, which improves the positioning accuracy of manhole cover positions and the accuracy of hidden danger identification, and effectively reduces the false detection rate and missed detection rate in traditional methods.

[0017] 2. Through data enhancement strategies and the introduction of a combined loss function, the present invention enables the model to maintain stable detection and classification performance in scenarios such as complex lighting conditions, rainy and snowy weather, or partial occlusion of manhole covers, thereby improving the robustness and generalization ability of the model.

[0018] 3. This paper uses TensorRT optimization technology to accelerate the YOLOv8 model, reducing model inference time and computing resource consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] The accompanying drawing is a flow chart of the method. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific examples and with reference to the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, but should not be understood as limiting the present invention.

[0022] The present invention provides a method for detecting hidden dangers in manhole covers, the process of which is shown in the accompanying drawings and includes the following steps:

[0023] Step 1: Target detection: Use the YOLOv8 model to quickly scan the input image, locate the target object in the image and give its bounding box.

[0024] Step 2: Target classification: The image area of the target object detected in step 1 is input into the ViT classification model for fine-grained category judgment.

[0025] Step 3: Output the results, output the category label, corresponding bounding box information and confidence score of each detected target object.

[0026] Specifically, the present invention is further described with reference to specific embodiments:

[0027] In the target detection step, the system uses the YOLOv8 target detector to perform target detection on the input image.

[0028] During training, the input image size of the YOLOv8 model was resized to 640×640. The batch size for validation was set to 16, and the batch size for training was dynamically adjusted based on the available GPU (Graphics Processing Unit) memory, set to 32 in the test environment of this example. The confidence threshold was set to 0.25, and the IoU (Intersection over Union) threshold was set to 0.6 to filter out low-confidence and redundant detection boxes in the post-processing stage. The initial learning rate was set to 0.0001, and the final learning rate was also set to 0.0001.

[0029] After target detection is complete, the system uses the NMS algorithm to post-process the preliminary detection results. Specifically, all detection boxes and their corresponding scores are first sorted in descending order. The highest-scoring detection box is then selected and added to the retained set, and redundant boxes with an IoU exceeding a threshold are removed. This process is repeated until all boxes have been processed. To improve the processing efficiency of NMS, this example uses the TensorRT EfficientNMS_TRT plug-in to implement this post-processing step. When configuring this NMS plug-in, the key parameters used are: the IoU threshold is set to 0.65, the confidence threshold is set to 0.25, and the maximum number of output boxes is set to 100.

[0030] To accelerate inference, the YOLOv8 model was converted to the TensorRT engine. Optimizations include: using FP16 for computation, enabling acceleration on GPUs supporting Tensor Cores and reducing model memory usage; using layer fusion to combine convolutional layers, batch normalization layers, and activation layers into a single CUDA (Compute Unified Device Architecture) core for improved computational efficiency; and leveraging TensorRT's memory optimization mechanisms to reduce storage requirements for intermediate results.

[0031] For each target object detected in step 1, the corresponding image region is input into the ViT classification model for refined category determination. This embodiment adopts a transfer learning strategy, and the network input size is adjusted to 224×224. After the backbone network, a custom fully connected layer is added, with 128 hidden units and 5 output categories, corresponding to the several object states that the system needs to identify (good, damaged, missing, uncovered, and well ring problem).

[0032] During model training, in order to improve the generalization and robustness of the model, the system uses a variety of data augmentation techniques, including: horizontal flipping (applied with a probability of 50%), random cropping and padding (applied with a probability of 25%), translation-scaling-rotation transformation (applied with a probability of 25%), channel random shuffling (applied with a probability of 50%), grid distortion or elastic transformation (applied with a probability of 25%), and coarse-grained random deletion (randomly erasing part of the image area, applied with a probability of 50%). These techniques effectively expand the diversity of training data by probabilistically changing the feature distribution of the input image, thereby enhancing the generalization ability of the model and suppressing overfitting to specific input patterns. The system implements two custom loss functions and performs a weighted combination to address the problem of class imbalance and improve the generalization ability of the model:

[0033] 1. FocalLoss: It aims to solve the problem of class imbalance in classification problems and give higher weights to samples that are difficult to classify. Its mathematical expression is: FL(p t )=-α(1-p t ) γ ·log(p t ), where p t is the model's predicted probability for the correct category, α is a balancing factor, which is set to 1 in this embodiment, and γ is a focusing parameter, which is set to 2 in this embodiment to reduce the weight of easy-to-classify samples.

[0034] 2. Label Smoothing Cross Entropy Loss: By converting hard labels into soft labels, the generalization ability of the model is improved. Its mathematical expression is: Among them, y is the true label, is the model prediction, ∈ is the smoothing parameter, which is set to 0.1 in this example, C is the total number of categories, which is set to 5 in this example, and CE is the standard cross entropy loss. The final joint loss function is a weighted combination of the above two losses: For the ViT classification model, during the training process, the system adopts a cosine annealing learning rate scheduling strategy, which gradually reduces the learning rate as the training progresses. Its mathematical expression is: Among them, η t is the learning rate of the tth epoch, η max is the initial learning rate, set to 0.0001, and T is the total number of training epochs, set to 10.

[0035] In the result output step, the system associates the target bounding box after NMS processing in step 1 and the category label output by the ViT classification model in step 2, and outputs the category label, corresponding bounding box information and confidence score of each detected target object. The detection results are output through a defined data structure.

[0036] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art may, within the technical scope disclosed by the present invention, make simple modifications, equivalent replacements or improvements to the technical solutions described in the aforementioned embodiments based on the technical solutions and inventive concepts of the present invention. As long as they do not depart from the spirit and scope of the technical solutions of the present invention, they shall be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be based on the scope of protection of the appended claims.

Claims

1. A method for detecting hidden dangers of manhole covers, characterized in that: The following steps are involved: Step 1: Object detection: Use the YOLOv8 (You Only Look Once version 8) model to locate manhole covers in the input image and output bounding boxes; Step 2: Target classification: Input the image area corresponding to the bounding box into the ViT (Vision Transformer) classification model, and classify the manhole cover status based on global feature extraction and transfer learning fine-tuning; Step 3: Output the result, including the category label, bounding box information and confidence score of the target object.

2. The method according to claim 1, characterized in that In the step 2, the ViT classification model is trained using a weighted combination loss function based on Focal Loss and label smoothed cross entropy loss.

3. The method according to claim 2, characterized in that The loss function of the ViT classification model adopts a weighted combination of FocalLoss (FL) and label smoothed cross entropy loss (LSL): Among them, the weight of label smoothed cross entropy loss is 1 / 6, and the weight of focal loss is 5 / 6.

4. The method according to claim 3, characterized in that The Focal Loss is defined as: FL(p t )=-α(1-p t ) γ ·log(p t ) Among them, the balance factor α is set to 1, the focusing parameter γ is set to 2, and p t Represents the classification probability.

5. The method according to claim 3, characterized in that The label smoothed cross entropy loss is defined as: Among them, y is the true label, is the model prediction, the smoothing parameter ∈ is set to 0.1, the total number of categories C is 5, and CE represents the cross entropy function.

6. The method according to claim 1, characterized in that The method accelerates the inference of the YOLOv8 model by using NVIDIA TensorRT. The inference acceleration includes using the EfficientNMS_TRT plug-in to implement non-maximum suppression operations, fusing convolutional layers, batch normalization layers, and activation layers, and using FP16 half-precision floating-point numbers for quantization optimization.

7. The method according to claim 1, characterized in that In step 2, the ViT classification model adopts a cosine annealing learning rate scheduling strategy, and its mathematical expression is: Among them, η t is the learning rate of the tth epoch, the initial learning rate ηmax is set to 0.0001, and the total number of training epochs T is set to 10.

Images