Pipeline blocking disease identification and rating method and device based on yov8

Through the yolov8-based pipeline sealing disease identification and rating method, the crawling robot collects images and Labelme annotation, combined with the yolov8n-seg.pt model, the automated, precise identification and rating of pipeline sealing disease is achieved, and the label in the traditional method is solved The problems of inconsistency and high labor costs improve detection efficiency and accuracy.

CN119068383AInactive Publication Date: 2024-12-03BEIJING JIANGXINGUANG ELECTROMECHANICAL TECH CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410984021.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-22
Publication Date
2024-12-03
Estimated Expiration
Not applicable · inactive patent

Smart Images

  • Figure CN119068383A_ABST
    Figure CN119068383A_ABST
Patent Text Reader

Abstract

The invention discloses a pipeline plugging disease identification and rating method and device based on the yov8. According to the invention, through image acquisition and analysis, a series of image preprocessing and data enhancement modes, a pipeline disease identification model is trained, automatic identification of each pipeline disease, including detection of the type, number and position of the pipeline disease, is realized, and through extraction of a plugging disease target mask and calculation of the area proportion, the pipeline disease is identified. According to the method, the degree grading of pipeline plugging diseases is realized, so that rapid, automatic, high-accuracy and multi-target identification can be realized, a disease target contour can be more accurately fit, the performance and reliability of pipeline plugging detection are remarkably improved, and the problems of label inconsistency and high labor cost in a traditional method are solved; according to the method, the automation level of data processing and the quality of model training are improved, the target contour can be accurately captured, and the mask area proportion can be calculated, so that finer, more accurate and more automatic disease degree distinguishing is provided, and the accuracy and efficiency of evaluation are remarkably improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence and machine learning, and in particular to a pipeline plugging disease identification and rating method based on yolov8, and a pipeline plugging disease identification and rating device based on yolov8. Background Art

[0002] With the acceleration of urbanization, the health of urban drainage pipe systems, as an important part of urban infrastructure, is directly related to the normal operation of the city and the quality of life of residents. The phenomenon that the water flow is blocked or completely stops flowing inside the pipe due to various reasons will cause pipe blockage. This disease seriously affects the efficiency and function of the drainage system, and may cause a series of problems such as waterlogging, sewage overflow, pipe corrosion and environmental pollution. However, urban drainage pipes are complex, and traditional manual detection methods have many limitations, such as the inability to fully detect the internal conditions of the pipes, safety risks, high labor intensity, and high costs. These problems have led to inefficient pipeline disease identification and rating, which is difficult to meet the growing demand for pipe network detection and maintenance.

[0003] At present, the technologies commonly used at home and abroad, such as pressure distribution method, pressure point method, mass balance method and acoustic wave method, require complex calculations and have large errors when conducting pipeline plugging fault detection.

[0004] Before target detection training, data augmentation is generally used to enrich the training data set samples. However, traditional data augmentation only enhances images. The enhanced images do not have corresponding label updates, and the enhanced images need to be manually re-labeled, which not only increases the workload and time cost, but also causes the annotation information to be inconsistent with the image content, affecting the model training effect.

[0005] When rating pipeline defects, the traditional method evaluates the severity of the defect by calculating the area of ​​the target box. The target box may not completely fit the actual contour of the target, resulting in inaccurate area estimation and loss of detail information, resulting in insufficiently precise severity grading. Summary of the invention

[0006] In order to overcome the defects of the prior art, the technical problem to be solved by the present invention is to provide a pipeline blockage disease identification and rating method based on yolov8, which can quickly, automatically, highly accurately and multi-target recognition, more accurately fit the target contour of the disease, significantly improve the performance and reliability of pipeline blockage detection, solve the problems of inconsistent labels and high labor costs in traditional methods, improve the automation level of data processing and the quality of model training, and can accurately capture the target contour and calculate the mask area ratio, thereby providing a more refined, accurate and automated distinction of the degree of disease, and significantly improving the accuracy and efficiency of the assessment.

[0007] The technical solution of the present invention is: the pipeline plugging disease identification and rating method based on yolov8 comprises the following steps:

[0008] (1) Use a crawling robot to collect images, image size: 1280*1080, acquisition frame rate: 25 frames / second, data rate: above 5683kbps, total bit rate: above 5683kbps, lens: wide-angle lens above 120° or fisheye lens, lens rotation speed is adjustable, video format: AVI format;

[0009] (2) Data preprocessing: label each image in the dataset and use labelme software to label polygon features;

[0010] (3) Based on the polygon label features, the yolov8n-seg.pt segmentation model is used as the weight basis;

[0011] (4) Configure the parameters of the recognition model;

[0012] (5) Training the recognition model;

[0013] (6) reading the pipeline image or pipeline video file and extracting key frames from the pipeline video;

[0014] (7) Call the trained recognition model and use the model to detect the disease type, quantity and location information;

[0015] (8) Extract the mask of the diseased target and count the number of non-zero pixels;

[0016] (9) Estimate the degree of blocked disease based on the percentage of masked area and classify the degree of disease;

[0017] (10) Save the recognition results and output the degree level.

[0018] The present invention trains a pipeline disease recognition model through image acquisition and analysis, and after a series of image preprocessing and data enhancement methods, realizes automatic recognition of various pipeline diseases, including detection of pipeline disease types, quantity and locations. When pipeline blockage diseases are identified, the degree of pipeline blockage diseases is classified by extracting the blockage disease target mask and calculating its area ratio. Therefore, it can quickly, automatically, highly accurately and multi-target recognition, more accurately fit the disease target contour, significantly improve the performance and reliability of pipeline blockage detection, solve the problems of inconsistent labels and high labor costs in traditional methods, improve the automation level of data processing and the quality of model training, and can accurately capture the target contour and calculate the mask area ratio, thereby providing more sophisticated, accurate and automated disease degree distinction, and significantly improving the accuracy and efficiency of evaluation.

[0019] A pipeline plugging disease identification and rating device based on yolov8 is also provided, which includes:

[0020] The image acquisition module is configured to use the crawling robot to acquire images.

[0021] Size: 1280*1080, acquisition frame rate: 25 frames / second, data rate: above 5683kbps, total bit rate: above 5683kbps, lens: wide-angle lens 120° or above or fisheye lens

[0022] Head, lens rotation speed is adjustable, video format: AVI format;

[0023] The data preprocessing module is configured to annotate each image in the dataset using

[0024] Labelme software is used to label polygon features;

[0025] The weighted basis module is configured based on the polygon label features and uses the yolov8n-seg.pt segmentation model as the weight basis;

[0026] A training module, which is configured to identify the parameters of the model and train the recognition model;

[0027] A data extraction module configured to read pipeline image or pipeline video files and extract key frames from pipeline videos;

[0028] The detection module is configured to call the trained recognition model and detect the disease type, quantity and location information through the model;

[0029] The degree classification module is configured to extract the mask of the disease target and count the number of non-zero pixels.

[0030] The number of blocked diseases is estimated according to the proportion of mask area, and the degree of the disease is graded; an output module is configured to maintain the recognition results and output the degree grade. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A flow chart of a pipeline plugging disease identification and rating method based on yolov8 according to the present invention is shown.

[0032] Figure 2 A flowchart showing a specific solution of the pipeline plugging disease identification and rating method based on yolov8 according to the present invention is shown. DETAILED DESCRIPTION

[0033] like Figure 1 As shown, this pipeline plugging disease identification and rating method based on yolov8 includes the following steps:

[0034] (1) Use a crawling robot to collect images, image size: 1280*1080, acquisition frame rate: 25 frames / second, data rate: above 5683kbps, total bit rate: above 5683kbps, lens: wide-angle lens above 120° or fisheye lens, lens rotation speed is adjustable, video format: AVI format;

[0035] (2) Data preprocessing: label each image in the dataset and use labelme software to label polygon features;

[0036] (3) Based on the polygon label features, the yolov8n-seg.pt segmentation model is used as the weight basis;

[0037] (4) Configure the parameters of the recognition model;

[0038] (5) Training the recognition model;

[0039] (6) reading the pipeline image or pipeline video file and extracting key frames from the pipeline video;

[0040] (7) Call the trained recognition model and use the model to detect the disease type, quantity and location information;

[0041] (8) Extract the mask of the diseased target and count the number of non-zero pixels;

[0042] (9) Estimate the degree of blocked disease based on the percentage of masked area and classify the degree of disease;

[0043] (10) Save the recognition results and output the degree level.

[0044] The present invention trains a pipeline disease recognition model through image acquisition and analysis, and after a series of image preprocessing and data enhancement methods, realizes automatic recognition of various pipeline diseases, including detection of pipeline disease types, quantity and locations. When pipeline blockage diseases are identified, the degree of pipeline blockage diseases is classified by extracting the blockage disease target mask and calculating its area ratio. Therefore, it can quickly, automatically, highly accurately and multi-target recognition, more accurately fit the disease target contour, significantly improve the performance and reliability of pipeline blockage detection, solve the problems of inconsistent labels and high labor costs in traditional methods, improve the automation level of data processing and the quality of model training, and can accurately capture the target contour and calculate the mask area ratio, thereby providing more sophisticated, accurate and automated disease degree distinction, and significantly improving the accuracy and efficiency of evaluation.

[0045] Preferably, in step (1), when inspecting the inside of the pipeline, the camera is located at the center of the pipeline to obtain the best viewing angle, and the image clarity is adjusted per meter according to the clarity of the inner wall of the pipeline, and the minimum focal length is maintained to avoid distortion; during the acquisition process, the lighting is adjusted to prevent overexposure, and the travel speed is controlled not to exceed half of the frame rate, at 8 cm / s, to ensure image continuity; for key areas, a surround acquisition is performed, and a stable acquisition time of at least 1 second is maintained after rotation; and auxiliary information in the video file is removed.

[0046] Preferably, in step (2), key frames are extracted from the video, which contain typical features of pipeline blockage diseases, and the blockage diseases are labeled; Labelme software is opened, and the blockage disease image to be labeled is loaded, and the boundary of the disease area is manually drawn using the polygon labeling function of Labelme; for each blockage disease area, a category label FD is input to represent the blockage, so as to distinguish between the image background and the blockage type of disease; and a corresponding JSON label file is generated, the JSON label file includes a labeling version, a flag, a shape array, an image path, an image data, and an image size, which are used to describe the objects in the image and their attributes.

[0047] Preferably, in step (2), while applying the data enhancement method to generate a new enhanced image, the label information is adjusted and converted to ensure that the generated enhanced image is consistent with the updated label file; a data enhancement class is defined to process the image, and predefined enhancement operations such as brightness adjustment, noise addition, random point addition, mirror flipping and translation are performed on the image, and ToolHelper is used to read the original JSON annotation file, update the enhanced image path and data, and then save the updated annotation information to a new JSON file. For each original image, the script generates a specified number of enhanced images and ensures that each enhanced image has a corresponding updated annotation file.

[0048] Preferably, in step (4), the pre-trained weight file yolov8n-seg.pt is selected as the starting point, the training parameters are set to optimize the model performance, and the YOLOv8.0.55 version provided by Ultralytics is used to train the target detection model in the Python 3.9.17 environment with the PyTorch 1.13.1+cu116 framework. The batch size is 6, the training cycle is 100 rounds, the input image size is 640 pixels, the confidence threshold is 0.3, the IoU threshold is 0.75, and the training device is an NVIDIA GeForce RTX 4060 Laptop GPU equipped with 8GB of video memory, running in a CUDA environment and running in a GPU environment.

[0049] Preferably, in step (5), the JSON file generated by detailed annotation and data enhancement is converted into the TXT format required for YOLOv8 training, and the data set is divided into a training set, a validation set and a test set; by testing on the validation set, the hyperparameters of the model are adjusted to monitor its performance on unseen data; the test set provides a standardized method for evaluating the generalization ability of the model and obtaining the performance of the model in an actual application environment.

[0050] Preferably, in step (9), the disease is classified into three levels: light, medium and severe according to the pixel area ratio occupied by the mask in the image, and the original image is saved in a designated classification folder accordingly.

[0051] Preferably, the method further comprises step (11), and the experimental results use recall rate R, precision P and the average precision AP of the two to evaluate the single disease detection performance; in the target detection task, the correctness of the prediction result is evaluated by calculating the intersection over union (IoU) between the predicted bounding box and the true bounding box. IoU is an indicator to measure the degree of overlap between two bounding boxes. Its calculation formula is the area of ​​the overlapping area divided by the area of ​​the union of the two bounding boxes. The IoU threshold is set to 0.75. When the IoU between the predicted box and the true box is higher than this threshold, the prediction is regarded as correctly detecting the positive sample TP. Based on TP, false positive FP and false negative FN, the recall rate and precision are further calculated to comprehensively evaluate the detection performance of the model;

[0052] The formula for recall is:

[0053]

[0054] The formula for accuracy is:

[0055]

[0056] The calculation formula of AP is:

[0057]

[0058] Among them, Precision k and Recall k are the precision and recall at the kth threshold, respectively, and n is the total number of thresholds.

[0059] Specifically, in order to evaluate the performance of YOLOv8 in detecting blocked diseases, all experimental results of this method use recall (R), precision (P) and the average precision (AP) of the two to evaluate the performance of single disease detection. In the target detection task, the correctness of the prediction result is evaluated by calculating the intersection over union (IoU) between the predicted bounding box and the true bounding box. IoU is an indicator to measure the degree of overlap between two bounding boxes. Its calculation formula is the area of ​​the overlapping area divided by the area of ​​the union of the two bounding boxes. An IoU threshold is set. The IoU threshold set by this method is 0.75. When the IoU between the predicted box and the true box is higher than this threshold, the prediction is regarded as a correct detection of the positive sample, that is, the true positive (TP). Based on the number of TPs and other statistical data, such as false positives (FP) and false negatives (FN), key performance indicators such as recall and precision can be further calculated to comprehensively evaluate the detection performance of the model.

[0060] Recall (R): Recall measures the proportion of positive samples detected by the model to all actual positive samples. The formula for recall is:

[0061]

[0062] Among them, TP (True Positives) is the number of samples correctly detected as positive, and FN (False Negatives) is the number of positive samples incorrectly detected as negative.

[0063] Precision (P): Precision measures the proportion of samples that are actually positive among the samples detected as positive by the model. The formula for precision is:

[0064]

[0065] Among them, FP (False Positives) is the number of negative samples that are mistakenly detected as positive.

[0066] Average Precision (AP): Average Precision is the area on the Precision-Recall Curve calculated at different confidence thresholds. The calculation formula for AP is:

[0067]

[0068] Among them, Precision k and Recall k are the precision and recall at the kth threshold, respectively, and n is the total number of thresholds.

[0069] In summary, this method uses precision (P), recall (R), and average precision (AP) to compare the original YOLOv8 model and the model after data enhancement. The results show that the AP of the YOLOv8 model reaches 84%, which performs well and can effectively identify blockage diseases. However, after data enhancement, the AP is further improved to 90%. Therefore, a good detection effect can be achieved for the detection of blockage diseases in urban pipelines.

[0070] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes the steps of the above-mentioned embodiment method, and the storage medium can be: ROM / RAM, disk, CD, memory card, etc. Therefore, corresponding to the method of the present invention, the present invention also includes a pipeline plugging disease identification and rating device based on yolov8, which is usually represented in the form of functional modules corresponding to the steps of the method. The device includes:

[0071] The image acquisition module is configured to use the crawling robot to acquire images.

[0072] Size: 1280*1080, acquisition frame rate: 25 frames / second, data rate: above 5683kbps, total bit rate: above 5683kbps, lens: wide-angle lens 120° or above or fisheye lens

[0073] Head, lens rotation speed is adjustable, video format: AVI format;

[0074] The data preprocessing module is configured to annotate each image in the dataset using

[0075] Labelme software is used to label polygon features;

[0076] The weighted basis module is configured based on the polygon label features and uses the yolov8n-seg.pt segmentation model as the weight basis;

[0077] A training module, which is configured to identify the parameters of the model and train the recognition model;

[0078] A data extraction module configured to read pipeline image or pipeline video files and extract key frames from pipeline videos;

[0079] The detection module is configured to call the trained recognition model and detect the disease type, quantity and location information through the model;

[0080] The degree classification module is configured to extract the mask of the disease target and count the number of non-zero pixels.

[0081] The number of blocked diseases is estimated according to the proportion of mask area, and the degree of the disease is graded; an output module is configured to maintain the recognition results and output the degree grade.

[0082] The actual application scenarios of this method are as follows:

[0083] (1) Using pipeline video and image data collected by the crawler robot, this method can efficiently and accurately perform pipeline defect identification and severity assessment. Through advanced image processing and pattern recognition algorithms, it can accurately analyze the internal conditions of the pipeline, automatically identify plugging diseases, and accurately classify them according to the characteristics and severity of the diseases.

[0084] (2) In addition, the detection and degree classification model of this method can be integrated and deployed into the camera system of the crawling robot to achieve real-time detection of pipeline blockage diseases. This integrated deployment allows the robot to instantly transmit analysis data back to the control center while performing the detection task. However, this real-time data processing and transmission method places higher requirements on the computing hardware performance of the robot and the real-time performance of the algorithm. It is necessary to ensure the rapid response and efficient operation of the detection system to meet the needs of on-site detection.

[0085] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the protection scope of the technical solution of the present invention.

Claims

1. The pipeline plugging disease identification and rating method based on yolov8 is characterized by: It includes the following steps: (1) Use a crawling robot to collect images, image size: 1280*1080, acquisition frame rate: 25 frames / second, data rate: above 5683kbps, total bit rate: above 5683kbps, lens: wide-angle lens above 120° or fisheye lens, lens rotation speed is adjustable, video format: AVI format; (2) Data preprocessing: label each image in the dataset and use labelme software to label polygon features; (3) Based on the polygon label features, the yolov8n-seg.pt segmentation model is used as the weight basis; (4) Configure the parameters of the recognition model; (5) Training the recognition model; (6) reading the pipeline image or pipeline video file and extracting key frames from the pipeline video; (7) Call the trained recognition model and use the model to detect the disease type, quantity and location information; (8) Extract the mask of the diseased target and count the number of non-zero pixels; (9) Estimate the degree of blocked disease based on the percentage of masked area and classify the degree of disease; (10) Save the recognition results and output the degree level.

2. The pipeline plugging disease identification and rating method based on yolov8 according to claim 1 is characterized by: In the step (1), when inspecting the interior of the pipeline, the camera is located at the center of the pipeline to obtain the best viewing angle, and the image clarity is adjusted per meter according to the clarity of the inner wall of the pipeline, and the minimum focal length is maintained to avoid distortion; during the acquisition process, the lighting is adjusted to prevent overexposure, and the travel speed is controlled not to exceed half of the frame rate, at 8 cm / s to ensure image continuity; for key areas, a surround acquisition is performed, and a stable acquisition time of at least 1 second is maintained after rotation; Remove auxiliary information from video files.

3. The pipeline plugging disease identification and rating method based on yolov8 according to claim 2 is characterized by: In the step (2), key frames are extracted from the video, which contain typical features of pipeline blockage diseases, and the blockage diseases are labeled; Labelme software is opened, and the blockage disease image to be labeled is loaded, and the boundary of the disease area is manually drawn using the polygon labeling function of Labelme; for each blockage disease area, a category label FD is input to represent the blockage, so as to distinguish between the image background and the blockage type of disease; and a corresponding JSON label file is generated, the JSON label file includes a labeling version, a flag, a shape array, an image path, an image data, and an image size, which is used to describe the objects in the image and their attributes.

4. The pipeline plugging disease identification and rating method based on yolov8 according to claim 3 is characterized by: In the step (2), while applying the data enhancement method to generate a new enhanced image, the label information is adjusted and converted to ensure that the generated enhanced image is consistent with the updated label file; a data enhancement class is defined to process the image, and predefined enhancement operations such as brightness adjustment, noise addition, random point addition, mirror flipping and translation are performed on the image, and ToolHelper is used to read the original JSON annotation file, update the enhanced image path and data, and then save the updated annotation information to a new JSON file. For each original image, the script generates a specified number of enhanced images and ensures that each enhanced image has a corresponding updated annotation file.

5. The pipeline plugging disease identification and rating method based on yolov8 according to claim 4 is characterized in that: In step (4), the pre-trained weight file yolov8n-seg.pt is selected as the starting point, and the training parameters are set to optimize the model performance. The YOLOv8.0.55 version provided by Ultralytics is used to train the target detection model in the Python 3.9.17 environment with the PyTorch1.13.1+cu116 framework. The batch size is 6, the training cycle is 100 rounds, the input image size is 640 pixels, the confidence threshold is 0.3, the IoU threshold is 0.75, and the training device is NVIDIA GeForce RTX 4060Laptop GPU, equipped with 8GB video memory, running in CUDA environment, running in GPU environment.

6. The pipeline plugging disease identification and rating method based on yolov8 according to claim 5 is characterized by: In the step (5), the JSON file generated by the detailed annotation and data enhancement is converted into the TXT format required for YOLOv8 training, and the data set is divided into a training set, a validation set, and a test set; by testing on the validation set, the hyperparameters of the model are adjusted to monitor its performance on unseen data; The test set provides a standardized method for evaluating the generalization ability of the model and obtaining the performance of the model in the actual application environment.

7. The pipeline plugging disease identification and rating method based on yolov8 according to claim 6 is characterized by: In the step (9), the disease is classified into three levels: light, medium and severe according to the pixel area ratio occupied by the mask in the image, and the original image is saved in a designated classification folder accordingly.

8. The pipeline plugging disease identification and rating method based on yolov8 according to claim 7 is characterized by: The method further comprises step (11), the experimental results use recall rate R, precision P and the average precision AP of the two to evaluate the single disease detection performance; In the object detection task, the correctness of the prediction result is evaluated by calculating the intersection over union (IoU) between the predicted bounding box and the true bounding box. IoU is an indicator to measure the degree of overlap between two bounding boxes. Its calculation formula is the area of ​​the overlapping area divided by the area of ​​the union of the two bounding boxes. The IoU threshold is set to 0.

75. When the IoU between the predicted box and the true box is higher than this threshold, the prediction is considered to be a correct detection of the positive sample TP. Based on TP, false positive FP and false negative FN, the recall rate and precision are further calculated to comprehensively evaluate the detection performance of the model. The formula for recall is: The formula for accuracy is: The calculation formula of AP is: Among them, Precision k and Recall k are the precision and recall at the kth threshold, respectively. n is the total number of thresholds.

9. Pipeline plugging disease identification and rating device based on yolov8, characterized by: It includes: Image acquisition module, which is configured to use the crawling robot to collect images, image size: 1280*1080, acquisition frame rate: 25 frames / second, data rate: above 5683kbps, total bit rate: above 5683kbps, lens: wide-angle lens above 120° or fisheye lens, lens rotation speed is adjustable, video format: AVI format; The data preprocessing module is configured to annotate each image in the dataset and perform polygon feature annotation using labelme software; The weighted basis module is configured based on the polygon label features and uses the yolov8n-seg.pt segmentation model as the weight basis; A training module, which is configured to identify the parameters of the model and train the recognition model; A data extraction module configured to read pipeline image or pipeline video files and extract key frames from pipeline videos; The detection module is configured to call the trained recognition model and detect the disease type, quantity and location information through the model; The degree grading module is configured to extract the mask of the disease target, count the number of non-zero pixels, estimate the degree of the blocked disease according to the mask area ratio, and grade the degree of the disease; The output module is configured to store the recognition result and output the degree level.