Construction method of drainage pipeline defect data set standardization process

Abstract

The invention discloses a construction method of a drainage pipeline defect data set standardization process, which comprises the following steps: video collection: collecting drainage pipeline shot videos and summarizing other data, and carrying out video frame extraction and image cleaning: reading drainage pipeline detection videos by utilizing a video processing library; basic information and defect pictures of the video are obtained and cleaned to obtain pictures meeting requirements. Image labeling and classification: labeling the screened defect images by using image labeling tools such as X-labeling and the like to generate a labeling file in a JSON format, and deep learning model training: constructing a deep learning model based on YOLOv8, and performing training and optimization through a standardized data set. The method has the advantages that manual intervention and misjudgment can be reduced, the data interactivity is enhanced, the drainage pipeline defect detection efficiency and accuracy are improved, and the application range is wider.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method for constructing a standardized process for a drainage pipe defect data set. Background Art

[0004] Existing research focuses on classifying, detecting, and segmenting sewer defects or other components in a given image. Many researchers use data sets from different countries, follow different inspection guidelines, or rely on experience to determine whether a sample meets the test conditions, without developing a scientific and reasonable sample library construction process. Among them, a series of processes such as sample screening, sample quantity, sample quality, annotation methods, and evaluation are rarely mentioned. Overall, the practicality and applicability of the automated recognition system for comprehensive defect detection and completeness are becoming increasingly important in practical applications.

[0005] Based on this, it is urgent to establish a set of standards for the construction of drainage pipe defect sample libraries, standardize the construction process of defect sample libraries and the quality control operations of each node, realize the standardization of sample libraries, improve the interactivity between sample libraries, and thus improve the training effect of algorithm models. The construction method of a drainage pipe defect dataset standardization process is not only of theoretical significance, but also of practical engineering application significance. Summary of the invention

[0006] In view of the defects of the existing database construction method, the present invention provides a method for constructing a standardized process of a drainage pipe defect data set, thereby improving the quality and standardization of the data set, thereby improving the accuracy and stability of the defect detection model.

[0007] In order to achieve the above invention object, the technical solution adopted by the present invention is as follows:

[0008] A method for constructing a standardized process for a drainage pipe defect dataset includes the following steps:

[0009] Step 1: Video collection:

[0010] Collect videos and summarize other information; collect existing information from urban drainage network survey and inspection archives and drainage geographic information system database, drainage pipeline inspection and evaluation data, operation and maintenance data in the business management process, collect field work equipment information in pipeline network inspection and evaluation projects, collect information on pipeline types, pipeline materials, pipeline defects and evaluation rules in drainage networks; classify and archive the collected videos, and check and update the sample library in a timely manner later;

[0011] The classification and archiving in step 1 includes establishing a pipeline attribute information table, which includes an image data source information table and a defect sample information table; the image data source information table records various types of field inspection images for summarizing massive inspection images; the defect sample information table records sample images, sample labels, pipeline types, pipe materials, pipe diameters and defect information in different inspection images for subsequent video frame extraction algorithm recognition.

[0012] Step 2: Video frame extraction and image cleaning:

[0013] Use the video processing library to read the drainage pipe inspection video, obtain the basic information of the video, determine the frame extraction interval, calculate the frame extraction interval according to the required number of frames and the total number of video frames, and extract image frames at the interval; then screen the extracted images and select the defective images with the best quality as samples; check the image clarity and shooting angle to ensure that the quality of the sample image meets the requirements;

[0014] Step 3: Image annotation and classification:

[0015] Use image annotation tools to annotate the screened defect images and generate annotation files in JSON format; classify the annotated images according to the defect types in the regulations and establish a standardized defect data set;

[0016] Step 3:

[0017] The sample annotation method is used to obtain defect targets from multi-source pipeline image data and obtain the label tags required by the detection model; the annotation principles are as follows:

[0018] Whether it is object detection or semantic segmentation, the annotated pixels are close to the edge pixels of the annotated object;

[0019] For defective images with slightly poor quality, blurred grayscale pixels will appear on the defect edges. When annotating the image, avoid annotating blurred pixels within the annotation range.

[0020] When there is a part that does not belong to the defect inside the marked defect, the part that does not belong to the defect itself will be deducted;

[0021] X-anylabeling is selected for labeling after comparing its image classification, target detection, image segmentation functions, labeling type, export format, memory usage, and support for multi-person collaboration. The labeled images are screened, and samples containing a single defect, samples containing multiple categories and multiple defects, and normal samples are separated from the images that meet the requirements and placed in their respective corresponding folders. Sample quality inspection is then performed: real-time quality inspection, sampling quality inspection, and full-sample quality inspection are performed on the labeled and classified images. Real-time quality inspection means that labeling is carried out in stages. After the sample labeling of a stage is completed, the quality inspector will inspect the labeling results of the data of that stage according to the labeling principles. If the labeled samples meet the specifications, they will be placed in the sample library completed by the labeler. If the labeling is unqualified, the labeler will immediately rework and correct the labeling. Full-sample quality inspection is required, and all labeled samples in the sample library will be inspected to finally build a complete sample library.

[0022] Step 4: Deep learning model training:

[0023] Build a deep learning model based on YOLOv8, train and optimize it through standardized datasets.

[0024] Compared with the prior art, the advantages of the present invention are:

[0025] Problems such as preventing urban waterlogging, ground collapse and abnormal sewage concentration have been effectively improved. This is a method for constructing a standardized process for a drainage pipe defect sample library, which can accurately and quickly determine the type of drainage pipe network defects. Through the standardized defect data set construction process, the data interactivity is enhanced, providing a solid foundation for further algorithm optimization and application. At the same time, the present invention also provides strong support for the maintenance and management of urban drainage pipe networks, and to a certain extent fills the gap in this field in China, providing a new technical path for the field of pipeline defect detection, and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 Schematic diagram of the construction process of the drainage pipe defect data set standardization according to an embodiment of the present invention;

[0027] Figure 2 It is a schematic diagram of the sample cleaning and classification process of the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and examples.

[0029] like Figure 1As shown, a method for constructing a standardized process of a drainage pipe defect dataset includes the following steps:

[0030] Step 1, video collection: collect videos and summarize other information. Collect existing information such as urban drainage network survey and inspection archives and drainage geographic information system database, drainage pipeline inspection and evaluation data, operation and maintenance data in the business management process, collect field work equipment information in pipeline network inspection and evaluation projects, and collect information such as pipeline types, pipeline materials, pipeline defects and evaluation rules in the drainage network. The collected videos should be classified and archived according to Table 1, and the sample library should be checked and updated in time later.

[0031] Table 1 Sample library related attribute information table

[0032]

[0033] The image data source information table mainly records various types of field inspection images, effectively summarizing massive inspection images to prevent subsequent confusion. The defect sample information table mainly records sample images, sample labels, pipeline types, pipe materials, pipe diameters and defect information in different inspection images to facilitate subsequent video frame extraction algorithm recognition.

[0034] Step 2, video frame extraction and image cleaning: Use the video processing library (such as OpenCV) to read the drainage pipe inspection video and obtain the basic information of the video (such as the total number of frames, frame rate, etc.). Determine the frame extraction interval, calculate the frame extraction interval according to the required number of frames and the total number of video frames, and extract image frames at the interval. Then screen the extracted images and select the defective images with the best quality as samples. Check the image clarity and shooting angle to ensure that the quality of the sample image meets the requirements.

[0035] The frame extraction method has not been improved: open the collected defective video, browse the entire video and find the time node where the defect appears in the video to record. Then record the frame extraction information including food name, frame positioning index, etc. in the frame extraction table. Then open the main frame extraction program, select the relevant file, fill in the column name, video path, frame extraction image storage path, and click the operation to successfully extract the frame.

[0036] However, the above frame extraction method takes a long time and the time nodes are difficult to control, so the frame extraction method is improved according to the frame extraction requirements.

[0037] The improved frame extraction method uses code to extract frames. In the code, you need to fill in the folder path where the frame extraction video is located, the folder path where the new frame extraction picture is stored, the format of the frame extraction picture, and the number of frame extraction pictures per second. Fill the improved code into the running software to get the picture after frame extraction. The relevant personnel will clean the picture later, such as Figure 2 shown.

[0038] Image cleaning involves the control of image quality. In this cleaning process, there are certain requirements for image quality, image quantity, and image viewing angle. Image quality: It can clearly distinguish the pipeline morphology and defect symptoms. The defect symptoms shooting angle is required to include all defects. The shooting clarity meets the requirements, there is no fog, no mirror blur, and it can be marked without image processing. The image pixel is preferably 1920*1080, so as to maintain a high clarity in the subsequent image processing process. In terms of the number of images, in order to make the model learn and identify better, the number of various defect images in the sample library should be evenly distributed to avoid a long-tail distribution. For the image viewing angle, sedimentation, residual wall dam roots, scum, deformation, foreign body penetration, branch pipe blind connection, misalignment, obstacles, scaling, tree roots, rupture, corrosion, leakage, interface material shedding, and disconnection adopt the front view angle, scaling, tree roots, rupture, corrosion, leakage, interface material shedding, and disconnection can also adopt the side view and oblique view angle.

[0039] Step 3: Image annotation and classification: Use image annotation tools such as X-anylabeling to annotate the screened defect images and generate annotation files in JSON format. Classify the annotated images according to the defect types in the regulations and establish a standardized defect dataset.

[0040] In order to meet the specific task requirements of target detection, semantic segmentation, etc., the sample annotation method is used to obtain defect targets from multi-source pipeline image data and obtain the label tags required by the detection model. The annotation principles are as follows:

[0041] Whether it is target detection (box selection) or semantic segmentation (stroke), the annotated pixels should be close to the edge pixels of the annotated object. The closer the annotation, the higher the quality and the greater the relative difficulty of annotation.

[0042] For defective images of slightly poor quality, blurred grayscale pixels will appear on the edges of the defects. When annotating the images, it is necessary to avoid annotating blurred pixels within the annotation range as much as possible.

[0043] When there is a part inside the marked defect that does not belong to the defect, the part that does not belong to the defect itself should be deducted.

[0044] After comparing the image classification, target detection, image segmentation functions, annotation types, export formats, memory usage, and support for multi-person collaboration, we selected the X-anylabeling annotation tool for annotation. We screened the annotated images and separated the samples containing a single defect, the samples containing multiple categories and multiple numbers of defects, and the normal samples from the images that met the requirements and put them into their respective folders.

[0045] Step 4: Deep learning model training: Build a deep learning model based on YOLOv8 and train and optimize it through a standardized dataset.

[0046] In the present invention, the construction of a drainage pipe defect data set is a key technology. This key technology requires the use of a video frame extraction algorithm and X-anylabeling annotation software, which play an important role in the video processing and data annotation process. The video frame extraction algorithm is the process of extracting representative static frames from a continuous video stream, and is commonly used in video analysis, monitoring, behavior recognition and other fields. Key technologies include frame difference method, optical flow method and frame extraction method based on deep learning. The frame difference method detects the motion area by calculating the pixel difference between adjacent frames and extracts frames with significant changes. This method is simple and computationally efficient, but sensitive to changes in illumination. The optical flow rule tracks moving objects by estimating the speed and direction of pixel movement in an image sequence, which can capture motion details more accurately, but has a high computational complexity. The frame extraction method based on deep learning uses models such as convolutional neural networks (CNN) and long short-term memory networks (LSTM) to automatically identify and extract key frames by learning features in a large amount of labeled data. This method has high robustness and accuracy, but requires a lot of computing resources and data support. X-anylabeling is an efficient labeling tool that combines artificial intelligence and human-computer interaction technology for accurate labeling of videos and images. The key technologies of the software include automatic labeling, intelligent auxiliary labeling, and multi-mode labeling. Automatic labeling uses deep learning models to perform preliminary analysis and labeling of video content, greatly reducing the workload of manual labeling. Intelligent auxiliary labeling helps users quickly complete labeling tasks by providing intelligent prompts and automatic completion functions. Multi-mode labeling supports a variety of labeling forms, such as box labeling, point labeling, line labeling, etc., to meet different application needs.

[0047] The present invention has high advantages in terms of cost, manpower and material resources, operational difficulty, accurate identification and rapid detection. Compared with previous pipeline defect detection methods, the present invention can accurately and quickly determine the type of drainage network defects, and through the standardized defect data set construction process, it enhances the interactivity of the data, providing a solid foundation for further algorithm optimization and application. For example, in the process of video frame extraction, there is no need to manually watch the video to find the defective node, so the algorithm can be used to read the defective frame in the video for positioning, making the video frame extraction more efficient and the positioning more accurate.

[0048] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the implementation methods of the present invention, and should be understood that the protection scope of the present invention is not limited to such special statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A method for constructing a standardized process for a drainage pipe defect dataset, characterized in that: The following steps are involved: Step 1: Video collection: Collect videos and summarize other information; collect existing information from urban drainage network survey and inspection archives and drainage geographic information system database, drainage pipeline inspection and evaluation data, operation and maintenance data in the business management process, collect field work equipment information in pipeline network inspection and evaluation projects, collect information on pipeline types, pipeline materials, pipeline defects and evaluation rules in drainage networks; classify and archive the collected videos, and check and update the sample library in a timely manner later; Step 2: Video frame extraction and image cleaning: Use the video processing library to read the drainage pipe inspection video, obtain the basic information of the video, determine the frame extraction interval, calculate the frame extraction interval according to the required number of frames and the total number of video frames, and extract image frames at the interval; then screen the extracted images and select the defective images with the best quality as samples; check the image clarity and shooting angle to ensure that the quality of the sample image meets the requirements; Step 3: Image annotation and classification: Use image annotation tools to annotate the screened defect images and generate annotation files in JSON format; classify the annotated images according to the defect types in the regulations and establish a standardized defect data set; Step 4: Deep learning model training: Build a deep learning model based on YOLOv8, train and optimize it through standardized datasets.

2. The method for constructing a standardized process of a drainage pipe defect dataset according to claim 1 is characterized by: The classification and archiving in step 1 includes establishing a pipeline attribute information table, which includes an image data source information table and a defect sample information table; The image data source information table records various types of field inspection images for summarizing massive inspection images; the defect sample information table records sample images, sample labels, pipeline types, pipe materials, pipe diameters and defect information in different inspection images for subsequent video frame extraction algorithm recognition.

3. The method for constructing a standardized process of a drainage pipe defect dataset according to claim 1, characterized in that: Step 3: The sample annotation method is used to obtain defect targets from multi-source pipeline image data and obtain the label tags required by the detection model; the annotation principles are as follows: Whether it is object detection or semantic segmentation, the annotated pixels are close to the edge pixels of the annotated object; For defective images with slightly poor quality, blurred grayscale pixels will appear on the defect edges. When annotating the image, avoid annotating blurred pixels within the annotation range. When there is a part that does not belong to the defect inside the marked defect, the part that does not belong to the defect itself will be deducted; After comparing the image classification, target detection, image segmentation functions, annotation type, export format, memory usage, and support for multi-person collaboration, X-anylabeling was selected for annotation. Screen the annotated images, and separate samples containing a single defect, samples containing multiple categories and multiple quantities of defects, and normal samples from the images that meet the requirements and put them into their respective corresponding folders; then conduct sample quality inspection: conduct real-time quality inspection, sampling quality inspection and full-sample quality inspection on the annotated and classified images. Real-time quality inspection means that the annotation is carried out in stages. When the sample annotation of a stage is completed, the quality inspector will check the annotation results of the data of that stage according to the annotation principles. If the annotated sample meets the specifications, it will be placed in the sample library completed by the annotator. If the annotation is unqualified, the annotator will be asked to rework and correct the annotation immediately; then conduct full-sample quality inspection on all the annotated samples in the sample library, and finally build a complete sample library.

Images