An intelligent quantification method for pipeline defects integrating vision and laser

Abstract

The present invention discloses an intelligent quantification method for pipeline defects that integrates vision and laser. The method includes acquiring detection data collected by a pipeline crawling robot, performing real-time calculations based on the detection data, and obtaining a panoramic view and a chromatogram of the pipeline; constructing a pipeline defect data set based on the panoramic view and the chromatogram, classifying and annotating the pipeline defect data set, establishing a pipeline defect database, and dividing the images in the pipeline defect database into a training set, a validation set, and a test set in proportion; carrying a pipeline defect segmentation model based on a feature pyramid, training the pipeline defect segmentation model, and detecting the panoramic view based on the pipeline defect segmentation model. When a defect is detected on the panoramic view, the laser radar contour data corresponding to the defect and the pipeline detection image data are associated to interpret and quantify the defect, provide detailed data for pipeline operation and maintenance management and repair, reduce the labor cost of drainage pipe detection engineering, and improve detection efficiency.

Description

Technical Field

[0001] The present invention relates to the interdisciplinary technical field of underground drainage pipeline engineering and computer vision, and in particular to an intelligent quantification method for pipeline diseases integrating vision and laser. Background Art

[0002] Drainage pipes are the lifeline of the city. With the corrosion and erosion of sewage and the long-term effects of natural conditions, various defects will appear inside the pipes. These defects gradually develop and have a great impact on the surrounding soil, buildings and the safety of vehicles and pedestrians. Therefore, quantitative inspections should be carried out regularly. At present, CCTV inspections generally use a rotating lens with a zoom factor of 10. The pipeline crawling robot uses a forward photography method when moving forward. When encountering defects, the pipeline crawling robot stops and uses a lateral photography method to scan the defects in detail. After the inspection is completed, the internal staff will watch the video and interpret the defects in combination with the on-site record sheet, and then output the inspection report. In this process, the level of the defect is qualitatively confirmed by the reader. The object of the reader's interpretation is the video.

[0003] The existing CCTV inspection of drainage pipes requires manual monitoring of the pipeline crawling robot throughout the process, stopping the pipeline crawling robot at the pipeline defect and looking around the pipeline defect part. The operation is cumbersome, the inspection efficiency is low, and it is easy to miss the inspection. After obtaining the pipeline inspection video, the defects need to be graded and classified manually. This work relies entirely on personal experience and does not effectively quantify the pipeline defect parts. On the other hand, there are also problems of insufficient accuracy and inability to quantify defects in the process of judging defects by artificial intelligence. The traditional video inspection method uses mobile inspection equipment to replace personnel entering the pipeline, reducing the inspection risk. However, this method relies on the subjective judgment of professionals, which is time-consuming and labor-intensive. In addition, repeated workloads and subjective evaluations lead to defect neglect and coding errors, reducing the detection accuracy.

[0004] Patent document with application number CN202111439363.1 discloses a drainage pipe detection device based on 3D laser, which uses a driving trolley to drive a high-definition camera and a 3D laser scanner to detect drainage pipes. However, pipeline defects still rely on manual interpretation, and intelligent quantification of pipeline defects cannot be achieved. Therefore, it is necessary to propose an intelligent quantification method for pipeline defects that integrates vision and laser to solve the above problems. Summary of the invention

[0005] The purpose of the present invention is to provide a pipeline defect intelligent quantification method integrating vision and laser, so as to solve the problem that the prior art cannot realize the intelligent quantification of pipeline defects.

[0006] The present invention provides a pipeline disease intelligent quantification method integrating vision and laser, comprising:

[0007] Acquire detection data collected by the pipeline crawling robot, the detection data including pipeline detection image data, laser radar contour data, travel mileage data and pipeline crawling robot posture data, perform real-time calculation based on the detection data to obtain a panoramic view and chromatogram of the pipeline;

[0008] Construct a pipeline defect data set according to the panoramic image and the chromatogram, classify and annotate the pipeline defect data set, establish a pipeline defect database, and divide the images in the pipeline defect database into a training set, a validation set, and a test set in proportion;

[0009] Equipped with a pipeline defect segmentation model based on a feature pyramid, the pipeline defect segmentation model is initialized using a transfer learning method, the pipeline defect segmentation model is trained based on the training set, the validation set, and the test set, the loss value and the accuracy change curves during the pipeline defect segmentation model training process are compared, and the optimal hyperparameters are adjusted and found;

[0010] The panoramic image is inspected based on the pipeline defect segmentation model. When a defect is detected on the panoramic image, the laser radar contour data corresponding to the defect and the pipeline inspection image data are associated to interpret and quantify the defect.

[0011] Receive real-time detection data sent back by the pipeline crawling robot during the field test, calculate a real-time panoramic image based on the real-time detection data, perform pipeline defect detection on the real-time panoramic image based on the pipeline defect segmentation model, and segment and quantify the pipeline defects.

[0012] Furthermore, the method for collecting detection data by the pipeline crawling robot includes:

[0013] After the pipeline crawling robot enters the pipeline, before officially starting the inspection, the parameters are determined, and the parameter determination includes calibrating the laser radar profile data, inputting the pipeline size, and selecting the appropriate expansion circle position and expansion width;

[0014] After the parameters are determined, a test is conducted to observe whether the panoramic expansion image is unclear or there is an obvious blind phenomenon;

[0015] If the panoramic expansion image is unclear or there is an obvious blinds phenomenon, adjust the parameters;

[0016] During the inspection process, if an impassable obstacle is encountered, the current image will be fully expanded so that the defect can be reflected in the panoramic image, and the defect can be interpreted through the panoramic image.

[0017] Furthermore, the laser radar profile data is calibrated, including:

[0018] Obtaining the actual pipe diameter measured manually and the diameter value displayed on the acquisition system of the pipe crawling robot;

[0019] Determine whether the actual pipe diameter measured manually is the same as the diameter value displayed on the collection system of the pipe crawling robot;

[0020] If the actual pipe diameter measured manually is different from the diameter value displayed on the collection system of the pipeline crawling robot, click the calibration button of the collection system and enter the difference between the two for calibration;

[0021] After the calibration is completed, the acquisition system prompts the recommended maximum walking speed of the pipeline crawling robot.

[0022] Furthermore, during the inspection process, if an impassable obstacle is encountered, the current image will be fully expanded so that the defect can be reflected in the panoramic view, and then in the step of defect interpretation through the panoramic view, the panoramic view will present the entire interior of the pipeline in the form of a picture. During the inspection process, there is no stopping, no rotation, no looking around, no adjustment of magnification, and no pausing for observation.

[0023] Furthermore, during the field test of the pipeline crawling robot, the water level inside the pipeline did not exceed 5%.

[0024] Furthermore, before the on-site test of the pipeline crawling robot, the light source intensity of the pipeline crawling robot is adjusted to ensure clear video images.

[0025] Furthermore, a pipeline defect data set is constructed based on the panoramic image and the chromatogram, and the pipeline defect data set is classified and labeled to establish a pipeline defect database, including:

[0026] Screen the images in the pipeline defect database, remove blurred images and images without defects, and obtain pipeline defect images;

[0027] Classify and label the pipeline defect image according to defect categories, label the background area pixels of the image as 0, and label the defect area pixels as 1, 2, 3, etc. according to the categories;

[0028] The calibrated pipeline defect image in png format is combined with the generated annotation information in json format, converted into a defect binary map through python code, and the pipeline defect image and the binary map are matched one by one according to the name to establish a pipeline defect image database.

[0029] Furthermore, when the panoramic image is inspected based on the pipeline defect segmentation model and a defect is detected on the panoramic image, the lidar contour data corresponding to the defect and the pipeline inspection image data are associated to perform the defect interpretation and quantification steps. The rupture size, deformation rate, and deposition amount can be measured on the lidar contour data, and the crack width and defect length can be confirmed on the plane unfolded image.

[0030] The beneficial effects of the present invention are as follows: The intelligent quantification method for pipeline defects that integrates vision and laser provided by the present invention can replace the traditional CCTV detection method and ensure that the pipeline crawling robot obtains all pipeline data after one travel. According to the actual size of the pipeline and the detected defects, the digital software quantitatively analyzes the type and grade of pipeline defects, internal siltation, pipeline undulation and deformation, and other information, and establishes a three-dimensional pipeline model to achieve 3D imaging output of the detection results. It provides detailed data for pipeline operation and maintenance management and repair, reduces the labor cost of drainage pipeline detection projects, improves detection efficiency, and provides an important basis for the comprehensive survey of urban pipeline networks and the management of pipeline network information data. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0032] Figure 1 It is a flow chart of an intelligent quantification method for pipeline defects integrating vision and laser according to the present invention;

[0033] Figure 2 This is a panoramic diagram of the pipeline. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described in conjunction with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. The technical solutions provided by the embodiments of the present invention are described in detail below in conjunction with the drawings.

[0035] See also Figure 1 and Figure 2 The present invention provides a pipeline disease intelligent quantification method integrating vision and laser, comprising:

[0036] S101, acquiring detection data collected by the pipeline crawling robot, wherein the detection data includes pipeline detection image data, laser radar contour data, mileage data, and pipeline crawling robot posture data, and performing real-time calculations based on the detection data to obtain a panoramic view and a chromatogram of the pipeline.

[0037] Specifically, the method for collecting detection data by the pipeline crawling robot includes:

[0038] After the pipeline crawling robot enters the pipeline, before officially starting the inspection, the parameters are determined. The parameter determination includes calibrating the laser radar profile data, inputting the pipeline size, and selecting the appropriate expansion circle position and expansion width.

[0039] After the parameters are determined, a test is performed to observe whether the panoramic expansion image is unclear or there is an obvious blind phenomenon; if the panoramic expansion image is unclear or there is an obvious blind phenomenon, the parameters are adjusted.

[0040] During the inspection process, if an impassable obstacle is encountered, the current image will be fully expanded so that the defect can be reflected in the panoramic image, and the defect can be interpreted through the panoramic image.

[0041] The panoramic view presents the whole interior of the pipeline in the form of a picture. During the inspection process, there is no stopping, no rotation, no looking around, no adjustment of magnification and no stopping for observation.

[0042] The LiDAR profile data is related to the environment. In different environments, the measured values ​​will have slight deviations. Calibration is required before the formal detection. Calibration can ensure more accurate profile data, and the subsequent chromatogram output and deformation calculation will be more accurate.

[0043] Calibrate the LiDAR profile data, including:

[0044] Obtaining the actual pipeline diameter measured manually and the diameter value displayed on the collection system of the pipeline crawling robot; determining whether the actual pipeline diameter measured manually and the diameter value displayed on the collection system of the pipeline crawling robot are the same;

[0045] If the actual pipe diameter measured manually is different from the diameter value displayed on the collection system of the pipeline crawling robot, click the calibration button of the collection system and enter the difference between the two for calibration;

[0046] After calibration, the acquisition system prompts the recommended maximum walking speed of the pipeline crawling robot. The pipeline crawling robot needs to crawl at a speed lower than this speed to ensure the detection effect.

[0047] The detection of pipeline crawling robots has the following requirements:

[0048] During vertical photography, the image should remain horizontal and the shooting angle and focal length should not be changed in the middle. During the movement of the pipeline crawling robot, the zoom function of the camera lens should not be used. When the zoom function is used, the pipeline crawling robot should remain stationary. When the pipeline crawling robot is required to continue moving, the focal length of the lens should be restored to the shortest focal length position first. When shooting sideways, the pipeline crawling robot should stop moving and change the shooting angle and focal length to obtain the best image. During the pipeline inspection process, the video footage should not have the phenomenon of screen pause, intermittent recording, and screen editing. When defects are found during the inspection process, the pipeline crawling robot should be stopped for at least 10 seconds at a position where the defects can be fully resolved to ensure that the captured image is clear and complete.

[0049] Specifically, the pipeline crawling robot has a panoramic laser quantitative detection device, and the basic performance of the panoramic laser quantitative detection device is:

[0050] The camera lens is a panoramic lens with a field of view of more than 170°; the laser radar probe has a field of view of 360° and can completely scan the cross-sectional profile of the pipeline; the pipeline crawling robot has the functions of forward, backward, neutral, speed change, and anti-rollover, and the wheel diameter and wheel spacing can be replaced or adjusted according to the size of the pipeline being inspected; the main controller has the function of synchronously displaying the date, time, pipe diameter, and distance traveled in the pipeline on the monitor, and can perform data processing; the light intensity can be adjusted steplessly. In order to achieve the best detection effect, the water level inside the pipeline does not exceed 5%. The influence of the light source on the panoramic video is very critical. Therefore, before starting the detection, the light intensity must be carefully adjusted to ensure a clear video image.

[0051] Specifically, panoramic inspection uses a special lens. The distance of the object from the lens is different, and the degree of deformation of the image is different. Therefore, during inspection, the moving track of the camera lens should be on the central axis of the pipeline, and the deviation should not be greater than 10% of the pipe diameter. When inspecting pipelines with special shapes, the camera position should be adjusted appropriately to obtain the best image.

[0052] Specifically, during the detection process, if the image quality deteriorates and is not ideal, the crawler should be stopped, the light source should be adjusted to ensure the best image effect, and then the detection should be continued. If the ideal image effect cannot be obtained, the detection should be terminated.

[0053] S102, constructing a pipeline defect data set according to the panoramic image and the chromatogram, classifying and annotating the pipeline defect data set, establishing a pipeline defect database, and dividing the images in the pipeline defect database into a training set, a validation set, and a test set in proportion.

[0054] Specifically, a pipeline defect data set is constructed according to the panoramic image and the chromatogram, the pipeline defect data set is classified and labeled, and a pipeline defect database is established, including:

[0055] Screen the images in the pipeline defect database, remove blurred images and images without defects, and obtain pipeline defect images;

[0056] Classify and label the pipeline defect image according to defect categories, label the background area pixels of the image as 0, and label the defect area pixels as 1, 2, 3, etc. according to the categories;

[0057] The calibrated pipeline defect image in png format is combined with the generated annotation information in json format, converted into a defect binary map through python code, and the pipeline defect image and the binary map are matched one by one according to the name to establish a pipeline defect image database.

[0058] S103, carrying a pipeline defect segmentation model based on a feature pyramid, using a transfer learning method to initialize the pipeline defect segmentation model, training the pipeline defect segmentation model based on the training set, validation set and test set, comparing the change curves of the loss value and accuracy during the training process of the pipeline defect segmentation model, and adjusting and finding the optimal hyperparameters.

[0059] S104, detecting the panoramic image based on the pipeline defect segmentation model, and when a defect is detected on the panoramic image, associating the laser radar contour data corresponding to the defect with the pipeline detection image data to interpret and quantify the defect.

[0060] The fracture size, deformation rate, and deposition amount can be measured on the LiDAR profile data, and the crack width and defect length can be confirmed on the plane development diagram.

[0061] Specifically, panoramic detection can obtain a full 360-degree horizontal field of view and a certain angle vertical field of view. The image formed by the panoramic annular lens on the detector is an annular image. The scene under the same field of view angle is located in the same circle on the image plane, and the radius of the circle is the image height. The inner radius of the annular image plane is determined by the field of view angle corresponding to the upper edge of the field of view of the annular lens, and the outer radius is determined by the field of view angle corresponding to the lower edge of the annular lens.

[0062] Specifically, the panoramic image of the interior of the pipeline obtained using panoramic laser quantitative detection technology is equivalent to cutting the pipeline open and then laying it flat. Through the plane unfolding image, the situation inside the pipeline can be intuitively seen without watching video, which greatly improves the judgment efficiency and reduces the amount of data.

[0063] Specifically, the working principle of laser radar is to transmit a detection laser beam to the target, and then compare the received signal reflected from the target with the transmitted signal. After appropriate processing, relevant information about the target, such as the target distance, can be obtained. It consists of a laser transmitter, an optical receiver, a turntable and a processing system. The processing system controls the turntable to rotate at a certain speed, and continuously measures the distance of the surrounding targets during the rotation process to obtain the distance of the surrounding targets from itself. When applied inside a pipeline, it is the distance from the inner surface of the pipeline to the laser radar in all directions.

[0064] Specifically, after acquiring the laser contour data of the pipeline, the center point of the contour is found through an algorithm, and then starting from the 12 o'clock direction, the distance from each laser measurement point to the center point is calculated and compared with the standard pipeline radius. The difference is displayed in different colors according to the standard pipeline radius.

[0065] Specifically, points of different colors on the chromatogram reflect the deviation of the pipe diameter at that location from the theoretical pipe diameter. A pipe in good condition will not have deviations, and the chromatogram will be a pure color picture showing a very uniform display. If there is deposition or rupture, it will be presented in different colors on the chromatogram.

[0066] Specifically, the chromatogram can be used to quickly identify deposits, ruptures, branch pipes, etc.

[0067] S105, receiving real-time detection data sent back by the pipeline crawling robot during the field test, calculating a real-time panoramic image according to the real-time detection data, performing pipeline defect detection on the real-time panoramic image based on the pipeline defect segmentation model, and segmenting and quantifying pipeline defects.

[0068] Specifically, when using this technology for inspection, there is no need to stay at the pipeline defect. Just crawling in the pipeline once can obtain the internal pipeline video, laser contour, panoramic image, and chromatogram, which greatly improves the inspection efficiency. According to the inspection results, the system can effectively quantify the defective parts in the pipeline and accurately identify the defect type, size and other information.

[0069] The present invention proposes a drainage pipe defect quantification method that integrates video and laser. A convolutional neural network algorithm based on feature pyramid is used to segment the pipe defects and map them on a panoramic expansion image, thereby realizing intelligent quantification and three-dimensional imaging output of the pipe defects.

[0070] The present invention integrates video and laser to quantitatively identify underground drainage pipe defects. It mainly develops a video detection data digitization system based on the previous research and development of video panoramic detection equipment. The study uses the panoramic image parameters recorded in real time to annularly expand the panoramic image, uses image registration and splicing technology to splice, and uses image fusion technology to generate a two-dimensional panoramic expansion image of the pipe wall. The study creates a three-dimensional pipe wall model through the ratio of the actual pipe diameter and pipe length (obtained from the detection) to achieve 3D imaging output.

[0071] According to the actual engineering detection results, the application of this technology can provide basic data support for the establishment of hydraulic models of urban drainage pipes in my country and the detection, repair and network management of buried drainage pipes. Compared with traditional relatively single detection technology, it greatly improves work efficiency and detection effects, and generates greater social benefits and economic value.

[0072] The defects that can and cannot be quantified by the pipeline disease intelligent quantification method integrating vision and laser of the present invention are shown in the table below.

[0073] Table 1 Panoramic laser inspection quantitative defect analysis table

[0074]

[0075] The same and similar parts between the various embodiments in this specification can be referred to each other. The above-mentioned embodiments of the present invention do not constitute a limitation on the protection scope of the present invention.

Claims

1. An intelligent pipeline disease quantification method integrating vision and laser, characterized in that: include: The detection data collected by the pipeline crawling robot is obtained, and the detection data includes pipeline detection image data, laser radar profile data, mileage data and pipeline crawling robot posture data. According to the detection data, real-time calculation is performed to obtain a panoramic view and a chromatogram of the pipeline; after the pipeline crawling robot enters the pipeline and before the formal detection begins, parameters are determined, and the parameter determination includes calibrating the laser radar profile data, inputting the pipeline size, selecting the appropriate expansion circle position and expansion width, obtaining the actual pipeline diameter measured manually and the diameter value displayed on the collection system of the pipeline crawling robot, and judging whether the actual pipeline diameter measured manually and the diameter value displayed on the collection system of the pipeline crawling robot are the same. If the actual pipeline diameter measured manually is If the diameter is different from the diameter value displayed on the acquisition system of the pipeline crawling robot, click the calibration button of the acquisition system and enter the difference between the two for calibration. After the calibration is completed, the acquisition system prompts the recommended maximum walking speed of the pipeline crawling robot; after the parameters are determined, a trial test is carried out to observe whether the panoramic expansion image is unclear or there is an obvious blind phenomenon; if there is an unclear panoramic expansion image or an obvious blind phenomenon, adjust the parameters; during the inspection process, if an obstacle that cannot be passed is encountered, the current image is fully expanded so that the defect is reflected in the panoramic image. The panoramic image presents the panoramic view of the inside of the pipeline in the form of a picture. During the inspection process, there is no stopping, no rotation, no looking around, no adjustment of zoom and no stopping for observation, so that the defects can be judged through the panoramic image; A pipeline defect dataset is constructed based on the panoramic image and the chromatogram, the pipeline defect dataset is classified and annotated, images in the pipeline defect database are screened, blurred images and images without defects are removed, and pipeline defect images are obtained; the pipeline defect images are classified and annotated according to defect categories, the background area pixels of the annotated images are marked as 0, and the defect area pixels are annotated as 1, 2, 3, etc. according to categories; the calibrated pipeline defect images in png format are combined with the generated annotation information in json format, converted into defect binary images through python code, and the pipeline defect images and the binary images are matched one by one according to the names, a pipeline defect database is established, and the images in the pipeline defect database are divided into a training set, a validation set, and a test set according to proportion; Equipped with a pipeline defect segmentation model based on a feature pyramid, the pipeline defect segmentation model is initialized using a transfer learning method, the pipeline defect segmentation model is trained based on the training set, the validation set, and the test set, the loss value and the accuracy change curves during the pipeline defect segmentation model training process are compared, and the optimal hyperparameters are adjusted and found; The panoramic image is inspected based on the pipeline defect segmentation model. When a defect is detected on the panoramic image, the laser radar contour data corresponding to the defect and the pipeline inspection image data are associated to interpret and quantify the defect. Receive real-time detection data sent back by the pipeline crawling robot during the field test, calculate a real-time panoramic image based on the real-time detection data, perform pipeline defect detection on the real-time panoramic image based on the pipeline defect segmentation model, and segment and quantify the pipeline defects.

2. The pipeline disease intelligent quantification method integrating vision and laser according to claim 1, characterized in that: During the field test of the pipeline crawling robot, the water level inside the pipeline did not exceed 5%.

3. The pipeline disease intelligent quantification method integrating vision and laser according to claim 1, characterized in that: Before the on-site test of the pipeline crawling robot, adjust the light intensity of the pipeline crawling robot to ensure clear video images.

4. The pipeline disease intelligent quantification method integrating vision and laser according to claim 1 is characterized in that: When the panoramic image is inspected based on the pipeline defect segmentation model and a defect is detected on the panoramic image, the laser radar contour data corresponding to the defect and the pipeline detection image data are associated to perform the defect interpretation and quantification steps. The rupture size, deformation rate, and deposition amount can be measured on the laser radar contour data, and the crack width and defect length can be confirmed on the plane unfolded image.

Images