Method and device for detecting defects of underground pipeline based on illumination compensation and point cloud filtering

Abstract

The application relates to the technical field of underground pipeline detection, and discloses an underground pipeline defect detection method and device based on light compensation and point cloud filtering. The underground pipeline defect detection method based on light compensation and point cloud filtering comprises the following steps: an adaptive gain coefficient is used to perform image enhancement processing on a pipeline inner wall color image to obtain a target enhanced gray image; pipeline three-dimensional point cloud data is filtered according to a radius threshold to obtain target three-dimensional point cloud data; edge image pixel sets are extracted from the target enhanced gray image, three-dimensional coordinate feature point sets are extracted from the target three-dimensional point cloud data, and a target mapping relationship is constructed according to the extraction results; three-dimensional coordinates of defects are determined according to the target mapping relationship; defect parameters are determined according to the three-dimensional coordinates of the defects, and the defect grade is determined according to the defect parameters. The application performs fine processing on images and three-dimensional point cloud data, and then judges defects according to the processed data, thereby improving the accuracy of pipeline defect judgment.

Description

Technical Field

[0001] This invention relates to the field of underground pipeline inspection technology, specifically to a method and apparatus for detecting defects in underground pipelines based on illumination compensation and point cloud filtering. Background Technology

[0002] As a core component of urban infrastructure and industrial production, the operational status of underground pipelines directly affects the normal operation of cities and the safety of industrial production. With the increase of service life and the influence of factors such as geological subsidence and media corrosion, underground pipelines are prone to deformation defects such as radial displacement, depressions, and bulges. If these defects are not detected and addressed in a timely manner, they may lead to safety accidents such as leaks and collapses.

[0003] To achieve efficient detection of pipeline defects, underground pipeline robot inspection technology is gradually replacing traditional manual excavation and inspection, becoming the mainstream inspection method. This technology uses robots equipped with high-definition cameras, LiDAR, and other sensors to penetrate deep into underground pipelines, collecting images and 3D point cloud data. Defect identification and assessment are then achieved through data processing.

[0004] The existing underground pipeline robot inspection method involves simple preprocessing of the acquired images and 3D point cloud data, followed by defect judgment based on the preprocessing results. However, this method results in significant noise interference in the images and 3D point cloud data, and different defect features are not effectively distinguished, leading to inaccurate defect detection results that do not reflect reality. Summary of the Invention

[0005] This invention provides a method and apparatus for detecting defects in underground pipelines based on illumination compensation and point cloud filtering, in order to solve the problem that the defect detection results of underground pipeline robot detection methods in related technologies are inaccurate and do not conform to the actual situation.

[0006] In a first aspect, the present invention provides a method for detecting defects in underground pipelines based on illumination compensation and point cloud filtering, comprising: acquiring a color image of the inner wall of a target underground pipeline and three-dimensional point cloud data of the pipeline collected by an underground pipeline robot; performing image enhancement processing on the color image of the inner wall of the pipeline using an adaptive gain coefficient to obtain a target enhanced grayscale image; the adaptive gain coefficient being a dynamic gain coefficient determined based on the grayscale value of the color image of the inner wall of the pipeline; determining a dynamic radius threshold based on the standard radius of the target underground pipeline; performing radius filtering on the three-dimensional point cloud data of the pipeline based on the radius threshold; and performing statistical filtering on the radius-filtered three-dimensional point cloud data of the pipeline to obtain the target three-dimensional point cloud image. The process involves: extracting edge image pixel sets from the enhanced grayscale image of the target; extracting the corresponding 3D coordinate feature point sets from the edge image pixel sets in the target 3D point cloud data; constructing a target mapping relationship based on the edge image pixel sets and the 3D coordinate feature point sets; using the target mapping relationship to characterize the mapping relationship between the pixel coordinates of the target underground pipeline image and the 3D coordinates of the point cloud; identifying defect areas in the enhanced grayscale image of the target; determining the 3D coordinates of the defect areas based on the defect pixel coordinates and the target mapping relationship; determining defect parameters based on the 3D coordinates of the defects; and judging the defect level of the defect areas based on the defect parameters to obtain the target defect level.

[0007] This invention presents a method for detecting defects in underground pipelines based on illumination compensation and point cloud filtering. It acquires color images of the inner wall of the target underground pipeline and 3D point cloud data of the pipeline collected by an underground pipeline robot. The color images of the inner wall are then enhanced using an adaptive gain coefficient to obtain an enhanced grayscale image of the target. This dynamic adaptive image enhancement method avoids compensation distortion and automatically adjusts the gain based on local grayscale features of the image, optimizing the illumination compensation effect and effectively improving the contrast of defect details in dark and shadowed areas of the pipeline inner wall. The invention determines a dynamic radius threshold based on the standard radius of the target underground pipeline. Radius filtering is then applied to the 3D point cloud data of the pipeline using this dynamic radius threshold. Statistical filtering is then applied to the radius-filtered 3D point cloud data to obtain the target 3D point cloud data. This combined radius filtering and statistical filtering effectively removes outlier noise points and interference points not on the pipeline inner wall from the 3D point cloud data, while retaining the effective geometric information of the pipeline inner wall, significantly improving the accuracy and stability of subsequent point cloud processing and defect parameter calculation. This invention extracts edge image pixel sets from a target enhanced grayscale image and extracts corresponding 3D coordinate feature point sets from the target 3D point cloud data. Based on the edge image pixel sets and 3D coordinate feature point sets, a target mapping relationship is constructed. By matching edge features, a mapping relationship between image pixels and the 3D point cloud is established, achieving precise alignment between 2D images and 3D information. This invention also identifies defect regions in a target enhanced grayscale image. Based on the defect pixel coordinates and the target mapping relationship, the 3D coordinates of the defect region are determined. Relying on the constructed target mapping relationship, the defect regions identified in the 2D image can be directly converted into 3D spatial coordinates, achieving rapid 3D defect localization. Defect parameters are determined based on the 3D defect coordinates, and the defect level of the defect region is judged based on the defect parameters to obtain the target defect level. Key parameters are calculated based on the 3D defect coordinates, enabling quantitative judgment of the defect level. Compared with related technologies, this invention utilizes an adaptive gain coefficient to enhance the color image of the inner wall of the pipe, and processes the three-dimensional point cloud data of the pipe through a combination of radius filtering and statistical filtering, thereby achieving refined processing of the image and point cloud data and improving the accuracy of the image and point cloud data. By establishing a target mapping relationship between the edge image pixel set and the three-dimensional coordinate feature point set, the three-dimensional spatial coordinate mapping of the defect area is realized, thereby improving the accuracy of the obtained defect level and making the defect level more consistent with the actual situation.

[0008] In one optional implementation, the process involves acquiring a color image of the inner wall of a target underground pipeline and 3D point cloud data of the pipeline collected by an underground pipeline robot, and performing image enhancement processing on the color image of the inner wall of the pipeline using an adaptive gain coefficient to obtain a target enhanced grayscale image. This includes: acquiring a color image of the inner wall of the pipeline collected by a high-definition camera mounted on the underground pipeline robot at a preset frame rate; acquiring 3D point cloud data of the pipeline collected by a lidar mounted on the underground pipeline robot at a preset density; converting the color image of the inner wall of the pipeline into an initial grayscale image; acquiring the grayscale values ​​of the grayscale image; determining the average grayscale value of the inner wall of the pipeline based on the grayscale values ​​of the grayscale image; obtaining a target difference value based on the difference between the standard grayscale average value and the average grayscale value of the inner wall of the pipeline; obtaining a target quotient value based on the quotient of the target difference value and the total number of grayscale levels; obtaining an adaptive gain coefficient based on the sum of the target quotient value and 1; acquiring the illumination component in the initial grayscale image; obtaining a corrected illumination component based on the quotient of the illumination component and the adaptive gain coefficient; extracting a reflection component based on the corrected illumination component and the illumination component; and normalizing the reflection component to obtain the target enhanced grayscale image.

[0009] In one optional implementation, a dynamic radius threshold is determined based on the standard radius of the target underground pipeline. Radius filtering is then applied to the pipeline's 3D point cloud data based on this radius threshold. Statistical filtering is then performed on the radius-filtered pipeline 3D point cloud data to obtain the target 3D point cloud data. This process includes: obtaining the dynamic radius threshold by multiplying the standard pipeline radius by an empirical coefficient; constructing a sphere with each data point in the pipeline 3D point cloud data as its center and the dynamic radius threshold as its radius; applying radius filtering to the pipeline 3D point cloud data based on the relationship between the number of neighboring point clouds within the sphere and the threshold number of neighboring point clouds; determining the average distance between each data point in the radius-filtered pipeline 3D point cloud data and multiple neighboring points; determining the mean distance and standard deviation of the average distances between multiple data points and their neighboring points; determining a confidence interval based on the sum of the mean distance and the standard deviation of the distance at a preset multiple, and the difference between the mean distance and the standard deviation of the distance at a preset multiple; and performing statistical filtering on the radius-filtered pipeline 3D point cloud data based on the confidence interval to obtain the target 3D point cloud data.

[0010] In one optional implementation, a target mapping relationship is constructed based on the edge image pixel set and the three-dimensional coordinate feature point set, including: constructing a first mapping relationship between the width dimension of the edge image pixel set and the first and second directional components of the three-dimensional coordinate feature point set based on the camera intrinsic parameters of the high-definition camera, the projection of the translation component of the LiDAR onto the image plane, the edge image pixel variables, and the three-dimensional coordinate feature point variables; constructing a second mapping relationship between the height dimension of the edge image pixel set and the third and second directional components of the three-dimensional coordinate feature point set based on the camera intrinsic parameters of the high-definition camera, the projection of the translation component of the LiDAR onto the image plane, the edge image pixel variables, and the three-dimensional coordinate feature point variables; the first directional component is perpendicular to the third directional component, and the second directional component is perpendicular to the plane formed by the first and third directional components; an initial mapping relationship is obtained based on the first and second mapping relationships; the three-dimensional coordinate feature point set is input into the initial mapping relationship to obtain the mapped edge image pixel set; and the initial mapping relationship is registered based on the average distance error between the mapped edge image pixel set and the edge image pixel set to obtain the target mapping relationship.

[0011] In one optional implementation, identifying defect regions in a target-enhanced grayscale image and determining the three-dimensional coordinates of the defect region based on the mapping relationship between the defect pixel coordinates of the defect region and the target includes: performing threshold segmentation binarization processing on the target-enhanced grayscale image; filtering initial defect regions based on the threshold segmentation binarization processing results; performing morphological processing on the initial defect regions to obtain the defect regions; and inputting the defect pixel coordinates of the defect regions into the target mapping relationship to obtain the three-dimensional coordinates of the defect regions.

[0012] In one optional implementation, the defect parameters include a maximum radial offset. Determining the defect parameters based on the three-dimensional coordinates of the defect, and judging the defect level of the defect area based on the defect parameters to obtain the target defect level, includes: obtaining the actual distance from each defect coordinate point in the three-dimensional coordinates of the defect to the axis of the target underground pipeline; obtaining the radial offset corresponding to each defect coordinate point based on the deviation between the actual distance corresponding to each defect coordinate point and a preset distance; selecting the maximum radial offset from the radial offsets corresponding to multiple defect coordinate points; determining the deformation rate based on the quotient of the maximum radial offset and the inner diameter of the target underground pipeline; comparing the deformation rate with a preset deformation rate threshold; and determining the target defect level based on the comparison result.

[0013] Secondly, this invention provides an underground pipeline defect detection device based on illumination compensation and point cloud filtering, comprising: an image enhancement module, used to acquire a color image of the inner wall of the pipeline and three-dimensional point cloud data of the pipeline collected by an underground pipeline robot in a target underground pipeline, and to perform image enhancement processing on the color image of the inner wall of the pipeline using an adaptive gain coefficient to obtain a target enhanced grayscale image; the adaptive gain coefficient is a dynamic gain coefficient determined based on the grayscale value of the color image of the inner wall of the pipeline; and a point cloud filtering module, used to determine a dynamic radius threshold based on the standard radius of the target underground pipeline, to perform radius filtering on the three-dimensional point cloud data of the pipeline based on the radius threshold, and to perform statistical filtering on the radius-filtered three-dimensional point cloud data of the pipeline to obtain the target three-dimensional point cloud data; The target mapping relationship determination module is used to extract edge image pixel sets from the target enhanced grayscale image and extract the corresponding 3D coordinate feature point sets from the target 3D point cloud data. Based on the edge image pixel sets and the 3D coordinate feature point sets, a target mapping relationship is constructed. The target mapping relationship is used to characterize the mapping relationship between the pixel coordinates of the target underground pipeline image and the 3D coordinates of the point cloud. The defect point cloud determination module is used to identify defect areas in the target enhanced grayscale image and determine the defect 3D coordinates of the defect area based on the defect pixel coordinates and the target mapping relationship. The defect level determination module is used to determine defect parameters based on the defect 3D coordinates and determine the defect level of the defect area based on the defect parameters to obtain the target defect level.

[0014] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the underground pipeline defect detection method based on illumination compensation and point cloud filtering described in the first aspect or any corresponding embodiment.

[0015] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the underground pipeline defect detection method based on illumination compensation and point cloud filtering described in the first aspect or any corresponding embodiment above.

[0016] Fifthly, the present invention provides a computer program product, including computer instructions, which are used to cause a computer to execute the underground pipeline defect detection method based on illumination compensation and point cloud filtering described in the first aspect or any corresponding embodiment above. Attached Figure Description

[0017] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of an application scenario according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the first process of the underground pipeline defect detection method based on illumination compensation and point cloud filtering according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the second process of the underground pipeline defect detection method based on illumination compensation and point cloud filtering according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the third process of the underground pipeline defect detection method based on illumination compensation and point cloud filtering according to an embodiment of the present invention. Figure 5 This is a schematic diagram of the fourth process of the underground pipeline defect detection method based on illumination compensation and point cloud filtering according to an embodiment of the present invention; Figure 6 This is a structural block diagram of an underground pipeline defect detection device based on illumination compensation and point cloud filtering according to an embodiment of the present invention. Figure 7 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0021] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0022] As an optional application scenario of this invention, such as Figure 1 As shown, the underground pipeline defect detection system based on illumination compensation and point cloud filtering may include at least one terminal device and at least one server. Figure 1 The system is illustrated in the example, which includes a computer 101, a mobile terminal 102, and a server 103, and the terminal devices such as the computer 101 and the mobile terminal 102 are connected to the server 103 through a network 110.

[0023] Specifically, the terminal device can be a smartphone, tablet, laptop, PDA, desktop computer, game console, smart TV, smart wearable device, in-vehicle terminal, VR (Virtual Reality) device, AR (Augmented Reality) device, etc. Server 103 can be a standalone physical server, a server cluster, a distributed system, or a cloud server providing cloud services. Network 110 can be a wired or wireless network, examples of which include, but are not limited to, the Internet, corporate intranet, local area network, wide area network, mobile communication network, and combinations thereof.

[0024] In related underground pipeline defect detection technologies, to address the problem of uneven image illumination, the traditional Retinex algorithm (an image enhancement algorithm that simulates human vision) is mainly used for illumination compensation, achieving illumination equalization by decomposing the illumination and reflection components of the image. For lidar point cloud noise, single-radius filtering or statistical filtering algorithms are often used to remove noise points. Radar filtering filters point clouds with a fixed radius threshold to select those with a sufficient number of neighboring points, while statistical filtering removes outliers based on the statistical characteristics of the point cloud's neighborhood distance. In multi-sensor data registration, feature matching algorithms such as SIFT (Scale-Invariant Feature Transform) or ICP (Iterative Closest Point) algorithms are commonly used to register images and point clouds. For defect quantization, defect parameters are often calculated based on single-sensor data, or multi-source data fusion quantization is achieved through simple coordinate mapping relationships.

[0025] However, underground pipeline defect detection technologies have many shortcomings. Traditional Retinex algorithms, which use a fixed gain coefficient to adjust the illumination component, cannot adapt to the grayscale differences in different areas of underground pipelines. They are prone to undercompensation in dimly lit areas and overcompensation in areas with water vapor reflection, leading to blurred or distorted pipeline defect features. Furthermore, some improved Retinex algorithms have excessive computational requirements, making real-time deployment on robotic embedded systems difficult. Single-radius filtering, using a fixed radius threshold, cannot adapt to radius variations across different pipeline cross-sections. It easily misclassifies effective point clouds in deformed areas as noise or fails to completely remove dust noise points. Single statistical filtering also has limitations in removing dense dust noise. Furthermore, the filtering method is inaccurate and lacks sensitivity to noise points in the pipe radius direction, resulting in residual noise in the point cloud after filtering, which affects the accuracy of 3D modeling and deformation quantization. Traditional registration algorithms require a large amount of feature extraction, iterative calculation, or neighborhood search, resulting in high computational complexity and stringent requirements on the hardware computing power of the robot's embedded system. They are prone to registration delays, failing to meet real-time detection needs, and are prone to registration failures in feature-sparse regions. Related quantization methods are mostly based on single sensor data or simple fusion relationships to calculate defect parameters, without establishing a precise mapping model between image pixel scale and point cloud 3D coordinates. This makes it difficult to accurately calculate core parameters such as radial offset and maximum indentation depth of the deformed area, leading to deviations in defect classification.

[0026] This invention provides a method for detecting defects in underground pipelines based on illumination compensation and point cloud filtering. By performing fine processing on images and 3D point cloud data, defects can be judged based on the processed data, thereby improving the accuracy of pipeline defect judgment.

[0027] According to an embodiment of the present invention, an embodiment of a method for detecting defects in underground pipelines based on illumination compensation and point cloud filtering is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0028] This embodiment provides a method for detecting defects in underground pipelines based on illumination compensation and point cloud filtering, which can be used with computer equipment. Figure 2 This is a first flowchart of an underground pipeline defect detection method based on illumination compensation and point cloud filtering according to an embodiment of the present invention, as follows: Figure 2 As shown, the process includes the following steps: Step S201: Obtain the color image of the inner wall of the underground pipeline and the three-dimensional point cloud data of the pipeline collected by the underground pipeline robot in the target underground pipeline. Use the adaptive gain coefficient to perform image enhancement processing on the color image of the inner wall of the pipeline to obtain the target enhanced grayscale image.

[0029] The adaptive gain coefficient is a dynamic gain coefficient determined based on the grayscale values ​​of the color image of the inner wall of the pipe.

[0030] In some optional implementations, the underground pipeline robot is a pipeline operation device equipped with a high-definition camera and a lidar, used to enter the target pipeline to collect data on the inner wall; the color image of the pipeline inner wall is an RGB format image collected by the high-definition camera on the underground pipeline robot, recording the texture and defect morphology information of the inner wall; the pipeline 3D point cloud data is a set of 3D coordinates collected by the lidar / structured light sensor on the robot, recording the geometric shape of the inner wall; the adaptive gain coefficient is an enhancement coefficient dynamically adjusted based on the image grayscale distribution to avoid overexposure / underexposure caused by fixed gain; the target enhanced grayscale image is a single-channel grayscale image with improved contrast of defect details after enhancement processing.

[0031] In some optional implementations, an underground pipeline robot equipped with a high-definition camera, LiDAR, and a positioning module is placed inside the target underground pipeline, and the robot is controlled to move at a constant speed along the axis of the target underground pipeline. The high-definition camera acquires RGB images of the inner wall of the pipeline at a fixed frame rate, and the LiDAR acquires three-dimensional point cloud data of the pipeline interior at a set density, with the LiDAR scanning angle covering the entire cross-section of the pipeline; the positioning module synchronously records the robot's position information, achieving timestamp synchronization between the images and the point cloud data, ensuring that each frame of the image corresponds to a continuous segment of point cloud data.

[0032] In some optional implementations, the acquired color images of the inner wall of the pipe and the three-dimensional point cloud data of the pipe are standardized in format and invalid data is removed. The color images of the inner wall of the pipe and the three-dimensional point cloud data of the pipe used subsequently are all data after the format standardization and invalid data removal process.

[0033] Step S202: Determine the dynamic radius threshold based on the standard radius of the target underground pipeline, perform radius filtering on the pipeline 3D point cloud data based on the radius threshold, and perform statistical filtering on the radius-filtered pipeline 3D point cloud data to obtain the target 3D point cloud data.

[0034] Among them, the standard radius of the pipeline is the nominal inner wall radius of the target underground pipeline when it leaves the factory; the dynamic radius threshold is the filtering range set based on the standard radius; the radius filtering is to filter outout points that exceed the radius range with the pipeline axis as the center; the statistical filtering is to filter noise points that deviate from the mean based on the local neighborhood distance statistics of the point cloud, and retain the continuous geometric points of the inner wall of the pipeline; the target three-dimensional point cloud data is a set of point clouds that has no obvious noise and only contains the effective geometric information of the inner wall of the pipeline after double filtering.

[0035] Step S203: Extract the edge image pixel set from the target enhanced grayscale image, extract the three-dimensional coordinate feature point set corresponding to the edge image pixel set from the target three-dimensional point cloud data, and construct the target mapping relationship based on the edge image pixel set and the three-dimensional coordinate feature point set.

[0036] Among them, the target mapping relationship is used to characterize the mapping relationship between the pixel coordinates of the target underground pipeline image and the three-dimensional coordinates of the point cloud.

[0037] In some optional implementations, the edge image pixel set is the set of pixel coordinates of the inner wall contour and defect edge extracted by the edge detection algorithm in the target enhanced grayscale image; the three-dimensional coordinate feature point set is the set of three-dimensional coordinates corresponding to the edge image pixels in the target three-dimensional point cloud data, which includes the geometric information of the inner wall edge; the target mapping relationship is the correspondence between the two-dimensional image pixel coordinates and the three-dimensional point cloud coordinates.

[0038] Step S204: Identify the defect region in the enhanced grayscale image of the target, and determine the three-dimensional coordinates of the defect region based on the mapping relationship between the defect pixel coordinates of the defect region and the target.

[0039] Among them, the defect area is the abnormal area of ​​the inner wall of the pipe identified in the target enhanced grayscale image, such as cracks, deformation, depressions, corrosion areas, etc.; the defect pixel coordinates are the set of pixel positions of the defect area in the target enhanced grayscale image; and the defect three-dimensional coordinates are the set of spatial coordinates of the defect area in the target three-dimensional point cloud data.

[0040] Step S205: Determine the defect parameters based on the three-dimensional coordinates of the defect, and determine the defect level of the defect area based on the defect parameters to obtain the target defect level.

[0041] Among them, the defect parameters are geometric parameters calculated based on the three-dimensional coordinates of the defect, such as the maximum radial offset, the maximum indentation depth, the area of ​​the defect region, and the length of the defect extension; the target defect level is the final severity level corresponding to the defect region.

[0042] This embodiment provides a method for detecting defects in underground pipelines based on illumination compensation and point cloud filtering. It acquires color images of the inner wall of the target underground pipeline and 3D point cloud data of the pipeline collected by an underground pipeline robot. An adaptive gain coefficient is used to enhance the color image of the inner wall, resulting in an enhanced grayscale image of the target. A dynamic adaptive image enhancement method is employed to avoid compensation distortion. The gain is automatically adjusted based on local grayscale features of the image, optimizing the illumination compensation effect and effectively improving the contrast of defect details in dark and shadowed areas of the pipeline inner wall. This embodiment determines a dynamic radius threshold based on the standard radius of the target underground pipeline. Radius filtering is then applied to the 3D point cloud data of the pipeline using this dynamic radius threshold. Statistical filtering is then applied to the radius-filtered 3D point cloud data to obtain the target 3D point cloud data. This combined radius filtering and statistical filtering effectively removes outlier noise points and interference points not on the pipeline inner wall from the 3D point cloud data, while retaining the effective geometric information of the pipeline inner wall, significantly improving the accuracy and stability of subsequent point cloud processing and defect parameter calculation. This invention extracts edge image pixel sets from a target enhanced grayscale image and extracts corresponding 3D coordinate feature point sets from the target 3D point cloud data. Based on the edge image pixel sets and the 3D coordinate feature point sets, a target mapping relationship is constructed. By matching edge features, a mapping relationship between image pixels and the 3D point cloud is established, achieving precise alignment between 2D images and 3D information. This invention also identifies defect regions in a target enhanced grayscale image. Based on the defect pixel coordinates and the target mapping relationship, the 3D coordinates of the defect region are determined. Relying on the constructed target mapping relationship, the defect region identified in the 2D image can be directly converted into 3D spatial coordinates, achieving rapid 3D defect localization. Defect parameters are determined based on the 3D defect coordinates, and the defect level of the defect region is judged based on the defect parameters to obtain the target defect level. Key parameters are calculated based on the 3D defect coordinates, enabling quantitative judgment of the defect level. Compared with related technologies, the embodiments of the present invention utilize adaptive gain coefficients to perform image enhancement processing on the color image of the inner wall of the pipeline, and process the three-dimensional point cloud data of the pipeline through a combination of radius filtering and statistical filtering to achieve refined processing of the image and point cloud data, thereby improving the accuracy of the image and point cloud data. By establishing a target mapping relationship between the edge image pixel set and the three-dimensional coordinate feature point set, the three-dimensional spatial coordinate mapping of the defect area is realized, thereby improving the accuracy of the obtained defect level and making the defect level more consistent with the actual situation.

[0043] This embodiment provides a method for detecting defects in underground pipelines based on illumination compensation and point cloud filtering, which can be used with computer equipment. Figure 3 This is a second flowchart of the underground pipeline defect detection method based on illumination compensation and point cloud filtering according to an embodiment of the present invention, as follows: Figure 3 As shown, the process includes the following steps: Step S301: Obtain the color image of the inner wall of the underground pipeline and the three-dimensional point cloud data of the pipeline collected by the underground pipeline robot in the target underground pipeline. Use the adaptive gain coefficient to perform image enhancement processing on the color image of the inner wall of the pipeline to obtain the target enhanced grayscale image.

[0044] Specifically, step S301 includes: Step S3011: Obtain color images of the inner wall of the pipeline from a high-definition camera mounted on the underground pipeline robot at a preset frame rate, and obtain three-dimensional point cloud data of the pipeline from a lidar mounted on the underground pipeline robot at a preset density.

[0045] The preset frame rate can be set according to the actual situation. For example, the preset frame rate can be 30fps. The preset density can also be set according to the actual situation. For example, the preset density can be 1000 points / meter.

[0046] Step S3012: Convert the color image of the inner wall of the pipe into an initial grayscale image, obtain the grayscale value of the grayscale image, and determine the average grayscale value of the inner wall of the pipe based on the grayscale value of the grayscale image.

[0047] In this process, the values ​​of each channel in the color image of the pipe's inner wall are weighted and summed to obtain the grayscale value of the grayscale image. For example, the formula for determining the grayscale value is:

[0048] in, The grayscale value of the grayscale image. The value of the red channel in the color image of the pipe's inner wall. The green channel value of the color image of the inner wall of the pipe. This represents the blue channel value in the color image of the pipe's inner wall.

[0049] In some optional implementations, the pipe region in the initial grayscale image is traversed, and the average grayscale value of the inner wall of the pipe is calculated. For example, the formula for determining the average grayscale value of the inner wall of the pipe is:

[0050] in, This represents the average gray level of the pipe's inner wall. This represents the number of pixel rows in the pipe region. This represents the number of pixel columns in the pipe region. This represents the grayscale value of the grayscale image.

[0051] For example, the average gray value of the inner wall of the pipe can be 85, with a range of 0-255. Values ​​below 100 are considered dark areas.

[0052] Step S3013: Based on the difference between the standard grayscale mean and the grayscale mean of the inner wall of the pipe, obtain the target difference value; based on the quotient of the target difference and the total number of grayscale levels, obtain the target quotient value; based on the sum of the target quotient value and 1, obtain the adaptive gain coefficient.

[0053] Where the standard grayscale mean can be 128 and the total number of grayscale levels can be 256, the formula for determining the adaptive gain coefficient is:

[0054] in, For adaptive gain coefficients, This represents the average grayscale value of the inner wall of the pipe.

[0055] In some alternative implementations, when hour, To enhance light intensity; when When ≥128, ≤1, suppress overexposure.

[0056] In some alternative implementations, when the average grayscale value of the inner wall of the pipe can be 85, the adaptive gain coefficient is 1.168.

[0057] Step S3014: Obtain the illumination component in the initial grayscale image; obtain the corrected illumination component based on the quotient of the illumination component and the adaptive gain coefficient; extract the reflection component based on the corrected illumination component and the illumination component; normalize the reflection component to obtain the target enhanced grayscale image.

[0058] Specifically, the Retinex algorithm is used to collect the illumination components in the initial grayscale image. For example, Gaussian filtering is used to estimate the illumination components, which can be expressed as:

[0059] in, The illumination component in the initial grayscale image. The grayscale value of the grayscale image. For convolution operations, It is a Gaussian filter kernel.

[0060] In some alternative implementations, the corrected illumination component is obtained based on the quotient of the illumination component and the adaptive gain coefficient, which can be expressed as:

[0061] in, To correct the light component, The illumination component in the initial grayscale image. This is the adaptive gain coefficient.

[0062] In some optional implementations, extracting the reflection component based on the modified illumination component and the illumination component includes: inputting the modified illumination component and the illumination component into a preset extraction formula to obtain the reflection component. The preset extraction formula may be:

[0063] in, For the reflection component, The grayscale value of the grayscale image. To correct the illumination component.

[0064] In some alternative implementations, the preset extraction formula enhances detail contrast through logarithmic transformation while avoiding numerical overflow.

[0065] In some optional implementations, the reflection component is normalized, mapping its value range to 0-255, to obtain the enhanced target grayscale image, as shown in the formula:

[0066] in, To enhance the grayscale image of the target, For the reflection component, This is the minimum value of the reflection component. This represents the maximum value of the reflection component.

[0067] Step S302: Determine the dynamic radius threshold based on the standard radius of the target underground pipeline, perform radius filtering on the pipeline 3D point cloud data based on the radius threshold, and perform statistical filtering on the radius-filtered pipeline 3D point cloud data to obtain the target 3D point cloud data.

[0068] Specifically, step S302 includes: Step S3021: Based on the product of the standard radius of the pipeline and the empirical coefficient, the dynamic radius threshold is obtained. A sphere is constructed with each data point in the pipeline 3D point cloud data as the center and the dynamic radius threshold as the radius. Radius filtering is performed on the pipeline 3D point cloud data based on the relationship between the number of neighboring point clouds in the sphere and the threshold number of neighboring point clouds.

[0069] Specifically, the pipeline 3D point cloud data is converted to coordinate system one (to the robot body coordinate system), and invalid points (points whose distance is beyond the detection range of the lidar) are removed from the point cloud data. The pipeline 3D point cloud data after radius filtering is the data after coordinate system one and invalid point removal.

[0070] In some alternative implementations, the standard pipe radius can be 0.5 meters, and the empirical coefficient, obtained through extensive experimental verification, can retain effective point cloud data while eliminating dust and noise. The empirical coefficient can be 0.05, and the formula for determining the dynamic radius threshold is:

[0071] in, For the standard radius of the pipeline, This is an empirical coefficient.

[0072] In some alternative implementations, the dynamic radius threshold can be obtained as 0.025m according to the formula for the dynamic radius threshold.

[0073] In some alternative implementations, each data point in the pipeline's 3D point cloud data is traversed. ,by Construct a sphere with a center and a dynamic radius threshold as its radius, and count the number of neighborhood point clouds within the sphere. Set a threshold for the number of neighboring point clouds. For example, the threshold for the number of neighborhood point clouds can be 5.

[0074] In some optional implementations, the 3D point cloud data of the pipeline is subjected to radius filtering based on the relationship between the number of neighboring point clouds within the sphere and a threshold number of neighboring point clouds. This includes: if the number of neighboring point clouds is less than the threshold number of neighboring point clouds, the data point is determined to be a dust noise point and is removed; if the number of neighboring point clouds is greater than or equal to the threshold number of neighboring point clouds, the data point is retained.

[0075] Step S3022: Determine the average distance between each data point and multiple neighboring points in the radius-filtered 3D point cloud data of the pipeline, and determine the mean distance and standard deviation of the average distance between multiple data points and neighboring points.

[0076] The formula for determining the average distance between each data point and multiple neighboring points in the radius-filtered 3D point cloud data of the pipeline is as follows:

[0077] in, For the first The average distance between a data point and its multiple neighboring points. The total number of neighboring points. For the first Data points With the m-th neighboring point Euclidean distance.

[0078] In some alternative implementations, the number of neighboring points can be set according to the actual situation, and can be up to 10.

[0079] In some alternative implementations, the formulas for determining the mean distance and standard deviation of the distances between multiple data points and their neighboring points are as follows:

[0080]

[0081] in, The mean distance For the first The average distance between a data point and its multiple neighboring points. The total number of data points. This represents the standard deviation of the distance.

[0082] Step S3023: Determine the confidence interval based on the sum of the distance mean and the distance standard deviation of the preset multiple, and the difference between the distance mean and the distance standard deviation of the preset multiple. Perform statistical filtering on the radius-filtered pipeline 3D point cloud data based on the confidence interval to obtain the target 3D point cloud data.

[0083] The preset multiple can be 2. The sum of the standard deviations of the distances from the mean and the preset multiple is used as the lower limit, and the sum of the standard deviations of the distances from the mean and the preset multiple is used as the upper limit, thus obtaining the confidence interval. The confidence interval can be expressed as follows: For example, when the mean distance is 0.01 meters and the standard deviation of the distance is 0.003 meters, the confidence interval is: Data outside this range is discarded to ensure that valid point clouds of defective areas are preserved.

[0084] Step S303: Extract the edge image pixel set from the target enhanced grayscale image, extract the corresponding 3D coordinate feature point set from the target 3D point cloud data, and construct the target mapping relationship based on the edge image pixel set and the 3D coordinate feature point set. For details, please refer to... Figure 2 Step S203 of the illustrated embodiment will not be described again here.

[0085] Step S304: Identify the defect region in the enhanced grayscale image of the target, and determine the three-dimensional coordinates of the defect region based on the mapping relationship between the defect pixel coordinates and the target. For details, please refer to [link to relevant documentation]. Figure 2 Step S204 of the illustrated embodiment will not be described again here.

[0086] Step S305: Determine the defect parameters based on the three-dimensional coordinates of the defect, and determine the defect level of the defect area based on the defect parameters to obtain the target defect level. For details, please refer to [link to relevant documentation]. Figure 2 Step S205 of the illustrated embodiment will not be described again here.

[0087] The underground pipeline defect detection method based on illumination compensation and point cloud filtering provided in this embodiment solves the distortion problem of traditional algorithms by adaptively adjusting the gain coefficient based on the gray-scale mean, and has a small computational load. The improved point cloud filtering realizes dynamic threshold adjustment based on the pipeline radius prior constraint, accurately removes dust and noise while retaining effective deformed point clouds, thereby improving the point cloud quality.

[0088] This embodiment provides a method for detecting defects in underground pipelines based on illumination compensation and point cloud filtering, which can be used with computer equipment. Figure 4 This is a third flowchart of the underground pipeline defect detection method based on illumination compensation and point cloud filtering according to an embodiment of the present invention, as follows: Figure 4 As shown, the process includes the following steps: Step S401: Acquire the color image of the inner wall of the underground pipeline and the 3D point cloud data of the pipeline collected by the underground pipeline robot in the target underground pipeline. Perform image enhancement processing on the color image of the inner wall of the pipeline using an adaptive gain coefficient to obtain the enhanced grayscale image of the target. For details, please refer to [link to details]. Figure 3 Step S301 of the illustrated embodiment will not be described again here.

[0089] Step S402: Determine the dynamic radius threshold based on the standard radius of the target underground pipeline. Perform radius filtering on the pipeline's 3D point cloud data based on the radius threshold. Then, perform statistical filtering on the radius-filtered pipeline 3D point cloud data to obtain the target 3D point cloud data. For details, please refer to [link to details]. Figure 3 Step S302 of the illustrated embodiment will not be described again here.

[0090] Step S403: Extract the edge image pixel set from the target enhanced grayscale image, extract the three-dimensional coordinate feature point set corresponding to the edge image pixel set from the target three-dimensional point cloud data, and construct the target mapping relationship based on the edge image pixel set and the three-dimensional coordinate feature point set.

[0091] Specifically, step S403 includes: Step S4031: Extract the edge image pixel set from the target enhanced grayscale image, and extract the three-dimensional coordinate feature point set corresponding to the edge image pixel set from the target three-dimensional point cloud data.

[0092] In this process, feature points of the pipe's inner wall, such as weld seams and interface edges, are extracted from the target enhanced grayscale image. The Canny edge detection algorithm is then used to extract the edge image pixel set, which can be represented as:

[0093] in, For the set of pixels in the edge image, For the first The coordinates of the width dimension of each image pixel. For the first The coordinates of the height dimension of each image pixel.

[0094] In some optional implementations, a set of three-dimensional coordinate feature points corresponding to the edge image pixel set is extracted from the target three-dimensional point cloud data. The set of three-dimensional coordinate feature points can be represented as:

[0095] in, For a set of three-dimensional coordinate feature points, For the first The three-dimensional coordinates of each feature point.

[0096] Step S4032: Based on the camera intrinsic parameters of the high-definition camera, the projection of the translation component of the LiDAR onto the image plane, the edge image pixel variables, and the three-dimensional coordinate feature point variables, construct a first mapping relationship between the width dimension of the edge image pixel set and the first and second direction components of the three-dimensional coordinate feature point set.

[0097] Step S4033: Based on the camera intrinsic parameters of the high-definition camera, the projection of the translation component of the LiDAR onto the image plane, the edge image pixel variables, and the three-dimensional coordinate feature point variables, construct a second mapping relationship between the height dimension of the edge image pixel set and the third and second directional components of the three-dimensional coordinate feature point set.

[0098] Wherein, the first directional component is perpendicular to the third directional component, and the second directional component is perpendicular to the plane formed by the first directional component and the third directional component. For example, the first directional component is the component in the X-axis direction, the second directional component is the component in the Z-axis direction, and the third directional component is the component in the Z-axis direction.

[0099] Step S4034: Based on the first mapping relationship and the second mapping relationship, an initial mapping relationship is obtained. The three-dimensional coordinate feature point set is input into the initial mapping relationship to obtain the set of pixel points of the mapped edge image.

[0100] For example, the initial mapping relationship can be represented as:

[0101]

[0102] in, This represents the width dimension value of the pixel set in the edge image. This is the internal reference of the first camera. This is the first projection of the translation component of the lidar onto the image plane. For the X-axis direction component of the three-dimensional coordinate feature point set, The Z-axis component of the three-dimensional coordinate feature point set. The height dimension value of the pixel set in the edge image. This is the internal reference of the second camera. This is the second projection of the translation component of the lidar onto the image plane. This represents the Y-axis component of the three-dimensional coordinate feature point set.

[0103] Step S4035: Based on the average distance error between the pixel set of the mapped edge image and the pixel set of the edge image, the initial mapping relationship is registered to obtain the target mapping relationship.

[0104] The formula for determining the average distance error is as follows:

[0105] in, This represents the average distance error. This refers to the number of data points in the set of pixels in the mapped edge image, or the set of pixels in the edge image. For the edge image pixel set, the first The width dimension value of each data point For the set of pixels in the mapped edge image The width dimension value of each data point For the edge image pixel set, the first The height dimension value of each data point. For the set of pixels in the mapped edge image The height dimension value of each data point.

[0106] In some optional implementations, if the average distance error is less than or equal to 1, the registration is completed and the initial mapping relationship is used as the target mapping relationship. If the average distance error is greater than 1, the small angle (±0.5°) of the rotation matrix R is fine-tuned, and the mapping calculation is repeated until the accuracy requirement is met, and the target mapping relationship is obtained.

[0107] Step S404: Identify the defect region in the enhanced grayscale image of the target, and determine the three-dimensional coordinates of the defect region based on the mapping relationship between the defect pixel coordinates of the defect region and the target.

[0108] Specifically, step S404 includes: Step S4041: Threshold segmentation and binarization processing is performed on the target enhanced grayscale image. Initial defect regions are selected based on the threshold segmentation and binarization processing results. Morphological processing is then performed on the initial defect regions to obtain the defect regions.

[0109] Specifically, the global grayscale distribution of the target enhanced grayscale image is calculated, the grayscale boundary value between the defect and the background is selected as the threshold, the grayscale image is converted into a binary image according to the threshold, the region of the enhanced grayscale image with pixel value less than the threshold is set to 1, the region of the enhanced grayscale image with pixel value greater than or equal to the threshold is set to 0, the connected regions with value of 1 in the binary image are marked, and the connected regions with an area greater than the preset minimum defect size (such as greater than or equal to 10 pixels) are filtered out to obtain the initial defect region.

[0110] In some optional implementations, the initial defect region is first subjected to erosion to remove small burrs and noise points at the edges; then, dilation is performed to restore the main size of the defect and eliminate false defects; the morphologically processed image is then subjected to connected component analysis to remove false defects with an area smaller than a preset threshold, while the contour of each defect region is extracted to obtain a final defect region with clear boundaries and no noise interference.

[0111] Step S4042: Input the defect pixel coordinates of the defect area into the target mapping relationship to obtain the three-dimensional coordinates of the defect area.

[0112] Step S405: Determine the defect parameters based on the three-dimensional coordinates of the defect, and determine the defect level of the defect area based on the defect parameters to obtain the target defect level.

[0113] Specifically, step S405 includes: Step S4051: Obtain the actual distance from each defect coordinate point in the three-dimensional coordinates of the defect to the axis of the target underground pipeline. Based on the deviation between the actual distance corresponding to each defect coordinate point and the preset distance, obtain the radial offset corresponding to each defect coordinate point.

[0114] The formula for determining the actual distance is:

[0115] in, For the defect in three-dimensional coordinates, the first The actual distance from the coordinate point of the defect to the axis of the target underground pipeline. For the defect in three-dimensional coordinates, the first The coordinates of the defect coordinate points.

[0116] In some alternative implementations, the formula for determining the radial offset is:

[0117] in, For the first The deviation between the actual distance and the preset distance corresponding to each defect coordinate point For the defect in three-dimensional coordinates, the first The actual distance from the coordinate point of the defect to the axis of the target underground pipeline. This is a preset distance, representing the distance from the normal area of ​​the pipeline to the axis.

[0118] Step S4052: Select the maximum radial offset from the radial offsets corresponding to multiple defect coordinate points, and determine the deformation rate based on the quotient of the maximum radial offset and the inner diameter of the target underground pipeline.

[0119] The maximum radial offset can be 0.08 meters.

[0120] In some alternative implementations, the formula for the deformation rate can be:

[0121] in, For deformation rate, For the first The deviation between the actual distance and the preset distance corresponding to each defect coordinate point The inner diameter of the target underground pipeline.

[0122] In some optional implementations, the defect parameters also include the maximum indentation depth, the defect area, and the defect extension length, wherein the formula for determining the indentation depth is:

[0123] in, For the defect area Defect depth of each data point This is the standard radius for a normal pipeline. For the first in the defect region Coordinates of data points.

[0124] In some alternative implementations, the maximum depression depth is selected as the maximum depression depth; for example, the maximum depression depth can be 0.06 meters.

[0125] In some optional implementations, the area of ​​the defect region is obtained by multiplying the number of pixels in the defect region by a scale mapping coefficient. The formula for determining the scale mapping coefficient is as follows:

[0126] in, These are the scale mapping coefficients. This is the standard radius for a normal pipeline. The width of the image in pixels.

[0127] For example, when rice, hour, The value is 0.001 meters per pixel, meaning that 1 pixel corresponds to an actual length of 0.001 meters.

[0128] In some alternative implementations, the area of ​​the defect region can be... .

[0129] In some alternative implementations, the length along the pipe axis is calculated by the difference in the Z coordinates of the point cloud. For example, the defect extension length can be 0.5 meters.

[0130] Step S4053: Compare the deformation rate with the preset deformation rate threshold, and determine the target defect level based on the comparison result.

[0131] The preset deformation rate threshold can be set according to the actual situation. For example, when the deformation rate is less than or equal to 5% of the pipe diameter, the target defect level is determined to be level 1 (minor). When the deformation rate is greater than 5% of the pipe diameter and less than or equal to 15% of the pipe diameter, the target defect level is determined to be level 2 (moderate). When the deformation rate is greater than 15% of the pipe diameter and less than or equal to 25% of the pipe diameter, the target defect level is determined to be level 3 (serious). When the deformation rate is greater than or equal to 25% of the pipe diameter, the target defect level is determined to be level 4 (major).

[0132] In some optional implementations, defects are divided into two main categories: structural defects and functional defects. Structural defects primarily affect the strength, stiffness, and service life of the pipeline. They are identified through a combination of registered image defect features (such as cracks and offset edges at interfaces) and point cloud 3D coordinate deviations (such as abnormal radial distances and surface depressions / protrusions), and include 10 categories: deformation (BX), cracking (PL), corrosion (FS), misalignment (CK), undulation (QF), discontinuity (TJ), interface material detachment (TL), concealed branch connection (AJ), foreign object penetration (CR), and leakage (SL). Functional defects primarily affect the pipeline's cross-sectional area and drainage performance. They are identified through a comprehensive analysis of the morphology of foreign objects in the image, the proportion of point cloud volume, and the reduction ratio of the cross-sectional area, and include 6 categories: sedimentation (CJ), scaling (JG), obstructions (ZW), residual wall / dam roots (CQ), tree roots (SG), and scum (FZ).

[0133] In some alternative implementations, all defects are uniformly classified into four levels: Level 1 (minor defect), Level 2 (moderate defect), Level 3 (serious defect), and Level 4 (major defect).

[0134] In some alternative implementations, for cracking (PL) defects: Grade 1 (crack), damage score 0.5; Grade 2 (crack opening), damage score 2; Grade 3 (fracture), damage score 5; Grade 4 (collapse), damage score 10; for corrosion (FS) defects: Grade 1 (mild, slight surface peeling), damage score 0.5; Grade 2 (moderate, significant surface corrosion), damage score 2; Grade 3 (severe, corrosion to the inner wall), damage score 5; for deposition (CJ) defects: 1 Level 1 (sediment thickness 20%~30% of pipe diameter), damage score 0.5; Level 2 (30%~40%), damage score 2; Level 3 (40%~50%), damage score 5; Level 4 (>50%), damage score 10; For root (SG) defects: Level 1 (water flow cross-sectional loss ≤15%), damage score 0.5; Level 2 (15%~25%), damage score 2; Level 3 (25%~50%), damage score 5; Level 4 (>50%), damage score 10.

[0135] In some optional implementations, when a single pipe segment has only one type of defect, the defect level is directly determined according to the above standards, which is the comprehensive defect level of the pipe segment. When a pipe segment has two or more defects, the level and corresponding score of each type of defect are determined separately. Then, the scores of all defects within a 1m range are summed (if the summed score exceeds 10 points, it is counted as 10 points). Finally, the level corresponding to the summed score or the highest level among all types of defects is taken as the comprehensive defect level of the pipe segment. For example, a pipe segment with a standard diameter of 0.5m, after registration and parameter calculation, has the following results: deformation is 0.06m, corresponding to a deformation rate of 12%, which is determined to be level 2 deformation (medium defect, corresponding to a score of 2) according to the grading standard. At the same time, slight corrosion is detected on the inner wall of the pipe segment, which is determined to be level 1 corrosion (minor defect, corresponding to a score of 0.5). According to the comprehensive judgment rule, the summed defect scores within a 1m range are 2.5 points, corresponding to a level 2 defect standard. Therefore, the comprehensive defect level of the pipe segment is determined to be level 2 (medium defect), providing a clear basis for the subsequent repair plan.

[0136] The underground pipeline defect detection method based on illumination compensation and point cloud filtering provided in this embodiment simplifies the coordinate mapping registration algorithm, significantly reducing computational power requirements and achieving real-time registration. A quantization model based on geometric projection establishes a precise mapping relationship between multi-source data, resulting in high accuracy in defect parameter calculation and reliable classification results. This embodiment can be directly adapted to existing underground pipeline robot hardware platforms without large-scale hardware modifications, resulting in low deployment costs and significantly improved detection efficiency and accuracy, effectively supporting intelligent operation and maintenance detection of underground pipelines.

[0137] This embodiment provides a method for detecting defects in underground pipelines based on illumination compensation and point cloud filtering, which can be used with computer equipment. Figure 5This is a fourth flowchart of the underground pipeline defect detection method based on illumination compensation and point cloud filtering according to an embodiment of the present invention, as follows: Figure 5 As shown, the process includes the following steps: Multi-source data acquisition within the pipeline; adaptive illumination compensation processing; improved point cloud filtering; simplified coordinate mapping and registration; construction and parameter calculation of a defect quantification model based on geometric projection; defect classification and determination; result output and storage.

[0138] This embodiment also provides an underground pipeline defect detection device based on illumination compensation and point cloud filtering. This device is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0139] This embodiment provides an underground pipeline defect detection device based on illumination compensation and point cloud filtering, such as... Figure 6 As shown, it includes: The image enhancement module 601 is used to acquire the color image of the inner wall of the underground pipeline and the three-dimensional point cloud data of the pipeline collected by the underground pipeline robot in the target underground pipeline, and to perform image enhancement processing on the color image of the inner wall of the pipeline using an adaptive gain coefficient to obtain the target enhanced grayscale image; the adaptive gain coefficient is a dynamic gain coefficient determined based on the grayscale value of the color image of the inner wall of the pipeline.

[0140] The point cloud filtering module 602 is used to determine a dynamic radius threshold based on the standard radius of the target underground pipeline, perform radius filtering on the pipeline's three-dimensional point cloud data based on the radius threshold, and perform statistical filtering on the radius-filtered pipeline's three-dimensional point cloud data to obtain the target's three-dimensional point cloud data.

[0141] The mapping relationship determination module 603 is used to extract the edge image pixel set in the target enhanced grayscale image, extract the three-dimensional coordinate feature point set corresponding to the edge image pixel set in the target three-dimensional point cloud data, and construct the target mapping relationship based on the edge image pixel set and the three-dimensional coordinate feature point set; the target mapping relationship is used to characterize the mapping relationship between the pixel coordinates of the target underground pipeline image and the three-dimensional coordinates of the point cloud.

[0142] The defect point cloud determination module 604 is used to identify defect regions in the target enhanced grayscale image and determine the three-dimensional coordinates of the defect region based on the mapping relationship between the defect pixel coordinates of the defect region and the target.

[0143] The defect level determination module 605 is used to determine the defect parameters based on the three-dimensional coordinates of the defect, and to determine the defect level of the defect area based on the defect parameters to obtain the target defect level.

[0144] In some alternative implementations, the image enhancement module 601 includes: The data acquisition unit is used to acquire color images of the inner wall of the pipeline collected by a high-definition camera configured on the underground pipeline robot at a preset frame rate, and to acquire three-dimensional point cloud data of the pipeline collected by a lidar configured on the underground pipeline robot at a preset density.

[0145] The grayscale mean determination unit is used to convert the color image of the inner wall of the pipe into an initial grayscale image, obtain the grayscale value of the grayscale image, and determine the grayscale mean of the inner wall of the pipe based on the grayscale value of the grayscale image.

[0146] The gain coefficient determination unit is used to obtain the target difference value based on the difference between the standard grayscale mean value and the grayscale mean value of the inner wall of the pipe, obtain the target quotient value based on the quotient of the target difference value and the total number of grayscale levels, and obtain the adaptive gain coefficient based on the sum of the target quotient value and 1.

[0147] The enhanced image determination unit is used to obtain the illumination component in the initial grayscale image, obtain the corrected illumination component based on the quotient of the illumination component and the adaptive gain coefficient, extract the reflection component based on the corrected illumination component and the illumination component, and perform normalization processing on the reflection component to obtain the target enhanced grayscale image.

[0148] In some alternative implementations, the point cloud filtering module 602 includes: The radius filtering unit is used to obtain the dynamic radius threshold based on the product of the standard radius of the pipeline and an empirical coefficient. A sphere is constructed with each data point in the pipeline's 3D point cloud data as the center and the dynamic radius threshold as the radius. The radius filtering is performed on the pipeline's 3D point cloud data based on the relationship between the number of neighboring point clouds in the sphere and the threshold number of neighboring point clouds.

[0149] The standard parameter determination unit is used to determine the average distance between each data point and multiple neighboring points in the radius-filtered 3D point cloud data of the pipeline, and to determine the mean distance and standard deviation of the average distance between multiple data points and their neighboring points.

[0150] The statistical filtering unit is used to determine the confidence interval based on the sum of the distance mean and the distance standard deviation of a preset multiple, and the difference between the distance mean and the distance standard deviation of a preset multiple. Based on the confidence interval, the 3D point cloud data of the pipeline after radius filtering is statistically filtered to obtain the target 3D point cloud data.

[0151] In some optional implementations, the mapping relationship determination module 603 includes: The first mapping relationship construction unit is used to construct a first mapping relationship between the width dimension of the edge image pixel set and the first and second direction components of the three-dimensional coordinate feature point set based on the camera intrinsic parameters of the high-definition camera, the projection of the translation component of the lidar onto the image plane, the edge image pixel variables, and the three-dimensional coordinate feature point variables.

[0152] The second mapping relationship construction unit is used to construct a second mapping relationship between the height dimension of the edge image pixel set and the third direction component and the second direction component of the three-dimensional coordinate feature point set based on the camera intrinsic parameters of the high-definition camera, the projection of the translation component of the lidar onto the image plane, the edge image pixel point variables, and the three-dimensional coordinate feature point variables; the first direction component is perpendicular to the third direction component, and the second direction component is perpendicular to the plane formed by the first direction component and the third direction component.

[0153] The pixel mapping unit is used to obtain an initial mapping relationship based on the first mapping relationship and the second mapping relationship, input the three-dimensional coordinate feature point set into the initial mapping relationship, and obtain the pixel point set of the mapped edge image.

[0154] The mapping relationship registration unit is used to register the initial mapping relationship based on the average distance error between the pixel set of the mapped edge image and the pixel set of the edge image, so as to obtain the target mapping relationship.

[0155] In some alternative implementations, the defect point cloud determination module 604 includes: In the target enhanced grayscale image, defect regions are identified, and the three-dimensional coordinates of the defect regions are determined based on the mapping relationship between the defect pixel coordinates and the target, including: The defect region determination unit is used to perform threshold segmentation and binarization processing on the target enhanced grayscale image, select the initial defect region based on the threshold segmentation and binarization processing result, and perform morphological processing on the initial defect region to obtain the defect region.

[0156] The defect coordinate determination unit is used to input the defect pixel coordinates of the defect area into the target mapping relationship to obtain the three-dimensional coordinates of the defect area.

[0157] In some optional implementations, the defect level determination module 605 includes: The radial offset determination unit is used to obtain the actual distance from each defect coordinate point in the three-dimensional coordinates of the defect to the axis of the target underground pipeline. Based on the deviation between the actual distance corresponding to each defect coordinate point and the preset distance, the radial offset corresponding to each defect coordinate point is obtained.

[0158] The deformation rate determination unit is used to select the maximum radial offset from the radial offsets corresponding to multiple defect coordinate points, and to determine the deformation rate based on the quotient of the maximum radial offset and the inner diameter of the target underground pipeline.

[0159] The defect level determination unit is used to compare the deformation rate with a preset deformation rate threshold and determine the target defect level based on the comparison result.

[0160] The underground pipeline defect detection device based on illumination compensation and point cloud filtering provided in this embodiment of the invention can execute the underground pipeline defect detection method based on illumination compensation and point cloud filtering provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method. Further functional descriptions of the above modules and units are the same as in the corresponding embodiments described above, and will not be repeated here.

[0161] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0162] The following is a detailed reference. Figure 7 This diagram illustrates a suitable structural schematic for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 701, which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) 702 or a program loaded from memory 708 into random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device. The processor 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0163] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic devices to exchange data via wireless or wired communication with other devices. Although Figure 7 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0164] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 709, or installed from a memory 708, or installed from a ROM 702. When the computer program is executed by the processor 701, it performs the functions defined in the underground pipeline defect detection method based on illumination compensation and point cloud filtering according to embodiments of the present invention.

[0165] Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0166] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the underground pipeline defect detection method based on illumination compensation and point cloud filtering shown in the above embodiments is implemented.

[0167] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0168] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for detecting defects in underground pipelines based on illumination compensation and point cloud filtering, characterized in that, The method includes: The system acquires color images of the inner wall of the underground pipeline and three-dimensional point cloud data of the pipeline collected by the underground pipeline robot in the target underground pipeline. The color images of the inner wall of the pipeline are then enhanced using an adaptive gain coefficient to obtain an enhanced grayscale image of the target. The adaptive gain coefficient is a dynamic gain coefficient determined based on the grayscale value of the color images of the inner wall of the pipeline. The process of acquiring color images of the inner wall of a target underground pipeline and 3D point cloud data of the pipeline collected by an underground pipeline robot, and performing image enhancement processing on the color images of the inner wall of the pipeline using an adaptive gain coefficient to obtain an enhanced grayscale image, includes: acquiring color images of the inner wall of the pipeline collected by a high-definition camera configured on the underground pipeline robot at a preset frame rate; acquiring 3D point cloud data of the pipeline collected by a lidar configured on the underground pipeline robot at a preset density; converting the color images of the inner wall of the pipeline into an initial grayscale image; acquiring the grayscale values ​​of the grayscale image; and, based on the... The grayscale values ​​of the grayscale image are used to determine the average grayscale value of the inner wall of the pipe; a target difference value is obtained based on the difference between the standard average grayscale value and the average grayscale value of the inner wall of the pipe; a target quotient value is obtained based on the quotient of the target difference value and the total number of grayscale levels; the adaptive gain coefficient is obtained based on the sum of the target quotient value and 1; the illumination component in the initial grayscale image is obtained; a corrected illumination component is obtained based on the quotient of the illumination component and the adaptive gain coefficient; a reflection component is extracted based on the corrected illumination component and the illumination component; the reflection component is normalized to obtain the target enhanced grayscale image; A dynamic radius threshold is determined based on the standard radius of the target underground pipeline. Radius filtering is then applied to the pipeline's 3D point cloud data based on the radius threshold. Statistical filtering is then applied to the radius-filtered pipeline's 3D point cloud data to obtain the target 3D point cloud data. Edge image pixel set is extracted from the enhanced grayscale image of the target, and three-dimensional coordinate feature point set corresponding to the edge image pixel set is extracted from the three-dimensional point cloud data of the target. A target mapping relationship is constructed based on the edge image pixel set and the three-dimensional coordinate feature point set. This target mapping relationship characterizes the mapping relationship between the pixel coordinates of the target underground pipeline image and the three-dimensional coordinates of the point cloud. The edge image pixel set is the set of pixel coordinates of the inner wall contour and defect edges extracted by an edge detection algorithm in the enhanced grayscale image of the target. The three-dimensional coordinate feature point set is the set of three-dimensional coordinates corresponding to the edge image pixels in the three-dimensional point cloud data of the target. In the target enhanced grayscale image, a defect region is identified, and the three-dimensional coordinates of the defect region are determined based on the mapping relationship between the defect pixel coordinates of the defect region and the target. The step of identifying defect regions in the enhanced grayscale image of the target and determining the three-dimensional coordinates of the defect regions based on the mapping relationship between the defect pixel coordinates of the defect regions and the target includes: performing threshold segmentation binarization processing on the enhanced grayscale image of the target; filtering initial defect regions based on the threshold segmentation binarization processing result; performing morphological processing on the initial defect regions to obtain the defect regions; and inputting the defect pixel coordinates of the defect regions into the target mapping relationship to obtain the three-dimensional coordinates of the defect regions. Based on the three-dimensional coordinates of the defect, the defect parameters are determined, and based on the defect parameters, the defect level of the defect area is determined to obtain the target defect level.

2. The method according to claim 1, characterized in that, The process involves determining a dynamic radius threshold based on the standard radius of the target underground pipeline, performing radius filtering on the pipeline's 3D point cloud data based on the radius threshold, and then performing statistical filtering on the radius-filtered pipeline's 3D point cloud data to obtain the target 3D point cloud data, including: The dynamic radius threshold is obtained by multiplying the standard radius of the pipeline with an empirical coefficient. A sphere is constructed with each data point in the pipeline's 3D point cloud data as the center and the dynamic radius threshold as the radius. Radius filtering is performed on the pipeline's 3D point cloud data based on the relationship between the number of neighboring point clouds in the sphere and the threshold number of neighboring point clouds. Determine the average distance between each data point and multiple neighboring points in the 3D point cloud data of the pipeline after radius filtering, and determine the mean distance and standard deviation of the average distance between the multiple data points and neighboring points; A confidence interval is determined based on the sum of the distance mean and the distance standard deviation of a preset multiple, and the difference between the distance mean and the distance standard deviation of a preset multiple. The target three-dimensional point cloud data is then obtained by statistical filtering the pipeline three-dimensional point cloud data after radius filtering based on the confidence interval.

3. The method according to claim 1, characterized in that, The step of constructing a target mapping relationship based on the edge image pixel set and the three-dimensional coordinate feature point set includes: Based on the camera intrinsic parameters of the high-definition camera, the projection of the translation component of the lidar onto the image plane, the edge image pixel variables, and the three-dimensional coordinate feature point variables, a first mapping relationship is constructed between the width dimension of the edge image pixel set and the first and second direction components of the three-dimensional coordinate feature point set. Based on the camera intrinsic parameters of the high-definition camera, the projection of the translation component of the LiDAR onto the image plane, the edge image pixel variables, and the three-dimensional coordinate feature point variables, a second mapping relationship is constructed between the height dimension of the edge image pixel set and the third directional component and the second directional component of the three-dimensional coordinate feature point set; the first directional component is perpendicular to the third directional component, and the second directional component is perpendicular to the plane formed by the first directional component and the third directional component. Based on the first mapping relationship and the second mapping relationship, an initial mapping relationship is obtained. The three-dimensional coordinate feature point set is input into the initial mapping relationship to obtain the mapped edge image pixel point set. Based on the average distance error between the mapped edge image pixel set and the edge image pixel set, the initial mapping relationship is registered to obtain the target mapping relationship.

4. The method according to claim 1, characterized in that, The defect parameters include the maximum radial offset; the process of determining the defect parameters based on the three-dimensional coordinates of the defect, and determining the defect level of the defect region based on the defect parameters to obtain the target defect level includes: The actual distance from each defect coordinate point in the three-dimensional coordinates of the defect to the axis of the target underground pipeline is obtained. Based on the deviation between the actual distance corresponding to each defect coordinate point and the preset distance, the radial offset corresponding to each defect coordinate point is obtained. The maximum radial offset is selected from the radial offsets corresponding to multiple defect coordinate points, and the deformation rate is determined based on the quotient of the maximum radial offset and the inner diameter of the target underground pipeline. The deformation rate is compared with a preset deformation rate threshold, and the target defect level is determined based on the comparison result.

5. A defect detection device for underground pipelines based on illumination compensation and point cloud filtering, characterized in that, The device includes: The image enhancement module is used to acquire color images of the inner wall of the underground pipeline and three-dimensional point cloud data of the pipeline collected by the underground pipeline robot in the target underground pipeline, and to perform image enhancement processing on the color images of the inner wall of the pipeline using an adaptive gain coefficient to obtain a target enhanced grayscale image; the adaptive gain coefficient is a dynamic gain coefficient determined based on the grayscale value of the color images of the inner wall of the pipeline. The image enhancement module includes: a data acquisition unit, used to acquire color images of the inner wall of the pipeline acquired by a high-definition camera configured on the underground pipeline robot at a preset frame rate, and to acquire three-dimensional point cloud data of the pipeline acquired by a lidar configured on the underground pipeline robot at a preset density; a grayscale mean determination unit, used to convert the color image of the inner wall of the pipeline into an initial grayscale image, acquire the grayscale values ​​of the grayscale image, and determine the grayscale mean of the inner wall of the pipeline based on the grayscale values ​​of the grayscale image; a gain coefficient determination unit, used to obtain a target difference value based on the difference between the standard grayscale mean value and the grayscale mean value of the inner wall of the pipeline, obtain a target quotient value based on the quotient of the target difference value and the total number of grayscale levels, and obtain an adaptive gain coefficient based on the sum of the target quotient value and 1; and an enhanced image determination unit, used to acquire the illumination component in the initial grayscale image, obtain a corrected illumination component based on the quotient of the illumination component and the adaptive gain coefficient, extract a reflection component based on the corrected illumination component and the illumination component, and normalize the reflection component to obtain the target enhanced grayscale image; The point cloud filtering module is used to determine a dynamic radius threshold based on the standard radius of the target underground pipeline, perform radius filtering on the pipeline's three-dimensional point cloud data based on the radius threshold, and perform statistical filtering on the radius-filtered pipeline's three-dimensional point cloud data to obtain the target's three-dimensional point cloud data. The mapping relationship determination module is used to extract an edge image pixel set from the target enhanced grayscale image, extract a set of three-dimensional coordinate feature points corresponding to the edge image pixel set from the target three-dimensional point cloud data, and construct a target mapping relationship based on the edge image pixel set and the three-dimensional coordinate feature point set. The target mapping relationship characterizes the mapping relationship between the pixel coordinates of the target underground pipeline image and the three-dimensional coordinates of the point cloud. The edge image pixel set is the set of pixel coordinates of the inner wall contour and defect edges extracted by an edge detection algorithm in the target enhanced grayscale image. The three-dimensional coordinate feature point set is the set of three-dimensional coordinates corresponding to the edge image pixels in the target three-dimensional point cloud data. The defect point cloud determination module is used to identify defect regions in the target enhanced grayscale image and determine the three-dimensional coordinates of the defect region based on the mapping relationship between the defect pixel coordinates of the defect region and the target. The defect point cloud determination module includes: a defect region determination unit, used to perform threshold segmentation binarization processing on the target enhanced grayscale image, filter the initial defect region according to the threshold segmentation binarization processing result, and perform morphological processing on the initial defect region to obtain the defect region; and a defect coordinate determination unit, used to input the defect pixel coordinates of the defect region into the target mapping relationship to obtain the defect three-dimensional coordinates of the defect region. The defect level determination module is used to determine defect parameters based on the three-dimensional coordinates of the defect, and to determine the defect level of the defect area based on the defect parameters to obtain the target defect level.

6. An electronic device, characterized in that, include: The system includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes the computer instructions to perform the underground pipeline defect detection method based on illumination compensation and point cloud filtering as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the underground pipeline defect detection method based on illumination compensation and point cloud filtering as described in any one of claims 1 to 4.

8. A computer program product, characterized in that, The method includes computer instructions for causing a computer to execute the underground pipeline defect detection method based on illumination compensation and point cloud filtering as described in any one of claims 1 to 4.

Images