Visual-based three-dimensional quantification method and device for sewer defects

Abstract

The application relates to the technical field of image processing, in particular to a visual-based three-dimensional quantification method and device for defects of a drainage pipeline, wherein the method comprises the following steps: at least one of sparse point cloud reconstruction, dense point cloud reconstruction, grid reconstruction, texture mapping and scale calibration is performed on collected visual images to obtain a real three-dimensional model of the drainage pipeline; based on the real three-dimensional model, the center point of the drainage pipeline is calculated to calculate the center line of the drainage pipeline according to the center point; and based on the center line, at least one actual defect of the drainage pipeline is identified from the following defects: a joint defect, a disconnection defect, a fluctuation defect and a deformation defect, so that actual defect information of the drainage pipeline is output according to the at least one actual defect. Therefore, the problems of low efficiency, missing details, incomplete identification of cracks, leakage and sediments and misjudgment caused by the fact that the related art is greatly affected by artificial subjective judgment or laser point clouds are solved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a vision-based three-dimensional quantification method and apparatus for defects in drainage pipes. Background Technology

[0002] Drainage pipes are an important part of urban infrastructure, undertaking the core function of collecting sewage and rainwater and transporting them to sewage treatment plants or natural water bodies. They are crucial for the sustainable use of water resources, prevention of urban flooding, and road safety.

[0003] In related technologies, visual inspection technologies such as CCTV (Closed-Circuit Television Inspection) or QV (Quick View) are commonly used. These technologies acquire images of the inside of pipes through camera devices, and the video images are then analyzed manually. This allows for the identification of surface defects such as cracks, damage, leaks, deformation, and scaling, and has the advantages of simple equipment and convenient operation. Alternatively, laser scanning 3D reconstruction technology can be used for drainage pipes. By scanning with lasers, spatial geometric data of the inner wall of the pipe can be obtained, and a 3D model can be built to identify structural defects such as misalignment, ellipticity, deformation, and deposit thickness.

[0004] However, due to the significant influence of human subjective judgment on related technologies, or the sparse laser point cloud, detection efficiency is easily reduced, and details are easily missing. This can lead to incomplete identification or misjudgment of information such as cracks, leaks, and deposits, affecting the accuracy of pipeline structural health assessment and the reliability of maintenance decisions, which urgently needs to be addressed. Summary of the Invention

[0005] This invention provides a vision-based three-dimensional quantification method and device for drainage pipe defects, which solves the problem in related technologies that are easily inefficient and lack detail due to the influence of human subjective judgment or sparse laser point clouds, resulting in incomplete or misjudged identification of information such as cracks, leaks, and deposits.

[0006] A first aspect of the present invention provides a vision-based three-dimensional quantification method for defects in drainage pipes, comprising the following steps: acquiring a visual image of the drainage pipe, and performing at least one of the following processing on the visual image: sparse point cloud reconstruction, dense point cloud reconstruction, mesh reconstruction, and texture mapping, to construct an initial three-dimensional model of the drainage pipe; scaling the initial three-dimensional model to obtain a true three-dimensional model of the drainage pipe; calculating the center point of the drainage pipe based on the true three-dimensional model, and calculating the centerline of the drainage pipe based on the center point; and identifying at least one actual defect of the drainage pipe, including misalignment defects, disconnection defects, undulation defects, and deformation defects, based on the centerline, and outputting actual defect information of the drainage pipe based on the at least one actual defect.

[0007] Through the above technical means, the embodiments of the present invention can calculate the centerline of the drainage pipeline based on the real three-dimensional model, and then identify actual defects such as misalignment defects, disconnection defects, undulation defects, and deformation defects. It can realize quantitative analysis of pipeline defects, intuitively present the internal structure of the pipeline, accurately capture defect details, and reduce the missed detection rate. At the same time, it can reduce the workload of manual inspection, save inspection time and maintenance costs, provide a reliable basis for pipeline structural health assessment and maintenance decisions, and provide high-precision data support for pipeline modification, operation monitoring and long-term management.

[0008] Optionally, in one embodiment of the present invention, the step of outputting the actual defect information of the drainage pipe according to the at least one actual defect includes: calculating the projected area of ​​any two types of point clouds on the plane perpendicular to the centerline, determining the single-sided area based on the projected area, and classifying the misalignment defect according to the single-sided area as one of the actual defect information; obtaining the maximum distance between the two types of point clouds, classifying the disjoint defect according to the maximum distance as one of the actual defect information; projecting the centerline onto a plane parallel to the centerline, and using the start-end line of the drainage pipe as a reference line, calculating the maximum height difference between the centerline and the reference line, classifying the undulation defect according to the maximum height difference as one of the actual defect information; calculating the theoretical radius and minimum radius of each segment based on the segments of the drainage pipe, and calculating the deformation rate of each segment according to the theoretical radius and the minimum radius, classifying the deformation defect according to the deformation rate as one of the actual defect information.

[0009] Through the above technical means, the embodiments of the present invention can classify defects according to geometric characteristic parameters such as the single-sided area, maximum distance, and maximum height difference of the defects, so as to realize quantitative assessment of defects, accurately reflect the severity of defects, assist in pipeline health assessment and maintenance decision-making, and ensure the safety of pipeline operation and the scientific nature of maintenance work.

[0010] Optionally, in one embodiment of the present invention, the step of performing at least one of sparse point cloud reconstruction, dense point cloud reconstruction, mesh reconstruction, and texture reconstruction on the visual image to construct an initial three-dimensional model of the drainage pipe includes: using a preset universal camera and motion recovery structure system to perform camera pose recovery and sparse point cloud reconstruction processing on the visual image to obtain a processed sparse point cloud image; and using a preset open multi-view stereo vision library to perform dense point cloud reconstruction, mesh reconstruction, and texture mapping processing on the sparse point cloud image to establish the initial three-dimensional model.

[0011] Through the above technical means, the embodiments of the present invention can construct an initial three-dimensional model that accurately reflects the spatial structure and surface details of the drainage pipe through sparse point cloud reconstruction, dense point cloud reconstruction, mesh reconstruction, texture reconstruction and other processing, accurately capture the details of the drainage pipe to reduce the missed detection rate, and can also serve as the basis for subsequent scale calibration, center point and center line calculation and defect identification.

[0012] Optionally, in one embodiment of the present invention, calculating the center point of the drainage pipe to calculate the centerline of the drainage pipe based on the center point includes: calculating the center point based on a preset generalized rotational symmetry axis; performing natural spline interpolation on the center point multiple times to obtain multiple interpolation results; and fitting the multiple interpolation results to obtain the centerline.

[0013] Through the above technical means, the embodiments of the present invention can calculate the center point according to the generalized rotational symmetry axis to accurately determine the spatial position of the pipeline along the axial direction; then, the center point is interpolated by natural spline along the pipeline axial direction to generate a continuous and smooth pipeline centerline, thereby accurately describing the pipeline direction, curvature and geometry, and providing a reliable reference benchmark for defect location, spatial measurement and automated inspection path planning.

[0014] Optionally, in one embodiment of the present invention, identifying at least one actual defect among misalignment, disjointness, undulation, and deformation defects of the drainage pipe based on the centerline includes: calculating the dot product of the normal vector of the drainage pipe's surface and the gradient of the centerline; determining the surface whose absolute value of the dot product is greater than a preset threshold; and determining the at least one actual defect based on the surface.

[0015] Through the above technical means, the embodiments of the present invention can determine the actual defects based on the dot product of the surface normal vector of the drainage pipe and the gradient of the centerline. This can effectively improve the identification accuracy of misalignment defects, disjoint defects, undulation defects and deformation defects, enhance the detection capability of abnormal internal pipe structure, reduce the false negative rate, and provide more reliable data support for pipeline health assessment and maintenance decisions.

[0016] A second aspect of the present invention provides a vision-based three-dimensional quantization device for drainage pipe defects, comprising: a construction module for acquiring visual images of the drainage pipe and performing at least one of the following processing methods on the visual images: sparse point cloud reconstruction, dense point cloud reconstruction, mesh reconstruction, and texture mapping, to construct an initial three-dimensional model of the drainage pipe; a calibration module for performing scale calibration on the initial three-dimensional model to obtain a true three-dimensional model of the drainage pipe; a calculation module for calculating the center point of the drainage pipe based on the true three-dimensional model, and calculating the centerline of the drainage pipe based on the center point; and a quantization module for identifying at least one actual defect of the drainage pipe, including misalignment defects, disjoint defects, undulation defects, and deformation defects, based on the centerline, and outputting actual defect information of the drainage pipe based on the at least one actual defect.

[0017] Optionally, in one embodiment of the present invention, the quantization module includes: a first division unit, configured to calculate the projected area of ​​any two types of point clouds on a plane perpendicular to the centerline, to determine the single-sided area based on the projected area, and to classify the misalignment defect according to the single-sided area as one of the actual defect information; a second division unit, configured to obtain the maximum distance between the two types of point clouds, to classify the disjoint defect according to the maximum distance as one of the actual defect information; a third division unit, configured to project the centerline onto a plane parallel to the centerline, and using the start-end line of the drainage pipe as a reference line, calculate the maximum height difference between the centerline and the reference line, to classify the undulation defect according to the maximum height difference as one of the actual defect information; and a fourth division unit, configured to calculate the theoretical radius and minimum radius of each segment based on the segmentation of the drainage pipe, and calculate the deformation rate of each segment according to the theoretical radius and the minimum radius, to classify the deformation defect according to the deformation rate as one of the actual defect information.

[0018] Optionally, in one embodiment of the present invention, the construction module includes: a processing unit, configured to perform camera pose recovery and sparse point cloud reconstruction processing on the visual image using a preset universal camera and motion recovery structure system to obtain a processed sparse point cloud image; and a construction unit, configured to perform dense point cloud reconstruction, mesh reconstruction, and texture mapping processing on the sparse point cloud image using a preset open multi-view stereo vision library to establish the initial three-dimensional model.

[0019] Optionally, in one embodiment of the present invention, the calculation module includes: a first calculation unit, configured to calculate the center point according to a preset generalized rotational symmetry axis; an interpolation unit, configured to perform natural spline interpolation on the center point multiple times to obtain multiple interpolation results; and a fitting unit, configured to fit the multiple interpolation results to obtain the center line.

[0020] Optionally, in one embodiment of the present invention, the quantization module includes: a second calculation unit for calculating the dot product of the surface normal vector of the drainage pipe and the gradient of the center line; a first determination unit for determining the surface whose absolute value of the dot product is greater than a preset threshold; and a second determination unit for determining the at least one actual defect based on the surface.

[0021] A third aspect of the present invention provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the vision-based three-dimensional quantification method for drainage pipe defects as described in the above embodiments.

[0022] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described vision-based three-dimensional quantification method for drainage pipe defects.

[0023] A fifth aspect of the present invention provides a computer program product, including a computer program that, when executed, is used to implement the above-described vision-based three-dimensional quantification method for drainage pipe defects.

[0024] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0025] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0026] Figure 1 A flowchart illustrating a vision-based three-dimensional quantification method for drainage pipe defects according to an embodiment of the present invention;

[0027] Figure 2 This is a schematic diagram of a GoPro 11 and a captured image according to an embodiment of the present invention;

[0028] Figure 3 This is a flowchart of a three-dimensional reconstruction process for a drainage pipe according to an embodiment of the present invention;

[0029] Figure 4A schematic diagram of camera pose and sparse point cloud generated by COLMAP according to an embodiment of the present invention;

[0030] Figure 5 This is a schematic diagram of a dense point cloud of a drainage pipe according to an embodiment of the present invention;

[0031] Figure 6 This is a schematic diagram of a drainage pipe mesh model according to an embodiment of the present invention;

[0032] Figure 7 This is a schematic diagram of a pipe texture model according to an embodiment of the present invention;

[0033] Figure 8 This is a schematic diagram showing the comparison of drainage pipe dimensions before and after calibration according to an embodiment of the present invention;

[0034] Figure 9 This is a schematic diagram illustrating the calculation principle of the centerline parameter equation of a drainage pipeline according to an embodiment of the present invention;

[0035] Figure 10 This is a schematic diagram illustrating the principle of cubic natural spline interpolation fitting for the center point of a pipe according to an embodiment of the present invention.

[0036] Figure 11 This is a schematic diagram of the centroid offset of the pipeline point cloud according to an embodiment of the present invention;

[0037] Figure 12 This is a schematic diagram illustrating the robustness of ROSA (Generalized Rotational Symmetry Axis) points to point cloud defects according to an embodiment of the present invention.

[0038] Figure 13 This is a schematic diagram of the error channel and node detachment cloud screening according to an embodiment of the present invention;

[0039] Figure 14 This is a schematic diagram of defect point cloud classification according to an embodiment of the present invention;

[0040] Figure 15 This is a schematic diagram of the single-sided area of ​​a misalignment defect according to an embodiment of the present invention;

[0041] Figure 16 A schematic diagram of the decoupling distance calculated by the defect quantification algorithm of decoupling according to an embodiment of the present invention;

[0042] Figure 17 This is a schematic diagram of the projection of the centerline of a pipe model in an embodiment of the present invention onto the ZY plane;

[0043] Figure 18 This is a schematic diagram of the projection of the center line of a pipe model and a reference straight line according to an embodiment of the present invention;

[0044] Figure 19 This is a schematic diagram of the quantification results of the fluctuation defect in a pipeline model according to an embodiment of the present invention;

[0045] Figure 20 This is a flowchart illustrating the calculation of the deformation rate of a pipe model according to an embodiment of the present invention.

[0046] Figure 21 This is a flowchart illustrating a vision-based three-dimensional quantification method for drainage pipe defects according to an embodiment of the present invention.

[0047] Figure 22 A block diagram illustrating a vision-based three-dimensional quantification device for drainage pipe defects according to an embodiment of the present invention;

[0048] Figure 23 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention.

[0049] Figure label:

[0050] 10-Vision-based 3D quantization device for drainage pipe defects; 100-Construction module, 200-Calibration module, 300-Computation module, 400-Quantization module; 2301-Memory, 2302-Processor, 2303-Communication interface. Detailed Implementation

[0051] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0052] The following describes a vision-based three-dimensional quantification method and apparatus for drainage pipe defects according to embodiments of the present invention, with reference to the accompanying drawings. Addressing the technical problems mentioned in the background art, such as the significant influence of subjective human judgment or sparse laser point clouds leading to low efficiency and missing details, resulting in incomplete or misjudged identification of information such as cracks, leaks, and sediments, the present invention provides a vision-based three-dimensional quantification method for drainage pipe defects. In this method, the acquired visual images undergo sparse point cloud reconstruction, dense point cloud reconstruction, mesh reconstruction, texture mapping, and scale calibration to construct a realistic three-dimensional model of the drainage pipe. Then, based on the realistic three-dimensional model, the centerline of the drainage pipe is calculated to identify misalignment defects in the drainage pipe. This system identifies defects such as discontinuity, undulation, and deformation in drainage pipelines, outputting actual defect information. It enables precise description of the overall spatial morphology of drainage pipelines, allowing for quantitative analysis of defects and a direct visual representation of the internal structure. This precise capture of defect details reduces the rate of missed detections. Simultaneously, it reduces reliance on manual inspections, saving inspection time and maintenance costs, and improving the efficiency and scientific rigor of pipeline structural health assessments. Furthermore, it provides a reliable data foundation for pipeline maintenance planning, defect evolution monitoring, and subsequent pipeline modifications. It is scalable and repeatable, suitable for large-scale pipeline inspection and long-term operation management. This solves the problems of inefficiency and missing details caused by the significant influence of subjective human judgment or sparse laser point clouds in related technologies, leading to incomplete or misjudged identification of information such as cracks, leaks, and deposits.

[0053] Specifically, Figure 1 This is a flowchart illustrating a vision-based three-dimensional quantification method for drainage pipe defects provided in an embodiment of the present invention.

[0054] like Figure 1 As shown, the vision-based three-dimensional quantification method for drainage pipe defects includes the following steps:

[0055] In step S101, a visual image of the drainage pipe is acquired, and at least one of the following processes is performed on the visual image: sparse point cloud reconstruction, dense point cloud reconstruction, mesh reconstruction, and texture mapping, in order to construct an initial three-dimensional model of the drainage pipe.

[0056] The acquisition methods may include, but are not limited to, fixing a camera to a mobile inspection device (such as a pipeline robot, crawler, or small inspection vehicle) to capture images of the inner wall of the pipeline in real time as it travels along the pipeline; or using an adjustable-direction camera installed in a pipeline inspection well to capture images of the pipeline inlet and key nodes at fixed points; the acquired images can be used for subsequent defect identification, crack detection, and sediment analysis.

[0057] It can be noted that methods for sparse point cloud reconstruction, dense point cloud reconstruction, mesh reconstruction, and texture mapping include, but are not limited to, structure-driven motion algorithms, multi-view stereo matching algorithms, triangulation, or Poisson reconstruction algorithms. Specifically, sparse point cloud reconstruction can extract and match feature points from multi-view images to estimate the camera's external pose parameters and the initial spatial feature point distribution of the scene, providing geometric constraints and an initial model for subsequent dense reconstruction. Dense point cloud reconstruction can generate high-density 3D point cloud data based on sparse point clouds and camera pose information, using disparity estimation and depth fusion techniques to obtain more complete information about the pipe's inner wall morphology. Mesh reconstruction can generate continuous 3D surface models. Texture mapping can map the original image texture information onto the surface of a 3D model, achieving realistic visualization of the pipe's internal structure and spatial localization of defects.

[0058] As a concrete example, such as Figure 2 As shown, this embodiment of the invention can use a GoPro 11 camera + MAX LENMOD fisheye lens (190° field of view, 27 megapixels), attached to the front end of a pipeline robot, utilizing the robot's built-in lighting system for supplemental illumination, to capture visual images of the inside of the drainage pipe. The fisheye lens is chosen because of its wide field of view, which reduces the number of frames captured and eliminates the need for image stitching. The robot can first quickly move forward to the end of the pipe, then reverse at a constant speed of 0.1 m / s to capture video, ensuring uniform image spacing along the pipe length (0.1 m / frame); uniform reversal avoids image jitter when the robot crosses obstacles. Furthermore, this embodiment of the invention can use FFmpeg to extract frames at 1 second / frame, adding a water surface mask to images containing water; the frame extraction interval of 0.1 m ensures that images directly above defects are captured and meets the matching requirement of 3D reconstruction that "the same object must appear in at least 3 images."

[0059] This invention can accurately restore the spatial geometry and surface texture features of the inner wall of a drainage pipe by acquiring visual images of the pipe and performing sparse point cloud reconstruction, dense point cloud reconstruction, mesh reconstruction, and texture mapping on the visual images. This allows for the construction of an initial 3D model that is consistent with the actual pipe shape, which can realistically reflect the internal dimensions, deformation, and surface condition of the pipe. It can also provide a reliable geometric basis for subsequent scale calibration, centerline extraction, and structural defect identification.

[0060] Optionally, in one embodiment of the present invention, at least one of sparse point cloud reconstruction, dense point cloud reconstruction, mesh reconstruction, and texture reconstruction is performed on the visual image to construct an initial three-dimensional model of the drainage pipe, including: using a preset universal camera and motion recovery structure system to perform camera pose recovery and sparse point cloud reconstruction processing on the visual image to obtain a processed sparse point cloud image; and using a preset open multi-view stereo vision library to perform dense point cloud reconstruction, mesh reconstruction, and texture mapping processing on the sparse point cloud image to establish an initial three-dimensional model.

[0061] It can be explained that, for example Figure 3 As shown, the preset COLMAP, as a graphical user interface software tool, can be used for camera pose recovery and sparse point cloud reconstruction of images. By estimating the extrinsic and intrinsic parameters of the camera under different viewpoints, it recovers the spatial pose relationship of the camera and generates a preliminary sparse 3D point cloud model based on feature matching results, providing basic data support for subsequent dense point cloud reconstruction and meshing processing. The preset OpenMVS (Open Multi-View Stereo) can be used to process the input sparse point cloud and camera pose data into dense point cloud data, generating a high-precision initial 3D model. It can further support mesh reconstruction and texture mapping processing, converting dense point clouds into continuous 3D surface models and mapping the original image texture onto the 3D mesh surface, thereby realizing realistic visualization of the internal structure of the pipeline and spatial location of defects.

[0062] Combination Figure 4 , Figure 5 , Figure 6 , Figure 7 As shown below, embodiments are listed to provide a detailed description of the sparse point cloud reconstruction, dense point cloud reconstruction, mesh reconstruction, and texture reconstruction processes of the present invention.

[0063] For example, such as Figure 4 As shown, the embodiments of the present invention can use the COLMAPv3.8 open source framework to output camera pose and sparse point cloud (point density of about 50 points / cm²) through "feature detection → feature matching → structure and motion reconstruction". Among them, sequential matching can save computation time, and since the images are extracted in chronological order, there is visual overlap between consecutive frames.

[0064] Furthermore, such as Figure 5 , Figure 6 , Figure 7As shown, in this embodiment of the invention, the COLMAP result can be read through the OpenMVS framework, a dense point cloud (point density ≥ 200 points / cm²) can be generated through the Patch-Match algorithm, a mesh model can be generated through Jancosek algorithm triangulation, and finally the original image texture can be mapped using the Waechter algorithm to obtain an initial three-dimensional model with realistic colors.

[0065] In step S102, the initial three-dimensional model is scaled to obtain a true three-dimensional model of the drainage pipe.

[0066] Among them, scale calibration can be used to map the 3D reconstruction model to the actual physical size, and to convert the units of sparse or dense point clouds from the relative coordinate system to the real world scale, thereby realizing the quantitative measurement and precise positioning of the 3D model.

[0067] As one possible way to achieve this, such as Figure 8 As shown, due to the scale uncertainty of the initial 3D model in visual reconstruction, the embodiments of the present invention can use the actual pipe diameter as a reference and scale the model coordinates in MeshLab software; for example, the pipe diameter is 3.0 (scalar) before calibration, and the actual pipe diameter is 500mm. It is necessary to scale the coordinates of all points in the model by a factor of 167.1 so that the size of the real 3D model after scale calibration is consistent with the real pipe.

[0068] In actual implementation, after reconstructing a 200.4cm long, 79.6cm diameter concrete pipe containing dents and protrusions, the measured values ​​of the real three-dimensional model were compared with the actual values: the average relative error was 2.3%, and the maximum error was 6.2%, which shows that the accuracy of the real three-dimensional model meets the requirements for defect quantification.

[0069] In step S103, the center point of the drainage pipe is calculated based on the real three-dimensional model, so as to calculate the center line of the drainage pipe based on the center point.

[0070] Among them, the center point and center line can be used to determine the spatial position and geometric direction of the pipeline, and describe the spatial curvature, bending angle and direction change of the pipeline. The center line can be further used as a reference benchmark for sparse or dense point clouds and mesh models to realize the calibration and measurement of the relative position of defects. At the same time, the center line can also be used to guide the movement path of automated inspection tools, improving the accuracy and efficiency of pipeline internal inspection and defect location.

[0071] It can be noted that the methods for calculating the center point may include, but are not limited to, using the geometric center, or using the least squares fitting method to determine the center of the cross section; the methods for calculating the center line may include, but are not limited to, cubic spline curves, B-splines, least squares curve fitting, etc.; the calculation method can be set by those skilled in the art according to the actual situation, and no specific restrictions are imposed here.

[0072] Optionally, in one embodiment of the present invention, calculating the center point of the drainage pipe to calculate the centerline of the drainage pipe based on the center point includes: calculating the center point based on a preset generalized rotational symmetry axis; performing natural spline interpolation on the center point multiple times to obtain the results of multiple interpolations; and fitting the results of multiple interpolations to obtain the centerline.

[0073] Combination Figure 9 , Figure 10 , Figure 11 , Figure 12 The following is a detailed explanation of how to calculate the centerline of a drainage pipe using a specific example, which may include the following steps:

[0074] (1) ROSA calculation center point:

[0075] like Figure 9 As described above, in this embodiment of the invention, a section can be taken every 0.5m along the Z-axis of the pipeline, and the point cloud (including coordinates and normal vectors) of the section can be projected onto the XY plane (the pipeline section). Using the centroid of the point cloud as the initial position, the ROSA center point is obtained by minimizing the sum of the distances from the center point to the extensions of the normal vectors of each point through gradient descent. This can solve the problem of centroid shift caused by uneven or missing point cloud distribution. Simultaneously, as... Figure 11 , Figure 12 As shown, ROSA points can still maintain robustness even when the point cloud is missing.

[0076] (2) Fitting the center line of a cubic natural spline:

[0077] Furthermore, such as Figure 10 As shown, embodiments of the present invention can perform cubic natural spline interpolation on the ROSA center point to construct a continuously differentiable centerline parameter equation. x(z)=a 1 z 3 +b 1 z 2 +c 1 z+d 1 y(z)=a 2 z 3 +b 2 z 2 +c 2 z+d2; where the fitting should satisfy 4n-4 linear equations of “point constraint, first derivative constraint, second derivative constraint, and natural constraint” to ensure that the centerline is smooth and conforms to the actual pipeline direction.

[0078] In step S104, based on the centerline, at least one actual defect among misalignment defect, disconnection defect, undulation defect, and deformation defect of the drainage pipe is identified, so as to output the actual defect information of the drainage pipe according to at least one actual defect.

[0079] It can be noted that the identification method may include, but is not limited to, the relative offset of the center line, analysis of texture maps, or the distribution of axial curvature of the pipe. These methods can be set by those skilled in the art according to the actual situation, and no specific restrictions are imposed here.

[0080] Among them, misalignment defects can refer to axial or radial offset between adjacent pipe sections, causing misalignment of pipe joints; disconnection defects can refer to loose connections at pipe joints, with gaps or cracks; undulation defects can refer to local bulges or depressions along the axial direction of the pipe, resulting in uneven inner wall surfaces; deformation defects can refer to the overall or partial distortion, ellipticization, or flattening of the pipe's shape; actual defects may lead to decreased pipe sealing performance, leakage, uneven water flow, sediment accumulation, and abnormal structural stress, thereby affecting the pipe's drainage capacity, operating efficiency, and structural safety, increasing maintenance costs, and potentially causing serious accidents such as urban flooding or pipe rupture.

[0081] In embodiments of the present invention, actual defect information may include, but is not limited to, parameters such as defect level, defect quantity, defect size and spatial distribution, to achieve a quantitative description of defects such as cracks, disconnections, misalignments, undulations and deformations, thereby providing a reliable basis for pipeline structure health assessment, maintenance decisions and risk warning.

[0082] Optionally, in one embodiment of the present invention, identifying at least one actual defect among misalignment, disconnection, undulation, and deformation defects of the drainage pipe based on the centerline includes: calculating the dot product of the normal vector of the drainage pipe surface and the gradient of the centerline; determining the surface whose absolute value of the dot product is greater than a preset threshold; and determining at least one actual defect based on the surface.

[0083] The preset threshold can be 0, which means that the two vectors have a certain similarity or consistency in the spatial direction; or it can be other values, which can be set by those skilled in the art according to the actual situation, and no specific restrictions are made here.

[0084] Specifically, embodiments of the present invention may include the following steps:

[0085] (1) Defect detection:

[0086] like Figure 13 As shown, embodiments of the present invention can calculate the dot product of the pipe surface normal vector and the centerline gradient, and filter out surfaces with an absolute dot product value greater than a threshold (corresponding to an angle deviation of 90° > 10°); for example, the normal vector of a normal pipe wall surface is perpendicular to the centerline gradient (dot product ≈ 0), while the normal vector of a defective surface deviates (dot product ≠ 0).

[0087] (2) Defect identification:

[0088] like Figure 14 As shown, embodiments of the present invention can use the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm to cluster candidate patches (e.g., eps =0.5m, min_samples =50), if the projection distance between the two clustered point clouds on the XY plane is greater than 1 / 5 of the pipe diameter, it can be determined as misalignment; if the projection distance is approximately 0 and distributed along the Z-axis, it can be determined as disconnection; misaligned point clouds are symmetrically distributed, while disconnected point clouds are parallelly distributed.

[0089] Optionally, in one embodiment of the present invention, outputting actual defect information of the drainage pipe based on at least one actual defect includes: calculating the projected area of ​​any two types of point clouds on the plane perpendicular to the centerline, determining the single-sided area based on the projected area, and classifying the misalignment defect level based on the single-sided area as one of the actual defect information; obtaining the maximum distance between any two types of point clouds, classifying the disjoint defect level based on the maximum distance as one of the actual defect information; projecting the centerline onto a plane parallel to the centerline, and using the starting-end line of the drainage pipe as a reference line, calculating the maximum height difference between the centerline and the reference line, classifying the undulation defect level based on the maximum height difference as one of the actual defect information; and calculating the theoretical radius and minimum radius of each segment based on the segments of the drainage pipe, calculating the deformation rate of each segment based on the theoretical radius and minimum radius, and classifying the deformation defect level based on the deformation rate as one of the actual defect information.

[0090] The following examples illustrate the actual defect information of the drainage pipes output by embodiments of the present invention in detail.

[0091] (1) Misalignment defect:

[0092] like Figure 15 As shown, this embodiment of the invention can calculate the projected area of ​​two types of point clouds on the vertical centerline plane, and take the larger value as the single-sided area, which can be classified according to Table 1; wherein, Table 1 is a table of common reinforced concrete drainage pipe specifications and corresponding misaligned single-sided areas.

[0093] Table 1

[0094]

[0095] (2) Disconnection defect:

[0096] like Figure 16 As shown, the embodiments of the present invention can measure the maximum distance of two types of point clouds along the Z-axis, and can classify the levels according to Table 2; wherein, Table 2 is the definition table of disjoint distance for the disjoint defect quantification algorithm.

[0097] Table 2

[0098]

[0099] (3) Undulation defects:

[0100] like Figure 17 , Figure 18 , Figure 19 As shown, in this embodiment of the invention, the centerline can be projected onto the ZY plane, and the line connecting the beginning and end of the pipe can be used as a reference line. The maximum height difference h between the centerline and the reference line can be calculated, and the level can be divided according to h / pipe diameter. The level can be marked with different colors, as shown in Table 3. Table 3 is a table of names, codes, level classifications and scores for undulation defects.

[0101] Table 3

[0102]

[0103] (4) Deformation defects:

[0104] like Figure 20 As shown, in this embodiment of the invention, the pipeline can be divided into 200 segments along the Z-axis. For each segment, the theoretical radius R (the average distance from the point cloud within the segment to the ROSA center point) and the minimum radius minR (the average of the first 10% of the minimum distances) are calculated, and then the deformation rate is calculated. The deformation defects are classified into different levels according to their deformation rate, as shown in Table 4. Table 4 lists the names, codes, levels, scores, and sample charts of the deformation defects.

[0105] Table 4

[0106]

[0107] In actual implementation, this embodiment of the invention was tested on an 84m long, 500mm diameter rainwater pipe containing 24 structural defects. The vision-based three-dimensional quantification method for drainage pipe defects in this embodiment of the invention achieved a defect recall rate of 95.8% (23 / 24 defects were identified), an accuracy rate of 80.8% (21 / 26 identification results were consistent with the actual results), a single-sided area error of 3.8% for misalignment, and a dislocation distance error of 1.5mm, which can meet the requirements of engineering applications.

[0108] Furthermore, the present invention compares the advantages of the present invention with laser scanning-based 3D reconstruction technology, as shown in Table 5, where Table 5 is a comparison table between laser scanning-based 3D reconstruction technology and the present invention.

[0109] Table 5

[0110]

[0111] As a specific example, this embodiment of the invention compares the real three-dimensional model established by the vision-based three-dimensional quantification method for drainage pipe defects with its actual value, and the comparison results are shown in Table 6; wherein, Table 6 is a comparison table of the real three-dimensional model established by the vision-based three-dimensional quantification method for drainage pipe defects with the actual value.

[0112] Table 6

[0113]

[0114] Furthermore, the defect level classification in this embodiment of the invention conforms to the "Technical Specification for Inspection and Evaluation of Urban Drainage Pipelines" (CJJ181-2012):

[0115] (1) The misalignment level corresponds to "deviation ≤ 1 / 2 of wall thickness (mild) - deviation > 2 times wall thickness (severe)";

[0116] (2) The dislocation level corresponds to "dislocation ≤ 20mm (moderate) - dislocation > 50mm (severe)";

[0117] (3) The classification of undulation and deformation levels is consistent with the description in the regulations, and there is no need to modify the existing engineering evaluation system, which can facilitate industry promotion.

[0118] The following is a specific example, such as Figure 21 The flowchart illustrates the vision-based three-dimensional quantification method for drainage pipe defects according to an embodiment of the present invention, which may include the following steps:

[0119] (1) Three-dimensional reconstruction of drainage pipes based on SfM (Structure from Motion) and MVS (Multi-View Stereo).

[0120] In step S2101, an image of the drainage pipe is acquired:

[0121] In this embodiment of the invention, a GoPro 11 with a fisheye lens can be attached to a pipeline robot to capture pipeline video at a constant speed of 0.1 m / s, and FFmpeg can be used to extract frames at 1 s / frame to add a mask to the water surface image in order to obtain the required drainage pipeline image.

[0122] In step S2102, COLMAP is used to recover the camera pose and sparse point cloud.

[0123] In step S2103, OpenMVS is used to reconstruct the dense point cloud, reconstruct the mesh, and the texture map.

[0124] In step S2104, the three-dimensional model of the drainage pipe is:

[0125] In this embodiment of the invention, MeshLab can be used to calibrate the model based on the actual pipe diameter and evaluate the quality of the pipe model to obtain the final three-dimensional model of the drainage pipe.

[0126] (2) Structural defect identification and quantification algorithm based on pipeline three-dimensional model.

[0127] In step S2105, the center point of the pipe model is calculated:

[0128] In this embodiment of the invention, a section can be taken every 0.5m along the Z-axis of the pipeline, and the point cloud of the section can be projected onto the XY plane. Furthermore, in this embodiment of the invention, the center point of the section can be calculated using the ROSA algorithm.

[0129] In step S2106, the centerline of the pipe model is calculated:

[0130] The embodiments of the present invention can perform cubic natural spline interpolation on the center point to obtain a continuously differentiable centerline parametric equation, thereby obtaining the centerline of the pipeline model.

[0131] In step S2107, defect identification and quantification:

[0132] 1) Algorithm for identifying and quantifying misaligned and disconnected parts:

[0133] First, the embodiments of the present invention can calculate the dot product of the surface normal vector and the centerline gradient to filter out defective surfaces with a deviation >10°; second, the DBSCAN algorithm can be used to cluster defective surfaces; next, the embodiments of the present invention can distinguish between misalignment (>1 / 5 of the pipe diameter) and disjointness (≈0) according to the XY plane projection distance; finally, the embodiments of the present invention can calculate the single-sided area of ​​the misalignment or the disjointness distance and classify the level according to Tables 1 and 2 above.

[0134] 2) Algorithm for identifying and quantifying fluctuations:

[0135] In this embodiment of the invention, the centerline can be projected onto the ZY plane, the height difference h between the centerline and the reference line can be calculated, and further, the levels can be classified according to Table 3;

[0136] 3) Deformation recognition and quantization algorithm:

[0137] As one possible implementation method, embodiments of the present invention can divide the pipeline into 20 segments, calculate the theoretical radius R and the minimum radius minR of each segment, and then classify them according to Table 4.

[0138] (3) Drainage pipe structural defect identification and quantification system

[0139] In step S2108, the drainage pipe model is read.

[0140] The embodiments of the present invention can read the three-dimensional model data of drainage pipes obtained from the three-dimensional reconstruction of drainage pipes based on SfM and MVS.

[0141] In step S2109, the center line of the pipe model.

[0142] Specifically, embodiments of the present invention can calculate the center point of the pipeline based on the generalized rotational symmetry axis algorithm of the three-dimensional model, and then use the natural spline interpolation algorithm to generate a continuous and smooth pipeline centerline to accurately describe the spatial geometric orientation of the pipeline.

[0143] In step S2110, the defect type, quantification value, defect level, and location are obtained.

[0144] As a specific example, embodiments of the present invention can identify misalignment defects, disconnection defects, undulation defects, and deformation defects in a pipeline based on the dot product relationship between the pipeline surface normal vector and the centerline gradient; and calculate the quantitative value of the defect based on parameters such as single-sided area, maximum distance, and maximum height difference to determine the corresponding defect level and spatial location.

[0145] In step S2111, the rendering interface of the pipeline model and defects, as well as the defect level, are output.

[0146] The embodiments of the present invention can visualize and overlay the identified defect information with the three-dimensional model of the drainage pipeline, and output a rendering interface that includes defect category, quantification value, level and location label, so as to realize the intuitive display and evaluation results of pipeline structural defects.

[0147] The vision-based 3D quantification method for drainage pipeline defects proposed in this invention performs sparse point cloud reconstruction, dense point cloud reconstruction, mesh reconstruction, texture mapping, and scale calibration on the acquired visual images to construct a realistic 3D model of the drainage pipeline. Based on this model, the centerline of the drainage pipeline is calculated to identify defects such as misalignment, disconnection, undulation, and deformation, and the actual defect information of the drainage pipeline is output. This method enables accurate description of the overall spatial morphology of the drainage pipeline, allowing for quantitative analysis of defects and providing a clear view of the internal structure of the pipeline. It accurately captures defect details and reduces the rate of missed detections. Simultaneously, it reduces reliance on manual inspections, saving inspection time and maintenance costs, and improving the efficiency and scientific basis of pipeline structural health assessments. Furthermore, it provides a reliable data foundation for pipeline maintenance planning, defect evolution monitoring, and subsequent pipeline modifications, possessing scalability and repeatability, and is suitable for large-scale pipeline inspection and long-term operation management.

[0148] Next, with reference to the accompanying drawings, a vision-based three-dimensional quantification device for drainage pipe defects according to an embodiment of the present invention is described.

[0149] Figure 22 This is a block diagram of a vision-based three-dimensional quantification device for drainage pipe defects according to an embodiment of the present invention.

[0150] like Figure 22 As shown, the vision-based three-dimensional quantification device 10 for drainage pipe defects includes: a construction module 100, a calibration module 200, a calculation module 300, and a quantification module 400.

[0151] The construction module 100 is used to acquire visual images of the drainage pipe and perform at least one of the following processing on the visual images: sparse point cloud reconstruction, dense point cloud reconstruction, mesh reconstruction, and texture mapping, in order to construct an initial three-dimensional model of the drainage pipe.

[0152] The calibration module 200 is used to calibrate the scale of the initial three-dimensional model to obtain a true three-dimensional model of the drainage pipe.

[0153] The calculation module 300 is used to calculate the center point of the drainage pipe based on the real 3D model, so as to calculate the center line of the drainage pipe based on the center point.

[0154] The quantization module 400 is used to identify at least one actual defect among misalignment, disjointness, undulation, and deformation defects in drainage pipes based on the centerline, so as to output actual defect information of drainage pipes according to at least one actual defect.

[0155] Optionally, in one embodiment of the present invention, the quantization module 400 includes: a first division unit, a second division unit, a third division unit, and a fourth division unit.

[0156] The first division unit is used to calculate the projected area of ​​any two types of point clouds on the vertical centerline plane, so as to determine the single-sided area based on the projected area, and classify the misalignment defect level based on the single-sided area, which is one of the actual defect information.

[0157] The second segmentation unit is used to obtain the maximum distance between any two types of point clouds, so as to classify the level of the disconnection defect based on the maximum distance, and use it as one of the actual defect information.

[0158] The third division unit is used to project the centerline onto a plane parallel to the centerline, and to calculate the maximum height difference between the centerline and the reference line using the line connecting the beginning and end of the drainage pipe as a reference line. The maximum height difference is used to classify the level of undulation defects as one of the actual defect information.

[0159] The fourth segmentation unit is used to calculate the theoretical radius and minimum radius of each segment based on the segmentation of the drainage pipe, and to calculate the deformation rate of each segment based on the theoretical radius and minimum radius, so as to classify the level of deformation defects according to the deformation rate, as one of the actual defect information.

[0160] Optionally, in one embodiment of the present invention, the construction module 100 includes a processing unit and a construction unit.

[0161] The processing unit is used to perform camera pose recovery and sparse point cloud reconstruction on the visual image using a preset general camera and motion recovery structure system to obtain the processed sparse point cloud image.

[0162] The building unit is used to perform dense point cloud reconstruction, mesh reconstruction, and texture mapping on sparse point cloud images using a pre-defined open multi-view stereo vision library in order to establish an initial 3D model.

[0163] Optionally, in one embodiment of the present invention, the calculation module 300 includes: a first calculation unit, an interpolation unit, and a fitting unit.

[0164] The first calculation unit is used to calculate the center point based on a preset generalized rotational symmetry axis.

[0165] The interpolation unit is used to perform natural spline interpolation on the center point multiple times to obtain the results of multiple interpolations.

[0166] The fitting unit is used to fit the results of multiple interpolations to obtain the centerline.

[0167] Optionally, in one embodiment of the present invention, the quantization module 400 includes: a second calculation unit, a first determination unit, and a second determination unit.

[0168] The second calculation unit is used to calculate the dot product of the surface normal vector of the drainage pipe and the gradient of the centerline.

[0169] The first determining unit is used to determine the patches whose absolute value of the dot product is greater than a preset threshold.

[0170] The second determining unit is used to determine at least one actual defect based on the facet.

[0171] It should be noted that the foregoing explanation of the embodiment of the vision-based three-dimensional quantification method for drainage pipe defects also applies to the vision-based three-dimensional quantification device for drainage pipe defects in this embodiment, and will not be repeated here.

[0172] The vision-based 3D quantification device for drainage pipeline defects proposed in this invention performs sparse point cloud reconstruction, dense point cloud reconstruction, mesh reconstruction, texture mapping, and scale calibration on the acquired visual images to construct a realistic 3D model of the drainage pipeline. Based on this model, the centerline of the drainage pipeline is calculated to identify defects such as misalignment, disconnection, undulation, and deformation, and the actual defect information of the drainage pipeline is output. This allows for a precise description of the overall spatial morphology of the drainage pipeline, enabling quantitative analysis of defects and providing a clear view of the internal structure of the pipeline. It accurately captures defect details and reduces the rate of missed detections. Simultaneously, it reduces reliance on manual inspections, saving inspection time and maintenance costs, and improving the efficiency and scientific basis of pipeline structural health assessments. Furthermore, it provides a reliable data foundation for pipeline maintenance planning, defect evolution monitoring, and subsequent pipeline modifications. It is scalable and repeatable, suitable for large-scale pipeline inspection and long-term operation management.

[0173] Figure 23 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. The electronic device may include:

[0174] The memory 2301, the processor 2302, and the computer program stored on the memory 2301 and capable of running on the processor 2302.

[0175] When the processor 2302 executes the program, it implements the vision-based three-dimensional quantification method for drainage pipe defects provided in the above embodiments.

[0176] Furthermore, electronic devices also include:

[0177] Communication interface 2303 is used for communication between memory 2301 and processor 2302.

[0178] The memory 2301 is used to store computer programs that can run on the processor 2302.

[0179] The memory 2301 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0180] If the memory 2301, processor 2302, and communication interface 2303 are implemented independently, then the communication interface 2303, memory 2301, and processor 2302 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized into address buses, data buses, control buses, etc. For ease of representation, Figure 23 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0181] Optionally, in a specific implementation, if the memory 2301, processor 2302, and communication interface 2303 are integrated on a single chip, then the memory 2301, processor 2302, and communication interface 2303 can communicate with each other through an internal interface.

[0182] Processor 2302 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0183] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described vision-based three-dimensional quantification method for drainage pipe defects.

[0184] This invention also provides a computer program product, including a computer program that can run computer instructions. When the computer instructions are executed by a processor, they implement the vision-based three-dimensional quantification method for drainage pipe defects provided in this invention.

[0185] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0186] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0187] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0188] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0189] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or more of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0190] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0191] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0192] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A vision-based three-dimensional quantification method for defects in drainage pipes, characterized in that, Includes the following steps: Visual images of the drainage pipe are acquired, and sparse point cloud reconstruction, dense point cloud reconstruction, mesh reconstruction, and texture mapping are performed on the visual images to construct an initial three-dimensional model of the drainage pipe. The initial three-dimensional model is scaled to obtain a true three-dimensional model of the drainage pipe; Based on the real 3D model, the center point of the drainage pipe is calculated, and the centerline of the drainage pipe is calculated based on the center point; Based on the centerline, identify the actual defects among the misalignment, disjointness, undulation, and deformation defects of the drainage pipe, and output the actual defect information of the drainage pipe according to the actual defects. The step of outputting the actual defect information of the drainage pipe based on the actual defect includes: Calculate the projected area of ​​any two types of point clouds on the vertical centerline plane, determine the single-sided area based on the projected area, and classify the misalignment defect level based on the single-sided area as one of the actual defect information; Obtain the maximum distance between any two types of point clouds, and classify the level of the disconnection defect according to the maximum distance, as one of the actual defect information; Project the centerline onto a plane parallel to the centerline, and use the line connecting the beginning and end of the drainage pipe as a reference line to calculate the maximum height difference between the centerline and the reference line. Based on the maximum height difference, classify the level of the undulation defect as one of the actual defect information. Based on the segmentation of the drainage pipe, the theoretical radius and minimum radius of each segment are calculated, and the deformation rate of each segment is calculated according to the theoretical radius and the minimum radius. The deformation defect is classified into levels according to the deformation rate, which is one of the actual defect information. The calculation of the center point of the drainage pipe, and the calculation of the centerline of the drainage pipe based on the center point, includes: The center point is calculated based on a preset generalized rotational symmetry axis; The center point is subjected to natural spline interpolation multiple times to obtain multiple interpolation results. The results of the multiple interpolation are then fitted to obtain the centerline. Specifically, the center point is subjected to cubic natural spline interpolation to obtain a continuously differentiable centerline parametric equation, and then the centerline is obtained. The process of identifying actual defects among the misalignment, disconnection, undulation, and deformation defects of the drainage pipe based on the centerline includes: Calculate the dot product of the surface normal vector of the drainage pipe and the gradient of the centerline; Identify the patches whose absolute value of the dot product is greater than a preset threshold; The actual defect is determined based on the surface patch.

2. The method according to claim 1, characterized in that, The process of performing sparse point cloud reconstruction, dense point cloud reconstruction, mesh reconstruction, and texture mapping on the visual image to construct an initial 3D model of the drainage pipe includes: The visual image is processed by camera pose restoration and sparse point cloud reconstruction using a preset COLMAP to obtain a processed sparse point cloud image. The sparse point cloud image is reconstructed using a pre-defined open multi-view stereo vision library, and then subjected to dense point cloud reconstruction, mesh reconstruction, and texture mapping to establish the initial 3D model.

3. A vision-based three-dimensional quantification device for defects in drainage pipes, characterized in that, include: The construction module is used to acquire visual images of the drainage pipe and perform sparse point cloud reconstruction, dense point cloud reconstruction, mesh reconstruction, and texture mapping on the visual images to construct the initial three-dimensional model of the drainage pipe. The calibration module is used to perform scale calibration on the initial three-dimensional model to obtain the true three-dimensional model of the drainage pipe; The calculation module is used to calculate the center point of the drainage pipe based on the real three-dimensional model, so as to calculate the centerline of the drainage pipe based on the center point; The quantization module is used to identify actual defects among the misalignment, disjointness, undulation, and deformation defects of the drainage pipe based on the centerline, and output the actual defect information of the drainage pipe according to the actual defects. The quantization module includes: The first division unit is used to calculate the projected area of ​​any two types of point clouds on the vertical centerline plane, so as to determine the single-sided area based on the projected area, and to classify the misalignment defect level based on the single-sided area as one of the actual defect information. The second division unit is used to obtain the maximum distance between any two types of point clouds, so as to classify the level of the disconnection defect according to the maximum distance, and use it as one of the actual defect information; The third division unit is used to project the centerline onto a plane parallel to the centerline, and use the line connecting the beginning and end of the drainage pipe as a reference line to calculate the maximum height difference between the centerline and the reference line, so as to classify the level of the undulation defect according to the maximum height difference, as one of the actual defect information; The fourth segmentation unit is used to calculate the theoretical radius and minimum radius of each segment based on the segmentation of the drainage pipe, and to calculate the deformation rate of each segment according to the theoretical radius and the minimum radius, so as to classify the deformation defect level according to the deformation rate as one of the actual defect information; The calculation of the center point of the drainage pipe, and the calculation of the centerline of the drainage pipe based on the center point, includes: The center point is calculated based on a preset generalized rotational symmetry axis; The center point is subjected to natural spline interpolation multiple times to obtain multiple interpolation results. The results of the multiple interpolation are then fitted to obtain the centerline. Specifically, the center point is subjected to cubic natural spline interpolation to obtain a continuously differentiable centerline parametric equation, and then the centerline is obtained. The process of identifying actual defects among the misalignment, disconnection, undulation, and deformation defects of the drainage pipe based on the centerline includes: Calculate the dot product of the surface normal vector of the drainage pipe and the gradient of the centerline; Identify the patches whose absolute value of the dot product is greater than a preset threshold; The actual defect is determined based on the surface patch.

4. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, the processor executing the program to implement the vision-based three-dimensional quantification method for drainage pipe defects as described in any one of claims 1-2.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the vision-based three-dimensional quantification method for drainage pipe defects as described in any one of claims 1-2.

6. A computer program product, comprising a computer program, characterized in that, The computer program is executed to implement the vision-based three-dimensional quantification method for drainage pipe defects as described in any one of claims 1-2.

Images