A data enhancement method for semantic segmentation of urban sewer defect point cloud
By simulating urban drainage pipe defect scenarios in Unity3D and generating synthetic point cloud data, the problem of data scarcity in deep learning of 3D point clouds of urban drainage pipes was solved, and efficient model training and detection were achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- DALIAN UNIV OF TECH
- Filing Date
- 2024-05-24
- Publication Date
- 2026-06-02
Smart Images
Figure CN118628732B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of urban lifeline engineering structure evaluation and automation, specifically involving a data augmentation method for semantic segmentation of defect point clouds in urban drainage pipelines. Background Technology
[0002] Urban drainage pipeline systems are among the most valuable infrastructure components ensuring urban resilience and sustainable development, often referred to as the "lifeline of the city." On the one hand, the length of urban drainage pipelines laid in my country is increasing rapidly year by year; on the other hand, they also face serious problems of aging and disrepair. Therefore, automated and intelligent routine inspections of urban drainage pipelines are crucial for reducing time and cost.
[0003] For the past thirty years, the most widely used technology for routine inspections of urban drainage pipelines has been vision-based closed-circuit television (CCTV) monitoring systems. In the last decade or so, thanks to the rapid development of deep learning in object detection and segmentation of two-dimensional images, the efficiency and accuracy of defect detection in CCTV monitoring systems have been greatly improved. However, due to the lack of depth information, limited camera field of view, and lens distortion effects, two-dimensional images are generally only used for defect classification tasks, rather than for three-dimensional quantitative evaluation. LiDAR technology, with its advantages of non-contact operation, no need for illumination, and rapid and dense emission of point clouds to obtain accurate three-dimensional geometric mappings of the measured object, has received significant attention in the pipeline field in recent years. Various pipeline service condition assessment algorithms based on three-dimensional point clouds have been proposed to solve different problems, but they all share a common premise: the need to first identify and segment defect point clouds from a cluttered point cloud lacking semantic information. The defect forms of concrete water supply and drainage pipelines are extremely complex, including structural defects such as deformation, spalling, corrosion, misalignment, disconnection, and rubber ring detachment, as well as functional defects such as siltation and obstructions. Due to the large scale of pipeline point clouds and the complexity of pipeline defects, manually annotating semantic information is very time-consuming, labor-intensive, and prone to errors.
[0004] Deep learning technology has become a benchmark for automating the solution of complex real-world problems and is transforming the construction engineering field. Besides high-performance neural network algorithms suitable for the target task, training a successful model largely depends on high-quality training datasets. However, for the underground pipeline industry, real-world data obtained from inspections is both scarce and expensive. Due to confidentiality agreements and asset privacy policies, underground pipeline managers often do not annotate large amounts of real-world data, let alone disclose it publicly. Therefore, data scarcity greatly limits the development of 3D point cloud deep learning in the underground pipeline field and may even hinder the industry from keeping up with the ongoing trend of automation. Current synthetic point cloud generation methods mainly focus on component segmentation in indoor buildings and bridge structures. However, compared to these conventional scenarios, the defect forms of concrete water supply and drainage pipes are more complex and more random, yet methods for generating synthetic point cloud data with automated annotation capabilities have not yet been developed. Furthermore, how to strategically and maximally utilize artificially synthesized data to train deep learning models is also a technical challenge. Summary of the Invention
[0005] To overcome the data scarcity problem faced by urban drainage pipelines in the field of 3D point cloud deep learning, this invention provides a data augmentation method for semantic segmentation of point clouds of urban drainage pipeline defects. It provides a large amount of low-cost, high-quality, fast-collection, and highly random artificial synthetic data for training deep learning models of urban drainage pipeline defects, and provides an efficient way to train deep learning models using artificial synthetic data.
[0006] The technical solution adopted by this invention to solve its technical problem is as follows:
[0007] A data augmentation method for semantic segmentation of point clouds of defects in urban drainage pipes, comprising the following steps:
[0008] The first step is to use BIM software to create a 3D digital model of the pipeline, output an OBJ format file, and import it into the Unity3D engine to create a prefab.
[0009] The second step is to establish structured domain randomization rules in the Unity3D virtual environment;
[0010] The third step is to set global parameters, establish simulation algorithms for defects such as pipe disconnection and misalignment, rubber ring detachment, concrete spalling, corrosion, siltation, and obstruction blockage, and randomly generate simulation scenarios of urban drainage pipe defects.
[0011] Step 4: Based on the Unity3D physics engine, establish a simulation algorithm for the scanning and detection process of the LiDAR robot inside the pipe, and set the parameters related to the sensor equipment;
[0012] Step 5: Use an automated semantic annotation-based synthetic point cloud generation algorithm to achieve automated collection of synthetic point cloud defect data; synthetic point cloud defect labels include "normal", "peeling", "obstacle", "corrosion", "misalignment", "accumulation", "disconnection" and "rubber ring defect";
[0013] Step 6: Use a pre-trained and fine-tuned deep learning data augmentation strategy. First, train the basic model with a large amount of synthetic data, and then optimize the model parameters by combining a small amount of actual test data from a specific project to obtain a semantic segmentation model for pipeline defects.
[0014] Furthermore, the first step specifically includes: establishing a three-dimensional digital geometric model of the pipeline in BIM software according to the pipeline design drawings used in the project. The created three-dimensional digital geometric model pre-divides the pipeline into three sections: spigot section, intermediate pipe section, and socket section. The total length of each pipe section is divided into effective length and interface length; the lengths of the socket section and spigot section are respectively within 10cm from the end of the effective length of the pipeline; then, the OBJ format file is exported from the BIM software and imported into Unity3D to create a prefabricated pipeline; the prefabricated pipeline is set with four control points, namely "first point", "socket end point", "intermediate point" and "spigot end point", thereby dividing the effective length of the pipeline into four equal parts.
[0015] Furthermore, the second step specifically involves: Structural domain randomization, from the global to the local level, includes three types of components: global parameters, rule lines, and objects placed along the rule lines. It establishes probabilistic relationships between different components of structural domain randomization. Global parameters are set in the simulated scene, controlling the rule lines aligned with the generated defect objects. These global parameters include the number of control points, the probability of each defect occurring, and the scanning direction. The number of control points is determined by the number of simulated pipes in the scene and the number of control points on each pipe section. The rule lines of structural domain randomization include the pipe top line, pipe center line, and pipe bottom line. Generating rule lines requires first installing pipe prefabs along the pipe axis direction, where the pipe prefab axis direction is pre-set to the z-axis direction in the Unity world coordinate system. The pipe center line is generated from control points, and then the pipe center line is offset vertically by one unit of pipe radius along the y-axis to obtain the other two rule lines, namely the pipe top line and the pipe bottom line. Two scanning directions are set: scanning along the positive z-axis and scanning along the negative z-axis. The random scanning direction is achieved by randomly flipping the z-coordinates of all control points.
[0016] Furthermore, the simulation algorithm for pipe disconnection and misalignment described in step three is established according to the following formula:
[0017]
[0018] In equation (1): P coord,dis These are the coordinates (x, y, z) of the pipeline control point after the disconnection defect occurred; Pcoord,x P coord,y P coord,z These are the initial pipe coordinates; and These are the positive zero random displacements along the x-axis and y-axis, respectively, in meters; L2 is the joint length of the pipe socket section. P represents a positive zero random displacement along the z-axis, in meters. coord,mis These are the coordinates (x, y, z) of the pipeline control point after the misalignment defect occurs; It is a random displacement along the z-axis that is greater than zero, and the unit is meters. These are random rotations along the x, y, and z axes, respectively.
[0019] The simulation algorithm for rubber ring detachment is established as follows: First, rubber rings of different shapes are modeled in BIM software, and then imported into Unity3D to create rubber ring prefabricated bodies; the rubber ring prefabricated bodies are placed on the misalignment or disengagement defect control points of the pipe top line in a way that randomly selects shapes and probabilities, while being randomly scaled U(-T) along the x and y axes. rubb ,T rubb ), randomly rotated along the xoy plane between -180° and 180°, and randomly scaled U(S) along the x, y, and z axes. rubb,min ,S rubb,max ); where U represents a uniform random distribution, T rubb S represents the distance from the edge to the center of the rubber ring preform. rubb,min and S rubb,max These represent the minimum and maximum scaling factors for the rubber ring size, respectively;
[0020] The simulation algorithm for concrete spalling is established as follows: Based on the surface bending characteristics of the pipeline, a digital model of the concrete spalling defect is constructed on the inner diameter of the pipeline BIM design model, and the origin of the spalling defect is set at the center of the cross-section; the spalling defect consists of two parts: the spalling surface and the spalling opening, where the spalling opening represents the laser entrance during the scanning process to achieve the occlusion relationship of the spalling; finally, the OBJ digital model of the concrete spalling defect is imported into Unity3D and a precast of the spalling defect is created; when generating the precast of the spalling defect, the depth and shape of the spalling are randomly selected, and the spalling is randomly rotated between -180° and 180° around the z-axis, and randomly translated U(-T) along the z-axis. spall ,T spall ), randomly scale U(S) along the z-axis spall,min ,S spall,max ), and place it on the centerline of the tube excluding the "first point" control point according to the probability parameters; where U represents a uniform random distribution, T spallSet to half the distance between the two control points, S spall,min and S spall,max These are the minimum and maximum scaling factors for the peeling size, respectively;
[0021] The corrosion simulation algorithm is established as follows: the modeling of corrosion defects is implemented in the Unity3D environment through mesh programming using vertices and triangles. The coordinates of the vertices of the corroded surface are generated by the following formula:
[0022]
[0023] The edge vertex of type I corrosion (θ=-θ) corros Or θ = θ corros The edge position of the corrosion will be simulated by setting it individually as (x,y) = polarToCart(θ,r), while Type II corrosion will simulate the case of complete corrosion of the entire cross section. Triangular faces will be used to connect the edge vertices to form a circular mesh.
[0024] In equation (2): polarToCart(*) represents the function to convert the polar coordinate system to the rectangular coordinate system; r is the design inner diameter of the pipe; U represents a random uniform distribution, reflecting the degree of corrosion; the random distribution of the corrosion level is set to U(0,αt), where t is the pipe wall thickness and α is the percentage of corrosion loss; θ corros Set to (θ) min ,θ max Generate random values within the range of 2θ. min up to 2θ max The random corrosion angle, θ min and θ max These represent the minimum and maximum corrosion ranges along the pipe cross-section, respectively; z scale The length of corrosion along the longitudinal direction of the pipe is represented by a random value U(S). corr,min ,S corr,max ), S corr,min and S corr,max These represent the minimum and maximum corrosion ranges along the longitudinal direction of the pipe, respectively; new Vector3(*) indicates the creation of a three-dimensional coordinate vector in Unity;
[0025] The simulation algorithm for siltation is established as follows: Perlin noise and fractal Brownian motion (fBm) noise are used to simulate the topographic undulations of siltation defects. Besides the variation in the y-coordinate, the range of the siltation defect along the x and z axes is defined as (-r, r), where r is the design radius of the pipe. The x, y, and z coordinates follow the world coordinate system of Unity3D. The variation in siltation depth is simulated by random translation along the y-axis. Where y h The y-axis fluctuation height of the fBm algorithm is between 0 and... Silt is generated within a random depth range;
[0026] The obstacle blockage simulation algorithm is established as follows: Brick models of various shapes and sizes are modeled in BIM software, then imported into Unity3D to create obstacle prefabs. Subsequently, Rigidbody components are added to the obstacle prefabs to simulate gravity and motion resistance based on the physics engine. Simultaneously, Collider components are added to the obstacle prefabs, pipe prefabs, and siltation mesh. The Random.onUnitSphere() function is used to generate obstacles from random directions according to probability parameters, simulating their fall to the bottom of the pipe by gravity.
[0027] Furthermore, the fourth step specifically includes: simulating the virtual robot's LiDAR scanning process using Ray rays and the `Physics.RaycastAll()` function in Unity3D; setting the point cloud resolution: the distance to a point at a position of 10m is x, and the angular step size of the laser ray in both the vertical and horizontal directions is θ. step =arctan(x / 10000)·180° / π; The scanning range of the virtual robot is set to the vertical angle (θ). longi_start ,θ longi_end The angle between the horizontal direction and the horizontal direction is (θ) circum_start ,θ circum_end The virtual robot lidar emits light along the vertical direction (θ). longi_end -θ longi_start ) / θ step Using ray tracing to simulate a vertical scan, with θ step The horizontal rotation is performed using an angular step size to simulate a horizontal scan. Therefore, the maximum number of point clouds that can be collected in each scan is [(θ longi_end -θ longi_start ) / θ step ]·[(θ circum_end -θ circum_start ) / θ step In the virtual robot's scanning simulation parameter settings, the sensor length is set to the farthest reaching distance of the scanning ray. After a single scan, the virtual robot will move to the next control point coordinate of the pipe centerline. When the virtual robot moves to the last control point coordinate in the scene and the point cloud acquisition task is not yet completed, a new concrete pipe defect scene will be automatically regenerated to continue the point cloud acquisition task.
[0028] Furthermore, various scanning errors are introduced into the LiDAR scanning process of the virtual robot to make it more closely resemble the real point cloud situation; the scanning errors include random errors, distance errors, and device errors, as shown in the following formula:
[0029]
[0030] In the formula: OP' coord These are the scan coordinates after applying the error; OP coord E represents the idealized scan coordinates. random It is a random error, including components e on the x, y, and z axes. x e y and e z And each component follows a normal distribution; μ is the mean of the normal distribution, σ is the standard deviation of the normal distribution, and N(μ,σ) represents a normal distribution with mean μ and standard deviation σ; all are determined by the technical parameters of the simulated equipment; E distance It is the distance error; σ perDistanceUnit It is the additional standard deviation per unit ray distance, determined according to the technical parameters; D hit V represents the distance from the scanning sensor position to the target point. ray This is the normalized vector of the scanning ray vector.
[0031] Furthermore, the specific process of the automated semantic annotation synthetic point cloud generation algorithm described in step five is as follows: ① Input the information of the laser ray simulated by the physics engine after colliding with the object: hits = {point i =(x i ,y i ,z i ), i = [0, N]}, where hits[0] = point0 is the information of the first object hit by the ray, hits[1] = point1 is the second object, and so on; point i It is the coordinate position in the world coordinate system; (x s ,y s ,z s ) is the position coordinate of the virtual robot in the world coordinate system; ② The output result is set as: the coordinates (x,y,z) of the point cloud collected by the laser ray in the scanner coordinate system and the tag, where tag=0 indicates normal, 1 indicates peeling, 2 indicates obstacle, 3 indicates corrosion, 4 indicates misalignment, 5 indicates siltation, 6 indicates disintegration, and 7 indicates rubber ring defect; ③ Obtain the type of the first object hit by the ray: hits[0].transform.tag; ④ The following are judged in order. If hits[0].transform.tag is an obstacle: (x,y,z)=(x s ,y s ,z s -point0+E random +e distance ;tag=2;⑤If hits[0].transform.tag is a rubber ring defect: (x,y,z)=(x s ,ys ,z s -point0+E random +E distance ;tag=7;⑥If hits0].transform.tag is siltation: (x,y,z)=(x s ,y s ,z s -point0+E random +E distance ;tag=5;⑦If hits[0].transform.tag is the spalling opening, the first case is that no corrosion occurs at the same time, N≥2 and hits[2].transform.tag is the spalling surface: (x,y,z)=(x s ,y s ,z s -point2+E random +E distance ;tag=1; The second case is when the corrosion depth is less than the spalling depth, N≥3 and hist[2].transform.tag is the corrosion surface and hits[3].transform.tag is the spalling surface: (x,y,z)=(x s ,y s ,z s -point3+E random +E distance ;tag=1;⑧If the pipe is disconnected and hits[0].transform.tag is a spigot or socket: (x,y,z)=(x s ,y s ,z s -point0+E random +E distance ;tag=6;⑨If the pipe is misaligned and hits[0].transform.tag is a spigot or socket: (x,y,z)=(x s ,y s ,z s -point0+E random +E distance ;tag=4;⑩If corrosion occurs in the pipeline, N≥1, hits[0].transform.tag is the pipe section, spigot or socket, and hits[1].transform.tag is the corrosion surface: (x,y,z)=(x s ,y s ,z s -point1+E random +E distance tag=3; Finally, if hits[0].transform.tag is a pipe segment, spigot, or socket: (x,y,z)=(x s ,y s ,z s -point0+E random +E distance tag=0; Obtain point cloud data (x, y, z, tag); all datasets (x, y, z, tag) will be further augmented with normal vector data (n) using the Python toolkit in Open3D. x ,n y ,n z ).
[0032] The beneficial effects of this invention include:
[0033] 1. This invention targets urban drainage pipes, providing a large amount of low-cost, high-quality, fast-collection, and highly random artificially synthesized data for the semantic segmentation of defects in this type of pipe, thus solving the problems of data scarcity and limited variety faced by deep learning in practical engineering applications.
[0034] 2. The artificial point cloud generated by this invention, in theory, is limited only by the computing power and data storage capacity of the computing server. Therefore, compared with collecting a large amount of real data on-site, which is costly and slow, it provides a low-cost and fast way to expand training data.
[0035] 3. This invention simulates a defect scenario of urban drainage pipes based on the Unity3D game engine and BIM design software. By integrating 3D modeling and graphics rendering technologies, it adopts advanced structural domain randomization simulation technology to construct a realistic digital defect scenario.
[0036] 4. This invention combines the emerging lidar in-pipe detection technology in underground pipeline network inspection. The proposed artificial synthetic data generation method integrates advanced lidar scanning simulation technology, realizes automated acquisition and semantic annotation of artificial synthetic point clouds, and takes into account various noise effects of lidar acquisition, laying the foundation for generating high-quality, realistic and semantically rich artificial synthetic point clouds.
[0037] 5. The artificial point cloud generation method proposed in this invention includes urban drainage pipe defect categories with complex geometric forms and highly random distribution: "stripping", "obstacles", "corrosion", "misalignment", "siltation", "disconnection" and "rubber ring defects".
[0038] 6. The method proposed in this invention first uses a large number of low-cost, fast-acquisition artificial point clouds to obtain a pre-trained basic model, and then uses a smaller number of real point clouds from engineering sites for fine-tuning. This method can better utilize artificial point clouds and provides an effective way for knowledge transfer between synthetic point clouds and real point clouds.
[0039] 7. This invention provides a new data augmentation approach for semantic segmentation of defects in urban drainage pipelines, enabling more efficient identification of complex defects and laying the foundation for quantitative evaluation of defects in urban drainage pipelines. Attached Figure Description
[0040] Figure 1 This is a schematic diagram of a pipe prefab in the Unity3D engine.
[0041] Figure 2 This is a schematic diagram illustrating the generation of control points and rule lines using the structured domain randomization technique.
[0042] Figure 3 This is a schematic diagram of generating concrete spalling defects in Unity3D.
[0043] Figure 4 This is a schematic diagram of generating concrete corrosion defects in Unity3D.
[0044] Figure 5 This is a diagram illustrating the generation of various defect scenarios in Unity3D.
[0045] Figure 6 This is a schematic diagram of the LiDAR scanning process for a virtual robot in Unity3D.
[0046] Figure 7 This is a schematic diagram of a scheme for using artificially synthesized data based on pre-trained data augmentation strategies.
[0047] In the diagram: 1. Spigot section, 2. Intermediate pipe section, 3. Socket section, 4. Effective length, 5. Interface length, 6. First point, 7. Socket end point, 8. Intermediate point, 9. Spigot end point, 10. Pipe top line, 11. Pipe center line, 12. Pipe bottom line, 13. Disconnection, 14. Misalignment, 15. Type I defect, 16. Type II defect, 17. Peeling surface, 18. Peeling hole, 19. Laser, 20. Type I corrosion, 21. Type II corrosion, 22. Pipe wall thickness, 23. Virtual robot, 24. Ray ray, 25. Vertical angle, 26. Horizontal angle, 27. Idealized scanning coordinate point, 28. Distance from scanning sensor position to target point. Detailed Implementation
[0048] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0049] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0050] The purpose of this invention is to provide a data augmentation method for semantic segmentation of point clouds representing defects in urban drainage pipelines. This method provides a large amount of low-cost, high-quality, fast-collection, and highly randomized artificially synthesized data for training deep learning models of urban drainage pipeline defects, and offers an efficient way to train deep learning models using artificially synthesized data. This method integrates Unity game engine technology, structured domain randomization technology, and laser scanning simulation technology, providing a new technical means to solve the data scarcity problem faced by urban pipe network companies in the field of 3D point cloud deep learning. The generated synthetic point cloud data with automated annotation has the advantages of fast acquisition speed, low cost, and high randomness, providing a big data foundation for the automation and intelligentization of drainage pipelines, and promoting the further development of 3D point cloud deep learning for pipelines. The specific measures are as follows: A 3D digital model of the pipeline is established using BIM software and output in OBJ format; structural domain randomization rules are established in the Unity3D virtual environment, and global parameters are set; simulation algorithms are established for pipeline discontinuity, misalignment, rubber ring detachment, concrete spalling, corrosion, siltation, and obstruction blockage defects, and simulated scenarios of urban drainage pipeline defects are randomly generated; a simulation algorithm for the scanning and detection process of a LiDAR robot inside the pipe is established based on the Unity3D physics engine, and parameters related to the sensor equipment are set; an automated semantic annotation synthetic point cloud generation algorithm is proposed to achieve automated acquisition of synthetic data; a pre-training-fine-tuning deep learning data augmentation strategy is used, first training the basic model on a large amount of synthetic data, and then optimizing the model parameters by combining a small amount of measured data from specific projects to obtain a semantic segmentation model for pipeline defects, providing strong support for the quantitative evaluation of urban drainage pipeline performance.
[0051] Example 1
[0052] A data augmentation method for semantic segmentation of point clouds of defects in urban drainage pipes, the specific implementation steps of which include:
[0053] The first step is to use Trimble SketchUp, a professional BIM design software, to create a 3D digital model of the pipeline, output an OBJ format file, and import it into the Unity3D engine to create prefabs, such as... Figure 1As shown. A 3D digital model of the pipeline is created in Trimble SketchUp software according to the pipeline design drawings used in the project, such as a concrete pipe with an inner diameter of 800mm designed according to specifications. The created 3D digital geometric model needs to pre-divide the pipeline into three sections: spigot section 1, intermediate pipe section 2, and socket section 3. According to the pipeline design specifications, the total length of each pipe section is divided into an effective length 4 and an interface length 5. The lengths of socket section 3 and spigot section 1 are respectively within 10cm of the end of the effective length 4 of the pipeline. Then, an OBJ format file is exported from the BIM software and imported into Unity3D to create a prefabricated pipeline. The prefabricated pipeline has four control points: "first point 6", "socket end point 7", "intermediate point 8", and "spigot end point 9", thus dividing the effective length of the pipeline into four equal parts.
[0054] The second step is to establish structured domain randomization rules in the Unity3D virtual environment, such as... Figure 2 As shown. Structural domain randomization comprises three components from global to local: global parameters, rule lines, and objects placed along the rule lines. Probabilistic relationships are established between the different components of structural domain randomization. Global parameters are set in the simulated scene, controlling the rule lines aligned with the generated defect objects. These global parameters include the number of control points, the probability of each defect occurring, and the scanning direction. The number of control points is determined by the number of pipes simulated in the scene and the number of control points on each pipe section. For example, if the scene simulates a pipe system consisting of 50 socket-type pipes, there will be 50 × 4 = 200 control points. The rule lines of structural domain randomization include the pipe top line (10), pipe center line (11), and pipe bottom line (12). Generating rule lines requires first installing pipe prefabs along the pipe axis direction, where the pipe prefab axis direction is pre-set to the z-axis direction in the Unity world coordinate system. The pipe center line is generated from the control points, and then the pipe center line is offset vertically by one unit pipe radius along the y-axis to obtain the other two rule lines: the pipe top line and the pipe bottom line. The scanning direction can be set to two types: scanning along the positive z-axis and scanning along the negative z-axis. Random scanning direction is achieved by randomly flipping the z-coordinates of all control points.
[0055] Step 3: Set global parameters and establish pipe disconnection and misalignment ( Figure 2 Rubber ring detachment, concrete spalling Figure 3 ),corrosion( Figure 4 Simulation algorithms for defects such as siltation and blockage by obstacles are used to randomly generate simulated scenarios of defects in urban drainage pipes. Figure 5 Set the probability of defect occurrence for global parameters, and install prefabricated sections according to the pipe axis direction.
[0056] The pipe disconnection 13 and misalignment 14 are generated according to the following formula during the construction of control points:
[0057]
[0058] In the formula: P coord,dis These are the coordinates (x, y, z) of the pipeline control point after the disconnection defect occurred; P coord,x P coord,y P coord,z These are the initial pipe coordinates; and These are the random displacements along the x-axis and y-axis, respectively, for example (-0.05 to 0.05), in meters; L2 is the interface length of the pipe socket end; P represents a positive zero random displacement along the z-axis, for example (0 to 0.1), in meters; coord,mis These are the coordinates (x, y, z) of the pipeline control point after the misalignment defect occurs; It is a random displacement along the z-axis, for example (0.05~0.1), in meters; These are random rotations along the x, y, and z axes, for example, (-10° to 10°).
[0059] Generation of Pipe Rubber Ring Defects: Rubber ring detachment defects are likely to occur at control points where misalignment or disengagement occurs. First, rubber rings of different shapes are modeled in BIM design software, and then imported into Unity3D to create rubber ring prefabricated bodies. These prefabricated bodies are placed at the misalignment or disengagement defect control points on the pipe top line using randomly selected shapes and probabilities, while being randomly scaled U(-T) along the x and y axes. rubb ,T rubb ), randomly rotated along the xoy plane between -180° and 180°, and randomly scaled U(S) along the x, y, and z axes. rubb,min ,S rubb,max Where U represents a uniform random distribution, T rubb S represents the distance from the edge to the center of the precast rubber ring, for example, 0.05m. rubb,min and S rubb,max These represent the minimum and maximum scaling factors of the rubber ring size, for example (0.9 to 1.1).
[0060] Pipe spalling defect generation: Digital mesh modeling of spalling defects of different depths and shapes is performed in BIM software. Based on engineering practice experience, 5mm, 8mm, 12mm, 18mm, 25mm, and 30mm can be selected as typical depths for modeling, but this can still be expanded based on prior knowledge of specific projects. Regarding defect shapes, the two most common spalling defect types in practice are considered, such as... Figure 3As shown. Type I defect 15 is a concave defect with steep edges, and Type II defect 16 is an arc-shaped defect with gentle edges. Based on the surface bending characteristics of the pipe, a digital model of the spalling defect is constructed on the inner diameter of the pipe BIM design model, and the coordinate origin of the spalling defect is set at the center of the cross-section. The spalling defect consists of two parts: the spalling surface 17 and the spalling opening 18. The opening represents the laser 19 entrance during the scanning process to achieve the occlusion relationship of the spalling. Finally, the OBJ model of the spalling component is imported into Unity3D and a spalling prefab is created. When generating the spalling defect, the depth and shape of the spalling are randomly selected, and it is randomly rotated around the z-axis between -180° and 180°, and randomly translated along the z-axis by U(-T). spall ,T spall ), randomly scale U(S) along the z-axis spall,min ,S spall,max ), and place it on the centerline of the pipe excluding the "first point" control point according to the probability parameters. T spall Set it to half the distance between the two control points, for example, 0.1m, S spall,min and S spall,max These are the minimum and maximum scaling factors for the peel size, for example (0.5 to 1.5).
[0061] Pipeline corrosion defect generation: Corrosion defects are implemented using C# scripting within the Unity3D environment. Modeling of corrosion defects is achieved through mesh programming using vertices and triangles. The coordinates of the vertices on the corroded surface are generated using the following formula:
[0062]
[0063] The edge vertex of type I corrosion 20 (θ=-θ) corros Or θ = θ corros The edge position of the corrosion will be simulated by setting it individually as (x,y) = polarToCart(θ,r), while Type II corrosion 21 will simulate the case of complete corrosion of the entire cross section, using triangular faces to connect the edge vertices to form a circular mesh.
[0064] In the above formula: polarToCart(*) represents the function to convert the polar coordinate system to the rectangular coordinate system; r is the design inner diameter of the pipe; U represents a random uniform distribution, reflecting the degree of corrosion. The random distribution of the corrosion level is set to U(0, αt), where t is the pipe wall thickness (22), and α is the percentage of corrosion loss. For example, α = 10% for a low corrosion level and α = 30% for a medium corrosion level; θ corros Set to (θ) min ,θ max Generate random values within the range of 2θ. min up to 2θ max The random corrosion angle, θ min and θ maxThese represent the minimum and maximum corrosion ranges along the pipe cross-section, such as 45° and 165°; scale The length of corrosion along the longitudinal direction of the pipe is represented by a random value U(S). corr,min ,S corr,max ), S corr,min and S corr,max These are the minimum and maximum corrosion ranges along the longitudinal direction of the pipeline, for example, 0.2m and 1m.
[0065] Siltation Defect Generation: Siltation defects are generated using scripts based on mesh programming. Perlin noise and fractal Brownian motion (fBm) noise are used to simulate the terrain undulations of the siltation defects. Except for variations in the y-coordinate, the range of the siltation defects along the x and z axes is defined as (-r, r), where r is the design radius of the pipe, and the x, y, and z coordinates follow the Unity3D world coordinate system. Variations in siltation depth are simulated by random translation along the y-axis. Where y h The y-axis fluctuation height of the fBm algorithm is between 0 and... Silt is generated within a random depth range.
[0066] Obstacle Defect Generation: Model bricks of various shapes and sizes in BIM software, then import them into Unity3D to create obstacle prefabs. Next, add Rigidbody components to the obstacle prefabs, which can simulate gravity and drag based on the physics engine. Simultaneously, add Collider components to the obstacle prefabs, pipe prefabs, and silt meshes. Using the Random.onUnitSphere() function, generate obstacles from random directions with probabilistic parameters and simulate their fall to the bottom of the pipe using gravity.
[0067] Step 4: Based on the Unity3D physics engine, establish a simulation algorithm for the LiDAR robot's scanning and detection process within the pipe, and set parameters related to the sensor equipment, such as... Figure 6 As shown. The laser radar scanning process of the virtual robot 23 is simulated using Ray rays 24 and the Physics.RaycastAll() function in Unity3D. The point cloud resolution is set as follows: the distance to a point at a position of 10m is x, for example, 49.1mm, and the angular step size of the laser ray in both the vertical and horizontal directions is θ. step =arctan(x / 10000), which is 0.28°. The scanning range of the virtual robot is set to a vertical angle of 25° (θ). longi_start ,θ longi_end For example, (-60°, 90°), the horizontal angle is 26° (θ). circum_start ,θ circum_endFor example, (-180°, 180°). The virtual robot lidar emits (θ) along the vertical direction. longi_end -θ longi_start ) / θ step That is, 536 Ray rays are used to simulate a vertical scan, with θ step The horizontal rotation is performed using an angular step size to simulate a horizontal scan. Therefore, the maximum number of point clouds that can be collected in each scan is [(θ longi_end -θ longi_start ) / θ step ]·[(θ circum_end -θ circum_start ) / θ step This translates to 689,296 data points. In the virtual robot's scanning simulation parameter settings, the sensor length is set to the furthest reach distance of the scanning ray. After a single scan, the virtual robot will move to the next control point coordinates on the pipe centerline. When the virtual robot moves to the last control point coordinates in the scene and the point cloud acquisition task is still not complete, a new concrete pipe defect scene will be automatically regenerated to continue the point cloud acquisition task.
[0068] Various errors are introduced during the lidar scanning process of the virtual robot 23 to make it more closely resemble the real point cloud. Scanning errors mainly include random errors, distance errors, and device errors, as shown in the following formula. Random errors are typically described as errors that follow a random normal distribution. Distance errors can be determined according to the manufacturer's specifications. Device errors can be eliminated through calibration before scanning.
[0069]
[0070] In the formula: OP' coord These are the scan coordinates after applying the error; OP coord For the idealized scan coordinate point 27; E random It is a random error, including components e on the x, y, and z axes. x e y and e z Furthermore, each component follows a normal distribution; μ is the mean of the normal distribution, and σ is the standard deviation of the normal distribution, both determined by the technical parameters of the equipment being simulated, such as the FAROFocus simulation. S For scanners, when the scanning distance is within 10m: μ = 0, σ = 2mm; when the scanning distance is greater than 10m, μ = 0, σ = 3.5mm; E distance It is the distance error; σ perDistanceUnit It is the additional standard deviation per unit ray distance, determined according to technical parameters, such as FARO Focus. S When the scanner's scanning distance exceeds 25m, σ perDistanceUnit =0.1mm / m; Dhit The distance from the scanning sensor position to the target point (28); V ray This is the normalized vector of the scanning ray vector;
[0071] Step 5: Generate synthetic point clouds with automated semantic annotation to achieve automated acquisition of synthetic data. During the point cloud acquisition process by the virtual robot, each target point hit by the ray needs to be classified into a semantic segmentation category. At the location of the pipe socket, the point cloud should be labeled as "normal", "misaligned", or "disconnected" depending on whether the pipe has misaligned or disjointed. In addition, the occlusion relationships of complex geometric defects such as corrosion and peeling are considered sequentially during the label generation process. The specific process of the developed automated point cloud annotation algorithm is as follows: ① Input the information after the laser ray simulated by the physics engine hits = {point i =(x i ,y i ,z i ), i=[0,N]}, where hits[0]=point0 is the information of the first object hit by the ray, hits[1]=point1 is the second object, and so on. i It represents the coordinate position in the world coordinate system. (x) s ,y s ,z s ) is the robot's position coordinates in the world coordinate system. ② The output result is set as: the coordinates (x,y,z) of the point cloud collected by the laser ray in the scanner coordinate system and the tag, where tag=0 indicates normal, 1 indicates peeling, 2 indicates obstacle, 3 indicates corrosion, 4 indicates misalignment, 5 indicates accumulation, 6 indicates disintegration, and 7 indicates rubber ring defect. ③ Obtain the type of the first object hit by the ray: hits[0].transform.tag. ④ The following are judged in order. If hits[0].transform.tag is an obstacle: (x,y,z)=(x s ,y s ,z s -point0+E random +E distance ;tag=2. ⑤If hits[0].transform.tag is a rubber ring defect: (x,y,z)=(x s ,y s ,z s -point0+E random +E distance ;tag = 7. ⑥If hits[0].transform.tag is a siltation: (x,y,z) = (x s ,y s ,z s -point0+Erandom +E distance ;tag=5. ⑦If hits[0].transform.tag is the spalling opening, the first case is that no corrosion occurs at the same time, N≥2 and hits[2].transform.tag is the spalling surface: (x,y,z)=(x s ,y s ,z s -point2+E random +E distance ;tag=1. The second case is when the corrosion depth is less than the spalling depth, N≥3 and hits[2].transform.tag is the corrosion surface and hits3].transform.tag is the spalling surface: (x,y,z)=(x s ,y s ,z s -point3+E random +E distance ;tag=1. ⑧If the pipe is disconnected and hits[0].transform.tag is a spigot or socket: (x,y,z)=(x s ,y s ,z s -point0+E random +E distance ;tag=6. ⑨ If the pipe is misaligned and hits[0].transform.tag is a spigot or socket: (x,y,z)=(x s ,y s ,z s -point0+E random +E distance ;tag=4. ⑩ If corrosion occurs in the pipeline, N≥1, hits[0].transform.tag is the pipe section, spigot or socket, and hits[1].transform.tag is the corrosion surface: (x,y,z)=(x s ,y s ,z s -point1+E random +E distance ;tag=3. Finally, if hits[0].transform.tag is a pipe segment, spigot, or socket: (x,y,z)=(x s ,y s ,z s -point0+E random +E distance ;tag=0. Obtain point cloud data (x, y, z, tag). All datasets (x, y, z, tag) will then have normal vector data (n) further added using Python tools in Open3D. x ,n y ,n z );
[0072] Step 6: Using a pre-trained and fine-tuned deep learning data augmentation strategy, first, a base model is trained on a large amount of synthetic data, and then the model parameters are optimized by combining a small amount of measured data from specific projects to obtain a semantic segmentation model for pipeline defects. This provides strong support for the quantitative evaluation of urban drainage pipeline performance. Figure 7 As shown, after the training loss and training accuracy of a large amount of synthetic data stabilize, a basic model is obtained. Then, in the fine-tuning stage, real data collected in the field are added while keeping the learning rate constant to obtain the final deep learning model.
[0073] In summary, the proposed method provides a pathway for faster and more cost-effective deep learning semantic segmentation in urban drainage pipeline defect assessment. The developed automated synthetic data, based on structured domain randomization and laser scanning simulation techniques, boasts advantages such as high quality, realism, and rich semantics, alleviating the data scarcity problem in engineering. Furthermore, the proposed pre-training data augmentation strategy effectively addresses the knowledge transfer problem between synthetic and real point clouds.
[0074] This invention targets urban drainage pipelines, providing a large amount of low-cost, high-quality, fast-collection, and highly randomized artificially synthesized data for the semantic segmentation of defects in this type of pipeline. This solves the problems of data scarcity and limited data types faced by deep learning in practical engineering applications. Theoretically, the amount of artificially synthesized point cloud generated by this invention is limited only by the computing power and data storage capacity of the computing server. Therefore, compared to collecting large amounts of costly and slow real data on-site, it provides a low-cost and rapid way to expand training data. Based on the Unity3D game engine and BIM design software, this invention simulates defect scenarios in urban drainage pipelines. By integrating 3D modeling and graphics rendering technologies, it employs advanced structured domain randomization simulation technology to construct realistic digital defect scenarios. This invention combines the emerging LiDAR pipe-in-pipe detection technology in underground pipe network inspection. The proposed artificially synthesized data generation method integrates advanced LiDAR scanning simulation technology, realizing automated collection and semantic annotation of artificially synthesized point clouds. It also considers various noise effects from LiDAR collection, laying the foundation for generating high-quality, realistic, and semantically rich artificially synthesized point clouds. The artificial point cloud generation method proposed in this invention considers the following defect categories for urban drainage pipelines: "normal," "stripping," "obstacles," "corrosion," "misalignment," "siltation," "disconnection," and "rubber ring defects." This invention first uses a large amount of low-cost, high-speed artificial point cloud data to obtain a pre-trained base model, and then uses a smaller amount of real-world engineering point cloud data for fine-tuning. This method better utilizes the artificial point cloud and provides an effective way to transfer knowledge between the synthetic and real point clouds.
[0075] In summary, this invention integrates Unity game engine technology, structured domain randomization technology, and laser scanning simulation technology to provide a method for generating artificially synthesized data. This artificially synthesized data is characterized by low cost, high quality, fast acquisition speed, large data volume, and strong randomness. It provides a way to achieve faster and more economical deep learning semantic segmentation tasks in urban drainage pipeline defect assessment, and identifies complex defects in a more efficient way, laying the foundation for the quantitative evaluation of defects in urban drainage pipelines.
[0076] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.
Claims
1. A data augmentation method for semantic segmentation of point clouds representing defects in urban drainage pipes, characterized in that, Includes the following steps: The first step is to use BIM software to create a 3D digital model of the pipeline, output an OBJ format file, and import it into the Unity3D engine to create a prefab. The second step is to establish structured domain randomization rules in the Unity3D virtual environment; The third step is to set global parameters and establish simulation algorithms for pipe disconnection and misalignment, rubber ring detachment, concrete spalling, pipe corrosion, pipe siltation, and obstruction blockage, so as to randomly generate simulation scenarios of urban drainage pipe defects. Step 4: Based on the Unity3D physics engine, establish a simulation algorithm for the scanning and detection process of the LiDAR robot inside the pipe, and set the parameters related to the sensor equipment; Step 5: Use an automated semantic annotation-based synthetic point cloud generation algorithm to achieve automated collection of synthetic point cloud defect data; Step 6: Use deep learning data augmentation strategies based on pre-training and fine-tuning to train the base model using synthetic data, and then combine it with engineering measured data to optimize the model parameters and obtain a pipeline defect semantic segmentation model. The simulation algorithm for pipe disconnection and misalignment described in step three is established according to the following formula: (1)； In formula (1): These are the coordinates of the pipeline control points after a disconnection defect occurs; These are the initial pipe coordinates; , and They are disjointed along x axis, y The random displacements of the x-axis and z-axis that are greater than zero, in meters; It is the joint length of the pipe socket section; The coordinates of the pipeline control points after the misalignment defect occurs. ; It's a wrong path along z The random displacement of the axis that is greater than zero, in meters; They are along x , y and z The random rotation angle of the axis; The fourth step specifically includes: Simulating the virtual robot's LiDAR scanning process using Ray rays and the `Physics.RaycastAll()` function in Unity3D; setting the point cloud resolution: the distance between points at a position of 10m is... The angular step size of the laser beam in both the vertical and horizontal directions is... The scanning range of the virtual robot is set to a vertical angle of [value missing]. The angle between the horizontal direction and the horizontal direction is The virtual robot's lidar emits signals vertically. Ray rays are used to simulate a vertical scan. If a horizontal rotation is performed using an angular step size to simulate a horizontal scan, then the maximum number of point clouds that can be collected in each scan is... In the virtual robot's scanning simulation parameter settings, the sensor length is set to the farthest reaching distance of the scanning ray. After a single scan, the virtual robot will move to the next control point coordinates of the pipe centerline. When the virtual robot moves to the last control point coordinates in the scene and the point cloud acquisition task is not yet completed, a new concrete pipe defect scene will be automatically regenerated, and the point cloud acquisition task will continue.
2. The data augmentation method for semantic segmentation of point clouds of urban drainage pipeline defects according to claim 1, characterized in that, The first step specifically includes: creating a 3D digital geometric model of the pipeline in BIM software according to the pipeline design drawings used in the project; the created 3D digital geometric model pre-divides the pipeline into three sections: spigot section, intermediate pipe section, and socket section; the length of each pipe section is divided into effective length and interface length; exporting OBJ format files from BIM software and importing them into Unity3D to create a prefabricated pipeline; setting four control points on the prefabricated pipeline, namely the starting point, socket end point, intermediate point, and spigot end point, thereby dividing the effective length of the pipeline into four equal parts.
3. The data augmentation method for semantic segmentation of point clouds of urban drainage pipeline defects according to claim 2, characterized in that, The second step involves: Structural domain randomization, from global to local, comprises three components: global parameters, rule lines, and objects placed along the rule lines. It establishes probabilistic relationships between different components of structural domain randomization. Global parameters are set in the simulated scene, controlling the rule lines aligned with the generated defect objects. These parameters include the number of control points, the probability of each defect occurring, and the scanning direction. The number of control points is determined by the number of simulated pipes in the scene and the number of control points on each pipe section. The rule lines of structural domain randomization include the pipe top line, pipe center line, and pipe bottom line. Generating rule lines requires first installing pipe prefabs along the pipe axis, with the pipe prefab's axis direction pre-set to the z-axis direction in the Unity world coordinate system. The pipe center line is generated from control points, and then offset vertically by one unit of pipe radius along the y-axis to obtain the other two rule lines: the pipe top line and the pipe bottom line. The scanning direction is set to scan along the positive z-axis, scan along the negative z-axis, and scan in a random direction.
4. The data augmentation method for semantic segmentation of point clouds of urban drainage pipeline defects according to claim 3, characterized in that, The simulation algorithm for rubber ring detachment described in step three is established as follows: First, rubber rings of different shapes are modeled in BIM software, and then imported into Unity3D to create rubber ring prefabricated bodies; the rubber ring prefabricated bodies are placed on the misalignment or disengagement defect control points of the pipe top line in a randomized and probabilistic manner, while simultaneously along... x and y Axis Random Scaling along xoy The plane rotates randomly between -180° and 180°, along... x , y , z Axis Random Scaling ;in, This indicates the distance from the edge to the center of the rubber ring preform. and These represent the minimum and maximum scaling factors for the rubber ring size, respectively; The simulation algorithm for concrete spalling is established as follows: Based on the surface bending characteristics of the pipeline, a digital model of the concrete spalling defect is constructed on the inner diameter of the pipeline BIM design model, and the origin of the spalling defect is set at the center of the cross-section; the spalling defect consists of a spalling surface and a spalling opening, with the spalling opening representing the laser's entrance during the scanning process, to achieve the occlusion relationship of the spalling; then, the OBJ digital model of the concrete spalling defect is imported into Unity3D and a precast spalling defect is created; when generating the precast spalling defect, the depth and shape of the spalling are randomly selected, and the spalling is considered... z The axis rotates randomly between -180° and 180°, along z random translation of axis ,along z Axis Random Scaling And place it on the centerline of the pipe excluding the first control point according to the probability parameters; where, Set it to half the distance between the two control points. and These are the minimum and maximum scaling factors for the peeling size, respectively; The simulation algorithm for pipeline corrosion is established as follows: the modeling of corrosion defects is implemented in the Unity3D environment through mesh programming using vertices and triangles. The coordinates of the vertices on the corroded surface are generated by the following formula: (2)； Set the edge vertices of Type I erosion separately To simulate the edge location of corrosion, Type II corrosion simulates the case of complete corrosion of the entire cross section, and triangular faces are used to connect the edge vertices to form a circular mesh; In formula (2): This represents a function that converts a polar coordinate system to a rectangular coordinate system. Let be the design inner diameter of the pipe; d is a random variable representing the corrosion level, and ,in, It's the pipe wall thickness. Percentage of corrosion loss; Set as Random corrosion angles within the range, and These represent the minimum and maximum corrosion angles of the pipe cross-section, respectively. This represents the length of corrosion along the longitudinal direction of the pipe, and is a random value. , and These represent the minimum and maximum corrosion lengths along the longitudinal direction of the pipeline, respectively. This refers to creating a three-dimensional coordinate vector in Unity; the simulation algorithm for pipe siltation is established according to the following method: Berlin noise and fractal Brownian motion noise are used to simulate the topographical undulations of siltation defects, except... y Besides the change in coordinates, the accumulation defect is in x shaft and z The range of the axis is defined as The simulation of changes in siltation depth was carried out along... y random translation of the axis , For fractal Brownian motion algorithms in y The height of the undulation on the axis; The obstacle blockage simulation algorithm is established as follows: Brick models of various shapes and sizes are constructed in BIM software, then imported into Unity3D to create obstacle prefabs. Subsequently, Rigidbody components are added to the obstacle prefabs to simulate gravity and motion resistance based on the physics engine. Simultaneously, Collider components are added to the obstacle prefabs, pipe prefabs, and siltation mesh. The Random.onUnitSphere() function is used to generate obstacles from random directions according to probability parameters, and gravity is used to simulate obstacles falling to the bottom of the pipe.
5. The data augmentation method for semantic segmentation of point clouds of urban drainage pipeline defects according to claim 1, characterized in that, Various scanning errors are introduced during the lidar scanning process of the virtual robot to make it more closely resemble the real point cloud situation; the scanning errors include random errors, distance errors, and equipment errors, as shown in the following formula: (3)； In the formula: These are the scan coordinate points after applying the error; These are idealized scan coordinate points; It is random error, including x , y and z Components on the axis , and And each component follows a normal distribution; It is the mean of a normal distribution. It is the standard deviation of the normal distribution. The mean is The standard deviation is The normal distribution of the values is determined by the technical parameters of the simulated equipment. Indicates distance error; Indicates the additional standard deviation per unit ray distance; To scan the distance from the sensor position to the target point; This is the normalized vector of the scanning ray vector.
6. The data augmentation method for semantic segmentation of point clouds of urban drainage pipeline defects according to claim 5, characterized in that, The specific process of the synthetic point cloud generation algorithm using automated semantic annotation in the fifth step is as follows: S5.1, Input information about the impact between the laser beam and the object, simulated by the physics engine. ,in, This indicates information about the first object hit by the ray. For the second object, and so on; It refers to the coordinate position in the world coordinate system; S5.2, the output result is set as: the coordinates of the point cloud acquired by the laser beam in the laser radar scanning coordinate system of the virtual robot. and tags tag ,in, This indicates that it is normal. 1 indicates peeling. 2 represents an obstacle. 3 indicates corrosion. 4 indicates a wrong mouth. 5 indicates siltation. 6 indicates a disconnect. 7 indicates a defect in the rubber ring; S5.3, Get the type of the first object hit by the ray. . transform.tag ; S5.4, if . transform.tag If it is an obstacle, then ; ,in, These are the position coordinates of the virtual robot in the world coordinate system; like . transform.tag If it is a defect in the rubber ring, then ; ; like . transform.tag For siltation, then ; ; like . transform.tag For the opening to peel off, the first scenario is that corrosion did not occur simultaneously. and . transform.tag If it is a peeling surface, then ; The second scenario is that the corrosion depth is less than the peeling depth. and . transform.tag For corrosion surface and . transform.tag If it is a peeling surface, then ; ; If the pipe becomes disconnected and . transform.tag If it is a spigot or socket, then ; ; If the pipe is misaligned and . transform.tag If it is a spigot or socket, then ; ; If the pipe corrodes , . transform.tag For pipe sections, spigots, or sockets, and . transform.tag If it is a corroded surface, then ; ; like . transform.tag If it is a pipe section, spigot, or socket, then ; ; S5.5, Obtaining Point Cloud Data Furthermore, normal vector data is added using Python toolkits in Open3D.