A concrete pipe damage identification and volume quantification method based on SSA-PointNet++ neural network

Abstract

This invention discloses a method for concrete pipe damage detection and volume quantification based on the SSA-PointNet++ neural network. The method includes the steps of acquiring a 3D point cloud dataset of concrete pipe damage, processing and segmenting the data, constructing and training an SSA-PointNet++ neural network model, adjusting model parameters, testing the model, and finally quantifying the volume. This method develops an algorithm suitable for concrete pipe damage detection and volume quantification. The model is trained based on concrete pipe damage data collected by a depth camera to improve the model's robustness and generalization capabilities. The method utilizes parameterized point cloud projection and registration, and uses a Poisson surface reconstruction algorithm to complete 3D damage reconstruction and volume quantification.

Description

Technical Field

[0001] The present invention relates to the technical field of pipeline damage identification and volume quantification, and in particular to a concrete pipeline damage identification and volume quantification method based on an SSA-PointNet++ neural network. Background Art

[0002] Urban underground drainage systems are one of the most critical infrastructures in modern cities. On one hand, they ensure the separation of rainwater and sewage, significantly contributing to resident health, sewage treatment, and environmental protection. On the other hand, they ensure urban traffic safety during extreme weather conditions. However, with the continuous development of the economy, the total length of underground drainage pipelines in my country has continued to grow. At the same time, various problems with underground drainage systems have also become more prominent, such as cracks, misalignments, and corrosion. These problems can affect drainage efficiency, causing road flooding during rainy days, or even lead to road collapses, resulting in casualties and property damage. Therefore, it is crucial to regularly inspect existing drainage pipelines, promptly identify potential safety hazards, and implement effective repair measures.

[0003] Currently, the main pipeline inspection systems available domestically and internationally include the pipeline scanning and evaluation system (SSET), pipeline endoscope sonar inspection, multi-sensor inspection systems, periscope inspection, pipeline closed-circuit television (CCTV) inspection systems, ground-penetrating radar, and infrared temperature recording and analysis. Among these, pipeline closed-circuit television (CCTV) inspection systems have been widely used in underground drainage pipeline inspections. However, this inspection method requires a significant amount of time for professional identification and evaluation, and is also subject to significant subjective factors. However, by using CCTV inspection technology to obtain image data and intelligently train large numbers of images using deep learning methods, the trained network can automatically classify drainage pipeline defects and segment and label each type of defect, which is more efficient and accurate than traditional methods. However, while two-dimensional images can detect some structural damage, they cannot quantify the damage volume. The emergence of three-dimensional information has provided insights for other inspection methods.

[0004] Therefore, there is an urgent need for a new automatic detection method for concrete pipe damage volume that can automatically complete concrete pipe damage volume detection at different positions and angles. Summary of the Invention

[0005] To address the problems existing in the prior art, the present invention provides a method for concrete pipe damage identification and volume quantification based on the SSA-PointNet++ neural network. Model training is performed based on three-dimensional data collected by a depth camera to improve the method's robustness and generalization capabilities. Parametric point cloud projection and registration are employed, and a Poisson surface reconstruction algorithm is used to complete three-dimensional reconstruction and volume quantification of the damaged point cloud. This method has a wide range of applications and can achieve pipeline damage detection and surface reconstruction. It has high detection accuracy and efficiency, and high-quality surface reconstruction, thus addressing the problems mentioned in the aforementioned background technology.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for concrete pipe damage identification and volume quantification based on the SSA-PointNet++ neural network, comprising the following steps:

[0007] S1. Acquire a 3D point cloud dataset of concrete pipe damage: collect a depth image of the concrete pipe damage and generate a 3D point cloud from it to obtain a 3D point cloud dataset;

[0008] S2. Data processing and division: Classify and annotate the 3D point cloud dataset obtained in step S1, then amplify it using data augmentation methods, and finally divide the dataset into a training set, a validation set, and a test set;

[0009] S3. Build and train the SSA-PointNet++ neural network model: Build the SSA-PointNet++ neural network model and use transfer learning to initialize the model, set hyperparameters, import the training set into the SSA-PointNet++ neural network model, and train the model.

[0010] S4. Model parameter tuning: Adjust hyperparameters sequentially, import validation set data into the model, compare the loss and accuracy curves of the model under different hyperparameters, find the optimal hyperparameters, and obtain the optimal SSA-PointNet++ neural network model;

[0011] S5. Model testing: Import the test set data into the optimal SSA-PointNet++ neural network model for testing, output various numerical evaluation indicators, and determine whether they meet the expected values;

[0012] S6. Volume quantification: Segment the damage and project the damage point cloud according to the pipeline surface parameters. Then register it with the original damage point cloud to generate a complete damage point cloud. Use the Poisson equation to reconstruct the surface of the complete damage point cloud. Perform validity testing on the reconstructed damage volume to complete volume quantification.

[0013] Preferably, the specific steps of obtaining the concrete pipe damage three-dimensional point cloud dataset in step S1 are:

[0014] S11, collecting a depth image of the concrete pipe damage by using a depth camera to obtain an original depth dataset;

[0015] S12, convert the depth image in the dataset into an image with a resolution of 640×576 pixels;

[0016] S13. Use MATLAB to calibrate the depth camera intrinsic parameters, and use the camera intrinsic parameters to convert the depth data into a three-dimensional point cloud in world coordinates to obtain a three-dimensional point cloud dataset.

[0017] Preferably, the classification and labeling in step S2 is performed using CloudCompare labeling program, and the classification and labeling are performed as normal or damaged;

[0018] The data enhancement method includes translation transformation, scale scaling, rotation transformation, symmetry transformation, stretching transformation or dithering transformation;

[0019] The data set is divided into a training set, a validation set and a test set in a ratio of 6:3:1 using a python classification program.

[0020] Preferably, the SSA-PointNet++ neural network model in step S3 introduces the self-attention mechanism into the PointNet++ neural network; the transfer learning is relative to random parameter initialization, and the neural network is initialized using the PointNet++ pre-trained model that performs well in the S3DIS and Semantic3D public data sets; the hyperparameters are the learning rate, the number of three-dimensional point clouds, the batch size, the momentum coefficient, the decay period and the decay rate.

[0021] Preferably, the various numerical evaluation indicators in step S5 include precision, accuracy and recall.

[0022] Preferably, the specific steps of step S6 are:

[0023] S61. Use the trained SSA-PointNet++ neural network to segment concrete pipe damage.

[0024] S62, Damage point cloud projection: The surface parameters for the damage 3D point cloud projection are obtained by using the normal concrete pipe 3D point cloud segmented by SSA-PointNet++;

[0025] S63. After the projected point cloud and the damage point cloud are registered, the Poisson surface reconstruction algorithm is used to complete the concrete pipe damage surface reconstruction work, and the validity of the reconstructed damage volume is tested to complete the volume quantification.

[0026] Preferably, the registration method used for registering the projection point cloud and the damage point cloud in step S63 is the ICRP algorithm.

[0027] Preferably, the indicators for effectiveness detection in step S63 are relative error and average relative error value.

[0028] Preferably, the Poisson surface reconstruction algorithm in step S63 obtains an implicit equation representing the surface information described by the point cloud model by solving the Poisson equation, and extracts isosurfaces from the equation to obtain a surface model with geometric entity information, thereby completing the surface reconstruction.

[0029] Preferably, the relative error value REV is the ratio of the absolute value of the difference between the true volume and the test volume to the true volume.

[0030]

[0031] The mean relative error value MRE is the average value of the relative error value.

[0032]

[0033] The beneficial effects of the present invention are:

[0034] 1) This invention uses a three-dimensional point cloud method to quantify the damage volume, which can intuitively represent the damage shape, volume and area, providing more detailed and specific information for repair projects;

[0035] 2) Based on the SSA-PointNet++ neural network model, this paper introduces a self-attention mechanism into the point cloud semantic segmentation CNN to enhance the network model's ability to learn features from sparsely sampled point neighborhoods, thereby improving the model's robustness and generalization ability, and enhancing recognition accuracy;

[0036] 3) The present invention uses the SSA-PointNet++ neural network combined with the ICRP algorithm and the Poisson surface reconstruction algorithm to automate the concrete pipe damage detection, segmentation, registration, and quantification processes, thereby improving the recognition efficiency of concrete pipe damage. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Flow chart of the steps of the method of the present invention;

[0038] Figure 2 Schematic diagram of the overall structure of the SSA-PointNet++ neural network of the method of the present invention. DETAILED DESCRIPTION

[0039] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0040] See also Figure 1-Figure 2 The present invention provides a technical solution: a method for concrete pipe damage identification and volume quantification based on SSA-PointNet++ neural network. The method flow is as follows: Figure 1 As shown, the following steps are included:

[0041] Step 1: Obtaining a 3D point cloud dataset of concrete pipe damage

[0042] The depth image of concrete pipe damage is collected and converted into a 3D point cloud dataset. The specific process is as follows:

[0043] 1.1 The original depth dataset is captured by the Microsoft Azure Kinect DK depth camera;

[0044] 1.2 The original acquired depth image will be automatically divided into datasets with a resolution of 640×576 pixels

[0045] 1.3 Use Matlab to calibrate the depth camera internal parameters;

[0046] 1.4 Use camera intrinsic parameters to convert depth data into a 3D point cloud in world coordinates;

[0047] Specifically, the depth image should have the following characteristics:

[0048] (1) The images should be diverse. The concrete pipe dataset should include images of light intensity, shadows, etc.

[0049] (2) The shooting distance of the pictures is wide, with different shooting distances, which improves the generalization ability;

[0050] (3) The image perspectives should be diverse, including frontal, side, and oblique views;

[0051] (4) The disease target should be representative and have obvious disease characteristics.

[0052] Step 2: Data processing and division

[0053] Based on the dataset obtained in step 1, the concrete pipe damage dataset is classified and labeled, amplified using the data augmentation method, and divided into a training set, a validation set, and a test set in proportion to construct the concrete pipe damage dataset. The specific process is as follows:

[0054] 2.1 Use the CloudCompare annotation program to classify and annotate concrete pipe damage into normal and damaged types;

[0055] 2.2 The labeled dataset is amplified using data augmentation technology;

[0056] 2.3 Use a python classification program to divide the dataset into training set, validation set and test set in a ratio of 6:3:1.

[0057] Specifically, the main methods of data enhancement include translation transformation, scaling, rotation transformation, symmetry transformation, stretching transformation, and jitter transformation.

[0058] Specifically, the images in the training set, validation set, and test set do not overlap with each other, which is conducive to testing the robustness and generalization ability of the model.

[0059] Step 3: Build and train the SSA-PointNet++ neural network model

[0060] Equipped with the SSA-PointNet++ neural network model framework, it uses transfer learning to initialize the model, set hyperparameters, import the training set into the neural network model, and train the model;

[0061] 3.1 Equipped with the SSA-PointNet++ neural network model framework, this framework introduces the self-attention mechanism into the PointNet++ neural network, such as Figure 2 As shown;

[0062] 3.2 Transfer learning is different from random parameter initialization. It uses the PointNet++ pre-trained model that performs well on the S3DIS and Semantic3D public datasets to initialize the neural network, speeding up training efficiency and improving detection accuracy.

[0063] Specifically, the SSA-PointNet++ neural network introduces the self-attention mechanism into PointNet++ to construct a point cloud semantic segmentation network, and clearly divides the self-attention of the sampling point neighborhood into two parts: central self-attention and neighborhood self-attention. The two are integrated and combined with different spatial encoding methods to enhance the network model's learning of the topological structure of the sampling point neighborhood. Then, an attention pooling module is constructed to enhance the effective transmission of important information in the network, and multiple global features extracted by attention pooling and maximum pooling are integrated through differential pooling functions to improve the robustness and generalization ability of the point cloud semantic segmentation results.

[0064] Step 4: Model parameter adjustment

[0065] Adjust the hyperparameters sequentially, introduce the validation set data to test the model, compare the loss value and accuracy curves of the model under different hyperparameters (loss value decrease curve and accuracy increase curve), find the optimal hyperparameters, and obtain the optimal SSA-PointNet++ neural network model;

[0066] As the preferred solution, this example has been tuned several times, and the final model hyperparameters are: initial learning rate of 0.001, batch size of 8, momentum coefficient of 0.9, decay period of 100,000, and decay rate of 0.96.

[0067] Specifically, the learning rate controls the speed at which weight parameters are updated. During model training, setting a learning rate that is too high can cause the model to miss the optimal solution and overfit; setting a learning rate that is too low can slow down learning. The learning rate should be set based on experience and continuous experimentation.

[0068] Specifically, in order to prevent the learning rate from being too large and oscillating back and forth when converging to the global optimum, it is necessary to let the learning rate decrease exponentially with the number of training rounds and converge the learning step of the gradient descent.

[0069] Step 5: Model testing

[0070] Based on the test set data, the optimal model is tested and various numerical evaluation indicators are output to determine whether the expected values ​​are achieved;

[0071] Specifically, the numerical evaluation indicators include precision, accuracy, and recall.

[0072] Specifically, whether the model has achieved the expected value is determined by whether the obtained precision and accuracy are greater than 80% and whether the recall rate is greater than 85%.

[0073] Step 6: Volume Quantification

[0074] The damage is segmented and the damage point cloud is projected according to the pipeline surface parameters. The damage point cloud is then registered with the original damage point cloud to generate a complete damage point cloud. The surface of the complete damage point cloud is reconstructed using the Poisson equation, and the validity of the reconstructed damage is tested. The specific process is as follows:

[0075] 4.1 Use the trained SSA-PointNet++ neural network to segment concrete pipe damage;

[0076] 4.2 The surface parameters for projection of the damaged 3D point cloud are obtained from the 3D point cloud of the normal concrete pipe segmented by SSA-PointNet++;

[0077] 4.3 After the projected point cloud and the damage point cloud are registered, the Poisson surface reconstruction algorithm is used to complete the concrete pipe damage surface reconstruction work;

[0078] 4.4 Perform validity check on the reconstructed lesion volume.

[0079] Furthermore, the registration method used in the registration of the projection point cloud and the damage point cloud in 4.3 is the ICRP algorithm. The ICRP algorithm is based on the ICP algorithm and performs a secondary reverse search for the nearest point. It uses the principle that rigid motion should keep the corresponding points in the overlapping area of ​​the measured object unchanged, thereby eliminating the effect of invalid registration points.

[0080] Furthermore, the Poisson surface reconstruction algorithm 4.3 solves the Poisson equation to obtain an implicit equation representing the surface information described by the point cloud model, and extracts isosurfaces from the equation to obtain a surface model with geometric entity information, thereby completing the surface reconstruction.

[0081] Furthermore, the effectiveness detection indicators in 4.4 are relative error value and average relative error value, wherein the relative error value REV is the ratio of the absolute value of the difference between the true volume and the test volume to the true volume,

[0082]

[0083] The mean relative error value MRE is the average value of the relative error value.

[0084]

[0085] Where N represents the number of test volumes, and i represents the number corresponding to the test volume.

[0086] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A concrete pipe damage identification and volume quantification method based on SSA-PointNet++ neural network, characterized in that: The steps include: S1. Acquire a 3D point cloud dataset of concrete pipe damage: collect a depth image of the concrete pipe damage and generate a 3D point cloud from it to obtain a 3D point cloud dataset; S2. Data processing and division: Classify and annotate the 3D point cloud dataset obtained in step S1, then amplify it using data augmentation methods, and finally divide the dataset into a training set, a validation set, and a test set; S3. Build and train the SSA-PointNet++ neural network model: Build the SSA-PointNet++ neural network model and use transfer learning to initialize the model, set hyperparameters, import the training set into the SSA-PointNet++ neural network model, and train the model. S4. Model parameter tuning: Adjust hyperparameters sequentially, import validation set data into the model, compare the loss and accuracy curves of the model under different hyperparameters, find the optimal hyperparameters, and obtain the optimal SSA-PointNet++ neural network model; S5. Model testing: Import the test set data into the optimal SSA-PointNet++ neural network model for testing, output various numerical evaluation indicators, and determine whether they meet the expected values; S6. Volume quantification: Segment the damage and project the damage point cloud based on the pipeline surface parameters. Regulate the damage point cloud with the damage point cloud to generate a complete damage point cloud. Use the Poisson equation to reconstruct the surface of the complete damage point cloud. Perform validity testing on the reconstructed damage volume to complete volume quantification. The specific steps are as follows: S61. Use the trained SSA-PointNet++ neural network to segment concrete pipe damage. S62, Damage point cloud projection: The surface parameters for the damage 3D point cloud projection are obtained by using the normal concrete pipe 3D point cloud segmented by SSA-PointNet++; S63. After the projected point cloud and the damage point cloud are registered, the Poisson surface reconstruction algorithm is used to complete the concrete pipe damage surface reconstruction work, and the validity of the reconstructed damage volume is tested to complete the volume quantification.

2. The method for concrete pipe damage identification and volume quantification based on the SSA-PointNet++ neural network according to claim 1 is characterized by: The specific steps of obtaining the concrete pipe damage three-dimensional point cloud dataset in step S1 are: S11, collecting a depth image of the concrete pipe damage by using a depth camera to obtain an original depth dataset; S12, convert the depth image in the dataset into an image with a resolution of 640×576 pixels; S13. Use MATLAB to calibrate the depth camera intrinsic parameters, and use the camera intrinsic parameters to convert the depth data into a three-dimensional point cloud in world coordinates to obtain a three-dimensional point cloud dataset.

3. The method for concrete pipe damage identification and volume quantification based on the SSA-PointNet++ neural network according to claim 1 is characterized by: The classification and labeling in step S2 is performed using the CloudCompare labeling program, and the classification and labeling are marked as normal or damaged; The data enhancement method includes translation transformation, scale scaling, rotation transformation, symmetry transformation, stretching transformation or dithering transformation; The data set is divided into a training set, a validation set and a test set in a ratio of 6:3:1 using a python classification program.

4. The method for concrete pipe damage identification and volume quantification based on the SSA-PointNet++ neural network according to claim 1 is characterized by: The SSA-PointNet++ neural network model in step S3 introduces the self-attention mechanism into the PointNet++ neural network. The transfer learning is relative to random parameter initialization, and the neural network is initialized using the PointNet++ pre-trained model that performs well in the S3DIS and Semantic3D public datasets. The hyperparameters are the learning rate, the number of three-dimensional point clouds, the batch size, the momentum coefficient, the decay period, and the decay rate.

5. The method for concrete pipe damage identification and volume quantification based on the SSA-PointNet++ neural network according to claim 1 is characterized by: The various numerical evaluation indicators in step S5 include precision, accuracy and recall.

6. The method for concrete pipe damage identification and volume quantification based on the SSA-PointNet++ neural network according to claim 1 is characterized by: The registration method used for registering the projection point cloud and the damage point cloud in step S63 is the ICRP algorithm.

7. The method for concrete pipe damage identification and volume quantification based on the SSA-PointNet++ neural network according to claim 1 is characterized by: The indicators for effectiveness detection in step S63 are relative error and average relative error value.

8. The method for concrete pipe damage identification and volume quantification based on the SSA-PointNet++ neural network according to claim 1 is characterized by: The Poisson surface reconstruction algorithm in step S63 solves the Poisson equation to obtain an implicit equation representing the surface information described by the point cloud model, and extracts isosurfaces from the equation to obtain a surface model with geometric entity information, thereby completing the surface reconstruction.

9. The method for concrete pipe damage identification and volume quantification based on the SSA-PointNet++ neural network according to claim 7 is characterized in that: The relative error value REV is the ratio of the absolute value of the difference between the true volume and the test volume to the true volume. The mean relative error value MRE is the average value of the relative error value.

Images