Causal reasoning based tunnel point cloud multi-object segmentation integrated method and system

Abstract

The application discloses a tunnel point cloud multi-target segmentation integrated method and system based on causal reasoning, and the method comprises the following steps: collecting point cloud data of a tunnel and converting the point cloud data into a two-dimensional point graph based on a circle projection algorithm; extracting global features of the point cloud data based on the two-dimensional point graph and a two-dimensional Unet model, and extracting local features of the point cloud data based on the two-dimensional point graph and edge convolution; performing data enhancement on the point cloud data of the tunnel based on a causal reasoning model; respectively classifying the point cloud data of the tunnel after data enhancement according to the global features and the local features; and performing feature fusion on the classification results based on an improved D-S evidence-based feature fusion method. In view of the effective and rapid solution to the three-dimensional point cloud semantic segmentation problem, the application provides an integrated method based on point projection and a dynamic graph convolutional neural network, processes input point cloud, and performs good semantic reasoning, so that seepage categories can be identified and determined at high precision and high speed.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional point cloud multi-target detection and recognition technology in shield tunneling construction, and more specifically, relates to a method and system for multi-target segmentation and integration of tunnel point clouds based on causal reasoning. Background Technology

[0002] Subway tunnels, as a crucial support for subway operations, are affected by external environmental and internal factors, such as train vibration, soil pressure, geological conditions, and aging processes. If timely inspection and maintenance measures are not implemented, tunnel deterioration can worsen, impacting normal subway operations. In particular, seepage caused by deterioration can gradually evolve, affecting the tunnel structure's performance and potentially leading to partial or complete collapse, jeopardizing the subway's safe operation. This is a common problem in tunnel operation and management. A traditional solution is manual tunnel inspection, which, while effective, is inefficient, time-consuming, and labor-intensive, making it unsuitable for large tunnels. Therefore, developing advanced methods is of great significance and a priority.

[0003] Currently, both image-based and point cloud-based methods have been proposed for tunnel monitoring. These methods undoubtedly improve the effectiveness and efficiency of tunnel exploration. However, the information collected by images is limited by the field of view, and these images cannot fully cover the entire tunnel. Furthermore, the nature of two-dimensional (2D) images makes it difficult to represent three-dimensional (3D) objects in the real world. In addition, the weak lighting conditions inside tunnels may lead to poor image quality, resulting in significantly different evaluation results. Point clouds, on the other hand, are acquired through 3D laser scanning, requiring no additional lighting. Therefore, they can provide 3D scene descriptions and are applicable to dark environments within tunnels. 3D point clouds can provide detailed spatial and attribute information for precise and fine-grained object description, making them more suitable for tunnel object representation than visual images. Point cloud-based methods work directly on 3D point clouds and perform well, but further optimization is still needed in terms of time efficiency and accuracy. At the same time, the unstructured and irregular nature of 3D point clouds can interfere with feature learning in the model, thus seriously hindering effective result analysis. In summary, most of the above research has achieved good results in specific scenarios, improving model performance, enriching the domain knowledge base, and promoting industry development. These studies are mainly based on feature engineering, which strengthens feature extraction, continuously improves models, and proposes new models. Effective feature extraction enables models to capture and learn features, thereby improving performance.

[0004] Causal intervention breaks down spurious connections between different classes, enabling the model to learn independently decoupled target features. While causal intervention methods can achieve efficient image generalization feature extraction, their effectiveness in reducing background interference and feature coupling has not been fully applied to point cloud-based segmentation tasks, and most related work focuses on enhancing model performance on image-based tasks.

[0005] To overcome the problem of effective point feature extraction and improve segmentation accuracy, this invention develops an integrated tunnel segmentation model, proposes an integrated framework based on projection point segmentation and point segmentation, and designs an integrated segmentation model that can effectively and efficiently segment various targets, including seepage, in large-scale three-dimensional tunnel point clouds. Summary of the Invention

[0006] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a multi-target segmentation integration method and system for tunnel point clouds based on causal reasoning. For effectively and rapidly solving the problem of semantic segmentation of 3D point clouds, an integration method based on point projection and dynamic graph convolutional neural networks is proposed. This method processes the input point cloud and performs sound semantic reasoning, thereby enabling high-precision and rapid identification and determination of seepage categories.

[0007] To achieve the above objectives, according to one aspect of the present invention, a method for multi-target segmentation and integration of tunnel point clouds based on causal reasoning is provided, comprising:

[0008] The point cloud data of the tunnel is collected and converted into a two-dimensional point map based on the circular projection algorithm;

[0009] Global features of point cloud data are extracted based on 2D point plots and 2D Unet models, while local features of point cloud data are extracted based on 2D point plots and edge convolution.

[0010] Data augmentation of tunnel point cloud data based on causal reasoning model;

[0011] The tunnel point cloud data are classified based on both global and local feature data enhancements. The classification results are then fused using an improved DS-based evidence-based feature fusion method.

[0012] Furthermore, the process of collecting the point cloud data of the tunnel and converting it into a two-dimensional point map based on a circular projection algorithm includes:

[0013] The point cloud data of the tunnel is transformed from spherical coordinate space to Cartesian coordinate space according to the first coordinate transformation matrix;

[0014] The vectors of the point cloud data of the tunnel in the Cartesian coordinate space are sorted and grouped according to the gradient interval, and the cross section of each small segment of the tunnel is divided into a group.

[0015] The center of the cross section is determined by the least squares circle fitting algorithm. The grouped point cloud data is then projected and transformed using the center of the cross section and the second coordinate transformation matrix to obtain a two-dimensional point map.

[0016] Furthermore, the first coordinate transformation matrix includes:

[0017]

[0018] Where x, y, and z represent the three-dimensional coordinates of each point in the point cloud data, and R, α, and β are the radius, elevation, and azimuth of the collected tunnel point cloud data, respectively.

[0019] Further, the step of determining the cross-section center using the least squares circle fitting algorithm, and then performing a projection transformation on the grouped point cloud data based on the cross-section center and the second coordinate transformation matrix to obtain a two-dimensional point map, includes:

[0020] The center of the cross section (x) is determined based on the least squares circle fitting algorithm. c ,z c );

[0021] Based on the cross-section center and the second coordinate transformation matrix, the point cloud data is transformed from Cartesian coordinate space to polar coordinates, and the coordinates of the point cloud data in polar coordinates are determined.

[0022] Furthermore, the extraction of local features from the point cloud based on two-dimensional point maps and edge convolution includes:

[0023] Select a rectangular region of size N×N from the two-dimensional point map, and group the points within the rectangular region.

[0024] The k-edge vector is obtained by finding the k nearest points of each point within a rectangular region using the KNN algorithm.

[0025] Features are extracted from the k-edge vector using a 1×1 convolutional layer to obtain k features;

[0026] The max pooling operation integrates k features into one feature, resulting in a local feature.

[0027] Furthermore, data augmentation is performed on the tunnel point cloud data based on a causal inference model, including:

[0028] Construct a causal structure model;

[0029] Based on the causal structure model and the point cloud data of the tunnel, a causal structure graph corresponding to the point cloud is constructed.

[0030] A nonlinear converter is constructed as a dry calculation unit to enhance the tunnel point cloud data based on the tunnel point cloud data, the dry calculation unit, and the causal structure graph.

[0031] Furthermore, the said budgeting unit includes:

[0032] Φ(x)=Φ I (x)+Φ P (x)

[0033] Where, Φ I (x) and Φ P (x) are two nonlinear converters, φ I (x) is a shallow convolutional network consisting of two convolutional layers and a Leaky ReLU layer, used to generate various transformation functions for the intensity at each point; Φ P (x) is a random vector applied to Cartesian coordinates.

[0034] Furthermore, based on the improved DS evidence-based feature fusion method, feature fusion is performed on the classification results, including:

[0035] Identify the evidence for each classification result and compile the evidence into an evidence set;

[0036] Determine the degree of conflict of evidence based on the evidence and the evidence set; determine the evidence trust factor based on the degree of conflict of evidence; and determine the evidence weight based on the evidence trust factor.

[0037] Feature fusion is performed based on evidence and evidence weights.

[0038] Furthermore, the improved DS-based evidence-based feature fusion method for performing feature fusion on the classification results also includes: evaluating the classification results after feature fusion based on intersection-union ratio, recall, precision, and F1 score.

[0039] According to a second aspect of the present invention, a tunnel point cloud multi-target segmentation integration system based on causal reasoning is provided, comprising:

[0040] The first main module is used to collect point cloud data of the tunnel and convert it into a two-dimensional point map based on the circular projection algorithm;

[0041] The second main module is used to extract global features of point cloud data based on two-dimensional point maps and two-dimensional Unet models, and to extract local features of point cloud data based on two-dimensional point maps and edge convolution.

[0042] The third main module is used to perform data augmentation on the point cloud data of the tunnel based on the causal reasoning model;

[0043] The fourth main module is used to classify the tunnel point cloud data based on the global and local feature data enhancements, respectively, and to perform feature fusion on the classification results based on the improved DS evidence-based feature fusion method.

[0044] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects:

[0045] 1. The tunnel point cloud multi-target segmentation integration method of the present invention transforms three-dimensional point cloud data into two-dimensional point maps, extracts global and local features of the point cloud data, and performs data augmentation on the tunnel point cloud data based on a causal inference model; classifies the tunnel point cloud data after data augmentation based on global and local features respectively, and performs feature fusion on the classification results based on an improved DS evidence-based feature fusion method. This invention aims to effectively and rapidly solve the problem of three-dimensional point cloud semantic segmentation by proposing an integration method based on point projection and dynamic graph convolutional neural networks, which processes the input point cloud and performs good semantic inference, thereby achieving high-precision and rapid identification and determination of seepage categories.

[0046] 2. The tunnel point cloud multi-target segmentation and integration method of this invention combines projection strategies with point-based methods. Based on this, a causal inference data augmentation method is proposed to alleviate confusion between segmentation classes and seepage classes. During the testing phase, the DS evidence fusion method is used to fuse local and global features from different CNN models. This provides an effective technical solution for tunnel safety and health status checks and maintenance during tunnel operation.

[0047] 3. The tunnel point cloud multi-target segmentation ensemble method of the present invention exhibits the best segmentation score compared with original point-based segmentation models of different baselines. Compared with the same ensemble framework based on other baselines (including PointNet, PCT, PointNet2, and PointMLP), the framework based on the DGCNN baseline design performs the best. The data augmentation and feature fusion methods based on causal inference can effectively assist the model in data segmentation, achieving excellent results.

[0048] 4. The tunnel point cloud multi-target segmentation and integration method of the present invention provides a new idea and implementation approach for autonomously realizing tunnel multi-target segmentation based on three-dimensional point clouds and constructing intelligent autonomous unmanned tunnel operation safety status monitoring. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the integrated model of segmented tunnel point clouds according to an embodiment of the present invention;

[0050] Figure 2 This is a schematic diagram of data acquisition for the present invention ((a) point cloud data; (b) real scene of different target objects in the tunnel);

[0051] Figure 3 This is a structural description of the global feature extraction module in an embodiment of the present invention;

[0052] Figure 4 This is a schematic diagram of the local feature extraction module according to an embodiment of the present invention;

[0053] Figure 5 This is a schematic diagram of a causal model according to an embodiment of the present invention;

[0054] Figure 6 This is a visualization diagram of the segmentation results in an embodiment of the present invention. Figure 6 (a) Baseline results of the original method; (b) Results of the improved method; (c) Baseline results of the original method; (d) Results of the improved method;

[0055] Figure 7 This is a schematic diagram of a tunnel point cloud multi-target segmentation and integration system based on causal reasoning, according to an embodiment of the present invention.

[0056] Figure 8 This is a flowchart of a multi-target segmentation and integration method for tunnel point clouds based on causal reasoning, according to an embodiment of the present invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0058] In the description of this invention, unless otherwise explicitly specified and limited, the terms "connected," "linked," and "fixed" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0059] Those skilled in the art will understand that, unless otherwise stated, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the word “comprising” as used in the specification of this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0060] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as in the embodiments of this application.

[0061] This invention provides a multi-target segmentation and integration method for tunnel point clouds based on causal reasoning, which can be applied to the field of point cloud segmentation technology, and is especially suitable for point cloud segmentation in environments such as tunnels with dim ambient light.

[0062] Point cloud classification assigns a semantic label to each point. Point cloud classification categorizes points into different point cloud sets. Point clouds within the same set share similar or identical attributes, such as ground, trees, or people. This is also known as point cloud semantic segmentation.

[0063] There are three main methods for segmenting point cloud data: edge-based segmentation, surface-based segmentation, and clustering-based segmentation.

[0064] 1) Edge-based segmentation methods require first identifying feature lines. Feature lines are lines connecting feature points. The most common method for extracting feature points is based on curvature and normal vectors. Points where curvature or normal vectors change abruptly are typically considered feature points, such as inflection points or corners. After extracting the feature lines, the region enclosed by these lines is then segmented.

[0065] 2) The surface-based method is an iterative process that finds points with the same surface properties, divides the set of points belonging to the same basic geometric features into the same region, determines the surface to which these points belong, and finally determines the boundary between surfaces by adjacent surfaces.

[0066] 3) Clustering-based methods classify data points with similar geometric features. Geometric features can be calculated based on Gaussian curvature and mean curvature, and then clustered. Finally, the data is segmented according to the class to which it belongs.

[0067] The tunnel point cloud multi-target segmentation and integration method based on causal reasoning provided in this application can be executed by a server or a terminal. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, in-vehicle terminal, smart TV, etc., and the specific device can be determined based on the actual application scenario requirements, without limitation here.

[0068] like Figure 8 The flowchart shown is a multi-target segmentation and integration method for tunnel point clouds based on causal reasoning according to an embodiment of the present invention. The multi-target segmentation and integration method for tunnel point clouds based on causal reasoning includes steps S100 to S400.

[0069] Step S100: Collect point cloud data of the tunnel and convert it into a two-dimensional point map based on the circular projection algorithm;

[0070] Specifically, embodiments of the present invention employ a three-dimensional laser scanning system to acquire tunnel point cloud data. For example... Figure 2 As shown, there are various target objects in the tunnel environment, such as power supply rails, cables, cable supports, pipe segments, and seepage points. After data acquisition using 3D lasers, feature extraction of the 3D point cloud data is required, and the objects are classified according to their characteristics.

[0071] To convert 3D point clouds into 2D pixels, this invention proposes a circular projection algorithm, which transforms 3D point clouds into 2D point maps.

[0072] Step S100 includes steps S101 to S103.

[0073] Step S101: Transform the point cloud data of the tunnel from spherical coordinate space to Cartesian coordinate space according to the first coordinate transformation matrix;

[0074] The first coordinate transformation matrix includes:

[0075]

[0076] Where x, y, and z represent the three-dimensional coordinates of each point in the point cloud data, and R, α, and β are the radius, elevation, and azimuth of the collected tunnel point cloud data, respectively.

[0077] Step S102: Sort the vectors of the point cloud data of the tunnel in the Cartesian coordinate space and group them according to the gradient interval, and divide the cross section of each small segment of the tunnel into a group.

[0078] In Cartesian coordinates, points are arranged in ascending order of y-coordinate. Then, the sorted vectors are grouped according to their y-gradient intervals. Alternatively, the sorting can be done by either x or z, followed by grouping based on their gradient intervals. In other words, the cross-sections of each small segment in the tunnel data are grouped together.

[0079] Step S103: Determine the cross-sectional center of the cross section according to the least squares circle fitting algorithm, and perform projection transformation on the grouped point cloud data according to the cross-sectional center and the second coordinate transformation matrix to obtain a two-dimensional point map.

[0080] The process of determining the cross-sectional center based on the least squares circle fitting algorithm, and then performing a projection transformation on the grouped point cloud data based on the cross-sectional center and the second coordinate transformation matrix to obtain a two-dimensional point map, includes: determining the cross-sectional center (x) based on the least squares circle fitting algorithm. c ,z c Based on the cross-section center and the second coordinate transformation matrix, the point cloud data is transformed from Cartesian coordinate space to polar coordinates, and the coordinates of the point cloud data in polar coordinates are determined.

[0081] The second coordinate transformation matrix transforms the point cloud from Cartesian space coordinates to polar coordinates. The second coordinate transformation matrix includes:

[0082]

[0083] Where, x p and z p These are the coordinates of each point on each cross section, with the cross section center (x) c , z c In the output two-dimensional point map of the projection, each pixel of the point map represents the intensity of the corresponding point cloud.

[0084] By reducing the dimensionality of a 3D point cloud to 2D, features can be extracted using a 2D CNN method.

[0085] Step S200: Extract global features of point cloud data based on two-dimensional point map and two-dimensional U-net model; extract local features of point cloud data based on two-dimensional point map and edge convolution.

[0086] like Figure 3 The diagram illustrates the extraction of global features from point cloud data using a 2D U-net model based on a 2D point map. By introducing the 2D U-net model, global features of large-scale point clouds, especially seepage features, are effectively extracted. However, this method inevitably leads to information loss. Sometimes, grouping points of different categories into the same group requires introducing pattern scores to select the output category among several different point categories, which results in segmentation errors, especially in seepage edge regions. To further enhance the segmentation results in edge regions, we employ an effective local feature extraction method.

[0087] like Figure 4 The diagram illustrates Local Feature Extraction (LFE) of a 2D point map. The extraction of local features from point cloud data based on 2D point map and edge convolution includes:

[0088] Select a rectangular region of size N×N from the two-dimensional point map, and group the points within the rectangular region.

[0089] The k-edge vector is obtained by finding the k nearest points of each point within a rectangular region using the KNN algorithm.

[0090] Features are extracted from the k-edge vector using a 1×1 convolutional layer to obtain k features;

[0091] The max pooling operation integrates k features into one feature, resulting in a local feature.

[0092] To extract local features, we randomly select an N×N rectangular region from the 2D point cloud image and group all points within this region into groups X = {x1, ..., x2}. n} represents a point. Each point contains a four-dimensional value, namely [x, y, z, 1]. In the feature space of each point, for each point x... i The KNN algorithm is used to find the k nearest neighbors. These k neighbors are sorted in ascending order of distance, denoted as . k-edge vector points are derived from Then, a convolutional layer with a kernel size of 1×1 is introduced to extract features from the k-edge vector, thus obtaining local features.

[0093] Step S300: Perform data augmentation on the point cloud data of the tunnel based on the causal reasoning model;

[0094] The causal structure model of this invention is as follows: Figure 5 As shown. x f This represents effective features for point cloud types, such as percolation types. And x b It may be related to background elements, such as pipe segments. For segmentation, Y represents point cloud classification. Based on the causal model, the neural network model extracts global and local point features to classify tunnel points, meaning that seepage points and seepage points in the adjacent background are inevitably perceived simultaneously. Therefore, x b and x f Both can affect the output of the segmentation model. The spurious correlation between the seepage class and the segment class can confuse the neural network model and affect feature extraction.

[0095] Step S300 includes steps S301 to S303.

[0096] Step S301: Construct a causal structure model;

[0097] Causal structure models are used to determine the relationship between point cloud targets (such as seepage objects) and background (such as pipe segments). It should be noted that the construction of causal structure models is an existing technology and will not be elaborated here.

[0098] Step S302: Based on the causal structure model and the point cloud data of the tunnel, construct the causal structure diagram corresponding to the point cloud;

[0099] Specifically, the target features of the point cloud are the effective features of the point cloud class to be identified. Based on the causal structure model, the correlation between the point cloud target (such as seepage class) and the background (such as pipe segment class) is determined, and the spurious correlation between seepage class and pipe segment class is isolated.

[0100] Step S303: Construct a nonlinear converter as a dry calculation unit to enhance the tunnel's point cloud data based on the tunnel's point cloud data, the dry calculation unit, and the causal structure graph.

[0101] The dry calculation module includes two converters for point intensity and location. By converting points in the causal structure graph using the dry calculation module, it overcomes the erroneous connections between seepage class and segment class, thereby enhancing the point cloud data of the tunnel.

[0102] The budgeting unit includes:

[0103] Φ(x)=Φ I (x)+Φ P (x)

[0104] Where, Φ I (x) and Φ P (x) are two nonlinear converters, Φ I (x) is a shallow convolutional network consisting of two convolutional layers and a Leaky ReLU layer, used to generate various transformation functions for the intensity at each point; Φ P (x) is a random vector applied to Cartesian coordinates.

[0105] In each iteration, the training points of the segment class are assigned different positions and intensities, thereby overcoming the erroneous connections between the seepage class and the segment class and improving the segmentation performance of the seepage class.

[0106] Step S400: Classify the point cloud data of the data-enhanced tunnel according to global features and local features respectively, and perform feature fusion on the classification results based on the improved DS evidence-based feature fusion method.

[0107] For tunnel point cloud data, before processing, the point cloud data is labeled into several categories based on expert opinions or point cloud segmentation requirements, such as segment data, seepage data, etc. After extracting global and local features, the point cloud data is classified based on the global and local features respectively.

[0108] To improve the performance of point cloud segmentation (Intersection over Union (IOU), Recall (R), Precision (P), and F1 score), classification results based on global and local features are fused.

[0109] Step S400 includes steps S401 to S403.

[0110] Step S401: Determine the evidence for each classification result and summarize the evidence into an evidence set;

[0111] Step S402: Determine the degree of conflict of evidence based on the evidence and the evidence set; determine the evidence trust factor based on the degree of conflict of evidence; and determine the evidence weight based on the evidence trust factor.

[0112] Step S403: Perform feature fusion based on evidence and evidence weight.

[0113] For a segmentation result i of category k, there is an evidence confidence factor Crd. i，k Evidence E i，k The evidence is compiled into an evidence set. For any piece of evidence E... i，k Evidence E j，k With Evidence E i，k The degree of conflict of evidence is

[0114]

[0115] Where d represents two pieces of evidence E i，k and E j，k The Jousselme distance between them. It should be noted that the method for calculating the Jousselme distance is existing technology and will not be elaborated here.

[0116] S k It is the measure of evidence E i，k An indicator of the degree of conflict, reflecting the degree of difference between two pieces of evidence. When S k When it is smaller, it means E i，k It is more consistent with other evidence and has a lower degree of conflict. Conversely, when S k When it is large, it means E i，k It differs significantly from other evidence, and the evidence is highly divergent and conflicting.

[0117] According to the degree of conflict of evidence S k Obtain the evidence trust factor Crd i，k The evidence confidence factor for any given piece of evidence is the sum of the evidence conflict degrees of each piece of evidence with the sum of the evidence conflict degrees of the other pieces of evidence.

[0118] After obtaining the trust factors for all evidence, the evidence weight can be derived from these trust factors:

[0119] Crd max =max(Crd1,Crd2, ...,Crd n )

[0120]

[0121] Among them, Crd1, Crd2, ..., Crd n For each piece of evidence relative to evidence E i，k Evidence trust factor, Crd max This represents the maximum value of the trust factor for all evidence.

[0122] The point cloud segmentation result obtained by feature fusion based on evidence weights and evidence is as follows:

[0123]

[0124] Among them, Out k This is the comprehensive result of segmenting the k-th type of point cloud.

[0125] After performing feature fusion on the classification results, the improved DS-based evidence-based feature fusion method further includes evaluating the segmentation performance based on intersection-union ratio, recall, precision, and F1 score.

[0126] Specifically, Intersection over Union (IOU) is a concept used in object detection, which is the overlap ratio between the generated candidate bounding boxes and the original ground truth bounding boxes, i.e., the ratio of their intersection to their union; Precision is the ratio of TP to the total number of positive samples classified by the model; Recall is the ratio of TP to the total number of actual positive samples; and F1 score is a combination of precision and recall.

[0127]

[0128] Precision = TP / (TP + FP)

[0129] Recall rate = TP / (TP + FN)

[0130]

[0131] Where A is the true bounding box, B is the predicted bounding box, TP is the total number of detected data, FP is the number of false positives, and FN is the number of false negatives.

[0132] Example 1

[0133] To make the specific implementation process of the method of the present invention clearer and more explicit, this application uses the three-dimensional point cloud segmentation of a certain tunnel data for specific description.

[0134] The method model of this invention is implemented using the PyTorch framework. All experiments were conducted using a GeForce RTX 3090 GPU with 24GB of RAM, and an i7 CPU with 16GB of RAM. The learning rate was initialized to 1×10⁴. The model training optimizer used in this example is Adam.

[0135] Step 1: Collect point cloud data of the tunnel and convert it into a two-dimensional point map based on the circular projection algorithm;

[0136] This example uses the classic Leica "SiTrack:ONE" 3D laser scanning system to acquire tunnel point cloud data. The dataset contains 34,407,083 points, manually labeled into seven classes by domain experts using CloudCompare software. Furthermore, 80% of the tunnel points (approximately 27.5 million points) were used as the training dataset, and the remaining tunnel points (approximately 6.88 million points) were used as the test dataset for evaluation.

[0137] The point cloud data of the tunnel is transformed from spherical coordinate space to Cartesian coordinate space according to the first coordinate transformation matrix;

[0138] The first coordinate transformation matrix includes:

[0139]

[0140] Where x, y, and z represent the three-dimensional coordinates of each point in the point cloud data, and R, α, and β are the radius, elevation, and azimuth of the collected tunnel point cloud data, respectively.

[0141] Step S102: Sort the vectors of the point cloud data of the tunnel in the Cartesian coordinate space and group them according to the gradient interval, and divide the cross section of each small segment of the tunnel into a group.

[0142] In Cartesian coordinates, points are arranged in ascending order of y-coordinate. Then, the sorted vectors are grouped according to their y-gradient intervals. Alternatively, the sorting can be done by either x or z, followed by grouping based on their gradient intervals. In other words, the cross-sections of each small segment in the tunnel data are grouped together.

[0143] Step S103: Determine the cross-sectional center of the cross section according to the least squares circle fitting algorithm, and perform projection transformation on the grouped point cloud data according to the cross-sectional center and the second coordinate transformation matrix to obtain a two-dimensional point map.

[0144] The process of determining the cross-sectional center based on the least squares circle fitting algorithm, and then performing a projection transformation on the grouped point cloud data based on the cross-sectional center and the second coordinate transformation matrix to obtain a two-dimensional point map, includes: determining the cross-sectional center (x) based on the least squares circle fitting algorithm. c ,z c Based on the cross-section center and the second coordinate transformation matrix, the point cloud data is transformed from Cartesian coordinate space to polar coordinates, and the coordinates of the point cloud data in polar coordinates are determined.

[0145] The second coordinate transformation matrix transforms the point cloud from Cartesian space coordinates to polar coordinates. The second coordinate transformation matrix includes:

[0146]

[0147] Where, x p and z p These are the coordinates of each point on each cross section, with the center coordinates of the cross section being (x...). c , z c In the output two-dimensional point map of the projection, each pixel of the point map represents the intensity of the corresponding point cloud.

[0148] By reducing the dimensionality of a 3D point cloud to 2D, features can be extracted using a 2D CNN method.

[0149] Step 2: Extract global features of point cloud data based on 2D point map and 2D Unet model, and extract local features of point cloud data based on 2D point map and edge convolution.

[0150] Two-dimensional point plots reduce the dimensionality of the data, allowing two-dimensional CNNs to extract features. (See reference...) Figure 3 As shown, by introducing the classic 2D Unet model, global features of large-scale point clouds, especially seepage features, are effectively extracted. However, this method inevitably leads to information loss. Sometimes, grouping points of different categories into the same group requires introducing pattern scores to select the output category among several different point categories, which leads to segmentation errors, especially in seepage edge regions. To further enhance the segmentation results in edge regions, we employ an effective local feature extraction method.

[0151] To extract local features, we randomly select an N×N rectangular region from the 2D point cloud image and group all points within this region into groups X = {x1, ..., x2}. n The symbol} represents a point. Each point contains a four-dimensional value, namely [x, y, z, I]. In the feature space of each point, a classic KNN algorithm is proposed to obtain the k nearest neighbors for each point. The k points are arranged in ascending order of distance, denoted as . k-edge vector point x i Source We then introduce a convolutional layer with a kernel size of 1×1 to extract features from the k-edge vectors, as shown in the following equation:

[0152]

[0153] Where F i This is the extracted i-th feature. Finally, max pooling is applied to combine the above k features into one feature, as shown in the following equation:

[0154]

[0155] Where output i For x i Output edge features.

[0156] Point cloud segmentation local feature extraction architecture reference Figure 4 As shown. The input to the first LFE is point cloud data. In the deep neural network architecture, each subsequent LFE layer operates on the output of the previous LFE layer. Since the point density varies at different locations in the 2D point map, a random sampling strategy is introduced to extract a certain number of points N from a random rectangular region. in In other words, if the number of points in the extraction region is greater than N... in Then, these points are randomly downsampled, and N is selected. in Points are used as input to the partitioning architecture. On the other hand, if the number of points is less than N... in Then repeat the point sequence until the number reaches N. in To this end, for each point cloud dataset of shape N×4096, reducing the constant difference between point cloud data is a necessary condition for accelerating the training process. The mean vector of the points is calculated, and then each point cloud array is subtracted from its mean vector to remove distribution bias, thus becoming the input to the neural network architecture.

[0157] For the LFE method, the LFE kernel size (KS) and point sample number (PSN) are key factors affecting the performance of the integration framework. The segmentation results of our method under different hyperparameters are shown in Table 1.

[0158] Table 1. Influence of different hyperparameter models on tunnel point clouds

[0159]

[0160]

[0161] Combining the kernel size (KS) score and sample number (PSN) in Table 1, we selected two parameters for several different groups and compared their performance. Based on the results, the selected parameters were KS = 20 and PSN = 4096. The results for different hyperparameters also show that the larger the PSN, the higher the KS, and the higher the score. However, when KS is greater than 20 and PSN equals 4096, the segmentation performance degrades. This is because a larger KS results in a larger receptive field, which may ignore the details of local features during convolution. We retained KS = 20 and PSN = 4096 for subsequent experiments and used an ensemble model to segment the tunneled point cloud.

[0162] Step 3: Perform data augmentation on the point cloud data of the tunnel based on a causal reasoning model;

[0163] The causal model of this application embodiment is as follows (refer to...) Figure 5 As shown. x f This represents effective features for point cloud types, such as percolation types. And x b It may be related to background elements, such as pipe segments. For segmentation, Y represents point cloud classification. Based on the causal model, the neural network model extracts global and local point features to classify tunnel points, meaning that seepage points and seepage points in the adjacent background are inevitably perceived simultaneously. Therefore, x b and x f Both can affect the output of the segmentation model. The spurious correlation between percolation classes and segmentation classes can confuse the neural network model and affect feature extraction.

[0164] Because x f The P→x expression contains enough information to describe Y, therefore it is necessary to remove the confounding P→x by introducing a diffractometer. b ,like Figure 4 As shown. By breaking x b and x f The spurious correlation between them is mitigated. The transfer correlation effect from the training phase to the testing phase is alleviated. This example proposes a nonlinear converter to enhance point cloud data as a dry copier. The dry copier is shown in the following equation:

[0165] Φ(x)=Φ I (x)+Φ P (x)

[0166] Where, Φ I (x) and Φ P (x) represents two nonlinear converters. Φ I (x) is a shallow convolutional network consisting of two convolutional layers and a Leaky ReLU layer, used to generate various transformation functions for the intensity at each point. Φ P(x) is a random vector applied to [x, y, z] to randomly change the position of each point. In each iteration, the training points of the segmented class are assigned different positions and intensities. This overcomes the erroneous connections between the percolation class and the segmented class, improving the segmentation performance of the percolation class.

[0167] The tunnel point cloud data are classified based on both global and local feature data enhancements. The classification results are then fused using an improved DS-based evidence-based feature fusion method.

[0168] In this example, for each segmentation result i of category k, we define a trust factor Crd. i，k Evidence E i，k From evidence E i，k The Euclidean distance E to the evidence set is defined as follows:

[0169]

[0170] Where d represents two pieces of evidence E i，k and E j，k The Jousselme distance between them.

[0171] S k It is the measure of evidence E i，k An indicator of the degree of conflict, reflecting the degree of difference between two pieces of evidence. When S k When it is smaller, it means E i,k It is more consistent with other evidence and has a lower degree of conflict. Conversely, when S k When it is large, it means E i，k This evidence differs significantly from other evidence, and is highly divergent and conflicting. Evidence E... i，k Crd i，k Subsequently, further measures of support for this evidence were obtained from other sources. These support measures were used as the weight ω for this evidence. i，k The fusion result for each class is given by the following formula.

[0172] Crd max =max(Crd1,Crd2, ...,Crd n )

[0173]

[0174] Among them, Out k This is the comprehensive result of segmenting the k-th type of point cloud.

[0175] To verify the effectiveness of the designed method, different segmentation modules were tested on the test dataset to express quantitative results, as shown in Table 2. Under the proposed projection framework, data augmentation based on causal inference, and the Dempster-Shafer (DS) fusion method, the proposed ensemble model achieved the best results in terms of Intersection over Union (IOU), Recall (R), Precision (P), and F1 score, especially in the seepage class. That is, the projection framework and data augmentation based on causal inference contribute to the score improvement, by 0.101 and 0.01 respectively, further validating the superiority of the developed ensemble projection method. DS-feature fusion improved the seepage F1 score from 0.817 to 0.824, demonstrating its effectiveness. Experimental results show that using a suitable fusion model to fuse global point features with local features is an effective method to improve tunnel segmentation performance.

[0176] Table 2. Calculation results of the tunnel point cloud model.

[0177]

[0178] The segmentation results and model evaluation of this invention specifically include:

[0179] To better evaluate and compare the proposed method, supervised learning pixel-level segmentation was performed using 20% ​​of the tunnel point cloud to validate and improve its performance. This study employed five state-of-the-art (SOTA) point-based segmentation architectures: DGCNN, PointNet, Pointnet++, PointMLP, and PCT. Through comparison of AP scores, the DGCNN-based model was found to perform best in detecting objects such as seepage, meeting our expectations. In the categories of "Cable," "Segment," "Pipe," "PowerTrack," "Seepage," "Support," and "Track," the DGCNN-based projection method achieved AP scores of 0.9318, 0.8919, 0.9968, 0.954, 0.7362, 0.7718, and 0.9814, respectively. Therefore, the DGCNN backbone was applied in the following experiments. The PR curves for different categories show that most categories can be easily classified using point classification methods. Five out of the seven classes achieved AP scores exceeding 0.9. Seepage is the most difficult class to classify, and the best AP score for seepage segmentation is less than 0.75. Therefore, this paper proposes a module focused on improving seepage segmentation performance, and the performance of the module is verified in subsequent experiments.

[0180] To further demonstrate the superiority of the proposed integrated method, this example combines a five-point segmentation baseline with a projection method to segment tunnel data. The combined method is trained and tested on the same data configuration. Table 3 shows that the seepage segmentation results demonstrate that for each baseline, the proposed method outperforms the original method. This is because the point-based method excels at extracting local features, while the projection method focuses on global features. This method combines the two types of features to extract comprehensive features of the seepage class, achieving better scores in seepage segmentation. Table 3 also shows that the DGCNN-based method achieves the best IOU and F1 scores, consistent with the results in Table 1, reflecting the effectiveness of the DGCNN baseline for tunnel data segmentation.

[0181] Table 3. Results of seepage segmentation models for different trunks.

[0182]

[0183] This example combines the proposed ensemble method with a data augmentation model and a feature fusion model, using the Python-based Open3D library to generate segmentation results for a test point cloud. A visualization of the segmentation results is provided. Figure 6 As shown. Compared with the RawDGCNN method, this further verifies that the integral model proposed in this paper correctly segments most seepage points. Correct seepage pixels and correct other classes are marked in light blue and dark blue, while seepage points misclassified as other classes are marked in orange. Non-seepage points misclassified as seepage are marked in green. Figure 6 (The images are represented by varying shades of gray). Visualization results show that most of the error stems from incorrectly classifying non-seepage classes as seepage classes. This is because segment points and seepage points are very similar within a small area, and the point-based model is confused with local features. Compared to the original method, the number of green pixels is significantly reduced, further validating the superiority of the proposed method in this example. By combining local and global features using a projection-based framework, segment and seepage classes can be correctly classified over a large area. The accuracy and F1 score of the segmentation results are significantly improved.

[0184] The implementation of the various embodiments of the present invention is based on programmed processing through a device with a central processing unit. Therefore, in practical engineering, the technical solutions and functions of the various embodiments of the present invention can be encapsulated into various modules. Based on this reality, and building upon the above embodiments, the embodiments of the present invention provide a multi-target segmentation and integration system for tunnel point clouds based on causal reasoning, such as... Figure 7 As shown, the method for multi-target segmentation and integration of tunnel point clouds based on causal reasoning in the above method embodiments includes:

[0185] The first main module is used to collect point cloud data of the tunnel and convert it into a two-dimensional point map based on the circular projection algorithm;

[0186] The second main module is used to extract global features of point cloud data based on two-dimensional point maps and two-dimensional Unet models, and to extract local features of point cloud data based on two-dimensional point maps and edge convolution.

[0187] The third main module is used to perform data augmentation on the point cloud data of the tunnel based on the causal reasoning model;

[0188] The fourth main module is used to classify the tunnel point cloud data based on the global and local feature data enhancements, respectively, and to perform feature fusion on the classification results based on the improved DS evidence-based feature fusion method.

[0189] It should be noted that the apparatus in the device embodiments provided by the present invention can be used not only to implement the methods in the above method embodiments, but also to implement the methods in other method embodiments provided by the present invention. The only difference is that corresponding functional modules are set. The principle is basically the same as that of the above device embodiments provided by the present invention. As long as those skilled in the art can improve the apparatus in the above device embodiments by referring to the specific technical solutions in other method embodiments and combining technical features to obtain corresponding technical means and technical solutions composed of these technical means, on the basis of the above device embodiments, under the premise of ensuring the practicality of the technical solutions, so as to obtain corresponding device-type embodiments for implementing the methods in other method-type embodiments.

[0190] The methods in the embodiments of the present invention are implemented using electronic devices; therefore, it is necessary to describe the relevant electronic devices. For this purpose, embodiments of the present invention provide an electronic device comprising: at least one central processor, a communications interface, at least one memory, and a communication bus, wherein the at least one central processor, the communications interface, and the at least one memory communicate with each other via the communication bus. The at least one central processor can invoke logical instructions stored in the at least one memory to execute all or part of the steps of the methods provided in the foregoing method embodiments.

[0191] Furthermore, when the logical instructions in at least one of the aforementioned memories can be implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various method embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0192] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0193] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0194] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Based on this understanding, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or sometimes in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0195] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A tunnel point cloud multi-object segmentation ensemble method based on causal reasoning, characterized in that, The method comprises the following steps: Collecting point cloud data of a tunnel and converting the point cloud data into a two-dimensional point graph based on a circular projection algorithm; Extracting global features of the point cloud data based on the two-dimensional point graph and a two-dimensional Unet model, and extracting local features of the point cloud data based on the two-dimensional point graph and edge convolution; Performing data enhancement on the point cloud data of the tunnel based on a causal inference model; Classifying the point cloud data of the tunnel after data enhancement according to the global features and the local features respectively, and performing feature fusion on the classification results based on an improved D-S evidence-based feature fusion method; The data enhancement on the point cloud data of the tunnel based on the causal inference model comprises the following steps: Constructing a causal structure model; Based on the causal structure model and the point cloud data of the tunnel, a causal structure graph corresponding to the point cloud is constructed; A nonlinear converter is constructed as an intervention operator, and the point cloud data of the tunnel is enhanced based on the point cloud data of the tunnel, the intervention operator and the causal structure graph; The feature fusion on the classification results based on the improved D-S evidence-based feature fusion method comprises the following steps: Determine the evidence of each classification result, and aggregate the evidence into an evidence set; Determine the evidence conflict degree according to the evidence and the evidence set, determine the evidence trust factor according to the evidence conflict degree, and determine the evidence weight according to the evidence trust factor; Perform feature fusion according to the evidence and the evidence weight.

2. The integrated method of tunnel point cloud multi-object segmentation based on causal reasoning according to claim 1, characterized in that, The collection of the point cloud data of the tunnel and the conversion of the point cloud data into a two-dimensional point graph based on the circular projection algorithm comprises the following steps: Transform the point cloud data of the tunnel from a spherical coordinate space to a Cartesian coordinate space according to a first coordinate transformation matrix; Sort the vectors of the point cloud data of the tunnel in the Cartesian coordinate space and group them according to the gradient interval, and divide the cross section of each small section of the tunnel into a group; Determine the cross section center according to the least square circle fitting algorithm, and perform projection transformation on the grouped point cloud data according to the cross section center and a second coordinate transformation matrix to obtain a two-dimensional point graph.

3. The integrated method of tunnel point cloud multi-object segmentation based on causal reasoning according to claim 2, characterized in that, The first coordinate transformation matrix comprises: Wherein, x, y, z represent the three-dimensional coordinates of each point in the point cloud data, R, a, β are the radius, elevation and azimuth of the collected point cloud data of the tunnel respectively.

4. The integrated method of tunnel point cloud multi-object segmentation based on causal reasoning according to claim 2, characterized in that, The determination of the cross section center according to the least square circle fitting algorithm and the projection transformation of the grouped point cloud data according to the cross section center and the second coordinate transformation matrix to obtain a two-dimensional point graph comprises the following steps: determining the cross-sectional center (x c , z c ) based on a least square circle fitting algorithm; Convert the point cloud data from the Cartesian coordinate space to the polar coordinate based on the cross section center and the second coordinate transformation matrix, and determine the coordinates of the point cloud data in the polar coordinate.

5. The integrated method of tunnel point cloud multi-object segmentation based on causal reasoning according to claim 1, characterized in that, The extraction of the local features of the point cloud based on the two-dimensional point graph and the edge convolution comprises the following steps: Select a rectangular region with a size of N×N from the two-dimensional point graph, and group the points in the rectangular region; Obtain k-edge vectors by obtaining k nearest points of each point in the rectangular region based on KNN algorithm; Extract features from the k-edge vectors based on a 1×1 convolution layer to obtain k features; Integrate the k features into one feature based on the maximum pooling operation to obtain the local features.

6. The integrated method of tunnel point cloud multi-object segmentation based on causal reasoning according to claim 1, characterized in that, The intervention operator comprises: Φ(x) = Φ I (x) + Φ P (x) where Φ I (x) and Φ P (x) are two nonlinear transformers, Φ I (x) is a shallow convolutional network including two convolutional layers and a Leaky ReLU layer to produce various transformation functions for the intensity of each point; Φ P (x) is a random vector applied to the Cartesian coordinates.

7. The integrated method of tunnel point cloud multi-object segmentation based on causal reasoning according to claim 1, characterized in that, The feature fusion on the classification results based on the improved D-S evidence-based feature fusion method further comprises: evaluating the classification results after feature fusion according to the intersection over union, recall rate, precision and F1 score.

8. A causal-reasoning-based tunnel point cloud multi-object segmentation integrated system, characterized in that, The method comprises the following steps: The first main module is used for collecting point cloud data of the tunnel and converting the point cloud data into a two-dimensional point graph based on a circle projection algorithm; The second main module is used for extracting global features of the point cloud data based on the two-dimensional point graph and a two-dimensional Unet model, and extracting local features of the point cloud data based on the two-dimensional point graph and edge convolution; The third main module is used for performing data enhancement on the point cloud data of the tunnel based on a causal inference model; The data enhancement on the point cloud data of the tunnel based on the causal inference model comprises: constructing a causal structure model; constructing a causal structure graph corresponding to the point cloud based on the causal structure model and the point cloud data of the tunnel; constructing a nonlinear converter as an intervention operator, and enhancing the point cloud data of the tunnel based on the point cloud data of the tunnel, the intervention operator and the causal structure graph; The fourth main module is used for classifying the point cloud data of the tunnel after data enhancement according to the global features and the local features respectively, and performing feature fusion on the classification results based on an improved D-S evidence-based feature fusion method; the feature fusion on the classification results based on the improved D-S evidence-based feature fusion method comprises: determining evidences of each classification result, and collecting the evidences into an evidence set; determining an evidence conflict degree according to the evidence and the evidence set, determining an evidence trust factor according to the evidence conflict degree, and determining an evidence weight according to the evidence trust factor; performing feature fusion according to the evidence and the evidence weight.

Images