Point cloud filtering method for large-scale scene data

Abstract

The invention relates to the technical field of three-dimensional image calculation processing, in particular to a large-scale scene data-oriented point cloud filtering method, which comprises the following steps of: constructing a point cloud filtering network SC-CNN on the basis of a multi-layer perceptron (MLP) network Point Net; a gradient constrained farthest point sampling module is used for carrying out adaptive subregion division, gradient pre-classification and subregion sample sampling on an original point cloud, and a Copula correlation and attention mechanism intra-class point feature enhancement module is combined to take a ground point and a non-ground point in the point cloud as variables; the Copula model is used for carrying out quantitative analysis on the correlation of variable data to enhance the capability of learning classification features from sparse point clouds, reduce classification errors to the greatest extent and improve the overall filtering precision; and finally, combining the intra-class points verified by Copula with the calibrated ambiguous points by combining a Copula correlation and confidence coefficient estimation classification error correction module to obtain a filtering result.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional image computing and processing, and in particular to a point cloud filtering method for large-scale scene data. Background Art

[0002] Due to the diversity of terrain, the complexity of attached objects, and the non-uniform and disordered characteristics of point clouds themselves, the development of large-scale point cloud filtering technology suitable for various terrain conditions is still a very challenging task. Point cloud filtering is essentially a binary classification problem. In recent years, deep learning represented by convolutional neural networks (CNN) has shown excellent capabilities in implicit model expression, high-dimensional feature description and extraction. Designing high-precision and robust point cloud filtering CNN has become a current research hotspot and can be roughly divided into two categories: point cloud filtering based on optimization and point cloud filtering based on deep learning.

[0003] Starting from the laser foot point information (generally elevation), the optimized point cloud filtering extracts the local geometric features of the point cloud and determines the discrimination rules and establishes the search strategy to separate the ground points and non-ground points. Representative methods include slope filtering, morphological filtering, surface fitting filtering, etc. The basic idea of ​​slope filtering is to calculate the slope value between each point and its neighboring points, and regard the points with a slope less than the set threshold as ground points. The basic principle of morphological filtering is to filter out the points with large elevation changes in the opening operation as non-ground points. This type of algorithm has good performance in data sets with flat terrain. The basic idea of ​​surface fitting filtering is to first establish an initial terrain surface model such as a triangulated network through interpolation, and then calculate the geometric error between the point to be filtered and the surface model. Points less than the given error threshold will be regarded as ground points and used to optimize the initial surface model. Although the above methods can achieve good results for specific terrains, they rely on the correct design and selection of feature vectors. Existing point cloud filtering methods based on deep learning can be divided into indirect point cloud filtering and direct point cloud filtering. The former uses mature image target classification and semantic segmentation CNN learning to obtain point cloud spatial features (relationships) for filtering purposes by performing 2D / 3D grid structured preprocessing on the original point cloud. However, when the 3D point cloud is projected along a certain line of sight to generate a 2D view, the intrinsic spatial geometric relationship of the object will be destroyed and the depth information will be lost. The filtering result is not ideal and is extremely sensitive to occlusion and projection perspective. Direct point cloud filtering is to apply convolution operations directly to the original point cloud data. The premise is to solve the problems of point cloud disorder and permutation invariance. This pioneering work was implemented by Qi et al. in the networks PointNet and PointNet++. Compared with indirect point cloud filtering, the independent operation of direct point cloud filtering on each point undoubtedly makes the point cloud processing process more flexible and the information utilization more fully. However, the overly independent operation of each point is not conducive to capturing the local object structure well, the contextual terrain information and the learning of large-scale terrain spatial relationships, thereby limiting the generalization ability of complex scenes.

[0004] However, the direct point cloud filtering networks based on PointNet and PointNet++ currently generally use the farthest point sampling (FPS) to solve the problem of large-scale 3D point cloud data network implementation calculation, but FPS is an "indifferent" point cloud sampling method. When there is irregular undulating terrain in the scene, it is difficult for KNN to capture the neighborhood (point) information of the same category from the training samples extracted by FPS. In addition, for the two types of abstract objects, ground points and non-ground points, the semantic clues to distinguish the two are relatively vague, especially when the terrain has complex spatial characteristics such as slope slopes, mountain faults, ups and downs, and near-ground objects, it is impossible to strengthen the consistent expression of intra-class features and highlight the differences in inter-class features. Summary of the invention

[0005] The purpose of the present invention is to provide a point cloud filtering method for large-scale scene data, aiming to ensure that sample points under different terrain conditions can be effectively learned by the network, and at the same time use the Copula model to quantitatively analyze the correlation of variable data to enhance the network's ability to learn classification features from sparse point clouds and minimize classification errors, thereby improving the overall filtering accuracy.

[0006] To achieve the above object, the present invention provides a point cloud filtering method for large-scale scene data, comprising the following steps:

[0007] Step 1: Based on the multi-layer perceptron MLP network PointNet, build a point cloud filtering network SC-CNN;

[0008] Step 2: Set the farthest point sampling module of slope constraint;

[0009] Step 3: Intra-class point feature enhancement module combining Copula correlation and attention mechanism;

[0010] Step 4: Classification error correction module combining Copula correlation and confidence estimation.

[0011] Optionally, the point cloud filtering network SC-CNN includes a farthest point sampling module, a downsampling layer, an upsampling layer, a fully connected layer and a classification error correction module, and the downsampling layer is implanted with an in-class point feature enhancement module.

[0012] Optionally, the execution process of step 2 includes the following steps:

[0013] Step 2.1: Adaptive sub-region division;

[0014] Step 2.2: Neighborhood compensation sampling;

[0015] Step 2.3: Slope pre-classification.

[0016] Optionally, in the process of adaptive sub-region division, the calculation formula of the sub-region side length L corresponding to the number of samples n is as follows:

[0017] L=[√(△x·△y / [N / n])]

[0018] Wherein, [·] represents the rounding symbol, [N / n] represents the number of sub-regions, △x and △y are the total horizontal areas of the scene, which are obtained by dividing the total area by the number of sub-regions and taking the square root;

[0019] Then, L / 2 is used as the step size to divide the point cloud into overlapping sub-regions.

[0020] Optionally, in step 2, the calculated side length L of the sub-region is increased by 1.2 times. If the number of points still cannot meet the requirement, the magnification factor is increased by 0.1 until the number of point clouds covered by the sub-region meets the sampling requirement.

[0021] Optional, slope angle θ in slope pre-classification i , the calculation formula is as follows:

[0022] θ i =arctan(|△z i | / S i ),S i =√[(x i -x min )2+(y i -y min )2]

[0023] Among them, △z i is u i Dot and u min The elevation difference of the point, S i is u i Dot and u min The horizontal straight-line distance of the point, reference point u min The point is the point with the smallest z value, which is taken from the point cloud U in each sub-area n ={u1,u2,…u i}z coordinate value.

[0024] Optionally, the expression of the Copula correlation and the intra-class point feature enhancement module of the attention mechanism in step 3 is as follows:

[0025]

[0026] Among them, τ∈(0,1) is the model parameter of Copula. When τ→1, C(·)→1, and all points have a high correlation; when τ→0, C(·)→0, and all points tend to be independent.

[0027] Optionally, in step 4, the classification error correction module combining Copula correlation and confidence estimation analyzes and corrects the classification results output by the fully connected layer. Specifically, the category score of the ambiguous point is corrected according to the confidence interval and the calibration parameter, and the in-class points verified by Copula are merged with the calibrated ambiguous points to obtain the filtering results of ground points and non-ground points.

[0028] The invention provides a point cloud filtering method for large-scale scene data. First, the farthest point sampling module constrained by slope is used to perform adaptive sub-region division, slope pre-classification and sub-region sample sampling on the original point cloud. Then, the feature encoding module is used in the downsampling layer to extract features of each training sample. The extracted neighborhood features of each sampling point are sent to the Copula correlation judgment module for in-class point analysis, and then the in-class point features are enhanced by the attention mechanism and spliced ​​with the non-in-class point features. At the same time, the upper sampling layer network uses 3D-interpolation and jump connection to achieve bottom-level-high-level feature fusion. The network outputs the optimal classification model by iterative training, and uses the fully connected layer to classify the target point cloud point by point. The classification results are input into the MLP layer according to the category for feature extraction, and the extracted features are used for Copula correlation analysis. The features corresponding to the positively correlated points are used to predict calibration parameters and solve the confidence interval. On the other hand, the scores of the categories to which the ambiguous points belong are corrected according to the confidence interval and the calibration parameters. Finally, the in-class points verified by Copula and the calibrated ambiguous points are merged to obtain the filtering results of ground points and non-ground points. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 It is a schematic diagram of the framework structure of the point cloud filtering network SC-CNN of the present invention.

[0031] Figure 2 It is a flow chart of a point cloud filtering method for large-scale scene data of the present invention.

[0032] Figure 3 A schematic diagram of the structure of the farthest point sampling module for slope constraint in the present invention.

[0033] Figure 4 It is a schematic diagram of the adaptive sub-region division process of the present invention.

[0034] Figure 5It is a schematic diagram of a local area of ​​the samp11 scene in the ISPRS standard point cloud filtering data set according to an embodiment of the present invention.

[0035] Figure 6 It is a structural diagram of the intra-class feature enhancement module under the attention mechanism of the present invention.

[0036] Figure 7 It is a structural schematic diagram of a verification module combining confidence and Copula correlation of the present invention.

[0037] Figure 8 It is a schematic diagram of the filtering effect of each module in three random scenes in the ablation experiment of an embodiment of the present invention.

[0038] Fig. 9 It is a schematic diagram showing the results of each module T.I and T.II in each scenario of the ablation experiment in an embodiment of the present invention.

[0039] Fig.10 Schematic diagram of the ISPRS Vaihingen 3D data set according to Example 2 of the present invention.

[0040] Fig.11 It is a schematic diagram of the filtering effects of different methods in Example 2 of the present invention.

[0041] Fig.12 It is a schematic diagram showing the results of methods T.I and T.II in Example 2 of the present invention. DETAILED DESCRIPTION

[0042] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0043] The English abbreviations in the present invention are explained as follows:

[0044] SC-CNN: LiDARpoint cloud filtering CNN under slope and Copulacorrelation constraint;

[0045] FPS: farthestpoint sampling.

[0046] The present invention provides a point cloud filtering method for large-scale scene data, comprising the following steps:

[0047] S1: Based on the multi-layer perceptron MLP network PointNet, a point cloud filtering network SC-CNN is constructed;

[0048] S2: The farthest point sampling module for setting slope constraints;

[0049] S3: Intra-class point feature enhancement module combining Copula correlation and attention mechanism;

[0050] S4: Classification error correction module combining Copula correlation and confidence estimation.

[0051] The point cloud filtering network SC-CNN includes a farthest point sampling module, a downsampling layer, an upsampling layer, a fully connected layer, and a classification error correction module. The downsampling layer is implanted with an intra-class point feature enhancement module. The network structure is as follows Figure 1 shown.

[0052] Specifically, the overall process of the present invention is as follows Figure 2 As shown in the figure, the farthest point sampling module with slope constraint is first used to perform adaptive sub-region division, slope pre-classification and sub-region sample sampling on the original point cloud. Then, the feature encoding module is used in the downsampling layer to extract features from each training sample. The extracted neighborhood features of each sampling point are sent to the Copula correlation judgment module for in-class point analysis, and then the in-class point features are enhanced and spliced ​​with the non-in-class point features using the attention mechanism. At the same time, the network in the upsampling layer uses 3D-interpolation and jump connection to achieve bottom-to-high-level feature fusion. The network outputs the optimal classification model through iterative training, and uses the fully connected layer to classify the target point cloud point by point. The classification results are input into the MLP layer according to category for feature extraction, and the extracted features are used for Copula correlation analysis. The features corresponding to the positively correlated points are used to predict the calibration parameters and solve the confidence interval. On the other hand, the scores of the categories to which the ambiguous points belong are corrected according to the confidence interval and the calibration parameters. Finally, the in-class points verified by Copula and the calibrated ambiguous points are merged to obtain the filtering results of ground points and non-ground points.

[0053] The following is an auxiliary explanation of the important parts of the execution steps:

[0054] 1. Slope-constrained farthest point sampling module

[0055] The farthest point sampling module designed in the present invention for slope constraint includes three parts: adaptive sub-region division, neighborhood compensation sampling and slope pre-classification. The module structure is as follows: Figure 3 shown.

[0056] When existing 3D point cloud feature extraction networks process large-scale point cloud data, they usually set the side length and step size of the sub-region according to historical experience. This processing method not only makes it impossible to fully extract the information in dense areas, but also causes the problem of insufficient points in sparse areas. Ultimately, it leads to the inability to fully extract features in dense areas and the duplication and redundancy of single features in sparse areas. In order to solve this problem, the present invention proposes an adaptive sub-region division method, the specific process of which is as follows: Figure 4 shown.

[0057] Figure 4 is a bird's-eye view of a scene. The algorithm first reads the coordinates of all points in the scene and extracts the maximum and minimum values ​​of the x and y coordinates. The side lengths △x and △y of the scene are calculated based on the maximum and minimum coordinates, such as Figure 4 (a) The total number of point clouds N in the scene is a fixed value, while the number of sampling points n in the sub-area can be freely set by the user (usually 2 n ,n∈[10,14]). The side length of the sub-region corresponding to the number of samples n can be calculated according to the following formula:

[0058] L=[√(△x·△y / [N / n])] (1)

[0059] In the formula, [·] represents the rounding symbol, and [N / n] represents the number of sub-regions. △x and △y are the total horizontal area of ​​the scene. The total area is divided by the number of sub-regions and then the square root is taken to get the side length L of the sub-region, as follows: Figure 4 (b) As shown. From a physical perspective, a partitioning strategy with a step length equal to the side length of a sub-region will destroy the geometric structural integrity of the scene to a certain extent. Therefore, the present invention uses L / 2 as the step length to perform overlapping sub-region partitioning of the point cloud, as shown in Figure 4 (c) This division method ensures that the same point cloud structure exists between adjacent sub-regions, thus preventing the block operation from destroying the overall structure of the original scene.

[0060] Although adaptive sub-region division can effectively alleviate the problem of uneven point cloud density distribution, the problem that the number of point clouds in sparsely distributed areas cannot meet the number of sub-region sampling has not been solved. At the same time, the number of point clouds contained in some sub-regions may not be enough to meet the sampling requirements, such as Figure 4(c) The sub-region in the first row and first column (the upper left sub-region) and the sub-region in the first row and tenth column (the upper right sub-region). The existing direct point cloud feature extraction network has two solutions to the above problems: 1. When the number of points in the sub-region is far less than the sampling point requirement, the divided region and the point cloud contained therein are directly discarded; 2. When the number of points in the sub-region is less than (not far less than) the sampling point requirement, the point cloud in the sub-region is randomly copied until the number of sampling points is met. Among them, method 1 will cause serious loss of scene edge information and minority class sample information, and method 2 will lead to overfitting of target feature learning. In order to solve the above problems, the present invention proposes a neighborhood compensation sampling method within the sub-region. The calculated sub-region side length L is increased by 1.2 times (if the number of points still cannot meet the requirement, the magnification is increased by 0.1 until the number of point clouds covered by the sub-region meets the sampling requirement), so that the sub-region with insufficient points can cover more point clouds. When the sub-region is expanded to cover the adjacent sub-region, the point cloud of the current sub-region is fully sampled, and the insufficient points are obtained from the adjacent sub-region according to the FPS. Neighborhood compensation sampling can retain the spatial structure of the original point cloud to the maximum extent and ensure the diversity of samples within the divided area. When the point cloud is located at the edge of the divided area, it will be able to effectively avoid the serious loss of sample information, thereby promoting the model to fully learn the features of different categories, thereby improving the overall model training accuracy.

[0061] Although adaptive sub-region division and neighborhood compensation sampling try to keep the number of points of training samples consistent with that of original data, the problem of irregular distribution of data itself still exists. Existing point cloud downsampling methods mainly include FPS, random sampling, uniform cloud sampling and geometric sampling. The downsampling method represented by FPS is a sampling method that continuously and iteratively selects the farthest point from the existing sampling point set. This method can better obtain the outer contour structure of the target when processing regular, uniform and continuous data. However, when the data set contains complex characteristics such as the same object at different heights, different targets stacked and aggregated in local areas, and discontinuously distributed faults, FPS can only collect the overall contour points in the sub-region and cannot sample the outer contours of similar points in the sub-region. At this time, the point cloud sampled by FPS is difficult to provide the network with rich ground point or non-ground point information, which in turn affects the filtering accuracy of the network. The literature (Rashidi P et al., 2018) points out that the point cloud slope filtering algorithm has the characteristics of simple principle, easy implementation, relatively stable and applicable to different scenarios. At the same time, the literature (WanP et al., 2018) indicates that slope information can be used as an important parameter of other algorithms. Combining the slope filtering algorithm with other methods is conducive to improving the filtering accuracy. In order to enable the network to fully learn the local geometric information of similar points, the present invention is inspired by the above and designs a slope pre-classification module to pre-classify and assign categories to the points in each sub-area. First, the slope filtering algorithm is used to roughly classify and label the ground points and non-ground points in the sub-area, and then the FPS is used to perform intra-class downsampling on the labeled similar point cloud. The final training sample consists of a large number of intra-class points (target main body information) and a small number of inter-class points (target edge information). The slope angle calculation of the slope pre-classification module and the slope classification algorithm process are as follows: Figure 5 shown.

[0062] Figure 5 It is a local area of ​​the samp11 scene in the ISPRS standard point cloud filtering dataset. The red and blue point clouds are non-ground points and ground points respectively. The algorithm flow of slope sampling is as follows: First, read the point cloud U in each sub-area n ={u1,u2,…u i}, take the point with the smallest z value as the reference point u min , the remaining n-1 points are related to u min Calculate the slope angle θ i , the calculation formula is as follows:

[0063] θ i =arctan(|△z i | / S i ),S i =√[(x i -x min )2+(yi -y min )twenty two)

[0064] Where △z i is u i Dot and u min The elevation difference of the point, S i is u i Dot and u min The horizontal straight line distance of the point can be calculated by using the inverse trigonometric function arctan to obtain the slope angle θ i , the mathematical meaning of formula (2) is as follows Figure 5 As shown in the dashed triangle on the left. Calculate the slope angle θ between each point and the minimum point i Then, the slope angle threshold θ in the sub-area is calculated by distance weighted average, and the calculation formula is as follows:

[0065] θ=∑ i＝1 (θ i ·S i / S),S=∑ i＝1 S i ,i=(1,2,..,n-1) (3)

[0066] Where S is the sum of the horizontal distances between n-1 points and the minimum point, S i / S is the slope angle θ i Corresponding distance weight. The n-1 slope angles are judged with the slope angle threshold θ calculated by formula (3). If it is greater than θ, it is classified as a non-ground point and assigned a value of 1. If it is less than θ, it is classified as a ground point and assigned a value of 0. On the basis of the slope pre-classification, the training data in the sub-area are sampled from similar points by FPS. The present invention uses the distance weighted average calculation method instead of the arithmetic average calculation method. This is because when there are multiple points in the sub-area with the same elevation distance and attribute characteristics but different slope angles (each point has a different horizontal distance from the minimum point), the slope angle threshold obtained by the arithmetic mean easily divides the points with close horizontal distances and similar points with long horizontal distances into two categories, and the misclassification problem caused by distance differences can be effectively alleviated by distance weighting.

[0067] 2. Intra-class point feature enhancement combining Copula correlation and attention mechanism

[0068] Although the slope-constrained farthest point sampling module can promote the possibility of fully learning similar points in the sub-region, the problem of insufficient number of point clouds in sparsely distributed areas making it difficult to effectively extract features still exists. The weight-shared convolutional layer will increase the computational cost and fail to highlight the differences in features between classes, thereby limiting the network's ability to judge the edges of various types of objects. In addition, PointNet's overly single feature encoding module also makes it difficult to extract complete geometric structure features from complex scenes, thereby affecting the classification accuracy of the network. In order to enable the network model to effectively extract various features from training samples, the present invention uses an intra-class feature enhancement module under the attention mechanism to improve the network, such as Figure 6 shown.

[0069] Depend on Figure 6 It can be obtained that first the point cloud V = {h i |i=1,…,N} to perform simple PointNet feature extraction, and then use KNN to extract feature neighborhood points. Based on the characteristics of KNN, the neighborhood points with smaller feature distances in the feature space are more likely to belong to the same category as the sampling points. It can be concluded that the neighborhood points obtained through KNN in the feature space help the network learn the feature information of the category to which the sampling points belong, and at the same time increase the distinction between inter-class features. Since the feature encoding method of PointNet has limited expression capabilities for the topological structure and spatial scale of point clouds, Yew et al. introduced the Point Pair Features (PPF) encoding method based on PointNet to enhance neighborhood data relationship learning. Inspired by this document, the present invention uses the following formula to encode local features of sampling points and their neighborhood points:

[0070] δ i k =MLP(h i ⊕h i,k ⊕√(h i -h i,k ) 2 ⊕(h i -h i,k )⊕F(h i ,h i,k )) (4)

[0071] F(h i ,h i,k )=(∠(m i ,(h i -h i,k ))⊕∠(m i,k ,(h i -h i,k ))⊕∠(m i ,m i,k )) (5)

[0072] In formula (4), h i and h i,k They represent the sampling point and its neighboring points respectively. The Euclidean distance, direction vector and 4D point pair features between points are √(h i -h i,k )2、(h i -h i,k ) and F(h i ,h i,k ). 4D point pair feature F(h i ,,h i,k ) i and m i,k Represent the normal vectors of the sampling point and the neighborhood point, respectively, which are solved by the method in the literature (Deng et al., 2018). At the same time, ∠(·,·) in formula (5) represents the inverse trigonometric function, which is used to calculate the angle between the vectors. Formulas (4) and (5) not only focus on the absolute spatial coordinates of each point, but also construct the local neighborhood topological structure and spatial scale feature descriptor through the point spacing, point normals and the angle between them.

[0073] However, δ ik The feature extraction is performed through the weight-sharing MLP layer, which results in each feature being given the same weight and failing to highlight the importance of intra-class features and the differences between inter-class features. In order to further improve the classification accuracy of the network, the present invention introduces the Copula correlation analysis module to judge intra-class points and inter-class points. First, the sampling points, neighborhood points, Euclidean distances between points, and neighborhood structure information in formula (4) are extracted to construct the autocorrelation coefficient, which is defined as follows:

[0074]

[0075] Where △h i,k is the Euclidean distance between points√(h i -h i,k )2. Then, the (e i ,d i ,p i ) into the Kendall correlation coefficient (Gao J et al., 2016) calculation formula:

[0076]

[0077] In formula (7), i≠j, && represents the AND logic, || represents the OR logic, and P{(·)>0} represents the number of groups of a pair of positively correlated points. The judgment method is (e i ,p i ,d i ) is greater than (e j ,pj ,d j ); N{(·)<0} indicates how many groups of negatively correlated points there are, and the judgment method is (e i ,p i ,d i ) contains any element less than (e j ,p j ,d j ); Q{·∪·∪·} represents the combination of selecting two points from k+1 points without considering the order. Substituting τ calculated by formula (7) into the following Copula distribution function (Oh DH 2014) can construct the Copula correlation analysis module of the present invention:

[0078]

[0079] Where: τ∈(0,1) is the model parameter of Copula. When τ→1, C(·)→1, and all points have a high correlation; when τ→0, C(·)→0, and all points tend to be independent. The mathematical characteristics of formula (7) are used to judge each local neighborhood point, and the feature output of all positively correlated points is P i,k , and the negatively correlated point feature output is N i,k . Using the attention mechanism proposed in the literature (Hu Q et al., 2020) i,k Perform feature enhancement, and then use the enhanced P i,k With N i,k For splicing, the formula is as follows:

[0080]

[0081] In formula (9), c(·) is a 1×1 convolution operation. i,k The self-attention score is calculated and assigned to the aggregation to obtain a feature vector that reflects the significance of the geometric structure of the intra-class points. At the same time, in order to maintain the consistency of the feature dimension, the negatively correlated point features are retained in the present invention and spliced ​​with the enhanced intra-class point features, so as to achieve the intra-class feature enhancement effect under the attention mechanism of the present invention. Obviously, by calculating the correlation between point pair features and features, the intra-class points can be effectively found, and the features of the intra-class points can be enhanced by the attention mechanism while retaining the inter-class point features, which can improve the difference of the inter-class features to a certain extent, thereby improving the overall network's ability to learn features and classify targets.

[0082] 3. Classification error correction combining Copula correlation and confidence estimation

[0083] Although the present invention optimizes and improves the feature encoding method in the downsampling stage through the above work, the layer-by-layer downsampling requires the use of pooling operations to achieve dimensionality reduction transmission of features. As one of the commonly used pooling methods, maximum pooling is suitable for extracting local features such as edges, lines and textures in feature maps. It is easy to ignore the point cloud position distribution, resulting in the loss of most spatial structure information, and ultimately making it difficult to further improve the network classification accuracy. In order to solve this problem, the present invention proposes a verification module that combines confidence and Copula correlation to analyze and correct the classification results output by the fully connected layer. The algorithm flow is as follows: Figure 7 shown.

[0084] Figure 7 The fully connected network (FC) classifies the point cloud data set to be filtered point by point and uses formula (7) to extract the features F1' and F2' of the ground points and non-ground points. Different from the network training process, the feature extraction module of the present invention is implemented by a single-layer PointNet. The features F1' and F2' extracted from the classification results are respectively sent to the Copula correlation analysis module for intra-class point and ambiguous value point analysis. The features between intra-class points are similar, so the Copula distribution function classifies the positively correlated features into the consistent point cloud set, while the independent noise points, outlier points and negatively correlated inter-class points are classified into the ambiguous value point cloud set. Then, the features of the consistent point cloud are extracted for calibration parameter estimation and confidence interval calculation. The ambiguous points in the ground point set and the ambiguous points in the non-ground point set are put into the set R, and the category score rd corresponding to each point in R is read from the fully connected layer. Inspired by the literature (Kang B et al., 2019; Zhang S et al., 2021), the present invention uses the following formula to adjust rd:

[0085] dPls(Fd)×(y+Bel(Fld)×rd(10)

[0086] In the formula, the calibration parameters γ and ζ are predicted by the shared parameter MLP and the softplus activation function, Bel(Fn|d) is the lower limit of the confidence interval, indicating the confidence that the Fn feature belongs to the d-th class point; Pls(Fn|d) is the upper limit of the confidence interval, indicating the likelihood that the Fn feature belongs to the d-th class point (Bai N et al., 2023). On the other hand, the confidence intervals of ground points and non-ground points are calculated by the following formula:

[0087]

[0088] Where |·| is the vector modulus operator, Nd is the total number of sample point clouds of the dth class, σ(·) is the standard deviation calculation function, and 1.96 is the proportional parameter corresponding to the 95% confidence level. The confidence interval estimated by formula (10) is used to adjust the degree of effect of the calibration parameters so that the adjusted results are closer to the actual situation. Finally, the adjusted r d 'The corresponding label can be obtained by inputting the softmax function, and the updated label is combined with the original consistent label and outputted as the filtering result finally outputted by the network of the present invention.

[0089] The present invention is further described by specific embodiments:

[0090] The effectiveness of each module in the present invention is verified on the ISPRS standard filtering dataset. In the first training, 12 samples are randomly selected as the training set, and the remaining 3 are used as the test set. Then the samples in the test set are exchanged with the samples in the training set that have not been tested, and the training process is iterated until all samples are tested. On PointNet, the farthest point sampling module (+FSC) with slope constraint, the intra-class point feature enhancement (+AM) combining Copula correlation and attention mechanism, the classification error correction (+CCV) combining Copula correlation and confidence estimation, and the ablation experiment under the full module combination are designed, as shown in Table 1. At the same time, Table 2 shows the filtering results of each module on 15 samples, and Table 3 shows the time required for a single training batch of each module.

[0091] Table 1. Introduction to each module

[0092]

[0093] Table 2. Filtering accuracy of each module on the ISPRS standard filtering data set (%)

[0094]

[0095] Table 3. Duration (seconds) required for each batch of training for each module

[0096]

[0097]

[0098] As can be seen from Table 2, compared with the MT.E (26.6%) of the baseline, when only FSC is considered in the model, except for the TE of Samp54 which remains unchanged, the rest have decreased to varying degrees. The reason is that the baseline only extracts features from samples through MLP, Maxpool and upsampling of jump connections, making network training a learning process with equal weights and no preference. In the end, it is difficult for the network to highlight the difference between key features and secondary features in complex scenes, thus limiting the further improvement of network accuracy.

[0099] On the other hand, in order to verify the effectiveness of the CCV module, when the baseline is only combined with the CCV module, the feature encoding method of each module in CCV is consistent with the baseline. The combination of CCV and the baseline reduces MT.E by 7.2%, and is higher than the baseline only in the first type error of samp12 and samp51. This is because Samp12 is an urban street with many cars and Samp51 is a slope covered with vegetation. Both are small near-ground targets, and their features are similar to those of ground points. The feature encoding method that only considers the spatial coordinates of sampling points and neighborhood points not only causes misjudgment in the Copula correlation analysis module, but also causes deviations in the parameter prediction and confidence interval estimation modules, thereby misclassifying some correctly classified points again. The combination of the AM module and the baseline reduces MT.E by 24.1%. This fully proves that AM is better than the baseline in local intra-class point search, intra-class spatial structure learning, and key feature extraction. When all three modules are loaded on the baseline, except for Samp31 and Samp61, the error rates of all remaining categories are the lowest, and MT.E also achieves the best result of 2.1%. It is worth noting that since the feature encoding methods of AM and SC-CNN are consistent, this not only enables SC-CNN to extract key features from the rich sample points provided by FSC, but also allows CCV to perform more accurate verification through point coordinates, point spacing, and point normals.

[0100] Further observation of Table 3 shows that, since the feature encoding methods of the FSC module, CCV module and baseline are consistent, the single batch training time of the three methods is not much different. However, the AM module and SC-CNN have more complex feature encoding modes, feature correlation calculation processes and attention mechanisms, which leads to the network requiring more training time and computing costs for feature extraction. In order to more intuitively demonstrate the effects of the ablation experiments of each module of the present invention, Figure 8 The filtering effect of three random scenes is shown. Fig. 9 The first and second category misclassified points in each scenario are shown.

[0101] Figure 8 The three columns of filtering results are Samp12, Samp23 and Samp24 from left to right, and the rows of pictures are baseline filtering effect, baseline filtering effect with FSC module loaded, baseline filtering effect with AM module loaded, baseline filtering effect with CCV module loaded, SC-CNN filtering effect and Ground Truth from top to bottom. Fig. 9 The scenes and Figure 8 For a one-to-one correspondence, Fig. 9 The T.Ⅰ and T.Ⅱ point clouds of each module in three scenes are displayed.

[0102] Figure 8 , Fig. 9 The experimental results combined with the filtering error rate in Table 2 show that the combination of the three optimization modules of data preprocessing, feature enhancement and Copula verification can significantly enhance the model's ability to describe the features of ground points and non-ground points in various scenarios. At the same time, SC-CNN can effectively identify targets with irregular geometric features or discontinuous structures. On the other hand, the ISPRS standard filtering data set contains ridges, faults, and scenes where the same type of targets are at different elevations. Compared with the KNN neighborhood point search method in Euclidean space, the neighborhood points searched by the feature KNN method in the AM module are more likely to be similar points. Therefore, feature encoding in this way can effectively improve the network's ability to learn intra-class point features, and at the same time improve the accuracy of parameter estimation in the CCV module, thereby improving the filtering ability of the network model in complex scenarios.

[0103] In addition, this embodiment also conducted a filtering performance comparison experiment:

[0104] Eight top filtering algorithms in recent publications were selected for comparison with the proposed method, including IMF (Hui Z et al., 2016), CSF (Zhang W et al., 2016), MHC (Chen C et al., 2013), SMRF (Pingel T J et al., 2013), ASF (Hu H et al., 2014), SITA (Chen C et al., 2021), DCNN (Hu X et al., 2016) and MSCNN (Wang B et al., 2022B). The total error rate and average total error rate of 15 samples of the proposed method and the eight filtering algorithms on the ISPRS standard filtering data set are shown in Table 4. In Table 4, MSCNN, DCNN and the proposed method are all deep learning methods. In order to ensure the comparability of the filtering results, the training sets of the three methods are kept consistent.

[0105] Table 4. Accuracy of each method on the ISPRS standard filter dataset

[0106]

[0107]

[0108] In Table 4, the lowest error rate is in bold, and the second lowest error rate is in italics. It can be seen from the table that the methods Samp21, Samp22, Samp23, Samp42, Samp51, Samp54, Samp71 and MT.E of the present invention rank first, with 0.72%, 1.89%, 2.14%, 0.51%, 1.02%, 2.01%, 1.08% and 2.1% respectively, and the error rates are lower than the second place by 0.24%, 0.82%, 0.41%, 0.44%, 0.41%, 0.04%, 0.15% and 0.14% respectively. The total error rates of Samp12, Samp41, Samp53 and Samp61 rank second, with 2.05%, 2.39%, 2.23% and 0.9% respectively, which are 0.11%, 0.98% and 0.04% lower than the first place respectively. Samp11 and Samp52 ranked third, Samp31 ranked fourth, and Samp24 ranked fifth, which are at the middle level of the network models in the listed literature. Usually, due to the complexity of the scene, the irregularity of the shape of the objects, and the ridge faults, it is difficult for the network to obtain the lowest error rate of 15 types of samples at the same time. The reason is that the non-ground points in the flat area have targets with different shapes and structures, while the ground points in the slope mountainous area have the problem of step stratification. During the training process, no matter which type of terrain feature learning the network model tends to favor, the accuracy of another type will be sacrificed. In general, the present invention extracts the training samples corresponding to each type of point through FSC, so that the network can fully learn the information of each type of sample; the introduction of AM extracts key features from a large number of sample information for transmission; the CCV module cross-validates the preliminary prediction results of the training model, and reclassifies the erroneous points in each type of classification results. The combination of the three enables the model of the present invention to achieve the best results in MT.E (2.1%).

[0109] Example 2, Vasik map data experiment:

[0110] The total number of points in the ISPRS Vaihingen 3D semantic labeling competition standard point cloud dataset is 1,165,598. The original 3D point cloud consists of Powerline, Car, Each point contains 3D coordinates and RGB information. The number and proportion of each type of ground feature are shown in Table 5. The overall data set is as follows: Fig.10 shown.

[0111] Table 5. Vaihingen 3D dataset details

[0112]

[0113] As can be seen from the table, like the ISPRS standard filter dataset, the Vaihingen 3D dataset is also a complex dataset containing multiple targets, and both contain irregular terrain structures. However, the Vaihingen 3D dataset is larger than the ISPRS standard filter dataset. The recently published PointNet++ (Qi CR et al., 2017B), BSH-Net (Deng C et al., 2023a), PointNAC (Deng C et al., 2023b), SMRF (Pingel TJ et al., 2013), Slope ( AH et al., 2016), CSF (Zhang W et al., 2016) and IR (Wang B et al., 2022A) were compared with the proposed method. In order to quantitatively evaluate the performance of each method, the deep learning methods were Fig.10 (b) and together with the optimization method Fig.10 (a) is used for filtering test. The above methods all use the first type, second type and overall error as standard metrics, as shown in Table 6. Fig.11 The filtering visualization effects of Groundtruth, BSH-Net, PointNAC, SMRF, Slope and the proposed method are shown. Fig.12 The first and second type of misclassified points are shown.

[0114] Table 6. Filtering error of different methods (%)

[0115]

[0116] In Table 6, the lowest error rate is in bold, and the second lowest error rate is in italics. As can be seen from Table 6, compared with the filtering results of other filtering algorithms on the Vaihingen 3D dataset, SC-CNN has the smallest T.Ⅱ and TE, which are 0.32% and 1.93% lower than the error rate of the second-best method SMRF, respectively. T.Ι ranks second (3.05%) and is 0.67% higher than the best method IR. Observation Fig.11 The segmentation effect of the method of the present invention and other methods in the red box shows that the filtering effect of the method of the present invention is generally consistent with that of Groundtruth. Fig.12The T.Ⅰ and T.Ⅱ point clouds of SC-CNN and other methods are available, and the two types of erroneous point clouds of the present invention are the rarest. The Slope filtering method has a good performance in the dataset where the terrain surface is smooth and there is no drastic change in the local terrain. However, the Vaihingen 3D dataset not only has irregular terrain structure but also complex scene information, so the TE of the Slope method is only 10.89%, ranking last. And in Fig.11 , 12 In the 3D dataset, the Slope method misclassifies a large number of roofs as ground points, and misclassifies some ground points with higher terrain as non-ground points. The performance of SMRF depends on the setting of parameters such as elevation threshold, slope threshold, and initial filter window threshold. On the other hand, the Vaihingen 3D dataset has almost no outliers and noise. Fig.11 and 12 The number of SMRF error points is also relatively rare, second only to the method of the present invention. It is worth noting that the Vaihingen 3D scene contains points that are very similar to Shrub, Hedge, Impervious surface, and Low vegetation. CSF cannot effectively extract the points at the intersection of the four types of targets through the estimated approximate value, which makes T.Ι the last (10.64%). IR proposed a similarity sampling and optimal plane selection method based on RANSAC. This method has good recognition ability for ground points and achieved the lowest T.Ι (2.38%), but this method, like the above three methods, requires multiple experiments to obtain the optimal parameters corresponding to the data set. PointNet++, BSH-Net, and PointNAC are all deep learning methods, and are optimized and designed based on PointNet. Therefore, these three methods not only have small deviations in T.Ⅰ and T.Ⅱ, but also gradually increase the filtering accuracy. In general, for large-scale outdoor point cloud data after adaptive slope sampling, the proposed method can effectively learn the spatial structure of each sampling point while highlighting the differences in point features between classes through Intraclass feature enhancement under attention mechanism, thereby improving the network's ability to identify ground points and non-ground points. Finally, the confidence-based Copula verification module is used to further analyze and correct the prediction results, making the network's filtering results more reliable.

[0117] In summary, the point cloud filtering experiments on the two public datasets ISPRS and 3D Vaihingen show that SC-CNN can improve the overall filtering accuracy by fully learning the classification features between ground points and non-ground points under complex terrain conditions. The average total error MT.E on the ISPRS dataset is 0.17% lower than the suboptimal method, and the average total error MT.E on the 3D Vaihingen dataset is 1.93% lower than the suboptimal method. Compared with the filtering effects of other methods on the two public datasets, the algorithm of the present invention has the best performance. The point cloud filtering experiments on the above two public datasets have demonstrated the good performance of SC-CNN, which is of great significance and value for automatic DEM generation.

[0118] What is disclosed above is only a preferred embodiment of the present invention, and it certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A point cloud filtering method for large-scale scene data, characterized in that: The following steps are involved: Step 1: Based on the multi-layer perceptron MLP network PointNet, build a point cloud filtering network SC-CNN; Step 2: Set the farthest point sampling module of slope constraint; Step 3: Intra-class point feature enhancement module combining Copula correlation and attention mechanism; Step 4: Classification error correction module combining Copula correlation and confidence estimation.

2. The point cloud filtering method for large-scale scene data according to claim 1, characterized in that: The point cloud filtering network SC-CNN includes a farthest point sampling module, a downsampling layer, an upsampling layer, a fully connected layer and a classification error correction module, and the downsampling layer is implanted with an intra-class point feature enhancement module.

3. The point cloud filtering method for large-scale scene data according to claim 2, characterized in that: The execution process of step 2 includes the following steps: Step 2.1: Adaptive sub-region division; Step 2.2: Neighborhood compensation sampling; Step 2.3: Slope pre-classification.

4. The point cloud filtering method for large-scale scene data according to claim 3, characterized in that: In the process of adaptive sub-region division, the calculation formula of the sub-region side length L corresponding to the number of samples n is as follows: L=[√(△x·△y / [N / n])] Wherein, [·] represents the rounding symbol, [N / n] represents the number of sub-regions, △x and △y are the total horizontal areas of the scene, which are obtained by dividing the total area by the number of sub-regions and taking the square root; Then, L / 2 is used as the step size to divide the point cloud into overlapping sub-regions.

5. The point cloud filtering method for large-scale scene data according to claim 4, characterized in that: In step 2, the calculated sub-region side length L is increased by 1.2 times. If the number of points still cannot meet the requirement, the magnification factor is increased by 0.1 until the number of point clouds covered by the sub-region meets the sampling requirement.

6. The point cloud filtering method for large-scale scene data according to claim 5, characterized in that: Slope angle θ in slope pre-classification i , the calculation formula is as follows: θ i =arctan(|△z i | / S i ),S i =√[(x i -x min )2+(and i -and min )2] Among them, △z i is u i Dot and u min The elevation difference of the point, S i is u i Dot and u min The horizontal straight-line distance of the point, reference point u min The point is the point with the smallest z value, which is taken from the point cloud U in each sub-area n ={u1,u2,…u i }z coordinate value.

7. The point cloud filtering method for large-scale scene data according to claim 6, characterized in that: The expression of the Copula correlation and the intra-class point feature enhancement module of the attention mechanism in step 3 is as follows: Among them, τ∈(0,1) is the model parameter of Copula. When τ→1, C(·)→1, and all points have a high correlation; when τ→0, C(·)→0, and all points tend to be independent.

8. The point cloud filtering method for large-scale scene data according to claim 7, characterized in that: In step 4, the classification error correction module combining Copula correlation and confidence estimation analyzes and corrects the classification results output by the fully connected layer. Specifically, the category scores of the ambiguous points are corrected according to the confidence interval and calibration parameters. The in-class points verified by Copula are merged with the calibrated ambiguous points to obtain the filtering results of ground points and non-ground points.

Images