A class-unbalanced building facade point cloud segmentation method based on a topology-aware graph network

Abstract

The application provides a class uneven building facade point cloud segmentation method based on a topology-aware graph network, and specifically comprises the following steps: constructing a facade topology extraction module to capture abstract object-level topology features between facade elements; constructing an enhanced sampling geometry extraction module to extract point-level hierarchical geometric features between facade elements; constructing a dual-feature attention fusion module to fuse the obtained two different granularity and different type features, so that the advantages of the two features are complementary; the three modules are combined to form a topology-aware graph network; and the input data set is divided into a training set, a validation set and a test set, and the network is tested and the result is evaluated after being well trained. The method provided by the application solves the problem that the prior art cannot effectively extract topology features between building facade elements and effectively fuse the topology features with geometric features, and can effectively segment class uneven building facade point clouds.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of three-dimensional point cloud intelligent processing and geographic space information data application research, and particularly relates to a category-unbalanced building facade point cloud segmentation method based on a topology-aware graph network. BACKGROUND

[0002] Building facade point cloud segmentation is a technology that extracts strong spatial features to segment and analyze semantic elements of building facades (such as windows, walls, doors, balconies, etc.). Building facade point cloud segmentation technology is an important core of urban scene understanding and has been widely used in various fields, such as solar potential estimation, building structure health monitoring, digital twin city construction, etc.

[0003] Nowadays, laser scanning technology has become an important means of spatial point cloud data acquisition due to its high precision and high density. Point cloud data contains rich geometric features, shape attributes and topological information, providing key information for building facade segmentation. At the same time, due to the efficient computing power and rich feature description ability, deep learning has been widely popularized in the field of point cloud processing. However, the existing building facade point cloud segmentation method based on deep learning still has the following difficulties: the building facade point cloud often has serious occlusion and point density variation; in the building facade scene, the number ratio of different facade elements is usually seriously unbalanced, which makes the feature learning of small sample targets (such as doors and balconies) insufficient, and further affects the segmentation accuracy; the spatial topology features between facade elements are crucial for segmentation, but how to effectively extract and integrate facade topology and geometric features of different granularities is still challenging.

[0004] The existing methods based on deep learning to extract point cloud features can be roughly divided into three categories: point-based methods, graph-based methods, and attention-based methods. Point-based methods focus on extracting and aggregating the features of each point in the point cloud. The pioneering research on deep learning-based point cloud segmentation and classification is PointNet and PointNet++. Identifying local features can provide an important basis for segmentation. Some studies have proposed neighborhood feature aggregation methods. For example, Point SIFT divides the spatial quadrant to search for neighboring points in different directions and aggregates multi-directional feature information through convolution operations. For large-scale point clouds, some scholars have proposed a highly efficient and lightweight neural network called RandLA. This method uses a random sampling strategy and a local feature aggregation module to extract detailed neighborhood features. Graph-based methods mainly use the permutation invariance of graph structures to capture geometric features. By aggregating graph structures and node features, convolution is used to extract significant geometric features. For example, PointGCN uses global pooling and multi-resolution pooling to capture global and local information. In addition, some scholars have proposed RGCNN, which uses Chebyshev polynomial approximation to extract features through graph convolution and adds a regularization term to the loss function to constrain the model parameters. In addition, there are methods based on spatial graph convolution. For example, DGCNN constructs a local dynamic neighborhood graph and effectively captures hierarchical features in point clouds by dynamically updating the neighborhood. Some scholars have proposed RadDGCNN, which replaces the KNN method with a spherical neighborhood to achieve more effective facade segmentation. Attention-based methods introduce an attention mechanism to adaptively learn the importance of different regions. For example, some scholars have proposed AFGL-Net, which enhances local features through position and direction encoding. The Transformer module is integrated into AFGL-Net, which uses attention-based feature fusion to flexibly balance local and global features. Some scholars have proposed DLA-Net, which is embedded into an encoder-decoder network architecture and uses self-attention and attention pooling modules to learn detailed local features, achieving significant progress in facade segmentation.

[0005] In addition to the above methods, scholars have also focused on researching the use of deep learning networks to capture abstract topological features. In the field of three-dimensional reconstruction, FoldingNet and AtlasNet capture abstract topological information in three-dimensional target point clouds by fusing folding and Riemannian two-dimensional grids. The role of topological features in semantic segmentation has also gradually become prominent. For example, some scholars have proposed a plug-and-play module called SRN to extract intrinsic topological information in the local structure of three-dimensional objects. Some scholars have introduced the topological theory of persistent homology into the deep learning framework and proposed the TopoSeg module and a loss function compatible with point cloud segmentation. However, existing deep learning methods still have limitations in capturing abstract topological information between facade elements, and cannot effectively extract and fuse abstract topological features between building facade elements and geometric features. SUMMARY

[0006] In order to solve the above problems in the prior art, the method provided by the application extracts two different granularity and type features of topological features and geometric features by designing a topological perception graph network FTG-Net including a facade topology extraction module, an enhanced sampling geometric feature extraction module and a double-feature attention fusion module, improves the small sample class segmentation performance, combines the complementary advantages of the two features, and improves the overall performance and robustness of the class-uniform building facade point cloud segmentation.

[0007] In order to achieve the above technical target, the application provides the following technical scheme:

[0008] A class-uniform building facade point cloud segmentation method based on a topological perception graph network, which specifically comprises the following steps:

[0009] S1, a facade topology extraction module is constructed to capture abstract object-level topological features between facade elements;

[0010] S2, an enhanced sampling geometric extraction module is constructed to extract hierarchical geometric features between facade elements at the point level;

[0011] S3, a double-feature attention fusion module is constructed to fuse the two different granularity and type features obtained in steps S1 and S2 to realize the complementary advantages of the two features;

[0012] S4, the facade topology extraction module, the enhanced sampling geometric extraction module and the double-feature attention fusion feature module are combined to form a topological perception graph network FTG-Net; after the building facade sample data is divided into a training set, a validation set and a test set and input into the FTG-Net, the overall network is trained, and then the test and evaluation are performed to realize the class-uniform building facade point cloud segmentation.

[0013] Further, step S1 comprises:

[0014] S11, K-Means uniform sampling is performed on each building facade sample data to obtain complete facade point clouds, the sampled facade point clouds are used as input units, a topological information encoder is used to encode the input units to capture high-dimensional encoded features with strong distinguishability;

[0015] S12, a topological information decoder enhanced by topological attention is used to extract abstract object-level topological features between facade elements from the high-dimensional encoded information.

[0016] More specifically, step S11 specifically comprises:

[0017] The facade point cloud after sampling is first expanded by two consecutive edge convolution operations to learn global coarse-grained features containing topological information between facade elements; then dimension expansion is performed by three-layer convolution operation with dimension , which makes up for the limitation of edge convolution that can only extract coarse-grained features, and obtains high-dimensional coding features with strong discriminability ; the formula is:

[0018] ;

[0019] ;

[0020] wherein, is a point in the facade point cloud after sampling, and there are N points in total; is its neighboring point, and there are K points in total; and respectively represent the facade input point and its neighboring point after the first edge convolution operation; and respectively are the edge convolution operation and the three-layer convolution operation with dimension ; is a feature concatenation operation.

[0021] More specifically, step S12 specifically comprises:

[0022] S121, introduce a two-dimensional Riemann square grid for the topological information decoder; set the two-dimensional Riemann square grid to , then the number N of the facade input point is a 2 ; learn the initial topological information f c from the high-dimensional coding feature and the two-dimensional Riemann square grid , and the formula is:

[0023] ;

[0024] wherein, is the three-layer convolution operation with dimension ; is a feature concatenation operation;

[0025] S122, add a topological attention enhancement module in the decoding process; calculate the attention weight matrix W based on f c , and perform linear transformation; then generate the offset feature f sa according to f c , V and the attention weight matrix W, so as to reduce the strong related information reserved by the weak related topological information; f sa then sequentially passes through linear transformation, batch normalization and ReLU activation function to obtain the object-level topological feature ; the formula is:

[0026] ;

[0027] ;

[0028] ;

[0029] wherein, 、 、 are query, key, value in attention mechanism; is a Softmax activation function; is a linear transformation; is batch normalization.

[0030] Further, the step S2 specifically comprises:

[0031] S21, using an adaptive weighted sampling technique, adaptively adjusting the sampling weights of various facade elements to alleviate the class imbalance problem, thereby sampling the complete facade point cloud and dividing it into multiple overlapping facade point cloud blocks;

[0032] S22, for the facade point cloud blocks obtained in step S21, learn multi-scale geometric features by stacking three hierarchical geometric extraction modules to effectively capture hierarchical geometric features and enhance the description of facade elements.

[0033] More specifically, the calculation process of the sampling weight in step S21 is specifically:

[0034] For the input building facade sample data, use i to represent a facade element, p i represents the proportion of the sample capacity of this facade element to the overall building facade sample data; an adaptive weighted sampling function is constructed to balance the sampling weights between small sample classes and large sample classes; the formula of the adaptive weighted sampling function is:

[0035] ;

[0036] wherein, 、 、 ; 、 、 ; 、 are piecewise curve coefficients; s and l represent the boundary values of small sample classes and large sample classes, which are determined based on the proportion of small sample classes and large sample classes;

[0037] Then, the sampling number n of each facade element is calculated according to the sampling weight​i :

[0038] ;

[0039] wherein N represents the number of points in each facade point cloud block, and C is the number of categories.

[0040] More specifically, step S22 specifically comprises:

[0041] S221, in each hierarchical geometry extraction module, the points in each point cloud block are sequentially subjected to adaptive weighted sampling, K-neighbor searching, edge convolution operation, and strip pooling operation. , its neighboring points are searched through K-neighbor searching method , and edge features are obtained through edge convolution operation , which is expressed by the formula:

[0042] ;

[0043] wherein, is the edge convolution operation; is the feature concatenation operation;

[0044] S222, the strip pooling operation is used to enhance long-distance geometric features; the strip pooling operation, convolution operation and batch normalization are sequentially performed in the horizontal and vertical directions respectively to obtain horizontal features and vertical features ; which is expressed by the formula:

[0045] ;

[0046] ;

[0047] wherein, , are the horizontal strip pooling operation and the vertical strip pooling operation respectively; is the copy operation;

[0048] S223, after the horizontal features and the vertical features are superimposed, they are sequentially subjected to ReLU activation function, convolution operation and Sigmoid normalization to obtain fusion features , which is expressed by the formula:

[0049] ;

[0050] The inner product between the edge features and the fusion features is calculated, and then added point by point to obtain the extraction features of the hierarchical geometry extraction block; the extraction features of the previous hierarchical geometry extraction block are used as the input of the next hierarchical geometry extraction block.

[0051] S224, aggregate the extracted features of the three hierarchical geometry extraction blocks, adopt convolution operation and maximum pooling, and then copy to the same dimension as the original features to aggregate and convolve again, and finally output the hierarchical geometry features at the point level.

[0052] Further, the step S3 specifically comprises:

[0053] S31, constructing a mapping index mechanism to accurately associate and aggregate the object-level topological features and the hierarchical geometry features;

[0054] S32, fusing the topological and geometry features by adopting a channel attention mechanism to retain important topological and geometry features in the segmentation of facade elements, and to realize robust and accurate segmentation of the facade elements.

[0055] More specifically, the step S31 specifically comprises:

[0056] S311, creating a mapping index mechanism to associate each facade point cloud with the facade point cloud block divided by it; for a facade point cloud with an index of , the index of the corresponding facade point cloud block is , the facade point cloud block index is taken as the value, and the index of the facade point cloud to which it belongs is taken as the key, to construct the mapping mechanism; wherein d represents the number of facade point clouds, and j is the number of facade point cloud blocks divided by each facade point cloud;

[0057] For each facade point cloud block, search for the corresponding facade point cloud by the mapping mechanism index; for each point in the facade point cloud block, search for its nearest point in the facade to form a pair; the pair of points after pairing is represented as: ; wherein represents the rth point in the point cloud block, represents the tth point in the corresponding facade;

[0058] S312, connect the hierarchical geometry features carried by all points in each point cloud block with the object-level topological features carried by the corresponding facade points to obtain double-feature fusion features :

[0059] ;

[0060] wherein, is an aggregation operation; K and L are the union of the indices of the points in the point pairs that have been formed in the facade and the point cloud block.

[0061] More specifically, the step S32 specifically comprises:

[0062] S321, based on the double-feature fusion features , calculate the context attention coefficient , the formula is expressed as:

[0063] ;

[0064] wherein, indicates a global average pooling operation;

[0065] S322, utilize the context attention coefficient As a feature weight, it is copied with the same dimension as the original feature and multiplied to make the finally extracted feature focus on the most relevant information; finally, the prediction value of each category is obtained through the MLP operation and the Dropout operation.

[0066] Based on the above technical scheme, the present application has at least the following beneficial effects:

[0067] 1. Through the facade topology extraction module, the topological features between the building facade elements are extracted, that is, the spatial structure information of the facade elements and the spatial relationship between the facade elements are captured, which is beneficial to distinguish the building facade elements;

[0068] 2. Design an enhanced sampling geometry extraction module, combine adaptive weighted sampling and hierarchical geometric feature extraction, extract local and global geometric features, and improve the segmentation performance of small sample categories;

[0069] 3. Design a double-feature attention fusion module to adaptively fuse topological and geometric features of different granularities, improve the overall performance and robustness of facade point cloud segmentation by combining the complementary advantages of the two types of features;

[0070] 4. Through the above three modules, the method proposed by the present application can accurately and robustly identify the building facade elements with serious sample imbalance. BRIEF DESCRIPTION OF DRAWINGS

[0071] Figure 1 is the topological perception graph network FTG-Net framework diagram constructed by the method proposed by the present application;

[0072] Figure 2 is the structure diagram of the facade topology extraction module in FTG-Net;

[0073] Figure 3 is the adaptive weighted sampling function curve diagram in the method proposed by the present application;

[0074] Figure 4 is the structure diagram of the enhanced sampling geometry extraction module in FTG-Net;

[0075] Figure 5 is the structure diagram of the double-feature attention fusion module in FTG-Net;

[0076] Figure 6 is a summary diagram of the building facade point cloud dataset collected in the application example of the present application;

[0077] Figure 7 is a test set segmentation result diagram in the application example of the present application. DETAILED DESCRIPTION

[0078] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0079] The steps in the present application are arranged with labels, but are not used to limit the order of the steps, unless the order of the steps is explicitly stated or the execution of a certain step needs other steps as a basis, otherwise the relative order of the steps can be adjusted. It can be understood that the term "and / or" used herein relates to and covers any and all possible combinations of one or more of the associated listed items.

[0080] The present application proposes a class imbalance building facade point cloud segmentation method based on a topological perception graph network, which provides a topological perception graph network FTG-Net for class imbalance building facade point cloud segmentation, as shown in Figure 1 , which realizes the segmentation function through three modules. In the facade topology extraction module, the abstract topological information between the facade elements is captured to enhance the description of the facade elements; in the sampling enhanced geometry extraction module, the hierarchical geometric features at the point level are extracted through adaptive sampling to further improve the description of small sample facade elements; in the dual feature attention fusion module, the advantages of two features are complementary through adaptive fusion of topological and geometric features of different granularities and different types, and the overall performance of the facade segmentation is improved.

[0081] The method proposed in the present application can realize effective segmentation of class imbalance building facade point cloud, which specifically includes the following steps:

[0082] S1, a facade topology extraction module is constructed to capture abstract object-level topological features between facade elements;

[0083] As a preferred embodiment, step S1 specifically includes:

[0084] S11, K-Means uniform sampling is adopted for each building facade sample data to obtain a facade point cloud, the sampled facade point cloud is taken as an input unit, a topological information encoder is used to encode the input unit to capture high-dimensional encoding features with strong discriminability;

[0085] As shown in Figure 2 , in the topological information encoder:

[0086] The facade point cloud after sampling is first expanded by two consecutive edge convolution operations to learn global coarse-grained features containing topological information between facade elements; and then dimension expansion is performed by three-layer convolution operations with a dimension of to make up for the limitation of edge convolution that can only extract coarse-grained features, so as to obtain high-dimensional encoding features with strong discriminability ; the formula is as follows:

[0087] ;

[0088] ;

[0089] wherein, is a point in the facade point cloud after sampling, and there are N points in total; is its neighboring point, and there are K points in total; and respectively represent the facade input point and its neighboring point after the first edge convolution operation; and are the edge convolution operation and the three-layer convolution operation with a dimension of respectively; is a feature concatenation operation; in this embodiment, a=45 and ;

[0090] S12, using a topological information decoder enhanced by topological attention, abstract object-level topological features between facade elements are extracted from high-dimensional encoding information;

[0091] As shown in Figure 2 , the topological information decoding process is as follows:

[0092] S121, a two-dimensional Riemann square grid is introduced for the topological information decoder; the two-dimensional Riemann square grid is set to , and the number N of facade input points is a 2 The initial topological information f is learned from the high-dimensional encoding features and the two-dimensional Riemann square grid c , and the formula is as follows:

[0093] ;

[0094] wherein, is a three-layer convolution operation with a dimension of ; is a feature concatenation operation; in this embodiment, a=45 and ;

[0095] S122, a topological attention enhancement module is added in the decoding process; based on fc Calculate the attention weight matrix W and perform a linear transformation; then, based on f... c V and attention weight matrix W are used to generate offset features f. sa To reduce weakly correlated topologies while preserving strongly correlated ones; f sa Then, object-level topological features are obtained by sequentially applying linear transformation, batch normalization, and the ReLU activation function. The formula is expressed as:

[0096] ;

[0097] ;

[0098] ;

[0099] in, , , For the attention mechanism, the query, key, and value are used. Use the Softmax activation function; It is a linear transformation; For batch normalization;

[0100] In this embodiment, the facade topology extraction module is trained using the Adam optimizer with an initial learning rate of 0.001, a training batch size of 600, and a batch size of 2. Due to network limitations, 2025 3D points are input each time. Furthermore, the module reconstructs the faceted point cloud to ensure the completeness and accuracy of the extracted topological information between facade objects. Additionally, a chamfer distance is introduced to constrain the facade topology extraction module. To evaluate the original point cloud P and the reconstructed point cloud The similarity between them can be expressed by the formula:

[0101] ;

[0102] in, This calculates the minimum distance between two point pairs in a point cloud. The first term represents the distance from any point X in P to... The sum of the minimum distances, the latter term represents any point in The sum of the minimum distances to P.

[0103] S2. Construct an enhanced sampling geometry extraction module to extract hierarchical geometric features at the point level between facade elements;

[0104] In a preferred embodiment, step S2 specifically includes:

[0105] S21, using adaptive weighted sampling technology, adaptively adjusting the sampling weights of various facade elements to alleviate the class imbalance problem; after sampling, a plurality of overlapping facade point cloud blocks are obtained;

[0106] The specific process of the sampling weight is:

[0107] For the input building facade sample data, let i represent a facade element of a certain class, p i represents the proportion of the sample capacity of the facade element of this class to the overall building facade sample data; an adaptive weighted sampling function is constructed to balance the sampling weights between small sample classes and large sample classes; the formula of the adaptive weighted sampling function is:

[0108] ;

[0109] Among them, , , ; , , ; , , are all piecewise curve coefficients; s and l represent the boundary values of small sample classes and large sample classes, which are determined based on the proportion of small sample classes and large sample classes; in this embodiment, as shown in Figure 3 , a curve graph of the adaptive weighted sampling function is given;

[0110] Then, according to the sampling weight, the sampling number n i of each facade element is calculated:

[0111] ;

[0112] Among them, N represents the number of points in each facade point cloud block, and C is the number of classes;

[0113] S22, for the facade point cloud block obtained in step S21, three hierarchical geometric extraction modules are stacked to learn multi-scale geometric features, so as to effectively capture hierarchical geometric features and enhance the description of facade elements;

[0114] More specifically, as shown in Figure 4 , the sampled facade point cloud is divided into a plurality of facade point cloud blocks; and for each point in the facade point cloud block, the process of capturing hierarchical geometric features is:

[0115] S221, in each hierarchical geometric extraction module, for each point after adaptive weighted sampling, its neighboring points are searched through K-nearest neighbor search method, and edge features are obtained through edge convolution operation, which is expressed by a formula as follows:

[0116] ;

[0117] wherein, is an edge convolution operation; is a feature concatenation operation;

[0118] S222, a strip pooling operation is used to enhance long-distance geometric features; a strip pooling operation, a convolution operation and a batch normalization are sequentially performed in a horizontal direction and a vertical direction respectively to obtain horizontal features and vertical features ; which is expressed by a formula as follows:

[0119] ;

[0120] ;

[0121] wherein, , are a horizontal strip pooling operation and a vertical strip pooling operation respectively; is a copy operation;

[0122] S223, after the horizontal features and the vertical features are superimposed, the superimposed features are sequentially subjected to a ReLU activation function, a convolution operation and a Sigmoid normalization to obtain fused features , which is expressed by a formula as follows:

[0123] ;

[0124] The inner product between the edge features and the fused features is calculated again, and the inner product is added point by point to obtain the extraction features of the hierarchical geometric extraction block; the extraction features of the previous hierarchical geometric extraction block are used as the input of the next hierarchical geometric extraction block;

[0125] S224, the extraction features of the three hierarchical geometric extraction blocks are aggregated, a convolution operation is performed and maximum pooling is performed, and then the extraction features are copied to the same dimension as the original features to be aggregated and convolved again, and finally the hierarchical geometric features at a point level are output.

[0126] In the embodiment, the enhanced sampling geometric extraction module is trained using an SGD optimizer, the initial learning rate is set to 0.001, the training batch is 100, and the batch size is 8; considering the high point density and large size of the facade scene, the facade is divided into 6m x 6m point cloud blocks, and the adjacent blocks overlap by 3m; at the same time, a cross-entropy loss function is used to constrain the parameters of the enhanced sampling geometric extraction module, the function can effectively measure the difference between the predicted probability distribution and the real label distribution, and performs well especially when dealing with classification and segmentation problems, and is denoted as , the specific formula is:

[0127] ;

[0128] wherein N and C represent the number of samples and the number of categories respectively, X uw represents the real label of the u-th sample in the w-th category, p uw represents the probability that the model predicts that the u-th sample belongs to the w-th category.

[0129] In addition, since the Lovász-Softmax loss can significantly optimize the mIoU index, and thus reduce the influence of class imbalance. Therefore, the embodiment also constructs a Lovász-Softmax loss function:

[0130] ;

[0131] wherein e (i) represents the ranking error of each category i, which is obtained according to the difference between the predicted probability and the real label; y represents a hot encoding real label of the point; represents the IoU change of category i;

[0132] The above two loss functions and the chamfer distance are combined as the total loss function of the facade topology extraction module and the enhanced sampling geometry extraction module, and the optimal model is obtained by training; the total loss function is represented as:

[0133] ;

[0134] wherein, and are balance coefficients.

[0135] S3, a dual-feature attention fusion module is constructed to fuse the two different granularities and different types of features obtained in steps S1 and S2, so as to realize the complementary advantages of the two features.

[0136] As a preferred embodiment, step S3 specifically comprises:

[0137] S31, a mapping index mechanism is constructed to accurately associate and aggregate the object-level topology features and the hierarchical geometric features.

[0138] More specifically, as shown in Figure 5 , step S31 specifically comprises:

[0139] S311, a mapping index mechanism is created to associate each facade point cloud and the facade point cloud block divided by it; for a facade point cloud with an index of , the corresponding facade point cloud block index is The facade point cloud block index is taken as a value, and the facade point cloud index to which the facade point cloud block belongs is taken as a key, to construct a mapping mechanism; wherein d represents the number of facade point clouds, and j represents the number of facade point cloud blocks into which each facade point cloud is divided

[0140] For each facade point cloud block, the corresponding facade point cloud is searched by the mapping mechanism index; for each point in the facade point cloud block, the nearest point in the facade is searched to form a pair; the pair of points after pairing is represented as: ; wherein, represents the rth point in the point cloud block, represents the tth point in the corresponding facade;

[0141] S312, all points in each point cloud block carry hierarchical geometric features , and the object-level topological features carried by the corresponding facade point are connected to obtain double-feature fusion features

[0142] ;

[0143] wherein, is an aggregation operation; K and L are the set of indexes of each point in the point pair that has been formed in the facade and the point cloud block.

[0144] S32, the channel attention mechanism is used to fuse the topological and geometric features, important topological and geometric features in the segmentation of facade elements are reserved, and robust and accurate segmentation of the facade elements is realized;

[0145] As shown in Figure 5 , the step S32 specifically includes:

[0146] S321, based on the double-feature fusion features , a context attention coefficient is calculated, and the formula is represented as:

[0147] ;

[0148] wherein, represents a global average pooling operation;

[0149] S322, the context attention coefficient is used as a feature weight, the context attention coefficient is copied to the same dimension as the original feature and multiplied to make the finally extracted features focus on the most relevant information; finally, the prediction value of each category is obtained through an MLP operation and a Dropout operation.

[0150] ​S4, the facade topology extraction module, the enhanced sampling geometry extraction module, and the double-feature attention fusion feature module are combined to form a topology-aware graph network FTG-Net; after the building facade sample data is divided into a training set, a verification set and a test set and input into the FTG-Net, the overall network is trained, and then the test is performed and the result is evaluated, so that the building facade point cloud segmentation of the uneven categories is realized.

[0151] In this embodiment, according to the designed total loss function, after the optimal model of the facade topology extraction module and the enhanced sampling geometry extraction module is obtained through the training set and the verification set, the test set is input, the object-level topology feature and the hierarchical geometry feature are extracted through the optimal model, and after the double-feature attention fusion feature module, the classification of the facade element is finally realized.

[0152] The effect of the method proposed in the application is verified by combining with a specific application example.

[0153] In this application example, two sets of building facade point cloud data sets are created, such as Figure 6 indicated: a campus data set and a city street data set. The two data sets are collected by different terrestrial laser scanners and have diversified facade styles. The data sets are labeled by combining an automatic segmentation algorithm with manual labeling. In the labeling process, in order to facilitate the learning of the topological information between the facade elements, each facade element needs to be labeled individually rather than as a whole. In the labeled facade data set, 51 facades are used for training, 18 are used for verification, and 14 are used for testing.

[0154] To evaluate the facade segmentation, the following indicators are used in this application example: class precision (Precision), recall (Recall), F1 score and intersection over union (IoU), and overall accuracy (OA) of all samples, and the specific calculation formula is as follows:

[0155] ;

[0156] ;

[0157] ;

[0158] ;

[0159] ;

[0160] Wherein, TP, FN and FP are true positive, false negative and false positive counts, respectively.

[0161] After the test set is input, the result of the building facade point cloud segmentation (or facade element classification) is as shown in Figure 7 ; Figure 7Fig. 1-8 are segmentation results of eight different building facade point clouds in the campus dataset, Figure 7 Fig. 9-16 are segmentation results of eight different building facade point clouds in the urban street dataset; different colors in the figures represent different facade elements, including windows, walls, balconies, and doors; (it should be noted that the color of each facade element in the figures is only used for differentiation from other facade elements, and does not mean that each facade element is fixed to one color). Figures 1-7

[0162] The corresponding indicators of each building facade point cloud segmentation result are shown in Table 1 as follows:

[0163] Table 1 Facade segmentation result indicators

[0164]

[0165]

[0166] From Table 1 and Figure 7 It can be seen from Table 1 and

[0167] It should be understood by those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description, and all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any reference signs in the claims should not be regarded as limiting the claims involved.

[0168] Furthermore, it should be understood that although the present specification is described in terms of embodiments, each embodiment does not contain only one independent technical solution, and the description manner of the specification is only for clarity, and those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can be properly combined to form other embodiments that can be understood by those skilled in the art.​

Claims

1. A method for segmenting point clouds of building facades with uneven class distribution based on topology-aware graph networks, characterized in that, Specifically, the following steps are included: S1. Construct a facade topology extraction module to capture abstract object-level topological features between facade elements, specifically including: S11. Complete facade point clouds are obtained by sampling data from each building facade sample. The sampled facade point clouds are used as input units, and a topological information encoder is used to encode the input units, capturing high-dimensional encoded features with strong discriminative power. Specifically: The sampled facade point cloud is first subjected to two consecutive edge convolution operations to expand the receptive field; then it is subjected to a dimension of The three-layer convolutional operation is used to expand the dimension, resulting in high-dimensional encoded features with strong discriminative power. The formula is expressed as: ； ； in, There are N points in the sampled facade point cloud; There are K neighboring points; and These represent the elevation input point and its neighboring points after the first edge convolution operation, respectively. and These are edge convolution operations and operations with dimension 1. The three-layer convolution operation; It is a feature concatenation operation; S12. Using a topological information decoder enhanced with topological attention, abstract object-level topological features between facade elements are extracted from high-dimensional encoded information. S2. Construct an enhanced sampling geometry extraction module to extract hierarchical geometric features at the point level between facade elements, specifically including: S21. Adaptive weighted sampling technology is adopted to adaptively adjust the sampling weight of various facade elements to alleviate the class imbalance problem, thereby sampling the complete facade point cloud and dividing it into multiple overlapping facade point cloud blocks. S22. For the facade point cloud blocks obtained in step S21, multi-scale geometric features are learned by stacking three hierarchical geometric extraction modules. S3. Construct a dual-feature attention fusion module to fuse the two different granularities and types of features obtained in steps S1 and S2. Specifically, this includes: S31. Construct a mapping index mechanism to associate and aggregate object-level topological features and hierarchical geometric features; S32. Employ a channel attention mechanism to fuse topological and geometric features, retaining important topological and geometric features when segmenting facade elements, thereby achieving the segmentation of facade elements; S4, the facade topology extraction module, the enhanced sampling geometry extraction module, and the dual-feature attention fusion feature module are combined to form the topology-aware graph network FTG-Net; the building facade sample data is divided into training set, validation set and test set and then input into FTG-Net. After the overall network is trained, it is tested and the results are evaluated.

2. The point cloud segmentation method for uneven building facades according to claim 1, characterized in that, In step S11, K-Means uniform sampling is used to obtain a complete facade point cloud for each building facade sample data.

3. The point cloud segmentation method for uneven building facades according to claim 2, characterized in that, Step S12 specifically includes: S121. Introduce a two-dimensional Riemannian square mesh to the topology information decoder; set the two-dimensional Riemannian square mesh as... Then the number of input points N on the facade is a. 2 Features encoded by high dimensions and two-dimensional Riemann square grid Learn the initial topological information f c The formula is expressed as: ； in, For dimension The three-layer convolution operation; It is a feature concatenation operation; S122. Add a topology attention enhancement module during the decoding process; based on f c Calculate the attention weight matrix W and perform a linear transformation; then, based on f... c V and attention weight matrix W are used to generate offset features f. sa To reduce weakly correlated topologies while preserving strongly correlated ones; f sa Then, object-level topological features are obtained by sequentially applying linear transformation, batch normalization, and the ReLU activation function. The formula is expressed as: ； ； ； in, , , For the attention mechanism, the query, key, and value are used. Use the Softmax activation function; It is a linear transformation; For batch normalization.

4. The point cloud segmentation method for uneven building facades according to claim 1, characterized in that, The specific calculation process of the sampling weight in step S21 is as follows: For the input building facade sample data, let i represent a certain type of facade element, p i This represents the proportion of the sample size of this type of facade element to the total building facade sample data; an adaptive weighted sampling function is constructed. To balance the sampling weights between small-sample and large-sample classes, the adaptive weighted sampling function is expressed as follows: ； in, , , ; , , ; , , All are piecewise curve coefficients; s and l represent the boundary values ​​between the small sample class and the large sample class, and the boundary values ​​are determined based on the ratio of the small sample class to the large sample class. Then, the number of samples for each type of facade element is calculated based on the sampling weight. : ； Where N represents the number of points in each facade point cloud block, and C represents the number of categories.

5. The point cloud segmentation method for uneven building facades according to claim 1, characterized in that, Step S22 is as follows: S221. In each hierarchical geometry extraction module, for each point after adaptive weighted sampling Search for its neighboring points using the K-nearest neighbor search method. Then, edge features are obtained through edge convolution operations. The formula is expressed as: ； in, This is an edge convolution operation; It is a feature concatenation operation; S222. Strip pooling is used to enhance long-range geometric features; strip pooling, convolution, and batch normalization are performed sequentially in the horizontal and vertical directions to obtain horizontal features. and vertical features The formula is expressed as: ； ； in, , These are horizontal strip pooling operations and vertical strip pooling operations, respectively. For copy operation; S223: Horizontal and vertical features are superimposed, then sequentially processed through ReLU activation, convolution, and Sigmoid normalization to obtain the fused features. The formula is expressed as: ； Recalculate edge features and fusion features The inner product between the layers is calculated and added point by point to obtain the extracted features of the hierarchical geometric extraction block; the extracted features of the previous hierarchical geometric extraction block are then used as the input of the next hierarchical geometric extraction block. S224. The extracted features of the three hierarchical geometric extraction blocks are aggregated, convolutional operation is performed and max pooling is applied, and then copied to the same dimension as the original features for aggregation and convolution again, finally outputting point-level hierarchical geometric features.

6. The point cloud segmentation method for uneven building facades according to claim 1, characterized in that, Step S31 specifically includes: S311. Create a mapping index mechanism to associate each facade point cloud with its corresponding facade point cloud blocks; for an index of... The facade point cloud, and its corresponding facade point cloud block index is A mapping mechanism is constructed by using the facade point cloud block index as the value and the facade point cloud index to which it belongs as the key; where d represents the number of facade point clouds and j is the number of facade point cloud blocks into which each facade point cloud is divided. For each facade point cloud block, the corresponding facade point cloud is searched using the mapping mechanism index; for each point in the facade point cloud block, its nearest point is searched in the facade to form a pair; the paired point pair is represented as follows: ;in, This represents the r-th point in the point cloud block. This represents the t-th point on its corresponding elevation. S312. Assign hierarchical geometric features to all points in each point cloud block. The object-level topological features carried by the corresponding facade points By performing a connection, a dual-feature fusion feature is obtained. : ； in, It is an aggregation operation; K and L are the set of point indices in the point pairs already formed in the elevation and point cloud blocks.

7. The point cloud segmentation method for uneven building facades according to claim 6, characterized in that, Step S32 specifically includes: S321, Based on dual-feature fusion features Calculate the context attention coefficient The formula is expressed as: ； in, This indicates a global average pooling operation; S322. Utilizing the Contextual Attention Coefficient As feature weights, they are copied to the same dimension as the original features and multiplied to concentrate the extracted features on the most relevant information; finally, the predicted values ​​for each category are obtained through MLP operations and Dropout operations.

Images