Building point cloud lightweight surface reconstruction method and device based on deep learning

Abstract

The invention discloses a building point cloud lightweight surface reconstruction method and device based on deep learning, and the method comprises the steps: (1) constructing a data set, and carrying out the related preprocessing; (2) predicting planes in the building point cloud and a plane adjacent relation; (3) extracting vertexes in the building according to the adjacent relation of the building point cloud planes; (4) optimizing the building vertexes, deleting error points in the building vertexes and merging repeated points; (5) extracting edges of the building; and (6) extracting a building vertex sequence in the building point cloud plane, ensuring the building vertex sequence to be in a clockwise direction, and outputting a building polygon mesh model. According to the invention, high-precision training data is provided for a deep learning model, rapid and accurate prediction of the plane of the building in the building point cloud and the adjacent relation of the planes can be realized, and an effective post-processing flow is provided for a prediction result. Redundant points and wrong vertexes of the building can be effectively and rapidly removed, a main body structure of the building is reconstructed, and finally a lightweight building polygon mesh model is generated.

Description

Technical Field

[0001] The present invention belongs to the field of three-dimensional reconstruction technology, and specifically focuses on an innovative method for automatically generating LOD2-level building models from three-dimensional point clouds, aiming to provide an efficient building point cloud surface reconstruction method and device for urban scene digitization. Background Art

[0002] In recent years, with the popularity of drones, it is no longer difficult to quickly obtain a large number of high-resolution urban images. With its flexible flight capabilities, low cost and easy operation, drones have become one of the main tools for urban data collection. Among them, buildings are the most important components of cities. How to accurately convert buildings into digital three-dimensional models is a key issue in 3D urban reconstruction. With the help of high-resolution cameras and laser radar (LiDAR) sensors carried by drones, a large number of building surface images and point cloud models can be obtained in a short time. Although the point cloud model of the building retains the real environmental information of the building through a large number of discrete points and captures the geometric information and complex shape of the building, it has the following disadvantages:

[0003] 1) Storage: The large amount of data leads to high processing and storage costs;

[0004] 2) In terms of visualization: the noise and missing points in the point cloud will lead to poor visualization effects.

[0005] In this context, a lightweight surface reconstruction method of building point cloud based on deep learning came into being. It aims to predict the main structure of the building with the help of deep learning methods and use polygonal meshes to represent it. In terms of storage volume, the use of polygonal meshes to represent buildings is far lower than point cloud data. In terms of visualization, polygonal mesh data retains the sharp features of the building and is less affected by noise.

[0006] Most of the existing lightweight building point cloud surface reconstruction methods extract the building plane, and then reconstruct the grid model by segmenting and selecting the space according to the plane. In terms of plane extraction, most traditional methods such as RANSAC and region growing method are used to extract the point cloud plane. When using the RANSAC method, it is necessary to continuously adjust the parameters for different point cloud building models to obtain a better result. At the same time, the computational overhead of the RANSAC method is also relatively large. The effect of the region growing method depends on the selection of seed points. Inappropriate seed points may lead to incomplete or erroneous region expansion. The method is also sensitive to noise and outliers. Noise may cause erroneous region growth, thus affecting the accuracy of the final reconstruction result. At the same time, the region growing method may fall into a local optimal solution, resulting in some real surface features not being captured. There are also some methods that use deep learning methods to predict the position of vertices or edges in the point cloud to reconstruct the main structure of the building, but this is a great challenge for deep learning networks because there is usually noise in building point clouds;

[0007] In view of the difficulty of point cloud plane extraction, the present invention creatively proposes a building point cloud plane prediction method based on Hough voting in building point cloud plane extraction; at the same time, in order to realize the reconstruction of the main structure of the building point cloud, the present invention also creatively uses a deep learning method to realize the prediction of the adjacent relationship between the building point cloud planes; then, the lightweight surface reconstruction of the building point cloud is realized according to the predicted building point cloud planes and the adjacent relationship between the planes; Summary of the invention

[0008] The present invention aims to overcome the above-mentioned shortcomings of the prior art and provide a method and device for lightweight surface reconstruction of building point clouds based on deep learning.

[0009] The present invention adds a self-attention mechanism to the PointNet++ network (the PointNet++ network model of the present invention is from the literature: Qi CR, Yi L, Su H, et al. Pointnet++: Deep hierarchical feature learning on point sets in a metric space [J]. Advances in neural information processing systems, 2017, 30.), which can effectively improve the ability to extract point cloud features; at the same time, in order to solve the problem of difficulty in extracting planes in building point clouds, the present invention applies the idea of ​​Hough voting to the deep learning network, and the building point clouds vote themselves to predict the building planes, and then predict the adjacent relations between the building planes; finally, the polygonal mesh model of the building is reconstructed according to the predicted building planes and the adjacent relations.

[0010] The present invention provides a method for lightweight surface reconstruction of building point clouds based on deep learning, and the specific steps are as follows:

[0011] S1: Constructing the dataset: The original data of the dataset comes from the 3D spatial data station in Estonia, which is LOD2 level triangular mesh data. The input of this step is a triangular mesh data, and the output is the point cloud data with labels. The specific steps include:

[0012] S1.1: Point cloud sampling based on barycentric coordinates: Input a normalized triangular mesh building data, whose vertex range is between -1 and 1, set the set of all triangular facets in the building to S, and let S i represents a triangle in the set, where i = 1, 2…n, n is the number of triangles; calculate the area of ​​each triangle, set to S i _area; Let the sampling density of the triangle patch be D, and the area of ​​each triangle patch S i The number of sampling points per triangle is obtained by multiplying _area and sampling density D, which is set to S i _sample; then, according to the principle of barycentric coordinates, S is randomly generated inside the triangle patch i _sample points, set to S i _points, S i _points is the sampling point cloud in each triangle patch. The above steps only sample the point cloud inside the triangle patch, but not the point cloud on the edge of the triangle patch. Therefore, 10 points are collected at equal intervals according to the length of the triangle patch edge, and these points are also added to S i _points; at the same time, calculate the normal of each triangle and set it to S i _normal, which is also the point set S i The normal of each point in _points;

[0013] S1.2: Point cloud clustering based on building planes; First, roughly filter out the triangles on the same building plane based on the normals of the triangles and the vertical distances between the triangles. The specific content of the rough screening is: 1) First, randomly select a triangle S from S i , all triangles in the set S and S i_normal triangles with normal angles less than 5 degrees are taken out and temporarily classified into the same category, and the above operation is repeated until all triangles in S are classified; 2) Then the following operations are performed on the set of triangles classified in the previous step: a triangle is randomly taken out, and the vertical distances from the three vertices on the triangle to the other triangles are calculated and averaged to obtain the vertical distance between the triangles; finally, the triangles with vertical distances less than 0.05 between the triangles are temporarily divided into a set of triangles belonging to the same building plane to obtain the rough screening result, which is set as M;

[0014] The divided triangle face set is then further screened. The specific steps are: (1) Take a triangle face from the triangle face set M and add it to the set M′. (2) Then repeatedly compare M′ with M. Each time, take out all the triangle facets with common points from M and add them to M′ until no new triangle facets can be taken out from M. At this time, M′ stores the triangle facet set belonging to the same building plane. (3) If there are still triangle facets remaining in M, repeat steps (1) and (2) until there are no triangle facets remaining in M. The above steps realize the classification of S, and set the face triangle facet set belonging to the same building plane as G. j , where j = 1, 2…, m, m is the number of building planes in the building triangular mesh data. Since each triangular face corresponds to its own sampling point cloud, the sampling point cloud on each building plane can be further obtained, which is expressed as G j _points, which represents the point cloud on each building plane. At this time, the correspondence between the plane and the point cloud in the building is obtained;

[0015] S1.3: Calculate the adjacent relationship between building planes; extract G j Then remove the duplicate edges to get the outer edge of the building plane, denoted as G j _edge, then extract G j The vertices in _edge are the vertices of the building plane, denoted by G j _vertices; Finally, compare the building planes in pairs to see if there are repeated vertices between the two building planes. If there are repeated vertices, it means that the two building planes are adjacent, and the adjacent relationship of the building planes in G can be obtained. Subsequently, the adjacent relationship of the planes in the building point cloud can be obtained from the corresponding relationship between the building planes and the point cloud;

[0016] S1.4: Calculate the minimum bounding box of the building plane; G j The edges with an angle less than 1.5 degrees in _edge are merged, and the length of the new edge is the sum of the lengths of the two edges; then the longest edge is selected as the main direction, and then the AABB algorithm is used to generate G j The minimum bounding box Gj _box, the minimum bounding box is represented by four vertices; then, the minimum bounding box of the plane in the building point cloud can be obtained by the correspondence between the building plane and the point cloud; at the same time, the center point and normal of the building point cloud plane can be obtained by calculating the center point and normal of the bounding box;

[0017] After the above operations, the building grid data can be converted into normalized building point cloud data. Each point cloud in the building point cloud data has an accurate normal, and the point clouds belonging to the same building plane are classified. At the same time, the adjacent relationship between building planes and the minimum bounding box of each building plane are established.

[0018] S2: Prediction of planes in building point clouds and adjacent relationships of planes: Input point cloud data with normals, first use the PoingNet++ network combined with the self-attention mechanism to select voting points and extract features, then cluster the point cloud based on the Hough voting method, and then predict the planes in the building point cloud based on the clustering results, and then predict the adjacent relationships between the planes of the building point cloud;

[0019] Specifically, the step S2 specifically includes:

[0020] S2.1: Select voting points and extract voting point features: The selection of voting points and the extraction of voting point features are carried out through the backbone network. The backbone network is designed based on the PointNet++ network model and consists of 4 SetAbstraction layers (SA layers for short) and 3 Feature Propagation layers (FP layers for short). At the same time, a self-attention mechanism module is added to each SA layer to improve the feature extraction capability. The input of the network is N×(d+C), where N is the number of point clouds, d is the position of the point cloud, and C is the feature channel of the point cloud. The output after passing through the backbone network is N vote ×(d vote +C vote ), where N vote is the number of polling stations, d vote is the location of the polling station, C vote for the characteristics of the polling place;

[0021] S2.2: Point cloud clustering based on Hough voting: The voting network consists of an MLP network, with input voting point N vote ×(d vote +V vote ), output after voting network Among them, C voted is the feature of the post-voting point, is the position of the center point of the building point cloud plane considered by the voting point; in the loss function, the mean square error is used; at the same time, in order to allow the voting points on the plane with a relatively small area in the building point cloud plane to vote to the correct center point position of the building point cloud plane, the present invention assigns different weights to the voting points according to the area size of the building point cloud plane; the specific formula of the loss function of the voting network module is as follows:

[0022]

[0023] where x i Indicates the position of the center point of the building point cloud plane in the label, x i_voted represents the location of the polling station after voting, N face Indicates the number of actual building point cloud planes, N i_pv Represents x i_vote The number of voting points on the point cloud plane of the building;

[0024] S2.3 Aggregate the characteristics of the plane of the building point cloud: Since the number of planes in the building point cloud is unknown, the present invention assumes that there are 64 planes in the building point cloud, which is more than the number of planes in the actual building point cloud. The farthest point sampling method is used to sample 64 center points, denoted as K i , where i = 1, 2…64; then for each K i Clustering: Perform a ball query with a radius of 0.1 to obtain the 16 points closest to itself. If there are less than 16 points, copy the nearest point as a replacement for completion, recorded as K×16×C voted ; Then, an MLP is used to aggregate the features of the voting points to obtain the features K×C of the building point cloud plane face ;

[0025] S2.4 Building point cloud plane prediction: Based on the features of the building point cloud plane obtained in step S2.3, the present invention uses three MLPs to predict the center point, normal, and confidence of the building point cloud plane; the predicted set of all building point cloud planes is set as F pre , the prediction of the center point of the building point cloud plane uses the mean square error to implement the constraint on the network; on the label, the center point of the minimum bounding box of the building point cloud plane obtained by step S1 is the center point of the building point cloud plane to which the voting point belongs; the prediction of the normal of the building point cloud plane uses the mean square error to implement the constraint on the network; the normal of the minimum bounding box of the building point cloud plane obtained by step S1 is the normal of the building point cloud plane to which the voting point belongs; the confidence prediction of the building point cloud plane uses the binary cross entropy to implement the constraint on the network; the similarity between the predicted building point cloud plane and the building point cloud plane in the label is the confidence label;

[0026] The similarity calculation method of the building point cloud plane is: calculate the vertical distance d between the two center points, and then normalize it. Since the point cloud range is [-1,1], the maximum distance between the two points is So the normalization process is Then, the normalized d is used to calculate the similarity. The specific formula is as follows:

[0027]

[0028] Among them, Sim center is the similarity of the center point of the building point cloud plane;

[0029] The cosine similarity of the normals of the two building point cloud planes is calculated, and then the absolute value of the cosine similarity is taken. At this time, the absolute value of the cosine similarity is assumed to be Cos sim , then the calculation formula of normal similarity is as follows:

[0030]

[0031] Among them, Sim normal is the normal similarity of the building point cloud plane;

[0032] The similarity of two building point cloud planes is calculated based on the vertical distance. First, a unit vector a that is not parallel to the normal of the building point cloud plane is taken outside the first building point cloud plane, and then the vector a and the normal are cross-producted to obtain a unit vector u in the building point cloud plane. Then, the vector u and the normal are cross-producted to obtain another unit vector v in the building point cloud plane that is perpendicular to u and the normal at the same time; then, taking the center point of the building point cloud plane as the starting point, four different points on the building point cloud plane are obtained by ±0.1 in the u and v directions; the same operation is used on the other building point cloud plane to obtain four different points on the building point cloud plane; finally, the four points on the two building point cloud planes are calculated in sequence according to the calculation method of the similarity of the center points of the building point cloud planes, and the results are averaged to obtain the similarity of the vertical distance: Sim distance ;

[0033] The proportions of the center point similarity, normal similarity, and vertical distance similarity of the building point cloud plane in the total similarity are 0.3, 0.4, and 0.3 respectively. The calculation formula is shown in the figure below:

[0034] Sim=0.3×Sim center +0.4×Sim normal +0.3×Sim distance (4)

[0035] Among them, Sim is the similarity between two building point cloud planes;

[0036] S2.5 Removal of redundant building point cloud planes: In F pre According to the confidence level, the building point cloud planes with confidence level less than 0.9 are removed to obtain the coarsely filtered building point cloud plane set F k Next, for F k Do the following: From E k Take out the building point cloud plane with the highest confidence and add it to the building point cloud plane set F used to predict the adjacent relationship of the building point cloud plane nms In the k The similarity of the remaining building point cloud planes in F is calculated, and the building point cloud planes with similarity higher than 0.9 are removed from F k Repeat the above steps until F k There are no building point cloud planes left;

[0037] S2.6 Prediction of adjacent relationship of building point cloud planes: For the building point cloud plane set F obtained in step S2.5 nms To predict the adjacent relationship between two objects, a paired face attention module is first used to fuse the features of two building point cloud planes, and an MLP network is used to predict the adjacent relationship between building point cloud planes. The paired face attention module consists of two MLPs, a sigmoid activation function, and a maximum pooling layer. nms The building point cloud planes in the image are paired in pairs. After each building point cloud plane pair passes through the face attention module to fuse the building point cloud plane pair features, an MLP network is used to output the prediction result Face_pairs×1, where Face_pairs is the dimension of the number of building point cloud plane pairs. In terms of the loss function, the module uses binary cross entropy for calculation.

[0038] S3: Extract building vertices: Based on step S.2.6, the adjacent probability of the building point cloud plane pair can be obtained: Face_pairs×1. The building point cloud plane pairs with the median of the adjacent probability Face_pairs×1 greater than 0.5 are assumed to be adjacent. At this time, X pairs of adjacent building point cloud plane pairs can be obtained; then, triples are extracted from the X pairs of building point cloud plane pairs. The so-called triples refer to three adjacent building point cloud planes; then, the intersection of these three building point cloud planes is performed to obtain the vertex positions of the building. Let the vertex position set be M i , where i = 1, 2…n, n is the number of building vertices; at the same time, the triple set of each building vertex is retained, denoted by M i _triple.

[0039] S4: Optimizing building vertices: There are two main problems with the building vertices obtained in step S3. One is that building vertices that do not actually exist are generated, and the other is that duplicate building vertices are generated;

[0040] The reason why the building vertices that do not actually exist are generated is that the adjacent relationship in the label is determined by whether there are common points between two building point cloud planes, so some non-existent building vertices will be generated; however, the positions of these building vertices are far away from the building, so the present invention directly calculates M i The closest distance to the point cloud in the original point cloud data. If the distance is greater than 0.1, it means that the building vertex is not the actual vertex of the building, and it is directly removed from the vertex set M i Delete from;

[0041] The reason for generating duplicate vertices is that there may be more than three building planes near a vertex in the building structure, so the triplet of building vertices is not unique; due to calculation errors, M i The building vertices obtained by the _triple intersection are not in the same position, but they are close to each other. Therefore, the duplicate building vertices can be removed by merging the building vertices within a distance of 0.05. The optimized building vertex set is set to V, and each vertex is set to V i , where i = 1, 2…n, where n is the number of retained building vertices; detect M i For all building vertices in the vertices, if the distance between building vertices is within 0.05, it means that they are repeated building vertices and need to be merged. The positions of the new vertices are directly averaged and stored in the building vertex set V. If the building vertices are not repeated, they can be directly stored in the vertex set V. When merging vertices, the triples M of these building vertices will be obtained. i _triple is also merged, and the plane set of the building point cloud after merging each building vertex triple is set to P, P i It is a set of building point cloud planes of optimized building vertices.

[0042] S5: Extract building edges: Based on the optimized vertices V and the building point cloud plane set P obtained in step S4, we can determine the building edges by i Building point cloud plane P i To determine whether two building vertices form a building edge. From the knowledge of geometric structure, we know that if two building vertices form an edge, then there must be two identical building planes near the two building vertices. Therefore, in this step, we can determine whether two building vertices form an edge by directly judging whether there are two or more coplanar surfaces in the building point cloud planes corresponding to the two building vertices. The building edge set is recorded as E i, where i = 1, 2…m, where m is the number of edges.

[0043] S6: Constructing a polygonal mesh model of the building: The building vertices and edges obtained in step S4 and step S5 have formed a wireframe model of the building. It can be seen from step S5 that the generation of each building edge is determined by detecting whether there are two or more common faces; therefore, it is only necessary to obtain the edges and vertices on each building point cloud plane according to which edges are involved in the generation of the building point cloud plane; after obtaining the edges and vertices on the building point cloud plane, the vertices and edges of these buildings are formed into a graph, and then the depth-first search method is used to search for the largest loop, which is the sequential structure of the building polygonal faces composed of the building vertices and edges; then it is necessary to detect whether the vertex order is clockwise, if not, it needs to be flipped; after all the vertex orders are adjusted, the final output polygonal mesh model is obtained;

[0044] The clockwise detection algorithm for the building vertex sequence is: first, take the first three building vertices v1, v2, and v3 in the loop order, and then use these three building vertices to form two vectors in the building point cloud plane in the loop order: a = v2-v1, b = v3-v2, and then calculate the cross product of vectors a and b to obtain the normal vector direction vector normal of the building point cloud plane direction ; Then, take an adjacent face of the building point cloud plane, average the center points of the two building point cloud planes, and get a point p inside the building; use point p and v1 to form a vector pointing to the outside of the building: out direction =v1-p; then the vector normal direction and out direction Perform dot product. If the value is greater than 0, it means that the current loop is in a clockwise direction, otherwise it means that the order of building vertices is counterclockwise.

[0045] The second aspect of the present invention relates to a lightweight surface reconstruction device for building point clouds based on deep learning, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement a lightweight surface reconstruction method for building point clouds based on deep learning of the present invention.

[0046] The technical concept of the present invention is: for the extraction of planes in building point clouds, a plane detection method based on Hough voting is proposed, and redundant faces and erroneous faces are removed from the building plane prediction results, and the adjacent relationship between building planes is predicted. And according to the predicted building planes and the adjacent relationship of building planes, building vertex extraction, optimization and building edge extraction are performed, and finally the main structure of the building is obtained. Subsequently, after the order of vertices in the building plane is extracted and guaranteed to be in a clockwise direction, a polygonal mesh model of the building is obtained.

[0047] The advantages of the present invention are: it can realize fast and accurate prediction of planes in building point clouds and the adjacent relationships between planes, and provide an effective post-processing process for the prediction results, which can effectively and quickly remove redundant building vertices and erroneous building vertices and reconstruct the main structure of the building, and finally generate a lightweight building polygonal mesh model; it can effectively reduce the storage burden for digital three-dimensional city models and improve the visualization effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 Constructing a flow chart for the dataset of the present invention;

[0049] FIG2(a) and FIG2(b) are the original data and the processed data of the data set of the present invention, wherein FIG2(a) is the original data and FIG2(b) is the processed building point cloud model;

[0050] Figure 3 This is a general flow chart of the method for lightweight surface reconstruction of building point clouds based on deep learning of the present invention;

[0051] Figure 4 This is a network structure diagram based on PointNet++ of the present invention;

[0052] Figure 5 It is a flow chart of the building plane prediction part of the present invention;

[0053] Figure 6 This is the calculation flow chart of the building plan similarity;

[0054] Figure 7 Flowchart for predicting adjacent relations of building planes;

[0055] Figure 8 A flowchart for extracting building vertices based on the adjacent relationship of building planes;

[0056] Fig. 9 Flowchart for optimizing building vertices;

[0057] Fig.10 Flowchart for converting a building wireframe model into a building polygonal mesh model. Specific implementation plan

[0058] The present invention will be further described in detail below with reference to the accompanying drawings.

[0059] Example 1

[0060] This embodiment provides a method for lightweight surface reconstruction of building point clouds based on deep learning. The specific steps are as follows: S1: constructing a data set. The data set construction process is as follows: Figure 1The original data of the dataset comes from the 3D spatial data station in Estonia, which is LOD2 level triangular mesh data; the input of this step is a triangular mesh data, and the output is the point cloud data with labels; the specific steps include:

[0061] S1.1: Point cloud sampling based on barycentric coordinates. Input a mesh data, as shown in the left figure of Figure 2(a), set the set of all triangular facets in the building as S, and let S i represents a triangle in the set, where i = 1, 2…n, n is the number of triangles; calculate the area of ​​each triangle, set to S i _area; Let the sampling density of the triangle patch be D, and the area of ​​each triangle patch S i The number of sampling points per triangle is obtained by multiplying _area and sampling density D, which is set to S i _sample; then, according to the principle of barycentric coordinates, S is randomly generated inside the triangle patch i _sample points, set to S i _points, S i _points is the sampling point cloud in each triangle patch. The above steps only sample the point cloud inside the triangle patch, but not the point cloud on the edge of the triangle patch. Therefore, 10 points are collected at equal intervals according to the length of the triangle patch edge, and these points are also added to S i _points; at the same time, calculate the normal of each triangle and set it to S i _normal, which is also the point set S i The normal of each point in _points;

[0062] S1.2: Point cloud clustering based on building planes; First, roughly filter out the triangles on the same building plane based on the normals of the triangles and the vertical distances between the triangles. The specific content of the rough screening is: 1) First, randomly select a triangle S from S i , all triangles in the set S and S i _normal triangles with normal angles less than 5 degrees are taken out and temporarily classified into the same category, and the above operation is repeated until all triangles in S are classified; 2) Then the following operations are performed on the set of triangles classified in the previous step: a triangle is randomly taken out, and the vertical distances from the three vertices on the triangle to the remaining triangles are calculated and averaged to obtain the vertical distance between the triangles; finally, the triangles with vertical distances less than 0.05 between the triangles are temporarily divided into a set of triangles belonging to the same building plane to obtain the rough screening result, which is set as M;

[0063] The divided triangle face set is then further screened. The specific steps are: (1) Take a triangle face from the triangle face set M and add it to the set M′. (2) Then repeatedly compare M′ with M. Each time, take out all the triangle facets with common points from M and add them to M′ until no new triangle facets can be taken out from M. At this time, M′ stores the triangle facet set belonging to the same building plane. (3) If there are still triangle facets remaining in M, repeat steps (1) and (2) until there are no triangle facets remaining in M. The above steps realize the classification of S, and set the face triangle facet set belonging to the same building plane as G. j , where j = 1, 2…, m, m is the number of building planes in the building triangular mesh data. Since each triangular face corresponds to its own sampling point cloud, the sampling point cloud on each building plane can be further obtained, which is expressed as G j _points, which represents the point cloud on each building plane. At this time, the correspondence between the plane and the point cloud in the building is obtained;

[0064] S1.3: Calculate the adjacent relationship between building planes; extract G j Then remove the duplicate edges to get the outer edge of the building plane, denoted as G j _edge, then extract G j The vertices in _edge are the vertices of the building plane, denoted by G j _vertices; Finally, compare the building planes in pairs to see if there are repeated vertices between the two building planes. If there are repeated vertices, it means that the two building planes are adjacent, and the adjacent relationship of the building planes in G can be obtained. Subsequently, the adjacent relationship of the planes in the building point cloud can be obtained from the corresponding relationship between the building planes and the point cloud;

[0065] S1.4: Calculate the minimum bounding box of the building plane; G j The edges with an angle less than 1.5 degrees in _edge are merged, and the length of the new edge is the sum of the lengths of the two edges; then the longest edge is selected as the main direction, and then the AABB algorithm is used to generate G j The minimum bounding box G j _box, the minimum bounding box is represented by four vertices; then, the minimum bounding box of the plane in the building point cloud can be obtained by the correspondence between the building plane and the point cloud; at the same time, the center point and normal of the building point cloud plane can be obtained by calculating the center point and normal of the bounding box;

[0066] After the above operations, the building grid data can be converted into normalized building point cloud data, as shown in Figure 2(b). Each point cloud in the building point cloud data has an accurate normal, and the point clouds belonging to the same building plane are classified. At the same time, the adjacent relationship between building planes and the minimum bounding box of each building plane are established.

[0067] Figure 3 This is the overall flow chart of the lightweight surface reconstruction method of building point cloud based on deep learning. The following is a detailed introduction.

[0068] S2: Prediction of planes in building point clouds and adjacent relationships of planes: Input point cloud data with normals, and then use the PoingNet++ network combined with the self-attention mechanism (the PointNet++ network model of the present invention comes from the literature: Qi CR, Yi L, Su H, et al. Pointnet++: Deep hierarchical feature learning on point sets in a metric space [J]. Advances in neural information processing systems, 2017, 30.) to select voting points and extract features, then cluster the point cloud based on the Hough voting method, and then predict the planes in the building point cloud based on the clustering results, and then predict the adjacent relationships between the planes of the building point cloud; the specific network structure is as follows Figure 4 shown.

[0069] Specifically, the step S2 specifically includes:

[0070] S2.1: Select voting points and extract voting point features: The selection of voting points and the extraction of voting point features are carried out through the backbone network. The backbone network is designed based on the PointNet++ network model and consists of 4 SetAbstraction layers (SA layers for short) and 3 Feature Propagation layers (FP layers for short). At the same time, a self-attention mechanism module is added to each SA layer to improve the feature extraction capability. The input of the network is N×(d+C), where N is the number of point clouds, d is the position of the point cloud, and C is the feature channel of the point cloud. The output after passing through the backbone network is N vote ×(d vote +C vote ), where N vote is the number of polling stations, d vote is the location of the polling station, C vote for the characteristics of the polling place;

[0071] S2.2: Point cloud clustering based on Hough voting: The voting network consists of an MLP network, with input voting point N vote ×(d vote +C vote ), output after voting network Among them, C voted is the feature of the post-voting point, is the position of the center point of the building point cloud plane considered by the voting point; in the loss function, the mean square error is used; at the same time, in order to allow the voting points on the plane with a relatively small area in the building point cloud plane to vote to the correct center point position of the building point cloud plane, the present invention assigns different weights to the voting points according to the area size of the building point cloud plane; the specific formula of the loss function of the voting network module is as follows:

[0072]

[0073] where x i Indicates the position of the center point of the building point cloud plane in the label, x i_voted represents the location of the polling station after voting, N face Indicates the number of actual building point cloud planes, N i_pv Represents x i_vote The number of voting points on the point cloud plane of the building;

[0074] S2.3 Aggregate the characteristics of the plane of the building point cloud: Since the number of planes in the building point cloud is unknown, the present invention assumes that there are 64 planes in the building point cloud, which is more than the number of planes in the actual building point cloud. The farthest point sampling method is used to sample 64 center points, denoted as K i , where i = 1, 2…64; then for each K i Clustering: Perform a ball query with a radius of 0.1 to obtain the 16 points closest to itself. If there are less than 16 points, copy the nearest point as a replacement for completion, recorded as K×16×C voted ; Then, an MLP is used to aggregate the features of the voting points to obtain the features K×C of the building point cloud plane face ;

[0075] S2.4 Building point cloud plane prediction: Based on the surface features obtained in step S2.3, the surface prediction is realized. The present invention uses three MLPs to realize the prediction of surface center point, surface normal, and surface confidence. The specific process is as follows: Figure 5 shown.

[0076] Set the predicted plane set of all building point clouds to F pre, the prediction of the center point of the building point cloud plane uses the mean square error to implement the constraint on the network; on the label, the center point of the minimum bounding box of the building point cloud plane obtained by step S1 is the center point of the building point cloud plane to which the voting point belongs; the prediction of the normal of the building point cloud plane uses the mean square error to implement the constraint on the network; the normal of the minimum bounding box of the building point cloud plane obtained by step S1 is the normal of the building point cloud plane to which the voting point belongs; the confidence prediction of the building point cloud plane uses the binary cross entropy to implement the constraint on the network; the similarity between the predicted building point cloud plane and the building point cloud plane in the label is the confidence label;

[0077] The similarity calculation process of the building point cloud plane is as follows Figure 6 As shown in the figure, the details are as follows: the vertical distance d between the two center points is calculated and then normalized. Since the point cloud ranges from [-1,1], the maximum distance between the two points is So the normalization process is Then, the normalized d is used to calculate the similarity. The specific formula is as follows:

[0078]

[0079] Among them, Sim center is the similarity of the center point of the building point cloud plane;

[0080] The cosine similarity of the normals of the two building point cloud planes is calculated, and then the absolute value of the cosine similarity is taken. At this time, the absolute value of the cosine similarity is assumed to be Cos sim , then the calculation formula of normal similarity is as follows:

[0081]

[0082] Among them, Sim normal is the normal similarity of the building point cloud plane;

[0083] The similarity of two building point cloud planes is calculated based on the vertical distance. First, a unit vector a that is not parallel to the normal of the building point cloud plane is taken outside the first building point cloud plane, and then the vector a and the normal are cross-producted to obtain a unit vector u in the building point cloud plane. Then, the vector u and the normal are cross-producted to obtain another unit vector v in the building point cloud plane that is perpendicular to u and the normal at the same time; then, taking the center point of the building point cloud plane as the starting point, four different points on the building point cloud plane are obtained by ±0.1 in the u and v directions; the same operation is used on the other building point cloud plane to obtain four different points on the building point cloud plane; finally, the four points on the two building point cloud planes are calculated in sequence according to the calculation method of the similarity of the center points of the building point cloud planes, and the results are averaged to obtain the similarity of the vertical distance: Sim distance ;

[0084] The proportions of the center point similarity, normal similarity, and vertical distance similarity of the building point cloud plane in the total similarity are 0.3, 0.4, and 0.3 respectively. The calculation formula is shown in the figure below:

[0085] Sim=0.3×Sim center +0.4×Sim normal +0.3×Sim distance (4)

[0086] Among them, Sim is the similarity between two building point cloud planes;

[0087] S2.5: Remove redundant building point cloud planes: In F pre According to the confidence level, the building point cloud planes with confidence level less than 0.9 are removed to obtain the coarsely filtered building point cloud plane set F k Next, for F k Do the following: From F k Take out the building point cloud plane with the highest confidence and add it to the building point cloud plane set F used to predict the adjacent relationship of the building point cloud plane nms In the k The similarity of the remaining building point cloud planes in F is calculated, and the building point cloud planes with similarity higher than 0.9 are removed from F k Repeat the above steps until F k There are no building point cloud planes left;

[0088] S2.6: Prediction of adjacent relationship of building point cloud planes: For the building point cloud plane set F obtained in step S2.5 nms To predict the adjacent relationship between two pairs, the specific process is as follows Figure 7As shown in Figure 1, a paired face attention module is first used to fuse the features of two building point cloud planes, and an MLP network is used to predict the adjacent relationship of the building point cloud planes; the paired face attention module consists of two MLPs, a sigmoid activation function, and a maximum pooling layer; F nms The building point cloud planes in the image are paired in pairs. After each building point cloud plane pair passes through the face attention module to fuse the building point cloud plane pair features, an MLP network is used to output the prediction result Face_pairs×1, where Face_pairs is the dimension of the number of building point cloud plane pairs. In terms of the loss function, the module uses binary cross entropy for calculation.

[0089] S3: Extracting building vertices: The specific process of extracting building vertices is as follows: Figure 8 As shown, based on step S.2.6, the adjacent probability of the building point cloud plane pair can be obtained: Face_pairs×1. The building point cloud plane pairs with the adjacent probability Face_pairs×1 median greater than 0.5 are assumed to be adjacent. At this time, X pairs of adjacent building point cloud plane pairs can be obtained. Then, triples are extracted from the X pairs of building point cloud plane pairs. The so-called triples refer to three adjacent building point cloud planes. Subsequently, the intersection of these three building point cloud planes is performed to obtain the vertex positions of the building. The vertex position set is denoted as M. i , where i = 1, 2…n, n is the number of building vertices; at the same time, the triple set of each building vertex is retained, denoted by M i _triple.

[0090] S4: Optimize building vertices. The specific process is as follows: Fig. 9 As shown: There are two main problems with the building vertices obtained in step S3. One is that building vertices that do not actually exist are generated, and the other is that duplicate building vertices are generated;

[0091] The reason why the building vertices that do not actually exist are generated is that the adjacent relationship in the label is determined by whether there are common points between two building point cloud planes, so some non-existent building vertices will be generated; however, the positions of these building vertices are far away from the building, so the present invention directly calculates M i The closest distance to the point cloud in the original point cloud data. If the distance is greater than 0.1, it means that the building vertex is not the actual vertex of the building, and it is directly removed from the vertex set M i Delete from;

[0092] The reason for generating duplicate vertices is that there may be more than three building planes near a vertex in the building structure, so the triplet of building vertices is not unique; due to calculation errors, M iThe building vertices obtained by the _triple intersection are not in the same position, but they are close to each other. Therefore, the duplicate building vertices can be removed by merging the building vertices within a distance of 0.05. The optimized building vertex set is set to V, and each vertex is set to V i , where i = 1, 2…n, where n is the number of retained building vertices; detect M i For all building vertices in the vertices, if the distance between building vertices is within 0.05, it means that they are repeated building vertices and need to be merged. The positions of the new vertices are directly averaged and stored in the building vertex set V. If the building vertices are not repeated, they can be directly stored in the vertex set V. When merging vertices, the triples M of these building vertices will be obtained. i _triple is also merged, and the plane set of the building point cloud after merging each building vertex triple is set to P, P i It is a set of building point cloud planes of optimized building vertices.

[0093] S5: Extract building edges: Based on the optimized vertices V and the building point cloud plane set P obtained in step S4, we can determine the building edges by i Building point cloud plane P i To determine whether two building vertices form a building edge. From the knowledge of geometric structure, we know that if two building vertices form an edge, then there must be two identical building planes near the two building vertices. Therefore, in this step, we can determine whether two building vertices form an edge by directly judging whether there are two or more coplanar surfaces in the building point cloud planes corresponding to the two building vertices. The building edge set is recorded as E i , where i = 1, 2…m, where m is the number of edges.

[0094] S6: Construct the polygonal mesh model of the building. The specific process is as follows Fig.10 As shown: the building vertices and building edges obtained based on step S4 and step S5 have formed a wireframe model of the building. It can be seen from step S5 that the generation of each building edge is determined by detecting whether there are two or more common faces; therefore, it is only necessary to obtain the edges and vertices on each building point cloud plane according to which edges are involved in the generation of the building point cloud plane; after obtaining the edges and vertices on the building point cloud plane, the vertices and edges of these buildings are formed into a graph, and then the depth-first search method is used to search for the largest loop therein, which is the sequential structure of the building polygonal faces composed of the building vertices and edges; then it is necessary to detect whether the vertex order is clockwise, if not, it needs to be flipped; after all the vertex orders are adjusted, the final output polygonal mesh model is obtained;

[0095] The clockwise detection algorithm for the building vertex sequence is: first, take the first three building vertices v1, v2, and v3 in the loop order, and then use these three building vertices to form two vectors in the building point cloud plane in the loop order: a = v2-v1, b = v3-v2, and then calculate the cross product of vectors a and b to obtain the normal vector direction vector normal of the building point cloud plane direction ; Then, take an adjacent face of the building point cloud plane, average the center points of the two building point cloud planes, and get a point p inside the building; use point p and v1 to form a vector pointing to the outside of the building: out direction =v1-p; then the vector normal direction and out direction Perform dot product. If the value is greater than 0, it means that the current loop is in a clockwise direction, otherwise it means that the order of building vertices is counterclockwise.

[0096] Example 2

[0097] The present embodiment relates to a lightweight surface reconstruction device for building point clouds based on deep learning, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement a lightweight surface reconstruction method for building point clouds based on deep learning of embodiment 1.

[0098] The contents described in the embodiments of this specification are merely an enumeration of the implementation forms of the inventive concept. The protection scope of the present invention should not be regarded as limited to the specific forms described in the embodiments. The protection scope of the present invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A lightweight surface reconstruction method for building point clouds based on deep learning, characterized in that: The following steps are involved: 1) Construct a data set; input a normalized building triangular mesh model, sample it into point cloud data with normals based on the barycentric coordinates, and calculate data labels such as the building point cloud plane, the adjacent relationship of the building point cloud plane, and the minimum bounding box of the building point cloud plane; 2) Predict the planes and adjacent relationships in the building point cloud. Input the point cloud data with normals, first use the PoingNet++ network combined with the self-attention mechanism to select voting points and extract features, then cluster the point cloud based on the Hough voting method, and then predict the planes in the building point cloud based on the clustering results, and then predict the adjacent relationships between the planes of the building point cloud; 3) Extract building vertices; According to the adjacent relationship between the planes of the building point cloud, the vertices of the building are obtained by intersection; 4) Optimize building vertices; Delete the building vertices that are not inside the building, and merge the duplicate building vertices to obtain an optimized building vertex set; 5) Extract building edges; Combine the optimized building vertices in pairs, and determine whether two building vertices are connected based on the building point cloud plane set corresponding to the building vertices, and obtain all edge sets in the building; 6) Construct a polygonal mesh model of the building; use a depth-first search method to obtain a clockwise loop of vertices in the building point cloud plane, and finally obtain a polygonal mesh model of the building.

2. The method for lightweight surface reconstruction of building point clouds based on deep learning according to claim 1, characterized in that: The step (1) of constructing a data set in claim 1 specifically includes: 11) Point cloud sampling based on barycentric coordinates: Input a normalized triangular mesh building data, whose vertex range is between -1 and 1, set the set of all triangular facets in the building to S, and let S i represents a triangle in the set, where i = 1, 2…n, n is the number of triangles; calculate the area of ​​each triangle, set to S i _area; Let the sampling density of the triangle patch be D, and the area of ​​each triangle patch S i The number of sampling points per triangle is obtained by multiplying _area and sampling density D, which is set to S i _sample; then, according to the principle of barycentric coordinates, S is randomly generated inside the triangle patch i _sample points, set to S i _points, S i _points is the sampling point cloud in each triangle patch. The above steps only sample the point cloud inside the triangle patch, but not the point cloud on the edge of the triangle patch. Therefore, 10 points are collected at equal intervals according to the length of the triangle patch edge, and these points are also added to S i _points; at the same time, calculate the normal of each triangle and set it to S i _normal, which is also the point set S i The normal of each point in _points; 12) Point cloud clustering based on building planes; First, roughly filter out the triangles on the same building plane according to the normals of the triangles and the vertical distances between the triangles. The specific contents of the rough screening are: 1) First, randomly select a triangle S from S, and cluster all the triangles in the triangle set S and S i _normal triangles with normal angles less than 5 degrees are taken out and temporarily classified into the same category, and the above operation is repeated until all triangles in S are classified; 2) Then the following operations are performed on the set of triangles classified in the previous step: a triangle is randomly taken out, and the vertical distances from the three vertices on the triangle to the other triangles are calculated and averaged to obtain the vertical distance between the triangles; finally, the triangles with vertical distances less than 0.05 between the triangles are temporarily divided into a set of triangles belonging to the same building plane to obtain the rough screening result, which is set as M; The divided triangle face set is then further screened. The specific steps are: (1) Take a triangle face from the triangle face set M and add it to the set M. ′ , (2) and then repeat the process of ′ Compare with M. Each time, take out all the triangles with common points from M and add them to M. ′ Until no new triangles can be taken out from M, then M ′ What is saved is the set of triangles belonging to the same building plane. (3) If there are still triangles left in M, repeat steps (1) and (2) until there are no triangles left in M. Through the above steps, the classification of S is realized, and the set of triangles belonging to the same building plane is set to G. j , where j = 1, 2…, m, m is the number of building planes in the building triangular mesh data. Since each triangular face corresponds to its own sampling point cloud, the sampling point cloud on each building plane can be further obtained, which is expressed as G j _points, which represents the point cloud on each building plane. At this time, the correspondence between the plane and the point cloud in the building is obtained; 13) Calculate the adjacent relationship between building planes; extract G j Then remove the duplicate edges to get the outer edge of the building plane, denoted as G j _edge, then extract G j The vertices in _edge are the vertices of the building plane, denoted by G j _vertices; Finally, compare the building planes in pairs to see if there are repeated vertices between the two building planes. If there are repeated vertices, it means that the two building planes are adjacent, and the adjacent relationship of the building planes in G can be obtained. Subsequently, the adjacent relationship of the planes in the building point cloud can be obtained from the corresponding relationship between the building planes and the point cloud; 14) Calculate the minimum bounding box of the building plane; j The edges with an angle less than 1.5 degrees in _edge are merged, and the length of the new edge is the sum of the lengths of the two edges; then the longest edge is selected as the main direction, and then the AABB algorithm is used to generate G j The minimum bounding box G j _box, the minimum bounding box is represented by four vertices; then, the minimum bounding box of the plane in the building point cloud can be obtained by the correspondence between the building plane and the point cloud; at the same time, the center point and normal of the building point cloud plane can be obtained by calculating the center point and normal of the bounding box.

3. The method for lightweight surface reconstruction of building point clouds based on deep learning according to claim 1, characterized in that: The prediction of the planes and the adjacent relations of the planes in the building point cloud described in step (2) specifically includes: 21) Select voting points and extract voting point features: The selection of voting points and the extraction of voting point features are carried out through the backbone network. The backbone network is designed based on the PointNet++ network model and consists of 4 Set Abstraction layers and 3 Feature Propagation layers. At the same time, a self-attention mechanism module is added to each SA layer to improve the feature extraction capability. The input of the network is N×(d+C), where N is the number of point clouds, d is the position of the point cloud, and C is the feature channel of the point cloud. The output after passing through the backbone network is N vote ×(d vote +C vote ), where N vote is the number of polling stations, d vote is the location of the polling station, C vote for the characteristics of the polling place; 22) Point cloud clustering based on Hough voting: The voting network consists of an MLP network, with input voting point N vote ×(d vote +C vote ), after the voting network, output N vote ×(x ivoted +C voted ), where C voted is the feature of the post-voting point, is the position of the center point of the building point cloud plane considered by the voting point; in the loss function, the mean square error is used; at the same time, in order to allow the voting points on the plane with a relatively small area in the building point cloud plane to vote to the correct center point position of the building point cloud plane, different weights are assigned to the voting points according to the area size of the building point cloud plane; the specific formula of the loss function of the voting network module is as follows: where x i Indicates the position of the center point of the building point cloud plane in the label, x i_voted represents the location of the polling station after voting, N face Indicates the number of actual building point cloud planes, N i_pv Represents x i_vote The number of voting points on the point cloud plane of the building; 23) Aggregate the features of the planes of the building point cloud: Since the number of planes in the building point cloud is unknown, it is assumed that there are 64 planes in the building point cloud, which is more than the number of planes in the actual building point cloud. vote × ivoted The farthest point sampling method is used to sample 64 center points, denoted as K i , where i = 1, 2…64; then for each K i Clustering: Perform a ball query with a radius of 0.1 to obtain the 16 points closest to itself. If there are less than 16 points, copy the nearest point as a replacement for completion, recorded as K×16×C voted ; Then, an MLP is used to aggregate the features of the voting points to obtain the features K×C of the building point cloud plane face ; 24) Building point cloud plane prediction: The input of this step is the features of the building point cloud plane obtained in step 23); three MLPs are designed to realize the prediction of the center point, normal line and confidence level of the plane in the building point cloud; in the loss function, the prediction of the center point and normal line of the building point cloud plane uses mean square error to constrain the network, and the prediction of the confidence level of the building point cloud plane uses binary cross entropy to constrain the network; the set of all predicted building point cloud planes is set to F pre ; The confidence of the predicted building point cloud plane is the similarity with the real building point cloud plane. The similarity is calculated from three aspects: the center point of the building point cloud plane, the normal line, and the vertical distance between the building point cloud planes; The similarity calculation method of the building point cloud plane is: calculate the vertical distance d between the two center points, and then normalize it. Since the point cloud range is [-1,1], the maximum distance between the two points is So the normalization process is Then, the normalized d is used to calculate the similarity. The specific formula is as follows: Among them, Sim center is the similarity of the center point of the building point cloud plane; The cosine similarity of the normals of the two building point cloud planes is calculated, and then the absolute value of the cosine similarity is taken. At this time, the absolute value of the cosine similarity is assumed to be Cos sim , then the calculation formula of normal similarity is as follows: Among them, Sim normal is the normal similarity of the building point cloud plane; The similarity of two building point cloud planes is calculated based on the vertical distance. First, a unit vector a that is not parallel to the normal of the building point cloud plane is taken outside the first building point cloud plane, and then the vector a and the normal are cross-producted to obtain a unit vector u in the building point cloud plane. Then, the vector u and the normal are cross-producted to obtain another unit vector v in the building point cloud plane that is perpendicular to u and the normal at the same time; then, taking the center point of the building point cloud plane as the starting point, four different points on the building point cloud plane are obtained by ±0.1 in the u and v directions; the same operation is used on the other building point cloud plane to obtain four different points on the building point cloud plane; finally, the four points on the two building point cloud planes are calculated in sequence according to the calculation method of the similarity of the center points of the building point cloud planes, and the results are averaged to obtain the similarity of the vertical distance: Sim distance ; The proportions of the center point similarity, normal similarity, and vertical distance similarity of the building point cloud plane in the total similarity are 0.3, 0.4, and 0.3 respectively. The calculation formula is shown in the figure below: Sim=0.3×SIm center +0.4×Sim normal +0.3×Try distance (4) Among them, Sim is the similarity between two building point cloud planes; 25) Remove redundant building point cloud planes: pre According to the confidence level, the building point cloud planes with confidence level less than 0.9 are removed to obtain the coarsely filtered building point cloud plane set F k Next, for F k Do the following: From F k Take out the building point cloud plane with the highest confidence and add it to the building point cloud plane set F used to predict the adjacent relationship of the building point cloud plane nms In the k The similarity of the remaining building point cloud planes in F is calculated, and the building point cloud planes with similarity higher than 0.9 are removed from F k Repeat the above steps until F There is no building point cloud plane left in k; 26) Prediction of adjacent relationship of building point cloud planes: The prediction of adjacent relationship of building point cloud planes first uses a paired face attention module to fuse the features of two building point cloud planes, and uses an MLP network to realize the prediction of adjacent relationship of building point cloud planes; the paired face attention module consists of two MLPs, a sigmoid activation function and a maximum pooling layer; F nms The building point cloud planes in the image are paired in pairs. After each building point cloud plane pair passes through the face attention module to fuse the building point cloud plane pair features, an MLP network is used to output the prediction result Face_pairs×1, where Face_pairs is the dimension of the number of building point cloud plane pairs. In terms of the loss function, the module uses binary cross entropy for calculation.

4. The method for lightweight surface reconstruction of building point clouds based on deep learning according to claim 1, characterized in that: The step (3) of extracting building vertices specifically includes: assuming that the building point cloud plane pairs whose median of adjacent probability Face_pairs×1 is greater than 0.5 are in an adjacent relationship, at this time, X pairs of adjacent building point cloud plane pairs can be obtained; then extracting triplets from the X pairs of building point cloud plane pairs, the so-called triplets refer to three building point cloud planes that are adjacent to each other; then, the intersection of these three building point cloud planes is performed to obtain the vertex position of the building, and the vertex position set is set to M i , where i = 1, 2…n, n is the number of building vertices; at the same time, the triple set of each building vertex is retained, denoted by M i _triple.

5. The method for lightweight surface reconstruction of building point clouds based on deep learning according to claim 1, characterized in that: The specific contents of optimizing the building vertices described in step (4) include: directly calculating M i The closest distance to the point cloud in the original point cloud data. If the distance is greater than 0.1, it means that the building vertex is not the actual vertex of the building. i Delete from; The optimized building vertex set is set to V, and each vertex is set to V i , where i = 1, 2…n, where n is the number of retained building vertices; detect M i For all building vertices in the vertices, if the distance between building vertices is within 0.05, they are repeated building vertices and need to be merged. The positions of the new vertices are directly averaged and stored in the building vertex set V. If the building vertices are not repeated, they can be directly stored in the vertex set V. When merging vertices, the triples M of these building vertices will be obtained. i _triple is also merged, and the plane set of the building point cloud after merging each building vertex triple is set to P, P i It is a set of building point cloud planes of optimized building vertices.

6. The method for lightweight surface reconstruction of building point clouds based on deep learning according to claim 1, characterized in that: The method for extracting building edges in step (5) specifically includes: firstly, combining the building vertices in V in pairs, and then checking whether the two building vertices in each building vertex pair have two or more identical building point cloud planes in the corresponding building point cloud plane set; if so, the two building vertices are connected and form an edge, and the building edge set is recorded as E i , where i = 1, 2…m, where m is the number of building edges.

7. The method for lightweight surface reconstruction of building point clouds based on deep learning according to claim 1, characterized in that: The step (6) of constructing the polygonal mesh model of the building specifically includes: the building edge set E and the building vertex set V have already formed a wireframe model of the building. At this time, traversing the building point cloud plane set F nms , obtain the building vertices and building edges on the building point cloud plane, then form these building vertices and edges into a graph, and then use the depth-first search method to obtain the maximum loop, which is the sequential structure of the building polygonal plane composed of the building vertices and edges on the building point cloud plane; then check whether the order of the building vertices is clockwise, if not, it needs to be flipped; The clockwise detection algorithm for the building vertex sequence is: first, take the first three building vertices v1, v2, and v3 in the loop order, and then use these three building vertices to form two vectors in the building point cloud plane in the loop order: a = v2-v1, b = v3-v2, and then calculate the cross product of vectors a and b to obtain the normal vector direction vector normal of the building point cloud plane direction ; Then, take an adjacent face of the building point cloud plane, average the center points of the two building point cloud planes, and get a point p inside the building; use point p and v1 to form a vector pointing to the outside of the building: out direction =v1-p; then the vector normal direction and out direction Perform dot product. If the value is greater than 0, it means that the current loop is in a clockwise direction. Otherwise, it means that the order of building vertices is in a counterclockwise direction. After adjusting the order of all building vertices, the final output polygonal mesh model is obtained.

8. A lightweight surface reconstruction device for building point clouds based on deep learning, characterized in that: It comprises a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used to implement a lightweight surface reconstruction method of a building point cloud based on deep learning as described in any one of claims 1-7.

Images