Method for constructing building layer household-level entity through laser point cloud fusion and combined surveying and mapping

Abstract

The invention provides a method for constructing a building layer household-level entity through laser point cloud fusion and combined surveying and mapping. The method comprises the following steps that a graph contour is converted into a point set; generating a primitive center point set, performing semantic fusion on completion acceptance measurement result attributes and real estate measurement result attributes based on a spatial relationship, and calculating surface primitives of real estate measurement to generate center points; fusing multi-modal point cloud data, and making up and correcting outer contour points of the building; building contour graph segmentation and extraction: taking the point cloud semantic information converted from the joint surveying and mapping result as a label for point cloud segmentation and extraction, segmenting fused point cloud data, extracting a building edge contour and an internal layering and household line, and reconstructing to generate building, floor and household entities; combined surveying and mapping data are converted into point cloud to form multi-modal point cloud data, and then the multi-modal point cloud data are fused with airborne laser radar, oblique photography and other data, so that accurate construction of a building, floor and even household entity model is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud maps, and in particular to a method for constructing entity models of building floors and households by fusing laser point clouds and joint surveying and mapping.

Background Art

[0002] Urban modeling and refined urban management have become research hotspots in recent years. The semantic extraction system for urban facilities can provide important support for urban modeling and urban management. Among them, real-scene three-dimensional is a digital virtual space that truly, stereoscopically, and time-sequentially reflects and expresses the production, life, and ecological space of human beings; the urban-level real-scene three-dimensional is not only an important part of comprehensively constructing the real-scene three-dimensional China, but also a unified and authoritative three-dimensional spatio-temporal information base for facing urban digital twin construction, refined management, emergency management, and three-dimensional stereoscopic management of natural resources. With the continuous development and maturity of oblique photography and laser point cloud three-dimensional modeling technologies, they have become two major mainstream technical means for real-scene three-dimensional urban construction in China.

[0003] Among them, the real-scene three-dimensional model of oblique photography is known for its high precision, high realism, all-round perception, and high degree of process automation, but it has insurmountable defects in model processing such as weak texture features, fine texture features, and far-surface texture features; while the laser point cloud is superior in its strong penetration, high precision, high efficiency, and diverse collection platforms and methods, but it also has deficiencies such as lack of texture features, large workload of in-house texture mapping, and incomplete small features of ground objects. By fusing oblique photography and airborne and vehicle-borne point cloud data to construct building entities, in complex urban scenes where the shapes, styles, and heights of buildings are different, the occlusion between buildings and the occlusion of buildings by other entities will cause missing building data, affecting the quality of the building three-dimensional model. Moreover, the buildings constructed by this method do not have the spatial segmentation data of each floor and household inside the building, and at the granularity level, only the entire building can be achieved, which cannot meet the application service requirements for urban refined management, emergency disaster reduction, three-dimensional stereoscopic management of natural resources, etc. If we want to achieve the granularity level of each floor and household for the building three-dimensional model, fine modeling by indoor laser scanning is not only costly and time-consuming, but also faces the problem of data loss caused by occlusion.

[0004] In the process of urban construction, through the management of construction projects, each functional competent department has accumulated a large amount of business topic data related to buildings, such as graphic results of joint surveying and mapping (real estate surveying and mapping, completion acceptance surveying, civil air defense surveying), etc. These graphic data not only have clear contour boundaries for the description of buildings, but also have high spatial accuracy, rich semantic attributes, and contain measurement value information such as building height and elevation that are suitable for constructing three-dimensional models. Although these data are drawn according to multiple business management rules and are not completely consistent with the real-scene three-dimensional building model in terms of spatial overlap, they can be fully used to correct and make up for the problems of reduced result accuracy and incomplete spatial shape expression caused by data loss due to occlusion in the collection of real-scene three-dimensional data.

[0005] At present, these topic business data have not been applied to the construction of real-scene three-dimensional fine models. How to quickly fuse the existing topic business data with the collected oblique models and airborne lidar data to construct building building, floor, and household entities with low cost, fine granularity, and high spatial accuracy has become a common problem in the industry, and there is an urgent need to propose a new technical method to solve the above problems.

Summary of the Invention

[0006] The present invention provides a method for constructing building building, floor, and household-level entities by fusing lidar point clouds with joint surveying and mapping. By converting joint surveying and mapping data into point clouds to form multi-modal point cloud data and then fusing data such as airborne lidar and oblique photography, accurate construction of building building, floor, and even household entity models can be achieved; it can not only achieve the construction of finer-grained and more accurate entities, solve the automatic adaptation problems of geometric semantics, attribute semantics, and determination of expression granularity in the construction of urban fine building entity models, but also make full use of the rich topic data existing in natural resources to realize the construction of real-scene three-dimensional fine building entities.

[0007] To achieve the above object, the following technical solutions are adopted:

[0008] A method for constructing building building, floor, and household-level entities by fusing lidar point clouds with joint surveying and mapping includes the following steps:

[0009] Step S1. Converting graphic contours into point sets

[0010] Convert the spatial surface primitives of real estate and completion acceptance surveying results into a point set representing their contour features;

[0011] Step S2. Generating a central point set of primitives

[0012] Semantically fuse the attributes of completion acceptance surveying results and real estate surveying results based on spatial relationships, and calculate the central points of the surface primitives of real estate surveying;

[0013] Assign the fused semantic attribute information to the center point, and the center point attributes will contain the floor number, room number, storey height, and floor slab elevation value attribute information;

[0014] Step S3. Multimodal point cloud data fusion to compensate for and correct the building outer contour points

[0015] Using the multimodal point cloud matching algorithm and the boundary contour constraint algorithm model, fuse the point cloud generated above with the point cloud generated by the radar field survey and the point cloud generated by the oblique model, remove the noise points, and correct the building outer contour points generated by the airborne radar point cloud;

[0016] Step S4. Building contour graph segmentation and extraction

[0017] Convert the point cloud semantic information obtained from the joint surveying and mapping results into the label for point cloud segmentation and extraction, segment the fused point cloud data, extract its building edge contour and internal hierarchical household lines, and reconstruct the building building, floor, and household entities.

[0018] Furthermore, the specific steps in the step S1 further include the following steps:

[0019] Step S1.1, first of all, in order to convert the joint surveying and mapping primitive data into a three-dimensional point cloud, it is necessary to first calculate the edge of the approximate area of the polygon for the primitive data;

[0020] Step S1.2, then, convert the polygon base points it has into a contour point set, and calculate the center point for each polygon to construct a center point set to store the entity semantic attributes of the current polygon;

[0021] Among them, use the Halcon operator to calculate the polygon matrix R. The rows and columns of each matrix represent the coordinates of the key points of the multi-variable sequence, and each polygon represents a connected area in the primitive data;

[0022] Although due to the strong accuracy of the surveying and mapping primitive data, the measurement error can be ignored, but in order to adapt to a wider range of applications, the Halcon operator introduces an error parameter Hausdorff distance h to calculate the maximum distance between the polygon area and the primitive closed area, so that the two areas can intersect as much as possible, as shown in formula (1):

[0023]

[0024] In formula (1), U represents the area of the surveying and mapping primitive data, d(·) calculates the Euclidean distance between two sets, and in this way, the non-empty subset of the metric space itself is transformed into a metric space for convenient subsequent calculations;

[0025] More importantly, if the calculated error parameter is smaller, the more key points of the contour are generated in the vector graphics. Therefore, an error parameter as small as possible is set to generate more key points to make the generated point cloud denser.

[0026] Further, the step S2 specifically further includes the following steps:

[0027] Step S2.1, in order to convert a planar polygon into spatial points, the spatial information it has must be considered;

[0028] The combined mapping primitive data attribute information contains accurate three-dimensional information. Calculate the center point for each polygon, assign the elevation information in its corresponding mapping primitive data attribute to the primitive data, and then convert it into the center point of the corresponding key point polygon, and use the elevation information it has as the attribute value of the key point;

[0029] Step S2.2, divide the grid in the two-dimensional plane and fill the key points in the two-dimensional grid. However, since it cannot be guaranteed that each point is on a regular grid point, it is necessary to resample the planar points and generate a completely regular dense grid point graph;

[0030] Define the resampling problem as an assignment problem, assign a key point to each grid point for filling, and require that the total movement cost is minimized. The specific processing method is shown in formula (2):

[0031]

[0032] The purpose of formula (2) is to optimize a set of permutation matrices W g and W i l , to uniquely assign the global feature F G and the local feature to the predefined two-dimensional grid points, where represents the mapping of W g relative to F G , As a redundant grid representation, only some grid points form a corresponding relationship with , N G is the dimension of the permutation matrix and the grid; formula (2) ensures that in the global and local neighborhoods, only a subset of grid points form a one-to-one context point, and all missing matching grid points are filled by the nearest neighbors. The matched points are mapped to the three-dimensional space according to the elevation values they have to form a dense point cloud shape.

[0033] Further, the step S3 specifically further includes the following steps:

[0034] The vertex set and edge set of the building outline point cloud are used to combine and jointly survey the floor household point cloud of the building for building outline correction.

[0035] Step S3.1. First, use the point cloud P scanned by the airborne lidar s and the point cloud P generated by the joint surveying and mapping primitive t to construct graph models G s and graph model G t ;

[0036] Match the vertices in the two graphs through the Hungarian algorithm. In the matching, the number of vertices in the point cloud contour graph must be greater than the number of vertices of the joint surveying and mapping primitive. Use the Euclidean distance to connect the predicted points with the primitive points;

[0037] Step S3.2. Use the graph matching algorithm to find the matching relationship between the point cloud P s and the point cloud P t ;

[0038] Use a multi-layer perceptron to extract the point features of the point cloud P s and encode them into a one-dimensional point sequence;

[0039] For the point cloud P t , since it is a diffusion grid point filled by neighboring points, therefore, use circular convolution to extract the structured feature sequence. The first-order graph matching similarity matrix is directly obtained by multiplying the first-order feature points in formula (3):

[0040] M p =U 1 U 2T ......(3)

[0041] To obtain the second-order similarity, first connect the two nodes corresponding to each edge to construct the second-order feature matrices of the two graphs, as shown in formula (4):

[0042]

[0043] Among them, for the first-order similarity in formula (3), U 1 and U 2 are the feature vectors of each node respectively, and the superscript represents different data sources;

[0044] For the second-order similarity in formula (4), X and Y are the edge feature matrices respectively. The construction method is that for the c-th edge starting from the i-th node and ending at the j-th node, set the descriptor of this edge as the concatenated matrix of the descriptors of the two nodes, W 2is a learnable parameter, so the second-order similarity has a learnable matching function; then, the problem of solving graph matching is approximated as the problem of solving the eigenvector corresponding to the largest eigenvalue of the similarity matrix, where power iteration is used for calculation;

[0045] The matching result of the graph is represented by a doubly stochastic matrix, which intuitively reflects the possibility of establishing a matching relationship between any pair of nodes.

[0046] First, normalize the matrix column by column, and then normalize it row by row. Calculate the doubly stochastic matrix through multiple iterations. However, since the element values in the same row and the same column are not very different, in order to increase the obvious difference and facilitate the calculation of the subsequent loss function, multiply the matrix by a constant and then use Softmax processing to obtain the probability matrix of each candidate node matching, and this matrix still satisfies the doubly stochastic property; finally, use the method based on the offset vector to calculate the loss function, calculate the offset between the matching points, and then perform weighted summation according to the probability matrix, so as to predict the offset vector and minimize the difference between the predicted vector and the true offset. The loss function is shown in formula (5):

[0047]

[0048] where d i and respectively represent the predicted and true offsets from the far point to the correct matching point. Φ(·) is a penalty term. In the calculation process of the final loss function, the discarded points do not affect the calculation of the loss function; through the above method, the airborne lidar point cloud and the joint mapping building floor household point cloud are matched and fused, and the fused point cloud is projected onto a two-dimensional plane to extract the hierarchical information;

[0049] The boundary contour of the building floor should satisfy the manifold hypothesis, where each vertex can only be connected to two adjacent edges;

[0050] Given the edge set generated in the previous step, select a subset of edges for calculation by using the constrained integer programming formula; x i ∈ {0, 1} is defined as a binary variable to describe whether the edge e i ∈ E is activated as part of the boundary state; the set of activated edges constitutes the polygonal boundary of the layer entity;

[0051] In addition, an energy function is also used to constrain the generation quality of the boundary contour, as shown in formula (6):

[0052] U(x) = (1 - λ)U fidelity (x) + λU complexity (x).......(6)

[0053] In formula (6), U fidelity(x) describes the consistency between state x and the input data, U complexity (x) measures the complexity of the output boundary shape, both are in the interval [0, 1], and λ ∈ (0, 1); U fidelity (x) evaluates the coherence between the activation edge and the boundary shape. Since the boundary shape divides the entire space into inner and outer domains, most points are inside the boundary shape, and a fidelity constraint is imposed as shown in Equation (7):

[0054] U fidelity (x) = βU points (x) + (1 - β)U walls (x)......(7)

[0055] The first term measures the percentage of input points enclosed by the boundary to retain different proportions of internal cells occupied by the input points, where β ∈ (0, 1); at the same time, the second term is added to measure the overlap between the edge and the actual wall to constrain the edge;

[0056] Furthermore, to control the complexity term of the final boundary shape, as shown in Equation (8):

[0057]

[0058] In Equation (8), |V| is the set of calculated vertices, and this term returns a boundary shape polygon with a small number of corner vertices. Γ is a discriminant used to determine whether the vertex v i ∈ V is a corner vertex; however, since the building facade may contain inclined planes, therefore, the average height of each point needs to be calculated and then compared with the single-layer height value in the entity attributes of the joint mapping primitive layer;

[0059] Then, use RANSAC to detect the planes with inconsistent z - axes from the point cloud, calculate the average height of the support point set corresponding to each plane, merge the points with an average height lower than the single - layer height, and merge the face polygons.

[0060] Furthermore, the specific steps in step S4 further include the following steps:

[0061] Step S4.1, the joint mapping primitive gives the instance labels of the layer units. Therefore, the layer segmentation problem is transformed into a Markov random field problem through an energy problem in standard form, and the energy function is as shown in Equation (9):

[0062]

[0063] In Equation (9), γ ∈ (0, 1) is a balancing term, E is all adjacent internal faces, j and k represent specific faces, and m is the total number of faces; D(l k)Assign a label to the closed surface that is consistent with the instance label of the layer number; for the pair V(l j , l k ), first return a planar cell with low complexity, and second, perform stratification through the shape within the boundary;

[0064] For the final layer segmentation, solve the above Markov problem through the map cutting optimization technology combined with the surveying and mapping primitive data. First, merge adjacent faces into large polygon faces, with each face representing a layer, and then simplify the planar graph by deleting vertices that are not connected to collinear edges to achieve accurate segmentation of the building facade layer household entities.

[0065] Advantages of the present invention:

[0066] Aiming at the main technical bottlenecks in the construction of fine real - scene 3D models in complex urban scenarios, such as the low semantic accuracy of data, difficult automated modeling, and high cost of point cloud annotation caused by incomplete single - data angles and insufficient data attributes;

[0067] Utilize data from combined surveys (real estate surveying and mapping, completion acceptance surveying, civil air defense surveying, etc.), convert them into point clouds, form multi - modal point cloud data, and then fuse data such as airborne lidar and oblique photography to achieve accurate construction of building building, layer, and even household entity models; convert the real estate and completion acceptance surveying results in the combined surveying and mapping results into 3D point cloud data, perform mutual optimization and mutual constraint with the measured airborne lidar point cloud, the optimized data is fitted into vector graphics, and after being associated with semantic attributes, perform segmentation of entities at different levels; use it to supplement the missing of airborne laser point cloud data, and also achieve the method of correcting the spatial shape, accuracy of the building and enriching the semantic attributes of the building, realize the fusion of thematic data and laser point cloud to construct urban - level fine building building, layer, and household entity models, and give full play to the great value of natural resource stock data. It can not only achieve the construction of more fine - grained and accurate entities, solve the automatic adaptation problems of geometric semantics, attribute semantics, and expression granularity determination when constructing urban fine building entity models, but also make full use of the existing rich thematic data of natural resources to realize the construction of real - scene 3D fine building entities.

[0068]

Description of the Drawings

[0069] Figure 1 is the implementation flowchart of the present invention.

Detailed Embodiment

[0070] The following further illustrates the content of the present invention through specific examples.

[0071] A method for constructing building building, layer, and household - level entities by fusing laser point cloud and combined surveying and mapping, as Figure 1 shown, includes the following steps:

[0072] Step S1. Convert the graphic outline into a point set

[0073] Convert the spatial surface primitives of the real estate and the completion acceptance measurement results into a point set representing their contour features;

[0074] Step S1.1. First, in order to convert the joint surveying and mapping primitive data into 3D point clouds, it is necessary to first calculate the edges of the approximate polygon area for the primitive data;

[0075] Step S1.2. Then, convert the polygon base points it has into a contour point set, and calculate the center point for each polygon to construct a center point set to store the entity semantic attributes of the current polygon;

[0076] Among them, use the Halcon operator to calculate the polygon matrix R. The rows and columns of each matrix represent the coordinates of the key points of the polygon sequence, and each polygon represents a connected area in the primitive data;

[0077] Although due to the strong accuracy of the surveying and mapping primitive data, the measurement error can be ignored, but in order to adapt to a wider range of applications, the Halcon operator introduces an error parameter Hausdorff distance h to calculate the maximum distance between the polygon area and the closed area of the primitive, so that the two areas can intersect as much as possible, as shown in formula (1):

[0078]

[0079] In formula (1), U represents the area of the surveying and mapping primitive data, d(·) calculates the Euclidean distance between two sets, and in this way, the non-empty subsets of the metric space are transformed into the metric space itself to facilitate subsequent calculations;

[0080] More importantly, if the calculated error parameter is smaller, then the more contour key points are generated in the vector graph. Therefore, an error parameter as small as possible is set to generate more key points to generate a denser point cloud;

[0081] Step S2. Generate the primitive center point set

[0082] Fuse the semantic attributes of the completion acceptance measurement results and the real estate measurement results based on the spatial relationship, and calculate the center points for the surface primitives of the real estate measurement;

[0083] Assign the fused semantic attribute information to the center points, and the center point attributes will contain attribute information such as floor number, room number, floor height, and floor slab elevation value;

[0084] Step S2.1. In order to convert the planar polygon into spatial points, it is necessary to consider the spatial information it has;

[0085] The combined surveying and mapping graphic element metadata contains precise three-dimensional information. Calculate the center point for each polygon, assign the elevation information in the corresponding surveying and mapping graphic element metadata to the graphic element data, and then convert it into the center point of the corresponding key point polygon. Use the elevation information it has as the attribute value of the key point;

[0086] Step S2.2, divide the grid in the two-dimensional plane and fill the key points in the two-dimensional grid. However, since it cannot be guaranteed that each point is on a regular grid point, it is necessary to resample the plane points and generate a completely regular dense grid point graph;

[0087] Define the resampling problem as an assignment problem, assign a key point to each grid point for filling, and require that the total movement cost is minimized. The specific processing method is shown in formula (2):

[0088]

[0089] The purpose of formula (2) is to optimize a set of permutation matrices W g and W i l , to uniquely assign the global feature F G and the local feature to the predefined two-dimensional grid points, where represents the mapping of W g relative to F G , As a redundant grid representation, only some grid points form a corresponding relationship with , N G is the dimension of the permutation matrix and the grid; formula (2) ensures that in the global and local neighborhoods, only a subset of grid points form one-to-one context points, and all missing matching grid points are filled by the nearest neighbors. The matched points are mapped to the three-dimensional space according to the elevation values they have to form a dense point cloud shape;

[0090] Step S3. Multimodal point cloud data fusion to compensate for and correct the building outer contour points

[0091] Use the multimodal point cloud matching algorithm and the boundary contour constraint algorithm model to fuse the point cloud generated above with the point cloud generated by the radar field survey and the point cloud generated by the oblique model, and remove the noise points to correct the building outer contour points generated by the airborne radar point cloud;

[0092] The vertex set and edge set of the building outer contour point cloud are used to combine with the floor household point cloud of the combined surveying and mapping building to correct the building outer contour;

[0093] Step S3.1, first, use the point cloud P scanned by the airborne lidar sand the point cloud P generated by the joint surveying and mapping primitive t Construct the graph models G s and the graph model G t ;

[0094] Match the vertices in the two graphs through the Hungarian algorithm. In the matching, the number of vertices in the point cloud contour graph must be greater than the number of vertices of the joint surveying and mapping primitive, and use the Euclidean distance to connect the predicted points with the primitive points;

[0095] Step S3.2, use the graph matching algorithm to find the matching relationship between the point cloud P s and the point cloud P t ;

[0096] Use a multi-layer perceptron to extract the point features of the point cloud P s and encode them into a one-dimensional point sequence;

[0097] For the point cloud P t , since it is a diffusion grid point formed by filling with neighboring points, therefore, use circular convolution to extract the structured feature sequence, and the first-order graph matching similarity matrix is directly obtained by multiplying the first-order feature points in formula (3):

[0098] M p =U 1 U 2T ......(3)

[0099] To obtain the second-order similarity, first connect the two nodes corresponding to each edge to construct the second-order feature matrices of the two graphs, as shown in formula (4):

[0100]

[0101] Among them, for the first-order similarity in formula (3), U 1 and U 2 are the feature vectors of each node respectively, and the superscript represents different data sources;

[0102] For the second-order similarity in formula (4), X and Y are the edge feature matrices respectively, and their construction method is that for the c-th edge starting from the i-th node and ending at the j-th node, set the descriptor of this edge as the concatenation matrix of the descriptors of the two nodes, and W 2 is a learnable parameter, so the second-order similarity has a learnable matching function; then, approximate the solution problem of graph matching as the problem of finding the eigenvector corresponding to the largest eigenvalue of the similarity matrix, where power iteration is used for calculation; then, approximate the solution problem of graph matching as the problem of finding the eigenvector corresponding to the largest eigenvalue of the similarity matrix, where power iteration is used for calculation;

[0103] The matching results of the figures are represented by a doubly stochastic matrix, which intuitively reflects the possibility of establishing a matching relationship between any pair of nodes;

[0104] First, the matrix is normalized column-wise and then row-wise. The doubly stochastic matrix is calculated through multiple iterations. However, since the element values in the same row and the same column are not very different, to increase the obvious differences and facilitate the subsequent calculation of the loss function, the matrix is multiplied by a constant and then processed using softmax to obtain the probability matrix for each candidate node to be matched, and this matrix still satisfies the doubly stochastic property. Finally, a method based on the offset vector is used to calculate the loss function, calculate the offset between the matching points, and then perform a weighted sum according to the probability matrix, so as to predict the offset vector and minimize the difference between the predicted vector and the true offset, as shown in formula (5):

[0105]

[0106] where, d i and respectively represent the predicted and true offsets from the origin point to the correct matching point. Φ(·) is a penalty term. In the calculation process of the final loss function, the discarded points do not affect the calculation of the loss function. Through the above method, the airborne lidar point cloud and the joint mapping building floor household point cloud are matched and fused, and the fused point cloud is projected onto a two-dimensional plane to extract the hierarchical information;

[0107] The boundary contour of the building floor should satisfy the manifold hypothesis, where each vertex can only be connected to two adjacent edges;

[0108] Given the edge set generated in the previous step, a subset of edges is selected for calculation by using the constrained integer programming formula; x i ∈ {0, 1} is defined as a binary variable to describe whether the edge e i ∈ E is activated as part of the boundary state; the set of activated edges constitutes the polygonal boundary of the layer entity;

[0109] In addition, an energy function is also used to constrain the generation quality of the boundary contour, as shown in formula (6):

[0110] U(x) = (1 - λ)U fidelity (x) + λU complexity (x).......(6)

[0111] In formula (6), U fidelity (x) describes the consistency between the state x and the input data, and U complexity (x) measures the complexity of the output boundary shape. Both terms are in the interval [0, 1], and λ ∈ (0, 1); U fidelity(x) Evaluate the coherence between the activation boundary and the boundary shape. Since the boundary shape divides the entire space into internal and external domains, most points are located within the boundary shape. A fidelity constraint is imposed as shown in Equation (7):

[0112] U fidelity (x) = βU points (x) + (1 - β)U walls (x)......(7)

[0113] The first term measures the percentage of input points enclosed by the boundary to retain different proportions of internal cells occupied by the input points, where β ∈ (0, 1); at the same time, the second term is added to measure the overlap between the edge and the actual wall to constrain the edge.

[0114] Furthermore, to control the complexity term of the final boundary shape, as shown in Equation (8):

[0115]

[0116] In Equation (8), |V| is the set of calculated vertices. This term returns a boundary shape polygon with a small number of corner vertices. Γ is a discriminant used to determine whether vertex v i ∈ V is a corner vertex; however, since the building facade may contain inclined planes, it is necessary to calculate the average height of each point and then compare it with the single - layer height value in the entity attributes of the joint mapping primitive layer.

[0117] Then, use RANSAC to detect planes with inconsistent z - axes from the point cloud, calculate the average height of the support point set corresponding to each plane, merge the points with an average height lower than the single - layer height, and merge the face polygons.

[0118] Step S4. Building contour graph segmentation and extraction

[0119] Use the point - cloud semantic information converted from the joint mapping results as the label for point - cloud segmentation and extraction. Segment the fused point - cloud data and extract its building edge contour and internal hierarchical household lines, and reconstruct and generate building entity of building blocks, floors, and households.

[0120] Step S4.1. Since the joint mapping primitive gives instance labels for layer units, the layer segmentation problem is transformed into a Markov random field problem through an energy problem in standard form, as shown in Equation (9):

[0121]

[0122] In Equation (9), γ ∈ (0, 1) is a balancing term, E is all adjacent internal faces, j and k represent specific faces, and m is the total number of faces; D(l k)Assign a label to the closed surface that is consistent with the layer number instance label; pair V(l j , l k )First, return a planar cell with low complexity;

[0123] Secondly, perform layering through the shape within the boundary;

[0124] For the final layer segmentation, solve the above Markov problem through the map cutting optimization technology of the combined surveying and mapping primitive data. First, merge adjacent faces into large polygon faces, with each face representing a layer. Then, simplify the planar graph by deleting vertices that are not connected to collinear edges to achieve accurate segmentation of the building facade layer household entities.

[0125] In this embodiment, data such as combined surveying and mapping (real estate surveying and mapping, completion acceptance surveying, civil air defense surveying) are utilized, converted into point clouds, and multi-modal point cloud data are formed. Then, data such as airborne lidar and oblique photography are fused to achieve accurate construction of building building, layer, and even household entity models; the real estate and completion acceptance surveying results in the combined surveying and mapping results are converted into three-dimensional point cloud data, which are mutually optimized and mutually constrained with the measured airborne lidar point clouds. The optimized data are fitted into vector graphics and segmented into entities at different levels after being associated with semantic attributes; this is used to supplement the missing of the airborne lidar point cloud data, and also to achieve the method of correcting the building spatial shape, accuracy, and enriching the building semantic attributes, so as to realize the fusion of thematic data and lidar point clouds to construct urban-level fine building building, layer, and household entity models, giving full play to the great value of the natural resource inventory data.

[0126] At the same time, it can not only achieve the construction of more fine-grained and accurate entities, solve the automatic adaptation problem of determining geometric semantics, attribute semantics, and expression granularity when constructing urban fine building entity models, but also make full use of the existing rich thematic data of natural resources to achieve the construction of real-scene three-dimensional fine building entities.

[0127] The above-described embodiments are only preferred embodiments of the present invention and do not limit the scope of implementation of the present invention. Except for the situations listed in the specific embodiments; all equivalent changes made according to the method and principle of the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for constructing building floor and household entities by laser point cloud fusion and joint surveying, characterized in that: The following steps are involved: Step S1. Convert the graphic outline into a point set Convert the spatial surface primitives of real estate and completion acceptance measurement results into a set of points representing their contour features; Step S2. Generate a set of primitive center points The attributes of the completion acceptance measurement results and the real estate measurement results are semantically fused based on spatial relationships, and the surface elements of the real estate measurement are calculated to generate the center point; The fused semantic attribute information is assigned to the center point, and the center point attributes will contain the attribute information of floor, room number, floor height and floor elevation value; Step S3. Multimodal point cloud data fusion, compensation and correction of building outline points The point cloud generated above is fused with the point cloud generated by the radar field scan and the point cloud generated by the tilt model using a multimodal point cloud matching algorithm and a boundary contour constraint algorithm model, and noise points are removed to correct the building outline points generated by the airborne radar point cloud. Step S4. Segmentation and extraction of building outlines The point cloud semantic information converted from the joint surveying and mapping results is used as the label for point cloud segmentation and extraction. The fused point cloud data is segmented, and the building edge contours and internal stratification and household lines are extracted to reconstruct the building, floor, and household entities.

2. According to claim 1, a method for constructing a building floor and household entity by laser point cloud fusion and joint surveying and mapping, characterized in that: The step S1 specifically also includes the following steps: Step S1.1, first, in order to convert the joint surveying and mapping metadata into a three-dimensional point cloud, it is necessary to calculate the polygonal approximate area edge for the metadata; Step S1.2, then, convert the polygon base points into a contour point set, and calculate the center point for each polygon to construct a center point set to store the entity semantic attributes of the current polygon; Among them, the Halcon operator is used to calculate the polygon matrix R, each row and column of the matrix represents the coordinates of the key points of the variable sequence, and each polygon represents a connected area in the image data; Although the measurement error can be ignored due to the high accuracy of the surveying and mapping metadata, in order to adapt to a wider range of applications, the Halcon operator introduces an error parameter, the Hausdorff distance h, to calculate the maximum distance between the polygon area and the closed area of ​​the primitive, so that the two areas can intersect as much as possible, as shown in formula (1): In formula (1), U represents the area of ​​the mapping metadata, and d(·) calculates the Euclidean distance between the two sets. In this way, the non-empty subset of the metric space itself is transformed into the metric space to facilitate subsequent calculations; More importantly, if the calculated error parameter is smaller, more contour key points are generated in the vector graphics. Therefore, setting the error parameter as small as possible generates more key points and generates a denser point cloud.

3. According to claim 1, a method for constructing a building floor and household entity by laser point cloud fusion and joint surveying and mapping, characterized in that: The step S2 specifically also includes the following steps: Step S2.1, in order to convert the polygons of the plane into spatial points, the spatial information they possess must be considered; The joint mapping data attribute information contains accurate three-dimensional information. The center point is calculated for each polygon, and the elevation information in the corresponding mapping data attribute is assigned to the data element, which is then converted into the center point of the corresponding key point polygon, and the elevation information is used as the attribute value of the key point. Step S2.2, divide the two-dimensional plane into grids and fill the key points in the two-dimensional grid. However, since it is impossible to ensure that each point is on a regular grid point, it is necessary to resample the plane points and generate a completely regular dense grid point graph; The resampling problem is defined as an assignment problem. A key point is assigned to each grid point for filling, and the total movement cost is required to be minimized. The specific processing method is shown in formula (2): The purpose of formula (2) is to optimize a set of permutation matrices W g and W i l , to uniquely transform the global feature F G and local features are assigned to a predefined 2D grid of points, where W g Relative to F G The mapping of Redundant grids represent grid points that only contain Form a corresponding relationship, N G is the dimension of the permutation matrix and the grid; Formula (2) ensures that in the global and local neighborhoods, only a subset of grid points form a one-to-one context point, and all missing matching grid points are filled in through the nearest neighbors. The matched points are mapped to the three-dimensional space according to their elevation values ​​to form a dense point cloud.

4. The method for constructing a building floor and household entity by laser point cloud fusion and joint surveying and mapping according to claim 1 is characterized in that: The step S3 specifically also includes the following steps: The vertex set and edge set of the building's outer contour point cloud are used to combine with the floor point cloud of the jointly surveyed building to correct the building's outer contour. Step S3.1, first, use the point cloud P scanned by the airborne laser radar s and point cloud P generated by joint mapping primitives t Construct the graph model G respectively s and graph model G t ; The vertices in the two graphs are matched by the Hungarian algorithm. During the matching, the number of vertices in the point cloud contour graph must be greater than the number of vertices in the joint mapping primitives. The predicted points are linked to the primitive points using the Euclidean distance. Step S3.2, use graph matching algorithm to find point cloud P s and point cloud P t The matching relationship; Extract point cloud P using multi-layer perceptron s The point features are encoded into a one-dimensional point sequence; For the point cloud P t , since it is a diffuse grid point formed by filling neighboring points, a circular convolution is used to extract the structured feature sequence, and the first-order graph matching similarity matrix is ​​directly obtained by multiplying the first-order feature points of formula (3): M p =U 1 U 2T ...... (3) In order to obtain the second-order similarity, we first connect the two nodes corresponding to each edge to construct the second-order feature matrix of the two graphs, as shown in formula (4): Among them, formula (3) is for the first-order similarity, U 1 and U 2 are the feature vectors of each node, respectively, and the superscripts indicate different data sources; Formula (4) For the second-order similarity, X and Y are edge feature matrices, which are constructed by setting the descriptor of the cth edge starting from the i-th node and ending at the j-th node to the concatenated matrix of the descriptors of the two nodes, W 2 is a learnable parameter, so the second-order similarity has a learnable matching function; then, the graph matching problem is approximated as the problem of solving the eigenvector corresponding to the maximum eigenvalue of the similarity matrix, in which power iteration is used for calculation; The matching results of the graph are represented by a double random matrix, which intuitively reflects the possibility of establishing a matching relationship between any pair of nodes. First, the matrix is ​​normalized by column and then by row. The double random matrix is ​​calculated through multiple iterations. However, since the element values ​​in the same row and column are not much different, in order to increase the obvious difference and facilitate the subsequent loss function calculation, the matrix is ​​multiplied by a constant and then processed using softmax to obtain the probability matrix of each candidate node match, and this matrix still satisfies the double random property. Finally, the loss function is calculated using the offset vector-based method to calculate the offset between the matching points, and then weighted summed according to the probability matrix to predict the offset vector and minimize the difference between the predicted vector and the true offset. The loss function is shown in formula (5): Among them, d i and They represent the predicted and true offsets from the far point to the correct matching point, respectively. Φ(·) is a penalty term. In the final loss function calculation process, the discarded points will not affect the calculation of the loss function. In the above way, the airborne lidar point cloud and the joint mapping building layer point cloud are matched and fused, and the fused point cloud is projected onto a two-dimensional plane to extract the layered information. The boundary outline of the building layer should satisfy the manifold assumption, where each vertex can only be connected to two adjacent edges; Given the edge set generated in the previous step, select a subset of edges to calculate by using the constrained integer programming formula; i ∈{0, 1} is defined as a binary variable to describe the edge e i Whether ∈E is activated as part of the boundary state; the set of activated edges constitutes the polygonal boundary of the layer entity; In addition, an energy function is used to constrain the generation quality of the boundary contour, as shown in formula (6): U(x)=(1-λ)U fidelity (x)+λU complexity (x)...... (6) In formula (6), U fidelity (x) describes the consistency of state x with the input data, U complexity (x) measures the complexity of the output boundary shape, both terms are in the interval [0, 1], and λ∈(0, 1); U fidelity (x) Evaluate the coherence between the activation edge and the boundary shape. Since the boundary shape divides the entire space into inner and outer domains, most points are located within the boundary shape and constrained by fidelity, as shown in formula (7): U fidelity (x)=βU points (x)+(1-β)U walls (x)...... (7) The first term measures the percentage of input points enclosed by the boundary to preserve different proportions of internal cells occupied by the input points, where β∈(0, 1); while the second term is added to constrain the edge by measuring the overlap between the edge and the actual wall; Furthermore, in order to control the complexity of the final boundary shape, as shown in formula (8): In formula (8), |V| is the set of vertices to be calculated. This term returns a polygon with a small number of corner vertices. Γ is a discriminant used to determine the vertex v i ∈V is a corner vertex; however, since the building facade may contain slopes, it is necessary to calculate the average height of each point and then compare it with the single-layer height value in the entity attributes of the joint surveying element layer; Then, RANSAC is used to detect planes with inconsistent z-axis from the point cloud, the average height of the support point set corresponding to each plane is calculated, the points with average height lower than the height of a single layer are merged, and the face polygons are merged.

5. The method for constructing a building floor and household entity by laser point cloud fusion and joint surveying and mapping according to claim 1 is characterized in that: The step S4 specifically also includes the following steps: Step S4.1, the joint mapping primitives are given instance labels of the layer units. Therefore, the layer segmentation problem is transformed into a Markov random field problem through the standard form of energy problem. The energy function is shown in formula (9): In formula (9), γ∈(0,1) is a balance term, E is all adjacent internal faces, j and k represent specific faces, and m is the total number of faces; D(l k ) assigns a label to the closed face that is consistent with the layer number instance label; the pair V(l j , l k ) first returns a flat cell with low complexity, and secondly, layers it by the shape within the boundary; For the final layer segmentation, the above Markov problem is solved by graph cut optimization technology combined with joint surveying and mapping metadata. First, adjacent faces are merged into large polygonal faces, each of which represents a layer. Then, the floor plan is simplified by deleting vertices that are not connected to colinear edges, thus achieving accurate entity segmentation of building facade layers.

Images