Building facade reconstruction method and device based on laser point cloud

Abstract

The invention discloses a building facade reconstruction method and device based on laser point clouds, and the method comprises the steps: extracting feature point clouds from facade point clouds through the facade semantic structure features of the point clouds; then, classifying the similar feature point clouds into one class by utilizing the point cloud similarity to obtain a plurality of classified point cloud sets; next, vertical face optimization is carried out from the two aspects of shape and position, that is, the spatial distribution position of the classified point cloud set is optimized to correct the position offset of the point cloud, so that an optimal center point is obtained; then, on the basis of the optimal center point, a vertical face contour is constructed, contour optimization is carried out, and the real structure of the vertical face is restored to the maximum extent; and finally, based on the optimized facade contour, reconstructing to obtain a building facade three-dimensional model. Therefore, the invention provides an automatic, object-level and high-accuracy building facade reconstruction method, and only a data source of the laser point cloud is needed, so that the reconstruction complexity is greatly reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of building three-dimensional reconstruction, and in particular relates to a method and device for building facade reconstruction based on laser point cloud. Background Art

[0003] At present, component-level reconstruction mainly relies on manual interactive editing to adapt to the diverse building structures in the city and overcome the problem of incomplete data caused by changes in lighting, occlusion of shooting angles, and limitations of acquisition equipment. Although researchers have conducted research on automated extraction, they mainly focus on the overall segmentation of building facades and top recognition, while there is less recognition of detailed facade structures. In addition, automated recognition mainly extracts point and line features of facade structures, rather than object-oriented extraction and structural optimization. Therefore, using existing automated facade extraction methods to extract facades will result in large reconstruction errors. In addition, there is also a method of using multi-source data fusion to reconstruct component-level real-life 3D models, but this method requires multi-source data and is too complicated. Based on this, how to provide a reconstruction method for component-level 3D models that is simple to operate and can accurately reconstruct building facades has become an urgent problem to be solved. Summary of the invention

[0004] The purpose of the present invention is to provide a method and device for building facade reconstruction based on laser point cloud, so as to solve the problems existing in the prior art of large error and the need for multi-source data, which leads to a complicated reconstruction process.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, a method for reconstructing a building facade based on a laser point cloud is provided, comprising:

[0007] Obtain the facade point cloud set of the target building;

[0008] Determining the facade semantic structure features of each facade point cloud in the facade point cloud set, and extracting feature point clouds from the facade point cloud set according to the facade semantic structure features of each facade point cloud to obtain a feature point cloud set;

[0009] Based on the point cloud similarity, the feature point cloud set is classified to obtain a plurality of classified point cloud sets;

[0010] Optimizing the spatial distribution position of each classification point cloud set, so as to obtain the optimal center point of each classification point cloud set after the spatial distribution position optimization processing;

[0011] According to the optimal center point of each classification point cloud set, the facade contour of each classification point cloud set is constructed, and the facade contour of each classification point cloud set is optimized to obtain the optimized facade contour of each classification point cloud set after the optimization process;

[0012] The three-dimensional model of the building facade of the target building is reconstructed using the optimized facade contours.

[0013] Based on the above disclosed content, the present invention first obtains the facade point cloud of the target building, and then determines the facade semantic structure features of each facade point cloud, and based on this, classifies the feature points and non-feature points, thereby obtaining a feature point cloud set; in this way, accurate data support can be provided for the subsequent facade extraction; then, the present invention classifies the extracted feature point cloud based on the point cloud similarity to obtain multiple classified point cloud sets, based on which similar point clouds can be classified into one category to facilitate subsequent facade extraction; then, the present invention corrects the position offset generated by optimizing the spatial distribution position of each classified point cloud set, thereby obtaining the optimal center point of each classified point cloud set; then, the facade contour of each classified point cloud set can be constructed according to the optimal center point, and the contour optimization can be performed at the same time, thereby restoring the true structure of the facade corresponding to each classified point cloud set to the greatest extent; finally, the three-dimensional model of the building facade of the target building can be reconstructed based on the optimized facade contour.

[0014] Through the above design, the present invention extracts feature point clouds from the facade point clouds of buildings through the facade semantic structure features of the point clouds; then, the point cloud similarity is used to classify similar feature point clouds into one category to obtain multiple classified point cloud sets; then, the facade is optimized from the two aspects of shape and position, that is, the position offset of the point cloud is corrected by optimizing the spatial distribution position of the classified point cloud set, so as to obtain the corresponding optimal center point; then, the facade contour is constructed based on the optimal center point, and the contour is optimized to restore the real structure of the facade to the greatest extent; finally, the three-dimensional model of the building facade can be reconstructed based on the optimized facade contour; in this way, the present invention provides an automated, object-level, and highly accurate facade reconstruction method for buildings, and only needs to use laser point clouds as a data source, thereby greatly reducing the complexity of reconstruction; thus, the present invention provides a breakthrough technical support for component-level real-scene three-dimensional construction, and is very suitable for large-scale application and promotion.

[0015] In a possible design, determining the facade semantic structure features of each facade point cloud in the facade point cloud set includes:

[0016] For any facade point cloud in the facade point cloud set, calculating the optimal neighborhood radius of any facade point cloud;

[0017] According to the optimal neighborhood radius, obtaining an optimal neighborhood point cloud set of any of the facade point clouds;

[0018] Based on the optimal neighborhood point cloud set and the centroid of the optimal neighborhood point cloud set, determining the optimal centroid displacement vector of any facade point cloud, and using the optimal centroid displacement vector as the facade semantic structure feature of any facade point cloud;

[0019] Correspondingly, according to the facade semantic structure features of each facade point cloud, a feature point cloud is extracted from the facade point cloud set to obtain a feature point cloud set, which includes:

[0020] Calculate the modulus of the best quality centroid displacement vector of each facade point cloud as the eigenvalue of each facade point cloud;

[0021] From the facade point cloud set, facade point clouds with characteristic values ​​greater than or equal to a characteristic threshold are extracted as characteristic point clouds, so as to form the characteristic point cloud set using the extracted characteristic point clouds.

[0022] In a possible design, the optimal neighborhood radius of any of the facade point clouds is calculated, including:

[0023] Randomly selecting a plurality of facade point clouds from the facade point cloud set as seed points;

[0024] Obtaining the neighboring points of each seed point, and calculating the distance between each seed point and the corresponding neighboring points, so as to take the average value of the sum of the distances between each seed point and the corresponding neighboring points as the point density of any facade point cloud, wherein the neighboring points of any seed point are the facade point clouds in the neighborhood area of ​​any seed point;

[0025] Determine a maximum neighborhood radius and a minimum neighborhood radius according to the point density of any of the facade point clouds;

[0026] Based on the minimum neighborhood radius and the maximum neighborhood radius, a radius search interval is determined, and a neighborhood radius search is performed within the radius search interval to search for a neighborhood radius that minimizes the Shannon entropy of any of the facade point clouds;

[0027] The neighborhood radius that minimizes the Shannon entropy of any of the facade point clouds is used as the optimal neighborhood radius of any of the facade point clouds. In a possible design, for any neighborhood radius within the radius search interval, the Shannon entropy of any of the facade point clouds under the neighborhood radius is calculated in the following manner;

[0028] Based on any neighborhood radius, obtaining a search neighborhood of any facade point cloud;

[0029] Determine the covariance matrix of any of the facade point clouds according to the facade point clouds in the search neighborhood, and calculate the first eigenvalue, the second eigenvalue and the third eigenvalue of the covariance matrix;

[0030] Calculating the probability that any one of the facade point clouds belongs to linear points and the probability that any one of the facade point clouds belongs to planar points according to the first eigenvalue, the second eigenvalue, and the third eigenvalue, and calculating the probability that any one of the facade point clouds belongs to scattered points according to the first eigenvalue and the third eigenvalue;

[0031] Based on the probability that any of the facade point clouds belongs to linear points, the probability that any of the facade point clouds belongs to planar points, and the probability that any of the facade point clouds belongs to scattered points, and using the following formula (1), the Shannon entropy of any of the facade point clouds at any of the neighborhood radius is calculated;

[0032] E(r i )=-(δ 1 lnδ 1 +δ 2 lnδ 2 +δ 3 lnδ 3 ) (1)

[0033] In the above formula (1), E(r i ) represents any facade point cloud within any neighborhood radius r i Shannon entropy under 1 ,δ 2 ,δ 3 represents the probability that any vertical point cloud belongs to a linear point, a plane point, and a scattered point, respectively, where r i ∈[r min ,r max ], and r min ,r max Represent the minimum neighborhood radius and the maximum neighborhood radius respectively.

[0034] In a possible design, based on the optimal neighborhood point cloud set and the centroid of the optimal neighborhood point cloud set, determining the optimal centroid displacement vector of any facade point cloud includes:

[0035] According to the following formula (2), the optimal centroid displacement vector of any vertical point cloud is calculated;

[0036]

[0037] In the above formula (2), ε i represents the optimal centroid displacement vector of any vertical point cloud, Ω(p i ) represents any of the vertical point clouds p iThe optimal neighborhood point cloud set, p j represents the jth neighborhood point cloud in the optimal neighborhood point cloud set, m j Indicates p j The centroid weighted point, Represents any of the facade point clouds p i To the centroid weighted point m j A vector of , ω(i,j) represents the vector weight;

[0038] in,

[0039] ω(i,j)=(-||p i -p j || 2 / R 2 ) (4)

[0040] In formula (3), c i represents the centroid of the optimal neighborhood point cloud set, Represents any of the facade point clouds p i To the jth neighborhood point cloud p j The vector of Represents any of the facade point clouds p i The vector to the centroid of the optimal neighborhood point cloud;

[0041] In formula (4), R represents the average distance between any facade point cloud and all neighboring point clouds in the optimal neighboring point cloud set, and || || represents the modulo operation symbol.

[0042] In a possible design, based on point cloud similarity, the feature point cloud set is classified to obtain multiple classified point cloud sets, including:

[0043] Performing vertical slicing processing on the feature point cloud set to obtain slice point cloud data;

[0044] Performing preliminary segmentation processing on the slice point cloud data by using a region growing algorithm, so as to obtain a plurality of point cloud clusters after the preliminary segmentation processing;

[0045] Extract the FPFH features of each feature point cloud in each point cloud cluster, and calculate the Hausdorff distance between each point cloud cluster based on the FPFH features of each feature point cloud in each point cloud cluster;

[0046] Based on the Hausdorff distance between each point cloud cluster, a Hausdorff distance matrix is ​​constructed;

[0047] According to the Hausdorff distance matrix, a hierarchical clustering algorithm is used to perform clustering processing on each point cloud cluster, so as to obtain a plurality of classified point cloud sets after clustering processing.

[0048] In a possible design, the spatial distribution position of each classification point cloud set is optimized to obtain the optimal center point of each classification point cloud set after the spatial distribution position optimization, including:

[0049] For any classified point cloud set, determine the center point and the plane bounding box of any classified point cloud set;

[0050] According to the plane bounding box, a plane coordinate system is established, wherein the origin of the plane coordinate system is the endpoint of the lower left corner of the plane bounding box, and the x-axis of the plane coordinate system is the horizontal direction, and the z-axis is the vertical direction;

[0051] Projecting the center point to the x-axis direction and the z-axis direction of the plane coordinate system respectively, and determining a first straight line and a second straight line, wherein the first straight line is a straight line co-columned with the projection point of the center point in the z-axis direction, and the second straight line is a straight line co-rowed with the projection point of the center point in the x-axis direction;

[0052] The intersection point between the first straight line and the second straight line is used as the optimal center point of any classification point cloud set.

[0053] In a possible design, any classified point cloud set contains multiple point cloud clusters, wherein the facade contour of each classified point cloud set is constructed according to the optimal center point of each classified point cloud set, including:

[0054] For any classified point cloud set, obtaining the outline of each point cloud cluster in the any classified point cloud set;

[0055] Taking the optimal center point of any classified point cloud set as the insertion anchor point, superimposing the contours corresponding to each point cloud cluster in the any classified point cloud set, so as to obtain the elevation contour of the any classified point cloud set after superimposing;

[0056] Accordingly, the elevation contours of each classified point cloud set are optimized to obtain the optimized elevation contours of each classified point cloud set after the optimization process, which includes:

[0057] For any classified point cloud set, determining a plurality of similar line segment groups in the elevation contour of the any classified point cloud set, wherein any similar line segment group contains two similar contour line segments;

[0058] Line segment merging processing is performed on two similar contour line segments in each similar line segment group, so as to obtain an optimized facade contour of any classified point cloud set after the line segment merging processing.

[0059] In a possible design, determining a plurality of similar line segment groups in the facade contour of any classified point cloud set includes:

[0060] For any contour line segment in the elevation contour of any classified point cloud set, calculate the angle difference, vertical distance and projection distance between any contour line segment and each target line segment in the target line segment set, wherein the target line segment set includes all contour line segments in the elevation contour of any classified point cloud set except the any contour line segment;

[0061] Calculating the line segment similarity between any contour line segment and each target line segment in the target line segment set based on the angle difference, vertical distance and projection distance between any contour line segment and each target line segment in the target line segment set;

[0062] Determine whether the smallest line segment similarity among the line segment similarities is less than or equal to the similarity threshold;

[0063] If so, any of the contour line segments and the target line segment with the smallest line segment similarity to any of the contour line segments are used to form a similar line segment group, and after all contour line segments of the facade contour of any of the classified point cloud sets are polled, several similar line segment groups are obtained.

[0064] In a second aspect, a building facade reconstruction device based on laser point cloud is provided, comprising:

[0065] An acquisition unit, used for acquiring a facade point cloud set of a target building;

[0066] A feature point extraction unit is used to determine the facade semantic structure feature of each facade point cloud in the facade point cloud set, and extract a feature point cloud from the facade point cloud set according to the facade semantic structure feature of each facade point cloud to obtain a feature point cloud set;

[0067] A classification unit, used for classifying the feature point cloud set based on point cloud similarity to obtain multiple classification point cloud sets;

[0068] A spatial distribution position optimization unit is used to optimize the spatial distribution position of each classification point cloud set, so as to obtain the optimal center point of each classification point cloud set after the spatial distribution position optimization processing;

[0069] A contour optimization unit is used to construct a facade contour of each classification point cloud set according to the optimal center point of each classification point cloud set, and optimize the facade contour of each classification point cloud set to obtain an optimized facade contour of each classification point cloud set after the optimization process;

[0070] The model reconstruction unit is used to reconstruct the three-dimensional model of the building facade of the target building by using each optimized facade contour.

[0071] In a third aspect, another device for reconstructing building facades based on laser point cloud is provided. Taking the device as an electronic device as an example, the device includes a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the method for reconstructing building facades based on laser point cloud as in the first aspect or any possible design in the first aspect.

[0072] In a fourth aspect, a storage medium is provided, on which instructions are stored. When the instructions are executed on a computer, the method for reconstructing a building facade based on laser point cloud as described in the first aspect or any possible design of the first aspect is executed.

[0073] In a fifth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, causes the computer to execute the method for reconstructing a building facade based on laser point cloud as described in the first aspect or any possible design of the first aspect.

[0074] Beneficial effects:

[0075] (1) The present invention provides an automated, object-level, and highly accurate method for reconstructing the facade of a building, and only requires the use of a laser point cloud as a data source, thereby greatly reducing the complexity of reconstruction. Therefore, the present invention provides a breakthrough technical support for component-level real-scene three-dimensional construction, and is very suitable for large-scale application and promotion.

[0076] (2) The present invention proposes a weighted centroid displacement method guided by Shannon entropy to measure features. In this way, effective feature information of point clouds can be extracted from shallow point clouds, thereby providing accurate data support for subsequent facade extraction.

[0077] (3) The present invention improves the Hausdorff distance used to calculate the similarity between two point cloud sets, and adds FPFH features so that it is not limited to the differences in Euclidean space. In this way, it is possible to identify the judgment of the similarity of point cloud sets by high-dimensional features at a deep level. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 A schematic diagram of the steps of a method for reconstructing a building facade based on laser point cloud provided by an embodiment of the present invention;

[0079] Figure 2 A schematic diagram of a centroid weighted point provided in an embodiment of the present invention;

[0080] Figure 3 A schematic diagram for determining the optimal center point provided by an embodiment of the present invention;

[0081] Figure 4A schematic diagram of constructing a three-dimensional model of a building facade provided by an embodiment of the present invention;

[0082] Figure 5 A schematic diagram of the structure of a building facade reconstruction device based on laser point cloud provided by an embodiment of the present invention;

[0083] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0084] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in combination with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0085] It should be understood that although the terms first, second, etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are only used to distinguish one unit from another unit. For example, a first unit can be referred to as a second unit, and similarly, a second unit can be referred to as a first unit without departing from the scope of the exemplary embodiments of the present invention.

[0086] It should be understood that the term "and / or" that may appear in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, B exists alone, and A and B exist at the same time. The term " / and" that may appear in this article describes another type of association object relationship, indicating that two relationships may exist. For example, A / and B can represent two situations: A exists alone, and A and B exist alone. In addition, the character " / " that may appear in this article generally indicates that the previous and next associated objects are in an "or" relationship.

[0087] Example:

[0088] See also Figure 1As shown, the building facade reconstruction method based on laser point cloud provided in this embodiment extracts feature point cloud from the building facade point cloud through the facade semantic structure feature of point cloud; then, similar feature point clouds are classified into one category by using point cloud similarity to obtain multiple classified point cloud sets; then, the facade is optimized from the two aspects of shape and position, that is, the position offset of the point cloud is corrected by optimizing the spatial distribution position of the classified point cloud set, so as to obtain the corresponding optimal center point; then, the facade contour is constructed based on the optimal center point, and the contour is optimized to restore the facade to the greatest extent. The real structure of the facade; finally, the three-dimensional model of the building facade can be reconstructed based on the optimized facade contour; in this way, the present embodiment provides an automated, object-level, and highly accurate method for reconstructing the facade of a building, and only requires the use of laser point cloud as a data source, thereby greatly reducing the complexity of reconstruction; thus, the present method provides a breakthrough technical support for component-level real-scene three-dimensional construction, and is very suitable for large-scale application and promotion; for example, the present method can be but is not limited to running on the building facade reconstruction end side, and optionally, the building facade reconstruction end can be but is not limited to a personal computer (PC). It can be understood that the aforementioned execution subject does not constitute a limitation on the embodiments of the present application, and accordingly, the operation steps of the present method can be but are not limited to the following steps S1 to S6.

[0089] S1. Obtain the facade point cloud set of the target building; in this embodiment, the original laser point cloud data of the target building can be obtained first, and then the facade point cloud data can be extracted from the original laser point cloud data to form a facade point cloud set (of course, the original laser point cloud data is two-dimensional point cloud data); wherein, the original laser point cloud data contains a large number of point clouds of the building itself and point cloud data of non-buildings, and while containing a large number of plane structures, it also has significant changes in the building outline boundary and subtle changes in the semantic structure of windows and doors; therefore, filtering and classification methods can be used to remove the noise of the original laser point cloud data, and extract the point cloud representing only the target building from the original laser point cloud data; then, the extracted point cloud of the target building is plane segmented (such as using the RANSAC plane segmentation method to perform segmentation), thereby segmenting the facade point cloud data of the target building from the aforementioned extracted point cloud of the target building; finally, the facade point cloud set can be formed based on the extracted facade point cloud data.

[0090] After obtaining the facade point cloud set of the target building, the facade feature point cloud can be extracted on this basis to provide accurate data support for the subsequent facade extraction (that is, the extraction of the window facade of the target building); wherein the feature point cloud extraction process can be but is not limited to as shown in the following step S2.

[0091] S2. Determine the facade semantic structure features of each facade point cloud in the facade point cloud set, and extract feature point clouds from the facade point cloud set according to the facade semantic structure features of each facade point cloud to obtain a feature point cloud set; in specific applications, this embodiment uses the best neighborhood discrimination and centroid shift optimization method guided by Shannon entropy to perform feature description of each facade point cloud in the facade point cloud set, and classifies feature points and non-feature points in combination with the quantization results; wherein, taking any facade point cloud as an example, it is explained that the extraction process of the facade semantic structure features can be but not limited to the following steps S21 to S23.

[0092] S21. For any facade point cloud in the facade point cloud set, calculate the optimal neighborhood radius of any facade point cloud; in specific implementation, the optimal neighborhood radius of any facade point cloud is determined based on the Shannon entropy of the any facade point cloud at different neighborhood radii; wherein, the optimization method for the optimal neighborhood radius based on Shannon entropy can be but is not limited to the following steps S21a to S21e.

[0093] S21a. Randomly select multiple facade point clouds from the facade point cloud set as seed points; in this embodiment, it is necessary to calculate the point density first, and then determine the search interval of the neighborhood radius based on the point density; then, traverse the search interval, and calculate the Shannon entropy of any facade point cloud at each neighborhood radius within the interval; finally, determine the optimal neighborhood radius based on the calculated Shannon entropies, wherein the point density calculation process can be but is not limited to as shown in the following step S21b.

[0094] S21b. Obtain the neighboring points of each seed point, and calculate the distance between each seed point and the corresponding neighboring points, so as to take the average value of the sum of the distances between each seed point and the corresponding neighboring points as the point density of any of the facade point clouds, wherein the neighboring points of any seed point are the facade point clouds in the neighborhood area of ​​any seed point; in specific applications, a preset radius can be set first, and then an area with the any seed point as the center and a radius of the aforementioned preset radius can be taken as the neighborhood area of ​​any seed point; in this way, the distance between any of the seed points and each of the facade point clouds in its neighborhood area can be calculated first, thereby obtaining multiple distances; then, by traversing all the seed points, the distances between all the seed points and the corresponding neighboring points can be obtained; finally, summing and averaging can be performed to obtain the point density of any of the facade point clouds.

[0095] After the point density is calculated based on step S21b, the maximum neighborhood radius and the minimum neighborhood radius can be determined based on this, and the process can be but not limited to as shown in the following step S21c.

[0096] S21c. Determine the maximum neighborhood radius and the minimum neighborhood radius according to the point density of any of the facade point clouds; in a specific implementation, for example, the maximum neighborhood radius is 10ρ and the minimum neighborhood radius is 3ρ, where ρ represents the point density.

[0097] Thus, after obtaining the maximum and minimum neighborhood radius, the optimal neighborhood radius can be searched within the search interval formed by the minimum and maximum neighborhood radius based on the Shannon entropy of any of the facade points. The process can be but is not limited to that shown in the following step S12d.

[0098] S21d. Based on the minimum neighborhood radius and the maximum neighborhood radius, a radius search interval is determined, and a neighborhood radius search is performed within the radius search interval to search for a neighborhood radius that minimizes the Shannon entropy of any of the facade point clouds. In a specific implementation, the principle of using Shannon entropy to optimize the neighborhood radius is as follows: Shannon entropy, as an indicator for measuring the randomness or uncertainty of information, is often used to quantify information measurement problems. In point cloud analysis, it is usually used to evaluate the amount of information in the neighborhood of a point cloud. The less information means the simpler the local features, and the Shannon entropy of a point cloud is related to its neighborhood area. Therefore, changes in the neighborhood radius will lead to changes in the Shannon entropy. A smaller Shannon entropy means that the neighborhood of the target point contains less information and the neighborhood structural features of the target point are simpler. Based on this, an adaptive selection of the radius can be performed based on this characteristic. In this way, the optimal radius is determined based on the principle of minimum Shannon entropy, which helps to improve the ability to express local fine structures.

[0099] Therefore, this embodiment sets reasonable maximum and minimum values ​​of the search radius, and combines the calculation of Shannon entropy to determine the optimal neighborhood radius; that is, traverse the aforementioned radius search interval to find the neighborhood radius that minimizes the Shannon entropy of any of the facade point clouds, and then use the neighborhood radius that minimizes the Shannon entropy of any of the facade point clouds as the optimal neighborhood radius.

[0100] For example, taking an example to illustrate, assuming that the radius search interval is [3,10] and the search step is 1, then the Shannon entropy of any facade point cloud at a neighborhood radius of 3, 4, 5, 6, 7, 8, 9, and 10 is calculated respectively; wherein, assuming that the Shannon entropy of any facade point cloud at a neighborhood radius of 4 is the smallest, then the neighborhood radius of 4 is used as the optimal neighborhood radius; of course, the above examples are only for illustration, and the search step and interval are not limited thereto.

[0101] Optionally, the following formula is used to express the optimization process of the aforementioned optimal neighborhood radius:

[0102]

[0103] In the above formula (5), r represents the optimal neighborhood radius, r min ,r max Respectively represent the minimum neighborhood radius and the maximum neighborhood radius, r i represents the search neighborhood radius between the minimum neighborhood radius and the maximum neighborhood radius, E(r i ) indicates that any of the facade point clouds is within the search neighborhood radius r i Shannon entropy under arg min[] represents the value of E(r i ) is the value of the variable when it reaches its minimum value.

[0104] Thus, according to the above formula (5), the optimal neighborhood radius can be searched out.

[0105] Furthermore, the following discloses a method for calculating the Shannon entropy of any facade point cloud at any neighborhood radius (i.e., the aforementioned search neighborhood radius) within the radius search interval when performing an optimal neighborhood radius search, and the process can be but is not limited to the first to fourth steps shown below.

[0106] Step 1: Based on any of the neighborhood radius, obtain the search neighborhood of any of the facade point clouds; in the present embodiment, it is equivalent to taking the area centered on any of the facade point clouds and with any of the neighborhood radius as the search radius as the search neighborhood; then, based on the search neighborhood, the Shannon entropy of any of the facade point clouds under any of the neighborhood radius can be calculated, and the process can be but not limited to the following second to fourth steps as shown.

[0107] Step 2: According to the facade point cloud in the search neighborhood, determine the covariance matrix of any facade point cloud, and calculate the first eigenvalue, second eigenvalue and third eigenvalue of the covariance matrix; in specific implementation, for example, but not limited to, the domain plane of the search neighborhood is fitted by the principal component analysis method to obtain its corresponding covariance matrix, and then, by solving the three eigenvalues ​​of the covariance matrix, the probability that any facade point cloud belongs to linear points, plane points and scattered points is calculated; finally, based on the above three probabilities, the Shannon entropy of any facade point cloud under the radius of any neighborhood can be calculated; of course, the above method of fitting the point cloud plane based on PCA (principal component analysis) to obtain the covariance matrix is ​​a commonly used method for point cloud fitting, and its principle will not be repeated.

[0108] After obtaining the three eigenvalues ​​of the covariance matrix, the probability that any facade point cloud belongs to different types of points can be calculated, and the process can be but not limited to the following third step.

[0109] Step 3: Based on the first eigenvalue, the second eigenvalue and the third eigenvalue, calculate the probability that any of the facade point clouds belongs to linear points and the probability that they belong to plane points, and based on the first eigenvalue and the third eigenvalue, calculate the probability that any of the facade point clouds belongs to scattered points.

[0110] In this embodiment, the following formulas (6), (7) and (8) may be used in sequence to calculate the probability that any vertical point cloud belongs to a linear point, the probability that it belongs to a plane point and the probability that it belongs to a scattered point.

[0111]

[0112]

[0113]

[0114] In the above formula (6), δ 1 represents the probability that any facade point cloud belongs to a linear point, λ 1 ,λ 2 Denote the first eigenvalue and the second eigenvalue, respectively.

[0115] In the above formula (7), δ 2 represents the probability that any vertical point cloud belongs to a plane point, λ 3 represents the third eigenvalue.

[0116] In the above formula (8), δ 3 represents the probability that any of the facade point clouds belongs to scattered points, and λ 1 ≥λ 2 ≥λ 3 .

[0117] Thus, through the aforementioned formulas (6)-(8), the probability that any of the vertical point clouds belongs to linear points, the probability that any of the vertical point clouds belongs to plane points, and the probability that any of the vertical point clouds belongs to scattered points can be calculated; then, based on the aforementioned three probabilities, the Shannon entropy of any of the vertical point clouds at any of the aforementioned neighborhood radii can be calculated, and the calculation process is shown in the fourth step below.

[0118] Step 4: Based on the probability that any of the vertical point clouds belongs to linear points, the probability that any of the vertical point clouds belongs to planar points, and the probability that any of the vertical point clouds belongs to scattered points, and using the following formula (1), calculate the Shannon entropy of any of the vertical point clouds at any of the neighborhood radius.

[0119] E(r i )=-(δ 1 lnδ 1 +δ 2 lnδ 2 +δ3 lnδ 3 ) (1)

[0120] In the above formula (1), E(r i ) represents any facade point cloud within any neighborhood radius r i Shannon entropy under 1 ,δ 2 ,δ 3 represents the probability that any vertical point cloud belongs to a linear point, a plane point, and a scattered point, respectively, where r i ∈[r min ,r max ], and r min ,r max Represent the minimum neighborhood radius and the maximum neighborhood radius respectively.

[0121] Therefore, through the first to fourth steps mentioned above, the Shannon entropy of any facade point cloud at each neighborhood radius within the radius search interval can be calculated, and then the neighborhood radius with the smallest Shannon entropy can be used as its corresponding optimal neighborhood radius; wherein, the process of determining the optimal neighborhood radius is shown in the following step S21e.

[0122] S21e. The neighborhood radius that minimizes the Shannon entropy of any of the facade point clouds is used as the optimal neighborhood radius of the any of the facade point clouds.

[0123] Therefore, through the aforementioned steps S21a to S21e, the optimal neighborhood radius of any facade point cloud can be determined; then, based on the optimal neighborhood radius, the optimal neighborhood point cloud set can be determined, and the facade semantic structure feature detection of any facade point cloud can be realized by using the centroid displacement of the optimal neighborhood point cloud set; wherein, the facade semantic structure feature detection process can be but is not limited to as shown in the following steps S22 and S23.

[0124] S22. According to the optimal neighborhood radius, the optimal neighborhood point cloud set of any one of the facade point clouds is obtained; in the present embodiment, the facade point cloud within the area with the radius of the optimal neighborhood radius being the center of any one of the facade point clouds is used to form the optimal neighborhood point cloud set; then, based on this, the optimal centroid displacement vector of any one of the facade point clouds can be determined, so as to use the optimal centroid displacement vector as the facade semantic structure feature of any one of the facade point clouds; optionally, the process of determining the optimal centroid displacement vector can be but is not limited to the process shown in the following step S23.

[0125] S23. Based on the optimal neighborhood point cloud set and the centroid of the optimal neighborhood point cloud set, determine the optimal centroid displacement vector of any facade point cloud, and use the optimal centroid displacement vector as the facade semantic structure feature of any facade point cloud; in this embodiment, the reason for not using the original centroid displacement vector of the optimal neighborhood point cloud set is: assuming that the centroid of the optimal neighborhood point cloud set is c i , then the original centroid displacement vector is: c i -p i , p i represents any of the above mentioned facade point clouds; wherein, the larger the modulus of the original centroid displacement vector is, the more complex the local shape of its neighborhood is; however, since the original centroid displacement vector is a uniform Laplace vector, its weight strategy is insensitive to surface changes, thus limiting its effectiveness in describing complex and detailed facade semantic structures; thus, in flat areas and shallow structures, since the vectors from the point to the centroid are very small, it will lead to problems of difficulty in distinguishing, and under noise interference, the vector from the point to the centroid will become longer, which may lead to p i Mistakenly identified as feature points.

[0126] Therefore, in order to increase the difference between the feature point cloud and the non-feature point cloud, thereby offsetting the influence of noise on feature detection, this embodiment proposes a solution based on local weighted centroid displacement to determine the optimal centroid displacement vector of any of the facade point clouds; wherein, this embodiment generates a corresponding centroid weighted point (which can be understood as a temporary point) for each neighborhood point cloud in the optimal neighborhood point cloud set, which can be seen in Figure 2 As shown ( Figure 2 m j represents the weighted centroid point of the jth neighborhood point cloud), which is used to construct the vector Then, the centroid weighted points of the neighborhood point cloud can be used to optimize the original centroid displacement vector to obtain the optimal centroid displacement vector.

[0127] Optionally, for example but not limited to, the following formula (2) may be used to calculate the optimal centroid displacement vector of any of the facade point clouds.

[0128]

[0129] In the above formula (2), ε i represents the optimal centroid displacement vector of any vertical point cloud, Ω(p i ) represents any of the vertical point clouds p i The optimal neighborhood point cloud set, p j represents the jth neighborhood point cloud in the optimal neighborhood point cloud set, m j Indicates p j The centroid weighted point, Represents any of the facade point clouds p i To the centroid weighted point m j , ω(i,j) represents the vector weight.

[0130] in,

[0131] ω(i,j)=(-||p i -p j || 2 / R 2 ) (4)

[0132] In formula (3), c i represents the centroid of the optimal neighborhood point cloud set, Represents any of the facade point clouds p i To the jth neighborhood point cloud p j The vector of Represents any of the facade point clouds p i The vector to the centroid of the optimal neighborhood point cloud.

[0133] In formula (4), R represents the average distance between any one of the facade point clouds and all the neighboring point clouds in the optimal neighboring point cloud set, and || || represents the modulo operation symbol; wherein the subtraction of two points in the above formula (4) is actually the subtraction of corresponding coordinates; at the same time, the sum of the neighborhood point clouds in formula (2) is the addition of corresponding coordinates.

[0134] In this way, the above-mentioned local weighted centroid displacement method is used to determine the best centroid displacement vector of any of the facade point clouds, which can amplify the difference between the points in the shallow features and the points in the flat area, thereby improving the accuracy of feature point recognition.

[0135] Based on this, through the aforementioned steps S21 to S23, the facade semantic structure features of each facade point cloud can be extracted; then, based on this, the feature point cloud and non-feature point cloud can be classified, wherein the feature point cloud extraction process is shown in the following steps S24 and S25.

[0136] S24. Calculate the modulus of the optimal centroid displacement vector of each facade point cloud as the eigenvalue of each facade point cloud.

[0137] S25. Extracting, from the facade point cloud set, facade point clouds whose characteristic values ​​are greater than or equal to a characteristic threshold as characteristic point clouds, so as to form the characteristic point cloud set using the extracted characteristic point clouds.

[0138] Therefore, through the aforementioned steps S21 to S25, the facade feature point cloud can be extracted from the facade point cloud set of the target building; then, the facade reconstruction can be performed based on the extracted feature point cloud; wherein, the present embodiment first classifies the feature point cloud, that is, classifies the point clouds with the same or similar structure into one category; then, on this basis, performs a double optimization of shape + position to obtain the optimal facade contour of the target building; finally, the facade model can be reconstructed according to the optimal facade contour; optionally, the classification process of the example feature point cloud can be but is not limited to as shown in the following step S3.

[0139] S3. Based on the similarity of point clouds, the feature point cloud set is classified and processed to obtain multiple classified point cloud sets; in the specific implementation, considering that the windows on the facades of large buildings in cities are often regularly arranged in the same or similar structural arrays, identifying and grouping windows of the same structure will help to overcome the problems of data omissions and point density differences under the limitation of scanning angle of view, thereby optimizing the results of semantic structure extraction; therefore, this embodiment adopts a two-time classification method to classify the feature point cloud, and the process is shown in the following steps S31 to S35.

[0140] S31. The feature point cloud set is vertically sliced ​​to obtain sliced ​​point cloud data. In specific implementation, vertical slicing is to slice the feature point cloud set in the vertical direction, and the initial thickness of the slice can be but is not limited to the floor height of the target building. At the same time, the present embodiment uses a sliding window to slice the feature point clouds corresponding to floors of different thicknesses, and gradually adjusts the slice thickness until a single slice can cover all areas of the same floor. At this time, the sliced ​​point cloud data can be obtained. In this way, vertical slicing can reduce the interference of irrelevant objects such as long connecting pipes attached to the building facade and clothes drying. Of course, vertical slicing of feature point clouds is a common technology for point cloud data capture, and its principle will not be repeated.

[0141] After completing the vertical slicing of the feature point cloud set, the region growing algorithm may be used to perform preliminary classification of the feature point cloud, and the process may be, but is not limited to, as shown in the following step S32.

[0142] S32. Use the region growing algorithm to perform preliminary segmentation processing on the slice point cloud data to obtain a number of point cloud clusters after the preliminary segmentation processing; in this embodiment, the region growing algorithm is used to divide the feature point cloud belonging to the same window into a point cloud cluster; among them, the region growing algorithm is a commonly used technology for point cloud classification, and its principle will not be repeated.

[0143] After completing the preliminary classification of the feature point cloud set, secondary classification can be performed; wherein, this embodiment is based on the improved Hausdorff distance to measure the point cloud similarity between two point cloud clusters, so as to complete the classification of the aforementioned several point cloud clusters; wherein, the secondary classification process can be but is not limited to as shown in the following steps S33 to S35.

[0144] S33. Extract the FPFH features of each feature point cloud in each point cloud cluster, and calculate the Hausdorff distance between each point cloud cluster based on the FPFH features of each feature point cloud in each point cloud cluster; in specific implementation, the Hausdorff distance is a metric for measuring the distance between two point sets, which is defined as the minimum distance between any pair of points in two relatively concentrated sets; among them, the Hausdorff distance can capture the similarity of objects, and is often used to compare and identify different geometric objects, especially when dealing with nonlinear or irregular shapes. It is very useful; however, the traditional distance measurement is performed in Euclidean space. For data with errors, especially when facing nonlinear data, the Euclidean distance may be enlarged. This is due to its characteristic of treating errors in all dimensions equally.

[0145] Based on this, in order to solve the above-mentioned drawbacks, this embodiment introduces FPFH (Fast Point Feature Histogram) to replace the traditional Euclidean distance; among them, the advantage of FPFH feature in expression ability is reflected in its ability to capture the local geometric characteristics of the point cloud through statistical analysis, thereby generating a 33-dimensional feature vector, which can effectively describe the local shape and surface changes of the point cloud; at the same time, in terms of multi-scale feature extraction, FPFH can adapt to point cloud data sets of different sizes by calculating point cloud features at different scales. This method not only reduces the need for parameter adjustment, but also improves the adaptability of the algorithm to changes in point cloud scale, ensuring that key geometric features can be accurately captured at different scales, and enhancing the comprehensiveness and robustness of feature description; therefore, this embodiment calculates the Hausdorff distance between each point cloud cluster based on the FPFH feature of the feature point cloud, thereby realizing the similarity measurement between point cloud clusters.

[0146] Optionally, for any two point cloud clusters, such as point cloud cluster A and point cloud cluster B, the following discloses a method for calculating the Hausdorff distance between the two, as shown below.

[0147] (1) For the mth feature point cloud in point cloud cluster A, the FPFH feature of the mth feature point cloud is compared with the FPFH feature of each feature point cloud in point cloud cluster B to obtain multiple feature difference vectors; in this embodiment, it has been explained above that the FPFH feature is a 33-dimensional feature vector. Therefore, the feature difference vector between the FPFH feature of the mth feature point cloud and the FPFH feature of any feature point cloud in point cloud cluster B is actually the difference vector between the FPFH features of the two.

[0148] (2) For any feature difference vector among the multiple feature difference vectors, the sum of all elements in the any feature difference vector is calculated to obtain the element sum of the any feature difference vector, and after all feature difference vectors are polled, the element sum of each feature difference vector is obtained.

[0149] (3) From all the elements and, select the smallest element and as the initial Hausdorff distance corresponding to the mth feature point cloud.

[0150] (4) Add 1 to m and re-compare the FPFH feature of the mth feature point cloud with the FPFH feature of each feature point cloud in point cloud cluster B until m is equal to M, and M initial Hausdorff distances are obtained, where the initial value of m is 1 and M is the total number of feature point clouds in point cloud cluster A.

[0151] (5) From the M initial Hausdorff distances, select the smallest initial Hausdorff distance as the Hausdorff distance between point cloud cluster A and point cloud cluster B.

[0152] In this way, through the above-mentioned method, the Hausdorff distance between each point cloud cluster can be calculated, and then, based on this, a Hausdorff distance matrix can be constructed to complete the classification of each point cloud cluster. The process can be but is not limited to the following steps S34 and S35.

[0153] S34. Based on the Hausdorff distance between each point cloud cluster, a Hausdorff distance matrix is ​​constructed.

[0154] After constructing the Hausdorff distance matrix, the hierarchical clustering algorithm can be combined to perform clustering (ie, secondary classification) of each point cloud cluster, and the process can be but is not limited to the following step S35.

[0155] S35. According to the Hausdorff distance matrix and using a hierarchical clustering algorithm, each point cloud cluster is clustered to obtain a plurality of classified point cloud sets after the clustering process; in the present embodiment, each row of the matrix is ​​summed and counted, and then, using a hierarchical clustering algorithm, point cloud clusters with similar structures can be grouped together, so that after the clustering process, a plurality of classified point cloud sets are obtained; wherein, for example, but not limited to, the number of clusters is determined according to the hierarchical clustering method, and then, the cluster similarity is calculated, and finally, similar clusters are merged to complete the classification; it is the same as the implementation process of the existing hierarchical clustering method, except that the original Euclidean distance is replaced by the Hausdorff distance; thus, any of the aforementioned classified point cloud sets contains a plurality of point cloud clusters, and each point cloud cluster contains a plurality of feature point clouds.

[0156] In this way, through the aforementioned steps S31 to S35, the classification of the feature point cloud can be completed, so that the feature point clouds of similar or identical structures can be classified into one category; then, the facade contour of the target building can be extracted based on the obtained classified point cloud set; wherein, in order to fully consider the similar associations of the window objects on the building facade in terms of spatial position and shape, the segmentation result errors caused by factors such as the omissions of the point cloud itself, the interference of noise, and the inapplicability of the segmentation method, this embodiment optimizes and enhances the facade semantic structure from two aspects of position and shape, so as to restore the most realistic window facade contour of the target building; optionally, the aforementioned position optimization process can be but is not limited to the following step S4.

[0157] S4. The spatial distribution position of each classification point cloud set is optimized to obtain the optimal center point of each classification point cloud set after the spatial distribution position optimization processing; in specific applications, the center point of each classification point cloud set generally conforms to the spatial relationship of co-column, co-row and equally spaced distribution, but the characteristics of point cloud scanning and the accuracy of the segmentation algorithm limit the distribution of the center point; therefore, this embodiment needs to optimize the spatial distribution position to obtain the optimal center point of each classification point cloud set.

[0158] Optionally, this embodiment takes any classified point cloud set as an example for explanation, and the process may be but is not limited to the following steps S41 to S44.

[0159] S41. For any classified point cloud set, determine the center point and the plane bounding box of any classified point cloud set; in the present embodiment, the center point of any classified point cloud set may be, but is not limited to, the average position of all feature point clouds in the any classified point cloud set (that is, the position corresponding to the average coordinates); and the plane bounding box of the point cloud is a commonly used technology for point cloud processing, and its construction process will not be repeated here.

[0160] After determining the plane bounding box of any of the classified point clouds, a plane coordinate system can be constructed based on it, so that the coordinate system can be used to project the center point later; wherein, the construction process of the plane coordinate system can be but is not limited to as shown in the following step S42.

[0161] S42. A plane coordinate system is established according to the plane bounding box, wherein the origin of the plane coordinate system is the end point of the lower left corner of the plane bounding box, and the x-axis of the plane coordinate system is the horizontal direction, and the z-axis is the vertical direction; in this embodiment, the two axes of the aforementioned plane coordinate system are the x-axis and the z-axis, respectively, the x-axis is the horizontal direction, and in the vertical direction, the z-axis is used instead of the traditional y-axis; in this way, the plane coordinate system can be constructed with the end point of the lower left corner of the plane bounding box as the origin; wherein, the schematic diagram of the plane coordinate system can be seen in Figure 3 shown.

[0162] After the plane coordinate system is constructed, the center point may be projected so as to obtain the optimal center point of any classified point cloud set based on the projection of the center point, wherein the projection process may be but is not limited to the following step S43.

[0163] S43. Project the center point to the x-axis direction and the z-axis direction of the plane coordinate system respectively, and determine a first straight line and a second straight line, wherein the first straight line is a straight line co-columned with the projection point of the center point in the z-axis direction, and the second straight line is a straight line co-rowed with the projection point of the center point in the x-axis direction; in this embodiment, it is equivalent to projecting the extracted center point to two coordinate axes (x-axis and z-axis) respectively, so as to find the first straight line co-columned with the projection point of the center point by projecting to the x-axis, and find the second straight line co-rowed with the projection point of the center point by projecting to the z-axis, wherein the schematic diagram of the first straight line and the second straight line can be seen in Figure 3 shown.

[0164] In this embodiment, although there is an error in the center point itself, the potential co-row and co-column characteristics can be easily identified in the projection structure. In this way, the intersection of the first straight line and the second straight line can be used to refit a center point to replace the original center point to achieve optimization of the spatial distribution position, wherein the process of determining the optimal center point can be but is not limited to as shown in the following step S44.

[0165] S44. The intersection point between the first straight line and the second straight line is used as the optimal center point of any classification point cloud set; in this embodiment, see Figure 3 As shown, Figure 3 G i represents the original center point, S i represents the optimal center point, so Figure 3It can be clearly seen that the position is optimized.

[0166] In this way, through the aforementioned steps S41 to S44, the position optimization of the center point of each classification point cloud set can be completed, so as to obtain the optimal center point of each classification point cloud set; then, the facade contour corresponding to each classification point cloud set can be constructed, and the contour optimization can be performed to restore the most realistic window facade structure of the target building; wherein, the facade contour construction and optimization process can be but is not limited to as shown in the following step S5.

[0167] S5. Based on the optimal center point of each classification point cloud set, construct the facade contour of each classification point cloud set, and optimize the facade contour of each classification point cloud set to obtain the optimized facade contour of each classification point cloud set after the optimization process; in the specific implementation, this embodiment takes into account the same structural arrangement characteristics of the building facade windows. In this shape optimization link, the windows of the same group are superimposed and fitted to obtain the most complete and accurate window contour, and match it to each center point, so as to solve the differences in scanning accuracy, omissions and interference of facade attachments; among them, any classification point cloud set is taken as an example for explanation, and the corresponding facade contour construction process can be but is not limited to the following steps S51 and S52.

[0168] S51. For any classified point cloud set, obtain the contours of each point cloud cluster in the said any classified point cloud set; in the specific implementation, for example, but not limited to, the α-shape algorithm can be used to extract the contour boundaries of each point cloud cluster; wherein, the α-shape algorithm can handle the concavity and convexity in the point cloud data, and can generate a polygonal shell to approximately represent the area occupied by the point cloud; of course, the α-shape algorithm is a commonly used algorithm for contour extraction, and its principle will not be repeated.

[0169] After the contours of each point cloud cluster in any classified point cloud set are obtained, the contours may be superimposed based on the optimal center point of any classified point cloud set, and the process may be but is not limited to that shown in the following step S52.

[0170] S53. Taking the optimal center point of any classified point cloud set as the insertion anchor point, the contours corresponding to each point cloud cluster in the any classified point cloud set are superimposed to obtain the facade contour of the any classified point cloud set after the superposition process; in this embodiment, contour superposition is performed based on the insertion anchor point, which is a commonly used technology for point cloud contour processing, and its principle will not be repeated here.

[0171] In this way, based on the aforementioned steps S51 and S52, the facade contours of each classified point cloud set can be constructed, and then the contour shape can be optimized; wherein, this embodiment merges similar contour line segments based on the similarity of the line segments in the contour, and the process can be but is not limited to as shown in the following steps S53 and S54.

[0172] S53. For any classified point cloud set, determine several similar line segment groups in the facade contour of any classified point cloud set, wherein any similar line segment group contains two similar contour line segments; in specific applications, for example, but not limited to, the following steps S53a to S53d can be used to determine the similar line segments in the facade contour of any of the aforementioned classified point cloud sets.

[0173] S53a. For any contour line segment in the facade contour of any classified point cloud set, calculate the angle difference, vertical distance and projection distance between any contour line segment and each target line segment in the target line segment set, wherein the target line segment set includes all contour line segments in the facade contour of any classified point cloud set except any contour line segment.

[0174] Optionally, the following formulas (9), (10) and (11) may be used, for example but not limited to, to calculate the angle difference, vertical distance and projection distance between any contour line segment and each target line segment in the target line segment set.

[0175] S θ (l a ,l b )=|L b ·sinθ| (9)

[0176]

[0177]

[0178] In the above formula (9), S θ (l a ,l b ) represents any contour segment l a The angle difference between the bth target line segment in the target line segment set, L b represents the length of the bth target line segment, and θ represents any contour line segment l a The angle between it and the bth target line segment.

[0179] In the above formula (10), S h (l a ,l b ) represents any contour segment l aThe vertical distance from the bth target line segment, h 1 ,h 2 Respectively represent the two endpoints of the bth target line segment, to any contour segment l a projection distance.

[0180] In the above formula (11), S d (l a ,l b ) represents any contour segment l a The projection distance between the bth target line segment, where intersects with means intersection, d p Represents any contour segment l a The minimum distance between the endpoints of the b-th target line segment; where b = 1, 2, 3, ..., Q, and Q is the total number of target line segments in the target line segment set.

[0181] In this way, based on the aforementioned formulas (9)-(11), after calculating the angle difference, vertical distance and projection distance between any contour line segment and each target line segment in the target line segment set, the line segment similarity between any contour line segment and each target line segment can be obtained based on this; wherein, the calculation process of the line segment similarity is shown in the following step S53b.

[0182] S53b. Based on the angle difference, vertical distance and projection distance between any one of the contour line segments and each target line segment in the target line segment set, calculate the segment similarity between any one of the contour line segments and each target line segment; in specific application, the b-th target line segment is still used as an example to illustrate, and the example may be but not limited to taking the angle difference, vertical distance and projection distance between any one of the contour line segments and the b-th target line segment as the segment similarity between the any one of the contour line segments and the b-th target line segment; in this way, after calculating the similarity between any one of the contour line segments and each target line segment in the aforementioned manner, the target line segment that is most similar to any one of the contour line segments may be selected based on this, and the process may be but not limited to as shown in the following steps S53c and S53d.

[0183] S53c. Determine whether the smallest line segment similarity among the line segment similarities is less than or equal to the similarity threshold; in specific implementation, if two line segments are similar, the above three similarity indicators all have smaller values. In other words, a smaller line segment similarity indicates that the two line segments are more similar. Therefore, this embodiment can set a similarity threshold to determine whether the target line segment with the smallest line segment similarity is similar to any contour line segment; wherein, if the smallest line segment similarity is less than or equal to the similarity threshold, then the target line segment corresponding to the smallest line segment similarity is similar to any contour line segment, and the two can be regarded as a group of similar line segment groups; otherwise, it is determined that the two are not similar line segments, and the next contour line segment needs to be determined; wherein, the process of determining the similar line segment group is shown in the following step S53d.

[0184] S53d. If so, then any one of the contour line segments and the target line segment with the smallest line segment similarity to any one of the contour line segments are used to form a similar line segment group, and after all the contour line segments in the facade contour of any one of the classified point cloud sets are polled, a number of similar line segment groups are obtained; in this embodiment, after a similar line segment group is determined, the straight lines in the similar line segment group can be deleted from the facade contour line segment set to avoid repeated calculation of the similarity; then, the next contour line segment is selected to perform a similarity determination until all the contour line segments in the facade contour selection set are polled; wherein, the facade contour line segment set contains all the contour line segments in the facade contour corresponding to any one of the classified point cloud sets.

[0185] Thus, through the aforementioned steps S53a to S53d, several similar line segment groups in the elevation contour of any of the classified point cloud sets can be determined; then, similar line segments can be merged, and the process can be but not limited to as shown in the following step S54.

[0186] S54. Perform segment merging processing on two similar contour line segments in each similar line segment group, so as to obtain the optimized facade contour of any classified point cloud set after the segment merging processing; in the specific implementation, since the merging process of the line segments in each similar line segment group is the same, the following takes any similar line segment group as an example for explanation, and its merging process can be but is not limited to the following steps S54a to S54f.

[0187] S54a. For any similar line segment group, the midpoint of the longest line segment of two similar contour line segments in the similar line segment group is used as a positioning point.

[0188] After the positioning point is determined based on the aforementioned step S54a, it is necessary to obtain the direction of the merged line segment, and the process may be but is not limited to the following steps S54b and S54c.

[0189] S54b. Obtain the similar line segment cluster corresponding to any of the similar line segment groups; in specific implementation, the target line segment whose line segment similarity is between the similarity threshold and the similarity upper limit value is used as the similar line segment cluster of the any of the similar line segment groups; as explained on the basis of the above examples; the line segment similarity of any of the contour line segments and each of the target line segments has been calculated in step S53b, then the target line segment whose line segment similarity is between the similarity threshold and the similarity upper limit value is used to form the similar line segment cluster of any of the contour line segments; of course, the similarity upper limit value is greater than the similarity threshold value, and can be specifically set according to actual use, and is not limited here.

[0190] In this way, after obtaining the similar line segment cluster, the main direction vector of the merged line segment can be determined based on the similar line segment cluster, and the determination process can be but not limited to the following step S54c.

[0191] S54c. The average vector of the direction vectors of all similar line segments in the similar line segment cluster is used as the main direction vector; after the main direction vector is determined based on step S54c, a new straight line can be generated in combination with the aforementioned positioning point, and the process can be but is not limited to as shown in the following step S54d.

[0192] S54d. Generate a new straight line corresponding to any of the similar line segment groups based on the positioning point and the main direction vector; in specific applications, it is equivalent to generating the new straight line with the aforementioned positioning point as the starting point and the main direction vector as the straight line direction; then, the new straight line can be intercepted to generate a merged line segment; wherein, the interception process of the new straight line can be but is not limited to as shown in the following step S54e.

[0193] S54e. Project the two endpoints of the longest line segment of two similar contour line segments in any similar line segment group onto the new straight line to obtain two projected endpoints.

[0194] After completing the projection of the two endpoints of the longest line segment between two similar contour line segments in any similar line segment group, the two projection endpoints can be found on the new straight line. Then, the line segment between the two projection endpoints can be used as the merged line segment corresponding to the two similar contour line segments in any similar line segment group. The process is shown in the following step S54f.

[0195] S54f. Take the line segment between the two projection endpoints on the new straight line as the merged line segment corresponding to any one of the similar line segment groups.

[0196] In this way, through the aforementioned steps S54a to S54f, the merging of all similar line segments in any classified point cloud set can be completed. At this time, the optimized facade contour of any classified point cloud set can be obtained; based on this, the optimization of the corresponding facade contours of the remaining classified point cloud sets can be completed using the same method as mentioned above.

[0197] Therefore, based on the aforementioned steps S51 to S54, multiple window outlines that can restore the real facade to the greatest extent can be obtained; then, based on this, a three-dimensional model of the building facade of the target building can be reconstructed, and the process can be but not limited to as shown in the following step S6.

[0198] S6. Reconstruct the three-dimensional model of the building facade of the target building using the optimized facade contours; in this embodiment, the facade layout diagram (i.e., the optimized facade contour) obtained in the aforementioned step S5 is used to optimize the facade structure of the 3D building model. At present, it is easy to obtain the LOD1 / 2 block model of the target building, wherein the facade is usually represented by a plane, and the image is used as a texture to describe the facade structure; in this way, using the facade layout diagram extracted above, the accurate distribution of the facade semantic structure that is closely consistent with the real structure can be achieved, thereby completing the refinement operation of the building model; wherein, Figure 4 The construction process of the three-dimensional model of the building facade is demonstrated, that is, the optimized facade contours (i.e., window contours) proposed in this embodiment are used in combination with the LOD1 / 2 block model of the target building to obtain the final three-dimensional model of the building facade; of course, reconstructing the three-dimensional facade model based on the two-dimensional facade contour is a common technology for facade reconstruction, and its principle will not be repeated here.

[0199] Thus, through the building facade reconstruction method based on laser point cloud described in detail in the aforementioned steps S1 to S6, the present invention extracts feature point cloud from the building facade point cloud through the facade semantic structure features of point cloud; then, using the point cloud similarity, similar feature point clouds are classified into one category to obtain multiple classified point cloud sets; then, the facade is optimized from the two aspects of shape and position, that is, the spatial distribution position of the classified point cloud set is first optimized to correct the position offset of the point cloud, so as to obtain the corresponding optimal center point; then, based on the optimal center point, the facade contour is constructed and the contour is optimized to restore the real structure of the facade to the greatest extent; finally, the three-dimensional model of the building facade can be reconstructed based on the optimized facade contour; thus, the present invention provides an automated, object-level, and highly accurate building facade reconstruction method, and only needs to use laser point cloud as a data source, thereby greatly reducing the complexity of reconstruction; thus, the present invention provides a breakthrough technical support for component-level real-scene three-dimensional construction, and is very suitable for large-scale application and promotion.

[0200] like Figure 5 As shown, the second aspect of this embodiment provides a hardware device for implementing the method for reconstructing a building facade based on laser point cloud described in the first aspect of the embodiment, including:

[0201] The acquisition unit is used to acquire the facade point cloud set of the target building.

[0202] The feature point extraction unit is used to determine the facade semantic structure features of each facade point cloud in the facade point cloud set, and extract feature point clouds from the facade point cloud set according to the facade semantic structure features of each facade point cloud to obtain a feature point cloud set.

[0203] The classification unit is used to classify the feature point cloud set based on the point cloud similarity to obtain multiple classification point cloud sets.

[0204] The spatial distribution position optimization unit is used to optimize the spatial distribution position of each classification point cloud set, so as to obtain the optimal center point of each classification point cloud set after the spatial distribution position optimization processing.

[0205] The contour optimization unit is used to construct the facade contour of each classification point cloud set based on the optimal center point of each classification point cloud set, and optimize the facade contour of each classification point cloud set to obtain the optimized facade contour of each classification point cloud set after the optimization processing.

[0206] The model reconstruction unit is used to reconstruct a three-dimensional model of the building facade of the target building using each optimized facade contour.

[0207] The working process, working details and technical effects of the device provided in this embodiment can be found in the first aspect of the embodiment and will not be described in detail here.

[0208] like Figure 6 As shown, the third aspect of this embodiment provides another device for reconstructing building facades based on laser point cloud. Taking the device as an electronic device as an example, it includes: a memory, a processor and a transceiver that are communicatively connected in sequence, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the method for reconstructing building facades based on laser point cloud as described in the first aspect of the embodiment.

[0209] For example, the memory may include, but is not limited to, random access memory (RAM), read only memory (ROM), flash memory, first input first output (FIFO) and / or first in last out (FILO), etc. Specifically, the processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). At the same time, the processor may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state.

[0210] In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. For example, the processor may not be limited to a microprocessor of the STM32F105 series, a reduced instruction set computer (RISC) microprocessor, an X86 architecture processor, or a processor with an integrated embedded neural network processing unit (NPU); the transceiver may be, but is not limited to, a wireless fidelity (WIFI) wireless transceiver, a Bluetooth wireless transceiver, a general packet radio service technology (General Packet Radio Service, GPRS) wireless transceiver, a ZigBee protocol (a low-power local area network protocol based on the IEEE802.15.4 standard, ZigBee) wireless transceiver, a 3G transceiver, a 4G transceiver and / or a 5G transceiver, etc. In addition, the device may also include, but is not limited to, a power module, a display screen, and other necessary components.

[0211] The working process, working details and technical effects of the electronic device provided in this embodiment can be found in the first aspect of the embodiment and will not be described in detail here.

[0212] The fourth aspect of this embodiment provides a storage medium storing instructions for the method for reconstructing a building facade based on laser point cloud as described in the first aspect of the embodiment, that is, the storage medium stores instructions, and when the instructions are run on a computer, the method for reconstructing a building facade based on laser point cloud as described in the first aspect of the embodiment is executed.

[0213] The storage medium refers to a carrier for storing data, which may include but is not limited to a floppy disk, a CD, a hard disk, a flash memory, a USB flash drive and / or a memory stick, and the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0214] The working process, working details and technical effects of the storage medium provided in this embodiment can be found in the first aspect of the embodiment and will not be described in detail here.

[0215] A fifth aspect of this embodiment provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method for reconstructing a building facade based on laser point cloud as described in the first aspect of the embodiment, wherein the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0216] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for building facade reconstruction based on laser point cloud, characterized in that: include: Obtain the facade point cloud set of the target building; Determining the facade semantic structure features of each facade point cloud in the facade point cloud set, and extracting feature point clouds from the facade point cloud set according to the facade semantic structure features of each facade point cloud to obtain a feature point cloud set; Based on the point cloud similarity, the feature point cloud set is classified to obtain a plurality of classified point cloud sets; Optimizing the spatial distribution position of each classification point cloud set, so as to obtain the optimal center point of each classification point cloud set after the spatial distribution position optimization processing; According to the optimal center point of each classification point cloud set, the facade contour of each classification point cloud set is constructed, and the facade contour of each classification point cloud set is optimized to obtain the optimized facade contour of each classification point cloud set after the optimization process; The three-dimensional model of the building facade of the target building is reconstructed using the optimized facade contours.

2. The method according to claim 1, characterized in that Determining the facade semantic structure features of each facade point cloud in the facade point cloud set includes: For any facade point cloud in the facade point cloud set, calculating the optimal neighborhood radius of any facade point cloud; According to the optimal neighborhood radius, obtaining an optimal neighborhood point cloud set of any of the facade point clouds; Based on the optimal neighborhood point cloud set and the centroid of the optimal neighborhood point cloud set, determining the optimal centroid displacement vector of any facade point cloud, and using the optimal centroid displacement vector as the facade semantic structure feature of any facade point cloud; Correspondingly, according to the facade semantic structure features of each facade point cloud, a feature point cloud is extracted from the facade point cloud set to obtain a feature point cloud set, which includes: Calculate the modulus of the best quality centroid displacement vector of each facade point cloud as the eigenvalue of each facade point cloud; From the facade point cloud set, facade point clouds with characteristic values ​​greater than or equal to a characteristic threshold are extracted as characteristic point clouds, so as to form the characteristic point cloud set using the extracted characteristic point clouds.

3. The method according to claim 2, characterized in that Calculating the optimal neighborhood radius of any of the facade point clouds, including: Randomly selecting a plurality of facade point clouds from the facade point cloud set as seed points; Obtaining the neighboring points of each seed point, and calculating the distance between each seed point and the corresponding neighboring points, so as to take the average value of the sum of the distances between each seed point and the corresponding neighboring points as the point density of any facade point cloud, wherein the neighboring points of any seed point are the facade point clouds in the neighborhood area of ​​any seed point; Determine a maximum neighborhood radius and a minimum neighborhood radius according to the point density of any of the facade point clouds; Based on the minimum neighborhood radius and the maximum neighborhood radius, a radius search interval is determined, and a neighborhood radius search is performed within the radius search interval to search for a neighborhood radius that minimizes the Shannon entropy of any of the facade point clouds; The neighborhood radius that minimizes the Shannon entropy of any of the facade point clouds is used as the optimal neighborhood radius of the any of the facade point clouds.

4. The method according to claim 3, characterized in that: For any neighborhood radius within the radius search interval, the Shannon entropy of any facade point cloud under the neighborhood radius is calculated in the following manner; Based on any neighborhood radius, obtaining a search neighborhood of any facade point cloud; Determine the covariance matrix of any of the facade point clouds according to the facade point clouds in the search neighborhood, and calculate the first eigenvalue, the second eigenvalue and the third eigenvalue of the covariance matrix; Calculating the probability that any one of the facade point clouds belongs to linear points and the probability that any one of the facade point clouds belongs to planar points according to the first eigenvalue, the second eigenvalue, and the third eigenvalue, and calculating the probability that any one of the facade point clouds belongs to scattered points according to the first eigenvalue and the third eigenvalue; Based on the probability that any of the facade point clouds belongs to linear points, the probability that any of the facade point clouds belongs to planar points, and the probability that any of the facade point clouds belongs to scattered points, and using the following formula (1), the Shannon entropy of any of the facade point clouds at any of the neighborhood radius is calculated; E(r i )=-(δ1·lnδ1+δ2·lnδ2+δ3·lnδ3) (1) In the above formula (1), E(r i ) represents any facade point cloud within any neighborhood radius r i Shannon entropy under δ1, δ2, δ3 represent the probability that any vertical point cloud belongs to a linear point, a plane point, and a scattered point respectively, where r i ∈[r min ,r max ], and r min ,r max Represent the minimum neighborhood radius and the maximum neighborhood radius respectively.

5. The method according to claim 2, characterized in that: Based on the optimal neighborhood point cloud set and the centroid of the optimal neighborhood point cloud set, determining the optimal centroid displacement vector of any facade point cloud, including: According to the following formula (2), the optimal centroid displacement vector of any vertical point cloud is calculated; In the above formula (2), ε i represents the optimal centroid displacement vector of any vertical point cloud, Ω(p i ) represents any of the vertical point clouds p i The optimal neighborhood point cloud set, p j represents the jth neighborhood point cloud in the optimal neighborhood point cloud set, m j Indicates p j The centroid weighted point, Represents any of the facade point clouds p i To the centroid weighted point m j A vector of , ω(i,j) represents the vector weight; in, ω(i,j)=(-||p i -p j || 2 / R 2 ) (4) In formula (3), c i represents the centroid of the optimal neighborhood point cloud set, Represents any of the facade point clouds p i To the jth neighborhood point cloud p j The vector of Represents any of the facade point clouds p i The vector to the centroid of the optimal neighborhood point cloud; In formula (4), R represents the average distance between any facade point cloud and all neighboring point clouds in the optimal neighboring point cloud set, and || || represents the modulo operation symbol.

6. The method according to claim 1, characterized in that Based on the point cloud similarity, the feature point cloud set is classified to obtain multiple classified point cloud sets, including: Performing vertical slicing processing on the feature point cloud set to obtain slice point cloud data; Performing preliminary segmentation processing on the slice point cloud data by using a region growing algorithm, so as to obtain a plurality of point cloud clusters after the preliminary segmentation processing; Extract the FPFH features of each feature point cloud in each point cloud cluster, and calculate the Hausdorff distance between each point cloud cluster based on the FPFH features of each feature point cloud in each point cloud cluster; Based on the Hausdorff distance between each point cloud cluster, a Hausdorff distance matrix is ​​constructed; According to the Hausdorff distance matrix, a hierarchical clustering algorithm is used to perform clustering processing on each point cloud cluster, so as to obtain a plurality of classified point cloud sets after clustering processing.

7. The method according to claim 1, characterized in that The spatial distribution position of each classification point cloud set is optimized to obtain the optimal center point of each classification point cloud set after the spatial distribution position optimization processing, including: For any classified point cloud set, determine the center point and the plane bounding box of any classified point cloud set; According to the plane bounding box, a plane coordinate system is established, wherein the origin of the plane coordinate system is the endpoint of the lower left corner of the plane bounding box, and the x-axis of the plane coordinate system is the horizontal direction, and the z-axis is the vertical direction; Projecting the center point to the x-axis direction and the z-axis direction of the plane coordinate system respectively, and determining a first straight line and a second straight line, wherein the first straight line is a straight line co-columned with the projection point of the center point in the z-axis direction, and the second straight line is a straight line co-rowed with the projection point of the center point in the x-axis direction; The intersection point between the first straight line and the second straight line is used as the optimal center point of any classification point cloud set.

8. The method according to claim 1, characterized in that Any classified point cloud set contains multiple point cloud clusters, among which, according to the optimal center point of each classified point cloud set, the facade outline of each classified point cloud set is constructed, including: For any classified point cloud set, obtaining the outline of each point cloud cluster in the any classified point cloud set; Taking the optimal center point of any classified point cloud set as the insertion anchor point, superimposing the contours corresponding to each point cloud cluster in the any classified point cloud set, so as to obtain the elevation contour of the any classified point cloud set after superimposing; Accordingly, the elevation contours of each classified point cloud set are optimized to obtain the optimized elevation contours of each classified point cloud set after the optimization process, which includes: For any classified point cloud set, determining a plurality of similar line segment groups in the elevation contour of the any classified point cloud set, wherein any similar line segment group contains two similar contour line segments; Line segment merging processing is performed on two similar contour line segments in each similar line segment group, so as to obtain an optimized facade contour of any classified point cloud set after the line segment merging processing.

9. The method according to claim 8, characterized in that Determining a plurality of similar line segment groups in the elevation contour of any of the classified point cloud sets comprises: For any contour line segment in the elevation contour of any classified point cloud set, calculate the angle difference, vertical distance and projection distance between any contour line segment and each target line segment in the target line segment set, wherein the target line segment set includes all contour line segments in the elevation contour of any classified point cloud set except the any contour line segment; Calculating the line segment similarity between any contour line segment and each target line segment in the target line segment set based on the angle difference, vertical distance and projection distance between any contour line segment and each target line segment in the target line segment set; Determine whether the smallest line segment similarity among the line segment similarities is less than or equal to the similarity threshold; If so, any of the contour line segments and the target line segment with the smallest line segment similarity to any of the contour line segments are used to form a similar line segment group, and after all contour line segments of the facade contour of any of the classified point cloud sets are polled, several similar line segment groups are obtained.

10. A building facade reconstruction device based on laser point cloud, characterized in that: include: An acquisition unit, used for acquiring a facade point cloud set of a target building; A feature point extraction unit is used to determine the facade semantic structure feature of each facade point cloud in the facade point cloud set, and extract a feature point cloud from the facade point cloud set according to the facade semantic structure feature of each facade point cloud to obtain a feature point cloud set; A classification unit, used for classifying the feature point cloud set based on point cloud similarity to obtain multiple classification point cloud sets; A spatial distribution position optimization unit is used to optimize the spatial distribution position of each classification point cloud set, so as to obtain the optimal center point of each classification point cloud set after the spatial distribution position optimization processing; A contour optimization unit is used to construct a facade contour of each classification point cloud set according to the optimal center point of each classification point cloud set, and optimize the facade contour of each classification point cloud set to obtain an optimized facade contour of each classification point cloud set after the optimization process; The model reconstruction unit is used to reconstruct the three-dimensional model of the building facade of the target building by using each optimized facade contour.

Images