A building instance segmentation method, device, terminal and storage medium

Abstract

The present invention provides a method, device, terminal, and storage medium for instanced building segmentation, relating to the technical field of three-dimensional point cloud processing. The method comprises: performing planar segmentation on a target building's building point cloud, extracting potential vertical plane point clusters from the segmented building plane point clusters; mapping all three-dimensional point clouds in the potential vertical plane point clusters to a two-dimensional feature space to obtain a two-dimensional point set, processing the two-dimensional point set using a preset dual-threshold judgment mechanism to extract the building's vertical structure point clusters; generating a building distance map based on the building's vertical structure point clusters, and generating instance anchor points on the distance map; extracting multi-level contours from the distance map based on the instance anchor points, constructing a tree-like hierarchical relationship corresponding to the multi-level contours, and constructing a labeling matrix based on the tree-like hierarchical relationship; and performing two-dimensional instanced building segmentation on the distance map based on the labeling matrix. The present invention, through an unsupervised hierarchical segmentation framework, can eliminate dependence on labeled data and improve segmentation accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional point cloud processing, and in particular to a method, device, terminal and storage medium for instantiating and segmenting a building. Background Art

[0002] At present, instance-level segmentation of three-dimensional models of urban buildings has become a core requirement in fields such as smart city management, real estate registration, and BIM (Building Information Modeling) modeling.

[0003] Current mainstream LiDAR and oblique photogrammetry technologies can quickly acquire large-scale point cloud data for urban scenes. However, achieving accurate segmentation of individual building instances still faces major technical bottlenecks. Traditional manual labeling methods are inefficient and limited by the subjective judgment of operators, making it difficult to meet the accuracy and timeliness requirements of city-level 3D modeling.

[0004] Furthermore, existing deep learning-based instance segmentation methods are highly data-dependent and require large-scale annotated datasets, which are expensive for urban scene annotation. These methods also suffer from poor generalization, resulting in a significant drop in model accuracy when migrating across scenarios. Furthermore, they have high hardware requirements, making them difficult to deploy on mobile devices. Furthermore, a clustering method based on local geometric features utilizes RANSAC (RANdom SAmple Consensus) plane fitting combined with Euclidean clustering to cluster building instances. However, this method clusters only based on local normal vector similarity, potentially leading to the incorrect merging of corridor-style buildings. Furthermore, this method has poor noise immunity, resulting in a high failure rate in facade structure extraction when the density of the 2D point cloud is uneven, and the clustering radius threshold requires repeated manual adjustment.

[0005] In summary, the existing technology has technical defects such as strong dependence on labeled data, poor generalization ability, missing global features, and projection distortion, which need to be improved and developed. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a building instance segmentation method, device, terminal and storage medium for the above-mentioned defects of the prior art, which can realize the accurate segmentation of single building instances, improve the segmentation accuracy, reduce the vertical structure misjudgment rate, and eliminate the dependence on labeled data.

[0007] The technical solutions adopted by the present invention to solve the technical problems are as follows:

[0008] A building instance segmentation method, wherein the method comprises:

[0009] Constructing a building point cloud of a target building, performing plane segmentation on the building point cloud, and extracting potential vertical plane point clusters from the segmented building plane point clusters;

[0010] Mapping all three-dimensional point clouds in the point cloud set corresponding to the potential vertical plane point cluster to a two-dimensional feature space to obtain a two-dimensional point set, using a preset double-threshold judgment mechanism and extracting a two-dimensional point cloud of the building's vertical structure based on the two-dimensional point set to obtain a building vertical structure point cluster;

[0011] generating a building distance map based on the building vertical structure point clusters, and generating building instance anchor points representing positions of potential building instances on the building distance map;

[0012] Extracting multi-level contours in the building distance map based on the building instance anchor points, constructing a tree-like hierarchical relationship corresponding to the multi-level contours, and constructing a corresponding labeling matrix based on the tree-like hierarchical relationship;

[0013] The building distance map is segmented into two-dimensional buildings based on the label matrix to obtain two-dimensional building instantiation objects; wherein the two-dimensional pixels of the two-dimensional building instantiation objects carry building instantiation labels.

[0014] In one implementation, performing plane segmentation on the building point cloud and extracting potential vertical plane point clusters from the segmented building plane point clusters includes:

[0015] Performing plane segmentation on the building point cloud using a plane region growing algorithm to obtain building plane point clusters;

[0016] Performing plane fitting on the building plane point cluster using the least square method to obtain an optimal plane equation;

[0017] Potential vertical plane point clusters are extracted based on the optimal plane equation and the size of the building plane point clusters.

[0018] In one implementation, mapping all three-dimensional point clouds in the point cloud set corresponding to the potential vertical plane point cluster to a two-dimensional feature space to obtain a two-dimensional point set includes:

[0019] Constructing a ground point cloud, and generating a ground horizontal reference plane based on the ground point cloud fitting;

[0020] On the ground horizontal reference plane, establishing a local coordinate system based on the centroid point of the ground point cloud;

[0021] Projecting all three-dimensional point clouds in the point cloud set corresponding to the potential vertical plane point cluster to the local coordinate system along the normal vector direction of the ground point cloud to obtain a two-dimensional point set;

[0022] The method of using a preset dual-threshold determination mechanism and extracting a two-dimensional point cloud of a building's vertical structure based on the two-dimensional point set to obtain a building's vertical structure point cluster includes:

[0023] Establishing a two-dimensional grid of the two-dimensional point set on the ground horizontal reference plane;

[0024] Counting the number of two-dimensional point clouds in each grid cell of the two-dimensional grid to obtain a vertical projection density feature corresponding to each grid cell;

[0025] Determining the highest point height and the lowest point height of each grid unit in the two-dimensional grid, and calculating the difference between the highest point height and the lowest point height to obtain a vertical height difference feature corresponding to each grid unit;

[0026] Determining a density threshold based on the vertical projection density feature, and determining a depth threshold based on the vertical height difference feature;

[0027] Using a preset dual-threshold determination mechanism, the vertical projection density feature corresponding to each grid unit is compared with the density threshold, and the vertical height difference feature is compared with the depth threshold;

[0028] When the vertical projection density feature is greater than the density threshold and the vertical height difference feature is greater than the depth threshold, the two-dimensional point cloud in the grid unit is extracted as a building vertical structure two-dimensional point cloud to obtain a building vertical structure point cluster.

[0029] In one implementation, determining a density threshold based on the vertical projection density feature and determining a depth threshold based on the vertical height difference feature includes:

[0030] Determining the mean and variation of the two-dimensional point cloud density within the grid unit based on the vertical projection density feature, and processing the mean and the variation using a preset density threshold calculation formula to obtain a density threshold;

[0031] The vertical height difference feature is processed using the preset depth threshold calculation formula to obtain the depth threshold;

[0032] The calculation formula for the preset density threshold is:

[0033] ;

[0034] The preset depth threshold calculation formula is:

[0035] ;

[0036] ;

[0037] and, represents the density threshold, represents the mean value of the 2D point cloud density within the grid cell, represents the variability of the 2D point cloud density within the grid cell, represents the depth threshold, Represents the vertical height difference feature, Indicates the height of the highest point in the grid cell, Indicates the lowest point height in the grid cell, Represents the threshold control coefficient.

[0038] In one implementation, generating a building distance map based on the building vertical structure point clusters and generating building instance anchor points representing positions of potential building instances on the building distance map includes:

[0039] Projecting all the two-dimensional point clouds of the building vertical structure in the building vertical structure point cluster onto the ground horizontal reference plane to obtain a building projection point set;

[0040] Determining the maximum longitudinal coordinate value and the minimum longitudinal coordinate value, as well as the maximum transverse coordinate value and the minimum transverse coordinate value of the building projection points in the projection point set;

[0041] Calculate the image width and image height based on a preset image resolution, the maximum value and the minimum value of the longitudinal coordinate, and the maximum value and the minimum value of the transverse coordinate;

[0042] Creating an initial binary image according to the image width and the image height;

[0043] Translating all the building projection points in the building projection point set to an image coordinate system constructed based on the initial binary image to obtain a target binary image;

[0044] Calculating the Euclidean distance between each pixel in the target binary image and the background pixel to obtain the distance value corresponding to each pixel;

[0045] Linearly mapping the distance value corresponding to each pixel to a standard range of grayscale values ​​to obtain a building distance map;

[0046] Traversing the building distance map in a sliding window manner, and sequentially calculating the average of the distance values ​​corresponding to all pixels in the window area to obtain an adaptive threshold corresponding to the window area;

[0047] The distance value corresponding to each pixel in the window area is compared with the adaptive threshold, and the pixel area with the distance value higher than the adaptive threshold is determined as the foreground, so as to obtain a building instance anchor point representing the position of the potential building instance on the building distance map.

[0048] In one implementation, before extracting the multi-level contours in the building distance map based on the building instance anchor points, the method further includes:

[0049] A morphological erosion operation is performed on the building instance anchor points on the building distance map.

[0050] In one implementation, after performing two-dimensional building instantiation segmentation on the building distance map based on the label matrix to obtain two-dimensional building instantiation objects, the method further includes:

[0051] Performing a morphological dilation operation on the two-dimensional building instantiation object, and extracting the maximum contour of the dilated two-dimensional building instantiation object;

[0052] Simplifying the maximum contour to obtain a simplified maximum contour, and determining the two-dimensional pixels within the simplified maximum contour as target building instantiation pixels;

[0053] Based on the pre-constructed mapping relationship between pixel coordinates and three-dimensional space coordinates, three-dimensional space mapping processing is performed on the instantiated pixels of the target building to obtain a three-dimensional instantiated object of the building.

[0054] The present invention also discloses a device for instantiating and segmenting a building, wherein the device comprises:

[0055] Building point cloud construction module, used to construct the building point cloud of the target building;

[0056] A plane point cluster extraction module is used to perform plane segmentation on the building point cloud and extract potential vertical plane point clusters from the segmented building plane point clusters;

[0057] A building vertical structure extraction module is used to map all three-dimensional point clouds in the point cloud set corresponding to the potential vertical plane point cluster to a two-dimensional feature space to obtain a two-dimensional point set, and to extract the two-dimensional point cloud of the building vertical structure based on the two-dimensional point set using a preset dual-threshold judgment mechanism to obtain a building vertical structure point cluster;

[0058] An instance anchor point generation module is used to generate a building distance map based on the building vertical structure point clusters, and generate building instance anchor points representing the positions of potential building instances on the building distance map;

[0059] A tree-like hierarchical relationship construction module is used to extract multi-level contours in the building distance map based on the building instance anchor points, and construct a tree-like hierarchical relationship corresponding to the multi-level contours;

[0060] A marker matrix construction module, configured to construct a corresponding marker matrix based on the tree-like hierarchical relationship;

[0061] The instantiation segmentation module is used to perform two-dimensional building instantiation segmentation on the building distance map based on the label matrix to obtain a two-dimensional building instantiation object; wherein the two-dimensional pixels of the two-dimensional building instantiation object carry a building instantiation label.

[0062] The present invention also discloses a terminal, which includes: a memory, a processor, and a building instantiation segmentation program stored in the memory and runnable on the processor. When the building instantiation segmentation program is executed by the processor, the steps of the building instantiation segmentation method described above are implemented.

[0063] The present invention also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program can be executed to implement the steps of the building instantiation segmentation method as described above.

[0064] The present invention provides a method, device, terminal and storage medium for building instance segmentation. The method comprises: constructing a building point cloud of a target building, performing plane segmentation on the building point cloud, and extracting potential vertical plane point clusters from the segmented building plane point clusters; mapping all three-dimensional point clouds in a point cloud set corresponding to the potential vertical plane point clusters to a two-dimensional feature space to obtain a two-dimensional point set, using a preset double-threshold judgment mechanism and extracting a two-dimensional point cloud of a building vertical structure based on the two-dimensional point set to obtain a building vertical structure point cluster; generating a building distance map based on the building vertical structure point cluster, and generating building instance anchor points representing the positions of potential building instances on the building distance map; extracting multi-level contours in the building distance map based on the building instance anchor points, constructing a tree-like hierarchical relationship corresponding to the multi-level contours, and constructing a corresponding labeling matrix based on the tree-like hierarchical relationship; performing two-dimensional building instance segmentation on the building distance map based on the labeling matrix to obtain a two-dimensional building instance object; wherein the two-dimensional pixels of the two-dimensional building instance object carry a building instance label. It can be seen from this that the present invention realizes accurate identification of vertical structures through a preset dual-threshold judgment mechanism, reduces the misjudgment rate of vertical structures, and realizes accurate division of complex building instances by mapping three-dimensional point clouds to two-dimensional feature spaces, thereby improving segmentation accuracy. It also realizes an unsupervised hierarchical segmentation framework through a three-level optimization architecture of three-dimensional point clouds, plane point clusters and instance anchors, as well as a contour tree structure. It can eliminate the dependence on labeled data in the process of segmenting individual building instances, realize rapid processing of city-level point clouds, significantly improve the processing capabilities of complex buildings such as special-shaped buildings and adhesion structures, and improve segmentation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a flow chart of a preferred embodiment of the building instance segmentation method of the present invention;

[0066] Figure 2 This is a schematic diagram of a specific process for extracting vertical structure point clusters of buildings disclosed in the present invention;

[0067] Figure 3 This is a schematic diagram of a specific adaptive building anchor point generation process disclosed in the present invention;

[0068] Figure 4 This is a schematic diagram of a specific morphology-based building instance segmentation process disclosed in the present invention;

[0069] Figure 5 This is a functional principle block diagram of a preferred embodiment of the building instantiation segmentation device of the present invention;

[0070] Figure 6 It is a functional principle block diagram of a preferred embodiment of the terminal in the present invention. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solutions and advantages of the present invention more clear and distinct, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0072] See Figure 1 , Figure 1 This is a flow chart of the building instance segmentation method in the present invention. Figure 1 As shown, the building instance segmentation method according to the embodiment of the present invention includes:

[0073] Step S11: construct a building point cloud of the target building, perform plane segmentation on the building point cloud, and extract potential vertical plane point clusters from the segmented building plane point clusters.

[0074] In this embodiment, for instance-level segmentation of a building three-dimensional model, a building point cloud of the target building is first constructed, and then the building point cloud is plane segmented to obtain building plane point clusters, and potential vertical plane point clusters are extracted from the segmented building plane point clusters.

[0075] Specifically, the planar region growing algorithm is used to segment the building point cloud into planes, generating building plane point clusters. The least squares method is then used to fit the building plane point clusters to obtain the optimal plane equation. Based on the optimal plane equation and the size of the building plane point clusters, potential vertical plane point clusters are extracted. The optimal plane equation can describe the overall geometric structure of the point cluster, such as the wall or roof plane of a building.

[0076] For example, when using the plane region growing algorithm to achieve plane segmentation of building point clouds, the minimum plane size can be set to 10 points and the maximum acceptable angle is , and the maximum point-to-plane distance is 0.5m. Then, the least squares method is used to perform plane fitting on the segmented building plane point clusters to obtain the optimal plane equation ax+by+cz+d=0. Assume that the size of the building plane point cluster is s, c and s are jointly constrained, and potential vertical plane point clusters are extracted based on the following formula, namely:

[0077] ;

[0078] in, is the size of the largest planar point cluster.

[0079] Step S12: Map all three-dimensional point clouds in the point cloud set corresponding to the potential vertical plane point cluster to a two-dimensional feature space to obtain a two-dimensional point set, use a preset dual-threshold judgment mechanism, and extract the two-dimensional point cloud of the building's vertical structure based on the two-dimensional point set to obtain a building vertical structure point cluster.

[0080] In this embodiment, all three-dimensional point clouds in the point cloud set corresponding to the potential vertical plane point cluster are mapped to the two-dimensional feature space to obtain a two-dimensional point set. Specifically, a ground point cloud is constructed, and a ground horizontal reference plane is generated based on the ground point cloud fitting; on the ground horizontal reference plane, a local coordinate system is established based on the centroid point of the ground point cloud; all three-dimensional point clouds in the point cloud set corresponding to the potential vertical plane point cluster are projected to the local coordinate system along the normal vector direction of the ground point cloud to obtain a two-dimensional point set.

[0081] For example, the point cloud set corresponding to the potential vertical plane point cluster can be , and then all points in the point cloud collection are aligned along the normal vector direction of the ground point cloud Projected onto the ground horizontal datum L , and the centroid of the ground point cloud , establish a local coordinate system ,Right now:

[0082] ;

[0083] Among them, the coordinates of the jth three-dimensional point in the point cloud set are: , ;

[0084] and, represents the x-coordinate of the j-th 3D point cloud in the point cloud set corresponding to the potential vertical plane point cluster, Represents the y coordinate of the jth 3D point cloud in the point cloud set corresponding to the potential vertical plane point cluster, Represents the z coordinate of the jth 3D point cloud in the point cloud set corresponding to the potential vertical plane point cluster.

[0085] Then, all point clouds in the point cloud collection are aligned along the normal vector direction of the ground point cloud. Projected to the local coordinate system, we get a two-dimensional point set , and the two-dimensional coordinates of the i-th point in the two-dimensional point set are , .

[0086] in, ;

[0087] and, represents the x-coordinate of the i-th two-dimensional point cloud in the two-dimensional point set, represents the y coordinate of the i-th two-dimensional point cloud in the two-dimensional point set, represents the i-th 3D point cloud in the point cloud set corresponding to the potential vertical plane point cluster, and .

[0088] In this embodiment, a preset dual-threshold judgment mechanism is used, and a two-dimensional point cloud of the vertical structure of the building is extracted based on the two-dimensional point set to obtain a vertical structure point cluster of the building. That is, the two-dimensional point set is processed using the preset dual-threshold judgment mechanism to extract the vertical structure point cluster of the building. The dual threshold in the preset dual-threshold judgment mechanism represents a dual threshold determined based on the two-dimensional point set, namely, a density threshold and a depth threshold, and is not a pre-set fixed threshold. The dual threshold is then used for judgment to extract the vertical structure point cluster of the building. Specifically, a two-dimensional grid of a two-dimensional point set is established on a horizontal reference plane of the ground; the number of two-dimensional point clouds in each grid cell of the two-dimensional grid is counted to obtain the vertical projection density feature corresponding to each grid cell; the highest point height and the lowest point height of each grid cell in the two-dimensional grid are determined, and the difference between the highest point height and the lowest point height is calculated to obtain the vertical height difference feature corresponding to each grid cell; the density threshold of the two-dimensional grid is determined based on the vertical projection density feature corresponding to each grid cell, and the depth threshold corresponding to the grid cell is determined based on the vertical height difference feature corresponding to the grid cell; using a preset dual-threshold judgment mechanism, the vertical projection density feature corresponding to each grid cell is compared with the density threshold, and the vertical height difference feature is compared with the depth threshold; when the vertical projection density feature corresponding to the grid cell is greater than the density threshold corresponding to the two-dimensional grid and the vertical height difference feature corresponding to the grid cell is greater than the depth threshold corresponding to the grid cell, the two-dimensional point cloud in the grid cell is extracted as the two-dimensional point cloud of the vertical structure of the building to obtain a point cluster of the vertical structure of the building.

[0089] In this embodiment, the dual threshold calculation method is specifically as follows: based on the vertical projection density feature, the mean and variation of the two-dimensional point cloud density within the grid unit are determined, and the mean and variation are processed using a preset density threshold calculation formula to obtain a density threshold; the vertical height difference feature is processed using a preset depth threshold calculation formula to obtain a depth threshold;

[0090] The preset density threshold calculation formula is:

[0091] ;

[0092] The preset depth threshold calculation formula is:

[0093] ;

[0094] ;

[0095] and, represents the density threshold corresponding to the two-dimensional grid, represents the mean value of the 2D point cloud density within the grid cell, represents the variability of the 2D point cloud density within the grid cell, represents the depth threshold corresponding to the grid cell, Represents the vertical height difference feature, Indicates the height of the highest point in the grid cell, Indicates the lowest point height in the grid cell, represents the threshold control coefficient, and the threshold control coefficient can be set to , at this time, the depth threshold can be .

[0096] For example, at the ground point horizontal datum L Build a 2D point cloud covering all 2D point sets Two-dimensional grid, count the number of two-dimensional point clouds in each grid cell, and obtain the vertical projection density feature , the spatial resolution is expressed as a two-dimensional density field to quantify the spatial distribution characteristics of the point cloud. At the same time, the height difference between the highest point and the lowest point in each grid cell in the two-dimensional grid is calculated to obtain the vertical height difference characteristics of each grid cell, which is used to quantitatively express the spatial heterogeneity in the vertical direction of the point cloud data, that is:

[0097] ;

[0098] in, represents the vertical projection density feature of the mth grid cell, Represents the mth grid unit, 1 is the characteristic function, which takes the value 1 when the condition is met, otherwise it is 0. That is, the characteristic function is a function that maps from the entire set to {0,1}. In image processing, the characteristic function can be used to mark whether a pixel belongs to the target area. Represents a grid cell The x-coordinate of the i-th 2D point cloud, where n represents the grid unit The number of inner 2D point clouds, represents the vertical height difference feature of the mth grid cell, Indicates the height of the highest point in the mth grid cell, Indicates the height of the lowest point in the mth grid cell.

[0099] For example, the mean and variation of the two-dimensional point cloud density within the grid unit are determined by statistical analysis methods, namely:

[0100] ;

[0101] Where M is the total number of grid cells in the two-dimensional grid.

[0102] Furthermore, a density threshold corresponding to the two-dimensional grid is set based on the mean and variation, and a relative depth threshold constraint for the column height is set based on the vertical height difference feature. Interference point clouds below the set threshold are eliminated, thereby enhancing the selectivity of relevant vertical features.

[0103] Finally, the double threshold is used to extract the final vertical structure of the building, namely:

[0104] .

[0105] For example, see Figure 2 As shown in the figure, the extraction process of the vertical structure of the building is as follows: the plane region growing algorithm is used to perform plane segmentation on the building point cloud to obtain the building plane point cluster, and multi-scale point cluster segmentation is realized, and then the three-dimensional feature calculation of the point cluster is performed. That is, the plane point cluster of the building is fitted using the least squares method to obtain the optimal plane equation ax+by+cz+d=0. The size of the building plane point cluster is set to s, and c and s are jointly constrained to extract potential vertical plane point clusters. At the same time, based on the ground point cloud, the ground horizontal reference plane is generated by fitting using the least squares method. L , and calculate the horizontal reference plane normal vector N, the ground point cloud centroid point , used as the subsequent projection reference plane, on the ground horizontal reference plane, a local coordinate system is established based on the centroid point of the ground point cloud, and all three-dimensional point clouds in the point cloud set corresponding to the potential vertical plane point cluster are projected to the local coordinate system along the normal vector direction of the ground point cloud to obtain a two-dimensional point set, and then the vertical projection density feature and the vertical height difference feature are calculated, and the density threshold and the height threshold are set based on the vertical projection density feature and the vertical height difference feature. Finally, the dual threshold is combined to extract the final vertical structure of the building, that is, through the two-dimensional projection density-depth dual threshold judgment mechanism, a local coordinate system is established on the ground horizontal reference plane as the reference plane to accurately identify the vertical structure, which can reduce the misjudgment rate of the vertical structure.

[0106] It is important to note that the vertical structure of a building is the primary basis for distinguishing different building instances. By mapping a 3D point cloud into a 2D feature space through constrained projection, the model overcomes the limitations of traditional single-dimensional feature analysis and solves the challenge of segmenting irregularly shaped buildings. During the instanced segmentation process, the model achieves an Intersection over Union (IoU) of 92.3% on a standard test set. IoU is a core evaluation metric for instance segmentation tasks, measuring the overlap between the predicted and true regions. A 92.3% IoU indicates that the model achieves extremely high segmentation accuracy on the standard test set, particularly for object boundaries and detail processing.

[0107] Step S13: generating a building distance map based on the building vertical structure point clusters, and generating building instance anchor points representing positions of potential building instances on the building distance map.

[0108] In this embodiment, a preset dual-threshold determination mechanism is used to process the two-dimensional point sets corresponding to potential vertical plane point clusters. After extracting building vertical structure point clusters, a building distance map is generated based on these clusters. This building distance map is a grayscale image obtained by mapping the Euclidean distance from each pixel in the binary image generated based on the building vertical structure point clusters to background pixels within a standard grayscale value range. Each pixel in this grayscale image has a uniquely corresponding distance value. Next, building instance anchor points are generated to represent the positions of potential building instances on the building distance map. In other words, building instance anchor points on the building distance map are adaptively generated. Building anchor points refer to the positions of potential building instances on the building distance map.

[0109] Specifically, all the two-dimensional point clouds of the vertical structure of the building in the vertical structure point cluster are projected onto the horizontal reference plane of the ground to obtain a set of building projection points; the maximum longitudinal coordinate and the minimum longitudinal coordinate, as well as the maximum transverse coordinate and the minimum transverse coordinate of the building projection points in the projection point set are determined; based on the preset image resolution, the maximum longitudinal coordinate and the minimum longitudinal coordinate, as well as the maximum transverse coordinate and the minimum transverse coordinate, the image width and image height are calculated respectively; an initial binary image is created according to the image width and image height; all the building projection points in the building projection point set are translated to the image coordinate system constructed based on the initial binary image to obtain the target image. A binary image is labeled; the Euclidean distance between each pixel in the target binary image and the background pixel is calculated to obtain the distance value corresponding to each pixel; the distance value corresponding to each pixel is linearly mapped to the standard range of grayscale values ​​to obtain a building distance map; the building distance map is traversed by a sliding window, and the average value of the distance values ​​corresponding to all pixels in the window area is calculated in turn to obtain an adaptive threshold corresponding to the window area; the distance value corresponding to each pixel in the window area is compared with the adaptive threshold, and the pixel area with a distance value higher than the adaptive threshold is determined as the foreground, thereby obtaining a building instance anchor point representing the position of the potential building instance on the building distance map.

[0110] For example, see Figure 3 As shown, all the original point clouds in the building vertical structure point cluster are projected onto the ground horizontal reference plane to obtain the building projection point set, and the maximum longitudinal coordinate and minimum longitudinal coordinate, as well as the maximum transverse coordinate and minimum transverse coordinate of the building projection point in the projection point set are determined, that is, , , , , preset image resolution It can be 0.08, and the image width and height are calculated as follows:

[0111] ;

[0112] ;

[0113] Then, an initial binary image Img is created based on the image width width and image height height. All building projection points in the building projection point set are translated to the image coordinate system constructed based on the initial binary image to obtain the target binary image, namely:

[0114] ;

[0115] ;

[0116] Among them, the preset offset , , height represents the image height, Represents the horizontal coordinate of the building projection point, Indicates the vertical coordinate of the building projection point.

[0117] Then, the Euclidean distance from each pixel in the target binary image to the background pixel (black pixel) is calculated, and the distance value is linearly mapped to the standard range of grayscale values, that is, the range of [0, 255], retaining the relative relationship with the distance value, and adapting the image display format to obtain a building distance map in grayscale display format. Then, the building distance map is traversed through the sliding window, and the average distance value corresponding to all pixels in the window area is calculated in turn. ,set up is the adaptive threshold, which determines whether the distance value corresponding to each pixel in the window area is greater than the adaptive threshold , the distance value is higher than the adaptive threshold The pixel area is set as the foreground (white), and the rest is the background (black). The different areas in the foreground are the building instance anchors.

[0118] Step S14: extracting multi-level contours in the building distance map based on the building instance anchor points, constructing a tree-like hierarchical relationship corresponding to the multi-level contours, and constructing a corresponding labeling matrix based on the tree-like hierarchical relationship.

[0119] In this embodiment, after generating building instance anchor points in the building distance map, building instance segmentation can be achieved based on morphology. That is, two-dimensional building instance segmentation is performed based on the morphological features of the building instance anchor points in the building distance map. First, multi-level contours in the building distance map are extracted based on the building instance anchor points, a tree-like hierarchical relationship corresponding to the multi-level contours is constructed, and a corresponding labeling matrix is ​​constructed based on the tree-like hierarchical relationship. Then, the subsequent two-dimensional building instance segmentation of step S15 is performed. Before extracting the multi-level contours in the building distance map based on the building instance anchor points, the process can further include performing a morphological erosion operation on the building instance anchor points in the building distance map. The erosion kernel size can be set to 2x2.

[0120] Among them, in the process of multi-level contour extraction and hierarchical relationship modeling, first, the multi-level contours in the building distance map are extracted, and then the corresponding tree-like hierarchical relationship can be constructed using the RETR_TREE mode. Among them, the RETR_TREE mode is one of the retrieval modes of the contour detection function, which is used to completely reconstruct the hierarchical relationship (tree structure) of the contour, index the parent and child nodes of each contour, and finally classify the contours into root nodes, intermediate nodes, and leaf nodes according to the hierarchical relationship, and visualize them through color marking, such as red marking the root node, the default color marking the intermediate node, and green marking the leaf node. In addition, in the process of marker matrix construction, the marker matrix markers is first initialized, and all contours are traversed. If the number of valid pixels in the contour is zero and the area is greater than the average value of the contour area, it is marked as background (Scalar (255), the gray value of the background is 255), and the contours with or less than 5 valid pixels are filtered. Only the contours without child nodes or root nodes are assigned unique index values, and the rest are marked as intermediate areas. In addition, the marker matrix can be optimized, that is, background anchor points are added at the four corners of the marker matrix to enhance the stability of edge segmentation. Among them, the labeling matrix is ​​a two-dimensional matrix used to initialize region labels in image segmentation algorithms (such as the watershed algorithm). Its essence is an integer matrix with the same size as the original image. Each pixel value represents the initial region category to which the position belongs. It can predefine the labels of key areas and guide the segmentation process to more accurately distinguish between the target and the background.

[0121] Step S15: performing two-dimensional building instantiation segmentation on the building distance map based on the label matrix to obtain a two-dimensional building instantiation object; wherein the two-dimensional pixels of the two-dimensional building instantiation object carry a building instantiation label.

[0122] In this embodiment, a labeling matrix corresponding to the multi-level contours in the distance map is constructed. Based on the labeling matrix, the building distance map can be segmented into two-dimensional building instances. Specifically, a watershed algorithm can be used to implement this segmentation based on the labeling matrix. For example, the labeling matrix is ​​input into the watershed algorithm and combined with image gradient information to perform regional submergence segmentation to obtain two-dimensional building instances. Building instance labels are then assigned to the two-dimensional pixels, meaning that the two-dimensional pixels of the segmented two-dimensional building instances carry the building instance labels. Image gradient information reflects local variations in pixel values ​​and is often used for edge detection. Regions with strong gradient strength typically correspond to object boundaries.

[0123] For example, see Figure 4 As shown in the figure, based on the morphological features of building anchor points in the building distance map, 2D building instance segmentation is performed. The specific process is as follows: morphological erosion is performed on the instanced building anchor points in the building distance map to extract the multi-level contours in the building distance map. The tree-like hierarchical relationship corresponding to the multi-level contours is constructed using the RETR_TREE mode. Then, a corresponding labeling matrix is ​​constructed based on the tree-like hierarchical relationship. Finally, the labeling matrix is ​​input into the watershed algorithm and regional submerged segmentation is performed in combination with image gradient information to obtain 2D building instance objects. Building instance labels are assigned to 2D pixels. In particular, the hierarchical organization strategy of the contour tree structure can improve the processing efficiency of scenes with large scale differences.

[0124] In this embodiment, after performing two-dimensional building instantiation segmentation on the building distance map based on the labeling matrix to obtain a two-dimensional building instantiation object, the method may further specifically include: performing a morphological dilation operation on the two-dimensional building instantiation object and extracting the maximum contour of the expanded two-dimensional building instantiation object; simplifying the maximum contour to obtain a simplified maximum contour, and determining the two-dimensional pixels within the simplified maximum contour as the target building instantiation pixels; and performing three-dimensional space mapping processing on the target building instantiation pixels based on a pre-constructed mapping relationship between pixel coordinates and three-dimensional space coordinates to obtain a three-dimensional building instantiation object.

[0125] For example, a morphological dilation operation is performed on each 2D instantiation object of a building to expand its boundary range, and the maximum contour after dilation is extracted. The maximum contour is simplified using the Douglas-Peucker algorithm, and the pixels within the simplified maximum contour are used as the target building instantiation pixels. Based on the mapping relationship between pixel coordinates and 3D space coordinates, a 3D space mapping process is performed on the target building instantiation pixels to obtain the 3D instantiation object of the building, namely:

[0126] ;

[0127] in, It is a height acquisition function that records the elevation value of the two-dimensional point corresponding to the three-dimensional point. represents the two-dimensional space coordinates of a two-dimensional point, Represents the three-dimensional space coordinates of a three-dimensional point, represents the minimum x value in three-dimensional space, Indicates the minimum y-value in three-dimensional space.

[0128] It can be seen that in the embodiment of the present invention, accurate recognition of vertical structures is achieved through a preset dual-threshold judgment mechanism, the misjudgment rate of vertical structures is reduced, and accurate division of complex building instances is achieved by mapping the three-dimensional point cloud to the two-dimensional feature space, thereby improving segmentation accuracy. An unsupervised hierarchical segmentation framework is also implemented through a three-level optimization architecture of supervoxels, planar point clusters, and instance anchors, as well as a contour tree structure. This eliminates the dependence on labeled data in the segmentation process of individual building instances, enables rapid processing of city-level point clouds, significantly improves the processing capabilities of complex buildings such as special-shaped buildings and adhesion structures, and improves segmentation accuracy.

[0129] That is, the above-mentioned technical solution of this application establishes an unsupervised segmentation mechanism that does not require labeled data, breaks through data dependence, realizes the coordinated optimization of three-dimensional geometric constraints and two-dimensional morphological features, thereby enhancing feature expression, and through adaptive spatial anchor point extraction, improves the single scene processing time, thereby improving computing efficiency, and also through vertical structural feature enhancement, improves the complex scene segmentation IoU, improves segmentation accuracy, and reduces hardware resource consumption. It can be widely used in smart city modeling, real estate three-dimensional registration and other fields.

[0130] In one embodiment, if Figure 5 As shown, based on the above-mentioned building instance segmentation method, the present invention also provides a building instance segmentation device, including:

[0131] A building point cloud construction module 11 is used to construct a building point cloud of a target building;

[0132] A plane point cluster extraction module 12 is used to perform plane segmentation on the building point cloud and extract potential vertical plane point clusters from the segmented building plane point clusters;

[0133] A building vertical structure extraction module 13 is configured to map all three-dimensional point clouds in the point cloud set corresponding to the potential vertical plane point cluster to a two-dimensional feature space to obtain a two-dimensional point set, and to extract a two-dimensional point cloud of the building vertical structure based on the two-dimensional point set using a preset dual-threshold judgment mechanism to obtain a building vertical structure point cluster;

[0134] An instance anchor point generation module 14 is configured to generate a building distance map based on the building vertical structure point clusters, and generate building instance anchor points representing positions of potential building instances on the building distance map;

[0135] A tree-like hierarchical relationship construction module 15 is configured to extract multi-level contours in the building distance map based on the building instance anchor points, and construct a tree-like hierarchical relationship corresponding to the multi-level contours;

[0136] A marker matrix construction module 16 is configured to construct a corresponding marker matrix based on the tree-like hierarchical relationship;

[0137] The instantiation segmentation module 17 is configured to perform two-dimensional building instantiation segmentation on the building distance map based on the label matrix to obtain a two-dimensional building instantiation object; wherein the two-dimensional pixels of the two-dimensional building instantiation object carry a building instantiation label.

[0138] Figure 6 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present application. The terminal may include:

[0139] Memory 501 , processor 502 , and computer programs stored in the memory 501 and executable on the processor 502 .

[0140] When the processor 502 executes the program, the building instantiation segmentation method provided in the above embodiment is implemented.

[0141] Furthermore, the terminal further includes:

[0142] The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0143] The memory 501 is used to store computer programs that can be run on the processor 502 .

[0144] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0145] If memory 501, processor 502, and communication interface 503 are implemented independently, communication interface 503, memory 501, and processor 502 can be interconnected via a bus to facilitate communication. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be categorized as address buses, data buses, and control buses. For ease of illustration, the figure uses only one line, but this does not imply that there is only one bus or only one type of bus.

[0146] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0147] The processor 502 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0148] This embodiment further provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the above-mentioned building instantiation segmentation method is implemented.

[0149] Other embodiments of the present invention will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the invention being indicated by the claims.

[0150] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0151] The logic and / or steps represented in the flowchart or otherwise described herein may be considered, for example, as a sequenced list of executable instructions for implementing the logical functions, and may be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can read and execute instructions from an instruction execution system, apparatus, or device).

[0152] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiment, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented using hardware, as in another embodiment, any of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having logic gate circuits for implementing logical functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0153] It should be understood that the application of the present invention is not limited to the above examples. For those skilled in the art, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A building instance segmentation method, characterized in that: The method comprises: Constructing a building point cloud of a target building, performing plane segmentation on the building point cloud, and extracting potential vertical plane point clusters from the segmented building plane point clusters; Mapping all three-dimensional point clouds in the point cloud set corresponding to the potential vertical plane point cluster to a two-dimensional feature space to obtain a two-dimensional point set, using a preset double-threshold judgment mechanism and extracting a two-dimensional point cloud of the building's vertical structure based on the two-dimensional point set to obtain a building vertical structure point cluster; generating a building distance map based on the building vertical structure point clusters, and generating building instance anchor points representing positions of potential building instances on the building distance map; Extracting multi-level contours in the building distance map based on the building instance anchor points, constructing a tree-like hierarchical relationship corresponding to the multi-level contours, and constructing a corresponding labeling matrix based on the tree-like hierarchical relationship; Performing a two-dimensional building instantiation segmentation on the building distance map based on the label matrix to obtain a two-dimensional building instantiation object; wherein the two-dimensional pixels of the two-dimensional building instantiation object carry a building instantiation label; Mapping all three-dimensional point clouds in the point cloud set corresponding to the potential vertical plane point cluster to a two-dimensional feature space to obtain a two-dimensional point set includes: Constructing a ground point cloud, and generating a ground horizontal reference plane based on the ground point cloud fitting; On the ground horizontal reference plane, establishing a local coordinate system based on the centroid point of the ground point cloud; Projecting all three-dimensional point clouds in the point cloud set corresponding to the potential vertical plane point cluster to the local coordinate system along the normal vector direction of the ground point cloud to obtain a two-dimensional point set; The method of using a preset dual-threshold determination mechanism and extracting a two-dimensional point cloud of a building's vertical structure based on the two-dimensional point set to obtain a building's vertical structure point cluster includes: Establishing a two-dimensional grid of the two-dimensional point set on the ground horizontal reference plane; Counting the number of two-dimensional point clouds in each grid cell of the two-dimensional grid to obtain a vertical projection density feature corresponding to each grid cell; Determining the highest point height and the lowest point height of each grid unit in the two-dimensional grid, and calculating the difference between the highest point height and the lowest point height to obtain a vertical height difference feature corresponding to each grid unit; Determining a density threshold based on the vertical projection density feature, and determining a depth threshold based on the vertical height difference feature; Using a preset dual-threshold determination mechanism, the vertical projection density feature corresponding to each grid unit is compared with the density threshold, and the vertical height difference feature is compared with the depth threshold; When the vertical projection density feature is greater than the density threshold and the vertical height difference feature is greater than the depth threshold, the two-dimensional point cloud in the grid unit is extracted as a building vertical structure two-dimensional point cloud to obtain a building vertical structure point cluster.

2. The building instance segmentation method according to claim 1, characterized in that: The performing plane segmentation on the building point cloud and extracting potential vertical plane point clusters from the segmented building plane point clusters comprises: Performing plane segmentation on the building point cloud using a plane region growing algorithm to obtain building plane point clusters; Performing plane fitting on the building plane point cluster using the least square method to obtain an optimal plane equation; Potential vertical plane point clusters are extracted based on the optimal plane equation and the size of the building plane point clusters.

3. The building instance segmentation method according to claim 1, characterized in that: The determining of a density threshold based on the vertical projection density feature and determining a depth threshold based on the vertical height difference feature includes: Determining the mean and variation of the two-dimensional point cloud density within the grid unit based on the vertical projection density feature, and processing the mean and the variation using a preset density threshold calculation formula to obtain a density threshold; The vertical height difference feature is processed using the preset depth threshold calculation formula to obtain the depth threshold; The calculation formula for the preset density threshold is: ； The preset depth threshold calculation formula is: ； ； and, represents the density threshold, represents the mean value of the two-dimensional point cloud density within the grid cell, represents the variability of the 2D point cloud density within the grid cell, represents the depth threshold, Represents the vertical height difference feature, Indicates the height of the highest point in the grid cell, Indicates the lowest point height in the grid cell, Represents the threshold control coefficient.

4. The building instance segmentation method according to claim 3, characterized in that: The generating of a building distance map based on the building vertical structure point clusters and generating building instance anchor points representing positions of potential building instances on the building distance map includes: Projecting all the two-dimensional point clouds of the building vertical structure in the building vertical structure point cluster onto the ground horizontal reference plane to obtain a building projection point set; Determining the maximum longitudinal coordinate value and the minimum longitudinal coordinate value, as well as the maximum transverse coordinate value and the minimum transverse coordinate value of the building projection points in the projection point set; Calculate the image width and image height based on a preset image resolution, the maximum value and the minimum value of the longitudinal coordinate, and the maximum value and the minimum value of the transverse coordinate; Creating an initial binary image according to the image width and the image height; Translating all the building projection points in the building projection point set to an image coordinate system constructed based on the initial binary image to obtain a target binary image; Calculating the Euclidean distance between each pixel in the target binary image and the background pixel to obtain the distance value corresponding to each pixel; Linearly mapping the distance value corresponding to each pixel to a standard range of grayscale values ​​to obtain a building distance map; Traversing the building distance map in a sliding window manner, and sequentially calculating the average of the distance values ​​corresponding to all pixels in the window area to obtain an adaptive threshold corresponding to the window area; The distance value corresponding to each pixel in the window area is compared with the adaptive threshold, and the pixel area with the distance value higher than the adaptive threshold is determined as the foreground, so as to obtain a building instance anchor point representing the position of the potential building instance on the building distance map.

5. The building instance segmentation method according to claim 1, characterized in that: Before extracting the multi-level contours in the building distance map based on the building instance anchor points, the method further includes: A morphological erosion operation is performed on the building instance anchor points on the building distance map.

6. The building instance segmentation method according to any one of claims 1 to 5, characterized in that: After performing two-dimensional building instantiation segmentation on the building distance map based on the label matrix to obtain two-dimensional building instantiation objects, the method further includes: Performing a morphological dilation operation on the two-dimensional building instantiation object, and extracting the maximum contour of the dilated two-dimensional building instantiation object; Simplifying the maximum contour to obtain a simplified maximum contour, and determining the two-dimensional pixels within the simplified maximum contour as target building instantiation pixels; Based on the pre-constructed mapping relationship between pixel coordinates and three-dimensional space coordinates, three-dimensional space mapping processing is performed on the instantiated pixels of the target building to obtain a three-dimensional instantiated object of the building.

7. A building instance segmentation device, characterized in that: The device comprises: Building point cloud construction module, used to construct the building point cloud of the target building; A plane point cluster extraction module is used to perform plane segmentation on the building point cloud and extract potential vertical plane point clusters from the segmented building plane point clusters; A building vertical structure extraction module is used to map all three-dimensional point clouds in the point cloud set corresponding to the potential vertical plane point cluster to a two-dimensional feature space to obtain a two-dimensional point set, and to extract the two-dimensional point cloud of the building vertical structure based on the two-dimensional point set using a preset dual-threshold judgment mechanism to obtain a building vertical structure point cluster; An instance anchor point generation module is used to generate a building distance map based on the building vertical structure point clusters, and generate building instance anchor points representing the positions of potential building instances on the building distance map; A tree-like hierarchical relationship construction module is used to extract multi-level contours in the building distance map based on the building instance anchor points, and construct a tree-like hierarchical relationship corresponding to the multi-level contours; A marker matrix construction module, configured to construct a corresponding marker matrix based on the tree-like hierarchical relationship; an instantiation segmentation module, configured to perform two-dimensional instantiation segmentation of the building distance map based on the label matrix to obtain a two-dimensional instantiation object of the building; wherein the two-dimensional pixels of the two-dimensional instantiation object of the building carry a building instantiation label; The building vertical structure extraction module is further specifically used for: Constructing a ground point cloud, and generating a ground horizontal reference plane based on the ground point cloud fitting; On the ground horizontal reference plane, establishing a local coordinate system based on the centroid point of the ground point cloud; Projecting all three-dimensional point clouds in the point cloud set corresponding to the potential vertical plane point cluster to the local coordinate system along the normal vector direction of the ground point cloud to obtain a two-dimensional point set; Establishing a two-dimensional grid of the two-dimensional point set on the ground horizontal reference plane; Counting the number of two-dimensional point clouds in each grid cell of the two-dimensional grid to obtain a vertical projection density feature corresponding to each grid cell; Determining the highest point height and the lowest point height of each grid unit in the two-dimensional grid, and calculating the difference between the highest point height and the lowest point height to obtain a vertical height difference feature corresponding to each grid unit; Determining a density threshold based on the vertical projection density feature, and determining a depth threshold based on the vertical height difference feature; Using a preset dual-threshold determination mechanism, the vertical projection density feature corresponding to each grid unit is compared with the density threshold, and the vertical height difference feature is compared with the depth threshold; When the vertical projection density feature is greater than the density threshold and the vertical height difference feature is greater than the depth threshold, the two-dimensional point cloud in the grid unit is extracted as a building vertical structure two-dimensional point cloud to obtain a building vertical structure point cluster.

8. A terminal, characterized in that: include: A memory, a processor, and a building instantiation segmentation program stored in the memory and executable on the processor, wherein the building instantiation segmentation program, when executed by the processor, implements the steps of the building instantiation segmentation method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which can be executed to implement the steps of the building instance segmentation method according to any one of claims 1 to 6.

Images