A precise indoor point cloud extraction method based on orthogonal direction and graph cut optimization

Abstract

This invention discloses a method for accurate extraction of indoor point clouds based on orthogonal direction and graph cut optimization. This method offers significant advantages in point cloud processing and building information extraction. First, through orthogonal direction estimation and nonlinear optimization, the point cloud data can be precisely aligned with the main axes of the building, ensuring consistency between the model and the actual structure. Second, the method effectively eliminates external point cloud data through a graph cut model, reducing noise interference and thus preserving valid point clouds inside the building. Furthermore, plane detection and orthogonal constraints improve the accuracy and completeness of indoor space extraction. Overall, this method is not only highly automated, significantly reducing the need for manual adjustments, but also possesses strong robustness, enabling robust extraction of internal building structural information even in the presence of missing data or noise.

Description

Technical Field

[0001] This invention belongs to the fields of point cloud processing, Building Information Modeling (BIM), and 3D spatial data analysis. It relates to the precise processing and analysis of point cloud data in complex building environments, specifically a method for precise extraction of indoor point clouds based on orthogonal direction and graph cut optimization. This includes automatic segmentation of the building's interior and exterior spaces, planar detection and estimation of orthogonal directions, and cleaning and filtering of point cloud data through graph cut and height range clipping methods. Background Technology

[0002] In the fields of 3D Building Information Modeling (BIM) and point cloud processing, the acquisition of high-precision 3D point cloud data has become increasingly common with advancements in laser scanning and photogrammetry technologies. This point cloud data is frequently used for precise building modeling, interior space analysis, and virtual reality applications. However, because point cloud data is often unstructured and contains significant noise and external interference (such as data about the building's external environment), effectively extracting the building's internal geometric information and aligning it with the building's main axis remains a challenging task.

[0003] Traditional methods often rely on manual labeling or simple plane fitting algorithms, which struggle to handle complex building structures and large amounts of noisy data. In recent years, many automated point cloud processing techniques, such as plane detection, orthogonal direction estimation, and graph cut optimization, have been proposed to improve the accuracy of extracting building interior spaces. These techniques can automatically identify and eliminate point cloud data outside buildings and accurately align the internal geometry of buildings through orthogonal direction constraints, providing a high-quality data foundation for building information modeling and analysis. However, existing technologies still face some shortcomings, such as insufficient accuracy in handling complex building structures, challenges in handling extreme noisy data, limitations of orthogonal direction constraints, and high computational complexity of global optimization. This invention aims to improve the accuracy and efficiency of point cloud processing and overcome the aforementioned shortcomings of existing technologies through an improved plane detection method, enhanced orthogonal direction estimation, and an optimized global graph cut algorithm. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method for accurate indoor point cloud extraction based on orthogonal direction and graph cut optimization. This invention employs a novel framework for point cloud preprocessing in complex indoor scenes. A brief description of this framework is as follows: Figure 1As shown, the method of this invention processes scatter points P generated by various scanning devices. These scatter points are typically affected by tilt and outliers. The algorithm does not require additional information such as intensity or scan location, but instead leverages the geometric features of the point cloud to broaden its application. Furthermore, this invention assumes a weak Manhattan world structure prior with arbitrary wall orientations to guide the processing of noisy point clouds. The preprocessing outputs are a reoriented point cloud RP and a cleaned point cloud CP, the purpose of which is to align the main orientations of buildings with the coordinate axes and remove background areas outside the building layout, thereby making the point cloud data more regular and accurate. Additionally, an interior planar primitive IP is one of the outputs, serving as a lightweight representation of the scene and providing high-level information for further reconstruction.

[0005] An accurate indoor point cloud extraction method based on orthogonal direction and graph cut optimization includes the following steps:

[0006] Step 1: Extract the initial plane from the scattered points P using a hybrid segmentation method. These scattered points P were generated by scanning from different devices and may contain outliers or error points.

[0007] The hybrid segmentation method combines the ideas of region growing, model fitting, and clustering. The specific operations include: First, based on the normal information of the point cloud, region growing is used to continuously increase the region on the smooth surface. Then, a plane S = {Si} is extracted from the grown region using the Random Sample Consensus Algorithm (RANSAC) with normal direction constraints. RANSAC randomly selects a set of points for plane model fitting, selecting points consistent with the fitted model to form a planar structure. The formed planar structure will be one of the output results, namely the indoor planar primitive IP. Next, Fast Euclidean Clustering (FEC) is introduced to separate the points in the planar structure Si into patches L = {li}. The FEC algorithm classifies points using Euclidean distance, dividing the point set of the same plane into different patches. These steps help extract accurate planes from the scattered points P and further clean the point cloud data.

[0008] Step 2: Assemble the extracted planar structures into planar clusters with similar orientations.

[0009] Specifically, the extracted planar structures are classified according to the similarity of their normal directions, forming several planar clusters. These clusters typically correspond to the main planar orientations of a building. After obtaining the approximately orthogonal planar clusters and their corresponding interior points, a nonlinear optimization algorithm with orthogonality constraints is used to determine the dominant orientation of the building. The nonlinear optimization algorithm optimizes the normal directions of the planar clusters to make them as consistent as possible with the preset orthogonality constraints, thereby identifying the dominant orientation of the building. This orientation is then used as a reference for coordinate system rotation, and the point cloud is rotated and adjusted so that the main orientation of the building in the point cloud is aligned with the coordinate axes, resulting in a reoriented point cloud. In this way, the orientation of the building is consistent with the orientation of the coordinate system, providing a unified reference framework for subsequent point cloud processing and reconstruction.

[0010] Step 3: Project the near-horizontal planar patch onto the grayscale image to form connected regions, and assign a label to each region using a graph cut method. One label is used for the building interior (i.e., labeled "Building Interior"), and the other label is used for the exterior space (i.e., labeled "Building Exterior"). Removal of the building exterior is accomplished by overlaying the mask converted from the interior region (i.e., the building interior) with the reoriented point cloud (i.e., the point cloud data after rotation adjustment in Step 2).

[0011] The beneficial effects of this invention are as follows:

[0012] This invention proposes a precise indoor point cloud extraction method based on orthogonal direction and graph cut optimization, which has significant benefits in point cloud processing and building information extraction. First, through orthogonal direction estimation and nonlinear optimization, the point cloud data can be precisely aligned with the main axes of the building, ensuring consistency between the model and the actual structure. Second, the method effectively eliminates external point cloud data through a graph cut model, reducing noise interference and thus preserving valid point cloud data from inside the building. Furthermore, plane detection and orthogonal constraints improve the accuracy and completeness of indoor space extraction. Overall, this method is not only highly automated, significantly reducing the need for manual adjustments, but also possesses strong robustness, enabling robust extraction of internal building structural information even in the presence of missing data or noise. Attached Figure Description

[0013] Figure 1 This is a flowchart of a method according to an embodiment of the present invention. Detailed Implementation

[0014] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0015] like Figure 1 As shown, the present invention proposes an accurate indoor point cloud extraction method based on orthogonal direction and graph cut optimization. The specific implementation steps are as follows:

[0016] Step 1: Extracting planar primitives;

[0017] Step (1.1) Point cloud integration;

[0018] To mitigate the negative impact of uneven point density and reduce the computational cost of redundant point clouds, a crucial first step is to perform voxel downsampling (voxel size 0.05m) on the original data. Specifically, voxel meshing merges points within the same voxel into a single point, reducing the density of the point cloud data and resulting in a downsampled point cloud. To calculate the representative coordinates of points within each voxel, the centroid method is employed, where the geometric center of each point within a voxel is calculated as the new representative point.

[0019] Subsequently, local surface features are estimated. These features include normals and curvature, which are used for the detection of planar structures (i.e., the process of identifying and extracting planar regions from point cloud data). Specifically, Principal Component Analysis (PCA) is used to compute the local surface features of the downsampled point cloud. PCA is used to estimate the normal direction and curvature of each point by fitting a local tangent plane, thereby obtaining the salient features of the points.

[0020] The parameters of the local surface (such as the coefficients of the plane equation) are solved using the total least squares algorithm. The total least squares algorithm calculates the coefficients of the plane equation by minimizing the distance from the point to the plane, thus obtaining the accurate parameters of the local surface.

[0021] Step (1.2) Hybrid plane detection;

[0022] Given the geometric characteristics of point clouds, a hybrid segmentation method combining region growing, model fitting, and clustering is designed to extract indoor structural planes. First, the downsampled point cloud with directional normals is projected onto a unit sphere using Gaussian mapping. Next, the point with the smallest curvature is selected as the seed region, and then expanded on the sphere based on the angular differences between the normal vectors (the expansion process involves gradually adding adjacent points with angular differences within a certain range to the seed region, starting from the current seed point, until no more points meet the criteria). The expansion of the seed region ends when no new points satisfy the expansion criteria or when a preset region size threshold is reached.

[0023] Through an expansion process, point clouds with similar normal vectors are clustered together, eventually forming clusters. The points in these clusters are typically distributed across a set of parallel planes. By borrowing the core idea of ​​the Random Sample Consensus (RANSAC) algorithm, the orientation of the plane is robustly captured from the normal vectors of points belonging to the same cluster, and this orientation is used as a constraint for subsequent planar structure extraction. Specifically, a unit normal vector of a point is randomly selected from the cluster as a reference, and the number of remaining normal vectors whose angle difference with it is less than a given threshold is calculated. This process aims to determine the optimal orientation, i.e., the orientation of the normal vector with the most interior points. This process is repeated to obtain the optimal orientation of the cluster. To ensure accuracy, it is typically necessary to traverse all possible reference normal vectors to obtain the optimal planar orientation.

[0024] Subsequently, the planar structure of each cluster is extracted using a RANSAC-based segmentation method with orientation constraints. The specific process includes: firstly, estimating the parameters of the planes using the RANSAC algorithm, and simultaneously applying orientation constraints to ensure that the extracted planes are consistent with the optimal orientation of the cluster. To ensure high accuracy and reliability of the planar structures extracted from the point cloud data, a hypothesis step and a verification step are proposed. In the hypothesis step, only candidate planes with an angle greater than a set threshold (preferably 85°) with the cluster orientation are considered. This threshold ensures that the candidate planes maintain a sufficient perpendicular relationship with the optimal orientation of the cluster, thereby improving the accuracy of the extracted planes. In the verification step, the optimal parameters of the planar model are determined based on the distance from the point to the candidate plane. The plane with an angle greater than the set threshold (preferably 85°) with the cluster orientation and the smallest distance from the point to the plane is selected as the final planar structure of that cluster. The planar structures of all extracted clusters will be one of the output results, namely the indoor planar primitive IP.

[0025] Furthermore, the Fast Euclidean Clustering (FEC) method is introduced to extract meaningful planar patches from the planar structure. FEC is an efficient clustering technique that, compared to traditional Euclidean clustering, reduces the number of traversals of neighboring points through a point-to-point approach, thus accelerating processing. Specifically, FEC first constructs a KD-tree on coplanar points to accelerate the search for neighboring points within a fixed range (i.e., a specified radius, 0.05m in this experiment). Then, by clustering points that are close to each other in space into subclusters and merging adjacent subclusters based on connectivity information, planar patches are formed. These planar patches represent actual planar structures in the point cloud, such as walls or floors, providing crucial information for subsequent building reconstruction and interior layout analysis.

[0026] Step 2: Coordinate system transformation;

[0027] Step (2.1) Virtual plane proxy creation;

[0028] For all the planar structures and their corresponding point sets extracted in Step 1, they are classified into different groups based on their positional relationships, with each group represented by a virtual planar proxy. The specific operation is as follows: Consider two relationships between shapes, namely parallelism and orthogonality, which are widespread in artificial architecture. First, use the parameter Ang to represent the maximum threshold of angular deviation to determine whether two planar structures are parallel. If the angle between the normal directions of two planar structures is within this threshold, they are considered parallel. Then, select the planar structure with the most coplanar points as the seed planar structure, and expand these planar structures outwards according to their parallelism with other planar structures, forming groups, until no more planar structures parallel to it are available for expansion. Then, reselect the seed planar structure from the remaining planar structures for expansion, repeating the above operation until no mutually parallel planar structures exist. A virtual planar proxy is generated in each resulting group, representing the nearly parallel planar structures in that group. The parameters of the virtual plane are determined by the average normal direction and anchor points of the planar structures within the group. This process ensures that the planar structures in each group are geometrically similar, thus simplifying subsequent coordinate system transformations.

[0029] The parameters of the virtual plane are determined through the following steps:

[0030] Calculation of the average normal direction: For planar structures within a group, first calculate the normal vector for each planar structure. Then, take a weighted average of these normal vectors to obtain the average normal direction of the planar structures within the group.

[0031] Determine the anchor point of the virtual plane: Within each group, select the plane with the largest area as the representative plane, and select the centroid point on the plane as the anchor point of the virtual plane (the anchor point is a specific point used to determine the position of the virtual plane, which, together with the average normal direction, defines the spatial position and orientation of the virtual plane).

[0032] The equation of the virtual plane is determined based on the mean normal direction and the anchor point. The plane equation is expressed as: n·(r-r0)=0. Here, n is the mean normal direction, r is the vector of any point on the plane, and r0 is the vector of the anchor point.

[0033] Step (2.2) Estimation of the dominant direction;

[0034] By comparing the normal directions of virtual plane proxies pairwise, the orthogonality between different virtual plane proxies is determined, and these relationships are used to obtain candidate dominant directions. Initially, virtual plane proxies are sorted in descending order of the number of interior points, and the normal direction of the virtual plane with the most interior points is marked as the first candidate direction. Then, the normal directions of other unmarked virtual planes are compared one by one. When the directional deviation between two virtual planes is less than a predefined threshold (π / 2 - Ang), the two virtual planes are determined to be perpendicular and marked as a candidate direction orthogonal to the first candidate direction. Next, a third plane direction orthogonal to both of the marked candidate directions is selected, resulting in three mutually orthogonal candidate directions. The virtual plane proxies corresponding to these three candidate directions are used as candidate virtual plane proxies. It is worth noting that these three candidate directions are closely related to the structural elements of the building. The first direction is roughly consistent with the direction of the ceiling or floor, while the other directions usually correspond to the direction of the walls. However, due to noise in the unorganized point cloud, the almost orthogonal candidate directions may have some deviation and cannot accurately reveal the interior layout.

[0035] Therefore, based on the assumption that the floor (ceiling) and walls intersect at right angles in a typical interior environment, global regularization is applied to the candidate virtual plane proxies to accurately extract the dominant directions. Specifically, a nonlinear optimization method with orthogonal constraints is employed, taking three candidate virtual plane proxies and their associated points as input, and outputting three completely perpendicular dominant directions that match the main axis of the building. The objective is to minimize the data fitting error from the interior point to the associated plane, while considering the orthogonality between the planes. The fitting error is accumulated by the square of the Euclidean distance from the point to the virtual plane. Furthermore, the influence of the point is weighted according to its original position and normal direction information.

[0036] The nonlinear optimization method with orthogonal constraints aims to minimize the data fitting error from the interior points to the relevant planes, while ensuring the orthogonality between the planar surrogates. The fitting error can be quantified by the following formula:

[0037]

[0038] Where, p i Represents the coordinates of the i-th point; n j d represents the unit normal vector corresponding to the candidate virtual plane proxy j; j ω represents the offset in the equation of plane j. i It is based on point p i The weights are determined by the original position and normal direction information.

[0039] Furthermore, the orthogonal constraint is expressed through this condition: n j ·n k= 0 (for all j ≠ k). Where n j and n k This represents the normal vectors of two distinct candidate virtual plane proxies.

[0040] The optimization algorithm extracts three orthogonal dominant directions that match the main axis of the building by minimizing the above error function and simultaneously satisfying the orthogonal constraints between planar proxies.

[0041] Step 3: Remove external building points;

[0042] After obtaining the dominant direction of the point cloud in step 2, a rotation matrix is ​​used to align the dominant direction of the downsampled point cloud with the principal axis of the global coordinate system to obtain a reoriented point cloud. This step achieves the reorientation of the point cloud, aligning the main planes (such as the ground, walls, etc.) in the point cloud with the global coordinate system and eliminating data skew caused by the complexity of the scanning equipment or scene. After solving the data skew problem, the next step is to remove outdoor points from the reoriented point cloud. First, the planar patch and its interior points are transformed into the global coordinate system, and horizontal planar patches are selected as candidate patches based on the angle between their normal vector and the z-axis. Specifically, planar patches with a normal angle to the z-axis that is less than a predetermined angle (5°) difference threshold are selected as nearly horizontal planar patches. Then, the candidate patches related to the ceiling, floor, and other horizontal components are projected onto the xy plane and rasterized into grayscale images with a fixed step size (0.1m). Each pixel value in the grayscale image is determined by the planar patch of the highest point (i.e., the point with the largest z-coordinate value) in each grid. Subsequently, grayscale thresholding is introduced to compare pixel values ​​in the image with a preset grayscale threshold, marking foreground and background regions. Then, connectivity analysis is applied to connect adjacent foreground pixels into connected regions, assigning each region a unique number to identify and distinguish different connected regions. These steps help refine the image and identify areas relevant to the building's exterior, thus achieving effective point cloud removal.

[0043] Next, the connected graph is partitioned into two disjoint and optimal subsets of regions using a graph cut algorithm. The graph cut algorithm involves constructing the graph, defining the cut cost function, and calculating the optimal cut boundary. Specifically, when constructing the graph, regions are used as vertices, and edges are established to represent the relationships between adjacent regions, thus generating an undirected graph G = <V,E>, where the vertex set V represents the obtained regions, and the edge set E represents pairs of adjacent regions. The cut cost function is defined based on the similarity between regions and known labels (referring to regions whose categories have been pre-determined), and the optimal cut boundary is solved using the maximum flow-minimum cut theorem. Finally, an initial label is added to each vertex, marking it as 1 (representing indoor space) or 0 (representing outdoor space) to achieve region classification.

[0044] For each vertex in the graph, the initial label is inferred from three geometric properties: area overlap, point cover, and wall support.

[0045] The area overlap metric assesses the degree of overlap between a given area and the ceiling and floor. Since ceilings and floors often appear in pairs, the degree of overlap between them is generally higher in indoor areas than in outdoor areas. Specifically, to calculate the overlap area between area A and the ceiling C and floor F: Area(A∩C) and Area(A∩F) represent the overlapping areas of area A with the ceiling and floor, respectively, while Area(A) represents the total area of ​​area A.

[0046] Point coverage assessment evaluates the distribution of points within an area, specifically the density and distribution characteristics of points in the area, and how well they match the point cloud data. This helps determine whether an area belongs to an indoor space. Indoor areas typically have higher point coverage density because point cloud data is denser in indoor spaces. Specifically, suppose the set of points within area A is P. A The total number of points is |P A If |, then the point cover can be evaluated by calculating the point density: P A | represents the total number of points in region A, while Area(A) represents the total area of ​​region A. High-density point coverage means that the region is more likely to be an indoor space.

[0047] The assessment considers whether an area is near or supported by a wall. Since walls are frequently present in interior spaces, areas near walls are more likely to be considered interior spaces. Specifically, assume there is a set of points P within area A. A And the set of points closest to the wall is P. W The wall support can then be evaluated by calculating the average distance from points within the area to the wall: Dist(p,w) represents the distance from point p to point w on the wall. A | represents the total number of points within region A. The lower the value for wall support, the closer the region is to a wall, and the more likely it is to be an interior space.

[0048] Based on these three attributes, an initial label is generated for each vertex, and a decision is made by setting a threshold: Threshold1, Threshold2, and Threshold3 are thresholds derived from experience or data training (the thresholds may vary in different scenarios) and are used to distinguish between indoor and outdoor areas.

[0049] The widely used graph cut method can efficiently optimize the energy function, thereby achieving optimal region segmentation. In the building exterior point removal step, firstly, the two region subsets divided based on the initial labels are mapped onto a two-dimensional grid to form a binary mask image, where 1 represents the indoor region and 0 represents the building exterior region.

[0050] The reoriented point cloud is then refined: To supplement missing wall information, firstly, vertical patches are rasterized with the same grid size (0.05m in this example) and attached to the boundary pixels of the region. Then, a flood fill algorithm is used to process internal holes in the image to completely cover the interior space. Furthermore, to remove outliers above the ceiling and below the floor, the height range of each grid needs to be calculated. First, histogram analysis is used to extract the heights of the top ceiling and bottom floor from near-horizontal planar structures (referring to planar structures nearly parallel to the ground, such as ceilings and floors) to determine an initial height threshold. Subsequently, local adjustments are made using the highest and lowest planar patches within the grid; specifically, the height range of each grid is dynamically adjusted based on the initial height threshold to remove outliers exceeding the set range.

[0051] Finally, by overlaying the refined, reoriented point cloud with a binary mask image and cropping it with an adjusted height range, the desired indoor points are retained while points outside the building are removed. This process ensures that only the point cloud data inside the building is preserved, and effectively removes external noise and irrelevant points.

[0052] The above description, in conjunction with specific / preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. Those skilled in the art can make various substitutions or modifications to these described embodiments without departing from the inventive concept, and all such substitutions or modifications should be considered within the scope of protection of the present invention.

[0053] The parts of this invention not described in detail are well-known to those skilled in the art.

Claims

1. A method for accurate extraction of indoor point clouds based on orthogonal direction and graph cut optimization, characterized in that, The steps include the following: Step 1: Extract the initial plane from the scattered points P using a hybrid segmentation method; The hybrid segmentation method combines the ideas of region growing, model fitting, and clustering. The specific operations include: First, based on the normal information of the point cloud, region growing is used to continuously increase the region on the smooth surface. Then, a plane S = {Si} is extracted from the grown region using the Random Sample Consensus Algorithm (RANSAC) with normal direction constraints. RANSAC randomly selects a set of points for plane model fitting, selecting points consistent with the fitted model to form a planar structure. The formed planar structure will be one of the output results, namely the indoor planar primitive IP. Next, Fast Euclidean Clustering (FEC) is introduced to separate the points in the planar structure Si into patches L = {li}. The FEC algorithm classifies points using Euclidean distance, dividing the point set of the same plane into different patches. Step 2: Assemble the extracted planar structures into planar clusters with similar orientations; Specifically, the extracted planar structures are classified according to the similarity of their normal directions, forming several planar clusters. These planar clusters correspond to the main planar directions in the building. After obtaining the approximately orthogonal planar clusters and their corresponding interior points, the dominant direction of the building is determined using nonlinear optimization with orthogonality constraints. The nonlinear optimization algorithm optimizes the normal directions of the planar clusters to make them as consistent as possible with the preset orthogonality constraints, thereby finding the dominant direction of the building. This direction is used as a reference for coordinate system rotation, and the point cloud is rotated and adjusted so that the main direction of the building in the point cloud is aligned with the coordinate axis to obtain a reoriented point cloud. Step 3: Project the near-horizontal planar patch onto the grayscale image to form a connected region, and assign a label to each region using a graph cut method, with one label for the interior of the building and another label for the exterior space; the removal of the exterior of the building is accomplished by overlaying a mask converted from the interior region with the reoriented point cloud.

2. The method for accurate indoor point cloud extraction based on orthogonal direction and graph cut optimization according to claim 1, characterized in that, Before extracting the initial plane in step 1, point cloud integration is required. The specific method is as follows: The original data is downsampled using voxels. Specifically, voxel meshing is used to merge points in the same voxel into one point to reduce the density of the point cloud data and obtain a downsampled point cloud. In order to calculate the representative coordinates of points in each voxel, the centroid method is used, that is, the geometric center of each point in each voxel is calculated as the new representative point. Subsequently, local surface features are estimated; these local surface features include normals and curvatures, which are used for the detection of planar structures; specifically, principal component analysis (PCA) is used to calculate the local surface features of the downsampled point cloud; PCA is used to estimate the normal direction and curvature of each point by fitting local tangent planes, thereby obtaining the salient features of the points; The parameters of the local surface are solved by the total least squares algorithm; the total least squares algorithm calculates the coefficients of the plane equation by minimizing the distance from the point to the plane, thereby obtaining the accurate parameters of the local surface.

3. The method for accurate indoor point cloud extraction based on orthogonal direction and graph cut optimization according to claim 2, characterized in that, The specific operation of the hybrid segmentation method is as follows: First, the downsampled point cloud with directional normals is projected onto a unit sphere using Gaussian mapping. Next, the point with the smallest curvature is selected as the seed region, and then the region is expanded on the sphere based on the angular differences between the normal vectors. Starting from the current seed point on the unit sphere, adjacent points with angular differences within a certain range are gradually added to the seed region until no more points meet the conditions. The expansion of the seed region ends when no new points meet the expansion conditions or when the preset region size threshold is reached. Through the expansion process, point clouds with similar normal vectors are grouped together to form clusters. The points in these clusters are typically distributed on a set of parallel planes. By borrowing the core idea of ​​the Random Sample Consensus (RANSAC) algorithm, the orientation of the plane is robustly captured from the normal vectors of points belonging to the same cluster, and this orientation is used as a constraint for subsequent planar structure extraction. Specifically, a unit normal vector of a point is randomly selected from the cluster as a reference, and the number of remaining normal vectors whose angle difference with it is less than a given threshold is calculated. This process is repeated to obtain the optimal orientation of the cluster. To ensure accuracy, all possible reference normal vectors need to be traversed to obtain the optimal planar orientation. Subsequently, a RANSAC-based segmentation method with orientation constraints is used to extract the planar structure of each cluster. The specific process includes: firstly, using the RANSAC algorithm to estimate the parameters of the plane, while applying orientation constraints to ensure that the extracted plane is consistent with the optimal orientation of the cluster; to ensure high accuracy and reliability of the planar structure extracted from the point cloud data, a hypothesis step and a verification step are proposed; in the hypothesis step, only candidate planes with an angle greater than a set threshold to the cluster orientation are considered. This threshold is used to ensure that the candidate plane maintains a sufficiently perpendicular relationship with the optimal orientation of the cluster, thereby improving the accuracy of the extracted plane; in the verification step, the optimal parameters of the planar model are determined based on the distance from the point to the candidate plane, and the plane with an angle greater than the set threshold to the cluster orientation and the smallest distance from the point to the plane is selected as the final planar structure of that cluster; the planar structures of all extracted clusters will be one of the output results, namely, the indoor planar primitive IP.

4. The method for accurate indoor point cloud extraction based on orthogonal direction and graph cut optimization according to claim 3, characterized in that, Meaningful planar patches are extracted from planar structures by introducing the Fast Euclidean Clustering (FEC) method. Specifically, FEC first constructs a KD tree on coplanar points to accelerate the search for neighboring points within a fixed range. Then, planar patches are formed by clustering points that are close to each other in space into subclusters and merging adjacent subclusters based on connectivity information.

5. The method for accurate indoor point cloud extraction based on orthogonal direction and graph cut optimization according to claim 1, characterized in that, Step 2 is explained in the following steps: Step (2.1) Virtual plane proxy creation; For all the planar structures and their corresponding point sets extracted in step 1, they are classified into different groups based on their positional relationships, with each group represented by a virtual planar proxy. The specific operation is as follows: Consider two relationships between shapes, namely parallelism and orthogonality. First, use the parameter Ang to represent the maximum threshold of angular deviation to determine whether two planar structures are parallel. If the angle between the normal directions of two planar structures is within this threshold range, they are considered parallel. Then, select the planar structure with the most coplanar points as the seed planar structure, and expand these planar structures outward according to their parallel relationship with other planar structures to form groups, until there are no more planar structures parallel to it available for expansion. Then, reselect the seed planar structure from the remaining planar structures for expansion, and repeat the above operation until there are no more mutually parallel planar structures. In each group, generate a virtual planar proxy, which represents the almost parallel planar structures in this group. The parameters of the virtual plane are determined by the average normal direction and anchor points of the planar structures within the group. Step (2.2) Estimation of the dominant direction; By comparing the normal directions of virtual plane proxies pairwise, the orthogonality between different virtual plane proxies is determined, and these relationships are used to obtain candidate dominant directions. Initially, virtual plane proxies are sorted in descending order of the number of interior points, and the normal direction of the virtual plane with the most interior points is marked as the first candidate direction. Then, the normal directions of other unmarked virtual planes are compared one by one. When the direction deviation between two virtual planes is less than a predefined threshold (π / 2-Ang), the two virtual planes are determined to be perpendicular and marked as a candidate direction orthogonal to the first candidate direction. Next, a third plane direction orthogonal to both of the marked candidate directions is selected, thus obtaining three mutually orthogonal candidate directions. The virtual plane proxies corresponding to the three candidate directions are used as candidate virtual plane proxies. Global regularization is applied to candidate virtual plane proxies to accurately extract dominant directions. Specifically, a nonlinear optimization method with orthogonal constraints is adopted, taking three candidate virtual plane proxies and their related points as input, and outputting three completely perpendicular dominant directions that match the main axis of the building. The goal is to minimize the data fitting error from the interior point to the related plane, while considering the orthogonality between the planes. The fitting error is accumulated by the square of the Euclidean distance from the point to the virtual plane. Furthermore, the influence of a point is weighted based on its original position and normal direction information; The nonlinear optimization method with orthogonal constraints aims to minimize the data fitting error from the interior point to the relevant plane, while ensuring the orthogonality between the planar surrogates; the fitting error can be quantified by the following formula: Where, p i Represents the coordinates of the i-th point; n j d represents the unit normal vector corresponding to the candidate virtual plane proxy j; j ω represents the offset in the equation of plane j. i It is based on point p i The weights are determined by the original position and normal direction information; Furthermore, the orthogonal constraint is expressed through this condition: n j ·n k =0, j≠k; where n j and n k Represents the normal vectors of two distinct candidate virtual plane proxies; The optimization algorithm extracts three orthogonal dominant directions that match the main axis of the building by minimizing the above error function and simultaneously satisfying the orthogonal constraints between planar proxies.

6. The method for accurate indoor point cloud extraction based on orthogonal direction and graph cut optimization according to claim 5, characterized in that, The parameters of the virtual plane are determined through the following steps: Calculation of average normal direction: For a group of planar structures, first calculate the normal vector of each planar structure; then, take a weighted average of these normal vectors to obtain the average normal direction of the planar structures in the group. Determine the anchor point of the virtual plane: Within each group, select the plane with the largest area as the representative plane, and select the centroid point on that plane as the anchor point of the virtual plane; The equation of the virtual plane is determined as follows: the equation of the virtual plane is defined based on the average normal direction and the anchor point; the plane equation is expressed as: n·(r-r0)=0; where n is the average normal direction, r is the vector of any point on the plane, and r0 is the vector of the anchor point.

7. A method for accurate indoor point cloud extraction based on orthogonal direction and graph cut optimization according to claim 1 or 5, characterized in that, The specific method for step 3 is as follows: After obtaining the dominant direction of the point cloud in step 2, the dominant direction in the downsampled point cloud is aligned with the principal axis of the global coordinate system using a rotation matrix to obtain a reoriented point cloud; outdoor points are removed from the reoriented point cloud. First, the planar patch and its interior points are transformed into the global coordinate system. Horizontal planar patches are selected as candidate patches based on the angle between their normal vector and the z-axis. Specifically, planar patches whose normal vector and z-axis angle is less than a predetermined angle difference threshold are selected as nearly horizontal planar patches. Then, candidate patches related to the ceiling, floor, and other horizontal components are projected onto the xy plane and rasterized into grayscale images with a fixed step size. Each pixel value in the grayscale image is determined by the planar patch at the highest point in each grid. Subsequently, grayscale thresholding is introduced to compare the pixel values ​​in the image with a preset grayscale threshold to mark the foreground and background regions. Then, connectivity analysis is applied to connect adjacent foreground pixels into connected regions, and each region is assigned a unique number to identify and distinguish different connected regions. Next, the connected graph is partitioned into two disjoint and optimal subsets of regions using a graph cut algorithm. The graph cut algorithm process includes constructing the graph, defining the cutting cost function, and calculating the optimal cutting boundary. Specifically, when constructing the graph, regions are used as vertices, and edges are established to represent the relationships between adjacent regions, thus generating an undirected graph G = ...<V,E> The vertex set V represents the obtained region, and the edge set E represents the pair of adjacent regions. The cutting cost function is defined based on the similarity between regions and the known labels, and the maximum flow-minimum cut theorem is used to solve for the optimal cutting boundary. Finally, an initial label is added to each vertex, marking it as 1 or 0, representing indoor space and outdoor space respectively, to achieve the classification of regions.

8. The method for accurate indoor point cloud extraction based on orthogonal direction and graph cut optimization according to claim 7, characterized in that, For each vertex in the graph, the initial label is inferred from three geometric properties: area overlap, point cover, and wall support. The area overlap metric assesses the degree of overlap between a given area and the ceiling and floor. Specifically, it calculates the overlap area between area A and the ceiling C and floor F: Where Area(A∩C) and Area(A∩F) represent the overlapping areas of area A with the ceiling and floor, respectively, while Area(A) represents the total area of ​​area A; Point coverage assesses the distribution of points within a given region; specifically, assuming the set of points within region A is P. A The total number of points is |P A If the point cover can be evaluated by calculating the point density, then: P A | represents the total number of points in region A, while Area(A) represents the total area of ​​region A; high-density point coverage means that the region is more likely to be an indoor space; The assessment focuses on whether the area is adjacent to or supported by a wall; specifically, assuming there is a set of points P within area A. A And the set of points closest to the wall is P. W The wall support can then be evaluated by calculating the average distance from points within the area to the wall: Dist(p,w) represents the distance from point p to point w on the wall. A | represents the total number of points within region A; the lower the value for wall support, the closer the region is to the wall, and the more likely it is to be an indoor space. Based on these three attributes, an initial label is generated for each vertex, and a decision is made by setting a threshold: Threshold1, Threshold2, and Threshold3 are thresholds derived from experience or data training, used to distinguish between indoor and outdoor areas.

9. The method for accurate indoor point cloud extraction based on orthogonal direction and graph cut optimization according to claim 7, characterized in that, In the building exterior point removal step, firstly, the two region subsets divided based on the initial labels are mapped into a two-dimensional grid to form a binary mask image, where 1 represents the indoor region and 0 represents the building exterior region; The reoriented point cloud is then further refined: To supplement missing wall information, firstly, vertical patches are rasterized with the same grid size and attached to the boundary pixels of the region; then, a flood fill algorithm is used to process internal holes in the image to completely cover the interior space; in addition, to remove outliers above the ceiling and below the floor, the height range of each grid needs to be calculated; firstly, the heights of the top ceiling and bottom floor are extracted from the near-horizontal planar structure through histogram analysis to determine the initial height threshold; subsequently, local adjustments are made using the highest and lowest planar patches within the grid; specifically, the adjustment method dynamically adjusts the height range of each grid based on the initial height threshold to remove outliers exceeding the set range; Finally, by overlaying the refined reoriented point cloud with a binary mask image and cropping it with an adjusted height range, the desired indoor points are retained while the external points of the building are removed.

Images