Indoor point cloud accurate extraction method based on orthogonal direction and graph cut optimization

Abstract

The invention discloses an indoor point cloud accurate extraction method based on orthogonal direction and graph cut optimization. The method has remarkable beneficial effects in point cloud processing and building information extraction. Firstly, through orthogonal direction estimation and nonlinear optimization, point cloud data can be aligned with the main axis of a building with high precision, and it is ensured that a model is consistent with an actual structure. Secondly, according to the method, external point cloud data of the building are effectively eliminated through a graph cut model, noise interference is reduced, and thus effective point cloud inside the building is reserved. In addition, by utilizing plane detection and orthogonal constraint, the accuracy and integrity of indoor space extraction are improved. On the whole, the method not only is high in automation degree and remarkably reduces manual adjustment requirements, but also has strong robustness, and can still stably extract the internal structure information of the building even under the condition of data missing or noise existence.

Description

Technical Field

[0001] The present invention belongs to the technical field of point cloud processing, building information modeling (BIM), and three-dimensional spatial data analysis, and relates to accurate processing and analysis of point cloud data in complex building environments, and specifically to an accurate indoor point cloud extraction method based on orthogonal direction and graph cut optimization. The method includes automatic segmentation of the interior and exterior spaces of a building, plane detection and estimation of orthogonal directions, and cleaning and filtering of point cloud data by graph cuts and height range clipping. Background Art

[0002] In the field of 3D building information modeling (BIM) and point cloud processing, with the advancement of laser scanning and photogrammetry technology, it is becoming more and more common to obtain high-precision 3D point cloud data. These point cloud data are often used for accurate modeling of buildings, indoor space analysis, and virtual reality applications. However, since point cloud data is usually unstructured and contains a lot of noise and external interference (such as data of the building's external environment), how to effectively extract the internal geometric information of the building and align it with the main axis of the building is a challenging task.

[0003] Traditional methods often rely on manual labeling or simple plane fitting algorithms, which are difficult to deal with complex building structures and a large amount of noisy data. In recent years, many automated point cloud processing technologies such as plane detection, orthogonal direction estimation and graph cut optimization have been proposed to improve the extraction accuracy of the interior space of the building. These technologies can automatically identify and eliminate the point cloud data outside the building, and accurately align the internal geometric structure of the building through orthogonal direction constraints, providing a high-quality data foundation for building information modeling and analysis. However, the existing technology still faces some shortcomings, such as insufficient accuracy in processing complex building structures, challenges in processing extreme noise data, limitations of orthogonal direction constraints, and high complexity of global optimization calculations. The present invention aims to improve the accuracy and efficiency of point cloud processing and overcome the shortcomings of the above-mentioned prior art through an improved plane detection method, enhanced orthogonal direction estimation and an optimized global graph cut algorithm. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides an accurate extraction method of indoor point cloud based on orthogonal direction and graph cut optimization. The present invention adopts a new framework for point cloud preprocessing of complex indoor scenes. A brief description of the framework is as follows Figure 1As shown. The method of the present invention processes scattered points P generated by various scanning devices, which are generated by scanning with different devices and are usually affected by tilt and outliers. The algorithm does not require additional information such as intensity or scanning position, but uses the geometric characteristics of the point cloud to expand its application range. In addition, the present invention assumes a weak Manhattan world structure prior with arbitrary wall directions to guide the processing of noisy point clouds. The output results of the preprocessing are the redirected point cloud RP and the cleaned point cloud CP, the purpose of which is to align the main directions of the building with the coordinate axes and remove the background areas outside the building layout, so that the point cloud data is more regular and accurate. In addition, the indoor plane primitive IP is one of the output results, which serves as a lightweight representation of the scene and provides high-level information for further reconstruction.

[0005] A method for accurately extracting indoor point clouds based on orthogonal directions and graph cut optimization includes the following steps:

[0006] Step 1: Extract the initial plane from the scattered points P through a hybrid segmentation method. These scattered points P are generated by scanning with different devices and may contain abnormal or error points.

[0007] The hybrid segmentation method combines the ideas of region growing, model fitting and clustering. The specific operations include: first, based on the normal information of the point cloud, the region on the smooth surface will continue to grow through region growing, and then, the plane S = {Si} is extracted from the grown region using the random sampling consensus algorithm RANSAC with normal direction constraints. RANSAC will randomly select a point set for plane model fitting, select points consistent with the fitting model, and form a plane structure. The formed plane structure will be used as one of the output results, namely the indoor plane primitive IP. Next, fast Euclidean clustering (FEC) is introduced to separate the points in the plane structure Si into patches L = {li}. The FEC algorithm classifies points by Euclidean distance and divides the point set of the same plane into different patches. These steps help to extract accurate planes from scattered points P and further clean up point cloud data.

[0008] Step 2: Assemble the extracted planar structures into planar clusters with similar orientations.

[0009] Specifically, the extracted plane structures are classified according to the similarity of their normal directions to form several plane clusters, which usually correspond to the main plane directions in the building. After obtaining the roughly orthogonal plane clusters and their corresponding internal points, the dominant direction of the building is determined using nonlinear optimization with orthogonality constraints. The nonlinear optimization algorithm optimizes the normal direction of the plane cluster to make it as consistent as possible with the preset orthogonal constraints, thereby finding the dominant direction of the building. Using this direction as a reference for the rotation of the coordinate system, the point cloud is rotated and adjusted so that the main direction of the building in the point cloud is aligned with the coordinate axis to obtain a redirected point cloud. In this way, the direction of the building can be consistent with the direction of the coordinate system, thereby providing a unified reference frame for subsequent point cloud processing and reconstruction.

[0010] Step 3: Project the nearly horizontal planar patches into the grayscale image to form connected regions, and assign labels to each region through the graph cut method, with one label for the interior of the building (i.e., labeled as "interior of the building") and another label for the exterior space (i.e., labeled as "exterior of the building"). The removal of the exterior of the building is completed by overlapping the mask converted from the indoor area (i.e., the interior of the building) with the reoriented point cloud (i.e., the point cloud data after rotation adjustment in step 2).

[0011] The beneficial effects of the present invention are as follows:

[0012] The present invention proposes a method for accurate extraction of indoor point clouds based on orthogonal directions and graph cut optimization, which has significant beneficial effects in point cloud processing and building information extraction. First, through orthogonal direction estimation and nonlinear optimization, the point cloud data can be aligned with the main axis of the building with high precision to ensure that the model is consistent with the actual structure. Secondly, the method effectively removes the point cloud data outside the building through the graph cut model to reduce noise interference, thereby retaining the valid point cloud inside the building. In addition, the accuracy and completeness of indoor space extraction are improved by plane detection and orthogonal constraints. Overall, this method not only has a high degree of automation and significantly reduces the need for manual adjustment, but also has strong robustness. Even in the absence of data or the presence of noise, it can still robustly extract the internal structural information of the building. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 The figure is a flow chart of a method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0014] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0015] like Figure 1 As shown, the present invention proposes an indoor point cloud accurate extraction method based on orthogonal direction and graph cut optimization, and the specific implementation steps are as follows:

[0016] Step 1: plane primitive extraction;

[0017] Step (1.1) point cloud integration;

[0018] In order to suppress the negative impact of uneven point density and reduce the computational cost of redundant point clouds, as a key first step, the original data is downsampled by voxel (voxel size is 0.05m). Specifically, through voxel gridding, the points in the same voxel are merged into one point to reduce the density of the point cloud data and obtain a downsampled point cloud. In order to calculate the representative coordinates of the points in each voxel, the centroid method is used, that is, the geometric center of the points in each voxel is calculated as the new representative point.

[0019] Subsequently, local surface features are estimated. Local surface features include normals and curvatures, which are used for planar structure detection (i.e., the process of identifying and extracting planar regions from point cloud data). Specifically, principal component analysis (PCA) is used to calculate the local surface features of the downsampled point cloud. PCA is used to estimate the normal direction and curvature of each point by fitting the local tangent plane, thereby obtaining the salient features of the point.

[0020] The parameters of the local surface (such as the coefficients of the plane equation) are solved by the total least squares algorithm. The total least squares algorithm calculates the coefficients of the plane equation by minimizing the distance from the point to the plane, thereby obtaining the accurate parameters of the local surface.

[0021] Step (1.2) hybrid plane detection;

[0022] In view of the geometric characteristics of point clouds, a hybrid segmentation method combining region growing, model fitting and clustering is designed to extract indoor structural planes. First, the downsampled point cloud with directional normals is projected onto the unit sphere by Gaussian mapping; next, the point with the smallest curvature is selected as the seed region, and then it is expanded on the sphere according to the angle difference between the normal vectors (the expansion process refers to starting from the current seed point on the unit sphere, and gradually adding adjacent points with angle differences within a certain range to the seed region until there are no more points that meet the conditions). The end condition of the expansion of the seed region is to stop the expansion when there are no new points that meet the expansion conditions or the preset region size threshold is reached.

[0023] Through the expansion process, point clouds with similar normal vectors are clustered together and eventually form clusters. The points in these clusters are usually distributed on a set of parallel planes. By borrowing the core idea of ​​the random sampling consensus algorithm RANSAC, the direction of the plane is robustly captured from the normal vectors of points belonging to the same cluster, and this direction is used as a constraint for subsequent plane structure extraction. Specifically, the unit normal vector of a point is randomly selected from the cluster as the reference, and the number of remaining normal vectors whose angle difference with it is less than a given threshold is calculated. This process is to determine the optimal direction, that is, the direction of the normal vector with the most internal points. Repeat this process to obtain the optimal direction of the cluster. In order to ensure accuracy, it is usually necessary to traverse all possible reference normal vectors to obtain the optimal plane direction.

[0024] Subsequently, the plane structure of each cluster is extracted by a RANSAC-based segmentation method with direction constraints. The specific process includes: first, the RANSAC algorithm is used to estimate the parameters of the plane, and the direction constraint is applied to ensure that the extracted plane is consistent with the optimal direction of the cluster. In order to ensure that the plane structure extracted from the point cloud data has high accuracy and reliability, a hypothesis step and a verification step are proposed. In the hypothesis step, only candidate planes whose angle with the direction of the cluster is greater than a set threshold (preferably 85°) are considered. This threshold is used to ensure that the candidate plane maintains a sufficiently vertical relationship with the optimal direction of the cluster to improve the accuracy of the extracted plane. In the verification step, the optimal parameters of the plane model are determined according to the distance from the point to the candidate plane, and the plane whose angle with the direction of the cluster is greater than a set threshold (preferably 85°) and the distance from the point to the plane is the smallest is selected as the final plane structure of the cluster. The plane structure of all extracted clusters will be one of the output results, namely the indoor plane primitive IP.

[0025] In addition, the fast Euclidean clustering (FEC) method is introduced to extract meaningful plane patches from planar structures. The FEC method is an efficient clustering technique. Compared with traditional Euclidean clustering, it reduces the number of traversals of neighboring points through a point-to-point scheme, thereby speeding up the processing speed. Specifically, FEC first constructs a KD tree on coplanar points to accelerate the search for neighboring points within a fixed range (i.e., within a specified radius, the specified radius in this experiment is 0.05m). Then, planar patches are formed by clustering points that are close to each other in space into subclusters and merging adjacent subclusters based on connection information. These planar patches represent the actual planar structures in the point cloud, such as walls or floors, and provide important information for subsequent building reconstruction and indoor layout analysis.

[0026] Step 2: Coordinate system transformation;

[0027] Step (2.1) virtual plane proxy creation;

[0028] For the plane structures of all clusters extracted in step 1 and their corresponding point sets, they are classified into different groups according to their positional relationships, and each group is represented by a virtual plane proxy. The specific operation is as follows: Consider two relationships between shapes, namely parallelism and orthogonality, which are widely present in artificial buildings. First, the parameter Ang is used to represent the maximum threshold of angle deviation to determine whether the two plane structures are parallel. If the angle between the normal directions of the two plane structures is within the threshold range, they are considered to be parallel. Then, the plane structure with the most coplanar points is selected as the seed plane structure, and these plane structures are expanded outward according to their parallel relationship with other plane structures to form a group until there are no more plane structures parallel to them to be expanded. Then, the seed plane structure is reselected from the remaining plane structures for expansion, and the above operation is repeated until there are no mutually parallel plane structures. A virtual plane proxy is generated in each obtained group, which serves as a representative of this group of almost parallel plane structures. The parameters of the virtual plane are determined by the average normal direction and anchor points of the plane structures in the group. This process ensures that the plane structures of each group are geometrically similar, thereby simplifying the subsequent coordinate system transformation.

[0029] The parameters of the virtual plane are determined by the following steps:

[0030] Calculation of average normal direction: For a planar structure within a group, first calculate the normal vector of each planar structure. Then, perform weighted average of these normal vectors to obtain the average normal direction of the planar structure within the group.

[0031] Determine the anchor point of the virtual plane: In each group, select the plane with the largest area as the representative plane, and select the centroid point from the plane as the anchor point of the virtual plane (the anchor point is a specific point used to determine the position of the virtual plane, which together with the average normal direction defines the spatial position and orientation of the virtual plane).

[0032] Determination of the equation of the virtual plane: The equation of the virtual plane is defined according to the average normal direction and the anchor point. The plane equation is expressed as: n·(r-r0)=0. Where n is the average normal direction, r is the vector of any point on the plane, and r0 is the vector of the anchor point.

[0033] Step (2.2) dominant direction estimation;

[0034] By comparing the normal directions of virtual plane proxies pair by pair, the orthogonal relationships between different virtual plane proxies are determined, and these relationships are used to obtain candidate dominant directions. Initially, the virtual plane proxies are arranged in descending order according to the number of inliers, and the normal direction of the virtual plane with the largest number of inliers is marked as the first candidate direction. Then, the normal directions of the virtual planes with the other unmarked virtual planes are compared one by one. When the direction deviation between the two virtual planes is less than the predefined threshold (π / 2-Ang), the two virtual planes are judged to be perpendicular and marked as the candidate direction orthogonal to the first candidate direction. Then, the third plane direction orthogonal to both marked candidate directions is further selected, thereby obtaining three mutually orthogonal candidate directions, and the virtual plane proxies corresponding to the three candidate directions are used as candidate virtual plane proxies. It is worth noting that the three candidate directions are closely related to the structural elements of the building. The first direction is roughly consistent with the direction of the ceiling or floor, while the remaining directions usually correspond to the directions of the walls. However, due to the presence of noise in the unorganized point cloud, the almost orthogonal candidate directions may have certain deviations and cannot accurately reveal the indoor layout.

[0035] Therefore, based on the assumption that the ground (ceiling) and the wall intersect at right angles in a typical indoor environment, the candidate virtual plane proxies are globally regularized to accurately extract the dominant directions. Specifically, a nonlinear optimization method with orthogonal constraints is adopted, taking three candidate virtual plane proxies and their associated points as input, and outputting three dominant directions that are completely perpendicular to each other, matching the main axes of the building. The goal is to minimize the data fitting error from the inner points to the associated planes while considering the orthogonal relationship between the planes. The fitting error is accumulated by the square of the Euclidean distance from the point to the virtual plane. In addition, the influence of the point is weighted according to its original position and normal direction information.

[0036] The goal of the nonlinear optimization method with orthogonality constraints is to minimize the data fitting error from the interior points to the relevant plane while ensuring the orthogonality between the plane agents. The fitting error can be quantified by the following formula:

[0037]

[0038] Among them, p i represents the coordinates of the i-th point; n j represents the unit normal vector corresponding to the candidate virtual plane proxy j; d j represents the offset in the plane equation of plane j; ω i According to point p i The weight is determined by the original position and normal direction information.

[0039] In addition, the orthogonality constraint is expressed by the condition: n j ·n k= 0 (for all j≠k). j and n k Represents the normal vectors of two different candidate virtual plane proxies.

[0040] The optimization algorithm extracts three orthogonal dominant directions that match the main axes of the building by minimizing the above error function and satisfying the orthogonality constraints between the plane agents.

[0041] Step 3: Removal of building external points;

[0042] After obtaining the dominant direction of the point cloud in step 2, the dominant direction in the downsampled point cloud is aligned with the principal axis of the global coordinate system through the rotation matrix to obtain a reoriented point cloud. This step realizes the reorientation of the point cloud so that the main planes in the point cloud (such as the ground, wall, etc.) are aligned with the global coordinate system, eliminating the data tilt caused by the scanning device or scene complexity. After solving the data tilt problem, the next step is to remove outdoor points from the reoriented point cloud. First, the plane patch and its internal points are transformed into the global coordinate system, and the horizontal plane patch is selected as the candidate patch according to the angle between its normal vector and the z-axis. Specifically, the plane patch whose normal angle with the z-axis is less than the predetermined angle (5°) difference threshold is selected and determined as a nearly horizontal plane patch. Then, the candidate patches related to the ceiling, floor and other horizontal components are projected to the xy plane and rasterized into a grayscale image with a fixed step size (0.1m). Each pixel value in the grayscale image is determined by the plane patch of the highest point (i.e., the point with the largest z coordinate value) in each grid. Subsequently, grayscale threshold segmentation is introduced to compare the pixel values ​​in the image with the preset grayscale threshold to mark the foreground and background areas. Then, connectivity analysis is applied to connect adjacent foreground pixels into connected regions and assign a unique number to each region to identify and distinguish different connected regions. These steps help refine the image and identify areas related to the exterior of the building, thereby achieving effective point cloud removal.

[0043] Next, the connected graph is divided into two non-overlapping and optimal region subsets through the graph cut algorithm. The process of the graph cut algorithm includes constructing a graph, defining a cutting cost function, and calculating the optimal cutting boundary. Specifically, when constructing the graph, the regions are used as vertices, and edges are established to represent the relationship between adjacent regions, thereby generating an undirected graph G = <V, E>, where the vertex set V represents the obtained region and the edge set E represents the pair of adjacent regions. The cutting cost function is defined based on the similarity between regions and known labels (referring to regions whose categories have been determined in advance), and the maximum flow-minimum cut theorem is used to solve the optimal cutting boundary. Finally, an initial label is added to each vertex, marking it as 1 (indoor space) or 0 (outdoor space) to achieve the classification of the region.

[0044] For each vertex in the graph, the initial label is inferred from three geometric properties, namely area overlap, point coverage, and wall support.

[0045] Among them, the area overlap evaluates the degree of overlap between a certain area and the ceiling and floor. Since ceilings and floors often appear in pairs, the degree of overlap between them is usually higher in indoor areas than in outdoor areas. Specifically, for calculating the overlapping area between area A and ceiling C and floor F: Where Area(A∩C) and Area(A∩F) represent the overlapping areas of area A with the ceiling and floor respectively, and Area(A) represents the total area of ​​area A.

[0046] Point coverage evaluates the distribution of points in the area, that is, the density of points in the area and the degree of matching of distribution characteristics with point cloud data, which can help determine whether it belongs to indoor space. Indoor areas usually have higher point coverage density because point cloud data in indoor spaces is denser. Specifically, suppose the point set in area A is P A , the total points are |P A |, then the point coverage can be evaluated by calculating the density of the points: P A | represents the total number of points in area A, and Area(A) represents the total area of ​​area A. A high density of point coverage means that the area is more likely to be an indoor space.

[0047] Wall support evaluates whether the area is close to a wall or supported by a wall. Since walls often exist in indoor areas, areas close to walls are more likely to belong to indoor spaces. Specifically, suppose there is a set of points P in area A. A , and the point set close to the wall is P W , then the wall support can be evaluated by calculating the average distance from the points in the area to the wall: Dist(p,w) represents the distance from point p to wall point w, P A | is the total number of points in region A. The lower the value of wall support, the closer the region is to the wall and the more likely it is to be an indoor space.

[0048] Based on these three properties, an initial label is generated for each vertex, and a decision is made by setting a threshold: Among them, Threshold1, Threshold2, and Threshold3 are thresholds obtained based on experience or data training (the thresholds may be different in different scenarios) and are used to distinguish indoor and outdoor areas.

[0049] Through the widely used graph cut method, the energy function can be efficiently optimized to achieve the best segmentation of the region. In the step of removing the external points of the building, first, the two region subsets divided based on the initial labels are mapped into a two-dimensional grid to form a binary mask image, where 1 represents the indoor area and 0 represents the external area of ​​the building.

[0050] The redirected point cloud is then improved: in order to supplement the missing wall information, first, the vertical patches are rasterized with the same grid size (0.05m is used in the embodiment) and attached to the boundary pixels of the area. Then, the internal holes of the image are processed using a flood filling algorithm to completely cover the indoor space. In addition, in order to remove outliers above the ceiling and below the floor, the height range of each grid needs to be calculated. First, the heights of the top ceiling and bottom floor are extracted from the nearly horizontal plane structure (referring to the plane structure that is nearly parallel to the ground, such as the ceiling and floor) through histogram analysis to determine the initial height threshold. Subsequently, local adjustments are made through the highest and lowest plane patches within the grid; the specific adjustment method is to dynamically adjust the height range of each grid according to the initial height threshold to remove outliers that exceed the set range.

[0051] Finally, the desired indoor points are retained while the building exterior points are removed by overlaying the refined reoriented point cloud with the binary mask image and cropping it with the adjusted height range. This process ensures that only the point cloud data inside the building is retained and the external noise and irrelevant points are effectively removed.

[0052] The above contents are further detailed descriptions of the present invention in combination with specific / preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, they can also make several substitutions or modifications to these described embodiments without departing from the concept of the present invention, and these substitutions or modifications should be regarded as belonging to the protection scope of the present invention.

[0053] Parts of the present invention that are not described in detail belong to the well-known technologies of those skilled in the art.

Claims

1. A method for accurate extraction of indoor point clouds based on orthogonal directions and graph cut optimization, characterized in that: The steps include: Step 1: Extract the initial plane from the scattered points P by hybrid segmentation method; The hybrid segmentation method combines the ideas of region growing, model fitting and clustering; the specific operations include: first, based on the normal information of the point cloud, the region on the smooth surface will continue to grow through region growing, and then, the plane S = {Si} is extracted from the grown region using the random sampling consensus algorithm RANSAC with normal direction constraints; RANSAC will randomly select a point set for plane model fitting, select points consistent with the fitting model, and form a plane structure; the formed plane structure will be used as one of the output results, that is, the indoor plane primitive IP; then, the fast Euclidean clustering FEC is introduced to separate the points in the plane structure Si into patches L = {li}; the FEC algorithm classifies the points by Euclidean distance and divides the point set on the same plane into different patches; Step 2: Assemble the extracted planar structures into planar clusters with similar directions; Specifically, the extracted plane structures are classified according to the similarity of their normal directions to form several plane clusters, which correspond to the main plane directions in the building; after obtaining the roughly orthogonal plane clusters and their corresponding inner points, the dominant direction of the building is determined using nonlinear optimization with orthogonality constraints; the nonlinear optimization algorithm optimizes the normal direction of the plane clusters to make it as consistent as possible with the preset orthogonal constraints, thereby finding the dominant direction of the building; using this direction as a reference for the rotation of the coordinate system, the point cloud is rotated and adjusted so that the main direction of the building in the point cloud is aligned with the coordinate axis to obtain a reoriented point cloud; Step 3: Nearly horizontal planar patches are projected into the grayscale image to form connected regions, and labels are assigned to each region through the graph cut method, where one label is for the interior of the building and the other label is for the exterior space; the removal of the building exterior is completed by overlapping the mask converted from the indoor area with the reoriented point cloud.

2. According to claim 1, a method for accurately extracting indoor point clouds based on orthogonal directions and graph cut optimization is characterized in that: Step 1: Point cloud integration is required before extracting the initial plane. The specific method is as follows: The original data is downsampled by voxel. Specifically, the points in the same voxel are merged into one point through voxel gridding to reduce the density of the point cloud data and obtain a downsampled point cloud. In order to calculate the representative coordinates of the points in each voxel, the centroid method is used, that is, the geometric center of the points in each voxel is calculated as the new representative point. Subsequently, local surface features are estimated; local surface features include normals and curvatures, which are used to detect planar structures; specifically, principal component analysis (PCA) is used to calculate local surface features of the downsampled point cloud; PCA is used to estimate the normal direction and curvature of each point by fitting the local tangent plane, thereby obtaining the salient features of the point; The parameters of the local surface are solved by the total least squares algorithm; the total least squares algorithm calculates the coefficients of the plane equation by minimizing the distance from the point to the plane, thereby obtaining the exact parameters of the local surface.

3. The method for accurately extracting indoor point clouds based on orthogonal directions and graph cut optimization according to claim 2 is characterized in that: The specific operation of the hybrid segmentation method is as follows: First, the downsampled point cloud with directional normals is projected onto the unit sphere through Gaussian mapping; next, the point with the smallest curvature is selected as the seed region, and then it is expanded on the sphere according to the angle difference between the normal vectors. Starting from the current seed point on the unit sphere, the adjacent points with angle differences within a certain range are gradually added to the seed region until there are no more points that meet the conditions; the end condition of the seed region expansion is to stop the expansion when there are no new points that meet the expansion conditions or the preset region size threshold is reached; Through the expansion process, point clouds with similar normal vectors are gathered together to eventually form clusters; the points in these clusters are usually distributed on a set of parallel planes; by borrowing the core idea of ​​the random sampling consensus algorithm RANSAC, the direction of the plane is robustly captured from the normal vectors of points belonging to the same cluster, and this direction is used as a constraint for subsequent plane structure extraction; specifically, the unit normal vector of a point is randomly selected from the cluster as a reference, and the number of remaining normal vectors whose angle difference with it is less than a given threshold is calculated; this process is repeated to obtain the optimal direction of the cluster; in order to ensure accuracy, it is necessary to traverse all possible reference normal vectors to obtain the optimal plane direction; Subsequently, the plane structure of each cluster is extracted by a RANSAC-based segmentation method with direction constraints. The specific process includes: first, the RANSAC algorithm is used to estimate the parameters of the plane, and the direction constraint is applied to ensure that the extracted plane is consistent with the optimal direction of the cluster; in order to ensure that the plane structure extracted from the point cloud data has high accuracy and reliability, a hypothesis step and a verification step are proposed; in the hypothesis step, only candidate planes whose angle with the direction of the cluster is greater than a set threshold are considered. This threshold is used to ensure that the candidate plane maintains a sufficiently vertical relationship with the optimal direction of the cluster to improve the accuracy of the extracted plane; in the verification step, the optimal parameters of the plane model are determined according to the distance from the point to the candidate plane, and the plane whose angle with the direction of the cluster is greater than the set threshold and whose distance from the point to the plane is the smallest is selected as the final plane structure of the cluster; the plane structure of all extracted clusters will be one of the output results, namely the indoor plane primitive IP.

4. The method for accurately extracting indoor point clouds based on orthogonal directions and graph cut optimization according to claim 3 is characterized in that: The fast Euclidean clustering (FEC) method is introduced to extract meaningful planar patches from planar structures. Specifically, FEC first constructs a KD tree on coplanar points to accelerate the search for neighboring points within a fixed range. Then, planar patches are formed by clustering points close to each other in space into subclusters and merging adjacent subclusters based on connectivity information.

5. The method for accurate extraction of indoor point clouds based on orthogonal directions and graph cut optimization according to claim 1, characterized in that: Step 2: Step (2.1) virtual plane proxy creation; For the plane structures of all clusters extracted in step 1 and their corresponding point sets, they are classified into different groups according to their positional relationships, and each group is represented by a virtual plane agent; the specific operations are as follows: consider two relationships between shapes, namely parallel and orthogonal; first, use the parameter Ang to represent the maximum threshold of angle deviation to determine whether the two plane structures are parallel; if the angle between the normal directions of the two plane structures is within the threshold range, they are considered to be parallel; then, select the plane structure with the most coplanar points as the seed plane structure, and expand these plane structures outward according to their parallel relationship with other plane structures to form a group until there are no more parallel plane structures to be expanded; then reselect the seed plane structure from the remaining plane structures for expansion, and repeat the above operations until there are no mutually parallel plane structures; generate a virtual plane agent in each obtained group, which serves as the representative of this group of almost parallel plane structures; the parameters of the virtual plane are determined by the average normal direction and anchor points of the plane structures in the group; Step (2.2) dominant direction estimation; By comparing the normal directions of virtual plane proxies pair by pair, the orthogonal relationships between different virtual plane proxies are determined, and these relationships are used to obtain candidate dominant directions; initially, the virtual plane proxies are arranged in descending order according to the number of inliers, and the normal direction of the virtual plane with the largest number of inliers is marked as the first candidate direction; then the normal directions are compared with other unmarked virtual planes one by one, and when the direction deviation between the two virtual planes is less than a predefined threshold (π / 2-Ang), the two virtual planes are determined to be perpendicular and marked as a candidate direction orthogonal to the first candidate direction; then, a third plane direction orthogonal to both marked candidate directions is further selected, thereby obtaining three mutually orthogonal candidate directions, and the virtual plane proxies corresponding to the three candidate directions are used as candidate virtual plane proxies; The candidate virtual plane proxies are globally regularized to accurately extract the dominant directions. Specifically, a nonlinear optimization method with orthogonal constraints is used to take three candidate virtual plane proxies and their associated points as inputs, and output three completely mutually perpendicular dominant directions that match the main axis of the building. The goal is to minimize the data fitting error from the inner point to the associated plane while considering the orthogonal relationship between the planes. The fitting error is accumulated by the square of the Euclidean distance from the point to the virtual plane. Furthermore, the influence of a point is weighted according to its original position and normal direction information; The goal of the nonlinear optimization method with orthogonality constraints is to minimize the data fitting error from the interior points to the relevant plane while ensuring the orthogonality between the plane agents; the fitting error can be quantified by the following formula: Among them, p i represents the coordinates of the i-th point; n j represents the unit normal vector corresponding to the candidate virtual plane proxy j; d j represents the offset in the plane equation of plane j; ω i According to point p i The weight determined by the original position and normal direction information; In addition, the orthogonality constraint is expressed by the condition: n j ·n k =0, j≠k; where n j and n k Represents the normal vectors of two different candidate virtual plane proxies; The optimization algorithm extracts three orthogonal dominant directions that match the main axes of the building by minimizing the above error function and satisfying the orthogonality constraints between the plane agents.

6. The method for accurate extraction of indoor point clouds based on orthogonal directions and graph cut optimization according to claim 5, characterized in that: The parameters of the virtual plane are determined by the following steps: Calculation of average normal direction: For a planar structure within a group, first calculate the normal vector of each planar structure; then, perform weighted average of these normal vectors to obtain the average normal direction of the planar structure within the group; Determine the anchor point of the virtual plane: In each group, select the plane with the largest area as the representative plane, and select the centroid point from the plane as the anchor point of the virtual plane; Determination of the equation of the virtual plane: The equation of the virtual plane is defined according to the average normal direction and the anchor point; the plane equation is expressed as: n·(r-r0)=0; where n is the average normal direction, r is the vector of any point on the plane, and r0 is the vector of the anchor point.

7. The method for accurately extracting indoor point clouds based on orthogonal directions and graph cut optimization according to claim 1 or 5, characterized in that: Step 3: After the dominant direction of the point cloud is obtained in step 2, the dominant direction in the downsampled point cloud is aligned with the principal axis of the global coordinate system through the rotation matrix to obtain a redirected point cloud; outdoor points are removed from the redirected point cloud; First, the plane patch and its internal points are transformed into the global coordinate system, and the horizontal plane patch is selected as the candidate patch according to the angle between its normal vector and the z-axis. Specifically, the plane patch whose normal angle with the z-axis is less than the predetermined angle difference threshold is selected as the nearly horizontal plane patch. Then, the candidate patches related to the ceiling, floor and other horizontal components are projected to the xy plane and rasterized into a grayscale image with a fixed step size. Each pixel value in the grayscale image is determined by the plane patch with the highest point in each grid. Then, grayscale threshold segmentation is introduced to compare the pixel values ​​in the image with the preset grayscale threshold to mark the foreground and background areas; Then, connectivity analysis is applied to connect adjacent foreground pixels into connected regions, and each region is assigned a unique number to identify and distinguish different connected regions; Next, the connected graph is divided into two disjoint and optimal region subsets through the graph cut algorithm. The process of the graph cut algorithm includes constructing the graph, defining the cutting cost function, and calculating the optimal cutting boundary. Specifically, when constructing the graph, the regions are used as vertices, and edges are established to represent the relationship between adjacent regions, thereby generating an undirected graph G =<V,E> , where the vertex set V represents the obtained area, and the edge set E represents the pair of adjacent areas; the cutting cost function is defined according to the similarity between regions and the known labels, and the maximum flow-minimum cut theorem is used to solve the optimal cutting boundary; finally, an initial label is added to each vertex, marking it as 1 or 0, representing indoor space and outdoor space respectively, to achieve the classification of the area.

8. The method for accurately extracting indoor point clouds based on orthogonal directions and graph cut optimization according to claim 7, characterized in that: For each vertex in the graph, the initial label is inferred from three geometric properties, namely, area overlap, point coverage, and wall support; Among them, area overlap evaluates the degree of overlap between a certain area and the ceiling and floor; specifically, for calculating the overlapping area between area A and ceiling C and floor F: Where Area(A∩C) and Area(A∩F) represent the overlapping areas of area A with the ceiling and floor respectively, and Area(A) represents the total area of ​​area A; Point coverage evaluates the distribution of points in the area; specifically, assuming that the point set in area A is P A , the total points are |P A |, then the point coverage can be evaluated by calculating the density of the points: P A | represents the total number of points in area A, and Area(A) represents the total area of ​​area A; high-density point coverage means that the area is more likely to be an indoor space; Wall support evaluates whether the area is close to a wall or supported by a wall; specifically, suppose there is a set of points P in area A. A , and the point set close to the wall is P W , then the wall support can be evaluated by calculating the average distance from the points in the area to the wall: Dist(p,w) represents the distance from point p to wall point w, P A | is the total number of points in area A; the lower the value of wall support, the closer the area is to the wall and the more likely it is to be an indoor space; Based on these three properties, an initial label is generated for each vertex, and a decision is made by setting a threshold: Among them, Threshold1, Threshold2, and Threshold3 are thresholds obtained based on experience or data training and are used to distinguish indoor and outdoor areas.

9. The method for accurately extracting indoor point clouds based on orthogonal directions and graph cut optimization according to claim 7, characterized in that: In the building exterior point removal step, first, the two region subsets divided based on the initial labels are mapped into a two-dimensional grid to form a binary mask image, where 1 represents the indoor area and 0 represents the building exterior area; The redirected point cloud is then improved: in order to supplement the missing wall information, first, the vertical patches are rasterized with the same grid size and attached to the boundary pixels of the area; then, the internal holes of the image are processed using a flood filling algorithm to completely cover the indoor space; in addition, in order to remove outliers above the ceiling and below the floor, the height range of each grid needs to be calculated; first, the heights of the top ceiling and bottom floor are extracted from the nearly horizontal plane structure through histogram analysis to determine the initial height threshold; then, local adjustments are made through the highest and lowest plane patches within the grid; the specific adjustment method is to dynamically adjust the height range of each grid according to the initial height threshold to remove outliers that exceed the set range; Finally, the desired indoor points are retained while the building exterior points are removed by overlaying the refined reoriented point cloud with the binary mask image and cropping it with the adjusted height range.

Images