Indoor super voxel segmentation method considering curved surface structure

Abstract

The invention discloses an indoor super voxel segmentation method considering a curved surface structure, and belongs to the technical field of environmental perception, and the method comprises the following steps: 1, after inputting an indoor three-dimensional point cloud, extracting indoor structure edge points based on a 2D-3D re-projection method, and dividing original data into edge points and non-edge points; 2, based on an improved TBPS super voxel segmentation method, plane super voxels are extracted on non-edge points; and step 3, based on a VCCS super voxel segmentation method, extracting curved surface super voxels on the edge points, and combining the curved surface super voxels with the plane super voxels obtained in the step 2 to realize indoor super voxel extraction. On the premise of indoor structure line extraction, the multi-resolution super voxel segmentation method suitable for the indoor scene is constructed, a good data premise can be provided for subsequent indoor structure extraction and indoor modeling, and a technical premise is provided for rapid processing of indoor three-dimensional point clouds.

Description

Technical Field

[0001] The present invention relates to the field of environmental perception technology, and in particular to an indoor supervoxel segmentation method taking into account curved surface structures. Background Art

[0002] With the rapid development of economic activities, the number of buildings has increased significantly, and human indoor activities have increased significantly. The internal structure of buildings has also become larger and more complex, bringing together various production and daily necessities. The understanding and semantic cognition of indoor scenes is imperative. With the increasing popularity of laser hardware and mobile scanning equipment, the 3D point cloud generated by 3D laser scanning has become an important way to obtain indoor scene information. How to achieve semantic segmentation and real-scene modeling of indoor scenes through 3D point clouds has become a hot topic in today's research.

[0003] Supervoxel is a 3D point cloud preprocessing method, a key preprocessing method for 3D point clouds, and an important part of semantic segmentation and scene modeling. However, the current supervoxel segmentation method is generally targeted at outdoor scenes, and there is no good solution for indoor scenes, especially those with curved structures. Summary of the invention

[0004] 1) Technical issues to be solved

[0005] Based on the above technical problems, the present invention provides an indoor super-voxel segmentation method that takes into account curved structures. Based on the extraction of indoor structural lines, a multi-resolution super-voxel segmentation method suitable for indoor scenes is constructed, which can provide a good data prerequisite for subsequent indoor structure extraction and indoor modeling, and provide a technical prerequisite for rapid processing of indoor three-dimensional point clouds.

[0006] (II) Technical solution

[0007] The present invention provides an indoor supervoxel segmentation method taking into account curved surface structures, which specifically comprises the following steps:

[0008] Step 1: After inputting the indoor 3D point cloud, the edge points of the indoor structure are extracted based on the 2D-3D reprojection method, and the original data are divided into edge points and non-edge points;

[0009] Step 2: extracting planar supervoxels at non-edge points based on the improved TBPS supervoxel segmentation method;

[0010] Step 3: Based on the VCCS supervoxel segmentation method, surface supervoxels are extracted at the edge points and merged with the plane supervoxels obtained in step 2 to achieve indoor supervoxel extraction.

[0011] Furthermore, the specific process of step 1 is as follows:

[0012] Step 1.1: pre-segment the indoor point cloud based on random sampling consistency, and divide the indoor point cloud into plane clusters, each cluster representing a structure;

[0013] Step 1.2, each cluster , according to its fitting plane , project the 3D point onto ; First in Using the formula Sure Center Point ,in, The value range of j is 1~n; Any point in along Project the normal vector onto its fitting plane to obtain the projection point Then, the center point As the zero point of the plane, the vector can be regarded as the x-axis of the plane , with the x-axis and The normal vector Orthogonal to the y-axis of the plane = ; All points The 2D plane coordinates ( , ) is calculated by the following formula:

[0014]

[0015] in, Center point The normal vector of

[0016] Step 1.3, construct a two-dimensional image based on the two-dimensional point information; first determine these points and The minimum and maximum values ​​on , , He Lai Indicates that the width of each pixel is , 2D point to 2D pixel coordinates ( , ) is as follows:

[0017]

[0018] If there is a 2D point in the grid, the grayscale value of the pixel is set to 255, otherwise it is 0, and a 2D image is constructed to convert the 3D point cloud into an image;

[0019] Step 1.4, use CannyLines on the image to detect the two-dimensional line segments in the image, and calculate the 3D-2D projection using formula (5).

[0020]

[0021] The endpoint coordinates of a 2D line segment ( , ) are sequentially projected into three-dimensional point coordinates ( , , ), project the two-dimensional lines into three-dimensional line segments and extract the indoor structure lines; through distance transformation, the original data are divided into edge points and non-edge points.

[0022] Furthermore, the specific process of step 2 is as follows:

[0023] Step 2.1, based on a given initial resolution , use the TBPS supervoxel segmentation method to achieve supervoxel segmentation of point clouds and generate supervoxel sets;

[0024] Step 2.2, for each supervoxel , through the formula , calculate the center point of the supervoxel , seek The 3D point about the center point The covariance matrix of , and , where the eigenvalue ;like The eigenvalues ​​of satisfy formulas (6) and (7), and the supervoxel Storing a planar supervoxel set ; Otherwise, store in the surface supervoxel set ;

[0025]

[0026] Step 2.3, based on current resolution , Repeat the TBPS supervoxel segmentation at the midpoint to generate a new supervoxel set; each supervoxel is further classified into a plane supervoxel and a curved surface supervoxel, and the plane supervoxel is stored in the plane supervoxel set , the surface supervoxel is stored in the new point set ;

[0027] Step 2.4, for the new collection , re-execute step 2.2;

[0028] Step 2.5, if completed, If it is not empty, repeat 2.3 and 2.4, and repeat the iteration until Empty or Less than a given threshold , get the plane supervoxel set .

[0029] (III) Beneficial effects

[0030] It can be seen from the above technical solutions that the present invention provides a simple and effective indoor supervoxel segmentation technology, which converts indoor three-dimensional point clouds into supervoxel sets on the basis of ensuring that supervoxels do not cross indoor structures. Existing supervoxel segmentation methods focus on outdoor scenes rather than indoor scenes. Indoor scenes are highly closed, with large amounts of data and severe data occlusion. In existing methods, supervoxel segmentation is affected by data occlusion, and supervoxels often cross structural boundaries. The present invention takes indoor structural line extraction as a prerequisite, and constructs a multi-resolution supervoxel segmentation method suitable for indoor scenes, which can provide a good data prerequisite for subsequent indoor structure extraction and indoor modeling, and provide a technical prerequisite for rapid processing of indoor three-dimensional point clouds. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Schematic diagram of a flow chart of an embodiment of the present invention.

[0032] Figure 2 : is the original point cloud image of the embodiment of the present invention.

[0033] Figure 3 Schematic diagram of boundary points according to an embodiment of the present invention.

[0034] Figure 4 FIG. 4 is a schematic diagram of a supervoxel segmentation result according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] Below, the present invention is further described in conjunction with the accompanying drawings.

[0036] Reference Figures 1 to 4 , a method for indoor supervoxel segmentation taking into account curved surface structures, the specific process is as follows:

[0037] Step 1: After inputting the indoor 3D point cloud, firstly extract the edge points of the indoor structure based on the 2D-3D reprojection method, and divide the original data into edge points and non-edge points, including the following sub-steps:

[0038] Step 1.1: pre-segment the indoor point cloud based on Random Sample Consensus (RANSAC), and divide the indoor point cloud into plane clusters, each cluster representing a structure;

[0039] Step 1.2, each cluster , according to its fitting plane , project the 3D point onto First, Using the formula Sure Center Point ,Will Any point in along Project the normal vector onto its fitting plane to obtain the projection point Then, the center point As the zero point of the plane, the vector can be regarded as the x-axis of the plane , with the x-axis and The normal vector Orthogonal to the y-axis of the plane = . All points The 2D plane coordinates ( , ) can be calculated by the following formula:

[0040]

[0041] in, Center point The normal vector of

[0042] Step 1.3, construct a two-dimensional image based on the two-dimensional point information. First, determine the points in and The minimum and maximum values ​​on , , He Lai Indicates that the width of each pixel is , 2D point to 2D pixel coordinates ( , ) is as follows:

[0043]

[0044] If there is a 2D point in the grid, the grayscale value of the pixel is set to 255, otherwise it is 0, and a 2D image is constructed to convert the 3D point cloud into an image;

[0045] Step 1.4, use CannyLines to detect the two-dimensional line segments in the image, and calculate the 3D-2D projection and the endpoint coordinates of the two-dimensional line segments using Formula 5 ( , ) are sequentially projected into three-dimensional point coordinates ( , , ), project the 2D lines into 3D line segments and extract the indoor structure lines. Through distance transformation, the original data are divided into edge points and non-edge points.

[0046]

[0047] Step 2, based on the improved TBPS supervoxel segmentation method, extracting planar supervoxels at non-edge points, including the following sub-steps:

[0048] Step 2.1, based on a given initial resolution , use the TBPS supervoxel segmentation method to achieve supervoxel segmentation of point clouds and generate supervoxel sets;

[0049] Step 2.2, for each supervoxel , using the formula , calculate the center point of the supervoxel , seek The 3D point about the center point The covariance matrix of , and , where the eigenvalue .like The eigenvalues ​​of satisfy formulas 6 and 7, and the supervoxel Storing a planar supervoxel set ; Otherwise, store in the surface supervoxel set ;

[0050]

[0051] Step 2.3, based on current resolution , Repeat the TBPS supervoxel segmentation at the midpoint to generate a new supervoxel set; each supervoxel is further classified into a plane supervoxel and a curved surface supervoxel, and the plane supervoxel is stored in the plane supervoxel set , the surface supervoxel is stored in the new point set ;

[0052] Step 2.4, for the new collection , re-execute step 2.2;

[0053] Step 2.5, if completed, If it is not empty, repeat 2.3 and 2.4, and repeat the iteration until Empty or Less than a given threshold , get the plane supervoxel set ;

[0054] Step 3: Based on the VCCS supervoxel segmentation method, surface supervoxels are extracted at the edge points and merged with the plane supervoxels obtained in step 2 to achieve indoor supervoxel extraction.

[0055] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0056] Moreover, the shapes and sizes of the components in the figures do not reflect the real size and proportion, but only illustrate the contents of the embodiments of the present disclosure. In addition, in the claims, any reference symbols between brackets shall not be constructed as limiting the claims.

[0057] The ordinal numbers used in the specification and claims, such as "first", "second", "third", etc., to modify the corresponding elements, do not themselves mean that the elements have any ordinal numbers, nor do they represent the order of one element and another element, or the order in the manufacturing method. The use of these ordinal numbers is only used to clearly distinguish a component with a certain name from another component with the same name.

Claims

1. A method for indoor supervoxel segmentation taking into account curved surface structures, characterized in that: The specific steps include: Step 1: After inputting the indoor 3D point cloud, the edge points of the indoor structure are extracted based on the 2D-3D reprojection method, and the original data are divided into edge points and non-edge points; Step 2: extracting planar supervoxels at non-edge points based on the improved TBPS supervoxel segmentation method; Step 3: Based on the VCCS supervoxel segmentation method, surface supervoxels are extracted at the edge points and merged with the plane supervoxels obtained in step 2 to achieve indoor supervoxel extraction.

2. The method for indoor supervoxel segmentation taking into account curved surface structures according to claim 1, characterized in that: The specific process of step 1 is as follows: Step 1.1: pre-segment the indoor point cloud based on random sampling consistency, and divide the indoor point cloud into plane clusters, each cluster representing a structure; Step 1.2, each cluster , according to its fitting plane , project the 3D point onto ; First in Using the formula Sure Center Point ,in, The value range of j is 1~n; Any point in along Project the normal vector onto its fitting plane to obtain the projection point Then, the center point As the zero point of the plane, the vector can be regarded as the x-axis of the plane , with the x-axis and The normal vector Orthogonal to the y-axis of the plane = ; All points The 2D plane coordinates ( , ) is calculated by the following formula: in, Center point The normal vector of Step 1.3, construct a two-dimensional image based on the two-dimensional point information; first determine these points and The minimum and maximum values ​​on , , He Lai Indicates that the width of each pixel is , 2D point to 2D pixel coordinates ( , ) is as follows: If there is a 2D point in the grid, the grayscale value of the pixel is set to 255, otherwise it is 0, and a 2D image is constructed to convert the 3D point cloud into an image; Step 1.4, use CannyLines on the image to detect the two-dimensional line segments in the image, and calculate the 3D-2D projection using formula (5). The endpoint coordinates of a 2D line segment ( , ) are sequentially projected into three-dimensional point coordinates ( , , ), project the two-dimensional lines into three-dimensional line segments and extract the indoor structure lines; through distance transformation, the original data are divided into edge points and non-edge points.

3. The method for indoor supervoxel segmentation taking into account curved surface structures according to claim 1, characterized in that: The specific process of step 2 is as follows: Step 2.1, based on a given initial resolution , use the TBPS supervoxel segmentation method to achieve supervoxel segmentation of point clouds and generate supervoxel sets; Step 2.2, for each supervoxel , through the formula , calculate the center point of the supervoxel , seek The 3D point about the center point The covariance matrix of , and , where the eigenvalue ;like The eigenvalues ​​of satisfy formulas (6) and (7), and the supervoxel Storing a planar supervoxel set ; Otherwise, store in the surface supervoxel set ; Step 2.3, based on current resolution , Repeat the TBPS supervoxel segmentation at the midpoint to generate a new supervoxel set; each supervoxel is further classified into a plane supervoxel and a curved surface supervoxel, and the plane supervoxel is stored in the plane supervoxel set , the surface supervoxels are stored in the new point set #imgpt63#; Step 2.4, re-execute step 2.2 for the new set #imgpt64#; Step 2.5, if the execution is completed and #imgpt65# is not empty, repeat 2.3 and 2.4, and repeat the iteration until #imgpt66# is empty or #imgpt67# is less than the given threshold #imgpt68#, and obtain the planar supervoxel set #imgpt69#.

Images