A wall structure analysis and reconstruction method for indoor point cloud scenes
Through occlusion detection and principal component analysis, combined with scanning light and KNN algorithm, the data loss and door state recognition problems in wall reconstruction in indoor point cloud scenarios are solved, and a more accurate and stable wall structure and door state reconstruction is achieved.
Patent Information
- Application Number
- CN202110673015.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-06-17
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2041-06-17
AI Technical Summary
The prior art is difficult to effectively reconstruct a complete wall model in indoor point cloud scenarios, especially due to the shading and diversity of doors and windows, resulting in data loss and neglected door state.
Through occlusion detection, partitioning of wall data and door recognition, three-dimensional voxel representation, scanning light construction, KNN algorithm, principal component analysis method and corner point detection are used to identify and reconstruct the status of the wall structure and door.
Improves the accuracy and robustness of wall reconstruction, can effectively deal with data loss caused by occlusion, and accurately identify and rebuild wall models that include closing and opening doors.
Smart Images

Figure CN113409448B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of computer vision and image processing methods, and relates to a wall structure analysis and reconstruction method for an indoor point cloud scene. Background Art
[0002] Various scenes in the real world are the most important components of human perception of the world. Among them, indoor scenes are an important component of real scenes, and the establishment of their three-dimensional models has broad application prospects.
[0003] In indoor scenes, walls are important elements, and the presence of detailed information such as doors and windows directly affects the realism of the wall model. At the same time, the occlusion of walls by objects in the scene and the diversity of wall structures such as doors and windows also make wall reconstruction a huge challenge. In the current research on wall reconstruction, there are few methods that can reconstruct a complete wall model. Most of them only focus on a certain wall structure, and some only focus on doors in a specific state, while ignoring doors in other states. Summary of the invention
[0004] The purpose of the present invention is to provide a wall structure analysis and reconstruction method for indoor point cloud scenes, which solves the problems of data loss due to occlusion and neglect of door status in the prior art.
[0005] The technical solution adopted by the present invention is a method for analyzing and reconstructing the wall structure of an indoor point cloud scene, which is specifically implemented according to the following steps:
[0006] Step 1, perform occlusion detection on the indoor scene to determine the occlusion area on the wall in the scene;
[0007] Step 2, extracting wall data in the occluded area obtained in step 1, and dividing the wall structure according to local features;
[0008] Step 3, identifying the door based on the wall data processed in step 2;
[0009] Step 4, determine the door status of the wall data processed in step 3 and reconstruct it.
[0010] The present invention is also characterized in that
[0011] Step 1 is as follows:
[0012] Step 1.1, set the voxel size to 10cm×10cm×10cm, represent the indoor scene with three-dimensional voxels, and then use the centroid of all points in each voxel to approximate the other points in the voxel, that is, all points in the voxel are finally represented by a centroid point, and finally all voxels are processed to obtain the downsampled point cloud scene, which is used as the input data for the next stage;
[0013] Step 1.2, determine the scanning position, scanning radius and scanning point, so as to construct the scanning light for detecting occlusion, specifically:
[0014] Step 1.2.1, determining the scanning position O according to the occluded area and the angle of the object causing the occlusion;
[0015] Step 1.2.2, calculate the scanning radius R scan , specifically:
[0016]
[0017] Vol=L x L y L z (2)
[0018] Where γ is the scale factor; Vol is the volume of the indoor scene; N is the number of points in the indoor scene; L x L is the length of the indoor scene in the X-axis direction; y is the length of the indoor scene in the Y-axis direction; L z is the length of the indoor scene in the Z-axis direction;
[0019] Step 1.2.3, determine the scanning point P scan , specifically:
[0020] According to the scanning radius R scan , divide the wall in the point cloud data into multiple scanning areas with a length of L x , height is L z The wall is divided into m×l scanning areas; the length is L y , height is L z The wall is divided into n×l scanning areas, where m=ceil(L x / (2×R scan )), n = ceil (L y / (2×R scan )), l = ceil(L z / (2×R scan )), ceil(x) is the rounding function, and then the center point of each scanning area is calculated, and the center point is the scanning point;
[0021] Step 1.2.4, for the wall parallel to XOZ, with the Z-axis coordinate as the reference, specify the order for the scanning points from the minimum value of the X coordinate to the maximum value of the X coordinate, expressed as
[0022] For walls parallel to YOZ, the order of the scan points is specified from the minimum value of the Y coordinate to the maximum value of the Y coordinate based on the Z-axis coordinate, expressed as
[0023] Step 1.2.5, after determining the scanning position, scanning radius and scanning point, a ray is emitted from the scanning position O to each scanning point, thereby constructing a scanning ray;
[0024] Step 1.3, based on the scanning rays constructed in step 1.2, use the ray tracing method to determine the occluded area on the wall in the scene.
[0025] Step 1.3 is as follows:
[0026] Step 1.3.1, get the unsearched scan points in the order specified in step 1.2
[0027] Step 1.3.2, apply KNN algorithm to search for scanning points Whether there are neighboring points within the scanning radius. If there are neighboring points, it means that the area is not an occluded area, and jump to step 1.3.1. Otherwise, it means that the area may be occluded, and jump to step 1.3.3;
[0028] Step 1.3.3, determine the scanning position O and the scanning point according to the two-point three-dimensional straight line equation (3) Substitute formula (4) into formula (3) to calculate the new scanning point (x, y, z). If the new scanning point is not between the scanning point and the scanning position, it means that the area is not an occluded area, and jump to step 1.3.1, otherwise jump to step 1.3.4;
[0029]
[0030]
[0031] Where: Scan point The coordinates of (o x ,o y ,o z ) is the coordinate of the scanning position O; R scan is the scanning radius;
[0032] Step 1.3.4, use the KNN algorithm to search whether there are neighboring points within the scanning radius of the scanning point. If there are neighboring points, it means that the area is an occluded area and jump to step 1.3.1, otherwise jump to step 1.3.3;
[0033] Step 1.3.5, until all scanning points have been searched, so as to determine all occluded areas on the walls in the scene and mark the empty areas caused by occlusion.
[0034] Step 2 is as follows:
[0035] Step 2.1, using the original point cloud scene data as input data to further process the wall data in the indoor scene;
[0036] Step 2.2, use principal component analysis to calculate features;
[0037] Step 2.3, divide the wall structure according to local features.
[0038] Step 2.1 is as follows:
[0039] Step 2.1.1, randomly select three points in the original point cloud scene data as local points and fit the plane;
[0040] Step 2.1.2, set the distance threshold, determine whether the points within the distance threshold around the plane fitted in step 2.1.1 are in-plane points, if yes, add these points to the plane, and update the plane;
[0041] Step 2.1.3, repeat the operation of step 2.1.2 for the updated plane until no more in-zone points are added to the plane. If the number of in-zone points of the plane is greater than the threshold of the number of in-zone points, the plane is output;
[0042] Step 2.1.4, extract the wall data from the plane output by step 2.1.3, specifically: first, determine the height H of the entire scan scene whole ; Then, since the wall is the structure that supports the entire scene, its height is the same as the scene height H whole The height of the door in the scene is generally 3 / 4 of the height of the wall. Therefore, based on this prior knowledge, the height threshold H for extracting the wall is set. wall is (H whole ×3 / 4,H whole ]; Finally, the planes are filtered according to the height threshold, and the height of each plane is calculated. If the height of the plane is within the height threshold, the plane is extracted to obtain the wall data.
[0043] Step 2.2 is as follows:
[0044] Step 2.2.1, establish a KD-tree to organize all points in the wall point cloud data extracted in step 2.1, and then for each point p, use KD-tree to query its k nearest neighbor points and use formula (5) to calculate the covariance matrix:
[0045] Where: ∑ is the 3x3 covariance matrix; P i is the i-th neighbor point of point p;
[0046] Step 2.2.2, solve the standard eigenvalue equation by singular value decomposition to obtain the normal vector of point p. The normal vector is the third eigenvector in the eigenvector matrix V, denoted by n p , represents the normal vector of point p;
[0047] λV=∑V (6)
[0048] Where: V is the eigenvector matrix; λ is the eigenvalue matrix;
[0049] Step 2.2.3, in the matrix of λ, λ3 represents the curvature value of the point, so the curvature value of point p is λ3, and it is recorded as λ p .
[0050] Step 2.3 is as follows:
[0051] Step 2.3.1, define a plane mark for each point in the wall point cloud data. The initial mark is 0, that is, Index = 0, indicating that the point is not marked. If Index = 1, it means that the point has been marked;
[0052] Step 2.2.2, create a stack to store points;
[0053] Step 2.3.3, select the point with the smallest curvature as the seed point, mark the seed point as Index = 1, and then store the seed point in the stack;
[0054] Step 2.3.4, if the stack is not empty, take out the seed point at the top of the stack, calculate the neighboring points of the seed point, and determine the cosine value cos of the angle between the normal vectors of the seed point and the neighboring points;
[0055] Step 2.3.5, if 1-cos<0.1, determine whether the curvature of the neighboring point is less than the set curvature threshold. If so, it is considered to be a point in the same plane. If the neighboring point is not marked, mark the neighboring point as Index=1 and push it into the stack; otherwise, jump to step 2.3.4;
[0056] Step 2.3.6, if the stack is empty, it means that a plane has been successfully extracted, and the marked points are deleted from the wall point cloud data, and then steps 2.3.3-2.3.5 are repeated for the remaining unmarked points until there are no unmarked points, and the plane segmentation of the wall is completed;
[0057] Step 2.3.7, after obtaining the different planes on the wall determined by steps 2.3.1-2.3.6, determine the side length L of the unit grid for dividing the wall according to formula (7);
[0058]
[0059] Where: β is the scale factor used to adjust the side length of the small grid; S is the area of the wall; N is the total number of wall points;
[0060] Step 2.3.8, calculate the number of grids divided by the wall according to formula (8): grid ;
[0061] num grid =num w ×num h (8)
[0062] num w = l w / L (9)
[0063] num h = l h / L (10)
[0064] Where: num grid is the number of grids in the wall; num w The number of grids occupied by the wall width; num h is the number of grids occupied by the wall height; l w is the width of the wall; l h is the height of the wall.
[0065] Step 2.3.9, calculate the number of points in each grid num ;
[0066] Step 2.3.10, if grid num = 0, determine whether the grid is occluded. If so, mark the grid as occluded and display it in blue. Otherwise, mark the grid as empty and display it in red.
[0067] Step 2.3.11, for grid numFor grids whose value is not equal to 0, the RGB information of the midpoint of each grid is used to represent the corresponding grid, and the number of grids is counted according to the RGB information of the grid, and they are sorted in descending order. The grids with the largest number are marked as walls and displayed in black. If there is another color besides walls, and the number of such grids is within the given range, the grid is marked as a candidate door, thereby realizing the division of the wall structure.
[0068] Step 3 is as follows:
[0069] Step 3.1, delete the segmentation planes whose points in the wall structure divided in step 2 are less than the quantity threshold, and then analyze the parallel relationship between the remaining wall segmentation planes and the XOZ coordinate axis. For the planes parallel to the coordinate axis, use the AABB bounding box to extract the corner points of the plane; for the planes not parallel to the coordinate axis, use the OBB bounding box to extract the corner points of the plane;
[0070] Step 3.2, calculate the volume V of each plane based on the corner point information of the bounding box box , where the plane using AABB bounding box for corner detection is calculated using formula (11), and the plane using OBB bounding box for corner detection is calculated using formula (12), and the plane with the largest volume is marked as a wall;
[0071] V box =(max_AABB x -min_AABB x )×(max_AABB y -min_AABB y )×(max_AABB z -min_AABB x ) (11)
[0072] V box =l1×l2×l1 (12)
[0073] Where: (max_AABB x ,min_AABB x ) is the maximum and minimum value of AABB on the x-axis; (max_AABB y ,min_AABB y ) is the maximum and minimum value of AABB on the y-axis; (max_AABB z ,min_AABB z ) is the maximum and minimum value of AABB on the z-axis; l1, l2, l3 are the lengths of the three axes of OBB;
[0074] Step 3.3, calculate the height of each plane, where the wall height is recorded as l wall, the heights of the other planes are denoted as l1, l2, ...;
[0075] Step 3.4, analyze the height ratio of each plane to the wall. If the height of a plane is within 3 / 4 of the wall height, then Then mark the plane as a door, otherwise mark it as other objects, where The fluctuation range.
[0076] Step 4 is as follows:
[0077] Step 4.1, using the angle between the normal vectors of the points in each plane determined in step 3 to extract the boundary;
[0078] Step 4.2, projecting the boundary points obtained in step 4.1, and fitting straight lines to these projected points;
[0079] Step 4.3, calculating the angle formed by the door and the wall according to the straight lines fitting each plane obtained in step 4.2, and analyzing the state of the door;
[0080] Step 4.4, based on the state of the door determined in step 4.3 and the corner point information of each plane obtained in step 3.1, up-sample and fit the plane to achieve reconstruction of the wall.
[0081] Step 4.1 is as follows:
[0082] Step 4.1.1, using principal component analysis (PCA) to extract the normal vectors of the points in each plane determined in step 3;
[0083] Step 4.1.2, for each point p i and its k nearest neighbors N j The point set X (j=0,1,…,k-1) i To its normal vector n i Projection is performed on the plane perpendicular to it, and the projection point set is X′ i ={(x′ i ,y′ i ,z′ i )|i=0,1,…t}, t is the number of projection points;
[0084] Step 4.1.3, with p i Projection point p′ i As the starting point, its neighboring point N j Projection point N′ j As the end point, define the vector p′ i N′ j (j=0,1,…,k-1), and find each vector p′ i N′ jThe cross product with the plane normal vector, that is, n i ×p′ i N′ j , where the result of the cross product of two vectors is a vector that is perpendicular to the sum of the two vectors;
[0085] Step 4.1.4, take any projection vector p′ i N′ j , find the angle α between other projection vectors and this vector j , and the cross product n with this vector i ×p′ i N′ j The angle β j , which stipulates that if β j ≥90°, then α j =360°-α j ;
[0086] Step 4.1.5, apply quick sort method to α j (j=0,1,…,k-1) are sorted so that they are ordered from large to small. Then, the angle between adjacent vectors can be expressed by formula (13). Then, it is estimated whether the maximum angle between adjacent vectors is greater than the angle threshold ε. If it is greater than the angle threshold, the point is a boundary point.
[0087]
[0088] Step 4.1.6, continue to search for the next boundary point until all boundary points of each plane are found;
[0089] Step 4.2 is as follows:
[0090] Step 4.2.1, determining the maximum and minimum values of the z-axis of each plane point cloud data obtained in step 3;
[0091] Step 4.2.2, take the average value of the z-axis as the projection plane;
[0092] Step 4.2.3, projecting the boundary points of the wall and the door obtained in step 4.1 onto the XOY plane to obtain the projection points of the point cloud data of the wall and the door;
[0093] Step 4.2.4, fitting a straight line based on these projection points by the least squares method;
[0094] Step 4.3 is as follows:
[0095] Step 4.3.1, obtain the equations of the two straight lines of the wall and the door fitted in step 4.2, and use formula (14) to calculate the intersection point ζ(x 12 ,y 12 );
[0096]
[0097] Step 4.3.2, divide the point cloud data used to fit the two straight lines into 10 segments, analyze the point density of each segment, if the difference between the point density of a segment and the point density of other segments is greater than the density threshold, the point density of this segment is significantly higher than that of other segments, if the point density of a segment is significantly higher than that of other segments, take the average value of x of all points in this segment, and get the y coordinate according to its straight line equation, otherwise, if the point density of each segment is uniform, randomly select a point of the straight line, and record the coordinate obtained in this step as θ(x θ ,y θ );
[0098] Step 4.3.3, according to the intersection point and coordinate θ(x θ ,y θ ), use formula (15) to calculate the slope k of the straight line, and calculate the slopes of the two straight lines as k1 and k2 respectively;
[0099]
[0100] Step 4.3.4, calculate the angle between the two straight lines according to the slopes k1 and k2. If k1·k2=-1, the angle between the two straight lines is 90°. Otherwise, calculate the tangent value tanλ of the two straight lines according to formula (16);
[0101]
[0102] Step 4.3.5, if k1·k2≠-1, calculate λ according to the inverse tangent function arctan. If λ is a negative number, the angle between the straight lines is π+λ, otherwise the angle between the straight lines is λ;
[0103] Step 4.3.6, judging the state of the door according to the obtained straight line angle, if the angle between the two straight lines is greater than or equal to 90°, it is an open door; if the angle between the two straight lines is in the range of [0°, 5°), it is a closed door; if the angle between the two straight lines is in the range of [5°, 90°), it is a half-open door;
[0104] Step 4.4 is as follows:
[0105] Step 4.4.1, based on the corner point information obtained in step 3.1, select the longest edge L in the corner point connection as the benchmark and set the required sampling number N up Then, the sampling distance, i.e. the distance between the sampling points, is calculated according to formula (17);
[0106]
[0107] Where: dis up——sampling distance; L——the longest side in the corner connection; N up ——Number of samples.
[0108] Step 4.4.2, starting from the point with the smallest coordinate in the corner point information obtained in step 3.1, according to the sampling distance dis up Output points in sequence to achieve data upsampling;
[0109] Step 4.4.3, use the random sampling consistency method to fit the plane, that is, randomly select three points on the upsampled data, determine a plane, add the points within the set distance of this plane to the plane, update the plane until all points are processed, complete the plane fitting, and realize the reconstruction of the wall.
[0110] The beneficial effects of the present invention are:
[0111] The present invention solves the problem of data loss caused by occlusion and improves the accuracy of reconstruction results. The wall reconstruction work of the present invention has greatly improved effectiveness, stability and accuracy, and has good robustness, enriching the method system of computer graphics and visual intelligence.
[0112] The present invention first divides the wall structure through occlusion detection and plane segmentation, effectively solving the data missing problem caused by occlusion and improving the accuracy of dividing the wall structure; then uses a door recognition method based on corner point detection to recognize the door elements on the wall with high accuracy; finally, according to the door state recognition and reconstruction method based on boundary extraction and straight line fitting, the state of the door is accurately analyzed, and the wall containing closed and open doors is completely reconstructed, and the final reconstruction effect is good. BRIEF DESCRIPTION OF THE DRAWINGS
[0113] Figure 1 The original data input in step 1 of the method embodiment of the present invention;
[0114] Figure 2 is the downsampling result obtained in step 1 of an embodiment of a method for analyzing and reconstructing the wall structure of an indoor point cloud scene of the present invention;
[0115] Figure 3 is the scanning light result constructed in step 1 of an embodiment of a method for analyzing and reconstructing the wall structure of an indoor point cloud scene of the present invention;
[0116] Figure 4 The data of the occlusion marking result obtained in step 1 of an embodiment of a method for analyzing and reconstructing the wall structure of an indoor point cloud scene of the present invention;
[0117] Figure 5is the scene segmentation result obtained in step 2 of an embodiment of a method for analyzing and reconstructing the wall structure of an indoor point cloud scene of the present invention;
[0118] Figure 6 The wall extraction data obtained in step 2 of an embodiment of a method for analyzing and reconstructing wall structure of an indoor point cloud scene of the present invention;
[0119] Figure 7 The wall structure division data obtained in step 2 of an embodiment of a method for analyzing and reconstructing wall structure of an indoor point cloud scene of the present invention;
[0120] Figure 8 is the corner point result of the point cloud data obtained in step 3 of an embodiment of a method for analyzing and reconstructing the wall structure of an indoor point cloud scene of the present invention;
[0121] Fig. 9 It is the point cloud data boundary point result obtained in step 4 of an embodiment of a method for analyzing and reconstructing the wall structure of an indoor point cloud scene of the present invention;
[0122] Fig.10 It is the result of fitting a straight line based on the projection points in step 4 of an embodiment of a method for analyzing and reconstructing the wall structure of an indoor point cloud scene of the present invention;
[0123] Fig.11 It is the wall reconstruction result in step 4 of an embodiment of a method for analyzing and reconstructing wall structure of an indoor point cloud scene of the present invention. DETAILED DESCRIPTION
[0124] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.
[0125] The present invention is a wall structure analysis and reconstruction method for indoor point cloud scenes, which is specifically implemented according to the following steps:
[0126] Step 1: Perform occlusion detection on the indoor scene to determine the occlusion area on the wall in the scene; specifically:
[0127] Step 1.1, set the voxel size to 10cm×10cm×10cm, and represent the indoor scene with 3D voxels, such as Figure 1 As shown, the centroid of all points in each voxel is then used to approximate other points in the voxel, that is, all points in the voxel are ultimately represented by a centroid point, and finally all voxels are processed to obtain a downsampled point cloud scene, as shown in Figure 2 As shown, and use it as input data for the next stage;
[0128] Step 1.2, determine the scanning position, scanning radius and scanning point, so as to construct the scanning light for detecting occlusion, such as Figure 3As shown, specifically:
[0129] Step 1.2.1, determining the scanning position O according to the occluded area and the angle of the object causing the occlusion;
[0130] Step 1.2.2, calculate the scanning radius R scan If the scanning radius is too large, the detection occlusion result will be inaccurate; if the scanning radius is too small, the detection efficiency will be reduced, and in serious cases, the program will crash. Therefore, this paper calculates the scanning radius R according to the spatial point density. scan . The scanning radius R scan The value of is taken to be smaller to ensure that the detection is carried out within the most appropriate range and reduce the detection time. Therefore, the scanning radius R is calculated scan , specifically:
[0131]
[0132] Vol=L x L y L z (2)
[0133] Where γ is the scale factor; Vol is the volume of the indoor scene; N is the number of points in the indoor scene; L x L is the length of the indoor scene in the X-axis direction; y is the length of the indoor scene in the Y-axis direction; L z is the length of the indoor scene in the Z-axis direction;
[0134] Step 1.2.3, determine the scanning point P scan , specifically:
[0135] According to the scanning radius R scan , divide the wall in the point cloud data into multiple scanning areas with a length of L x , height is L z The wall is divided into m×l scanning areas; the length is L y , height is L z The wall is divided into n×l scanning areas, where m=ceil(L x / (2×R scan )), n = ceil (L y / (2×R scan )), l = ceil(L z / (2×R scan )), ceil(x) is the rounding function, and then the center point of each scanning area is calculated, and the center point is the scanning point;
[0136] Specify the order of scanning points to facilitate the subsequent experiments. Since most walls are perpendicular to the ground and adjacent walls are perpendicular, the order of scanning points is divided into two categories.
[0137] Step 1.2.4, for the wall parallel to XOZ, with the Z-axis coordinate as the reference, specify the order for the scanning points from the minimum value of the X coordinate to the maximum value of the X coordinate, expressed as
[0138] For walls parallel to YOZ, the order of the scan points is specified from the minimum value of the Y coordinate to the maximum value of the Y coordinate based on the Z-axis coordinate, expressed as
[0139] Step 1.2.5, after determining the scanning position, scanning radius and scanning point, a ray is emitted from the scanning position O to each scanning point, thereby constructing a scanning ray;
[0140] Step 1.3, based on the scanning rays constructed in step 1.2, use the ray tracing method to determine the occluded area on the wall in the scene, such as Figure 4 As shown, specifically:
[0141] Step 1.3.1, get the unsearched scan points in the order specified in step 1.2
[0142] Step 1.3.2, apply KNN algorithm to search for scanning points Whether there are neighboring points within the scanning radius. If there are neighboring points, it means that the area is not an occluded area, and jump to step 1.3.1. Otherwise, it means that the area may be occluded, and jump to step 1.3.3;
[0143] Step 1.3.3, determine the scanning position O and the scanning point according to the two-point three-dimensional straight line equation (3) Substitute formula (4) into formula (3) to calculate the new scanning point (x, y, z). If the new scanning point is not between the scanning point and the scanning position, it means that the area is not an occluded area, and jump to step 1.3.1, otherwise jump to step 1.3.4;
[0144]
[0145]
[0146] Where: Scan point The coordinates of (o x ,o y ,o z ) is the coordinate of the scanning position O; Rscan is the scanning radius;
[0147] Step 1.3.4, use the KNN algorithm to search whether there are neighboring points within the scanning radius of the scanning point. If there are neighboring points, it means that the area is an occluded area and jump to step 1.3.1, otherwise jump to step 1.3.3;
[0148] Step 1.3.5, until all scanning points have been searched, so as to determine all the occluded areas on the walls in the scene and mark the empty areas caused by the occlusion;
[0149] Step 2, extracting wall data in the occluded area obtained in step 1, and dividing the wall structure according to local features; specifically:
[0150] Step 2.1, using the original point cloud scene data as input data to further process the wall data in the indoor scene; specifically:
[0151] Step 2.1.1, randomly select three points in the original point cloud scene data as local points and fit the plane;
[0152] Step 2.1.2, set the distance threshold, determine whether the points within the distance threshold around the plane fitted in step 2.1.1 are in-plane points, if yes, add these points to the plane, and update the plane;
[0153] Step 2.1.3, repeat the operation of step 2.1.2 for the updated plane until no internal points are added to the plane. If the number of internal points in the plane is greater than the threshold of the number of internal points, the plane is output, such as Figure 5 As shown;
[0154] Step 2.1.4, extract the wall data from the plane output by step 2.1.3, specifically: first, determine the height H of the entire scan scene whole ; Then, since the wall is the structure that supports the entire scene, its height is the same as the scene height H whole The height of the door in the scene is generally 3 / 4 of the height of the wall. Therefore, based on this prior knowledge, the height threshold H for extracting the wall is set. wall is (H whole ×3 / 4,H whole ]; Finally, the planes are filtered according to the height threshold, and the height of each plane is calculated. If the height of the plane is within the height threshold range, the plane is extracted to obtain the wall data, such as Figure 6 As shown;
[0155] Step 2.2, use principal component analysis to calculate features; specifically:
[0156] Step 2.2.1, establish a KD-tree to organize all points in the wall point cloud data extracted in step 2.1, and then for each point p, use KD-tree to query its k nearest neighbor points and use formula (5) to calculate the covariance matrix:
[0157] Where: ∑ is the 3x3 covariance matrix; P i is the i-th neighbor point of point p;
[0158] Step 2.2.2, solve the standard eigenvalue equation by singular value decomposition to obtain the normal vector of point p. The normal vector is the third eigenvector in the eigenvector matrix V, denoted by n p , represents the normal vector of point p;
[0159] λV=∑V (6)
[0160] Where: V is the eigenvector matrix; λ is the eigenvalue matrix;
[0161] Step 2.2.3, in the matrix of λ, λ3 represents the curvature value of the point, so the curvature value of point p is λ3, and it is recorded as λ p ;
[0162] Step 2.3, divide the wall structure according to local features; specifically:
[0163] Step 2.3.1, define a plane mark for each point in the wall point cloud data. The initial mark is 0, that is, Index = 0, indicating that the point is not marked. If Index = 1, it means that the point has been marked;
[0164] Step 2.2.2, create a stack to store points;
[0165] Step 2.3.3, select the point with the smallest curvature as the seed point, mark the seed point as Index = 1, and then store the seed point in the stack;
[0166] Step 2.3.4, if the stack is not empty, take out the seed point at the top of the stack, calculate the neighboring points of the seed point, and determine the cosine value cos of the angle between the normal vectors of the seed point and the neighboring points;
[0167] Step 2.3.5, if 1-cos<0.1, determine whether the curvature of the neighboring point is less than the set curvature threshold. If so, it is considered to be a point in the same plane. If the neighboring point is not marked, mark the neighboring point as Index=1 and push it into the stack; otherwise, jump to step 2.3.4;
[0168] Step 2.3.6, if the stack is empty, it means that a plane has been successfully extracted, and the marked points are deleted from the wall point cloud data, and then steps 2.3.3-2.3.5 are repeated for the remaining unmarked points until there are no unmarked points, and the plane segmentation of the wall is completed;
[0169] Step 2.3.7, after obtaining the different planes on the wall determined by steps 2.3.1-2.3.6, determine the side length L of the unit grid for dividing the wall according to formula (7);
[0170]
[0171] Where: β is the scale factor used to adjust the side length of the small grid; S is the area of the wall; N is the total number of wall points;
[0172] Step 2.3.8, calculate the number of grids divided by the wall according to formula (8): grid ;
[0173] num grid =num w ×num h (8)
[0174] num w = l w / L (9)
[0175] num h = l h / L (10)
[0176] Where: num grid is the number of grids in the wall; num w The number of grids occupied by the wall width; num h is the number of grids occupied by the wall height; l w is the width of the wall; l h is the height of the wall.
[0177] Step 2.3.9, calculate the number of points in each grid num ;
[0178] Step 2.3.10, if grid num = 0, determine whether the grid is occluded. If so, mark the grid as occluded and display it in blue. Otherwise, mark the grid as empty and display it in red.
[0179] Step 2.3.11, for grid num≠0, the corresponding grid is represented by the RGB information of the midpoint of each grid, and the number of each grid is counted according to the RGB information of the grid, sorted in descending order, and the grid with the largest number is marked as a wall and displayed in black; if there is another color other than the wall, and the number of such grids is within the given number range, then the grid is marked as a candidate door, thereby realizing the division of the wall structure, such as Figure 7 shown.
[0180] Step 3, identifying the door based on the wall data processed in step 2; specifically:
[0181] Step 3.1, delete the segmentation planes whose points in the wall structure divided in step 2 are less than the quantity threshold, and then analyze the parallel relationship between the remaining wall segmentation planes and the XOZ coordinate axis. For the planes parallel to the coordinate axis, use the AABB bounding box to extract the corner points of the plane; for the planes not parallel to the coordinate axis, use the OBB bounding box to extract the corner points of the plane, such as Figure 8 As shown;
[0182] Step 3.2, calculate the volume V of each plane based on the corner point information of the bounding box box , where the plane using AABB bounding box for corner detection is calculated using formula (11), and the plane using OBB bounding box for corner detection is calculated using formula (12), and the plane with the largest volume is marked as a wall;
[0183] V box =(max_AABB x -min_AABB x )×(max_AABB y -min_AABB y )×(max_AABB z -min_AABB x ) (11)
[0184] V box =l1×l2×l1 (12)
[0185] Where: (max_AABB x ,min_AABB x ) is the maximum and minimum value of AABB on the x-axis; (max_AABB y ,min_AABB y ) is the maximum and minimum value of AABB on the y-axis; (max_AABB z ,min_AABB z ) is the maximum and minimum value of AABB on the z-axis; l1, l2, l3 are the lengths of the three axes of OBB;
[0186] Step 3.3, calculate the height of each plane, where the wall height is recorded as l wall , the heights of the other planes are denoted as l1, l2, ...;
[0187] Step 3.4, analyze the height ratio of each plane to the wall. If the height of a plane is within 3 / 4 of the wall height, then Then mark the plane as a door, otherwise mark it as other objects, where The fluctuation range.
[0188] Step 4, determine the state of the door of the wall data processed in step 3 and reconstruct it, specifically:
[0189] Step 4.1, using the angle between the normal vectors of the points in each plane determined in step 3 to extract the boundary, such as Fig. 9 As shown, specifically:
[0190] Step 4.1.1, using principal component analysis (PCA) to extract the normal vectors of the points in each plane determined in step 3;
[0191] Step 4.1.2, for each point p i and its k nearest neighbors N j The point set X (j=0,1,…,k-1) i To its normal vector n i Projection is performed on the plane perpendicular to it, and the projection point set is X′ i ={(x′ i ,y′ i ,z′ i )|i=0,1,…t}, t is the number of projection points;
[0192] Step 4.1.3, with p i Projection point p′ i As the starting point, its neighboring point N j Projection point N′ j As the end point, define the vector p′ i N′ j (j=0,1,…,k-1), and find each vector p′ i N′ j The cross product with the plane normal vector, that is, n i ×p′ i N′ j , where the result of the cross product of two vectors is a vector that is perpendicular to the sum of the two vectors;
[0193] Step 4.1.4, take any projection vector p′ i N′ j , find the angle α between other projection vectors and this vectorj , and the cross product n with this vector i ×p′ i N′ j The angle β j , which stipulates that if β j ≥90°, then α j =360°-α j ;
[0194] Step 4.1.5, apply quick sort method to α j (j=0,1,…,k-1) are sorted so that they are ordered from large to small. Then, the angle between adjacent vectors can be expressed by formula (13). Then, it is estimated whether the maximum angle between adjacent vectors is greater than the angle threshold ε. If it is greater than the angle threshold, the point is a boundary point.
[0195]
[0196] Step 4.1.6, continue to search for the next boundary point until all boundary points of each plane are found;
[0197] Step 4.2, project the boundary points obtained in step 4.1, and fit straight lines to these projected points; specifically:
[0198] Step 4.2.1, determining the maximum and minimum values of the z-axis of each plane point cloud data obtained in step 3;
[0199] Step 4.2.2, take the average value of the z-axis as the projection plane;
[0200] Step 4.2.3, projecting the boundary points of the wall and the door obtained in step 4.1 onto the XOY plane to obtain the projection points of the point cloud data of the wall and the door;
[0201] Step 4.2.4, fit a straight line based on these projection points using the least squares method, such as Fig.10 As shown;
[0202] Step 4.3, calculate the angle formed by the door and the wall based on the straight lines fitting each plane obtained in step 4.2, and analyze the state of the door; specifically:
[0203] Step 4.3.1, obtain the equations of the two straight lines of the wall and the door fitted in step 4.2, and use formula (14) to calculate the intersection point ζ(x 12 ,y 12 );
[0204]
[0205] Step 4.3.2, divide the point cloud data used to fit the two straight lines into 10 segments, analyze the point density of each segment, if the difference between the point density of a segment and the point density of other segments is greater than the density threshold, the point density of this segment is significantly higher than that of other segments, if the point density of a segment is significantly higher than that of other segments, take the average value of x of all points in this segment, and get the y coordinate according to its straight line equation, otherwise, if the point density of each segment is uniform, randomly select a point of the straight line, and record the coordinate obtained in this step as θ(x θ ,y θ );
[0206] Step 4.3.3, according to the intersection point and coordinate θ(x θ ,y θ ), use formula (15) to calculate the slope k of the straight line, and calculate the slopes of the two straight lines as k1 and k2 respectively;
[0207]
[0208] Step 4.3.4, calculate the angle between the two straight lines according to the slopes k1 and k2. If k1·k2=-1, the angle between the two straight lines is 90°. Otherwise, calculate the tangent value tanλ of the two straight lines according to formula (16);
[0209]
[0210] Step 4.3.5, if k1·k2≠-1, calculate λ according to the inverse tangent function arctan. If λ is a negative number, the angle between the straight lines is π+λ, otherwise the angle between the straight lines is λ;
[0211] Step 4.3.6, judging the state of the door according to the obtained straight line angle, if the angle between the two straight lines is greater than or equal to 90°, it is an open door; if the angle between the two straight lines is in the range of [0°, 5°), it is a closed door; if the angle between the two straight lines is in the range of [5°, 90°), it is a half-open door;
[0212] Step 4.4, based on the state of the door determined in step 4.3 and the corner point information of each plane obtained in step 3.1, up-sample and fit the plane to achieve the reconstruction of the wall, specifically:
[0213] Step 4.4.1, based on the corner point information obtained in step 3.1, select the longest edge L in the corner point connection as the benchmark and set the required sampling number N up Then, the sampling distance, i.e. the distance between the sampling points, is calculated according to formula (17);
[0214]
[0215] Where: dis up——sampling distance; L——the longest side in the corner connection; N up ——Number of samples.
[0216] Step 4.4.2, starting from the point with the smallest coordinate in the corner point information obtained in step 3.1, according to the sampling distance dis up Output points in sequence to achieve data upsampling;
[0217] Step 4.4.3, use the random sampling consistency method to fit the plane, that is, randomly select three points on the upsampled data, determine a plane, add the points within the set distance of this plane to the plane, update the plane until all points are processed, complete the plane fitting, and realize the reconstruction of the wall, such as Fig.11 shown.
Claims
1. A wall structure analysis and reconstruction method for indoor point cloud scenes, It is characterized in that The specific steps are as follows: Step 1, perform occlusion detection on the indoor scene to determine the occlusion area on the wall in the scene; Step 2, extracting wall data in the occluded area obtained in step 1, and dividing the wall structure according to local features, specifically: Step 2.1: Use the original point cloud scene data as input data to further process the wall data in the indoor scene. Specifically: Step 2.1.1, randomly select three points in the original point cloud scene data as local points and fit the plane; Step 2.1.2, set the distance threshold, determine whether the points within the distance threshold around the plane fitted in step 2.1.1 are in-plane points, if yes, add these points to the plane, and update the plane; Step 2.1.3, repeat the operation of step 2.1.2 for the updated plane until no more in-zone points are added to the plane. If the number of in-zone points of the plane is greater than the threshold of the number of in-zone points, the plane is output; Step 2.1.4, extract the wall data from the plane output by step 2.1.3, specifically: first, determine the height of the entire scan scene ; Then, since the wall is the structure that supports the entire scene, its height is the same as the scene height. The height of the door in the scene is generally 3 / 4 of the height of the wall. Therefore, based on this prior knowledge, the height threshold for extracting the wall is set. for ]; Finally, the planes are filtered according to the height threshold, and the height of each plane is calculated. If the height of the plane is within the height threshold range, the plane is extracted to obtain the wall data; Step 2.2, use principal component analysis to calculate features, specifically: Step 2.2.1, establish a KD-tree to organize all the points in the wall point cloud data extracted in step 2.1, and then for each point , use KD-tree to query its k nearest neighbors, and use formula (5) to calculate the covariance matrix: (5) Where: for The covariance matrix of For point The i-th neighbor point of ; ; Step 2.2.2, solve the standard eigenvalue equation by singular value decomposition and get the point The normal vector is the eigenvector The third eigenvector in the matrix is denoted by , indicating a point The normal vector of (6) Where: is the eigenvector matrix; is the eigenvalue matrix; Step 2.2.3, In the matrix, represents the curvature value of the point, so the curvature value of point p is , and record it as ; Step 2.3, divide the wall structure according to local features, specifically: Step 2.3.1, define a plane mark for each point in the wall point cloud data. The initial mark is 0, that is, Index=0, indicating that the point is not marked. If Index=1, it means that the point has been marked. Step 2.2.2, create a stack to store points; Step 2.3.3, select the point with the smallest curvature as the seed point, mark the seed point as Index=1, and then store the seed point in the stack; Step 2.3.4, if the stack is not empty, take out the seed point at the top of the stack, calculate the neighboring points of the seed point, and determine the cosine value cos of the angle between the normal vectors of the seed point and the neighboring points; Step 2.3.5, if 1-cos<0.1, determine whether the curvature of the neighboring point is less than the set curvature threshold. If it is less than, it is considered to be a point in the same plane. If the neighboring point is not marked, mark the neighboring point as Index=1 and push it into the stack; otherwise, jump to step 2.3.4; Step 2.3.6, if the stack is empty, it means that a plane has been successfully extracted, and the marked points are deleted from the wall point cloud data, and then steps 2.3.3-2.3.5 are repeated for the remaining unmarked points until there are no unmarked points, and the plane segmentation of the wall is completed; Step 2.3.7, after obtaining the different planes on the wall determined by steps 2.3.1-2.3.6, determine the side length L of the unit grid for dividing the wall according to formula (7); (7) Where: is the scale factor used to adjust the side length of the small grid; S is the area of the wall; N is the total number of wall points; Step 2.3.8, calculate the number of grids divided by the wall according to formula (8) ; (8) (9) (10) Where: is the number of grids of the wall; is the number of grids occupied by the wall width; is the number of grids occupied by the wall height; is the width of the wall; is the height of the wall; Step 2.3.9, calculate the number of points in each grid ; Step 2.3.10, if , determine whether the grid is occluded. If so, mark the grid as occluded and display it in blue. Otherwise, mark the grid as empty and display it in red. Step 2.3.11, for The corresponding grid is represented by the RGB information of the midpoint of each grid, and the number of each grid is counted according to the RGB information of the grid, and sorted in descending order. The grid with the largest number is marked as a wall and displayed in black; if there is another color other than the wall, and the number of such grids is within a given range, then the grid is marked as a candidate door, thereby realizing the division of the wall structure; Step 3, identifying the door based on the wall data processed in step 2; Step 4, determine the state of the door of the wall data processed in step 3 and reconstruct it, specifically: Step 4.1, using the angle between the normal vectors of the points in each plane determined in step 3 to extract the boundary, specifically: Step 4.1.1, using principal component analysis (PCA) to extract the normal vectors of the points in each plane determined in step 3; Step 4.1.2, each point and its k nearest neighbors The point set To its normal vector The projection point set is , t is the number of projection points; Step 4.1.3, Projection Point As the starting point, its neighboring points Projection Point As the end point, define the vector , and find each vector The cross product with the plane normal vector is , where the result of the cross product of two vectors is a vector that is perpendicular to the sum of the two vectors; Step 4.1.4, take any projection vector , find the angle between other projection vectors and this vector , and the cross product with this vector Angle , which stipulates that if ,but ; Step 4.1.5, apply quick sort method to Sort them in order from large to small, then the angle between adjacent vectors can be expressed by formula (13), then estimate whether the maximum angle between adjacent vectors is greater than the angle threshold , if it is greater than the angle threshold, the point is a boundary point; (13) Step 4.1.6, continue to search for the next boundary point until all boundary points of each plane are found; Step 4.2, project the boundary points obtained in step 4.1, and fit straight lines to these projected points, specifically: Step 4.2.1, determining the maximum and minimum values of the z-axis of each plane point cloud data obtained in step 3; Step 4.2.2, take the average value of the z-axis as the projection plane; Step 4.2.3, projecting the boundary points of the wall and the door obtained in step 4.1 onto the XOY plane to obtain the projection points of the point cloud data of the wall and the door; Step 4.2.4, fitting a straight line based on these projection points by the least squares method; Step 4.3, calculate the angle between the door and the wall based on the straight lines fitting each plane obtained in step 4.2, and analyze the state of the door, specifically: Step 4.3.1, obtain the equations of the two straight lines of the wall and the door fitted in step 4.2, and use formula (14) to calculate the intersection point. ; (14) Step 4.3.2, divide the point cloud data used to fit the two straight lines into 10 segments, analyze the point density of each segment, if the difference between the point density of a segment and the point density of other segments is greater than the density threshold, then the point density of this segment is significantly higher than that of other segments; if the point density of a segment is significantly higher than that of other segments, then take all the points of this segment The average value of Otherwise, if the point density of each segment is uniform, a point on the line is randomly selected and the coordinates obtained in this step are recorded as ; Step 4.3.3, based on the intersection points and coordinates , use formula (15) to calculate the slope of the straight line , calculate the slopes of the two straight lines respectively , ; (15) Step 4.3.4, according to the slope and , calculate the angle between two straight lines, if , then the straight line angle is , otherwise calculate the tangent value of the two straight lines according to formula (16) ; (16) Step 4.3.5, if , then according to the inverse tangent function calculate ,like If is a negative number, the angle between the lines is , otherwise the straight line angle is ; Step 4.3.6, determine the state of the door based on the obtained straight line angle. If the angle between the two straight lines is greater than or equal to , then it is an open door; if the angle between the two straight lines is If the angle between the two straight lines is within range, then it is a half-open door; Step 4.4, based on the state of the door determined in step 4.3 and the corner point information of each plane obtained in step 3.1, up-sample and fit the plane to achieve the reconstruction of the wall, specifically: Step 4.4.1, based on the corner point information obtained in step 3.1, select the longest edge in the corner point connection As a benchmark, set the number of samples required Then, the sampling distance, i.e. the distance between the sampling points, is calculated according to formula (17); (17) Where: ——sampling distance; ——The longest side among the corner points; - number of samples; Step 4.4.2, starting from the point with the smallest coordinates in the corner point information obtained in step 3.1, according to the sampling distance Output points in sequence to achieve data upsampling; Step 4.4.3, use the random sampling consistency method to fit the plane, that is, randomly select three points on the upsampled data, determine a plane, add the points within the set distance of this plane to the plane, update the plane until all points are processed, complete the plane fitting, and realize the reconstruction of the wall.
2. According to claim 1, a wall structure analysis and reconstruction method for indoor point cloud scenes, It is characterized in that The step 1 is specifically as follows: Step 1.1, set the voxel size to , the indoor scene is represented by three-dimensional voxels, and then the centroid of all points in each voxel is used to approximate the other points in the voxel, that is, all points in the voxel are finally represented by a centroid point, and finally all voxels are processed to obtain the downsampled point cloud scene, which is used as the input data for the next stage; Step 1.2, determine the scanning position, scanning radius and scanning point, so as to construct the scanning light for detecting occlusion, specifically: Step 1.2.1, determine the scanning position based on the occluded area and the angle of the object causing the occlusion ; Step 1.2.2, calculate the scanning radius , specifically: (1) (2) in, is the scale factor; is the volume of the indoor scene; is the number of points in the indoor scene; For indoor scenes Length in the axial direction; For indoor scenes Length in the axial direction; For indoor scenes Length in the axial direction; Step 1.2.3, determine the scanning point , specifically: According to the scanning radius , divide the wall in the point cloud data into multiple scanning areas with a length of , the height is The walls that divide scanning area; length is , the height is The walls that divide scanning area, where , , , is the rounding function, and then the center point of each scanning area is calculated, and the center point is the scanning point; Step 1.2.4, for the wall parallel to XOZ, with the Z-axis coordinate as the reference, specify the order for the scanning points from the minimum value of the X coordinate to the maximum value of the X coordinate, expressed as ; For walls parallel to YOZ, the order of the scan points is specified from the minimum value of the Y coordinate to the maximum value of the Y coordinate based on the Z-axis coordinate, expressed as ; Step 1.2.5: After determining the scanning position, scanning radius and scanning point, A scan ray is constructed by emitting a ray to each scan point; Step 1.3, based on the scanning rays constructed in step 1.2, use the ray tracing method to determine the occluded area on the wall in the scene.
3. According to the wall structure analysis and reconstruction method of an indoor point cloud scene according to claim 2, It is characterized in that The step 1.3 is specifically as follows: Step 1.3.1, get the unsearched scan points in the order specified in step 1.2 , ; Step 1.3.2, apply KNN algorithm to search for scanning points Whether there are neighboring points within the scanning radius. If there are neighboring points, it means that the area is not an occluded area, and jump to step 1.3.
1. Otherwise, it means that the area may be occluded, and jump to step 1.3.3; Step 1.3.3, determine the scanning position according to the two-point three-dimensional straight line equation (3) With scanning point Substitute formula (4) into formula (3) to calculate the new scanning point (x, y, z). If the new scanning point is not between the scanning point and the scanning position, it means that the area is not an occluded area, and jump to step 1.3.1, otherwise jump to step 1.3.4; (3) (4) Where: Scan point The coordinates of Scan location The coordinates of is the scanning radius; Step 1.3.4, use the KNN algorithm to search whether there are neighboring points within the scanning radius of the scanning point. If there are neighboring points, it means that the area is an occluded area and jump to step 1.3.1, otherwise jump to step 1.3.3; Step 1.3.5, until all scanning points have been searched, so as to determine all occluded areas on the walls in the scene and mark the empty areas caused by occlusion.
4. According to claim 3, a wall structure analysis and reconstruction method for indoor point cloud scenes, It is characterized in that The step 3 is specifically as follows: Step 3.1, delete the segmentation planes whose points in the wall structure divided in step 2 are less than the quantity threshold, and then analyze the parallel relationship between the remaining wall segmentation planes and the XOZ coordinate axis. For the planes parallel to the coordinate axis, use the AABB bounding box to extract the corner points of the plane; for the planes not parallel to the coordinate axis, use the OBB bounding box to extract the corner points of the plane; Step 3.2: Calculate the volume of each plane based on the corner point information of the bounding box , where the plane using AABB bounding box for corner detection is calculated using formula (11), and the plane using OBB bounding box for corner detection is calculated using formula (12), and the plane with the largest volume is marked as a wall; (11) (12) Where: ) is the AABB in Maximum and minimum values on the axis; ) is the AABB in Maximum and minimum values on the axis; ) is the AABB in Maximum and minimum values on the axis; is the length of the three axes of OBB; Step 3.3, calculate the height of each plane, where the wall height is recorded as , and the other plane heights are recorded as ; Step 3.4, analyze the height ratio of each plane to the wall. If the height of a plane is within the height of the wall, within the range of , then mark the plane as a door, otherwise mark it as other objects, where The fluctuation range.
