Room layout division method based on unit decomposition and space segmentation
By adopting unit decomposition and spatial segmentation methods in indoor scenarios, combining morphological corrosion and wavefront expansion, the difficulty of room layout division caused by occlusion phenomena in complex indoor scenarios is solved, and the accurate division of room layout and spatial integrity are achieved.
Patent Information
- Application Number
- CN202210327868.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-30
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2042-03-30
AI Technical Summary
When the prior art deals with indoor scenes with complex internal structures, due to the occlusion phenomenon, it is difficult to accurately divide room layout by dividing the structural elements of the indoor scene.
The room layout division method based on unit decomposition and spatial segmentation is adopted. By extracting wall point clouds from indoor point cloud scenes, downsampling and regional growth clustering, the center of mass points are calculated and the straight lines are fitted, combining morphological corrosion and wavefront. The expansion method realizes unit and space segmentation of indoor scenes, and finally determines the room layout through overlap analysis.
It effectively solves the difficulty in dividing room layout caused by occlusion in indoor scenes, realizes the accurate division of room layout, and maintains the geometrical law and spatial integrity of the room.
Smart Images

Figure CN114677388B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of artificial intelligence computing methods and relates to a room layout division method based on unit decomposition and space division. Background Art
[0002] Indoor scenes are the places where people live and work most closely, and people also determine the complexity and diversity of their internal structures. For some indoor scenes with simple internal structures, the accurate division of room layout can be achieved by segmenting the indoor scene structural elements. Even if the acquired indoor scene data is missing, this method still has high robustness. However, in some indoor scenes with complex internal structures, there may be serious occlusion, and the room layout cannot be divided by segmenting the indoor scene structural elements. Summary of the invention
[0003] The purpose of the present invention is to provide a room layout division method based on unit decomposition and space segmentation, which solves the problem in the prior art that when dividing indoor scenes with complex internal structures by dividing indoor scene structural elements, there is serious occlusion phenomenon, making it impossible to achieve division.
[0004] The technical solution adopted by the present invention is a room layout division method based on unit decomposition and space division, which is specifically implemented according to the following steps:
[0005] Step 1: extract wall point cloud from the indoor point cloud scene, then divide the wall point cloud into several segments, calculate the centroid of each segment, fit the straight line corresponding to the wall point cloud according to the centroid, then fuse the fitted straight line, and then perform unit decomposition based on the result of the fusion processing to obtain the unit division result of the indoor scene;
[0006] Step 2: After downsampling the indoor scene, project the downsampled indoor scene onto a two-dimensional plane, discretize the projection points and generate a depth image; then binarize the depth image, use the morphological erosion operation to segment the indoor space to identify the number of rooms; finally, use the wavefront extension method to add the area that belongs to the room area but has not been visited to the room space to obtain the spatial segmentation result of the indoor scene;
[0007] Step 3, overlap the unit division result of the indoor scene obtained in step 1 with the spatial segmentation result of the indoor scene obtained in step 2, generate random points in the overlapping area, and determine which room the grid cell belongs to based on the number of color labels carried by the random points in each grid cell, thereby achieving accurate division of the room layout.
[0008] The present invention is also characterized in that:
[0009] Step 1 is as follows:
[0010] Step 1.1, downsample the indoor scene to obtain the downsampled indoor point cloud scene, use the downsampled indoor point cloud scene as input to segment it into several horizontal point cloud slices, use the region growing clustering method to segment the wall points and indoor debris in each horizontal point cloud slice, and finally retain the wall points to obtain the wall point cloud corresponding to the horizontal point cloud slice;
[0011] Step 1.2, divide each wall point cloud extracted in step 1.1 into several segments, calculate the centroid of each segment, use the centroid to represent the characteristics of all points in each segment, and then perform straight line fitting based on the centroid;
[0012] Step 1.3, project the straight line obtained in step 1.2, perform fusion processing after projection, and then extend the fused line segment, calculate the two-dimensional arrangement data structure according to the CGAL library, and finally obtain the two-dimensional unit decomposition result.
[0013] In step 1.1, the region growing clustering method is used to segment the wall points and indoor debris in each horizontal point cloud slice, and finally the wall points are retained to obtain the wall point cloud corresponding to the horizontal point cloud slice:
[0014] Step 1.1.1, calculate the curvature of each point in each horizontal point cloud slice, sort the points according to their curvature values, find the point with the smallest curvature, and add it to the seed point set;
[0015] Step 1.1.2, for each seed point, if its K nearest neighbor points satisfy both equation (1) and equation (2), then add the point to the potential seed point list;
[0016] ||n p ·n s ||>cos(θ th ) (1)
[0017] Among them, n p Represents the normal vector of the current seed point; n s Represents the normal vector of the K nearest neighbor points of the current seed point; θ th represents a smoothing threshold;
[0018] r p <r th (2)
[0019] Among them, r p Represents the curvature of the K nearest neighbor points of the current seed point, r th represents a curvature threshold;
[0020] Step 1.1.3, remove the point added to the potential seed point list in step 1.1.2 from the slice point cloud corresponding to the initial indoor point cloud;
[0021] Step 1.1.4, clustering is performed on the basis of step 1.1.3, the number of points of the minimum point cluster is set as Min, the number of points of the maximum point cluster is set as Max, all slices with the number of points between Min and Max are retained, and different slices are marked with different colors to distinguish them. The slice point cloud of the retained slices is the obtained wall point cloud;
[0022] Step 1.1.5, repeat steps 1.1.1 to 1.1.4 to complete the clustering of all slices and finally obtain the wall point cloud corresponding to each slice.
[0023] Step 1.2 is as follows:
[0024] Step 1.2.1, input the wall point cloud extracted in step 1.1, and save the x, y, and z coordinates of each point in each wall point cloud;
[0025] Step 1.2.2, calculate the length and width of the wall area in space based on the x and y coordinates of the wall point cloud, where the length is x max -x min , width y max -y min , x max 、x min ,y max ,y min Represent the maximum and minimum x and y coordinates respectively; set the step size l to 0.4m, divide the wall point cloud in space into l×l grids, then the total number of rows is raster_rows=x max -x min / l, the total number of columns is raster_cols=y max -y min / l;
[0026] Step 1.2.3, create a one-dimensional array vector_4, a two-dimensional array col_, and a three-dimensional array row_col. Set the size of the array vector_4 to 4. Then, according to the sizes of raster_rows and raster_cols, set the size of the array col_ to raster_cols·vector_4 and the size of the array row_col to raster_rows·col.
[0027] Step 1.2.4, store the points in the wall point cloud into corresponding rows and columns, where the position of any point (row_idx, col_idx) is expressed as (ceil((points[i].x-point_min.x) / l-1), ceil((points[i].y-point_min.y) / l-1)), point[i].x, point[i].y represent the x and y coordinates of any point, point_min.x, point_min.y represent the x and y coordinates of the minimum point;
[0028] Step 1.2.5, if the value of row_idx or col_idx is less than 0, set it to 0; then store the x, y, and z of the point selected in step 1.2.4 in the array row_col[row_idx][col-idx][0], row_col[row_idx][col-idx][1], and row_col[row_idx][col-idx][2] in sequence, and increase the value of the array row_col[row_idx][col-idx][2] by 1 each time a point is stored;
[0029] Step 1.2.6, repeat steps 1.2.4 and 1.2.5 to divide all points belonging to the wall point cloud into corresponding grid spaces, and calculate the centroid point Q of each grid space. The calculation formula is as shown in formula (3);
[0030]
[0031] Among them, x_mean, y_mean, z_mean represent the x, y, and z coordinates of the centroid point Q respectively; m represents the size of the array row_col; n represents the size of the array row_col[i]; row_xol[i][j][0] represents the x coordinate of any point in the space corresponding to the extracted wall point cloud; row_xol[i][j][1] represents the y coordinate of any point in the space corresponding to the extracted wall point cloud; row_xol[i][j][2] represents the z coordinate of any point in the space corresponding to the extracted wall point cloud; row_col[i][j][3] represents the number of points contained in the space corresponding to the wall point cloud extracted from each grid;
[0032] Step 1.2.7, randomly select three points from the multiple centroid points corresponding to each wall point cloud to construct an xoy plane, calculate the coefficients of the plane equation, and then calculate the normal vector v of the constructed plane. Reconstruct the plane according to the solved plane equation coefficients, and use the endpoints of the line segment in space and the normal vector of the plane to construct a projection line. Solve the intersection of the plane and the projection line again. The intersection is the projection point of one endpoint of the line segment in space. Finally, connect the corresponding projection points in sequence to obtain the projected line segment.
[0033] Step 1.3 is as follows:
[0034] Step 1.3.1, set a predefined length value of a wall segment, then sort the multiple line segments projected in step 1.2.7 from small to large according to their lengths, and directly delete the line segments whose lengths are less than the predefined length value;
[0035] Step 1.3.2, select the line segment with the largest length among the remaining line segments after processing in step 1.3.1 and add it to the final result set, then select a line segment from the remaining line segments to compare with the line segments in the final result set, set an angle threshold and a distance threshold, if the angle between the two line segments is less than the given threshold, then the two line segments are considered to be parallel; on the basis of parallelism, if the distance between the two parallel lines is less than the given threshold, then the two line segments are considered to be collinear, and the two line segments are fused;
[0036] Step 1.3.3, repeat step 1.3.2 until the number of remaining line segments is 0, then end the whole process to obtain the fused line segments;
[0037] In step 1.3.4, the fused line segments are extended, and then the two-dimensional arrangement data structure is calculated according to the CGAL library, and the two-dimensional plane of the indoor scene is divided using the extended line segments to decompose it into two-dimensional units.
[0038] Step 2 is as follows:
[0039] Step 2.1, convert the point cloud of the indoor scene space into a two-dimensional image to obtain the corresponding grayscale image. In order to make the image present an obvious black and white effect, the grayscale image is binarized to obtain a binary image that reflects the overall and local features of the image;
[0040] Step 2.2, by default, all doors leading to other rooms or corridors in each room are closed. Use a 3×3 structural element to iteratively erode the accessible pixels in the binary image obtained in step 2.1. The accessible pixels are white pixels in the binary image. After one erosion, an erosion result image is obtained. The contour on the erosion result image is used to determine whether there is a separable area. If a separable area is obtained, the iteration is stopped and the separable area is marked with different colors.
[0041] Step 2.3, detect the number of rooms according to the separable areas marked in step 2.2. At this time, there are three states on the binary image, where white pixels represent unmarked areas in the accessible area. Then, the white pixels are expanded using the wavefront propagation method to obtain the final room space segmentation result.
[0042] Step 2.1 is as follows:
[0043] Step 2.1.1, project the indoor point cloud after downsampling in step 1.1 onto the xoy plane, find the maximum x, y coordinates and the minimum x, y coordinates from the projected points, and determine the width and height of the image according to the maximum and minimum x, y coordinate values;
[0044] Step 2.1.2, discretize the projection points into a two-dimensional grid to obtain multiple pixel grids, set the pixel size to pixelsize, the size of each pixel is determined by the thickness of the wall and the size and density of the point cloud, and judge whether the pixel is gray or black according to the number of points contained in each pixel. If a pixel contains at least one point, it indicates that the pixel is gray; if a pixel does not contain a point, it indicates that the pixel is black, and then a grayscale image is obtained;
[0045] Step 2.1.3, through appropriate threshold selection, the grayscale value of the pixel on the grayscale image obtained in step 2.1.2 is set to 0 or 255. If the grayscale value of the pixel is greater than or equal to the threshold, it is judged to belong to a specific object, and its grayscale value is represented by 255; otherwise, these pixels are excluded from the object area, and their grayscale value is represented by 0, resulting in a binary image.
[0046] Step 2.2 is as follows:
[0047] Step 2.2.1, starting the first erosion from the center position of the binary image obtained in step 2.1, using a 3×3 structure element to erode accessible pixels in the binary image, where accessible pixels are white pixels in the binary image. After one erosion is completed, a erosion result image will be obtained;
[0048] Step 2.2.2, find the contour in the erosion result image, establish only two hierarchical relationships for all contours, namely the top layer and the inner layer, and count the number of contours found findContourNums;
[0049] Step 2.2.3, check each contour. If the current contour has no corresponding embedded contour, calculate the area contourArea1 enclosed by the current contour, thereby calculating the room area roomArea1. The calculation formula is shown in (4);
[0050] roomArea1=cellSize*cellSize*contourArea1 (4)
[0051] The parameter roomArea1 represents the room area; cellSize represents the size of a cell grid; contourArea1 represents the area of the area enclosed by the current contour;
[0052] Step 2.2.4: Based on step 2.2.3, determine whether the currently detected contour is an embedded contour of other contours. If the current contour is an embedded contour of one of the contours, calculate the area contourArea2 enclosed by the current contour, and update the room area roomArea2 at the same time. The area calculation formula is shown in formula (5);
[0053] roomArea2=cellSize*cellSize*contourArea2-roomArea1 (5)
[0054] The parameter roomArea2 represents the updated room area;
[0055] Step 2.2.5, determine whether the current room area is between the upper threshold and the lower threshold. The lower threshold represents the smallest room area in the data, and the upper threshold represents the largest room area. If the current area is between the upper threshold and the lower threshold, save its contour;
[0056] Step 2.2.6, repeat steps 2.2.1 to 2.2.5, iteratively erode the accessible pixels in the binary image until the area of all remaining regions is less than the lower threshold, then stop the iteration, obtain the separable regions, and then mark the obtained separable regions with different colors.
[0057] Step 2.3 is as follows:
[0058] Step 2.3.1, according to the binary image of step 2.1 combined with the contour of step 2.2, the room segmentation result is obtained, and this segmentation result is used as input. At this time, the room segmentation result image presents three states. The first is black pixels represented by black, that is, the black pixels obtained by step 2.1.2. The area where the black pixels are located represents the inaccessible area, which does not belong to the indoor scene; the second is white pixels represented by white, that is, the white pixels in the binary image obtained by step 2.1.3. The area where the white pixels are located represents the accessible area but has not been visited; the third is the area marked with different colors, that is, the area marked by step 2.2.6. Different colors represent different room areas composed of some pixels;
[0059] Step 2.3.2, select a pixel from the unmarked white pixels, determine the color of the current pixel based on the colors of other pixels in the 3×3 area around the pixel, and determine the channel values of its three channels starting from the pixel. If the channel values of the three channels of the pixel are greater than 250 at the same time, it means that the pixel is an accessible pixel; if the channel values of the three channels of the pixel are not 0 and are less than or equal to 250 at the same time, assign the channel values of the three channels of this pixel to the accessible pixel, realize the marking of the accessible pixel, and expand it to the room space;
[0060] Step 2.3.3, iterate steps 2.3.1 and 2.3.2 until all unmarked areas in the room segmentation result in step 2.3.1 are marked, thus achieving spatial segmentation of the indoor scene.
[0061] Step 3 is as follows:
[0062] Step 3.1, overlapping the unit division result of the indoor scene obtained in step 1 with the spatial segmentation result of the indoor scene obtained in step 2;
[0063] Step 3.2, create a set of random point sets in the overlapping space, add the center point in each unit grid to the random point set, and let the points in each unit grid extract label information from the spatial segmentation result;
[0064] Step 3.3, by calculating the number of points with the same label in each cell grid, and then assigning a label to the cell grid according to the label with the highest frequency in the cell grid, or according to the fact that a marked cell grid is surrounded by the same marked cell grid, then the cell grid is also marked with the same label as the surrounding cell grids;
[0065] In step 3.4, the unit grids of the segmentation result labels completed in steps 3.1 to 3.3 are visualized, and then the unit grids with the same labels are merged, and the unit grids of the inaccessible white pixels in the binary image in step 2.3 are deleted to obtain the final room layout segmentation result.
[0066] The beneficial effects of the present invention are as follows: the present invention decomposes the top view of the indoor scene into units through unit decomposition, thereby solving the problem of messy room layout; distributes room information to the pixels of each room unit decomposition through room segmentation, thereby solving the serious occlusion phenomenon that may exist in the indoor scene, and finally performs superposition analysis and processing to obtain the final room layout division result. This comprehensive division method not only maintains the geometric law of the room, but also maintains the integrity of the room space. BRIEF DESCRIPTION OF THE DRAWINGS
[0067] Figure 1A flow chart of a room layout partitioning method based on unit decomposition and space segmentation according to the present invention;
[0068] Figure 2 Schematic diagram of original scene data and downsampled scene data of Scene1 of the present invention;
[0069] Figure 3 Schematic diagram of original scene data and downsampled scene data of Scene2 of the present invention;
[0070] Figure 4 It is a schematic diagram of the decomposition result of the room unit of Scene 1 of the present invention;
[0071] Figure 5 It is a schematic diagram of the decomposition result of the room unit of Scene2 of the present invention;
[0072] Figure 6 It is a schematic diagram of the room space segmentation result of Scene 1 of the present invention;
[0073] Figure 7 It is a schematic diagram of the room space segmentation result of Scene 2 of the present invention;
[0074] Figure 8 It is a schematic diagram of the result of the superposition analysis of Scene 1 of the present invention;
[0075] Fig. 9 It is a schematic diagram of the result of the Scene2 overlay analysis of the present invention. DETAILED DESCRIPTION
[0076] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.
[0077] The room layout division method of the present invention is based on unit decomposition and space division, and its process is as follows: Figure 1 As shown, the specific steps are as follows:
[0078] Step 1: extract the wall point cloud from the indoor point cloud scene, then divide the wall point cloud into several segments, calculate the centroid of each segment, fit the straight line corresponding to the wall point cloud according to the centroid, then fuse the fitted straight line, and then perform unit decomposition based on the result of the fusion processing to obtain the unit division result of the indoor scene; specifically:
[0079] Step 1.1, after downsampling the indoor scene, a downsampled indoor point cloud scene is obtained, and the downsampled indoor point cloud scene is used as input to segment it into several horizontal point cloud slices, and the wall points and indoor debris in each horizontal point cloud slice are segmented using the regional growing clustering method, and the wall points are finally retained to obtain the wall point cloud corresponding to the horizontal point cloud slice; wherein, the wall points and indoor debris in each horizontal point cloud slice are segmented using the regional growing clustering method, and the wall points are finally retained to obtain the wall point cloud corresponding to the horizontal point cloud slice. Specifically:
[0080] Step 1.1.1, calculate the curvature of each point in each horizontal point cloud slice, sort the points according to their curvature values, find the point with the smallest curvature, and add it to the seed point set;
[0081] Step 1.1.2, for each seed point, if its K nearest neighbor points satisfy both equation (1) and equation (2), then add the point to the potential seed point list;
[0082] ||n p ·n s ||>cos(θ th ) (1)
[0083] Among them, n p Represents the normal vector of the current seed point; n s Represents the normal vector of the K nearest neighbor points of the current seed point; θ th represents a smoothing threshold;
[0084] r p <r th (2)
[0085] Among them, r p Represents the curvature of the K nearest neighbor points of the current seed point, r th represents a curvature threshold;
[0086] Step 1.1.3, remove the point added to the potential seed point list in step 1.1.2 from the slice point cloud corresponding to the initial indoor point cloud;
[0087] Step 1.1.4, clustering is performed on the basis of step 1.1.3, the number of points of the minimum point cluster is set as Min, the number of points of the maximum point cluster is set as Max, all slices with the number of points between Min and Max are retained, and different slices are marked with different colors to distinguish them. The slice point cloud of the retained slices is the obtained wall point cloud;
[0088] Step 1.1.5, repeat steps 1.1.1 to 1.1.4 to complete the clustering of all slices and finally obtain the wall point cloud corresponding to each slice.
[0089] Step 1.2, divide each wall point cloud extracted in step 1.1 into several segments, calculate the centroid of each segment, use the centroid to represent the characteristics of all points in each segment, and then perform straight line fitting based on the centroid; specifically:
[0090] Step 1.2.1, input the wall point cloud extracted in step 1.1, and save the x, y, and z coordinates of each point in each wall point cloud;
[0091] Step 1.2.2, calculate the length and width of the wall area in space based on the x and y coordinates of the wall point cloud, where the length is x max -x min , width y max -y min , x max 、x min ,y max ,y min Represent the maximum and minimum x and y coordinates respectively; set the step size l to 0.4m, divide the wall point cloud in space into l×l grids, then the total number of rows is raster_rows=x max -x min / l, the total number of columns is raster_cols=y max -y min / l;
[0092] Step 1.2.3, create a one-dimensional array vector_4, a two-dimensional array col_, and a three-dimensional array row_col. Set the size of the array vector_4 to 4. Then, according to the sizes of raster_rows and raster_cols, set the size of the array col_ to raster_cols·vector_4 and the size of the array row_col to raster_rows·col.
[0093] Step 1.2.4, store the points in the wall point cloud into corresponding rows and columns, where the position of any point (row_idx, col_idx) is expressed as (ceil((points[i].x-point_min.x) / l-1), ceil((points[i].y-point_min.y) / l-1)), point[i].x, point[i].y represent the x and y coordinates of any point, point_min.x, point_min.y represent the x and y coordinates of the minimum point;
[0094] Step 1.2.5, if the value of row_idx or col_idx is less than 0, set it to 0; then store the x, y, and z of the point selected in step 1.2.4 in the array row_col[row_idx][col-idx][0], row_col[row_idx][col-idx][1], and row_col[row_idx][col-idx][2] in sequence, and increase the value of the array row_col[row_idx][col-idx][2] by 1 each time a point is stored;
[0095] Step 1.2.6, repeat steps 1.2.4 and 1.2.5 to divide all points belonging to the wall point cloud into corresponding grid spaces, and calculate the centroid point Q of each grid space. The calculation formula is as shown in formula (3);
[0096]
[0097] Among them, x_mean, y_mean, z_mean represent the x, y, and z coordinates of the centroid point Q respectively; m represents the size of the array row_col; n represents the size of the array row_col[i]; row_xol[i][j][0] represents the x coordinate of any point in the space corresponding to the extracted wall point cloud; row_xol[i][j][1] represents the y coordinate of any point in the space corresponding to the extracted wall point cloud; row_xol[i][j][2] represents the z coordinate of any point in the space corresponding to the extracted wall point cloud; row_col[i][j][3] represents the number of points contained in the space corresponding to the wall point cloud extracted from each grid;
[0098] Step 1.2.7, randomly select three points from the multiple centroid points corresponding to each wall point cloud to construct an xoy plane, calculate the coefficients of the plane equation, and then calculate the normal vector v of the constructed plane. Reconstruct the plane according to the solved plane equation coefficients, and use the endpoints of the line segment in space and the normal vector of the plane to construct a projection line. Solve the intersection of the plane and the projection line again. The intersection is the projection point of one endpoint of the line segment in space. Finally, connect the corresponding projection points in sequence to obtain the projected line segment.
[0099] Step 1.3, project the straight line obtained in step 1.2, perform fusion processing after projection, and then extend the fused line segment, calculate the two-dimensional arrangement data structure according to the CGAL library, and finally obtain the two-dimensional unit decomposition result, which is specifically:
[0100] Step 1.3.1, set a predefined length value of a wall segment, then sort the multiple line segments projected in step 1.2.7 from small to large according to their lengths, and directly delete the line segments whose lengths are less than the predefined length value;
[0101] Step 1.3.2, select the line segment with the largest length among the remaining line segments after processing in step 1.3.1 and add it to the final result set, then select a line segment from the remaining line segments to compare with the line segments in the final result set, set an angle threshold and a distance threshold, if the angle between the two line segments is less than the given threshold, then the two line segments are considered to be parallel; on the basis of parallelism, if the distance between the two parallel lines is less than the given threshold, then the two line segments are considered to be collinear, and the two line segments are fused;
[0102] Step 1.3.3, repeat step 1.3.2 until the number of remaining line segments is 0, then end the whole process to obtain the fused line segments;
[0103] In step 1.3.4, the fused line segments are extended, and then the two-dimensional arrangement data structure is calculated according to the CGAL library, and the two-dimensional plane of the indoor scene is divided using the extended line segments to decompose it into two-dimensional units.
[0104] Step 2: After downsampling the indoor scene, project the downsampled indoor scene onto a two-dimensional plane, discretize the projection points and generate a depth image; then binarize the depth image, use the morphological erosion operation to segment the indoor space to identify the number of rooms; finally, use the wavefront extension method to add the area that belongs to the room area but has not been visited to the room space to obtain the spatial segmentation result of the indoor scene; specifically:
[0105] Step 2.1, convert the point cloud of the indoor scene space into a two-dimensional image to obtain the corresponding grayscale image. In order to make the image present an obvious black and white effect, the grayscale image is binarized to obtain a binary image that reflects the overall and local features of the image; specifically:
[0106] Step 2.1.1, project the indoor point cloud after downsampling in step 1.1 onto the xoy plane, find the maximum x, y coordinates and the minimum x, y coordinates from the projected points, and determine the width and height of the image according to the maximum and minimum x, y coordinate values;
[0107] Step 2.1.2, discretize the projection points into a two-dimensional grid to obtain multiple pixel grids, set the pixel size to pixelsize, the size of each pixel is determined by the thickness of the wall and the size and density of the point cloud, and judge whether the pixel is gray or black according to the number of points contained in each pixel. If a pixel contains at least one point, it indicates that the pixel is gray; if a pixel does not contain a point, it indicates that the pixel is black, and then a grayscale image is obtained;
[0108] Step 2.1.3, through appropriate threshold selection, the grayscale value of the pixel on the grayscale image obtained in step 2.1.2 is set to 0 or 255. If the grayscale value of the pixel is greater than or equal to the threshold, it is judged to belong to a specific object, and its grayscale value is represented by 255; otherwise, these pixels are excluded from the object area, and their grayscale value is represented by 0, resulting in a binary image.
[0109] Step 2.2, by default, all doors leading to other rooms or corridors in each room are closed. Use a 3×3 structural element to iteratively erode the accessible pixels in the binary image obtained in step 2.1. The accessible pixels are white pixels in the binary image. After one erosion, an erosion result image is obtained. The contour on the erosion result image is used to determine whether there is a separable area. If a separable area is obtained, the iteration is stopped and the separable area is marked with different colors. Specifically:
[0110] Step 2.2.1, starting the first erosion from the center position of the binary image obtained in step 2.1, using a 3×3 structure element to erode accessible pixels in the binary image, where accessible pixels are white pixels in the binary image. After one erosion is completed, a erosion result image will be obtained;
[0111] Step 2.2.2, find the contour in the erosion result image, establish only two hierarchical relationships for all contours, namely the top layer and the inner layer, and count the number of contours found findContourNums;
[0112] Step 2.2.3, check each contour. If the current contour has no corresponding embedded contour, calculate the area contourArea1 enclosed by the current contour, thereby calculating the room area roomArea1. The calculation formula is shown in (4);
[0113] roomArea1=cellSize*cellSize*contourArea1 (4)
[0114] The parameter roomArea1 represents the room area; cellSize represents the size of a cell grid; contourArea1 represents the area of the area enclosed by the current contour;
[0115] Step 2.2.4: Based on step 2.2.3, determine whether the currently detected contour is an embedded contour of other contours. If the current contour is an embedded contour of one of the contours, calculate the area contourArea2 enclosed by the current contour, and update the room area roomArea2 at the same time. The area calculation formula is shown in formula (5);
[0116] roomArea2=cellSize*cellSize*contourArea2-roomArea1 (5)
[0117] The parameter roomArea2 represents the updated room area;
[0118] Step 2.2.5, determine whether the current room area is between the upper threshold and the lower threshold. The lower threshold represents the smallest room area in the data, and the upper threshold represents the largest room area. If the current area is between the upper threshold and the lower threshold, save its contour;
[0119] Step 2.2.6, repeat steps 2.2.1 to 2.2.5, iteratively erode the accessible pixels in the binary image until the area of all remaining regions is less than the lower threshold, then stop the iteration, obtain the separable regions, and then mark the obtained separable regions with different colors.
[0120] Step 2.3, detect the number of rooms according to the separable areas marked in step 2.2. At this time, there are three states on the binary image, where white pixels represent unmarked areas in the accessible area. Then, the white pixels are expanded using the wavefront propagation method to obtain the final room space segmentation result; specifically:
[0121] Step 2.3.1, according to the binary image of step 2.1 combined with the contour of step 2.2, the room segmentation result is obtained, and this segmentation result is used as input. At this time, the room segmentation result image presents three states. The first is black pixels represented by black, that is, the black pixels obtained by step 2.1.2. The area where the black pixels are located represents the inaccessible area, which does not belong to the indoor scene; the second is white pixels represented by white, that is, the white pixels in the binary image obtained by step 2.1.3. The area where the white pixels are located represents the accessible area but has not been visited; the third is the area marked with different colors, that is, the area marked by step 2.2.6. Different colors represent different room areas composed of some pixels;
[0122] Step 2.3.2, select a pixel from the unmarked white pixels, determine the color of the current pixel based on the colors of other pixels in the 3×3 area around the pixel, and determine the channel values of its three channels starting from the pixel. If the channel values of the three channels of the pixel are greater than 250 at the same time, it means that the pixel is an accessible pixel; if the channel values of the three channels of the pixel are not 0 and are less than or equal to 250 at the same time, assign the channel values of the three channels of this pixel to the accessible pixel, realize the marking of the accessible pixel, and expand it to the room space;
[0123] Step 2.3.3, iterate steps 2.3.1 and 2.3.2 until all unmarked areas in the room segmentation result in step 2.3.1 are marked, thus achieving spatial segmentation of the indoor scene.
[0124] Step 3, overlap the unit division result of the indoor scene obtained in step 1 with the spatial segmentation result of the indoor scene obtained in step 2, generate random points in the overlapping area, and determine which room the grid cell belongs to according to the number of color labels of the random points in each grid cell, so as to achieve accurate division of the room layout; specifically:
[0125] Step 3.1, overlapping the unit division result of the indoor scene obtained in step 1 with the spatial segmentation result of the indoor scene obtained in step 2;
[0126] Step 3.2, create a set of random point sets in the overlapping space, add the center point in each unit grid to the random point set, and let the points in each unit grid extract label information from the spatial segmentation result;
[0127] Step 3.3, by calculating the number of points with the same label in each cell grid, and then assigning a label to the cell grid according to the label with the highest frequency in the cell grid, or according to the fact that a marked cell grid is surrounded by the same marked cell grid, then the cell grid is also marked with the same label as the surrounding cell grids;
[0128] In step 3.4, the unit grids of the segmentation result labels completed in steps 3.1 to 3.3 are visualized, and then the unit grids with the same labels are merged, and the unit grids of the inaccessible white pixels in the binary image in step 2.3 are deleted to obtain the final room layout segmentation result.
[0129] Example
[0130] Embodiments of the present invention The present invention uses two sets of data Scene1 and Scene2, such as Figure 2 (a) Figure 3 (a), where Figure 2 (a) is the original scene data of Scene1, Figure 3 (a) is the original scene data of Scene2. The specific process is as follows:
[0131] The room layout division method based on unit decomposition and space segmentation of the present invention is specifically implemented according to the following steps:
[0132] Step 1: extract the wall point cloud from the indoor point cloud scene, then divide the wall point cloud into several segments, calculate the centroid of each segment, fit the straight line corresponding to the wall point cloud according to the centroid, then fuse the fitted straight line, and then perform unit decomposition based on the result of the fusion processing to obtain the unit division result of the indoor scene; specifically:
[0133] Step 1.1, after downsampling the indoor scene, the downsampled indoor point cloud scene is obtained, corresponding to Figure 2 (b) and Figure 3 (b) where Figure 2 (b) is the downsampled scene data of Scene1. Figure 3 (b) is the downsampled scene data of Scene2. The downsampled indoor point cloud scene is used as input to segment it into several horizontal point cloud slices. The wall points and indoor debris in each horizontal point cloud slice are segmented using the region growing clustering method, and the wall points are finally retained to obtain the wall point cloud corresponding to the horizontal point cloud slice; wherein, the wall points and indoor debris in each horizontal point cloud slice are segmented using the region growing clustering method, and the wall points are finally retained to obtain the wall point cloud corresponding to the horizontal point cloud slice:
[0134] Step 1.1.1, calculate the curvature of each point in each horizontal point cloud slice, sort the points according to their curvature values, find the point with the smallest curvature, and add it to the seed point set;
[0135] Step 1.1.2, for each seed point, if its K nearest neighbor points satisfy both equation (1) and equation (2), then add the point to the potential seed point list;
[0136] ||n p ·n s ||>cos(θ th ) (1)
[0137] Among them, n p Represents the normal vector of the current seed point; n s Represents the normal vector of the K nearest neighbor points of the current seed point; θ th represents a smoothing threshold;
[0138] r p <r th (2)
[0139] Among them, r p Represents the curvature of the K nearest neighbor points of the current seed point, r th represents a curvature threshold;
[0140] In the two experiments of the present invention, the slice thickness of Scene 1 is 0.15 cm, the number of slices is 20, and θth is 7, r th is 1; the slice thickness of Scene2 is 0.15cm, the number of slices is 17, θ th is 7, r th is 1;
[0141] Step 1.1.3, remove the point added to the potential seed point list in step 1.1.2 from the slice point cloud corresponding to the initial indoor point cloud;
[0142] Step 1.1.4, clustering is performed on the basis of step 1.1.3, the number of points of the minimum point cluster is set as Min, the number of points of the maximum point cluster is set as Max, all slices with the number of points between Min and Max are retained, and different slices are marked with different colors to distinguish them. The slice point cloud of the retained slices is the obtained wall point cloud;
[0143] Step 1.1.5, repeat steps 1.1.1 to 1.1.4 to complete the clustering of all slices, and finally obtain the wall point cloud corresponding to each slice. The corresponding two sets of experimental results are as follows Figure 4 (a) Figure 5 (a) shown.
[0144] Step 1.2, divide each wall point cloud extracted in step 1.1 into several segments, calculate the centroid of each segment, use the centroid to represent the characteristics of all points in each segment, and then perform straight line fitting based on the centroid; specifically:
[0145] Step 1.2.1, input the wall point cloud extracted in step 1.1, and save the x, y, and z coordinates of each point in each wall point cloud;
[0146] Step 1.2.2, calculate the length and width of the wall area in space based on the x and y coordinates of the wall point cloud, where the length is x max -x min , width y max -y min , x max 、x min ,y max ,y min Represent the maximum and minimum x and y coordinates respectively; set the step size l to 0.4m, divide the wall point cloud in space into l×l grids, then the total number of rows is raster_rows=x max -x min / l, the total number of columns is raster_cols=y max -y min / l;
[0147] Step 1.2.3, create a one-dimensional array vector_4, a two-dimensional array col_, and a three-dimensional array row_col. Set the size of the array vector_4 to 4. Then, according to the sizes of raster_rows and raster_cols, set the size of the array col_ to raster_cols·vector_4 and the size of the array row_col to raster_rows·col.
[0148] Step 1.2.4, store the points in the wall point cloud into corresponding rows and columns, where the position of any point (row_idx, col_idx) is expressed as (ceil((points[i].x-point_min.x) / l-1), ceil((points[i].y-point_min.y) / l-1)), point[i].x, point[i].y represent the x and y coordinates of any point, point_min.x, point_min.y represent the x and y coordinates of the minimum point;
[0149] Step 1.2.5, if the value of row_idx or col_idx is less than 0, set it to 0; then store the x, y, and z of the point selected in step 1.2.4 in the array row_col[row_idx][col-idx][0], row_col[row_idx][col-idx][1], and row_col[row_idx][col-idx][2] in sequence, and increase the value of the array row_col[row_idx][col-idx][2] by 1 each time a point is stored;
[0150] Step 1.2.6, repeat steps 1.2.4 and 1.2.5 to divide all points belonging to the wall point cloud into corresponding grid spaces, and calculate the centroid point Q of each grid space. The calculation formula is as shown in formula (3);
[0151]
[0152] Among them, x_mean, y_mean, z_mean represent the x, y, and z coordinates of the centroid point Q respectively; m represents the size of the array row_col; n represents the size of the array row_col[i]; row_xol[i][j][0] represents the x coordinate of any point in the space corresponding to the extracted wall point cloud; row_xol[i][j][1] represents the y coordinate of any point in the space corresponding to the extracted wall point cloud; row_xol[i][j][2] represents the z coordinate of any point in the space corresponding to the extracted wall point cloud; row_col[i][j][3] represents the number of points contained in the space corresponding to the wall point cloud extracted from each grid;
[0153] Step 1.2.7, randomly select three points from the multiple centroid points corresponding to each wall point cloud to construct an xoy plane, calculate the coefficients of the plane equation, and then calculate the normal vector v of the constructed plane. Reconstruct the plane according to the solved plane equation coefficients, and use the endpoints of the line segment in space and the normal vector of the plane to construct a projection line. Solve the intersection of the plane and the projection line again. The intersection is the projection point of one endpoint of the line segment in space. Finally, connect the corresponding projection points in sequence to obtain the projected line segment. The corresponding two sets of experimental results are shown as follows: Figure 4 (b) Figure 5 (b) as shown.
[0154] Step 1.3, project the straight line obtained in step 1.2, perform fusion processing after projection, and then extend the fused line segment, calculate the two-dimensional arrangement data structure according to the CGAL library, and finally obtain the two-dimensional unit decomposition result, which is specifically:
[0155] Step 1.3.1, set a predefined length value of a wall segment, then sort the multiple line segments projected in step 1.2.7 from small to large according to their lengths, and directly delete the line segments whose lengths are less than the predefined length value;
[0156] Step 1.3.2, select the line segment with the largest length among the remaining line segments after processing in step 1.3.1 and add it to the final result set, then select a line segment from the remaining line segments to compare with the line segments in the final result set, set an angle threshold and a distance threshold, if the angle between the two line segments is less than the given threshold, then the two line segments are considered to be parallel; on the basis of parallelism, if the distance between the two parallel lines is less than the given threshold, then the two line segments are considered to be collinear, and the two line segments are fused;
[0157] Step 1.3.3, repeat step 1.3.2 until the number of remaining line segments is 0, then end the whole process to obtain the fused line segments. The corresponding two sets of experimental results are as follows: Figure 4 (c) Figure 5 (c)
[0158] Step 1.3.4, extend the fused line segments, then calculate the two-dimensional arrangement data structure according to the CGAL library, use the extended line segments to divide the two-dimensional plane of the indoor scene, and decompose it into two-dimensional units. The corresponding two groups of experimental results are shown in Figure 4 (d) Figure 5 (d) as shown.
[0159] Step 2: After downsampling the indoor scene, project the downsampled indoor scene onto a two-dimensional plane, discretize the projection points and generate a depth image; then binarize the depth image, use the morphological erosion operation to segment the indoor space to identify the number of rooms; finally, use the wavefront extension method to add the area that belongs to the room area but has not been visited to the room space to obtain the spatial segmentation result of the indoor scene; specifically:
[0160] Step 2.1, convert the point cloud of the indoor scene space into a two-dimensional image to obtain the corresponding grayscale image. In order to make the image present an obvious black and white effect, the grayscale image is binarized to obtain a binary image that reflects the overall and local features of the image; specifically:
[0161] Step 2.1.1, project the indoor point cloud after downsampling in step 1.1 onto the xoy plane, find the maximum x, y coordinates and the minimum x, y coordinates from the projected points, and determine the width and height of the image according to the maximum and minimum x, y coordinate values;
[0162] Step 2.1.2, discretize the projection points into a two-dimensional grid to obtain multiple pixel grids, set the pixel size to pixelsize, the size of each pixel is determined by the thickness of the wall and the size and density of the point cloud, and judge whether the pixel is gray or black according to the number of points contained in each pixel. If a pixel contains at least one point, it indicates that the pixel is gray; if a pixel does not contain a point, it indicates that the pixel is black, and then a grayscale image is obtained. The corresponding two sets of experimental results are shown as follows Figure 6 (a) Figure 7 (a)
[0163] Step 2.1.3, through appropriate threshold selection, the grayscale value of the pixel on the grayscale image obtained in step 2.1.2 is set to 0 or 255. If the grayscale value of the pixel is greater than or equal to the threshold, it is judged to belong to a specific object, and its grayscale value is represented by 255; otherwise, these pixels are excluded from the object area, and their grayscale value is represented by 0, and a binary image is obtained. The corresponding two sets of experimental results are shown as follows. Figure 6 (b) Figure 7 (b) as shown.
[0164] Step 2.2, by default, all doors leading to other rooms or corridors in each room are closed. Use a 3×3 structural element to iteratively erode the accessible pixels in the binary image obtained in step 2.1. The accessible pixels are white pixels in the binary image. After one erosion, an erosion result image is obtained. The contour on the erosion result image is used to determine whether there is a separable area. If a separable area is obtained, the iteration is stopped and the separable area is marked with different colors. Specifically:
[0165] Step 2.2.1, starting the first erosion from the center position of the binary image obtained in step 2.1, using a 3×3 structure element to erode accessible pixels in the binary image, where accessible pixels are white pixels in the binary image. After one erosion is completed, a erosion result image will be obtained;
[0166] Step 2.2.2, find the contour in the erosion result image, establish only two hierarchical relationships for all contours, namely the top layer and the inner layer, and count the number of contours found findContourNums;
[0167] Step 2.2.3, check each contour. If the current contour has no corresponding embedded contour, calculate the area contourArea1 enclosed by the current contour, thereby calculating the room area roomArea1. The calculation formula is shown in (4);
[0168] roomArea1=cellSize*cellSize*contourArea1 (4)
[0169] The parameter roomArea1 represents the room area; cellSize represents the size of a cell grid; contourArea1 represents the area of the area enclosed by the current contour;
[0170] Step 2.2.4: Based on step 2.2.3, determine whether the currently detected contour is an embedded contour of other contours. If the current contour is an embedded contour of one of the contours, calculate the area contourArea2 enclosed by the current contour, and update the room area roomArea2 at the same time. The area calculation formula is shown in formula (5);
[0171] roomArea2=cellSize*cellSize*contourArea2-roomArea1 (5)
[0172] The parameter roomArea2 represents the updated room area;
[0173] Step 2.2.5, determine whether the current room area is between the upper threshold and the lower threshold. The lower threshold represents the smallest room area in the data, and the upper threshold represents the largest room area. If the current area is between the upper threshold and the lower threshold, save its contour;
[0174] In the two experiments of the present invention, the pixel size in the binary image of Scene 1 is 60 mm, the largest room area is 10, and the smallest room area is 1; the pixel size in the binary image of Scene 2 is 50 mm, the largest room area is 25, and the smallest room area is 3;
[0175] Step 2.2.6, repeat steps 2.2.1 to 2.2.5, iteratively erode the accessible pixels in the binary image until the area of all remaining regions is less than the lower threshold, then stop the iteration, get the separable regions, and then mark the separable regions with different colors. The corresponding two sets of experimental results are shown in Figure 6 (c) Figure 7 (c) as shown.
[0176] Step 2.3, detect the number of rooms according to the separable areas marked in step 2.2. At this time, there are three states on the binary image, where white pixels represent unmarked areas in the accessible area. Then, the white pixels are expanded using the wavefront propagation method to obtain the final room space segmentation result; specifically:
[0177] Step 2.3.1, according to the binary image of step 2.1 combined with the contour of step 2.2, the room segmentation result is obtained, and this segmentation result is used as input. At this time, the room segmentation result image presents three states. The first is black pixels represented by black, that is, the black pixels obtained by step 2.1.2. The area where the black pixels are located represents the inaccessible area, which does not belong to the indoor scene; the second is white pixels represented by white, that is, the white pixels in the binary image obtained by step 2.1.3. The area where the white pixels are located represents the accessible area but has not been visited; the third is the area marked with different colors, that is, the area marked by step 2.2.6. Different colors represent different room areas composed of some pixels;
[0178] Step 2.3.2, select a pixel from the unmarked white pixels, determine the color of the current pixel based on the colors of other pixels in the 3×3 area around the pixel, and determine the channel values of its three channels starting from the pixel. If the channel values of the three channels of the pixel are greater than 250 at the same time, it means that the pixel is an accessible pixel; if the channel values of the three channels of the pixel are not 0 and are less than or equal to 250 at the same time, assign the channel values of the three channels of this pixel to the accessible pixel, realize the marking of the accessible pixel, and expand it to the room space;
[0179] Step 2.3.3, iterate steps 2.3.1 and 2.3.2 until all the unmarked areas in the room segmentation result in step 2.3.1 are marked, and the spatial segmentation of the indoor scene is achieved. The corresponding two sets of experimental results are shown as follows: Figure 6 (d) Figure 7 (d) as shown.
[0180] Step 3, overlap the unit division result of the indoor scene obtained in step 1 with the spatial segmentation result of the indoor scene obtained in step 2, generate random points in the overlapping area, and determine which room the grid cell belongs to according to the number of color labels carried by the random points in each grid cell, so as to achieve accurate division of the room layout; specifically, step 3.1, overlap the unit division result of the indoor scene obtained in step 1 with the spatial segmentation result of the indoor scene obtained in step 2, and the corresponding two groups of experimental results are shown as follows: Figure 8 (a) Fig. 9 (a)
[0181] Step 3.2, create a set of random points in the overlapping space, add the center point in each unit grid to the random point set, and the corresponding two sets of experimental results are as follows: Figure 8 (b) Fig. 9 As shown in (b), the points in each unit grid are allowed to extract label information from the spatial segmentation results;
[0182] Step 3.3, by calculating the number of points with the same label in each unit grid, and then assigning a label to the unit grid according to the label with the highest frequency in the unit grid, or according to the fact that a marked unit grid is surrounded by the same marked unit grid, then the unit grid is also marked with the same label as the surrounding unit grids. The corresponding two sets of experimental results are shown as follows: Figure 8 (c) Fig. 9 (c)
[0183] Step 3.4 visualizes the cell grids of the segmentation result labels completed from step 3.1 to step 3.3, then merges the cell grids with the same label, and deletes the cell grids of the inaccessible white pixels in the binary image in step 2.3. The corresponding two sets of experimental results are shown in Figure 8 (d) Fig. 9 As shown in (d), the final room layout division result is obtained.
Claims
1. A room layout division method based on unit decomposition and space segmentation, characterized in that: The specific steps are as follows: Step 1: extract wall point cloud from the indoor point cloud scene, then divide the wall point cloud into several segments, calculate the centroid of each segment, fit the straight line corresponding to the wall point cloud according to the centroid, then fuse the fitted straight line, and then perform unit decomposition based on the result of the fusion processing to obtain the unit division result of the indoor scene; Specifically: Step 1.1, after downsampling the indoor scene, a downsampled indoor point cloud scene is obtained, and the downsampled indoor point cloud scene is used as input to segment it into several horizontal point cloud slices, and the wall points and indoor debris in each horizontal point cloud slice are segmented using the region growing clustering method, and finally the wall points are retained to obtain the wall point cloud corresponding to the horizontal point cloud slice; Among them, the region growing clustering method is used to segment the wall points and indoor debris in each horizontal point cloud slice, and finally retain the wall points to obtain the wall point cloud corresponding to the horizontal point cloud slice: Step 1.1.1, calculate the curvature of each point in each horizontal point cloud slice, sort the points according to their curvature values, find the point with the smallest curvature, and add it to the seed point set; Step 1.1.2, for each seed point, if its K nearest neighbor points satisfy both equation (1) and equation (2), then add the point to the potential seed point list; (1) in, Represents the normal vector of the current seed point; Represents the normal vector of the K nearest neighbor points of the current seed point; represents a smoothing threshold; (2) in, Represents the curvature of the K nearest neighbor points of the current seed point, represents a curvature threshold; Step 1.1.3, remove the point added to the potential seed point list in step 1.1.2 from the slice point cloud corresponding to the initial indoor point cloud; Step 1.1.4, clustering is performed based on step 1.1.3, and the number of points in the minimum cluster is set to , the number of points in the largest cluster is , will generate points in and All slices between are retained, and different slices are marked with different colors to distinguish them. The slice point cloud of the retained slices is the obtained wall point cloud; Step 1.1.5, repeat steps 1.1.1 to 1.1.4 to complete the clustering of all slices, and finally obtain the wall point cloud corresponding to each slice; Step 1.2, divide each wall point cloud extracted in step 1.1 into several segments, calculate the centroid of each segment, use the centroid to represent the characteristics of all points in each segment, and then perform straight line fitting based on the centroid; specifically: Step 1.2.1, input the wall point cloud extracted in step 1.1, and save each point in each wall point cloud separately. coordinate; Step 1.2.2, based on the wall point cloud Coordinates are used to calculate the length and width of the wall area in space. The length is , the width is , Respectively represent the largest and smallest Coordinates; set step size is 0.4m, dividing the wall point cloud in space into In a grid of , the total number of rows is The total number of columns is ; Step 1.2.3, create a one-dimensional array , two-dimensional array , a three-dimensional array , set the array The size is 4, according to and Set the size of the array Size , array The size is ; Step 1.2.4, store the points in the wall point cloud into corresponding rows and columns, where the position of any point Expressed as , , represents the x and y coordinates of any point, Indicates the minimum point coordinate; Step 1.2.5, if or If the value of is less than 0, set it to 0; then set the point selected in step 1.2.4 Stored in array , , In each point array, The value at is increased by 1; Step 1.2.6, repeat steps 1.2.4 and 1.2.5 to divide all points belonging to the wall point cloud into corresponding grid spaces, and calculate the centroid point Q of each grid space. The calculation formula is as shown in formula (3); (3) in, They represent the center of mass point Q coordinate; Representing an array size; Representing an array size; Represents any point in the space corresponding to the extracted wall point cloud coordinate; Represents any point in the space corresponding to the extracted wall point cloud coordinate; Represents any point in the space corresponding to the extracted wall point cloud coordinate; Indicates the number of points contained in the space corresponding to the wall point cloud extracted from each grid; Step 1.2.7, randomly select three points from the multiple centroid points corresponding to each wall point cloud to construct a plane, calculate the coefficients of the plane equation, and then calculate the normal vector to construct the plane , reconstruct the plane according to the solved plane equation coefficients, and construct a projection line using the endpoints of the line segment in space and the normal vector of the plane, solve the intersection of the plane and the projection line again, the intersection is the projection point of one endpoint of the line segment in space, and finally connect the corresponding projection points in sequence to get the projected line segment; Step 1.3, project the straight line obtained in step 1.2, perform fusion processing after projection, and then extend the fused line segment, calculate the two-dimensional arrangement data structure according to the CGAL library, and finally obtain the two-dimensional unit decomposition result; specifically: Step 1.3.1, set a predefined length value of a wall segment, then sort the multiple line segments projected in step 1.2.7 from small to large according to their lengths, and directly delete the line segments whose lengths are less than the predefined length value; Step 1.3.2, select the line segment with the largest length among the remaining line segments after processing in step 1.3.1 and add it to the final result set, then select a line segment from the remaining line segments to compare with the line segments in the final result set, set an angle threshold and a distance threshold, if the angle between the two line segments is less than the given threshold, then the two line segments are considered to be parallel; on the basis of parallelism, if the distance between the two parallel lines is less than the given threshold, then the two line segments are considered to be collinear, and the two line segments are fused; Step 1.3.3, repeat step 1.3.2 until the number of remaining line segments is 0, then end the whole process to obtain the fused line segments; Step 1.3.4, extend the fused line segments, then calculate the two-dimensional arrangement data structure according to the CGAL library, use the extended line segments to divide the two-dimensional plane of the indoor scene, and decompose it into two-dimensional units; Step 2: After downsampling the indoor scene, project the downsampled indoor scene onto a two-dimensional plane, discretize the projection points and generate a depth image; then binarize the depth image, use the morphological erosion operation to segment the indoor space to identify the number of rooms; finally, use the wavefront extension method to add the area that belongs to the room area but has not been visited to the room space to obtain the spatial segmentation result of the indoor scene; specifically: Step 2.1, converting the point cloud of the indoor scene space into a two-dimensional image to obtain a corresponding grayscale image, binarizing the grayscale image to obtain a binary image reflecting the overall and local features of the image; Step 2.2, by default, all doors from each room to other rooms or corridors are closed, use The structural element of iteratively corrodes the accessible pixels in the binary image obtained in step 2.
1. The accessible pixels are white pixels in the binary image. After one corrosion is completed, an corrosion result image is obtained. The contour on the corrosion result image is used to determine whether there is a separable area. If a separable area is obtained, the iteration is stopped and the separable area is marked with different colors. Specifically: Step 2.2.1, start the first erosion from the center of the binary image obtained in step 2.1, using The structural element of erodes the accessible pixels in the binary image. The accessible pixels are white pixels in the binary image. After the erosion is completed once, a erosion result image will be obtained. Step 2.2.2, find contours in the corrosion result map, establish only two levels of relationships for all contours, namely the top layer and the inner layer, and count the number of contours found ; Step 2.2.3, check each contour, if the current contour has no corresponding embedded contour, calculate the area enclosed by the current contour , thus calculating the room area , the calculation formula is shown in (4); (4) The parameters Indicates the room area; Indicates the size of a unit grid; Indicates the area enclosed by the current contour; Step 2.2.4: Based on step 2.2.3, determine whether the currently detected contour is an embedded contour of other contours. If the current contour is an embedded contour of one of the contours, calculate the area enclosed by the current contour. , while updating the room area , the area calculation formula is shown in formula (5); (5) The parameters Indicates the updated room area; Step 2.2.5, determine whether the current room area is between the upper threshold and the lower threshold. The lower threshold represents the smallest room area in the data, and the upper threshold represents the largest room area. If the current area is between the upper threshold and the lower threshold, save its contour; Step 2.2.6, repeat steps 2.2.1 to 2.2.5, iteratively erode the accessible pixels in the binary image until the area of all remaining regions is smaller than the lower threshold, then stop the iteration, obtain the separable regions, and then mark the obtained separable regions with different colors; Step 2.3, detect the number of rooms according to the separable areas marked in step 2.
2. At this time, there are three states on the binary image, where white pixels represent unmarked areas in the accessible area. Then, the white pixels are expanded using the wavefront propagation method to obtain the final room space segmentation result; Step 3, overlap the unit division result of the indoor scene obtained in step 1 with the spatial segmentation result of the indoor scene obtained in step 2, generate random points in the overlapping area, and determine which room the grid cell belongs to according to the number of color labels of the random points in each grid cell, so as to achieve accurate division of the room layout; specifically: Step 3.1, overlapping the unit division result of the indoor scene obtained in step 1 with the spatial segmentation result of the indoor scene obtained in step 2; Step 3.2, create a set of random point sets in the overlapping space, add the center point in each unit grid to the random point set, and let the points in each unit grid extract label information from the spatial segmentation result; Step 3.3, by calculating the number of points with the same label in each cell grid, and then assigning a label to the cell grid according to the label with the highest frequency in the cell grid, or according to the fact that a marked cell grid is surrounded by the same marked cell grid, then the cell grid is also marked with the same label as the surrounding cell grids; In step 3.4, the unit grids of the segmentation result labels completed in steps 3.1 to 3.3 are visualized, and then the unit grids with the same labels are merged, and the unit grids of the inaccessible white pixels in the binary image in step 2.3 are deleted to obtain the final room layout segmentation result.
2. The room layout division method based on unit decomposition and space division according to claim 1 is characterized in that: The step 2.1 is specifically as follows: Step 2.1.1, project the indoor point cloud downsampled in step 1.1 to On the plane, find the largest , Coordinates and minimum , Coordinates, according to the largest , The coordinate values determine the width and height of the image; Step 2.1.2, discretize the projection points into a two-dimensional grid to obtain multiple pixel grids, and set the pixel size to The size of each pixel is determined by the thickness of the wall, the size of the point cloud, and the density of the point cloud. The number of points contained in each pixel determines whether the pixel is gray or black. If a pixel contains at least one point, it indicates that the pixel is gray; if a pixel does not contain a point, it indicates that the pixel is black, and a grayscale image is obtained; Step 2.1.3, through appropriate threshold selection, the grayscale value of the pixel on the grayscale image obtained in step 2.1.2 is set to 0 or 255. If the grayscale value of the pixel is greater than or equal to the threshold, it is judged to belong to a specific object, and its grayscale value is represented by 255; otherwise, these pixels are excluded from the object area, and their grayscale value is represented by 0, resulting in a binary image.
3. The room layout division method based on unit decomposition and space division according to claim 2 is characterized in that: The step 2.3 is specifically as follows: Step 2.3.1, according to the binary image of step 2.1 combined with the contour of step 2.2, the room segmentation result is obtained, and this segmentation result is used as input. At this time, the room segmentation result image presents three states. The first is black pixels represented by black, that is, the black pixels obtained by step 2.1.
2. The area where the black pixels are located represents the inaccessible area, which does not belong to the indoor scene; the second is white pixels represented by white, that is, the white pixels in the binary image obtained by step 2.1.
3. The area where the white pixels are located represents the accessible area but has not been visited; the third is the area marked with different colors, that is, the area marked by step 2.2.
6. Different colors represent different room areas composed of some pixels; Step 2.3.2, select a pixel from the unmarked white pixels, and The color of the current pixel is determined by the colors of other pixels in the area. The channel values of the three channels are determined starting from the pixel. If the channel values of the three channels of the pixel are greater than 250 at the same time, it means that the pixel is an accessible pixel; if the channel values of the three channels of the pixel are not 0 and are less than or equal to 250 at the same time, the channel values of the three channels of this pixel are assigned to the accessible pixel to realize the marking of the accessible pixel and extend it to the room space; Step 2.3.3, iterate steps 2.3.1 and 2.3.2 until all unmarked areas in the room segmentation result in step 2.3.1 are marked, thus achieving spatial segmentation of the indoor scene.
