Scene accessory component extraction method and device based on point cloud data

Abstract

The embodiment of the invention discloses a scene accessory component extraction method and device based on point cloud data. The method comprises the steps of obtaining point cloud data collected in a specified scene; performing plane segmentation processing on the point cloud data, and obtaining a point cloud cluster belonging to a wall plane in the specified scene based on a plane segmentation processing result; projecting the point cloud cluster of the wall plane to the corresponding wall plane to obtain an occupation grid map; detecting a shielding area in the occupied grid map, and performing filling processing on the shielding area; wherein the shielding area is an area, shielded by an object, of a wall body in the specified scene; and extracting the bounding box of the cavity area from the occupied grid map after filling processing, and determining the position information of the affiliated component of the wall in the specified scene based on the bounding box of the cavity area, so that the position information of the affiliated component can be extracted from the point cloud data robustly and accurately, and the data processing efficiency is improved.

Description

Technical Field

[0001] The present invention relates to three-dimensional reconstruction technology and point cloud processing technology, and in particular to a method and device for extracting scene accessory components based on point cloud data. Background Art

[0002] With the continuous development of LiDAR detection technology, the depth information of the target can be quickly obtained, and then the point cloud data of the target can be generated. Point cloud data carries high-precision, high-resolution and high-dimensional geometric information, which can intuitively represent the shape, surface and texture of objects in space, and can be used for applications such as 3D modeling and scene reconstruction. In an application example, after the point cloud data of an indoor scene is obtained by laser scanning, in addition to directly browsing the colored point cloud, it is often necessary to further perform scene analysis and other applications, such as extracting the position of the wall, extracting the position of the wall accessories such as doors and windows, etc. The existing method for calculating the position and size of the accessories in the point cloud data is to manually measure them in the point cloud editing software. This method is cumbersome to operate and inefficient, and the accuracy of the detection results is difficult to guarantee. Summary of the invention

[0003] In view of the above problems in the prior art, an embodiment of the present disclosure provides a method and device for extracting scene accessory components based on point cloud data.

[0004] A first aspect of the embodiments of the present disclosure provides a method for extracting scene attachment components based on point cloud data, comprising:

[0005] Get the point cloud data collected in the specified scene;

[0006] Performing plane segmentation processing on the point cloud data, and obtaining a point cloud cluster belonging to the wall plane in the specified scene based on the result of the plane segmentation processing;

[0007] Project the point cloud cluster of the wall plane onto the corresponding wall plane to obtain the occupancy grid map;

[0008] Detect the occluded area in the occupancy grid map and fill the occluded area; the occluded area is the area where the wall in the specified scene is occluded by the object;

[0009] The bounding box of the hole area is extracted from the occupancy grid map after the filling process, and the position information of the attached components of the wall in the specified scene is determined based on the bounding box of the hole area.

[0010] As a possible implementation of the first aspect, detecting an occluded area in an occupied grid map and performing filling processing on the occluded area further includes:

[0011] Perform a closing operation on the occupancy grid map;

[0012] The occluded area in the occupancy grid map after the closing operation is filled.

[0013] As a possible implementation of the first aspect, detecting an occluded area in an occupied grid map and performing filling processing on the occluded area includes:

[0014] For unoccupied pixels in the occupancy grid map, obtain the 3D points corresponding to the pixels in the point cloud data;

[0015] Calculate whether there is point cloud occlusion in the direction of the ray passing through the three-dimensional point; wherein the ray passing through the three-dimensional point is a ray that takes the three-dimensional point as an endpoint and extends into the room along the normal direction of the wall plane;

[0016] If there is a point cloud occlusion in the direction of the ray, the pixel point in the occupancy grid map is filled as occupied.

[0017] As a possible implementation of the first aspect, the method further includes:

[0018] Performing plane segmentation processing on the point cloud data, and obtaining point cloud clusters belonging to the top plane and the bottom plane in the specified scene respectively based on the result of the plane segmentation processing;

[0019] Fit the point cloud clusters of the top plane and the bottom plane to obtain the position information of the top plane and the bottom plane;

[0020] Based on the position information of the accessory component, the position information of the top plane and the bottom plane, the relative position information of the accessory component and the top plane, and the relative position information of the accessory component and the bottom plane are determined.

[0021] As a possible implementation of the first aspect, the method further includes:

[0022] Extract edge point cloud data whose distance from an edge line of the attached component is less than a first threshold value according to position information of the attached component of the wall in the specified scene;

[0023] Perform plane extraction on the edge point cloud data to obtain the position information of the edge plane. The edge plane is a plane parallel to the edge line of the attached component and perpendicular to the wall plane.

[0024] According to the position information of the edge plane, the position information of the auxiliary component is adjusted.

[0025] As a possible implementation of the first aspect, the method further includes:

[0026] Among multiple wall planes in the specified scene, a wall plane that is parallel to the wall plane and whose distance satisfies a preset condition is determined as the opposite wall plane to the wall;

[0027] According to the position information of the auxiliary component, detecting whether there is an auxiliary component on the opposite side at a corresponding position on the plane of the opposite wall;

[0028] According to the detection result of the auxiliary component on the opposite side, the position information of the auxiliary component is verified.

[0029] As a possible implementation of the first aspect, the position information of the auxiliary component is verified according to the detection result of the auxiliary component on the opposite side, including:

[0030] If it is detected that there is no opposite-side auxiliary component at the corresponding position of the opposite-side wall plane, it is determined that the position information of the auxiliary component is misjudged information.

[0031] As a possible implementation of the first aspect, the position information of the auxiliary component is verified according to the detection result of the auxiliary component on the opposite side, including:

[0032] If it is detected that there is an opposite-side auxiliary component at a corresponding position on the opposite-side wall plane, the position information of the auxiliary component is adjusted based on the position information of the opposite-side auxiliary component.

[0033] As a possible implementation manner of the first aspect, adjusting the position information of the auxiliary component based on the position information of the auxiliary component on the opposite side includes:

[0034] The intersection or union of the area where the attached component and the opposite-side attached component are located is adjusted to the area where the attached component is located.

[0035] A second aspect of the disclosed embodiment provides a scene accessory component extraction device based on point cloud data, comprising:

[0036] An acquisition unit, used for acquiring point cloud data collected in a specified scene;

[0037] A segmentation unit, used for performing plane segmentation processing on the point cloud data, and obtaining a point cloud cluster belonging to the wall plane in the specified scene based on the result of the plane segmentation processing;

[0038] A projection unit, used for projecting the point cloud cluster of the wall plane onto the corresponding wall plane to obtain an occupancy grid map;

[0039] A filling unit is used to detect the occluded area in the occupancy grid map and fill the occluded area; wherein the occluded area is the area where the wall in the specified scene is occluded by the object;

[0040] The extraction unit is used to extract the bounding box of the hole area in the occupancy grid map after the filling process, and determine the position information of the attached components of the wall in the specified scene based on the bounding box of the hole area.

[0041] As a possible implementation manner of the second aspect, the filling unit is further used for:

[0042] Perform a closing operation on the occupancy grid map;

[0043] The occluded area in the occupancy grid map after the closing operation is filled.

[0044] As a possible implementation manner of the second aspect, the filling unit is used to:

[0045] For unoccupied pixels in the occupancy grid map, obtain the 3D points corresponding to the pixels in the point cloud data;

[0046] Calculate whether there is point cloud occlusion in the direction of the ray passing through the three-dimensional point; wherein the ray passing through the three-dimensional point is a ray that takes the three-dimensional point as an endpoint and extends into the room along the normal direction of the wall plane;

[0047] If there is a point cloud occlusion in the direction of the ray, the pixel point in the occupancy grid map is filled as occupied.

[0048] As a possible implementation manner of the second aspect, the apparatus further includes a processing unit, configured to:

[0049] Performing plane segmentation processing on the point cloud data, and obtaining point cloud clusters belonging to the top plane and the bottom plane in the specified scene respectively based on the result of the plane segmentation processing;

[0050] Fit the point cloud clusters of the top plane and the bottom plane to obtain the position information of the top plane and the bottom plane;

[0051] Based on the position information of the accessory component, the position information of the top plane and the bottom plane, the relative position information of the accessory component and the top plane, and the relative position information of the accessory component and the bottom plane are determined.

[0052] As a possible implementation manner of the second aspect, the apparatus further includes an adjustment unit, configured to:

[0053] Extract edge point cloud data whose distance from an edge line of the attached component is less than a first threshold value according to position information of the attached component of the wall in the specified scene;

[0054] Perform plane extraction on the edge point cloud data to obtain the position information of the edge plane. The edge plane is a plane parallel to the edge line of the attached component and perpendicular to the wall plane.

[0055] According to the position information of the edge plane, the position information of the auxiliary component is adjusted.

[0056] As a possible implementation manner of the second aspect, the apparatus further includes a testing unit, and the testing unit includes:

[0057] A matching subunit, configured to determine, among multiple wall planes in a specified scene, a wall plane that is parallel to the wall plane and whose distance satisfies a preset condition as the opposite wall plane to the wall;

[0058] The detection subunit is used to detect whether there is an opposite side auxiliary component at a corresponding position on the opposite side wall plane according to the position information of the auxiliary component;

[0059] The verification subunit is used to verify the position information of the auxiliary component according to the detection result of the auxiliary component on the opposite side.

[0060] As a possible implementation manner of the second aspect, the check subunit is used to:

[0061] If it is detected that there is no opposite-side auxiliary component at the corresponding position of the opposite-side wall plane, it is determined that the position information of the auxiliary component is misjudged information.

[0062] As a possible implementation manner of the second aspect, the check subunit is used to:

[0063] If it is detected that there is an opposite-side auxiliary component at a corresponding position on the opposite-side wall plane, the position information of the auxiliary component is adjusted based on the position information of the opposite-side auxiliary component.

[0064] As a possible implementation manner of the second aspect, the check subunit is used to:

[0065] If it is detected that there is an opposite side auxiliary component at a corresponding position of the opposite side wall plane, the intersection or union of the auxiliary component and the area where the opposite side auxiliary component is located is adjusted to the area where the auxiliary component is located.

[0066] A third aspect of the present disclosure provides an electronic device, including:

[0067] a memory for storing a computer program product;

[0068] The processor is used to execute a computer program product stored in the memory, and when the computer program product is executed, the method of any one of the above-mentioned first aspects is implemented.

[0069] A fourth aspect of an embodiment of the present disclosure provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the method of any one of the above-mentioned first aspects is implemented.

[0070] Based on the embodiments of the present disclosure, by projecting the point cloud data in a specified scene into an occupancy grid map, and by detecting the occluded areas blocked by objects in the scene in the occupancy grid map, interference factors in the extracted information can be effectively filtered out, and the position information of the attached components can be robustly and accurately extracted from the point cloud data, thereby improving data processing efficiency.

[0071] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0073] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0074] Figure 1 A flowchart of an embodiment of the scene attachment component extraction method disclosed in the present invention;

[0075] Figure 2 A schematic diagram of point cloud data projection of an embodiment of the scene accessory component extraction method disclosed in the present invention;

[0076] Figure 3 A schematic diagram of the locations of attached components of an embodiment of the scene attached component extraction method disclosed in the present invention;

[0077] Figure 4 A schematic diagram of point cloud data projection of an embodiment of the scene accessory component extraction method disclosed in the present invention;

[0078] Figure 5 A schematic diagram of point cloud data projection of an embodiment of the scene accessory component extraction method disclosed in the present invention;

[0079] Figure 6 A flowchart of an embodiment of the scene attachment component extraction method disclosed in the present invention;

[0080] Figure 7 It is a structural schematic diagram of an embodiment of the scene-attached component extraction device disclosed in the present invention;

[0081] Figure 8 It is a structural schematic diagram of an embodiment of the scene-attached component extraction device disclosed in the present invention;

[0082] Fig. 9 It is a structural schematic diagram of an embodiment of the scene-attached component extraction device disclosed in the present invention;

[0083] Fig.10 A block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0084] The following first introduces the existing method, and then introduces the technical solution of the present application in detail.

[0085] In the related art, scene analysis is further performed on the point cloud data of the indoor scene obtained by laser scanning, such as extracting the position of the wall, extracting the position of the wall accessories such as doors and windows, etc., and manually measuring them in the point cloud editing software. This method has high labor costs, is cumbersome to operate, and is relatively inefficient, and the accuracy of the detection results is difficult to guarantee. Alternatively, there is also information extraction of accessory components based on image recognition algorithms in the related art, but this method requires additional camera configuration, such as the configuration of cameras and other equipment, which increases additional costs. In summary, the existing scene accessory component extraction methods are costly, inefficient, and difficult to guarantee accuracy.

[0086] Based on the technical problems existing in the above-mentioned prior art, an embodiment of the present application provides a method for extracting scene accessory components based on point cloud data. By projecting the point cloud data in a specified scene into an occupancy grid map, and by detecting the occluded areas blocked by objects in the scene in the occupancy grid map, the interference factors in the extracted information can be effectively filtered out, and the position information of the accessory components can be robustly and accurately extracted from the point cloud data, thereby improving the data processing efficiency without increasing additional labor costs and equipment costs, thereby solving the technical problems mentioned in the prior art of high cost, low efficiency and difficulty in ensuring accuracy.

[0087] Figure 1 FIG. 1 is a flow chart of an embodiment of a method for extracting scene attachment components based on point cloud data disclosed in the present invention. Figure 1 As shown, it may specifically include:

[0088] Step S110, acquiring point cloud data collected in a specified scene. In one example, the specified scene may include an indoor area of ​​a specified house. According to the layout of the house, the point cloud collection device may be controlled to move in each room in sequence to collect point cloud data. The point cloud collection device may be a mobile scanning device that emits lasers, such as a three-dimensional laser scanner, a laser radar, etc.

[0089] Step S120 , performing plane segmentation processing on the point cloud data, and obtaining a point cloud cluster belonging to the wall plane in the specified scene based on the result of the plane segmentation processing.

[0090] First, the point cloud data collected in the specified scene is segmented using the plane segmentation algorithm of the point cloud, and the points in the point cloud data are assigned to clusters belonging to the same plane to obtain point cloud clusters belonging to the wall plane in the specified scene. Then, the normal, length, and height information of the plane obtained by the plane segmentation process are used to obtain multiple wall planes in the specified scene. For example, a plane whose normal is along the horizontal plane and whose height and length exceed a certain threshold can be determined as a wall plane. Based on the above method, all wall planes in the specified scene are filtered.

[0091] An exemplary point cloud plane segmentation algorithm includes the following steps:

[0092] 1) Preprocess the point cloud data: including data filtering, downsampling and other operations.

[0093] 2) Plane detection: Use appropriate algorithms, such as RANSAC (Random Sample Consensus) or Region Growing, to detect planes in point cloud data. These algorithms find specific plane models by fitting or clustering samples or point sets in the data.

[0094] 3) Plane parameter estimation: After finding the plane, estimate its parameters. Usually, the least square method or the method of minimizing the sum of squared residuals is used to fit the representation and description of the plane for further use and analysis.

[0095] 4) Plane segmentation: Based on the plane model and corresponding parameters, the point cloud data is segmented according to the plane, and the points are assigned to clusters belonging to the same plane. This can be achieved by calculating the distance between the point and the plane or the angle between the normal vector.

[0096] Step S130, project the point cloud cluster of the wall plane onto the corresponding wall plane to obtain an occupancy grid map. For the wall plane fitted in the specified scene, project the point cloud cluster belonging to the wall plane onto the corresponding wall plane, convert the spatial data from the three-dimensional coordinate system to the two-dimensional coordinate system through projection technology, and generate an occupancy grid map in this process. The occupancy grid map obtained by projection refers to the occupancy grid map, which divides the space into multiple small grids (grids), each grid represents a specific spatial area, and can mark whether these areas are occupied.

[0097] Specifically, the projected point cloud data can be drawn onto the image at a certain resolution to obtain an occupancy grid map. The resolution can be set according to the density of the point cloud. A grid in the occupancy grid map represents a pixel. Resolution refers to the size of a pixel. For example, if the density of the point cloud is one point per 1 cm, and the point cloud is discretized onto the image, the resolution can be set to a pixel size of 1 cm. Project all points in the point cloud onto the plane. If a point is projected onto a grid on the plane, set the pixel corresponding to the grid to white, indicating that the grid has been occupied. After projecting the point cloud cluster on the wall plane, the occupied pixels indicate that the position belongs to the wall surface.

[0098] The occupancy grid obtained after projection is as follows Figure 2As shown in the figure, the black hole area in the occupancy grid map may be caused by the attached components of the wall such as windows, doors, and doorways. Since there is no wall in this area, the black hole area appears. Figure 2 The black hole area with the label number 1 is caused by a window on the wall. In addition, some black areas may also be caused by curtains, radiators, etc. blocking the wall, resulting in no point cloud data being collected on the wall in this area, resulting in a black hole area. Figure 2 The black hole area numbered 2 is caused by the occlusion of the curtain, and the black hole area numbered 3 is caused by the occlusion of the radiator.

[0099] Step S140, detecting the occluded area in the occupied grid image, and performing filling processing on the occluded area; wherein the occluded area is an area where the wall in the specified scene is occluded by the object.

[0100] Due to the presence of obstructions such as curtains, radiators, pendants, and decorative paintings, the wall point cloud may sometimes be incomplete. Black holes will appear in the obstructed areas in the occupancy grid map. In this case, the occupancy grid map needs to be further processed. First, the point cloud data of the space in front of the wall can be detected to determine whether the wall is obstructed by objects. If there are obstructions on the wall, additional processing will be performed on the obstructed area, and the area obstructed by the object will also be filled with white. Figure 2 In the example shown, if it is detected that there is no occlusion on the wall where the black hole area labeled 1 is located, and there are occlusions on the wall where the black hole areas labeled 2 and 3 are located, the two areas corresponding to the areas labeled 2 and 3 are filled with white. After the filling process, only the black area labeled 1 remains in the occupied raster image, which is not generated by the occlusion, but by the attached components of the wall such as windows.

[0101] Step S150, extracting a bounding box of the hole area in the filled occupancy grid map, and determining position information of an attached component of the wall in the specified scene based on the bounding box of the hole area.

[0102] Among them, the bounding box is an algorithm for solving the optimal bounding space of a discrete point set. The basic idea is to use a slightly larger geometric body with simpler characteristics (called a bounding box) to approximately replace complex geometric objects. Figure 2 In the example of FIG. 1 , the rectangular bounding box of the hole area labeled 1 can be extracted from the occupancy grid map after the filling process, and the position information of the window can be determined based on the bounding box of the hole area. Specifically, the area where the rectangular bounding box is located is used as the area where the window is located. The horizontal range of the rectangular bounding box corresponds to the width of the window, and the vertical range corresponds to the height of the window.

[0103] like Figure 3 As shown, the white rectangular border represents the border of the extracted rectangular bounding box, represents the border line of the extracted window, and the entire black area represents the entire wall area where the window is located.

[0104] In addition to windows, the above method can also be used to extract location information for other auxiliary components such as doors and doorways. When collecting point cloud data, the door of the room can be opened. In this way, when processing the point cloud data using the above method, the area where the door is located will produce obvious black holes in the occupancy grid map, making the extraction result more accurate.

[0105] In summary, based on the embodiments of the present disclosure, by projecting the point cloud data in a specified scene into an occupancy grid map, and by detecting the occluded areas blocked by objects in the scene in the occupancy grid map, the interference factors in the extracted information can be effectively filtered out, and the position information of the attached components can be robustly and accurately extracted from the point cloud data, thereby improving the data processing efficiency.

[0106] In one embodiment, detecting the occluded area in the occupancy grid map and filling the occluded area further includes:

[0107] Perform a closing operation on the occupancy grid map;

[0108] The occluded area in the occupancy grid map after the closing operation is filled.

[0109] See also Figure 2 , in the process of projecting the point cloud clusters on the wall plane to obtain the occupancy grid map, the projected point cloud data is drawn onto the image at a certain resolution. If the density of the point cloud is low, some positions on the wall may not be scanned. See Figure 2 , limited by the point cloud density, the white area corresponding to the wall in the occupancy grid map is basically one occupied point every other pixel, and the distribution is relatively sparse. The wall position on the occupancy grid map is not a complete and continuous area. In view of this situation, image morphological operations need to be performed on the image to aggregate the sparse points into a white area.

[0110] For example, you can close an image to fill small holes. Closing is an operation in mathematical morphology, defined as dilation followed by erosion. Specifically, closing is done by dilating the image first and then performing an erosion operation. Closing can eliminate small holes in an image, fill small breaks, and connect narrow gaps. In addition, closing can exclude small black holes (black areas) and usually connect narrow gaps to fill holes that are smaller than the structural element. The result of a closing operation is shown below. Figure 4 Compare Figure 2 and Figure 4 The result of the closing operation is Figure 2All the small black dots in the white area in the image are filled with white. The black hole areas left after filling are those areas where there is no point cloud projection in a large range, that is, Figure 2 and Figure 4 The areas are numbered 1, 2, and 3.

[0111] After the occupancy grid map is closed, the occlusion area in the occupancy grid map can be further detected and filled. Figure 4 On the basis of , the two areas corresponding to the labels 2 and 3 are also filled with white. After filling, we get the following Figure 5 As shown in the image, it can be seen that the black hole area left after filling is the area where the small holes and the occluded area are filled, leaving only the auxiliary components of the wall labeled 1, that is, the area where the window is located. Figure 5 In the occupancy grid map after filling processing shown in FIG. 1 , the bounding box of the hole area can be further extracted. The bounding box after extraction is as follows: Figure 3 As shown, the position information of the auxiliary component can be determined based on the bounding box. It can be seen that through the above filling process, the interference factors in the extracted information are effectively filtered out, based on which the position information of the auxiliary component can be accurately extracted from the point cloud data.

[0112] like Figure 6 As shown, in one embodiment, Figure 1 In step S140, the occluded area in the occupied grid image is detected and filled in, including:

[0113] Step S210, for unoccupied pixel points in the occupancy grid map, obtain the three-dimensional points corresponding to the pixel points in the point cloud data.

[0114] For the unoccupied pixels in the occupancy grid map, the plane information of the wall plane can be used to restore them to the original three-dimensional space, and the three-dimensional points corresponding to the pixels in the point cloud data can be obtained. Specifically, based on the three-dimensional plane equation corresponding to the wall plane, the three-dimensional coordinates of the pixels in the occupancy grid map in the three-dimensional space are obtained, and the positioning information in the three-dimensional space is obtained. In this way, the two-dimensional pixel points in the occupancy grid map are converted into a three-dimensional point in the three-dimensional space.

[0115] Step S220, calculating whether there is point cloud occlusion in the direction of the ray passing through the three-dimensional point; wherein the ray passing through the three-dimensional point is a ray that takes the three-dimensional point as an endpoint and extends into the room along the normal direction of the wall plane.

[0116] Specifically, a ray is emitted from a three-dimensional point into the room along the normal direction of the wall plane, and it is calculated whether there is occlusion of the original point cloud in the direction of the ray.

[0117] Step S230: If there is a point cloud occlusion in the ray direction, the pixel point is filled in the occupancy grid map as an occupied state.

[0118] See also Figure 4 In the two regions corresponding to 2 and 3, it is detected that there is a point cloud occlusion in the direction of the ray emitted from the pixel point as the endpoint, then the pixels in these two regions are filled as occupied in the occupancy grid map, that is, the two regions corresponding to 2 and 3 are also filled with white. After filling, the following is obtained Figure 5 The image shown. Figure 5 In the filled occupancy grid map shown in the figure, the bounding box of the hole area can be further extracted. Figure 5 The boundary line of the black and white areas in the image is drawn, and the bounding box after extraction is as follows Figure 3 As shown, the position information of the auxiliary component can be determined based on the bounding box. It can be seen that by filling the occluded area, the interference factors in the extracted information are effectively filtered out, so that only the auxiliary component is left on the image after filling. Based on this, the position information of the auxiliary component can be accurately extracted from the point cloud data.

[0119] In one embodiment, the above method further comprises:

[0120] Performing plane segmentation processing on the point cloud data, and obtaining point cloud clusters belonging to the top plane and the bottom plane in the specified scene respectively based on the result of the plane segmentation processing;

[0121] Fit the point cloud clusters of the top plane and the bottom plane to obtain the position information of the top plane and the bottom plane;

[0122] Based on the position information of the accessory component, the position information of the top plane and the bottom plane, the relative position information of the accessory component and the top plane, and the relative position information of the accessory component and the bottom plane are determined.

[0123] Among them, the top plane in the specified scene includes the plane where the ceiling of the room is located, and the bottom plane includes the plane where the floor of the room is located. First, the point cloud data collected in the specified scene is processed by plane segmentation using the plane segmentation algorithm of the point cloud, and the points in the point cloud data are assigned to clusters belonging to the same plane, so as to obtain point cloud clusters belonging to the top plane and the bottom plane respectively. Then, the normal, length, and height information of the plane obtained by the plane segmentation process are used to obtain the top plane and the bottom plane in the specified scene. For example, a plane whose normal is along the vertical direction and whose width and length exceed a certain threshold can be determined as a top plane or a bottom plane. Based on the above method, all the top planes or bottom planes in the specified scene are filtered out, and the height range of the room can also be determined based on the position information of the top plane or the bottom plane.

[0124] Furthermore, the position information of the accessory components in the occupancy grid map can be restored to the three-dimensional space to obtain the three-dimensional position information of the accessory components. In one example, the height of the lower edge of the window from the ground and the height of the upper edge of the window from the top of the room can be further calculated in combination with the height information of the ceiling and the ground. The relative position information of the accessory components and the top plane, and the accessory components and the bottom plane obtained above can provide on-site detection data for applications such as wall coloring, maintenance, and installation of accessory components.

[0125] In one example, the distance between the accessory component and the upper and lower edges of the image can also be measured in the occupancy grid map of the two-dimensional projection, so as to obtain the relative position information between the accessory component and the top plane, and the accessory component and the bottom plane. However, the upper edge of the two-dimensional projection image of the wall is not necessarily the location of the ceiling. For example, if there is a wall cabinet above the window, when extracting the point cloud data of the wall plane, only the area below the wall cabinet can be extracted, and the location of the ceiling cannot be extracted. Therefore, calculating the distance from the window to the ceiling in the two-dimensional projection image of the wall point cloud may be inaccurate, and it is necessary to combine the global ceiling information of the entire room to obtain the accurate value of the relative position information between the accessory component and the top plane, and the accessory component and the bottom plane.

[0126] In one embodiment, the above method further comprises:

[0127] Extract edge point cloud data whose distance from an edge line of the attached component is less than a first threshold value according to position information of the attached component of the wall in the specified scene;

[0128] Perform plane extraction on the edge point cloud data to obtain the position information of the edge plane. The edge plane is a plane parallel to the edge line of the attached component and perpendicular to the wall plane.

[0129] According to the position information of the edge plane, the position information of the auxiliary component is adjusted.

[0130] In the above step S150, a set of a series of auxiliary components in the specified scene can be obtained, and the rectangular bounding box corresponding to the auxiliary component can be used to describe its position and size. The vertices of the rectangular bounding box determine the position of the auxiliary component, and the length and width of the rectangular bounding box determine the size of the auxiliary component. On this basis, the size of the auxiliary component can be further optimized and adjusted to make it more accurate. Taking the window as an example, the optimization method is as follows:

[0131] A) Most of the windows are embedded in the wall, and there will be a plane perpendicular to the wall at the edge of the rectangle where the window frame is located, such as the window sill, the side, top, and upper edge of the window. First, according to the edge of the rectangular bounding box obtained in step S150, the point cloud data of a certain range around it is segmented. Specifically, the first threshold can be reasonably set to extract the edge point cloud data whose distance from the edge line of the window bounding box is less than the first threshold.

[0132] B) Extract the plane of the point cloud data extracted in step A to obtain a plane parallel to the edge line of the rectangular bounding box and perpendicular to the wall plane, that is, the edge plane, that is, the plane where the windowsill, the side, top, and upper edge of the window are located.

[0133] C) According to the position information of the edge plane, the position information of the corresponding window is adjusted to obtain a more accurate window size. The rectangular bounding box obtained in step S150 is obtained by projecting the information of the continuous three-dimensional space onto the two-dimensional image. Since the two-dimensional image is discrete, for example, there is one pixel point every 1 cm, its accuracy can reach 1 cm. On this basis, in order to further improve the accuracy, the edge plane where the window frame is located can be extracted from the point cloud data with the help of the original point cloud data, and the position information of the edge plane can be adjusted to the position information of the corresponding window frame to optimize the position and size of the window and further improve the accuracy of the position information.

[0134] Taking the door as an example, there will also be a plane perpendicular to the wall at the edge of the rectangle where the door frame is located, that is, the edge plane, such as the ground, the side, top, and upper edge of the door. The position information of the door can be optimized in the same way, and the above-mentioned window example can be referred to, which will not be repeated here.

[0135] In the above implementation manner, the edge plane of the accessory component is extracted from the original point cloud data to adjust and optimize the extraction result, thereby further improving the accuracy of the position information of the accessory component.

[0136] In one embodiment, the above method further comprises:

[0137] Among multiple wall planes in the specified scene, a wall plane that is parallel to the wall plane and whose distance satisfies a preset condition is determined as the opposite wall plane to the wall;

[0138] According to the position information of the auxiliary component, detecting whether there is an auxiliary component on the opposite side at a corresponding position on the plane of the opposite wall;

[0139] According to the detection result of the auxiliary component on the opposite side, the position information of the auxiliary component is verified.

[0140] Since laser point cloud scanning obtains three-dimensional information of the surface of an object, for the interior walls of a house, there are two matching wall surfaces on both sides of the wall, and these two wall surfaces are opposite wall surfaces. In this case, corresponding to the same accessory component on the wall, two matching accessory components will be detected at the corresponding positions on the two surfaces of the wall through the above steps, and these two accessory components are opposite accessory components. Taking windows as an example, two windows located at the same position on the two sides of the same wall surface will be detected in the set of accessory components obtained through the above processing steps. Based on this situation, the accessory components of the interior wall can be further filtered.

[0141] Specifically, multiple wall planes in the specified scene are matched with each other, and point cloud planes belonging to the two sides of the same wall are found to form a pair. For example, two planes that are parallel to each other and whose distance is approximately equal to the wall thickness are found and determined as point cloud planes on both sides of the same wall. The two wall planes that successfully match each other are the opposite wall surfaces. For a certain accessory component, based on the position information of the accessory component, it is detected whether the opposite accessory component exists at the corresponding position of the opposite wall plane. Based on the detection result of the opposite accessory component, the position information of the accessory component is verified. If the opposite accessory component is detected at the corresponding position of the opposite wall plane, it is considered that the detection result of the accessory component is accurate.

[0142] The disclosed embodiment further verifies the detection results of the accessory components by matching two corresponding accessory components on the point cloud planes on both sides of the same wall, thereby improving the accuracy of the accessory component extraction results.

[0143] In one embodiment, the position information of the accessory component is verified according to the detection result of the accessory component on the opposite side, including:

[0144] If it is detected that there is no opposite-side auxiliary component at the corresponding position of the opposite-side wall plane, it is determined that the position information of the auxiliary component is misjudged information.

[0145] The above misjudgment may be caused by incomplete or missing scans due to holes or mirrors on the wall. In addition, when the point cloud acquisition device performs laser scanning, the scanning angle and orientation are constantly changing, which may cause an area on the wall that is not scanned by the laser point. In this case, according to the above-mentioned auxiliary component extraction method, a hole like a window may be detected in the point cloud data on the wall surface, and this hole may also be misjudged as a window.

[0146] For the interior walls of a house, scanning the entire house can detect both sides of the same wall. If two matching accessory components are detected at the same position on the two side surfaces, it is considered that the detection result of the accessory component is accurate. If only one accessory component is detected on both side surfaces of the same wall, and no matching opposite accessory component is detected at the corresponding position of the opposite window plane, it is considered that the accessory component does not really exist, but is a misjudgment result caused by a hole in the wall.

[0147] The disclosed embodiment matches two corresponding accessory components on the point cloud planes on both sides of the same wall, and further verifies the detection results of the accessory components, which can effectively reduce the misjudgment rate of the accessory component extraction results and improve the accuracy of the extraction results.

[0148] In one embodiment, the position information of the accessory component is verified according to the detection result of the accessory component on the opposite side, including:

[0149] If it is detected that there is an opposite-side auxiliary component at a corresponding position on the opposite-side wall plane, the position information of the auxiliary component is adjusted based on the position information of the opposite-side auxiliary component.

[0150] If two matching accessory components are detected at the same position on the two sides of the wall, the detection result of the accessory component is considered to be accurate. On this basis, the final detection result of the position information can be adjusted based on the position information of the accessory component and the position information of the accessory component on the opposite side. For example, the average value of the position information of the accessory component and the accessory component on the opposite side is calculated, and the average value is used as the final detection result of the position information of the accessory component. The average value of the position information may be closer to the true value, and after adjustment, the detection error can be further reduced and the accuracy can be improved.

[0151] In one embodiment, adjusting the position information of the accessory component based on the position information of the opposite-side accessory component includes:

[0152] The intersection or union of the area where the attached component and the opposite-side attached component are located is adjusted to the area where the attached component is located.

[0153] If two matching accessory components are detected at the same position on the surfaces of both sides of the wall, the intersection or union of the areas where the two accessory components are located can be adjusted to the area where the corresponding accessory component is located. Specifically, the detection results of the position information of the two matching accessory components may not be completely equal, and there may be a relatively small difference between the two. In one example, in order to purchase paint for painting the wall, it is necessary to calculate the area of ​​the wall. In order to avoid insufficient amount of purchased paint, it is necessary to reduce the extracted window area as much as possible to expand the wall area so that the amount of purchased paint is sufficient. In this case, the intersection of the areas where the two matching accessory components are located can be adjusted to the area where the corresponding accessory component is located, so as to achieve the purpose of minimizing the extracted window area and making the amount of purchased paint sufficient, thereby meeting the needs of practical applications.

[0154] The disclosed embodiment further adjusts the detection results of the accessory components by matching two corresponding accessory components on the point cloud planes on both sides of the same wall, which not only improves the accuracy of the accessory component extraction results, but also makes the accessory component extraction results more meet the specific application requirements of actual usage scenarios, and has wide practicality in various specific application scenarios or environments.

[0155] like Figure 7 As shown, the present disclosure also provides a corresponding embodiment of a scene accessory component extraction device based on point cloud data. For the beneficial effects or technical problems solved by the device, please refer to the description of the method corresponding to each device, or refer to the description in the invention content, which will not be repeated here.

[0156] In an embodiment of the scene attachment component extraction device based on point cloud data, the device includes:

[0157] An acquisition unit 100 is used to acquire point cloud data collected in a specified scene;

[0158] The segmentation unit 200 is used to perform plane segmentation processing on the point cloud data, and obtain a point cloud cluster belonging to the wall plane in the specified scene based on the result of the plane segmentation processing;

[0159] A projection unit 300 is used to project the point cloud cluster of the wall plane onto the corresponding wall plane to obtain an occupancy grid map;

[0160] The filling unit 400 is used to detect the occluded area in the occupancy grid map and fill the occluded area; wherein the occluded area is the area where the wall in the specified scene is occluded by the object;

[0161] The extraction unit 500 is used to extract the bounding box of the hole area in the occupancy grid map after the filling process, and determine the position information of the auxiliary components of the wall in the specified scene based on the bounding box of the hole area.

[0162] In one embodiment, the filling unit 400 is further used for:

[0163] Perform a closing operation on the occupancy grid map;

[0164] The occluded area in the occupancy grid map after the closing operation is filled.

[0165] In one embodiment, the filling unit 400 is used to:

[0166] For unoccupied pixels in the occupancy grid map, obtain the 3D points corresponding to the pixels in the point cloud data;

[0167] Calculate whether there is point cloud occlusion in the direction of the ray passing through the three-dimensional point; wherein the ray passing through the three-dimensional point is a ray that takes the three-dimensional point as an endpoint and extends into the room along the normal direction of the wall plane;

[0168] If there is a point cloud occlusion in the direction of the ray, the pixel point in the occupancy grid map is filled as occupied.

[0169] like Figure 8 As shown, in one embodiment, the above device further includes a processing unit 600, which is used to:

[0170] Performing plane segmentation processing on the point cloud data, and obtaining point cloud clusters belonging to the top plane and the bottom plane in the specified scene respectively based on the result of the plane segmentation processing;

[0171] Fit the point cloud clusters of the top plane and the bottom plane to obtain the position information of the top plane and the bottom plane;

[0172] Based on the position information of the accessory component, the position information of the top plane and the bottom plane, the relative position information of the accessory component and the top plane, and the relative position information of the accessory component and the bottom plane are determined.

[0173] In one embodiment, the above device further includes an adjustment unit 700, which is used to:

[0174] Extract edge point cloud data whose distance from an edge line of the attached component is less than a first threshold value according to position information of the attached component of the wall in the specified scene;

[0175] Perform plane extraction on the edge point cloud data to obtain the position information of the edge plane. The edge plane is a plane parallel to the edge line of the attached component and perpendicular to the wall plane.

[0176] According to the position information of the edge plane, the position information of the auxiliary component is adjusted.

[0177] like Figure 8 and Fig. 9As shown, in one embodiment, the above device further includes a testing unit 800, and the testing unit 800 includes:

[0178] The matching subunit 810 is used to determine, among multiple wall planes in a specified scene, a wall plane that is parallel to the wall plane and whose distance satisfies a preset condition as the opposite wall plane of the wall;

[0179] The detection subunit 820 is used to detect whether there is an opposite side auxiliary component at a corresponding position on the opposite side wall plane according to the position information of the auxiliary component;

[0180] The verification subunit 830 is used to verify the position information of the auxiliary component according to the detection result of the auxiliary component on the opposite side.

[0181] In one implementation, the syndrome unit 830 is used to:

[0182] If it is detected that there is no opposite-side auxiliary component at the corresponding position of the opposite-side wall plane, it is determined that the position information of the auxiliary component is misjudged information.

[0183] In one implementation, the syndrome unit 830 is used to:

[0184] If it is detected that there is an opposite-side auxiliary component at a corresponding position on the opposite-side wall plane, the position information of the auxiliary component is adjusted based on the position information of the opposite-side auxiliary component.

[0185] In one implementation, the syndrome unit 830 is used to:

[0186] If it is detected that there is an opposite side auxiliary component at a corresponding position of the opposite side wall plane, the intersection or union of the auxiliary component and the area where the opposite side auxiliary component is located is adjusted to the area where the auxiliary component is located.

[0187] The scene accessory component extraction device based on point cloud data in the embodiment of the present disclosure corresponds to each other in specific implementation and beneficial technical effects with respect to the above-mentioned scene accessory component extraction method embodiments based on point cloud data in the present disclosure, and the relevant contents can be referenced to each other and will not be repeated here.

[0188] Below, reference Fig.10 The electronic device according to the embodiment of the present disclosure is described. The electronic device may be any one or both of the first device and the second device, or a stand-alone device independent of them, and the stand-alone device may communicate with the first device and the second device to receive the collected input signals from them.

[0189] Fig.10 A block diagram of an electronic device according to an embodiment of the present disclosure is illustrated.

[0190] like Fig.10As shown, the electronic device includes one or more processors and memory.

[0191] The processor may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0192] The memory may store one or more computer program products, and the memory may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program products may be stored on a computer-readable storage medium, and the processor may run the computer program products to implement the scene-attached component extraction method based on point cloud data and / or other desired functions of the various embodiments of the present disclosure described above.

[0193] In one example, the electronic device may further include: an input device and an output device, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0194] In addition, the input device may also include, for example, a keyboard, a mouse, and the like.

[0195] The output device can output various information to the outside, including the determined distance information, direction information, etc. The output device can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0196] Of course, to simplify, Fig.10 Only some of the components related to the present disclosure in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device may further include any other appropriate components.

[0197] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the scene-attached component extraction method based on point cloud data according to various embodiments of the present disclosure described in the above part of this specification.

[0198] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0199] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the scene accessory component extraction method based on point cloud data according to various embodiments of the present disclosure described in the above part of this specification.

[0200] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0201] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limiting. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0202] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0203] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0204] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0205] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0206] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0207] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A scene accessory component extraction method based on point cloud data, characterized in that: include: Get the point cloud data collected in the specified scene; Performing plane segmentation processing on the point cloud data, and obtaining a point cloud cluster belonging to the wall plane in the specified scene based on the result of the plane segmentation processing; Projecting the point cloud cluster of the wall plane onto the corresponding wall plane to obtain an occupancy grid map; Detecting an occluded area in the occupancy grid map and performing a filling process on the occluded area; wherein the occluded area is an area where a wall in the specified scene is occluded by an object; A bounding box of the hole area is extracted from the occupancy grid map after the filling process, and position information of the attached components of the wall in the specified scene is determined based on the bounding box of the hole area.

2. The method according to claim 1, characterized in that The detecting the blocked area in the occupancy grid map and filling the blocked area further includes: Performing a closing operation on the occupancy grid map; Fill the occluded area in the occupancy grid map after the closing operation.

3. The method according to claim 1, characterized in that The detecting the occluded area in the occupancy grid map and performing filling processing on the occluded area includes: For unoccupied pixel points in the occupancy grid map, obtaining the three-dimensional point corresponding to the pixel point in the point cloud data; Calculate whether there is point cloud occlusion in the direction of the ray passing through the three-dimensional point; wherein the ray passing through the three-dimensional point is a ray that takes the three-dimensional point as an endpoint and extends into the room along the normal direction of the wall plane; If there is a point cloud occlusion in the ray direction, the pixel point in the occupancy grid map is filled as an occupied state.

4. The method according to claim 1, characterized in that: The method further comprises: Performing plane segmentation processing on the point cloud data, and obtaining point cloud clusters respectively belonging to the top plane and the bottom plane in the specified scene based on the result of the plane segmentation processing; Fitting the point cloud clusters of the top plane and the bottom plane to obtain position information of the top plane and the bottom plane; Based on the position information of the accessory member, the position information of the top plane and the bottom plane, the relative position information of the accessory member and the top plane, and the relative position information of the accessory member and the bottom plane are determined.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Extracting edge point cloud data whose distance from an edge line of the attached component is less than a first threshold value according to position information of the attached component of the wall in the specified scene; Performing plane extraction on the edge point cloud data to obtain position information of an edge plane, wherein the edge plane is a plane parallel to the edge line of the accessory component and perpendicular to the wall plane; The position information of the auxiliary component is adjusted according to the position information of the edge plane.

6. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Among the multiple wall planes in the specified scene, a wall plane that is parallel to the wall plane and whose distance satisfies a preset condition is determined as the opposite wall plane to the wall; According to the position information of the auxiliary component, detecting whether there is an auxiliary component on the opposite side at a corresponding position on the plane of the opposite wall; The position information of the accessory component is verified according to the detection result of the accessory component on the opposite side.

7. The method according to claim 6, characterized in that The step of verifying the position information of the accessory component according to the detection result of the accessory component on the opposite side includes: If it is detected that there is no opposite-side auxiliary component at the corresponding position of the opposite-side wall plane, it is determined that the position information of the auxiliary component is misjudged information.

8. The method according to claim 6, characterized in that The step of verifying the position information of the accessory component according to the detection result of the accessory component on the opposite side includes: If it is detected that there is an opposite-side auxiliary component at a corresponding position of the opposite-side wall plane, the position information of the auxiliary component is adjusted based on the position information of the opposite-side auxiliary component.

9. The method according to claim 8, characterized in that The adjusting the position information of the auxiliary component based on the position information of the auxiliary component on the opposite side includes: The intersection or union of the regions where the accessory component and the opposite-side accessory component are located is adjusted to be the region where the accessory component is located.

10. A scene accessory component extraction device based on point cloud data, characterized in that: include: An acquisition unit, used for acquiring point cloud data collected in a specified scene; a segmentation unit, configured to perform plane segmentation processing on the point cloud data, and obtain a point cloud cluster belonging to the wall plane in the specified scene based on a result of the plane segmentation processing; A projection unit, used for projecting the point cloud cluster of the wall plane onto the corresponding wall plane to obtain an occupancy grid map; A filling unit, used to detect the occluded area in the occupancy grid map and fill the occluded area; wherein the occluded area is an area where the wall in the specified scene is occluded by an object; The extraction unit is used to extract the bounding box of the hole area in the occupancy grid map after the filling process, and determine the position information of the auxiliary components of the wall in the specified scene based on the bounding box of the hole area.

Images