Point cloud pose optimization method and device, electronic equipment, medium and program product

Abstract

The embodiment of the invention discloses a point cloud pose optimization method and device, electronic equipment, a medium and a program product, and the method comprises the steps: carrying out the clustering and blocking of point cloud frames of rooms and transition regions in a target space, independently building point-surface matching pairs for each room and transition region based on the point cloud frames, and constraining the pose of each room and transition region, and finally, carrying out overall pose optimization on the room and the transition area which are respectively optimized once again, and obtaining the point cloud of the target space after optimization is completed. According to the point cloud pose optimization method, the room and the transition area are used as independent whole bodies to carry out point cloud pose optimization, so that a part with a large error and a part with a small error in the point cloud of the whole target space can be processed separately, the point cloud pose optimization precision is improved, and the problem of wrong matching such as wall thickening, wall two-side crossing and overlapping and wall layering and aliasing is solved.

Description

Technical Field

[0001] The present disclosure relates to point cloud processing technology, and in particular to a point cloud pose optimization method, device, electronic device, medium and program product. Background Art

[0002] During the point cloud data collection process by laser radar and other point cloud acquisition devices, due to the large noise of laser short-range ranging, room transitions, thin walls and other scene structure limitations, the point cloud products of indoor SLAM (Simultaneous Localization and Mapping) systems based on laser point clouds often show phenomena such as thickened walls, cross-walls, and wall stratification and aliasing. Figure 1 The marked positions 1 and 2 shown in the figure are both the phenomenon of the two sides of the wall crossing and fighting.

[0003] In related technologies, point cloud optimization methods based on structured information such as planes are used to solve problems such as wall thickening, intersection of two walls, and wall stratification and aliasing. However, this method cannot fundamentally solve problems such as wall intersection and aliasing. The optimized point cloud still contains erroneous wall structure information such as wall thickening, intersection of two walls, and wall stratification and aliasing. Summary of the Invention

[0004] In order to solve the technical problems in the related art, the embodiments of the present disclosure provide a point cloud pose optimization method, device, electronic device, medium and program product.

[0005] According to a first aspect of an embodiment of the present disclosure, a point cloud pose optimization method is provided, the method comprising:

[0006] Determining a first local point cloud frame and a second local point cloud frame within the target space based on the first overall point cloud data of the target space, wherein the first local point cloud frame is point cloud frame data of any room within the target space, and the second local point cloud frame is point cloud frame data of a transition area between rooms within the target space;

[0007] Establishing a first point-surface matching pair based on the first partial point cloud frame, optimizing the point cloud pose of the room to which the first partial point cloud frame belongs based on the first point-surface matching pair, and merging the optimized first partial point cloud frames to obtain a third partial point cloud of the room;

[0008] Establishing a second point-surface matching pair based on the second partial point cloud frame, optimizing the point cloud pose of a transition region between rooms to which the second partial point cloud frame belongs based on the second point-surface matching pair, and merging the optimized second partial point cloud frames to obtain a fourth partial point cloud of the transition region;

[0009] performing point cloud splicing and fusion on the third partial point cloud of the room in the target space and the fourth partial point cloud of the transition area, and establishing a third point-surface matching pair, performing point cloud pose optimization on the room to which the third partial point cloud belongs and the transition area to which the fourth partial point cloud belongs based on the third point-surface matching pair, and merging the optimized third partial point cloud and fourth partial point cloud to obtain second overall point cloud data of the target space;

[0010] The first point-plane matching pair, the second point-plane matching pair, and the third point-plane matching pair all represent matching relationships between multiple point clouds and the planes to which they belong.

[0011] As an optional embodiment, determining the first local point cloud frame and the second local point cloud frame in the target space according to the first overall point cloud data of the target space includes:

[0012] Calculating a normal vector for each point cloud in the point cloud frame of the target space to obtain a normal vector for each point cloud;

[0013] Merging the point clouds with normal vectors to obtain first overall point cloud data of the target space;

[0014] Performing plane segmentation and recognition processing on the first overall point cloud data based on the normal vector of the point cloud to obtain the wall area and the door opening area of the target space;

[0015] The first local point cloud frame and the second local point cloud frame are obtained according to the point clouds of the wall area and the door opening area, respectively.

[0016] As an optional embodiment, establishing a first point-surface matching pair based on the first partial point cloud frame, optimizing the point cloud pose of the room to which the first partial point cloud frame belongs based on the first point-surface matching pair, and merging the optimized first partial point cloud frames to obtain a third partial point cloud of the room includes:

[0017] Searching for multiple target neighborhood points of a target point cloud through a preset search tree, wherein the target point cloud is any point cloud on a first point cloud frame in the first partial point cloud frame, and the multiple target neighborhood points are multiple point clouds on a second point cloud frame in the first partial point cloud frame;

[0018] Determining whether the target point cloud and its multiple target neighborhood points belong to the same point cloud plane;

[0019] When it is determined that the target point cloud and its multiple target neighborhood points belong to the same point cloud plane, determining that there is an overlapping area between the first point cloud frame and the second point cloud frame;

[0020] In a case where there is an overlapping area between the first point cloud frame and the second point cloud frame, establishing a point cloud-point cloud plane matching relationship between the multiple point clouds and the point cloud plane to which the multiple point clouds belong, to obtain the first point-plane matching pair, wherein the multiple point clouds include the target point cloud and its multiple target neighborhood points;

[0021] Adjusting the poses of the first point cloud frame and the second point cloud frame by using the first point-plane matching pair so that the first point-plane matching pair satisfies a first preset optimization condition;

[0022] Merging the first point-surface matching pairs that meet the first preset optimization condition to obtain a global point-surface matching pair of the room;

[0023] Adjusting the pose of the point cloud frame in the first local point cloud frame of the room by using a preset point cloud optimizer and the global point-surface matching pair so that the global point-surface matching pair satisfies a second preset optimization condition;

[0024] The point cloud frames in the first local point cloud frame of the room corresponding to the global point-surface matching pair satisfying the second preset optimization condition are merged, and the point cloud frames in the first local point cloud frame of the room corresponding to the merged global point-surface matching pair satisfying the second preset optimization condition are determined as the second local point cloud of the room.

[0025] As an optional embodiment, establishing a first point-surface matching pair based on the first partial point cloud frame, optimizing the point cloud pose of the room to which the first partial point cloud frame belongs based on the first point-surface matching pair, and merging the optimized first partial point cloud frames to obtain a third partial point cloud of the room further includes:

[0026] Before adjusting the poses of the first point cloud frame and the second point cloud frame by using the first point-plane matching pair so that the first point-plane matching pair satisfies a first preset optimization condition, determining whether the total number of the first partial point cloud frames exceeds a preset number of point cloud frames;

[0027] When the total number of the first partial point cloud frames exceeds a preset number of point cloud frames, dividing the first partial point cloud frames into point cloud frames of a plurality of subspaces;

[0028] For the point cloud frame of each subspace, the step of adjusting the posture of the first point cloud frame and the second point cloud frame through the first point-surface matching pair is performed so that the first point-surface matching pair satisfies the first preset optimization condition and the subsequent steps, so as to optimize the point cloud posture of the room to which the first local point cloud frame belongs, and merge the optimized first point cloud frame and the second point cloud frame of each subspace to obtain the third local point cloud of the room.

[0029] As an optional embodiment, searching for multiple target neighborhood points of the target point cloud through a preset search tree includes:

[0030] For the second point cloud frame in the first partial point cloud frame, establishing a preset search tree for the first point cloud frame in the first partial point cloud frame;

[0031] Traversing each point cloud in the first point cloud frame in the first partial point cloud frame, and calculating a dot product of a normal vector between the point cloud and each point cloud in the second point cloud frame in the first partial point cloud frame;

[0032] For each point cloud on the second point cloud frame in the first partial point cloud frame, determining whether the normal vector dot product meets a preset dot product condition;

[0033] Determining a point cloud on the second point cloud frame in the first partial point cloud frame, whose normal vector dot product satisfies a preset dot product condition, as a target search point cloud;

[0034] The target searched point cloud is searched through the preset search tree to determine the target searched point cloud that meets the search conditions of the preset search tree as the target area point.

[0035] As an optional embodiment, adjusting the poses of the first point cloud frame and the second point cloud frame by using the first point-plane matching pair so that the first point-plane matching pair satisfies a first preset optimization condition includes:

[0036] Coordinate rotation and / or coordinate translation are iteratively performed on the first point cloud frame and the second point cloud frame until the distance between the point cloud in the first point-plane matching pair and its point cloud plane meets a first preset optimization condition.

[0037] As an optional embodiment, the third partial point cloud of the room in the target space and the fourth partial point cloud of the transition area are subjected to point cloud stitching and fusion, and a third point-surface matching pair is established. Based on the third point-surface matching pair, point cloud pose optimization is performed on the room to which the third partial point cloud belongs and the transition area to which the fourth partial point cloud belongs, and the optimized third partial point cloud and fourth partial point cloud are combined to obtain the second overall point cloud data of the target space, including:

[0038] performing point cloud stitching and fusion on the third partial point cloud of the room in the target space and the fourth partial point cloud of the transition area, so that the optimized first point-surface matching pair of the third partial point cloud and the optimized second point-surface matching pair of the fourth partial point cloud are combined to form the third point-surface matching pair;

[0039] Coordinate rotation and / or coordinate translation are performed on the point cloud frames belonging to the room and the point cloud frames belonging to the transition area between the rooms in the second overall point cloud data respectively until the distance between the point cloud in the third point-surface matching pair and its point cloud plane meets the third preset optimization condition, thereby obtaining the second overall point cloud data of the target space.

[0040] According to a second aspect of an embodiment of the present disclosure, a point cloud pose optimization device is provided, the device comprising:

[0041] a local point cloud frame determination module, configured to determine a first local point cloud frame and a second local point cloud frame within the target space based on the first overall point cloud data of the target space, wherein the first local point cloud frame is point cloud frame data of any room within the target space, and the second local point cloud frame is point cloud frame data of a transition area between rooms within the target space;

[0042] a room pose optimization module, configured to establish a first point-surface matching pair based on the first partial point cloud frame, optimize the point cloud pose of the room to which the first partial point cloud frame belongs based on the first point-surface matching pair, and merge the optimized first partial point cloud frames to obtain a third partial point cloud of the room;

[0043] a transition region pose optimization module, configured to establish a second point-surface matching pair based on the second partial point cloud frame, optimize the point cloud pose of the transition region between the rooms to which the second partial point cloud frame belongs based on the second point-surface matching pair, and merge the optimized second partial point cloud frames to obtain a fourth partial point cloud of the transition region;

[0044] a spatial pose optimization module, configured to perform point cloud splicing and fusion on the third partial point cloud of the room in the target space and the fourth partial point cloud of the transition area, and establish a third point-surface matching pair, to optimize the point cloud pose of the room to which the third partial point cloud belongs and the transition area to which the fourth partial point cloud belongs based on the third point-surface matching pair, and to merge the optimized third partial point cloud and fourth partial point cloud to obtain second overall point cloud data of the target space;

[0045] The first point-plane matching pair, the second point-plane matching pair, and the third point-plane matching pair all represent matching relationships between multiple point clouds and the planes to which they belong.

[0046] As an optional embodiment, the local point cloud frame determination module includes:

[0047] a normal calculation unit, configured to calculate a normal vector for each point cloud in the point cloud frame of the target space to obtain a normal vector for each point cloud;

[0048] a point cloud merging unit, configured to merge the point clouds having normal vectors to obtain first overall point cloud data of the target space;

[0049] a segmentation and recognition unit, configured to perform plane segmentation and recognition processing on the first overall point cloud data based on a normal vector of the point cloud to obtain a wall area and a door opening area of the target space;

[0050] The first point cloud frame determining unit is configured to obtain the first local point cloud frame and the second local point cloud frame according to the point clouds of the wall area and the door opening area, respectively.

[0051] As an optional embodiment, the room posture optimization module includes:

[0052] a target neighborhood point search unit, configured to search for a plurality of target neighborhood points of a target point cloud through a preset search tree, wherein the target point cloud is any point cloud on a first point cloud frame in the first local point cloud frame, and the plurality of target neighborhood points are a plurality of point clouds on a second point cloud frame in the first local point cloud frame;

[0053] A same-plane judgment unit is used to judge whether the target point cloud and its multiple target neighboring points belong to the same point cloud plane;

[0054] An overlapping region determining unit, configured to determine whether an overlapping region exists between the first point cloud frame and the second point cloud frame when it is determined that the target point cloud and its multiple target neighborhood points belong to the same point cloud plane;

[0055] a first point-plane matching unit, configured to, when there is an overlapping area between the first point cloud frame and the second point cloud frame, establish a matching relationship between the multiple point clouds and the point cloud plane to which the multiple point clouds belong, to obtain the first point-plane matching pair, wherein the multiple point clouds include the target point cloud and its multiple target neighborhood points;

[0056] a first pose adjustment unit, configured to adjust the poses of the first point cloud frame and the second point cloud frame by using the first point-plane matching pair, so that the first point-plane matching pair satisfies a first preset optimization condition;

[0057] a second point-plane matching unit, configured to merge the first point-plane matching pairs that satisfy the first preset optimization condition to obtain a global point-plane matching pair of the room;

[0058] A second pose adjustment unit is configured to adjust the pose of the point cloud frame in the first local point cloud frame of the room by using a preset point cloud optimizer and the global point-surface matching pair, so that the global point-surface matching pair satisfies a second preset optimization condition;

[0059] The second point cloud frame determination unit is used to merge the point cloud frames in the first local point cloud frame of the room corresponding to when the global point-surface matching pair meets the second preset optimization condition, and determine the point cloud frame in the first local point cloud frame of the room corresponding to when the merged global point-surface matching pair meets the second preset optimization condition as the third local point cloud of the room.

[0060] As an optional embodiment, the room posture optimization module further includes:

[0061] a point cloud quantity determination unit, configured to determine whether the total number of the first partial point cloud frames exceeds a preset number of point cloud frames before adjusting the poses of the first point cloud frame and the second point cloud frame by using the first point-surface matching pair so that the first point-surface matching pair satisfies a first preset optimization condition;

[0062] a space division unit, configured to divide the first local point cloud frame into a plurality of subspace point cloud frames when the total number of the first local point cloud frames exceeds a preset number of point cloud frames;

[0063] The second pose adjustment unit is used to perform the step of adjusting the poses of the first point cloud frame and the second point cloud frame through the first point-surface matching pair for the point cloud frame of each subspace, so that the first point-surface matching pair satisfies the first preset optimization condition and its subsequent steps, so as to optimize the point cloud pose of the room to which the first local point cloud frame belongs, and merge the optimized first point cloud frame and the second point cloud frame of each subspace to obtain the second local point cloud frame of the room.

[0064] As an optional embodiment of the present disclosure, the target neighborhood point search unit includes:

[0065] A search tree establishing subunit, configured to establish a preset search tree for the first point cloud frame in the first partial point cloud frame with respect to the second point cloud frame in the first partial point cloud frame;

[0066] a normal vector dot product calculation subunit, configured to traverse each point cloud in a first point cloud frame in the first partial point cloud frame, and calculate a normal vector dot product between the point cloud and each point cloud in a second point cloud frame in the first partial point cloud frame;

[0067] a dot product condition judging subunit, configured to judge, for each point cloud on the second point cloud frame in the first partial point cloud frame, whether the dot product of the normal vector satisfies a preset dot product condition;

[0068] a searched point cloud determining subunit, configured to determine a point cloud on the second point cloud frame in the first partial point cloud frame, for which the normal vector dot product satisfies a preset dot product condition, as a target searched point cloud;

[0069] The target domain point determination subunit is configured to search the target searched point cloud through the preset search tree, so as to determine the searched target searched point cloud that meets the search conditions of the preset search tree as the target domain point.

[0070] As an optional embodiment of the present disclosure, the first posture adjustment unit includes:

[0071] The posture adjustment subunit is used to iteratively perform coordinate rotation and / or coordinate translation on the first point cloud frame and the second point cloud frame until the distance between the point cloud in the first point-plane matching pair and its point cloud plane meets a first preset optimization condition.

[0072] As an optional embodiment of the present disclosure, the spatial posture optimization module includes:

[0073] a point-surface matching pair acquisition unit, configured to perform point cloud splicing and fusion on the third partial point cloud of the room in the target space and the fourth partial point cloud of the transition area, so that the optimized first point-surface matching pair of the third partial point cloud and the optimized second point-surface matching pair of the fourth partial point cloud are combined to form the third point-surface matching pair;

[0074] The third posture adjustment unit is used to perform coordinate rotation and / or coordinate translation on the point cloud frames belonging to the room and the point cloud frames belonging to the transition area between the rooms in the second overall point cloud data, respectively, until the distance between the point cloud in the third point-surface matching pair and its point cloud plane meets the third preset optimization condition, thereby obtaining the second overall point cloud data of the target space.

[0075] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0076] a memory for storing a computer program product;

[0077] The processor is configured to execute the computer program product stored in the memory, and when the computer program product is executed, the method described in the first aspect above is implemented.

[0078] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described in the first aspect above is implemented.

[0079] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising computer program instructions, which implement the method described in the first aspect when executed by a processor.

[0080] The technical solution provided by the embodiments of the present disclosure optimizes the internal posture of the room by clustering and blocking the point cloud frames of the rooms and transition areas in the target space, and then establishing point-surface matching pairs based on the point cloud frames to constrain the internal posture of each room; the transition areas between the rooms are also optimized as a whole in a similar room-like manner, and finally the optimized rooms and transition areas are subjected to another overall posture optimization to obtain the optimized point cloud of the target space. Most of the mismatches such as wall thickening, wall cross-fighting, wall layering and aliasing are caused by the transition area between the rooms. Therefore, in the present disclosure, performing point cloud pose optimization on the transition area separately is beneficial to separate the parts with large errors and the parts with small errors in the point cloud of the entire target space, thereby improving the accuracy of point cloud pose optimization and solving the problems of mismatches such as wall thickening, wall cross-fighting, wall layering and aliasing; and clustering and blocking the point cloud frames of the rooms and transition areas in the target space and then performing point cloud pose optimization separately reduces the overall scale of the target space optimization at one time, and the optimization between the rooms and transition areas can also be processed in parallel, which greatly improves the processing performance and resource overhead of the algorithm.

[0081] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0083] The present disclosure can be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0084] Figure 1 A schematic diagram of indoor walls intersecting and overlapping with each other provided in an embodiment of the present disclosure.

[0085] Figure 2 This is one of the flowcharts of a point cloud pose optimization method according to an embodiment of the disclosed method.

[0086] Figure 3 This is the second flowchart of a point cloud pose optimization method according to an embodiment of the disclosed method.

[0087] Figure 4 This is the third flowchart of a point cloud pose optimization method according to an embodiment of the disclosed method.

[0088] Figure 5 This is a schematic diagram of the point cloud frame pose points of a room and a transition area in one embodiment of the method disclosed herein.

[0089] Figure 6This is a top view of the target space after point cloud optimization is completed in one embodiment of the method disclosed herein.

[0090] Figure 7 This is a structural block diagram of a point cloud pose optimization device according to an embodiment of the present invention.

[0091] Figure 8 This is a structural block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0092] When laser radar and other point cloud acquisition devices collect point cloud data of target spaces such as houses, the overall structure of the house is relatively complex, which may include multiple subspaces such as rooms, as well as transition areas between rooms such as doorways, etc. In addition, the thickness of the walls of the rooms varies, and some may be too thin. These factors will cause the point cloud products of the target space to have phenomena such as thickened walls, intersections of two walls, and overlapping of wall seals. Figure 1 As shown in the figure, the positions enclosed by the dotted boxes "1" and "2" are two walls of a wall between two rooms. The two walls are intertwined at the positions enclosed by the dotted boxes "1" and "2", making it difficult to distinguish between the inner wall and the outer wall at these two positions.

[0093] To solve the technical problems in the related art, the present disclosure provides a point cloud pose optimization method, apparatus, electronic device, computer-readable storage medium, and computer program product. The technical solutions of the present disclosure are described in detail below with reference to the accompanying drawings.

[0094] Figure 2 This is one of the flow charts of a point cloud pose optimization method according to an embodiment of the disclosed method. Figure 2 As shown, a point cloud pose optimization method according to an embodiment of the present disclosure may include the following steps:

[0095] Step 201 : determining a first partial point cloud frame and a second partial point cloud frame in the target space according to first overall point cloud data of the target space.

[0096] For example, the target space is a house with multiple rooms and doorways between them. The first partial point cloud frame is the point cloud frame data for any room within the target space, while the second partial point cloud frame is the point cloud frame data for the transition area between the rooms within the target space. This allows the point cloud frame data for each room to be separated from the point cloud frame data for doorways (transition areas) for separate processing in subsequent point cloud optimization, reducing the processing overhead of the point cloud frame data and improving processing efficiency.

[0097] As an embodiment of the present disclosure, step 201 can be further implemented through the following steps: first, a normal vector is calculated for each point cloud in the point cloud frame of the target space to obtain a normal vector for each point cloud; then, the point clouds with normal vectors are merged to obtain a first overall point cloud data of the target space; then, plane segmentation and recognition processing is performed on the first overall point cloud data based on the normal vector of the point cloud to obtain a wall area and a door opening area of the target space; finally, a first local point cloud frame and a second local point cloud frame are obtained according to the point clouds of the wall area and the door opening area, respectively. Among them, the normal vector of each point cloud can be calculated, for example, by the geometric relationship between the points in the point cloud. For example, for a point cloud M in any point cloud frame, the domain point set P(M) of M can be determined by neighborhood point search (for example, KD tree, octree), and then the vector between each point in the domain point set P(M) and M is calculated, and the calculated vector is then vector averaged, and the average vector obtained after averaging is normalized, and finally the normalized average normal vector of the point cloud of each point cloud frame is obtained, and used as the normal vector of each point cloud in the point cloud frame. It can be understood that the calculation method of the point cloud normal vector proposed in the above embodiment is only an example, and the present disclosure does not limit the calculation method of the point cloud normal vector.

[0098] In the disclosed embodiment, the point cloud frame of the target space may be composed of tens of thousands of point cloud frames, and each frame of point cloud may include a large number of point clouds (for example, more than 20,000 point clouds), and then the normal vector of each point is calculated, and then the point cloud frames with normal vector point clouds are merged together to obtain the overall point cloud with normal vector of the target space (i.e., the first overall point cloud data). Furthermore, the point cloud data of the walls and door openings of the target space can be identified by, for example, plane segmentation methods (such as region growing methods or other methods that can achieve flat point cloud plane segmentation) and wall and door opening recognition methods (such as some machine learning models that can achieve shape or object detection, etc.), so that the point cloud frame of the room in the target space can be obtained based on the point cloud data of the wall (i.e., the first local point cloud frame), and the point cloud frame of the transition area (i.e., the second local point cloud frame) can be obtained based on the point cloud data of the door opening. As Figure 5 As shown in the figure, the position marked "1" is the pose point of the point cloud frame in the transition area, and the area formed by the remaining lines is the pose point of the point cloud frame in each room. The transition area is the trajectory connection of the point cloud frame data, which connects the trajectories of the point clouds of each room together to form the point cloud trajectory of the entire target space.

[0099] In step 202 , a first point-surface matching pair is established based on the first partial point cloud frame, point cloud pose optimization is performed on the room to which the first partial point cloud frame belongs based on the first point-surface matching pair, and the optimized first partial point cloud frame is merged to obtain a third partial point cloud of the room.

[0100] In the previous step, the point cloud data of the room, i.e., the first local point cloud frame, is obtained. In this step, the pose optimization of the room point cloud is achieved by performing point cloud optimization on the first local point cloud frame.

[0101] Specifically, Figure 3 This is a flowchart of a point cloud pose optimization method according to an embodiment of the present disclosure, see Figure 3 As shown, step 202 can be specifically implemented as follows:

[0102] In step 2021, a preset search tree is used to search for multiple target neighborhood points of the target point cloud, where the target point cloud is any point cloud on the first point cloud frame in the first local point cloud frame, and the multiple target neighborhood points are multiple point clouds on the second point cloud frame in the first local point cloud frame.

[0103] In the first local point cloud frame, a neighborhood point search can be performed between any two point cloud frames to further determine whether there is an overlapping area between the point cloud frames in the first local point cloud frame. Specifically, first, a preset search tree of the first point cloud frame in the first local point cloud frame can be established for the second point cloud frame in the first local point cloud frame, and then each point cloud in the first point cloud frame in the first local point cloud frame is traversed to calculate the normal vector dot product of the point cloud and each point cloud on the second point cloud frame in the first local point cloud frame. Then, for each point cloud on the second point cloud frame in the first local point cloud frame, it is determined whether the normal vector dot product meets the preset dot product condition. Then, the point cloud on the second point cloud frame in the first local point cloud frame whose normal vector dot product meets the preset dot product condition is determined as the target searched point cloud. Finally, the target searched point cloud is searched through the preset search tree to determine the target searched point cloud that meets the search condition of the preset search tree as the target domain point. The preset search tree can be a KD tree (K-dimensional tree). For example, the KD tree can be used to search for several neighborhood matching points (i.e., target neighborhood points) of each point in a certain frame point cloud (e.g., the aforementioned first point cloud frame) on another frame (e.g., the aforementioned second point cloud frame). As an optional embodiment, when using the KD tree search, it is considered that the wall is two-sided (including the outer wall and the inner wall), that is, the point cloud data of the outer wall and the inner wall are included, and the directions of the normal vectors of the point clouds of the outer wall and the inner wall are opposite. The point cloud trajectory is collected from outside the door opening to inside the door opening. If the point cloud scanned outside the door opening is already in the KD tree, when searching for neighborhood points through the KD tree, it is easy to match the point cloud that does not belong to the wall, thereby causing incorrect matching of neighborhood points, and then leading to incorrect matching of subsequent wall point clouds. Therefore, the embodiment of the present disclosure uses normal vector constraints, that is, only the point cloud on the second point cloud frame in the first local point cloud frame, whose normal vector dot product meets the preset dot product condition, is determined as the target searched point cloud. In this way, it is ensured that the point cloud of the inner wall is only matched with the point cloud of the inner wall for neighborhood points, and the point cloud of the outer wall is only matched with the point cloud of the outer wall for neighborhood points, thereby improving the matching accuracy of the neighborhood points and providing a more accurate data basis for subsequent posture optimization. As another optional embodiment, since the point cloud acquisition trajectory is more likely to cause incorrect matching of the point cloud of the exterior wall, the neighborhood point search of the point cloud of the exterior wall can also be constrained by the normal vector. For example, the point cloud of the exterior wall only searches for the point cloud whose normal vector dot product in the exterior wall meets the preset dot product condition.Furthermore, the preset dot product condition is, for example, that the normal vector dot product must be greater than zero. For example, when searching for the target neighborhood point of point cloud point A on the first point cloud frame through the KD tree, assuming that only seven points B, C, D, E, F, G, and H on the second point cloud frame meet the preset dot product condition that the normal vector dot product is greater than 0, then only B, C, D, E, F, G, and H are searched during the KD tree search. This can also reduce the resource overhead of the KD tree search and improve the point cloud processing efficiency. If only B, C, D, E, and F meet the search conditions of the KD tree (for example, within its search radius), then the five points B, C, D, E, and F on the second point cloud frame are determined as the target neighborhood points of point cloud point A on the first point cloud frame.

[0104] Step 2022: Determine whether the target point cloud and its multiple target neighborhood points belong to the same point cloud plane.

[0105] Taking point A (the target point cloud) on the first point cloud frame in the previous step and its target neighboring points B, C, D, E, and F (the multiple target neighboring points of point A) on the second point cloud frame as an example, there are a total of 6 point clouds. In this step, we further determine whether these 6 point clouds belong to the same point cloud plane. To determine whether A, B, C, D, E, and F belong to the same plane, methods include but are not limited to the following: three-point plane determination method, RANSAC (Random Sample Consensus) algorithm, 3D Hough transform, etc. Taking the RANSAC algorithm as an example, it is a random sampling consistency algorithm that can be used to estimate the parameters of a mathematical model from a set of observation data containing outliers. The specific steps of using it to determine whether A, B, C, D, E, and F belong to the same plane may include: first, randomly select a point and its normal vector (for example, point A), assuming that point A and its normal vector define a plane M, then calculate the distance from points B, C, D, E, and F to this plane, and determine whether the distance is within a preset distance range (which can be determined based on prior knowledge and is not limited in this disclosure). If so, the points within the range and A are all points within plane M.

[0106] Step 2023 , when it is determined that the target point cloud and its multiple target neighborhood points belong to the same point cloud plane, it is determined that there is an overlapping area between the first point cloud frame and the second point cloud frame.

[0107] The first point cloud frame and the second point cloud frame are two different point cloud frames in space. If there is a point cloud between the two that belongs to the same point cloud plane, then the point cloud plane can be determined to be the overlapping area between the two, that is, it can be determined that there is an overlapping area between the two point cloud frames in the different spaces. For example, the T1 point cloud frame is collected in the range of 50° to 100°, while the T2 point cloud frame is collected from 90° to 150°. Then the T2 and T1 point cloud frames overlap in the range of 90° to 100°. Such an overlapping area is caused by the overlapping collection range, which will make the wall thickness less than the actual thickness of the wall, that is, the wall is too thin, which will lead to problems such as cross-fighting on both sides of the wall and wall layering and aliasing.

[0108] In step 2024, when there is an overlapping area between the first point cloud frame and the second point cloud frame, a matching relationship between the multiple point clouds and the point cloud plane to which the multiple point clouds belong is constructed to obtain a first point-surface matching pair, where the multiple point clouds include a target point cloud and its multiple target neighborhood points.

[0109] In the overlapping areas of the first and second point cloud frames, point-to-plane matching relationships are established to obtain multiple first point-to-plane matching pairs. Still taking the target point cloud point A and its target neighborhood points B, C, D, E, and F in the first point cloud frame as an example, these six points belong to point cloud plane M. Matching relationships between A, B, C, D, E, and F and M can be established, forming first point cloud matching pairs corresponding to points A, B, C, D, E, and F and point cloud plane M. After processing through the aforementioned steps, multiple first point-to-plane matching pairs can be established in this step.

[0110] Step 2025: Adjust the poses of the first point cloud frame and the second point cloud frame through the first point-surface matching pair, so that the first point-surface matching pair satisfies a first preset optimization condition.

[0111] As an optional embodiment of the present disclosure, when the point-plane matching relationship (first point-plane matching pair) between any first point cloud frame and the second point cloud frame meets a certain number (this number can be determined based on the degree of overlap between the two point cloud frames. If the degree of overlap is large, the number can be set higher. Conversely, if the degree of overlap is small, the number can be set lower. The present disclosure does not limit this), a posture optimization constraint between the two point cloud frames can be constructed based on the first point-plane matching pair. Exemplarily, the first point cloud frame and the second point cloud frame can be iteratively rotated and / or translated until the distance between the point cloud in the first point-plane matching pair and its point cloud plane meets the first preset optimization condition. The first preset optimization condition can be, for example, that the distance between each point in the first point-plane matching pair and the point cloud plane to which it belongs is minimized. For example, assuming that the displacement difference between the T1 point cloud frame and the T2 point cloud frame is 2cm, in order to satisfy the minimum distance between each point in the first point-surface matching pair in the T1 point cloud frame and the T2 point cloud frame and the point cloud plane to which it belongs, the point clouds in the T1 point cloud frame and the T2 point cloud frame can be rotated and / or translated in an iterative manner. If the displacement difference (2cm) between the T1 point cloud frame and the T2 point cloud frame is eliminated, the optimization constraint condition of the minimum distance between each point in the first point-surface matching pair in the T1 point cloud frame and the T2 point cloud frame and the point cloud plane to which it belongs is just satisfied, the iteration can be stopped, that is, the posture optimization of the T1 point cloud frame and the T2 point cloud frame is completed.

[0112] It can be understood that the first local point cloud frame in a room includes multiple first point cloud frames and second point cloud frames. The first local point cloud frame is traversed through the above method to complete the pose optimization of the point cloud frame.

[0113] Step 2026: Merge the first point-surface matching pairs that meet the first preset optimization condition to obtain a global point-surface matching pair of the room.

[0114] The first point-surface matching pairs in the point cloud frames that have completed pose optimization (meeting the first preset optimization condition) are merged, and finally a point-surface matching pair of a single room can be obtained, that is, a global point-surface matching pair.

[0115] Step 2027: Adjust the pose of the point cloud frame in the first local point cloud frame of the room through a preset point cloud optimizer and a global point-surface matching pair so that the global point-surface matching pair satisfies a second preset optimization condition.

[0116] A global point-plane matching pair in a single room is obtained (i.e., the point-plane matching pairs constructed frame-by-frame are put together), and then a preset point cloud optimizer (e.g., the least squares method) is used to perform overall optimization on the point cloud frame of the single room. That is, the coordinates of the point cloud frame of the single room are adjusted (rotated or translated) by the preset point cloud optimizer so that the geometric relationship between the points and planes in the global point-plane matching pair meets a second preset optimization condition. For example, the second preset optimization condition is that the distance between any point in the global point-plane matching pair and the point cloud plane to which it belongs is less than a preset distance threshold.

[0117] In step 2028, the point cloud frames in the first local point cloud frame of the room corresponding to the global point-surface matching pair satisfying the second preset optimization condition are merged, and the point cloud frames in the first local point cloud frame of the room corresponding to the merged global point-surface matching pair satisfying the second preset optimization condition are determined as the third local point cloud of the room.

[0118] The point cloud frames in the first local point cloud corresponding to the optimized global point-surface matching are merged and fused to finally obtain the third local point cloud of a single room, thus completing the point cloud optimization of a single room.

[0119] As an optional embodiment of the present disclosure, step 202 may further include: before executing step 2025, it may be determined whether the total number of the first local point cloud frames exceeds the preset number of point cloud frames (for example, 1000). If the total number of the first local point cloud frames exceeds the preset number of point cloud frames, the first local point cloud frame is divided into point cloud frames of multiple subspaces, and then steps 2025 to 2028 are executed for the point cloud frames of each subspace to optimize the point cloud pose of the room to which the first local point cloud frame belongs, so as to obtain the third local point cloud of the room. For example, when a single room has too many point cloud frames, the disclosed embodiments can randomly divide these point cloud frames into multiple subspaces. For example, for room P, the point cloud frames are divided to obtain point cloud frames of multiple subspaces such as P1, P2, P3, and P4. Steps 2025 to 2028 are performed on the point cloud frames of each subspace to optimize the point cloud frames of multiple subspaces such as P1, P2, P3, and P4, and then perform secondary optimization on the overall point cloud frame of P. In this way, optimizing each subspace separately can improve overall performance. If a single room is too large, it may include thousands of point cloud frames, and the number of first point-surface matching pairs obtained may be tens of thousands. By splitting it into multiple subspaces, computation can be reduced, saving resource overhead for point cloud processing.

[0120] In step 203, a second point-surface matching pair is established based on the second partial point cloud frame, and the point cloud pose of the transition area between the rooms to which the second partial point cloud frame belongs is optimized based on the second point-surface matching pair. The optimized second partial point cloud frames are merged to obtain a fourth partial point cloud of the transition area.

[0121] In the embodiment of the present disclosure, for the point cloud pose optimization of the transition area between rooms, each transition area is regarded as a separate entity. Refer to the point cloud pose optimization process of a single room in the aforementioned step 202, which will not be repeated here for the sake of brevity.

[0122] In step 204, the third partial point cloud of the room in the target space and the fourth partial point cloud of the transition area are stitched and fused, and a third point-surface matching pair is established. Based on the third point-surface matching pair, the point cloud pose of the room to which the third partial point cloud belongs and the transition area to which the fourth partial point cloud belongs are optimized, and the optimized third partial point cloud and fourth partial point cloud are merged to obtain the second overall point cloud data of the target space.

[0123] After completing the pose optimization of the point cloud frames of the rooms in the target space and the transition areas between the rooms through the previous steps, in this step, the point cloud frames of the rooms in the target space and the transition areas between the rooms that have completed the pose optimization are spliced and fused to obtain new overall point cloud data of the target space, that is, the second overall point cloud data of the target space.

[0124] It is understandable that each room in the target space is subjected to separate point cloud pose optimization, and the transition areas between rooms are subjected to point cloud pose optimization as multiple separate entities. After splicing, there may be room transitions between each room, between each transition area, and between rooms and transition areas, as well as wall thickening caused by thin walls, cross-wall collisions, and wall layering and aliasing. For this reason, in the embodiment of the present disclosure, after splicing and fusion to obtain the second overall point cloud data of the target space, the point cloud frame of the target space is further optimized for overall pose based on the second overall point cloud data, thereby improving the overall quality of the point cloud optimization of the target space and avoiding erroneous wall structure information such as wall thickening, cross-wall collisions, and wall layering and aliasing.

[0125] For example, Figure 4 As shown, step 204 can be implemented by the following steps:

[0126] In step 2041, point cloud splicing and fusion are performed on the third partial point cloud of the room in the target space and the fourth partial point cloud of the transition area, so that the optimized first point-surface matching pair of the third partial point cloud and the optimized second point-surface matching pair of the fourth partial point cloud are combined to form a third point-surface matching pair.

[0127] In step 2042, coordinate rotation and / or coordinate translation are performed on the point cloud frames belonging to the room and the point cloud frames belonging to the transition area between the rooms in the second overall point cloud data, respectively, until the distance between the point cloud in the third point-surface matching pair and its point cloud plane meets the third preset optimization condition, thereby obtaining the second overall point cloud data of the target space.

[0128] The third preset optimization condition, for example, is that the distance between the point cloud in the third point-plane matching pair and the point cloud plane to which it belongs is minimized or satisfies a preset distance threshold. Specifically, through this embodiment, pose optimization matching is further performed between the transition region and each room. The point cloud-plane distance between the transition region and each room is minimized or satisfies a preset distance threshold to adjust the pose of the point cloud frames between the transition region and each room, thereby achieving optimal matching between the point cloud frames in the target space, between the transition region, and between the transition region and each room. This further improves the overall quality of point cloud optimization in the target space and avoids erroneous wall structure information such as wall thickening, intersecting walls, and wall layering and aliasing. Furthermore, through this embodiment, pose matching is performed only between the rooms, between the transition region, and between the transition region and each room in the target space, eliminating repeated pose processing of the point cloud frames of individual rooms or transition regions. This reduces resource overhead for point cloud pose optimization and improves the efficiency of point cloud frame optimization processing in the target space.

[0129] As an optional embodiment of the present disclosure, it is also possible to further search for target neighborhood points of the target point cloud in the second overall point cloud data of the target space, and then recreate the point-surface matching relationship in the second overall point cloud data of the target space to obtain a third point-surface matching pair, and then optimize the overall point cloud pose of the target space with reference to the pose optimization of the point cloud frame of the aforementioned single room or single transition area based on the third point-surface matching. However, with this optimization method, since there are more point cloud frames in the target space, the number of point-surface matching pairs obtained will also be larger, resulting in a large overhead of point cloud computing processing resources.

[0130] like Figure 6 , which is a top view of the target space after point cloud optimization is completed according to an embodiment of the method disclosed herein. Figure 1 and Figure 6 Top view of the point cloud before and after point cloud optimization for the same target space. Figure 6 This is the effect diagram after point cloud pose optimization according to the embodiment of the present disclosure, and Figure 1 In comparison, it has been eliminated Figure 1 The walls at the locations indicated by the dotted boxes "1" and "2" are intertwined and overlapped. Figure 6 The positions indicated by the dotted circles "1" and "2".

[0131] In summary, the technical solution provided by the embodiments of the present disclosure optimizes the internal posture of the room by clustering and blocking the point cloud frames of the rooms and transition areas in the target space, and then establishing point-surface matching pairs based on the point cloud frames to constrain the posture inside each room; the transition areas between the rooms are also optimized as a whole in a similar room-like posture, and finally the optimized rooms and transition areas are subjected to another overall posture optimization to obtain the optimized point cloud of the target space. Most of the mismatches such as wall thickening, wall cross-fighting, wall layering and aliasing are caused by the transition area between the rooms. Therefore, in the present disclosure, performing point cloud pose optimization on the transition area separately is beneficial to separate the parts with large errors and the parts with small errors in the point cloud of the entire target space, thereby improving the accuracy of point cloud pose optimization and solving the problems of mismatches such as wall thickening, wall cross-fighting, wall layering and aliasing; and clustering and blocking the point cloud frames of the rooms and transition areas in the target space and then performing point cloud pose optimization separately reduces the overall scale of the target space optimization at one time, and the optimization between the rooms and transition areas can also be processed in parallel, which greatly improves the processing performance and resource overhead of the algorithm.

[0132] Correspondingly, the embodiments of the present disclosure also provide device embodiments corresponding to the aforementioned method embodiments. The device embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0133] Figure 7 This is a structural block diagram of a point cloud pose optimization device according to an embodiment of the present invention. Figure 7 As shown, a point cloud pose optimization device according to an embodiment of the present disclosure may include a local point cloud frame determination module 701, a room pose optimization module 702, a transition region pose optimization module 703, and a space pose optimization module 704, wherein:

[0134] The local point cloud frame determining module 701 is configured to determine a first local point cloud frame and a second local point cloud frame within the target space based on the first overall point cloud data of the target space, wherein the first local point cloud frame is point cloud frame data of any room within the target space, and the second local point cloud frame is point cloud frame data of a transition area between rooms within the target space;

[0135] a room pose optimization module 702 for establishing a first point-surface matching pair based on the first partial point cloud frame, optimizing the point cloud pose of the room to which the first partial point cloud frame belongs based on the first point-surface matching pair, and merging the optimized first partial point cloud frames to obtain a third partial point cloud of the room;

[0136] a transition region pose optimization module 703 for establishing a second point-surface matching pair based on the second partial point cloud frame, optimizing the point cloud pose of the transition region between the rooms to which the second partial point cloud frame belongs based on the second point-surface matching pair, and merging the optimized second partial point cloud frames to obtain a fourth partial point cloud of the transition region;

[0137] a spatial pose optimization module 704 for performing point cloud splicing and fusion on the third partial point cloud of the room in the target space and the fourth partial point cloud of the transition region, and establishing a third point-surface matching pair; performing point cloud pose optimization on the room to which the third partial point cloud belongs and the transition region to which the fourth partial point cloud belongs based on the third point-surface matching pair; and merging the optimized third and fourth partial point clouds to obtain second overall point cloud data of the target space;

[0138] The first point-plane matching pair, the second point-plane matching pair, and the third point-plane matching pair all represent matching relationships between multiple point clouds and the planes to which they belong.

[0139] The technical solution provided by the embodiments of the present disclosure optimizes the internal posture of the room by clustering and blocking the point cloud frames of the rooms and transition areas in the target space, and then establishing point-surface matching pairs based on the point cloud frames to constrain the internal posture of each room; the transition areas between the rooms are also optimized as a whole in a similar room-like manner, and finally the optimized rooms and transition areas are subjected to another overall posture optimization to obtain the optimized point cloud of the target space. Most of the mismatches such as wall thickening, wall cross-fighting, wall layering and aliasing are caused by the transition area between the rooms. Therefore, in the present disclosure, performing point cloud pose optimization on the transition area separately is beneficial to separate the parts with large errors and the parts with small errors in the point cloud of the entire target space, thereby improving the accuracy of point cloud pose optimization and solving the problems of mismatches such as wall thickening, wall cross-fighting, wall layering and aliasing; and clustering and blocking the point cloud frames of the rooms and transition areas in the target space and then performing point cloud pose optimization separately reduces the overall scale of the target space optimization at one time, and the optimization between the rooms and transition areas can also be processed in parallel, which greatly improves the processing performance and resource overhead of the algorithm.

[0140] exist Figure 7 Based on the embodiment shown, as an optional embodiment, the local point cloud frame determination module 701 includes:

[0141] a normal calculation unit, configured to calculate a normal vector for each point cloud in the point cloud frame of the target space to obtain a normal vector for each point cloud;

[0142] a point cloud merging unit, configured to merge the point clouds having normal vectors to obtain first overall point cloud data of the target space;

[0143] a segmentation and recognition unit, configured to perform plane segmentation and recognition processing on the first overall point cloud data based on a normal vector of the point cloud to obtain a wall area and a door opening area of the target space;

[0144] The first point cloud frame determining unit is configured to obtain the first local point cloud frame and the second local point cloud frame according to the point clouds of the wall area and the door opening area, respectively.

[0145] As an optional embodiment, the room posture optimization module 702 includes:

[0146] a target neighborhood point search unit, configured to search for a plurality of target neighborhood points of a target point cloud through a preset search tree, wherein the target point cloud is any point cloud on a first point cloud frame in the first local point cloud frame, and the plurality of target neighborhood points are a plurality of point clouds on a second point cloud frame in the first local point cloud frame;

[0147] A same-plane judgment unit is used to judge whether the target point cloud and its multiple target neighboring points belong to the same point cloud plane;

[0148] An overlapping region determining unit, configured to determine whether an overlapping region exists between the first point cloud frame and the second point cloud frame when it is determined that the target point cloud and its multiple target neighborhood points belong to the same point cloud plane;

[0149] a first point-plane matching unit, configured to, when there is an overlapping area between the first point cloud frame and the second point cloud frame, establish a matching relationship between the multiple point clouds and the point cloud plane to which the multiple point clouds belong, to obtain the first point-plane matching pair, wherein the multiple point clouds include the target point cloud and its multiple target neighborhood points;

[0150] a first pose adjustment unit, configured to adjust the poses of the first point cloud frame and the second point cloud frame by using the first point-plane matching pair, so that the first point-plane matching pair satisfies a first preset optimization condition;

[0151] a second point-plane matching unit, configured to merge the first point-plane matching pairs that satisfy the first preset optimization condition to obtain a global point-plane matching pair of the room;

[0152] A second pose adjustment unit is configured to adjust the pose of the point cloud frame in the first local point cloud frame of the room by using a preset point cloud optimizer and the global point-surface matching pair, so that the global point-surface matching pair satisfies a second preset optimization condition;

[0153] The second point cloud frame determination unit is used to merge the point cloud frames in the first local point cloud frame of the room corresponding to when the global point-surface matching pair meets the second preset optimization condition, and determine the point cloud frame in the first local point cloud frame of the room corresponding to when the merged global point-surface matching pair meets the second preset optimization condition as the third local point cloud of the room.

[0154] As an optional embodiment, the room posture optimization module 702 further includes:

[0155] a point cloud quantity determination unit, configured to determine whether the total number of the first partial point cloud frames exceeds a preset number of point cloud frames before adjusting the poses of the first point cloud frame and the second point cloud frame by using the first point-surface matching pair so that the first point-surface matching pair satisfies a first preset optimization condition;

[0156] a space division unit, configured to divide the first local point cloud frame into a plurality of subspace point cloud frames when the total number of the first local point cloud frames exceeds a preset number of point cloud frames;

[0157] The second pose adjustment unit is used to perform the step of adjusting the poses of the first point cloud frame and the second point cloud frame through the first point-surface matching pair for the point cloud frame of each subspace, so that the first point-surface matching pair satisfies the first preset optimization condition and its subsequent steps, so as to optimize the point cloud pose of the room to which the first local point cloud frame belongs, and merge the optimized first point cloud frame and the second point cloud frame of each subspace to obtain the second local point cloud frame of the room.

[0158] As an optional embodiment of the present disclosure, the target neighborhood point search unit includes:

[0159] A search tree establishing subunit, configured to establish a preset search tree for the first point cloud frame in the first partial point cloud frame with respect to the second point cloud frame in the first partial point cloud frame;

[0160] a normal vector dot product calculation subunit, configured to traverse each point cloud in a first point cloud frame in the first partial point cloud frame, and calculate a normal vector dot product between the point cloud and each point cloud in a second point cloud frame in the first partial point cloud frame;

[0161] a dot product condition judging subunit, configured to judge, for each point cloud on the second point cloud frame in the first partial point cloud frame, whether the dot product of the normal vector satisfies a preset dot product condition;

[0162] a searched point cloud determining subunit, configured to determine a point cloud on the second point cloud frame in the first partial point cloud frame, for which the normal vector dot product satisfies a preset dot product condition, as a target searched point cloud;

[0163] The target domain point determination subunit is configured to search the target searched point cloud through the preset search tree, so as to determine the searched target searched point cloud that meets the search conditions of the preset search tree as the target domain point.

[0164] As an optional embodiment of the present disclosure, the first posture adjustment unit includes:

[0165] The posture adjustment subunit is used to iteratively perform coordinate rotation and / or coordinate translation on the first point cloud frame and the second point cloud frame until the distance between the point cloud in the first point-plane matching pair and its point cloud plane meets a first preset optimization condition.

[0166] As an optional embodiment of the present disclosure, the spatial posture optimization module 705 includes:

[0167] a point-surface matching pair acquisition unit, configured to perform point cloud splicing and fusion on the third partial point cloud of the room in the target space and the fourth partial point cloud of the transition area, so that the optimized first point-surface matching pair of the third partial point cloud and the optimized second point-surface matching pair of the fourth partial point cloud are combined to form the third point-surface matching pair;

[0168] The third posture adjustment unit is used to perform coordinate rotation and / or coordinate translation on the point cloud frames belonging to the room and the point cloud frames belonging to the transition area between the rooms in the second overall point cloud data, respectively, until the distance between the point cloud in the third point-surface matching pair and its point cloud plane meets the third preset optimization condition, thereby obtaining the second overall point cloud data of the target space.

[0169] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0170] Below, reference Figure 8The electronic device according to the embodiment of the present disclosure is described. The electronic device may be either or both of the first device and the second device, or a standalone device independent of them, and the standalone device may communicate with the first device and the second device to receive collected input signals from them.

[0171] Figure 8 A block diagram of an electronic device according to an embodiment of the present disclosure is illustrated.

[0172] like Figure 8 As shown, the electronic device includes one or more processors and memory.

[0173] The processor may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0174] The memory can store one or more computer program products (hereinafter referred to as program products). The memory can include various forms of computer-readable storage media (hereinafter referred to as media), such as volatile memory and / or non-volatile memory. The volatile memory can include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory can include, for example, read-only memory (ROM), a hard disk, flash memory, etc. One or more computer program products can be stored on the computer-readable storage medium, and the processor can execute the computer program products to implement the point cloud pose optimization method of the various embodiments of the present disclosure described above and / or other desired functions.

[0175] In one example, the electronic device may further include an input device and an output device, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0176] In addition, the input device may also include, for example, a keyboard, a mouse, and the like.

[0177] The output device can output various information to the outside, including determined distance information, direction information, etc. The output device can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0178] Of course, to simplify, Figure 8 Only some of the components related to the present disclosure in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device may further include any other appropriate components according to specific application scenarios.

[0179] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the point cloud pose optimization method according to various embodiments of the present disclosure described in the above part of this specification.

[0180] The computer program product may be written in any combination of one or more programming languages to implement the operations of the disclosed embodiments, including object-oriented programming languages such as Java, C++, and conventional procedural programming languages such as C or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0181] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enable the processor to execute the steps of the point cloud pose optimization method according to various embodiments of the present disclosure described in the above part of this specification.

[0182] The computer-readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, include but is not limited to a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0183] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this disclosure are merely illustrative and not restrictive, and should not be construed as necessarily possessed by each embodiment of the present disclosure. Furthermore, the specific details disclosed above are provided for illustrative purposes and to facilitate understanding, rather than as limitations. These details do not limit the present disclosure to necessarily being implemented using these specific details.

[0184] Each embodiment in this specification is described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. References to the same or similar parts between the various embodiments are sufficient. For system embodiments, since they largely correspond to method embodiments, their description is relatively simple. For relevant parts, references to the description of the method embodiments are sufficient.

[0185] The block diagrams of the devices, devices, equipment, and systems involved in this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "include," "comprise," "have," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0186] The methods and apparatus of the present disclosure may be implemented in many ways. For example, the methods and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless otherwise specified. In addition, in some embodiments, the present disclosure may also be implemented as programs recorded in a recording medium, which include machine-readable instructions for implementing the methods according to the present disclosure. Thus, the present disclosure also covers recording media that store programs for executing the methods according to the present disclosure.

[0187] It should also be noted that in the apparatus, device, and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0188] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0189] The above description has been provided for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A point cloud pose optimization method, characterized in that: The method comprises: Determining a first local point cloud frame and a second local point cloud frame within the target space based on the first overall point cloud data of the target space, wherein the first local point cloud frame is point cloud frame data of any room within the target space, and the second local point cloud frame is point cloud frame data of a transition area between rooms within the target space; Establishing a first point-surface matching pair based on the first partial point cloud frame, optimizing the point cloud pose of the room to which the first partial point cloud frame belongs based on the first point-surface matching pair, and merging the optimized first partial point cloud frames to obtain a third partial point cloud of the room; Establishing a second point-surface matching pair based on the second partial point cloud frame, optimizing the point cloud pose of a transition region between rooms to which the second partial point cloud frame belongs based on the second point-surface matching pair, and merging the optimized second partial point cloud frames to obtain a fourth partial point cloud of the transition region; performing point cloud splicing and fusion on the third partial point cloud of the room in the target space and the fourth partial point cloud of the transition area, and establishing a third point-surface matching pair, performing point cloud pose optimization on the room to which the third partial point cloud belongs and the transition area to which the fourth partial point cloud belongs based on the third point-surface matching pair, and merging the optimized third partial point cloud and fourth partial point cloud to obtain second overall point cloud data of the target space; The first point-plane matching pair, the second point-plane matching pair, and the third point-plane matching pair all represent matching relationships between multiple point clouds and the planes to which they belong.

2. The method according to claim 1, characterized in that The determining of the first local point cloud frame and the second local point cloud frame in the target space according to the first overall point cloud data of the target space comprises: Calculating a normal vector for each point cloud in the point cloud frame of the target space to obtain a normal vector for each point cloud; Merging the point clouds with normal vectors to obtain first overall point cloud data of the target space; Performing plane segmentation and recognition processing on the first overall point cloud data based on the normal vector of the point cloud to obtain the wall area and the door opening area of the target space; The first local point cloud frame and the second local point cloud frame are obtained according to the point clouds of the wall area and the door opening area, respectively.

3. The method according to claim 1 or 2, characterized in that The step of establishing a first point-surface matching pair based on the first partial point cloud frame, optimizing the point cloud pose of the room to which the first partial point cloud frame belongs based on the first point-surface matching pair, and merging the optimized first partial point cloud frames to obtain a third partial point cloud of the room includes: Searching for multiple target neighborhood points of a target point cloud through a preset search tree, wherein the target point cloud is any point cloud on a first point cloud frame in the first partial point cloud frame, and the multiple target neighborhood points are multiple point clouds on a second point cloud frame in the first partial point cloud frame; Determining whether the target point cloud and its multiple target neighborhood points belong to the same point cloud plane; When it is determined that the target point cloud and its multiple target neighborhood points belong to the same point cloud plane, determining that there is an overlapping area between the first point cloud frame and the second point cloud frame; In a case where there is an overlapping area between the first point cloud frame and the second point cloud frame, establishing a point cloud-point cloud plane matching relationship between the multiple point clouds and the point cloud plane to which the multiple point clouds belong, to obtain the first point-plane matching pair, wherein the multiple point clouds include the target point cloud and its multiple target neighborhood points; Adjusting the poses of the first point cloud frame and the second point cloud frame by using the first point-plane matching pair so that the first point-plane matching pair satisfies a first preset optimization condition; Merging the first point-surface matching pairs that meet the first preset optimization condition to obtain a global point-surface matching pair of the room; Adjusting the pose of the point cloud frame in the first local point cloud frame of the room by using a preset point cloud optimizer and the global point-surface matching pair so that the global point-surface matching pair satisfies a second preset optimization condition; The point cloud frames in the first local point cloud frame of the room corresponding to the global point-surface matching pair satisfying the second preset optimization condition are merged, and the point cloud frames in the first local point cloud frame of the room corresponding to the merged global point-surface matching pair satisfying the second preset optimization condition are determined as the third local point cloud of the room.

4. The method according to claim 3, characterized in that The step of establishing a first point-surface matching pair based on the first partial point cloud frame, optimizing the point cloud pose of the room to which the first partial point cloud frame belongs based on the first point-surface matching pair, and merging the optimized first partial point cloud frames to obtain a third partial point cloud of the room further includes: Before adjusting the poses of the first point cloud frame and the second point cloud frame by using the first point-plane matching pair so that the first point-plane matching pair satisfies a first preset optimization condition, determining whether the total number of the first partial point cloud frames exceeds a preset number of point cloud frames; When the total number of the first partial point cloud frames exceeds a preset number of point cloud frames, dividing the first partial point cloud frames into point cloud frames of a plurality of subspaces; For the point cloud frame of each subspace, the step of adjusting the posture of the first point cloud frame and the second point cloud frame through the first point-surface matching pair is performed so that the first point-surface matching pair satisfies the first preset optimization condition and the subsequent steps, so as to optimize the point cloud posture of the room to which the first local point cloud frame belongs, and merge the optimized first point cloud frame and the second point cloud frame of each subspace to obtain the third local point cloud of the room.

5. The method according to claim 4, characterized in that The method of searching for multiple target neighborhood points of the target point cloud through a preset search tree includes: For the second point cloud frame in the first partial point cloud frame, establishing a preset search tree for the first point cloud frame in the first partial point cloud frame; Traversing each point cloud in the first point cloud frame in the first partial point cloud frame, and calculating a dot product of a normal vector between the point cloud and each point cloud in the second point cloud frame in the first partial point cloud frame; For each point cloud on the second point cloud frame in the first partial point cloud frame, determining whether the normal vector dot product meets a preset dot product condition; Determining a point cloud on the second point cloud frame in the first partial point cloud frame, whose normal vector dot product satisfies a preset dot product condition, as a target search point cloud; The target searched point cloud is searched through the preset search tree to determine the target searched point cloud that meets the search conditions of the preset search tree as the target area point.

6. The method according to claim 5, characterized in that The adjusting the poses of the first point cloud frame and the second point cloud frame by using the first point-plane matching pair so that the first point-plane matching pair satisfies a first preset optimization condition includes: Coordinate rotation and / or coordinate translation are iteratively performed on the first point cloud frame and the second point cloud frame until the distance between the point cloud in the first point-plane matching pair and its point cloud plane meets a first preset optimization condition.

7. The method according to claim 6, characterized in that The step of performing point cloud stitching and fusion on the third partial point cloud of the room in the target space and the fourth partial point cloud of the transition area, and establishing a third point-surface matching pair, performing point cloud pose optimization on the room to which the third partial point cloud belongs and the transition area to which the fourth partial point cloud belongs based on the third point-surface matching pair, and merging the optimized third partial point cloud and fourth partial point cloud to obtain second overall point cloud data of the target space, including: performing point cloud stitching and fusion on the third partial point cloud of the room in the target space and the fourth partial point cloud of the transition area, so that the optimized first point-surface matching pair of the third partial point cloud and the optimized second point-surface matching pair of the fourth partial point cloud are combined to form the third point-surface matching pair; Coordinate rotation and / or coordinate translation are performed on the point cloud frames belonging to the room and the point cloud frames belonging to the transition area between the rooms in the second overall point cloud data respectively until the distance between the point cloud in the third point-surface matching pair and its point cloud plane meets the third preset optimization condition, thereby obtaining the second overall point cloud data of the target space.

8. A point cloud pose optimization device, characterized in that: The device comprises: a local point cloud frame determination module, configured to determine a first local point cloud frame and a second local point cloud frame within the target space based on the first overall point cloud data of the target space, wherein the first local point cloud frame is point cloud frame data of any room within the target space, and the second local point cloud frame is point cloud frame data of a transition area between rooms within the target space; a room pose optimization module, configured to establish a first point-surface matching pair based on the first partial point cloud frame, optimize the point cloud pose of the room to which the first partial point cloud frame belongs based on the first point-surface matching pair, and merge the optimized first partial point cloud frames to obtain a third partial point cloud of the room; a transition region pose optimization module, configured to establish a second point-surface matching pair based on the second partial point cloud frame, optimize the point cloud pose of the transition region between the rooms to which the second partial point cloud frame belongs based on the second point-surface matching pair, and merge the optimized second partial point cloud frames to obtain a fourth partial point cloud of the transition region; a spatial pose optimization module, configured to perform point cloud splicing and fusion on the third partial point cloud of the room in the target space and the fourth partial point cloud of the transition area, and establish a third point-surface matching pair, to optimize the point cloud pose of the room to which the third partial point cloud belongs and the transition area to which the fourth partial point cloud belongs based on the third point-surface matching pair, and to merge the optimized third partial point cloud and fourth partial point cloud to obtain second overall point cloud data of the target space; The first point-plane matching pair, the second point-plane matching pair, and the third point-plane matching pair all represent matching relationships between multiple point clouds and the planes to which they belong.

9. An electronic device, characterized in that: include: a memory for storing a computer program product; A processor is configured to execute the computer program product stored in the memory, and when the computer program product is executed, implements the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method described in any one of claims 1 to 7 is implemented.

11. A computer program product comprising computer program instructions, characterized in that When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images