Point cloud pose optimization method and apparatus, and electronic device, medium and program product
By segmenting and maximizing the pose of point clouds generated by SLAM devices into room and transition areas, problems such as wall thickening and wall intersections in point clouds are solved, thus improving the accuracy and efficiency of point cloud pose optimization.
Patent Information
- Application Number
- PCT/CN2026/072028
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-07
- Filing Date
- 2026-01-12
- Publication Date
- 2026-11-12
AI Technical Summary
When generating point clouds, existing SLAM devices often exhibit phenomena such as wall thickening, wall overlap between two sides, and wall layering. Existing optimization methods based on structured information such as planes cannot completely solve these problems.
By distinguishing between the various rooms and transition areas within the target space, the point cloud frame data is processed in blocks, point-to-surface matching pairs are established to optimize the pose of the rooms and transition areas, and point cloud stitching and fusion are performed. Finally, overall pose optimization is carried out to improve the accuracy and processing efficiency of point cloud pose.
It effectively solves the mismatch problems such as wall thickening, wall cross-intersection, and wall layering, improves the accuracy and processing performance of point cloud pose optimization, reduces the overall optimization scale, and improves the processing performance and resource consumption of the algorithm.
Smart Images

Figure CN2026072028_12112026_PF_FP_ABST
Abstract
Description
Point cloud pose optimization methods, devices, electronic devices, media, and program products
[0001] This disclosure claims priority to Chinese Patent Application No. CN202510586112.8, filed on May 7, 2025, entitled “Point Cloud Pose Optimization Method, Apparatus, Electronic Device, Medium and Program Product”, the entire contents of which are incorporated herein by reference. Technical Field
[0002] This disclosure relates to point cloud processing technology, and in particular to a point cloud pose optimization method, apparatus, electronic device, medium, and program product. Background Technology
[0003] Simultaneous Localization and Mapping (SLAM) devices typically include a lidar unit and a camera. The lidar unit performs laser scanning of the surrounding environment to acquire 3D spatial information and generate a corresponding point cloud. The camera unit acquires images of the surrounding environment so that the color values of each point in the point cloud can be used to color the point cloud generated by the lidar or the 3D model obtained from the 3D reconstruction.
[0004] During the point cloud data acquisition process of LiDAR, due to the large noise of short-range laser ranging and the limitations of scene structure such as room transition and thin walls, the point cloud generated by SLAM equipment often exhibits phenomena such as thickened walls, overlapping walls on both sides, and layered walls. As shown in Figure 1, labeled position 1 and labeled position 2 are both examples of overlapping walls on both sides.
[0005] In related technologies, point cloud optimization methods based on structured information such as planes are used to solve problems such as wall thickening, wall overlap between two sides, and wall layering. However, this method cannot fundamentally solve problems such as wall overlap. The optimized point cloud still contains incorrect wall structure information such as wall thickening, wall overlap between two sides, and wall layering. Summary of the Invention
[0006] To address the technical problems in related technologies, embodiments of this disclosure provide a point cloud pose optimization method, apparatus, electronic device, medium, and program product.
[0007] According to a first aspect of the present disclosure, a point cloud pose optimization method is provided, the method comprising:
[0008] Based on the first overall point cloud data of the target space, a first local point cloud frame and a second local point cloud frame within the target space are determined, wherein the first local point cloud frame is the point cloud frame data of any room within the target space, and the second local point cloud frame is the point cloud frame data of the transition area between the rooms within the target space.
[0009] A first point-plane matching pair is established based on the first local point cloud frame. The point cloud pose of the room to which the first local point cloud frame belongs is optimized based on the first point-plane matching pair. The optimized first local point cloud frame is then merged to obtain the third local point cloud of the room.
[0010] A second point-plane matching pair is established based on the second local point cloud frame. The point cloud pose of the transition area between the rooms to which the second local point cloud frame belongs is optimized based on the second point-plane matching pair. The optimized second local point cloud frames are then merged to obtain the fourth local point cloud of the transition area.
[0011] The third local point cloud of the room in the target space and the fourth local point cloud of the transition area are stitched and fused together, and a third point-plane matching pair is established. Based on the third point-plane matching pair, the point cloud pose of the room to which the third local point cloud belongs and the transition area to which the fourth local point cloud belongs are optimized. The optimized third local point cloud and the fourth local point cloud are merged to obtain the second overall point cloud data of the target space.
[0012] The first point-plane matching pair, the second point-plane matching pair, and the third point-plane matching pair respectively represent the matching relationship between multiple point clouds and their respective planes.
[0013] As an optional embodiment, determining the first local point cloud frame and the second local point cloud frame within the target space based on the first overall point cloud data of the target space includes:
[0014] The normal vector of any point cloud in the point cloud frame of the target space is calculated.
[0015] The point clouds with normal vectors are merged to obtain the first overall point cloud data of the target space;
[0016] Based on the normal vectors of the point cloud, the first overall point cloud data is subjected to planar segmentation and recognition processing to obtain the wall area and doorway area of the target space;
[0017] The first local point cloud frame and the second local point cloud frame are obtained based on the point clouds of the wall area and the doorway area, respectively.
[0018] As an optional embodiment, the step of establishing a first point-plane matching pair based on the first local point cloud frame, optimizing the point cloud pose of the room to which the first local point cloud frame belongs based on the first point-plane matching pair, and merging the optimized first local point cloud frames to obtain a third local point cloud of the room includes:
[0019] Multiple target neighboring points of the target point cloud are searched by a preset search tree. The target point cloud is any point cloud in the first point cloud frame of the first local point cloud frame, and the multiple target neighboring points are multiple point clouds in the second point cloud frame of the first local point cloud frame.
[0020] Determine whether the target point cloud and its multiple target neighboring points belong to the same point cloud plane;
[0021] If the target point cloud and its multiple target neighboring points belong to the same point cloud plane, it is determined that the first point cloud frame and the second point cloud frame have an overlapping area.
[0022] In the case where there is an overlapping area between the first point cloud frame and the second point cloud frame, the multiple point clouds are matched with the point cloud plane to which the multiple point clouds belong to to form a matching relationship between the point cloud and the point cloud plane, and the first point-plane matching pair is obtained. The multiple point clouds include the target point cloud and its multiple target neighborhood points.
[0023] The poses of the first point cloud frame and the second point cloud frame are adjusted by the first point-plane matching pair so that the first point-plane matching pair meets the first preset optimization condition.
[0024] The first point-plane matching pairs that satisfy the first preset optimization conditions are merged to obtain the global point-plane matching pairs of the room.
[0025] The pose of the first point cloud frame in the first local point cloud frame of the room is adjusted by a preset point cloud optimizer and the global point-plane matching pair so that the global point-plane matching pair meets the second preset optimization condition.
[0026] The point cloud frames in the first local point cloud frame of the room corresponding to the global point-surface matching pair satisfying the second preset optimization condition are merged, and the point cloud frames in the first local point cloud frame of the room corresponding to the merged global point-surface matching pair satisfying the second preset optimization condition are determined as the third local point cloud of the room.
[0027] As an optional embodiment,
[0028] Before adjusting the poses of the first point cloud frame and the second point cloud frame through the first point-plane matching pair to ensure that the first point-plane matching pair meets the first preset optimization condition, the method further includes:
[0029] Determine whether the total number of the first local point cloud frames exceeds the preset number of point cloud frames;
[0030] If the total number of the first local point cloud frames exceeds the preset number of point cloud frames, the first local point cloud frames are divided into point cloud frames in multiple subspaces.
[0031] For any point cloud frame in any subspace, the pose of the first point cloud frame and the second point cloud frame are adjusted by the first point-plane matching pair so that the first point-plane matching pair meets the first preset optimization condition and its subsequent steps, so as to optimize the point cloud pose of the room to which the first local point cloud frame belongs, and merge the optimized first point cloud frame and the second point cloud frame of the multiple subspaces to obtain the third local point cloud of the room.
[0032] As an optional embodiment, the step of searching multiple target neighborhood points in the target point cloud through a preset search tree includes:
[0033] For the second point cloud frame in the first local point cloud frame, a preset search tree for the first point cloud frame in the first local point cloud frame is established.
[0034] Traverse each point cloud in the first point cloud frame of the first local point cloud frame, and calculate the dot product of the normal vector between the point cloud and each point cloud in the second point cloud frame of the first local point cloud frame;
[0035] For any point cloud in the second point cloud frame within the first local point cloud frame, determine whether the dot product of the normal vectors satisfies the preset dot product condition;
[0036] The point cloud in the second point cloud frame of the first local point cloud frame whose normal vector dot product satisfies the preset dot product condition is determined as the target search point cloud;
[0037] The target search point cloud is searched using the preset search tree, and the target search point cloud that meets the search conditions of the preset search tree is determined as the target neighborhood point.
[0038] As an optional embodiment, adjusting the poses of the first point cloud frame and the second point cloud frame through the first point-plane matching pair to make the first point-plane matching pair satisfy a first preset optimization condition includes:
[0039] Iteratively rotate and / or translate the first point cloud frame and the second point cloud frame until the distance between the point cloud in the first point-plane matching pair and its point cloud plane satisfies the first preset optimization condition.
[0040] As an optional embodiment, the step of stitching and fusing the third local point cloud of the room within the target space and the fourth local point cloud of the transition region, and establishing a third point-plane matching pair, to optimize the point cloud pose of the room to which the third local point cloud belongs and the transition region to which the fourth local point cloud belongs based on the third point-plane matching pair, and merging the optimized third local point cloud and fourth local point cloud to obtain the second overall point cloud data of the target space, includes:
[0041] The process involves stitching and fusing the third local point cloud of the room within the target space and the fourth local point cloud of the transition region, establishing a third point-plane matching pair, and optimizing the point cloud pose of the room to which the third local point cloud belongs and the transition region to which the fourth local point cloud belongs based on the third point-plane matching pair. The optimized third and fourth local point clouds are then merged to obtain the second overall point cloud data of the target space, including:
[0042] The third local point cloud of the room in the target space and the fourth local point cloud of the transition area are stitched and fused together. The optimized first point-surface matching pair of the third local point cloud and the optimized second point-surface matching pair of the fourth local point cloud are combined to form the third point-surface matching pair.
[0043] The coordinate rotation and / or coordinate translation of the point cloud frames belonging to each room and the point cloud frames belonging to the transition area between each room in the second overall point cloud data are performed iteratively until the distance between the point cloud in the third point-plane matching pair and its point cloud plane meets the third preset optimization condition, thereby obtaining the second overall point cloud data of the target space.
[0044] According to a second aspect of the present disclosure, a point cloud pose optimization apparatus is provided, the apparatus comprising:
[0045] The local point cloud frame determination module is used to determine a first local point cloud frame and a second local point cloud frame within the target space based on the first overall point cloud data of the target space. The first local point cloud frame is the point cloud frame data of any room within the target space, and the second local point cloud frame is the point cloud frame data of the transition area between the rooms within the target space.
[0046] The room pose optimization module is used to establish a first point-plane matching pair based on the first local point cloud frame, to optimize the point cloud pose of the room to which the first local point cloud frame belongs based on the first point-plane matching pair, and to merge the optimized first local point cloud frame to obtain the third local point cloud of the room.
[0047] The transition region pose optimization module is used to establish a second point-plane matching pair based on the second local point cloud frame, to optimize the point cloud pose of the transition region between the rooms to which the second local point cloud frame belongs based on the second point-plane matching pair, and to merge the optimized second local point cloud frames to obtain the fourth local point cloud of the transition region.
[0048] The spatial pose optimization module is used to perform point cloud stitching and fusion on the third local point cloud of the room and the fourth local point cloud of the transition area within the target space, and to establish a third point-plane matching pair. Based on the third point-plane matching pair, the module optimizes the point cloud pose of the room to which the third local point cloud belongs and the transition area to which the fourth local point cloud belongs, and merges the optimized third local point cloud and the fourth local point cloud to obtain the second overall point cloud data of the target space.
[0049] The first point-plane matching pair, the second point-plane matching pair, and the third point-plane matching pair all represent the matching relationship between multiple point clouds and their respective planes.
[0050] As an optional embodiment, the local point cloud frame determination module includes:
[0051] The normal vector calculation unit is used to calculate the normal vector of any point cloud in the point cloud frame of the target space to obtain the normal vector of any point cloud.
[0052] A point cloud merging unit is used to merge point clouds with normal vectors to obtain the first overall point cloud data of the target space;
[0053] The segmentation and recognition unit is used to perform planar segmentation and recognition processing on the first overall point cloud data based on the normal vector of the point cloud to obtain the wall area and doorway area of the target space;
[0054] The first point cloud frame determination unit is used to obtain the first local point cloud frame and the second local point cloud frame based on the point clouds of the wall area and the doorway area, respectively.
[0055] As an optional embodiment, the room pose optimization module includes:
[0056] The target neighborhood point search unit is used to search for multiple target neighborhood points of the target point cloud through a preset search tree. The target point cloud is any point cloud in the first point cloud frame of the first local point cloud frame, and the multiple target neighborhood points are multiple point clouds in the second point cloud frame of the first local point cloud frame.
[0057] The same plane determination unit is used to determine whether the target point cloud and its multiple target neighboring points belong to the same point cloud plane;
[0058] The overlapping region determination unit is used to determine that if the target point cloud and its multiple target neighboring points belong to the same point cloud plane, the first point cloud frame and the second point cloud frame have an overlapping region.
[0059] The first point-plane matching unit is used to construct a matching relationship between the multiple point clouds and the point cloud plane to which the multiple point clouds belong when there is an overlapping area between the first point cloud frame and the second point cloud frame, thereby obtaining the first point-plane matching pair. The multiple point clouds include the target point cloud and its multiple target neighborhood points.
[0060] The first pose adjustment unit is used to adjust the pose of the first point cloud frame and the second point cloud frame through the first point-plane matching pair so that the first point-plane matching pair meets the first preset optimization condition.
[0061] The second point-to-surface matching unit is used to merge the first point-to-surface matching pairs that satisfy the first preset optimization conditions to obtain the global point-to-surface matching pairs of the room.
[0062] The second pose adjustment unit is used to adjust the pose of the first point cloud frame in the first local point cloud frame of the room through a preset point cloud optimizer and the global point-plane matching pair, so that the global point-plane matching pair meets the second preset optimization condition.
[0063] The second point cloud frame determination unit is used to merge the point cloud frames in the first local point cloud frame of the room corresponding to the global point-surface matching pair satisfying the second preset optimization condition, and determine the point cloud frames in the first local point cloud frame of the room corresponding to the merged global point-surface matching pair satisfying the second preset optimization condition as the third local point cloud of the room.
[0064] As an optional embodiment, the room pose optimization module further includes:
[0065] The point cloud quantity determination unit is used to determine whether the total number of the first local point cloud frames exceeds the preset number of point cloud frames before adjusting the poses of the first point cloud frame and the second point cloud frame through the first point-plane matching pair so that the first point-plane matching pair meets the first preset optimization condition.
[0066] The spatial partitioning unit divides the first local point cloud frames into multiple sub-space point cloud frames if the total number of the first local point cloud frames exceeds the preset number of point cloud frames.
[0067] The second pose adjustment unit is used to perform the adjustment of the pose of the first point cloud frame and the second point cloud frame by means of the first point-plane matching pair for any point cloud frame in any subspace, so that the first point-plane matching pair meets the first preset optimization condition and its subsequent steps, so as to optimize the point cloud pose of the room to which the first local point cloud frame belongs, and merge the optimized first point cloud frame and the second point cloud frame of the multiple subspaces to obtain the third local point cloud of the room.
[0068] As an optional embodiment of this disclosure, the target neighborhood point search unit includes:
[0069] The search tree building subunit is used to build a preset search tree for the first point cloud frame in the first local point cloud frame for the second point cloud frame in the first local point cloud frame.
[0070] The normal vector dot product calculation subunit is used to traverse each point cloud in the first point cloud frame of the first local point cloud frame and calculate the normal vector dot product between the point cloud and each point cloud in the second point cloud frame of the first local point cloud frame.
[0071] The dot product condition judgment subunit is used to determine whether the dot product of the normal vector satisfies the preset dot product condition for any point cloud in the second point cloud frame in the first local point cloud frame.
[0072] The searched point cloud determination subunit is used to determine the point cloud in the second point cloud frame in the first local point cloud frame, where the dot product of the normal vector satisfies the preset dot product condition, as the target searched point cloud.
[0073] The target neighborhood point determination subunit is used to search the target search point cloud through the preset search tree, so as to determine the target search point cloud that meets the search conditions of the preset search tree as the target neighborhood point.
[0074] As an optional embodiment of this disclosure, the first pose adjustment unit includes:
[0075] The pose adjustment subunit is used to iteratively rotate and / or translate the first point cloud frame and the second point cloud frame until the distance between the point cloud in the first point-plane matching pair and its point cloud plane satisfies the first preset optimization condition.
[0076] As an optional embodiment of this disclosure, the spatial pose optimization module includes:
[0077] The point-surface matching pair acquisition unit is used to perform point cloud stitching and fusion on the third local point cloud of the room in the target space and the fourth local point cloud of the transition area, and to combine the optimized first point-surface matching pair of the third local point cloud and the optimized second point-surface matching pair of the fourth local point cloud to form the third point-surface matching pair.
[0078] The third pose adjustment unit is used to iteratively rotate and / or translate the point cloud frames belonging to each room and the point cloud frames belonging to the transition area between each room in the second overall point cloud data until the distance between the point cloud in the third point-plane matching pair and its point cloud plane meets the third preset optimization condition, so as to obtain the second overall point cloud data of the target space.
[0079] According to a third aspect of the present disclosure, an electronic device is provided, comprising:
[0080] Memory, used to store computer program products;
[0081] A processor is configured to execute a computer program product stored in the memory, and when the computer program product is executed, to implement the method described in the first aspect above.
[0082] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the method described in the first aspect.
[0083] According to a fifth aspect of the present disclosure, a computer program product is provided, including computer program instructions that, when executed by a processor, implement the method described in the first aspect.
[0084] Based on the technical solution provided in this disclosure, by distinguishing between each room in the target space and the transition area between each room, the point cloud frame data of the rooms and the transition area in the target space are divided into blocks, and point-plane matching pairs are established based on the point cloud frames to constrain the pose inside each room, thereby optimizing the pose inside the room; the transition area between rooms is also treated as a separate whole and its pose is optimized in a similar way to that of the room. Finally, the point clouds of the optimized rooms and the transition area are stitched together and then the overall pose is optimized again in a similar manner to obtain the optimized point cloud of the target space. Since most errors, such as wall thickening, wall overlap, and wall layering, originate from the transition areas between rooms, this embodiment optimizes the point cloud pose of the transition areas separately. This separates the parts with large and small errors in the point cloud of the entire target space, which helps improve the accuracy of point cloud pose optimization and solves the matching problems of wall thickening, wall overlap, and wall layering. In addition, the method of dividing the point cloud frames of the rooms and transition areas in the target space into blocks and optimizing the point cloud pose separately reduces the overall scale of the target space optimization at one time. Moreover, the optimization between rooms and transition areas can be processed in parallel, which greatly improves the processing performance and resource consumption of the algorithm.
[0085] The technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description
[0086] The accompanying drawings, which form part of this specification, illustrate embodiments of this disclosure and, together with the description, serve to explain the principles of this disclosure.
[0087] Figure 1 is a schematic diagram of an interior wall structure with overlapping and intersecting elements provided in an embodiment of this disclosure.
[0088] Figure 2 is one of the flowcharts of the point cloud pose optimization method according to an embodiment of this disclosure.
[0089] Figure 3 is a second flowchart of the point cloud pose optimization method according to an embodiment of this disclosure.
[0090] Figure 4 is a flowchart of the point cloud pose optimization method according to an embodiment of this disclosure.
[0091] Figure 5 is an exemplary schematic diagram of the point cloud frame pose points of the room and transition area in an embodiment of this disclosure.
[0092] Figure 6 is an exemplary top view of the target space after point cloud optimization in an embodiment of this disclosure.
[0093] Figure 7 is a structural block diagram of a point cloud pose optimization device according to an embodiment of the present disclosure.
[0094] Figure 8 is a structural block diagram of an electronic device according to an embodiment of the present disclosure. Detailed Implementation
[0095] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.
[0096] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of this disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.
[0097] It should also be understood that in the embodiments disclosed herein, "a plurality of" may refer to two or more, and "at least one" may refer to one, two or more.
[0098] It should also be understood that any component, data or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless expressly defined or given to the contrary in the context.
[0099] Furthermore, the term "and / or" in this disclosure is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this disclosure generally indicates that the preceding and following related objects have an "or" relationship.
[0100] It should also be understood that the description of the various embodiments in this disclosure emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.
[0101] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.
[0102] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.
[0103] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.
[0104] This disclosure can be applied to electronic devices such as SLAM devices, terminal devices, computer systems, and servers, and can operate with a wide range of other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as SLAM devices, terminal devices, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.
[0105] Electronic devices such as SLAM devices, terminal devices, computer systems, and servers can be described in the general context of computer-executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.
[0106] During the point cloud data acquisition process of LiDAR on target spaces such as houses, the overall structure of a house is relatively complex, possibly including multiple rooms and other sub-spaces, as well as transition areas between rooms, such as doorways. Furthermore, the thickness of the walls varies, with some being too thin. These factors can cause phenomena such as thickened walls, overlapping walls, and mixed wall layers in the point cloud products of the target space. As shown in Figure 1, the areas enclosed by the dashed boxes "1" and "2" are both walls of a wall between two rooms. At the locations enclosed by the dashed boxes "1" and "2," the walls overlap, making it difficult to distinguish between the inner and outer walls at these two locations.
[0107] To address the aforementioned technical problems, this disclosure provides a point cloud pose optimization method, apparatus, electronic device, computer-readable storage medium, and computer program product. The technical solutions of this disclosure are described in detail below with reference to the accompanying drawings.
[0108] Figure 2 is a flowchart of one embodiment of a point cloud pose optimization method according to the present disclosure. As shown in Figure 2, a point cloud pose optimization method according to an embodiment of the present disclosure may include the following steps:
[0109] Step 201: Determine the first local point cloud frame and the second local point cloud frame within the target space based on the first overall point cloud data of the target space.
[0110] For example, the target space can be any space for point cloud generation or 3D reconstruction, such as a house containing multiple rooms and doorways between them. The first local point cloud frame represents the point cloud data of any room within the target space, while the second local point cloud frame represents the point cloud data of the transition area between the rooms within the target space. This separation of the point cloud frame data of each room from the point cloud frame data of the doorways (transition area) allows for separate processing in subsequent point cloud optimization, reducing the processing overhead of the point cloud frame data and improving processing efficiency.
[0111] The first overall point cloud data in this embodiment can be point cloud data collected from the target space by a lidar mounted on a SLAM device or any other point cloud acquisition device.
[0112] In an optional example, step 201 can be executed by the processor calling the corresponding instructions stored in memory, or by the local point cloud frame determination module 701 run by the processor.
[0113] Step 202: Establish a first point-plane matching pair based on the first local point cloud frame, optimize the point cloud pose of the room to which the first local point cloud frame belongs based on the first point-plane matching pair, and merge the optimized first local point cloud frames to obtain the third local point cloud of the room.
[0114] The first point-plane matching pair represents the matching relationship between multiple point clouds in the first local point cloud frame and their respective planes.
[0115] The room point cloud data, i.e. the first local point cloud frame, is obtained through the previous step 201. In this step 202, the pose optimization of the room point cloud is achieved by optimizing the first local point cloud frame.
[0116] In an optional example, step 202 can be executed by the processor calling the corresponding instruction stored in memory, or by the room pose optimization module 702 run by the processor.
[0117] Step 203: Establish a second point-plane matching pair based on the second local point cloud frame, optimize the point cloud pose of the transition area between rooms to which the second local point cloud frame belongs based on the second point-plane matching pair, and merge the optimized second local point cloud frames to obtain the fourth local point cloud of the transition area.
[0118] The second point-plane matching pair represents the matching relationship between multiple point clouds in the second local point cloud frame and their respective planes.
[0119] In step 203, each transition area between rooms can be treated as a separate whole for point cloud pose optimization. For details, please refer to the point cloud pose optimization process for a single room in step 202 above. For the sake of brevity, it will not be repeated here.
[0120] In an optional example, step 203 can be executed by the processor calling the corresponding instruction stored in memory, or by the transition region pose optimization module 703 run by the processor.
[0121] Step 204: Perform point cloud stitching and fusion on the third local point cloud of the room in the target space and the fourth local point cloud of the transition area, and establish a third point-plane matching pair. Based on the third point-plane matching pair, optimize the point cloud pose of the room to which the third local point cloud belongs and the transition area to which the fourth local point cloud belongs, and merge the optimized third local point cloud and the fourth local point cloud to obtain the second overall point cloud data of the target space.
[0122] Among them, the third point-plane matching pair represents the matching relationship between multiple point clouds in the third and fourth local point clouds and their respective planes.
[0123] After completing the pose optimization of the point cloud frames of the rooms and transition areas between rooms in the target space through the aforementioned steps, in this step 204, the pose-optimized point cloud frames of the rooms and transition areas between rooms in the target space are stitched together and fused to obtain new overall point cloud data of the target space, namely the second overall point cloud data of the target space.
[0124] In an optional example, step 204 can be executed by the processor calling the corresponding instruction stored in memory, or by the spatial pose optimization module 704 run by the processor.
[0125] It is understandable that performing point cloud pose optimization separately for each room within the target space, and optimizing the point cloud pose of each transition area between rooms as a separate whole, and then stitching together the point clouds of each part, may result in phenomena such as wall thickening due to room transitions, wall overlap, and wall layering caused by excessively thin walls, between different rooms, between different transition areas, and between rooms and transition areas. Therefore, in this embodiment, after stitching and fusing to obtain the second overall point cloud data of the target space, further overall pose optimization of the point cloud frames of the target space is performed based on this second overall point cloud data. This helps improve the overall quality of the point cloud optimization of the target space and avoids erroneous wall structure information such as wall thickening, wall overlap, and wall layering.
[0126] Optionally, as some implementations of the embodiments of this disclosure, step 201 can be implemented by the following steps: First, calculate the normal vector of any point cloud in the point cloud frame of the target space to obtain the normal vector of any point cloud; then, merge the point clouds with normal vectors to obtain the first overall point cloud data of the target space; then, perform planar segmentation and recognition processing on the first overall point cloud data based on the normal vectors of each point cloud to obtain the wall region and doorway region of the target space; finally, obtain the first local point cloud frame and the second local point cloud frame according to the point clouds of the wall region and the doorway region respectively. The normal vectors of each point cloud can be calculated, for example, through the geometric relationships between points in the point cloud. For instance, for any point M in a point cloud frame, the set of neighboring points P(M) of point M can be determined through neighborhood point lookup (e.g., KD-tree, octree), etc.). Then, the vector between each point in the neighborhood set P(M) and point M is calculated. The calculated vectors are then averaged, and the average vector is normalized to obtain the normalized average normal vector of the point cloud in each point cloud frame, which serves as the normal vector of each point cloud in the point cloud frame. It is understood that the point cloud normal vector calculation method proposed in the above embodiments is only an example, and this disclosure does not limit the calculation method of point cloud normal vectors.
[0127] In this embodiment of the disclosure, the point cloud frame of the target space can be composed of tens of thousands of point cloud frames, and each point cloud frame can include a large number of point clouds (e.g., more than 20,000 points). By calculating the normal vector of each point cloud, the point cloud frames with normal vectors are merged together to obtain the overall point cloud of the target space with normal vectors (i.e., the first overall point cloud data). Further, the point cloud data of the walls and doorways in the target space can be identified by, for example, a plane segmentation method (e.g., region growing or other methods that can achieve plane segmentation of point clouds) and a wall and doorway recognition method (e.g., some machine learning models that can achieve shape or object detection). Thus, based on the point cloud data of the walls, the point cloud frames of the rooms in the target space (i.e., the first local point cloud frames) can be obtained, and based on the point cloud data of the doorways, the point cloud frames of the transition area (i.e., the second local point cloud frames) can be obtained. As shown in Figure 5, the position marked "1" in the figure is the pose point of the point cloud frame in the transition area. The area formed by the other lines is the pose point of the point cloud frame in each room. The transition area is the trajectory connection of the point cloud frame data. By connecting the trajectories of the point clouds in each room together, the point cloud trajectory of the entire target space can be formed.
[0128] Figure 3 is a second flowchart of the point cloud pose optimization method according to an embodiment of this disclosure. Referring to Figure 3, based on the above embodiment, step 202 may include the following steps:
[0129] Step 2021: Search for multiple target neighboring points of the target point cloud using a preset search tree. The target point cloud is any point cloud in the first point cloud frame of the first local point cloud frame, and the multiple target neighboring points are multiple point clouds in the second point cloud frame of the first local point cloud frame.
[0130] In the first local point cloud frame, a neighborhood point search can be performed between any two point cloud frames to determine whether there are overlapping regions between different point cloud frames in the first local point cloud frame. For example, in some implementation examples, a preset search tree for the first point cloud frame in the first local point cloud frame can be established first for the second point cloud frame in the first local point cloud frame. Then, each point cloud in the first point cloud frame in the first local point cloud frame is traversed, and the normal vector dot product between the point cloud and each point cloud in the second point cloud frame in the first local point cloud frame is calculated. Then, for each point cloud in the second point cloud frame in the first local point cloud frame, it is determined whether the normal vector dot product satisfies the preset dot product condition. Then, the point clouds in the second point cloud frame in the first local point cloud frame whose normal vector dot product satisfies the preset dot product condition are determined as target searched point clouds. Finally, the target searched point clouds are searched through the preset search tree to determine the target searched point clouds that satisfy the search conditions of the preset search tree as target neighborhood points. The preset search tree can be a KD tree. For example, a preset number of neighborhood matching points (i.e. target neighborhood points) of each point in a certain frame of point cloud (e.g., the aforementioned first point cloud frame) can be searched using a KD tree. As an optional embodiment, when using KD-tree search, it is considered that the wall has two sides (including the outer wall and the inner wall), including the point cloud data of the outer wall and the inner wall. The normal vectors of the point clouds of the outer wall and the inner wall are in opposite directions. The trajectory of the point cloud is collected from outside the doorway to inside the doorway. If the point cloud scanned outside the doorway is already in the KD-tree, it is easy to match point clouds that do not belong to the wall when searching for neighboring points through the KD-tree, thus causing incorrect matching of neighboring points, and consequently leading to incorrect matching of the wall point cloud in the subsequent process. To address this, this embodiment of the disclosure uses normal vector constraints, that is, only the point clouds in the second point cloud frame of the first local point cloud frame whose normal vector dot product satisfies the preset dot product condition are determined as the target search point clouds. In this way, it is ensured that the point cloud of the inner wall is only matched with the point cloud of the inner wall, and the point cloud of the outer wall is only matched with the point cloud of the outer wall, thereby improving the matching accuracy of neighboring points and providing a more accurate data foundation for subsequent pose optimization. As another optional embodiment, since the point cloud acquisition trajectory is more likely to cause incorrect matching of the point cloud of the outer wall, the search can also be carried out by constraining the neighborhood points of the point cloud of the outer wall only through the normal vector. For example, for the point cloud of the outer wall, only the point cloud in the outer wall whose normal vector dot product satisfies the preset dot product condition is searched.In this embodiment of the disclosure, the preset dot product condition can be, for example, the dot product of the normal vector being greater than zero or other values slightly greater than zero. For example, when searching for the target neighborhood points of point cloud A in the first point cloud frame using a KD tree, assuming that only 7 points (B, C, D, E, F, G, and H) in the second point cloud frame satisfy the preset dot product condition that the dot product of the normal vector is greater than 0, the KD tree search will only search for B, C, D, E, F, G, and H. This can also reduce the resource overhead of the KD tree search and improve the efficiency of point cloud processing. If only, for example, B, C, D, E, and F satisfy the KD tree search condition (e.g., within its search radius), then 5 points (B, C, D, E, and F) in the second point cloud frame are determined as the target neighborhood points of point cloud A in the first point cloud frame.
[0131] In an optional example, step 2021 can be executed by the processor calling the corresponding instructions stored in memory, or by the target neighborhood point search unit run by the processor.
[0132] Step 2022: Determine whether the target point cloud and its multiple target neighboring points belong to the same point cloud plane.
[0133] Taking point A (the target point cloud) in the first point cloud frame (step 2021) and its target neighbor points B, C, D, E, and F (multiple target neighbor points of point A) in the second point cloud frame as an example, there are a total of 6 point clouds. In this step 2022, it is determined whether these 6 point clouds belong to the same point cloud plane. To determine whether points A, B, C, D, E, and F belong to the same plane, methods can be used, but are not limited to, the following: the three-point plane determination method, the Random Sample Consensus (RANSAC) algorithm, and the 3D Hough transform. Taking the RANSAC algorithm as an example, it is a random sampling consensus algorithm that can be used to estimate the parameters of a mathematical model from a set of observation data containing outliers. The specific steps for determining whether points A, B, C, D, E, and F belong to the same plane can include: First, randomly select a point and its normal vector (e.g., point A). Assume that A and its normal vector define a plane M. Then, calculate the distances from points B, C, D, E, and F to this plane and determine whether the distance is within a pre-set distance range (which can be determined based on prior knowledge, and this embodiment does not impose any restrictions on this). If it is within the distance range, then points within the distance range and point A both belong to points within plane M.
[0134] In an optional example, step 2022 can be executed by the processor calling the corresponding instruction stored in memory, or by a coplanar decision unit run by the processor.
[0135] Step 2023: If the target point cloud and its multiple target neighboring points belong to the same point cloud plane, determine that there is an overlapping area between the first point cloud frame and the second point cloud frame.
[0136] The first and second point cloud frames are two different point cloud frames in space. If the points between them belong to the same point cloud plane, then that point cloud plane can be determined as the overlapping area between the two, thus confirming the existence of an overlapping area between the two different point cloud frames in the space. For example, if point cloud frame T1 has a sampling range of 50° to 100° for the space, while point cloud frame T2 has a sampling range of 90° to 150° for the space, then point cloud frames T2 and T1 overlap in the 90° to 100° range. This overlap is due to the overlapping sampling ranges, which can cause the wall thickness to be less than the actual wall thickness, i.e., the wall to be too thin. This can lead to problems such as walls overlapping on both sides and walls being layered and mixed.
[0137] In an optional example, step 2023 can be executed by the processor calling the corresponding instruction stored in memory, or by the overlapping region determination unit run by the processor.
[0138] Step 2024: In the case that there is an overlapping area between the first point cloud frame and the second point cloud frame, a matching relationship between the point cloud and the point cloud plane to which the multiple point clouds belong is constructed to obtain the first point-plane matching pair. The multiple point clouds include the target point cloud and its multiple target neighborhood points.
[0139] When there is an overlapping area between the first and second point cloud frames, point-to-plane matching relationships are established to obtain multiple first point-to-plane matching pairs. Taking target point A and its neighboring points B, C, D, E, and F in the first point cloud frame as an example, these six points all belong to point cloud plane M. Therefore, a matching relationship can be established between points A, B, C, D, E, and F and plane M, forming first point cloud matching pairs corresponding to points A, B, C, D, E, and F and point cloud plane M. After the aforementioned steps, multiple first point-to-plane matching pairs can be established through this step 2024.
[0140] In an optional example, step 2024 can be executed by the processor calling the corresponding instruction stored in memory, or by the first point-to-surface matching unit run by the processor.
[0141] Step 2025: Adjust the pose of the first point cloud frame and the second point cloud frame by the first point-plane matching pair so that the first point-plane matching pair meets the first preset optimization condition.
[0142] As an optional embodiment, when the point-plane matching relationship (first point-plane matching pair) between any first point cloud frame and second point cloud frame satisfies a preset number, pose optimization constraints between the two point cloud frames can be constructed based on the first point-plane matching pair. The preset number can be determined based on the degree of overlap between the two point cloud frames. If the overlap is large, the preset number can be set higher; conversely, if the overlap is small, the preset number can be set lower. This disclosure does not limit the specific value of the preset number. In a specific example, the first and second point cloud frames can be iteratively rotated and / or translated until the distance between the point cloud in the first point-plane matching pair and its point cloud plane satisfies a first preset optimization condition. The first preset optimization condition can, for example, be that the distance from each point in the first point-plane matching pair to its corresponding point cloud plane is minimized. For example, suppose the displacement difference between point cloud frames T1 and T2 is 2cm. In order to minimize the distance from each point in the first point-plane matching pair in T1 and T2 to its corresponding point cloud plane, the point clouds in T1 and T2 can be rotated and / or translated iteratively. If the optimization constraint of minimizing the distance from each point in the first point-plane matching pair in T1 and T2 to its corresponding point cloud plane is satisfied when eliminating the displacement difference (2cm) between T1 and T2, then the iteration can be stopped, and the pose optimization of T1 and T2 can be completed.
[0143] It is understandable that the first local point cloud frame in a room may include multiple first point cloud frames and second point cloud frames. Therefore, the pose optimization of the point cloud frames can be completed by traversing all the point cloud frames in the first local point cloud frame using the aforementioned method.
[0144] In an optional example, step 2025 can be executed by the processor calling the corresponding instruction stored in memory, or by the first pose adjustment unit run by the processor.
[0145] Step 2026: Merge the first point-plane matching pairs that meet the first preset optimization conditions to obtain the global point-plane matching pairs of the room.
[0146] The first point-plane matching pair in the point cloud frame that has completed pose optimization (meeting the first preset optimization condition) is merged to finally obtain the point-plane matching pair of a single room, that is, the global point-plane matching pair.
[0147] In an optional example, step 2026 can be executed by the processor calling the corresponding instruction stored in memory, or by the second point-to-surface matching unit run by the processor.
[0148] Step 2027: Adjust the pose of the point cloud frame in the first local point cloud frame of the room by using a preset point cloud optimizer and a global point-plane matching pair, so that the global point-plane matching pair meets the second preset optimization condition.
[0149] A global point-plane matching pair is obtained within a single room (that is, the point-plane matching pairs constructed frame-to-frame are put together). Then, a preset point cloud optimizer (e.g., least squares method) is used to optimize the point cloud frame of the single room as a whole. That is, the coordinates of the point cloud frame of the single room are adjusted by the preset point cloud optimizer (rotating or translating the point cloud frame) so that the geometric relationship between the points and planes in the global point-plane matching pair satisfies the second preset optimization condition. The second preset optimization condition can be, for example, that the distance between any point in the global point-plane matching pair and its corresponding point cloud plane is less than a preset distance threshold. The specific value of the preset distance threshold can be determined according to actual needs. This embodiment of the disclosure does not impose any restrictions on the value of the threshold.
[0150] In an optional example, step 2027 can be executed by the processor calling the corresponding instruction stored in memory, or by the second pose adjustment unit run by the processor.
[0151] Step 2028: Merge the point cloud frames in the first local point cloud frame of the room corresponding to the global point-surface matching pair when the second preset optimization condition is met, and determine the point cloud frames in the first local point cloud frame of the room corresponding to the merged global point-surface matching pair when the second preset optimization condition is met as the third local point cloud of the room.
[0152] The optimized global point-surface matching is used to merge and fuse the point cloud frames in the corresponding first local point cloud to finally obtain the third local point cloud of a single room, thus completing the point cloud optimization of a single room.
[0153] In an optional example, step 2028 can be executed by the processor calling the corresponding instruction stored in memory, or by the second point cloud frame determination unit run by the processor.
[0154] Optionally, as another implementation of the present disclosure, in step 202, before executing step 2025, it can be determined whether the total number of the first local point cloud frames exceeds the preset number of point cloud frames (e.g., 1000). If the total number of the first local point cloud frames exceeds the preset number of point cloud frames, the first local point cloud frame is divided into point cloud frames of multiple subspaces. Then, for each subspace point cloud frame, steps 2025 to 2028 are executed to optimize the point cloud pose of the room to which the first local point cloud frame belongs. After that, the optimized first and second point cloud frames of the multiple subspaces are merged to obtain the third local point cloud of the room. For example, when the number of point cloud frames in a single room is too large (i.e., exceeds the preset number of point cloud frames), these point cloud frames can be randomly divided into multiple subspaces. For instance, for room P, its point cloud frames can be divided into multiple subspaces such as P1, P2, P3, and P4. Steps 2025 to 2028 described above are performed on the point cloud frames of each subspace to optimize the point cloud frames of P1, P2, P3, and P4. Then, the optimized point cloud frames from multiple subspaces are combined to form the overall point cloud frame of room P for secondary optimization. In this way, optimizing each subspace individually can improve the overall performance. Furthermore, if a single room is too large, it may contain thousands of point cloud frames, resulting in tens of thousands of first point-to-surface matching pairs. By splitting it into multiple subspaces, computation can be reduced, saving resource overhead in point cloud processing.
[0155] Optionally, in some implementations of the embodiments of this disclosure, as shown in FIG4, step 204 can be implemented by the following steps:
[0156] Step 2041: Perform point cloud stitching and fusion on the third local point cloud of the room in the target space and the fourth local point cloud of the transition area, and combine the optimized first point-surface matching pair of the third local point cloud and the optimized second point-surface matching pair of the fourth local point cloud to form the third point-surface matching pair.
[0157] In an optional example, step 2041 can be executed by the processor calling the corresponding instruction stored in memory, or by the point-face matching pair acquisition unit run by the processor.
[0158] Step 2042: Perform coordinate rotation and / or coordinate translation on the point cloud frames belonging to each room and the point cloud frames belonging to the transition area between each room in the second overall point cloud data, until the distance between the point cloud in the third point-plane matching pair and its point cloud plane meets the third preset optimization condition, and obtain the second overall point cloud data of the target space.
[0159] The third preset optimization condition can be, for example, minimizing the distance between the point cloud in the third point-plane matching pair and its corresponding point cloud plane, or satisfying a preset distance threshold. The specific value of this preset distance threshold can be determined according to actual needs, and this embodiment does not impose any restrictions on it. Through this embodiment, pose optimization matching is further performed between the transition area and each room. By minimizing the point cloud-plane distance between the transition area and each room, or satisfying the preset distance threshold, the pose of the point cloud frames between the transition area and each room is adjusted, so that the poses of the point cloud frames between each room, between transition areas, and between the transition area and each room in the target space are optimally matched. This helps to further improve the overall quality of point cloud optimization in the target space and avoid erroneous wall structure information such as wall thickening, wall overlap, and wall layering. Furthermore, through this embodiment, only the poses of the point cloud frames between each room, between transition areas, and between the transition area and each room in the target space are matched, without repeatedly processing the poses of the point cloud frames of a single room or transition area. This reduces the resource overhead of point cloud pose optimization and improves the efficiency of point cloud frame optimization processing in the target space.
[0160] In an optional example, step 2042 can be executed by the processor calling the corresponding instruction stored in memory, or by the third pose adjustment unit run by the processor.
[0161] Optionally, in some implementations of the embodiments of this disclosure, the point-plane matching relationship can be recreated in the second overall point cloud data of the target space by searching the target neighborhood points of the target point cloud, to obtain a third point-plane matching pair. Then, based on the third point-plane matching, the pose optimization method of the point cloud frame of a single room or a single transition area is referred to to achieve the overall point cloud pose optimization of the target space. However, in this optimization method, since there are many point cloud frames in the target space, the number of point-plane matching pairs obtained will also be larger, which may lead to a large overhead of point cloud computing processing resources.
[0162] Figure 6 shows an exemplary top view of the target space after point cloud optimization according to the method embodiment of this disclosure. Figures 1 and 6 are top views of the point cloud before and after point cloud optimization of the same target space, respectively. Figure 6 is a diagram showing the effect of point cloud pose optimization according to the embodiment of this disclosure. Compared with Figure 1, the phenomenon of overlapping walls at the positions indicated by dashed boxes "1" and "2" in Figure 1 has been eliminated. See the positions indicated by dashed circles "1" and "2" in Figure 6.
[0163] In summary, based on the technical solution provided in the embodiments of this disclosure, by distinguishing between each room in the target space and the transition area between each room, the point cloud frame data of the rooms and the transition area in the target space are clustered and segmented respectively, and point-face matching pairs are established based on the point cloud frames to constrain the pose inside each room, thereby optimizing the pose inside the room; the transition area between rooms is also treated as a separate whole and its pose is optimized in a similar manner to that of the room; finally, the point clouds of the optimized rooms and the transition area are stitched together and then the overall pose is optimized again in a similar manner to obtain the optimized point cloud of the target space. Since most errors, such as wall thickening, wall overlap, and wall layering, originate from the transition areas between rooms, this embodiment optimizes the point cloud pose of the transition areas separately. This separates the parts with large and small errors in the point cloud of the entire target space, which helps improve the accuracy of point cloud pose optimization and solves the matching problems of wall thickening, wall overlap, and wall layering. In addition, the method of dividing the point cloud frames of the rooms and transition areas in the target space into blocks and optimizing the point cloud pose separately reduces the overall scale of the target space optimization at one time. Moreover, the optimization between rooms and transition areas can be processed in parallel, which greatly improves the processing performance and resource consumption of the algorithm.
[0164] Any pose optimization method provided in this disclosure can be executed by any suitable device with data processing capabilities, including but not limited to: terminal devices and servers. Alternatively, any pose optimization method provided in this disclosure can be executed by a processor, such as by a processor executing any pose optimization method mentioned in this disclosure by calling corresponding instructions stored in memory. Further details will not be elaborated below.
[0165] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as ROM, RAM, magnetic disk, or optical disk.
[0166] Correspondingly, this disclosure also provides apparatus embodiments corresponding to the foregoing method embodiments. The apparatus embodiments provided in this disclosure can be used to implement the above-described method embodiments. The apparatus embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.
[0167] Figure 7 is a structural block diagram of a point cloud pose optimization device according to an embodiment of the present disclosure. As shown in Figure 7, a point cloud pose optimization device provided in an embodiment of the present disclosure may include a local point cloud frame determination module 701, a room pose optimization module 702, a transition region pose optimization module 703, and a spatial pose optimization module 704, wherein:
[0168] The local point cloud frame determination module 701 is used to determine the first local point cloud frame and the second local point cloud frame in the target space based on the first overall point cloud data of the target space. The first local point cloud frame is the point cloud frame data of any room in the target space, and the second local point cloud frame is the point cloud frame data of the transition area between the rooms in the target space.
[0169] The room pose optimization module 702 is used to establish a first point-plane matching pair based on the first local point cloud frame, to optimize the point cloud pose of the room to which the first local point cloud frame belongs based on the first point-plane matching pair, and to merge the optimized first local point cloud frame to obtain the third local point cloud of the room.
[0170] The transition region pose optimization module 703 is used to establish a second point-plane matching pair based on the second local point cloud frame, to optimize the point cloud pose of the transition region between rooms to which the second local point cloud frame belongs based on the second point-plane matching pair, and to merge the optimized second local point cloud frames to obtain the fourth local point cloud of the transition region.
[0171] The spatial pose optimization module 704 is used to perform point cloud stitching and fusion on the third local point cloud of the room in the target space and the fourth local point cloud of the transition area, and establish a third point-plane matching pair. Based on the third point-plane matching pair, the point cloud pose of the room to which the third local point cloud belongs and the transition area to which the fourth local point cloud belongs are optimized, and the optimized third local point cloud and the fourth local point cloud are merged to obtain the second overall point cloud data of the target space.
[0172] Among them, the first point-plane matching pair, the second point-plane matching pair, and the third point-plane matching pair represent the matching relationship between multiple point clouds and their respective planes.
[0173] Based on the embodiment shown in Figure 7, as an optional embodiment, the local point cloud frame determination module 701 may include (not shown in the figure): a normal calculation unit, used to calculate the normal vector of any point cloud in the point cloud frame of the target space to obtain the normal vector of each point cloud; a point cloud merging unit, used to merge the point clouds with normal vectors to obtain the first overall point cloud data of the target space; a segmentation and recognition unit, used to perform planar segmentation and recognition processing on the first overall point cloud data based on the normal vector of the point cloud to obtain the wall area and the doorway area of the target space; and a first point cloud frame determination unit, used to obtain a first local point cloud frame and a second local point cloud frame according to the point clouds of the wall area and the doorway area, respectively.
[0174] As an optional embodiment, the room pose optimization module 702 may include (not shown in the figure): a target neighborhood point search unit, used to search for multiple target neighborhood points of a target point cloud through a preset search tree, wherein the target point cloud is any point cloud in the first point cloud frame of the first local point cloud frame, and the multiple target neighborhood points are multiple point clouds in the second point cloud frame of the first local point cloud frame; a plane judgment unit, used to determine whether the target point cloud and its multiple target neighborhood points belong to the same point cloud plane; an overlapping region determination unit, used to determine that there is an overlapping region between the first point cloud frame and the second point cloud frame when it is determined that the target point cloud and its multiple target neighborhood points belong to the same point cloud plane; and a first point-plane matching unit, used to construct a matching relationship between the point cloud and the point cloud plane based on the multiple point clouds and the point cloud plane to which the multiple point clouds belong when there is an overlapping region between the first point cloud frame and the second point cloud frame, to obtain the first point-plane matching pair, wherein the multiple The point cloud includes the target point cloud and its multiple target neighboring points; a first pose adjustment unit is used to adjust the pose of the first point cloud frame and the second point cloud frame through a first point-plane matching pair, so that the first point-plane matching pair meets a first preset optimization condition; a second point-plane matching unit is used to merge the first point-plane matching pairs that meet the first preset optimization condition to obtain the global point-plane matching pair of the room; a second pose adjustment unit is used to adjust the pose of the point cloud frames in the first local point cloud frame of the room through a preset point cloud optimizer and the global point-plane matching pair, so that the global point-plane matching pair meets a second preset optimization condition; a second point cloud frame determination unit is used to merge the point cloud frames in the first local point cloud frame of the room corresponding to the global point-plane matching pair meeting the second preset optimization condition, and determine the point cloud frames in the first local point cloud frame of the room corresponding to the merged global point-plane matching pair meeting the second preset optimization condition as the third local point cloud of the room.
[0175] As an optional embodiment, the room pose optimization module 702 may further include (not shown in the figure): a point cloud quantity judgment unit, used to determine whether the total number of the first local point cloud frames exceeds a preset number of point cloud frames before adjusting the pose of the first point cloud frame and the second point cloud frame through the first point-plane matching pair so that the first point-plane matching pair meets the first preset optimization condition; a space division unit, used to divide the first local point cloud frame into point cloud frames of multiple subspaces when the total number of the first local point cloud frames exceeds the preset number of point cloud frames; and a second pose adjustment unit, used to perform the adjustment of the pose of the first point cloud frame and the second point cloud frame through the first point-plane matching pair so that the first point-plane matching pair meets the first preset optimization condition and its subsequent steps for any subspace point cloud frame, so as to optimize the point cloud pose of the room to which the first local point cloud frame belongs, and merge the optimized first point cloud frames and the second point cloud frames of the multiple subspaces to obtain the second local point cloud frame of the room.
[0176] As an optional embodiment of this disclosure, the target neighborhood point search unit may include (not shown in the figure): a search tree building subunit, used to build a preset search tree for the first point cloud frame in the first local point cloud frame for the second point cloud frame in the first local point cloud frame; a normal vector dot product calculation subunit, used to traverse each point cloud in the first point cloud frame in the first local point cloud frame and calculate the normal vector dot product between the point cloud and each point cloud in the second point cloud frame in the first local point cloud frame; a dot product condition judgment subunit, used to determine whether the normal vector dot product satisfies a preset dot product condition for any point cloud in the second point cloud frame in the first local point cloud frame; a searched point cloud determination subunit, used to determine the point cloud in the second point cloud frame in the first local point cloud frame whose normal vector dot product satisfies the preset dot product condition as the target searched point cloud; and a target neighborhood point determination subunit, used to search the target searched point cloud through the preset search tree to determine the searched target searched point cloud that satisfies the search conditions of the preset search tree as the target neighborhood point.
[0177] As an optional embodiment of this disclosure, the first pose adjustment unit may include (not shown in the figure): a pose adjustment subunit, used to iteratively rotate and / or translate the first point cloud frame and the second point cloud frame until the distance between the point cloud in the first point-plane matching pair and its point cloud plane satisfies the first preset optimization condition.
[0178] As an optional embodiment of this disclosure, the spatial pose optimization module 704 may include (not shown in the figure): a point-plane matching pair acquisition unit, used to perform point cloud stitching and fusion on the third local point cloud of the room and the fourth local point cloud of the transition area in the target space, and combine the optimized first point-plane matching pair of the third local point cloud and the optimized second point-plane matching pair of the fourth local point cloud to form a third point-plane matching pair; a third pose adjustment unit, used to perform coordinate rotation and / or coordinate translation on the point cloud frames belonging to the room and the point cloud frames belonging to the transition area between the rooms in the second overall point cloud data, until the distance between the point cloud in the third point-plane matching pair and its point cloud plane meets the third preset optimization condition, thereby obtaining the second overall point cloud data of the target space.
[0179] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0180] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. The same, similar, or corresponding parts between the embodiments can be referenced, combined, or cited interchangeably. Since the method and apparatus embodiments are basically corresponding, relevant parts can be referred to in the corresponding descriptions, and the two can be referenced, combined, or cited interchangeably. The methods and apparatuses of the embodiments in this disclosure also correspond to each other in specific implementation and beneficial technical effects; related content can be referred to interchangeably and will not be repeated here.
[0181] Hereinafter, an electronic device according to an embodiment of the present disclosure will be described with reference to FIG8. The electronic device may be either or both of a first device and a second device, or a standalone device independent of them, which may communicate with the first device and the second device to receive acquired input signals from them.
[0182] Figure 8 illustrates a block diagram of an electronic device according to an embodiment of the present disclosure.
[0183] As shown in Figure 8, the electronic device includes one or more processors and a memory.
[0184] A processor can be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and can control other components in an electronic device to perform desired functions.
[0185] The memory can store one or more computer program products (hereinafter referred to as program products), and the memory can include various forms of computer-readable storage media (hereinafter referred to as media), such as volatile memory and / or non-volatile memory. The volatile memory can, for example, include random access memory (RAM) and / or cache memory. The non-volatile memory can, for example, include read-only memory (ROM), hard disk, flash memory, etc. One or more computer program products can be stored on the computer-readable storage medium, and the processor can run the computer program products to implement the point cloud pose optimization methods of the various embodiments of this disclosure described above and / or other desired functions.
[0186] In one example, the electronic device may also include input devices and output devices, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).
[0187] In addition, the input device may also include, for example, a keyboard, a mouse, etc.
[0188] This output device can output various information to the outside, including determined distance information, direction information, etc. The output device may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.
[0189] Of course, for simplicity, Figure 8 only shows some of the components of the electronic device that are relevant to this disclosure, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device may include any other suitable components depending on the specific application.
[0190] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products, including computer program instructions that, when executed by a processor, cause the processor to perform the steps in the point cloud pose optimization methods according to various embodiments of this disclosure as described in the foregoing portion of this specification.
[0191] The computer program product can be written in any combination of one or more programming languages to perform the operations of the embodiments of this disclosure. The programming languages include object-oriented programming languages such as Java and C++, as well as conventional procedural programming languages such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.
[0192] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform the steps in the point cloud pose optimization methods according to various embodiments of this disclosure as described in the foregoing portion of this specification.
[0193] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0194] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.
[0195] The block diagrams of the devices involved in this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.
[0196] The methods and apparatus of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this disclosure are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the methods according to this disclosure.
[0197] It should also be noted that in the apparatus and method of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.
[0198] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.
[0199] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.
Claims
1. A point cloud pose optimization method, characterized in that, The method comprises: determining a first local point cloud frame and a second local point cloud frame in the target space according to first overall point cloud data of the target space, wherein the first local point cloud frame is point cloud frame data of any room in the target space, and the second local point cloud frame is point cloud frame data of a transition area between rooms in the target space; establishing a first point-plane matching pair according to the first local point cloud frame, so as to perform point cloud pose optimization on a room to which the first local point cloud frame belongs based on the first point-plane matching pair, and merging the optimized first local point cloud frame to obtain a third local point cloud of the room; establishing a second point-plane matching pair according to the second local point cloud frame, so as to perform point cloud pose optimization on a transition area between rooms to which the second local point cloud frame belongs based on the second point-plane matching pair, and merging the optimized second local point cloud frame to obtain a fourth local point cloud of the transition area; performing point cloud splicing fusion on the third local point cloud of the room and the fourth local point cloud of the transition area in the target space, and establishing a third point-plane matching pair, so as to perform point cloud pose optimization on a room to which the third local point cloud belongs and a transition area to which the fourth local point cloud belongs based on the third point-plane matching pair, and merging the optimized third local point cloud and fourth local point cloud to obtain second overall point cloud data of the target space; wherein the first point-plane matching pair, the second point-plane matching pair and the third point-plane matching pair respectively represent the matching relationship between a plurality of point clouds and the planes to which they belong.
2. The method of claim 1, wherein, The method comprises: performing normal vector calculation on any point cloud in the point cloud frame of the target space to obtain the normal vector of the point cloud; merging the point cloud with the normal vector to obtain the first overall point cloud data of the target space; performing plane segmentation and identification processing on the first overall point cloud data based on the normal vector of the point cloud to obtain the wall area and the door hole area of the target space; obtaining the first local point cloud frame and the second local point cloud frame from the point cloud of the wall area and the door hole area respectively.
3. The method according to claim 1 or 2, characterized in that, The method comprises: searching for a plurality of target neighborhood points of a target point cloud through a preset search tree, wherein the target point cloud is any point cloud in a first point cloud frame in the first local point cloud frame, and the plurality of target neighborhood points are a plurality of point clouds in a second point cloud frame in the first local point cloud frame; determining whether the target point cloud and its plurality of target neighborhood points belong to the same point cloud plane; if the target point cloud and its plurality of target neighborhood points belong to the same point cloud plane, it is determined that the first point cloud frame and the second point cloud frame have an overlapping area; In a case where the first point cloud frame and the second point cloud frame exist an overlapping area, a matching relationship between point clouds and point cloud planes is constructed based on the plurality of point clouds and the point cloud planes to which the plurality of point clouds belong, the plurality of point clouds including the target point cloud and a plurality of target neighborhood points thereof, to obtain the first point-plane matching pair. The poses of the first point cloud frame and the second point cloud frame are adjusted through the first point-plane matching pair, so that the first point-plane matching pair satisfies a first preset optimization condition. The first point-plane matching pairs satisfying the first preset optimization condition are merged to obtain a global point-plane matching pair of the room. The pose of a first point cloud frame in a first local point cloud frame of the room is adjusted through a preset point cloud optimizer and the global point-plane matching pair, so that the global point-plane matching pair satisfies a second preset optimization condition. The point cloud frames in the first local point cloud frame of the room corresponding to the global point-plane matching pair satisfying the second preset optimization condition are merged, and the point cloud frames in the first local point cloud frame of the room corresponding to the global point-plane matching pair satisfying the second preset optimization condition after merging are determined as a third local point cloud of the room.
4. The method of claim 3, wherein, before the poses of the first point cloud frame and the second point cloud frame are adjusted through the first point-plane matching pair so that the first point-plane matching pair satisfies the first preset optimization condition, the method further comprises: determining whether the total number of the first local point cloud frames exceeds a preset point cloud frame number; if the total number of the first local point cloud frames exceeds the preset point cloud frame number, dividing the first local point cloud frames into point cloud frames of a plurality of subspaces; for any point cloud frame of a subspace, performing the adjustment of the poses of the first point cloud frame and the second point cloud frame through the first point-plane matching pair so that the first point-plane matching pair satisfies the first preset optimization condition and subsequent steps to optimize the point cloud poses of the room to which the first local point cloud frame belongs, and merging the first point cloud frames and the second point cloud frames of the plurality of subspaces after optimization to obtain a third local point cloud of the room. The searching of the plurality of target neighborhood points of the target point cloud through the preset search tree comprises:
5. The method of claim 4, wherein, establishing a preset search tree of the first point cloud frame in the first local point cloud frame for the second point cloud frame in the first local point cloud frame; calculating a normal vector dot product between each point cloud in the first point cloud frame in the first local point cloud frame and each point cloud in the second point cloud frame in the first local point cloud frame by traversing each point cloud in the first point cloud frame in the first local point cloud frame; for any point cloud in the second point cloud frame in the first local point cloud frame, determining whether the normal vector dot product satisfies a preset dot product condition; determining a target searched point cloud in the second point cloud frame in the first local point cloud frame as a point cloud for which the normal vector dot product satisfies the preset dot product condition; searching the target searched point cloud through the preset search tree to determine a target neighborhood point as the target searched point cloud searched and satisfying a search condition of the preset search tree. 6. The method of claim 5, wherein, The pose of the first point cloud frame and the second point cloud frame is adjusted by the first point-plane matching pair to make the first point-plane matching pair satisfy a first preset optimization condition, including: The first point cloud frame and the second point cloud frame are iteratively rotated and / or translated in coordinates until the distance between the point cloud and the point cloud plane in the first point-plane matching pair satisfies the first preset optimization condition.
7. The method of claim 6, wherein, The third local point cloud of the room and the fourth local point cloud of the transition area in the target space are fused by point cloud splicing, and a third point-plane matching pair is established to perform point cloud pose optimization on the room to which the third local point cloud belongs and the transition area to which the fourth local point cloud belongs based on the third point-plane matching pair, and the optimized third local point cloud and fourth local point cloud are merged to obtain second overall point cloud data of the target space, including: The third local point cloud of the room and the fourth local point cloud of the transition area in the target space are fused by point cloud splicing, and the optimized first point-plane matching pair of the third local point cloud and the optimized second point-plane matching pair of the fourth local point cloud are combined to form the third point-plane matching pair; The point cloud frames belonging to each room and the point cloud frames belonging to the transition area between each room in the second overall point cloud data are respectively rotated and / or translated in coordinates until the distance between the point cloud and the point cloud plane in the third point-plane matching pair satisfies a third preset optimization condition, to obtain the second overall point cloud data of the target space.
8. A point cloud pose optimization apparatus, comprising: The device comprises: A local point cloud frame determination module is configured to determine a first local point cloud frame and a second local point cloud frame in a target space according to first overall point cloud data of the target space, wherein the first local point cloud frame is point cloud frame data of any room in the target space, and the second local point cloud frame is point cloud frame data of a transition area between each room in the target space; A room pose optimization module is configured to establish a first point-plane matching pair according to the first local point cloud frame to perform point cloud pose optimization on a room to which the first local point cloud frame belongs based on the first point-plane matching pair, and merge the optimized first local point cloud frame to obtain a third local point cloud of the room; A transition area pose optimization module is configured to establish a second point-plane matching pair according to the second local point cloud frame to perform point cloud pose optimization on a transition area between rooms to which the second local point cloud frame belongs based on the second point-plane matching pair, and merge the optimized second local point cloud frame to obtain a fourth local point cloud of the transition area; A space pose optimization module is configured to fuse the third local point cloud of the room and the fourth local point cloud of the transition area in the target space by point cloud splicing, and establish a third point-plane matching pair to perform point cloud pose optimization on the room to which the third local point cloud belongs and the transition area to which the fourth local point cloud belongs based on the third point-plane matching pair, and merge the optimized third local point cloud and fourth local point cloud to obtain second overall point cloud data of the target space. The first point-plane matching pair, the second point-plane matching pair, and the third point-plane matching pair each represent a matching relationship between a plurality of point clouds and a plane to which the plurality of point clouds belong.
9. An electronic device, comprising: The computer program instructions are executed by the processor to implement the method in any one of claims 1-7. The computer program instructions are executed by the processor to implement the method in any one of claims 1-7. The computer program instructions are executed by the processor to implement the method in any one of claims 1-7.
10. A computer-readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions are executed by the processor to implement the method in any one of claims 1-7.
11. A computer program product comprising computer program instructions, characterised in that,
