Map merging method, device, robot, storage medium and program product

Abstract

The present application relates to a map merging method, device, robot, storage medium, and program product. The method comprises: obtaining multiple local point cloud maps in a target area, wherein each local point cloud map has overlapping areas, and each local point cloud map includes node information for constructing a local point cloud map; determining the relative pose transformation relationship between the starting nodes in each adjacent local point cloud map based on the overlapping areas of each local point cloud map and the node information in each local point cloud map; and performing coordinate alignment transformation on all node information in each local point cloud map based on the relative pose transformation relationship between the starting nodes in each adjacent local point cloud map to obtain a global point cloud map of the target area. This method can reduce the complexity of merging multiple local point cloud maps into a global point cloud map.

Description

Technical Field

[0001] The present application relates to the field of robot positioning and navigation technology, and in particular to a map merging method, device, robot, storage medium and program product. Background Art

[0002] Typically, a robot can be moved in the target area to collect data about the target area, and then the collected data can be analyzed to construct a point cloud map of the target area.

[0003] In related technologies, for a target area with a larger range, the target area will be divided into multiple sub-areas. The robot obtains the point cloud map corresponding to each sub-area by collecting data information of each sub-area, constructs a pose graph through the point cloud map corresponding to each sub-area, and merges the local point cloud maps of each target sub-area into a global point cloud map based on the global positioning system (GPS) installed in the robot.

[0004] However, the method of merging multiple local point cloud maps into a global point cloud map in the related art is relatively complex. Summary of the Invention

[0005] Based on this, it is necessary to provide a map merging method, device, robot, storage medium and program product that can reduce the complexity of the method of merging multiple local point cloud maps into a global point cloud map to address the above technical problems.

[0006] In a first aspect, the present application provides a map merging method, applied to a robot, comprising:

[0007] Acquire multiple local point cloud maps in the target area, where there are overlapping areas between the local point cloud maps, and each local point cloud map includes node information used to construct the local point cloud map;

[0008] Determine the relative position transformation relationship between the starting nodes in each adjacent local point cloud map based on the overlapping area of each local point cloud map and the node information in each local point cloud map;

[0009] Based on the relative pose transformation relationship between the starting nodes in each adjacent local point cloud map, the coordinate alignment transformation is performed on all node information in each local point cloud map to obtain the global point cloud map of the target area.

[0010] In one embodiment, determining the relative pose transformation relationship between the starting nodes in adjacent local point cloud maps based on the overlapping areas of the local point cloud maps and the node information in the local point cloud maps includes:

[0011] Determine all adjacent local point cloud maps in each local point cloud map according to the overlapping areas of each local point cloud map;

[0012] According to the node information in each local point cloud map, the matching node pairs in the adjacent local point cloud maps are obtained;

[0013] Obtaining the node loop constraint relationship between each matching node pair in each adjacent local point cloud map;

[0014] According to the node loop constraint relationship between each matching node pair in each adjacent local point cloud map, the relative pose transformation relationship between the starting nodes in each adjacent local point cloud map is determined.

[0015] In one embodiment, if each adjacent local point cloud map includes a first local point cloud map and a second local point cloud map; and the node information in each of the local point cloud maps includes a point cloud descriptor of the node;

[0016] Then, according to the node information in each local point cloud map, the matching node pairs in the adjacent local point cloud maps are obtained, including:

[0017] For each of the adjacent local point cloud maps, the point cloud descriptors of each reference node in the second local point cloud map are matched with the point cloud descriptors of each loop candidate node in the first local point cloud map.

[0018] The reference nodes and loop candidate nodes that are successfully matched by the point cloud descriptor are determined as matching node pairs, and the matching node pairs in each adjacent local point cloud map are obtained.

[0019] In one embodiment, obtaining a node loop constraint relationship between each matching node pair in each adjacent local point cloud map includes:

[0020] For any matching node pair, obtain the loop candidate local point cloud map corresponding to each loop candidate node in the matching node pair, and the reference local point cloud map corresponding to the reference node matching each loop candidate node;

[0021] Perform point cloud registration on the matching loop candidate local point cloud map and the reference local point cloud map;

[0022] After the point cloud registration is completed, the relative pose between the loop candidate node and the reference node in each matching node pair is determined as the node loop constraint relationship between each matching node pair.

[0023] In one embodiment, obtaining a loop candidate local point cloud map corresponding to each loop candidate node in a matching node pair and a reference local point cloud map corresponding to a reference node matching each loop candidate node includes:

[0024] Acquire a reference frame corresponding to each reference node and a loop candidate frame corresponding to each loop candidate node;

[0025] According to the poses of a preset number of key frames adjacent to and preceding each loop candidate frame, the loop candidate frame is spliced with the preset number of key frames to obtain a loop candidate local point cloud map corresponding to each loop candidate node;

[0026] According to the pose of each loop candidate frame, each reference frame is transformed to obtain a reference local point cloud map corresponding to the reference node matching each loop candidate node.

[0027] In one embodiment, determining the relative pose transformation between the starting nodes in each adjacent local point cloud map based on the node loop constraint relationship between each matching node pair in each adjacent local point cloud map includes:

[0028] Get the root mean square error between all matching node pairs in each adjacent local point cloud map;

[0029] Get the target matching node pair with the minimum root mean square error value;

[0030] The node loop constraint relationship between the reference node and the loop candidate node in the target matching node pair is determined as the relative pose transformation between the starting nodes in each adjacent local point cloud map; wherein, the reference node in the target matching node pair is the starting node in the second local point cloud map, and the loop candidate node in the target matching node pair is the starting node in the first local point cloud map.

[0031] In one embodiment, the method further comprises:

[0032] Obtain the difference between the root mean square error value of each matching node pair and the root mean square error value of the target matching node pair;

[0033] Eliminate matching node pairs whose difference is greater than a preset difference to obtain candidate matching node pairs.

[0034] In one embodiment, based on the relative pose transformation between the starting nodes in each adjacent local point cloud map, coordinate alignment transformation is performed on all node information in each local point cloud map to obtain a global point cloud map of the target area, including:

[0035] According to the relative pose transformation between the starting nodes in each adjacent local point cloud map, all node information in each local point cloud map is converted to the same coordinate system to obtain the initial global point cloud map of the target area;

[0036] The poses of the nodes in the initial global point cloud map are optimized by a preset optimizer to obtain a global point cloud map of the target area.

[0037] In one embodiment, the poses of nodes in the initial global point cloud map are optimized by a preset optimizer to obtain a global point cloud map of the target area, including:

[0038] The node poses and loop constraints in the candidate point cloud map are input into a preset optimizer, and the node poses are optimized by the optimizer to obtain the target node poses; the loop constraints include the node loop constraints between candidate matching node pairs, the loop constraints in each local point cloud map, and the node loop constraints between adjacent nodes in each local point cloud map;

[0039] Based on the target node pose and the point cloud data corresponding to each node, a global point cloud map of the target area is constructed.

[0040] In one embodiment, obtaining a plurality of local point cloud maps in a target area includes:

[0041] According to the initial local point cloud maps of multiple areas in the target area, construct a pose graph corresponding to each initial local point cloud map;

[0042] Through the node pose of each node in the pose graph, the point cloud data of each node is transformed into the global coordinate system of each initial local point cloud map to obtain each local point cloud map.

[0043] In a second aspect, the present application further provides a map merging device, the device comprising:

[0044] The first acquisition module is used to obtain multiple local point cloud maps in the target area, where there are overlapping areas between the local point cloud maps, and each local point cloud map includes node information of the robot during the construction of the local point cloud map;

[0045] A determination module, configured to determine the relative pose transformation between the starting nodes in each adjacent local point cloud map based on the overlapped area of each local point cloud map and the node information in each local point cloud map;

[0046] The conversion module is used to perform coordinate alignment conversion on all node information in each local point cloud map based on the relative pose transformation between the starting nodes in each adjacent local point cloud map to obtain a global point cloud map of the target area.

[0047] In a third aspect, the present application further provides a robot comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the method steps in any one of the embodiments of the first aspect above when executing the computer program.

[0048] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the method steps in any one of the embodiments of the first aspect when the computer program is executed by a processor.

[0049] In a fifth aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the method steps in any one of the embodiments of the first aspect above.

[0050] The above-mentioned map merging method, device, robot, storage medium and program product obtain multiple local point cloud maps in the target area, determine the relative pose transformation between the starting nodes in each adjacent local point cloud map based on the overlapping area of each local point cloud map and the node information in each local point cloud map, and based on the relative pose transformation between the starting nodes in each adjacent local point cloud map, perform coordinate alignment transformation on all node information in each local point cloud map to obtain a global point cloud map of the target area. There are overlapping areas between the local point cloud maps in this method, and each local point cloud map includes the node information of the robot in the process of constructing the local point cloud map. The relative posture between the starting nodes in each adjacent local point cloud map can be directly obtained through multiple local point cloud maps. According to the relative posture of the starting nodes, the local point cloud maps can be converted to the same coordinate system to obtain a global point cloud map. This process only involves the calculation and processing of point cloud map data, while the existing technology involves the calculation and processing of point cloud maps and GPS positioning data. Compared with the existing technology, the data volume and data dimension of this method are smaller, the calculation process is relatively simple, and it can reduce the complexity of merging multiple local point cloud maps into a global point cloud map. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A diagram showing an application environment of a map merging method in one embodiment;

[0052] Figure 2 1 is a flow chart of a map merging method according to an embodiment;

[0053] Figure 3 1 is a flow chart of a map merging method according to an embodiment;

[0054] Figure 4 1 is a flow chart of a map merging method according to an embodiment;

[0055] Figure 5 1 is a flow chart of a map merging method according to an embodiment;

[0056] Figure 6 1 is a flow chart of a map merging method according to an embodiment;

[0057] Figure 7 1 is a flow chart of a map merging method according to an embodiment;

[0058] Figure 8 1 is a flow chart of a map merging method according to an embodiment;

[0059] Figure 9 1 is a flow chart of a map merging method according to an embodiment;

[0060] Figure 10 1 is a flow chart of a map merging method according to an embodiment;

[0061] Figure 11 1 is a flow chart of a map merging method according to an embodiment;

[0062] Figure 12 1 is a flow chart of a map merging method according to an embodiment;

[0063] Figure 13 FIG. 4 is a structural block diagram of a map merging device in one embodiment. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0065] The map merging method provided in the embodiment of the present application can be applied to a robot, wherein the internal structure of the robot is as follows: Figure 1 As shown. The memory within the robot's internal structure includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database; the internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database is used to store multiple local point cloud map data and global point cloud map data. The network interface is used to communicate with an external terminal via a network connection. When executed by a processor, the computer program implements the map merging method provided in this application.

[0066] The following will explain in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems through embodiments and in combination with the accompanying drawings. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. It should be noted that the map merging method provided by the present application can be executed by a robot or a map merging device, which can be implemented as part or all of the robot through software, hardware, or a combination of software and hardware. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments.

[0067] In one embodiment, Figure 2 As shown, a map merging method is provided, which is applied to Figure 1 The robot in the example is used to illustrate the following steps:

[0068] S201, obtaining multiple local point cloud maps in the target area, where there are overlapping areas between the local point cloud maps, and each local point cloud map includes node information of the robot in the process of constructing the local point cloud map.

[0069] The target area represents the area corresponding to the point cloud map to be constructed. If the target area is large, it can be divided into multiple target sub-areas. The point cloud map constructed by the robot in each target sub-area serves as a local point cloud map, with each target sub-area corresponding to a local point cloud map. The ranges of the multiple target sub-areas can be the same size, and the ranges of the corresponding local point cloud maps can be the same size, or the ranges of the multiple target sub-areas can be different. The surface shape of the target area can be arbitrary, and the local point cloud map corresponding to the target area can also be arbitrary. For example, the surface shape of the target area and the local point cloud map corresponding to the target area can be triangular, rectangular, or circular. During the target area division process, the target area can be divided equally or according to actual needs. The robot's node information includes information such as node poses, loop constraints between nodes, and node point cloud descriptors. During the process of constructing a local point cloud map corresponding to a target sub-area by the robot, the robot moves within the target sub-area, and the robot's simultaneous localization and mapping (SLAM) system generates a local point cloud map for the target sub-area.

[0070] Optionally, the robot can divide the target area into multiple sub-areas, and by moving in the multiple target sub-areas, construct multiple local point cloud maps corresponding to the multiple target sub-areas in real time, and obtain multiple local point cloud maps in the target area. Optionally, the robot can also match the identification information of the target area with any identification information in the point cloud map library. If the match is successful, the robot obtains multiple pre-constructed local point cloud maps from the point cloud map library according to the identification information, and obtains multiple local point cloud maps in the target area. The point cloud map library includes multiple pre-constructed local point cloud maps and the correspondence between the multiple local point cloud maps and the identification information. The identification information includes keyword information and storage time information of the multiple point cloud maps of the target area.

[0071] Furthermore, it can be understood that since there are multiple adjacent target sub-areas in the target area, and the adjacent target sub-areas share the same edge, when the robot is in the process of constructing local point cloud maps corresponding to adjacent target sub-areas, when the robot's position is within a preset distance range of the common edge, there is an overlapping area between the two local point cloud maps constructed by the SLAM system in the robot.

[0072] S202 : Determine the relative position transformation between the starting nodes in each adjacent local point cloud map according to the overlapping area of each local point cloud map and the node information in each local point cloud map.

[0073] The starting node refers to the robot's starting position in each target sub-region when constructing a point cloud map for each target sub-region. The aforementioned pose includes position and attitude. Position refers to the robot's 3D coordinate position in the current world map, and attitude refers to the robot's 3D rotation angle. The relative pose transformation refers to the conversion between the position and attitude of two nodes. The relative pose transformation of the starting node refers to the conversion between the position and attitude of the starting positions of two adjacent local point cloud maps.

[0074] Specifically, the robot can determine the starting node and the node pose of each local point cloud map based on the node information of each local point cloud map, and then determine multiple adjacent local point cloud maps based on the overlapping area of each local point cloud map. For each pair of adjacent local point cloud maps, the node pose of the starting nodes in the two adjacent local point cloud maps is converted to the same world coordinate system to obtain the relative pose transformation of the node pose of the starting nodes in the two adjacent local point cloud maps in the same world coordinate system.

[0075] S203 , based on the relative pose transformation between the starting nodes in each adjacent local point cloud map, coordinate alignment transformation is performed on all node information in each local point cloud map to obtain a global point cloud map of the target area.

[0076] Specifically, the robot can perform coordinate alignment conversion on all nodes in adjacent local point cloud maps based on the relative pose transformation between the starting nodes in adjacent local point cloud maps. Adjacent local point cloud maps are converted to the same coordinate system, and two adjacent local point cloud maps are fused into a single point cloud map. Following the above method, the fused point cloud map is fused with its adjacent local point cloud maps. Similarly, all local point cloud maps are fused into a complete point cloud map, and the pose graph of the complete point cloud map is optimized to obtain a global point cloud map of the target area.

[0077] It can be understood that this method can not only complete the merging of point cloud maps, but also merge multiple 2D grid maps, merging multiple grid maps into a global grid map.

[0078] In the above-mentioned map merging method, the method obtains multiple local point cloud maps in the target area, determines the relative pose transformation between the starting nodes in each adjacent local point cloud map based on the overlapping area of each local point cloud map and the node information in each local point cloud map, and performs coordinate alignment transformation on all node information in each local point cloud map based on the relative pose transformation between the starting nodes in each adjacent local point cloud map to obtain a global point cloud map of the target area. In this method, there are overlapping areas between the local point cloud maps, and each local point cloud map includes the node information of the robot during the construction of the local point cloud map. The relative pose between the starting nodes in each adjacent local point cloud map can be directly obtained from the multiple local point cloud maps. Based on the relative pose of the starting nodes, the local point cloud maps can be converted to the same coordinate system to obtain a global point cloud map. This process only involves the calculation and processing of point cloud map data, while the existing technology involves the calculation and processing of point cloud maps and GPS positioning data. Compared with the existing technology, the data volume and data dimension of this method are smaller, the calculation process is simpler, and it can reduce the complexity of merging multiple local point cloud maps into a global point cloud map.

[0079] Figure 3 A flowchart of a map merging method provided in an embodiment of the present application. The embodiment of the present application relates to an optional implementation method for determining the relative pose transformation between the starting nodes in each adjacent local point cloud map based on the overlapping area of each local point cloud map and the node information in each local point cloud map. Figure 2 Based on the embodiment shown, Figure 3 As shown, the above method may include the following steps:

[0080] S301 : Determine all adjacent local point cloud maps in each local point cloud map according to the overlapping areas of each local point cloud map.

[0081] Specifically, the robot can number the overlapping areas of each local point cloud map, determine the two local point cloud maps corresponding to each number, and obtain all adjacent local point cloud maps in the multiple local point cloud maps. Among them, each local point cloud map may have one or more adjacent local point cloud maps. For example, if the length of the target area is 8 meters and the width is 2 meters, and the target area needs to be divided into 4 target sub-areas, the target area can be divided into 4 target sub-areas with a length of 2 meters and a width of 2 meters, or the target area can be divided into 4 target sub-areas with a length of 4 meters and a width of 1 meter. When the length of the target sub-area is 2 meters and the width is 2 meters, the two outermost target sub-areas of the four target sub-areas have an adjacent local point cloud map, and the two middle target sub-areas have two adjacent local point cloud maps; when the length of the target sub-area is 4 meters and the width is 1 meter, each target sub-area has two adjacent local point cloud maps.

[0082] S302: Acquire multiple matching node pairs in adjacent local point cloud maps based on the node information in each local point cloud map.

[0083] The matching node pair refers to two nodes in two local point cloud maps whose node information matching degree is greater than a preset threshold.

[0084] Specifically, there are multiple nodes in each local point cloud map, each node includes node information, and there is a one-to-one mapping relationship between nodes and node information. The robot can divide the adjacent local point cloud map into a first local point cloud map and a second local point cloud map, and match the node information of the node in the first local point cloud map with the node information of all nodes in the second local point cloud map in a preset order. If the node information of the node in the first local point cloud map successfully matches the node information of any node in the second local point cloud map, the two nodes in the adjacent local point cloud map are determined to be a matching node pair; if the similarity between the two nodes in the adjacent local point cloud map is greater than a preset threshold, the node information of the node in the first local point cloud map is matched with the node information of the next node in the second local point cloud map, until the node information of the node in the first local point cloud map is matched with the node information of all nodes in the second local point cloud map, thereby obtaining multiple matching node pairs in the adjacent local point cloud maps. Among them, there may be multiple matching node pairs in every two adjacent local point cloud maps.

[0085] S303: Obtain node loop constraint relationships between each matching node pair in each adjacent local point cloud map.

[0086] The loop constraint relationship of the node refers to the difference in posture between the starting node and the ending node in the multi-frame key frame.

[0087] Specifically, the robot can perform point cloud registration on adjacent local point cloud maps. Based on the relative pose transformation between matching node pairs, the adjacent local point cloud maps are converted to the same coordinate system to form a single point cloud map. For any matching node pair in the adjacent local point cloud maps, any matching node pair can be used as the starting and ending nodes of the adjacent local point cloud map. The relative pose transformation of each matching node pair in the adjacent local point cloud map is determined as a node loop constraint relationship.

[0088] S304: Determine the relative pose transformation between the starting nodes in each adjacent local point cloud map according to the node loop constraint relationship between each matching node pair in each adjacent local point cloud map.

[0089] Optionally, the robot can determine the node loop constraint relationship between any one of the matching node pairs in the adjacent local point cloud maps as the relative pose transformation between the starting nodes in the adjacent local point cloud maps. Optionally, the robot can sort the matching node pairs in the adjacent local point cloud maps according to the matching values between the matching node pairs, and determine the node loop constraint relationship between the matching node pairs with the highest matching values in the adjacent local point cloud maps as the relative pose transformation between the starting nodes in the adjacent local point cloud maps. This embodiment does not limit the method for determining the relative pose transformation between the starting nodes in each adjacent local point cloud map based on the node loop constraint relationship between each matching node pair in each adjacent local point cloud map.

[0090] In the above-mentioned map merging method, the method determines all adjacent local point cloud maps in each local point cloud map based on the overlapping area of each local point cloud map, obtains multiple matching node pairs in the adjacent local point cloud maps based on the node information in each local point cloud map, obtains the node loop constraint relationship between each matching node pair in each adjacent local point cloud map, and determines the relative pose transformation between the starting nodes in each adjacent local point cloud map based on the node loop constraint relationship between each matching node pair in each adjacent local point cloud map. This method can accurately determine all adjacent local point cloud maps in each local point cloud map based on the overlapping area of each local point cloud map, and for each pair of adjacent local point cloud maps, it can accurately determine multiple matching node pairs in the adjacent point cloud maps based on the node information in the adjacent local point cloud maps, and then accurately determine the relative pose transformation between the starting nodes in each adjacent local point cloud map based on the loop constraint relationship between each matching node pair, thereby improving the accuracy of obtaining the relative pose transformation between the starting nodes.

[0091] Figure 4A flowchart of a map merging method provided in an embodiment of the present application. The embodiment of the present application relates to an optional implementation method for obtaining multiple matching node pairs in adjacent local point cloud maps based on the node information in each local point cloud map, if each adjacent local point cloud map includes a first local point cloud map and a second local point cloud map; and the node information includes the point cloud descriptor of the node. Figure 3 Based on the embodiment shown, Figure 4 As shown, the above method may include the following steps:

[0092] S401 , for each adjacent local point cloud map, matching the point cloud descriptors of each reference node in the second local point cloud map with the point cloud descriptors of each loop candidate node in the first local point cloud map.

[0093] Specifically, the robot can extract point cloud descriptors to perform recognition, segmentation, resampling, or surface reconstruction on local point cloud maps. Each adjacent local point cloud map can be divided into a first local point cloud map and a second local point cloud map. Each node in the first local point cloud map is determined as a loop candidate node, and each node in the second local point cloud map is determined as a reference node. The robot can calculate the similarity between the point cloud descriptor of each reference node and the point cloud descriptor of each loop candidate node. If the similarity is greater than a preset threshold, the match is successful; if the similarity is less than the preset threshold, the match is unsuccessful. The node information can be a point cloud descriptor or other more robust descriptor.

[0094] S402: Determine the reference node and the loop candidate node that are successfully matched by the point cloud descriptor as a matching node pair, and obtain multiple matching node pairs in each adjacent local point cloud map.

[0095] Specifically, the robot can match the point cloud descriptor of the reference node with the point cloud descriptor of any one of the loop candidate nodes. If the point cloud descriptor of the reference node is successfully matched with the point cloud descriptor of any one of the loop candidate nodes, the reference node and the loop candidate node are determined as a matching node pair; if the point cloud descriptor of the reference node is not successfully matched with the loop candidate node, the reference node is matched with the next node in the loop candidate nodes until the point cloud descriptors of each reference node in the second local point cloud map are respectively matched with each loop candidate node in the first local point cloud map, and multiple matching node pairs in adjacent local point cloud maps are obtained.

[0096] It can be understood that the successfully matched node pairs mean that the robots have a certain probability of reaching the same area, resulting in the point cloud descriptors of the matched node pairs being relatively similar, that is, there may be loops between the matched node pairs in the two maps.

[0097] In the map merging method described above, for each adjacent local point cloud map, the point cloud descriptors of each reference node in the second local point cloud map are matched with the point cloud descriptors of each loop candidate node in the first local point cloud map. Reference nodes and loop candidate nodes whose point cloud descriptors successfully match are identified as matching node pairs, resulting in multiple matching node pairs in each adjacent local point cloud map. This method matches the point cloud descriptors of nodes in two adjacent local point cloud maps, which contain multiple pieces of information. This allows for comprehensive matching of multiple pieces of information between the two nodes, resulting in more accurate matching node pairs.

[0098] Figure 5 A flowchart of a map merging method provided in an embodiment of the present application. The embodiment of the present application relates to an optional implementation method for obtaining node loop constraint relationships between each matching node pair in each adjacent local point cloud map. Figure 4 Based on the embodiment shown, Figure 5 As shown, the above method may include the following steps:

[0099] S501 : For any matching node pair, obtain a loop candidate local point cloud map corresponding to each loop candidate node in the matching node pair, and a reference local point cloud map corresponding to a reference node matching each loop candidate node.

[0100] Specifically, there may be multiple matching node pairs in two adjacent local point cloud maps. For each matching node pair, the matching node pair includes a loop candidate node and a reference node. Optionally, the robot can obtain the loop candidate local point cloud map corresponding to the loop candidate node from the point cloud database based on the identification information of the loop candidate node; and obtain the reference point cloud map corresponding to the reference node from the point cloud database based on the identification information of the reference node. Optionally, the robot can splice the point cloud maps corresponding to multiple loop candidate frames in the loop candidate node to obtain the loop candidate local point cloud map corresponding to the loop candidate node; and splice the reference point cloud maps corresponding to multiple reference frames in the reference node to obtain the reference local point cloud map corresponding to the reference node.

[0101] Optional, Figure 6 A flowchart of a map merging method provided in an embodiment of the present application. The embodiment of the present application relates to an optional implementation method for obtaining a loop candidate local point cloud map corresponding to each loop candidate node in a matching node pair, and a reference local point cloud map corresponding to a reference node matching each loop candidate node. Figure 5 Based on the embodiment shown, Figure 6 As shown, the above method may include the following steps:

[0102] S601: Obtain a reference frame corresponding to each reference node and a loop candidate frame corresponding to each loop candidate node.

[0103] Specifically, the reference frame corresponding to a reference node refers to the point cloud image acquired by the robot at the reference node, and the loop candidate frame corresponding to a loop candidate node refers to the point cloud image acquired by the robot at the loop candidate node. The robot can obtain the reference frame corresponding to each reference node from the database based on the identification information of the reference node, and the robot can obtain the loop candidate frame corresponding to each loop candidate node from the database based on the identification information of the loop candidate node.

[0104] S602: performing a splicing operation on the loop candidate frames and the key frames according to the poses of a preset number of key frames adjacent to and preceding each loop candidate frame, to obtain a loop candidate local point cloud map corresponding to each loop candidate node.

[0105] Specifically, the loop candidate point cloud map is composed of point cloud maps corresponding to multiple loop candidate frames. For each loop candidate frame, the robot can determine the key frames adjacent to the loop candidate frame according to the generation time of the loop candidate frame, and splice the point cloud maps corresponding to each key frame and the loop candidate frame according to the pose of each key frame and the loop candidate frame, and combine the point cloud maps corresponding to multiple frames into a large point cloud map, and determine the point cloud map as the loop candidate local point cloud map. Each loop candidate frame is processed by the above method to obtain the loop candidate local point cloud map corresponding to each loop candidate node. For example, the 25 frames before and after the loop candidate frame can be determined as key frames to obtain 50 key frames. The point cloud maps corresponding to the 50 key frames and the loop candidate frame are spliced to obtain the loop candidate local point cloud map, and the loop candidate local point cloud map is recorded as pointcloud1.

[0106] S603: Convert each reference frame according to the pose of each loop candidate frame to obtain a reference local point cloud map corresponding to a reference node matching each loop candidate node.

[0107] Specifically, the reference local point cloud map is composed of point cloud maps corresponding to multiple reference frames. For each reference frame, the robot can convert the reference frame to the coordinate system corresponding to the loop candidate frame based on the pose of the loop candidate frame corresponding to the reference frame, and perform a splicing operation on the reference frame and the pose of a preset number of key frames adjacent to the reference frame before and after the reference frame, and combine the point cloud maps corresponding to multiple frames into a large point cloud map, and determine the point cloud map as the reference local point cloud map. For example, the 25 frames before and after the reference frame can be determined as key frames to obtain 50 key frames. The 50 key frames and the point cloud maps corresponding to the reference frame are spliced to obtain a reference local point cloud map, and the reference local point cloud map is recorded as pointcloud2.

[0108] In the above-mentioned map merging method, the method obtains the reference frame corresponding to each reference node and the loop candidate frame corresponding to each loop candidate node, and splices the loop candidate frame and the key frame according to the pose of a preset number of key frames adjacent to each loop candidate frame to obtain a loop candidate local point cloud map corresponding to each loop candidate node, and converts each reference frame according to the pose of each loop candidate frame to obtain a reference local point cloud map corresponding to the reference node that matches each loop candidate node. This method splices the reference frame and the preset number of key frames adjacent to the loop candidate frame to accurately obtain the loop candidate local point cloud map; converts the reference frame to the loop candidate frame to accurately obtain the reference local point cloud map; at the same time, the reference node and the loop candidate node are placed in the same world coordinate system, which makes it more convenient to compare the loop candidate local point cloud map and the reference point cloud map, so that the result is more accurate.

[0109] S502: performing point cloud registration on the matching loop candidate local point cloud map and the reference local point cloud map.

[0110] Among them, point cloud registration refers to the transformation of two point sets so that the two point sets are in the same coordinates to form a new point set.

[0111] Specifically, the robot can convert the loop candidate local point cloud map and the reference local point cloud map through a related point cloud registration algorithm so that the loop candidate local point cloud map and the reference local point cloud map are in the same coordinate system to form a local point cloud map. For example, related point cloud registration algorithms include the Iterative Closest Point (ICP) algorithm, the Kernel Correlation (KC) algorithm, and the Robust Point Matching (RPM) algorithm. For example, the loop candidate local point cloud map pointcloud1 and the reference local point cloud map pointcloud2 are point cloud registered, and pointcloud1 and pointcloud2 are fused into a point cloud map.

[0112] S503: After the point cloud registration is completed, the relative pose between the loop candidate node and the reference node in each matching node pair is determined as the node loop constraint relationship between each matching node pair.

[0113] Specifically, after the point cloud registration of the loop candidate local point cloud map and the reference local point cloud map is completed through step S502, the loop candidate local point cloud map and the reference local point cloud map form a point cloud map, the loop candidate node is determined as the first node in the point cloud map, and the reference node is determined as the last node in the point cloud map, that is, the node loop constraint relationship in the point cloud map is the relative posture between the loop candidate node and the reference node.

[0114] It can be understood that a successful point cloud registration is considered to be a high similarity between the point cloud data of the reference node and the loop candidate node, that is, there is a loop between the matching node pairs in the two maps.

[0115] In the above-mentioned map merging method, for any matching node pair, a loop candidate local point cloud map corresponding to each loop candidate node in the matching node pair and a reference local point cloud map corresponding to the reference node matching each loop candidate node are obtained. Point cloud registration is performed on the matching loop candidate local point cloud map and the reference local point cloud map. After the point cloud registration is completed, the relative pose between the loop candidate node and the reference node in each matching node pair is determined as the node loop constraint relationship between each matching node pair. This method performs point cloud registration on the local point cloud map and the reference point cloud map. On the basis of determining that the robot has a certain probability of reaching the same area, it further determines that the robot reaches the same area in adjacent local point cloud maps. This makes the process of determining the node loop constraint relationship more accurate, and the resulting node loop constraint relationship is also more accurate.

[0116] Figure 7A flowchart of a map merging method provided in an embodiment of the present application. The embodiment of the present application relates to an optional implementation method for determining the relative pose transformation between the starting nodes in each adjacent local point cloud map based on the node loop constraint relationship between each matching node pair in each adjacent local point cloud map. Figure 3 Based on the embodiment shown, Figure 7 As shown, the above method may include the following steps:

[0117] S701, obtaining root mean square error values between all matching node pairs in adjacent local point cloud maps.

[0118] Specifically, the root mean square error (RMS) is the square root of the ratio of the square of the deviation between the predicted value and the true value to the number of observations. The robot can calculate the RMS error between matching node pairs in each adjacent local point cloud map using the node information and relevant calculation formulas. This method can be used to calculate the RMS error between all matching node pairs in each adjacent local point cloud map.

[0119] S702: Obtain a target matching node pair with the minimum root mean square error value.

[0120] Specifically, the robot can sort the matching node pairs according to the size of the root mean square error values between each pair, with the ones with larger root mean square error values placed in front and the ones with smaller root mean square error values placed in the back, and determine the matching node pair corresponding to the last root mean square error value in the sorted root mean square error values as the target matching node pair.

[0121] S703, determining the node loop constraint relationship between the reference node and the loop candidate node in the target matching node pair as the relative pose transformation between the starting nodes in each adjacent local point cloud map; wherein, the reference node in the target matching node pair is the starting node in the second local point cloud map, and the loop candidate node in the target matching node pair is the starting node in the first local point cloud map.

[0122] Specifically, after point cloud registration is performed on the first local point cloud map and the second local point cloud map, the first local point cloud map and the second local point cloud map are in the same coordinate system, and the first local point cloud map and the second local point cloud map are merged into the same point cloud map. For this point cloud map, the reference node can be used as the starting node of the point cloud map, and the loop candidate node can be used as the ending node of the point cloud map. The node loop constraint relationship between the reference node and the loop candidate node is the relative pose transformation between the starting nodes in adjacent local point cloud maps. The relative pose transformation between the starting nodes in each adjacent local point cloud map is determined by the above method.

[0123] In the above-mentioned map merging method, the method obtains the root mean square error value between all matching node pairs in each adjacent local point cloud map, obtains the target matching node pair with the smallest root mean square error value, and determines the node loop constraint relationship between the reference node and the loop candidate node in the target matching node pair as the relative pose transformation between the starting nodes in each adjacent local point cloud map. The reference node in the target matching node pair in this method is the starting node in the second local point cloud map, and the loop candidate node in the target matching node pair is the starting node in the first local point cloud map. After the two local point cloud maps are point cloud aligned, the first local point cloud map and the second local point cloud map are fused into one point cloud map. The reference node and the loop candidate node are used as the starting node and the ending node of the point cloud map respectively, so that a more accurate relative pose transformation between the starting nodes in the adjacent local point cloud maps can be obtained.

[0124] Figure 8 A flowchart of a map merging method provided in an embodiment of the present application. The embodiment of the present application relates to an optional implementation method for obtaining candidate matching node pairs. Figure 7 Based on the embodiment shown, Figure 8 As shown, the above method may include the following steps:

[0125] S801 , obtaining the difference between the root mean square error value of each matching node pair and the root mean square error value of the target matching node pair.

[0126] Specifically, after the root mean square error value of each matching node pair is obtained in step S701, the difference between the root mean square error value of each matching node pair and the root mean square error value of the target matching node pair is calculated to obtain multiple difference values.

[0127] S802 , eliminating matching node pairs whose difference is greater than a preset difference, and obtaining candidate matching node pairs.

[0128] Specifically, when the difference between the root mean square error value of the matching node pair and the root mean square error value of the target matching node pair is greater than the preset difference, it means that the relative posture transformation between the matching node pairs cannot be used as a constraint condition, and the matching node pairs with differences greater than the preset difference are filtered out, and the remaining matching node pairs are determined as candidate matching node pairs.

[0129] In the aforementioned map merging method, candidate matching node pairs are obtained by obtaining the difference between the root mean square error (RMSE) value of each matching node pair and the target matching node pair, eliminating matching node pairs with a difference greater than a preset value. This method eliminates matching node pairs that do not meet the RMS error threshold, preventing these matching node pairs from affecting the merging process of multiple local point cloud maps and improving the efficiency of local point cloud map merging.

[0130] Figure 9 A flowchart of a map merging method provided in an embodiment of the present application. The embodiment of the present application relates to an optional implementation method of performing coordinate alignment conversion on all node information in each local point cloud map based on the relative pose transformation between the starting nodes in each adjacent local point cloud map to obtain a global point cloud map of the target area. Figure 8 Based on the embodiment shown, Figure 9 As shown, the above method may include the following steps:

[0131] S901: According to the relative pose transformation between the starting nodes in each adjacent local point cloud map, all node information in each local point cloud map is converted into the same coordinate system to obtain an initial global point cloud map of the target area.

[0132] Specifically, for each adjacent local point cloud map, the robot can transform the relative posture between the starting nodes in the adjacent local point cloud maps, take one local point cloud map as the basis, and convert all node information in the other local point cloud map to the global coordinate system of the other local point cloud map, so that the adjacent local point cloud maps are in the same coordinate system, and determine the point cloud map obtained after conversion as the initial global point cloud map of the target area.

[0133] S902: Optimize the positions and poses of the nodes in the initial global point cloud map using a preset optimizer to obtain a global point cloud map of the target area.

[0134] Among them, the optimizer is a tool that guides the various parameters of the objective function to update in the correct direction and to the appropriate size during the back-propagation process of deep learning, so that the updated parameters make the value of the objective function continuously approach the global minimum.

[0135] Specifically, the robot can use a preset optimizer to optimize the posture corresponding to each node in the initial global point cloud map. In the process of optimizing the posture corresponding to each node in the initial global point cloud map, the object of optimization is the posture corresponding to each node, and the error of the posture corresponding to each node is optimized to a preset range. The posture corresponding to the optimized node is output, and the global point cloud map is constructed according to the optimized node posture and the point cloud data corresponding to each node.

[0136] In the above-mentioned map merging method, the method converts all node information in each local point cloud map into the same coordinate system based on the relative pose transformation between the starting nodes in each adjacent local point cloud map, obtains the initial global point cloud map of the target area, and optimizes the pose of the nodes in the initial global point cloud map through a preset optimizer to obtain a global point cloud map of the target area. This method converts all node information in each local point cloud map into the same coordinate system, and quickly combines each local point cloud map into an initial global map of the target area. Since there may be errors between the poses of each node in the initial global map, it is necessary to optimize the pose of the nodes in the initial global point cloud map through a preset optimizer, so that the accuracy of the obtained global point cloud map of the target area is higher.

[0137] Figure 10 A flowchart of a map merging method provided in an embodiment of the present application. The embodiment of the present application relates to an optional implementation method of optimizing the poses of nodes in an initial global point cloud map by a preset optimizer to obtain a global point cloud map of a target area. Figure 9 Based on the embodiment shown, Figure 10 As shown, the above method may include the following steps:

[0138] S1001, input the node poses and loop constraint relationships in the candidate point cloud map into a preset optimizer, and optimize the node poses through the optimizer to obtain the target node poses; the loop constraints include the node loop constraint relationships between candidate matching node pairs, the loop constraint relationships in each local point cloud map, and the node loop constraint relationships between adjacent nodes in each local point cloud map.

[0139] Among them, taking two adjacent local point cloud maps as an example, the two adjacent local point cloud maps include a first local point cloud map and a second local point cloud map, and the loop constraints between the two adjacent local point cloud maps include: inter-frame constraints between adjacent nodes within the first local point cloud map, inter-frame constraints between adjacent nodes within the second local point cloud map, loop constraints within the first local point cloud map, loop constraints within the second local point cloud map, and loop constraints between the first local point cloud map and the second local point cloud map.

[0140] Specifically, the robot can take the node poses and loop constraint relationships in the candidate point cloud map as input to a preset optimizer, optimize the node poses in the candidate point cloud map through the preset optimizer, so that the node poses in the candidate point cloud map satisfy multiple loop constraint conditions, output the optimized node poses through the optimizer, and determine the optimized node poses as the target node poses.

[0141] S1002: Construct a global point cloud map of the target area based on the target node pose and the point cloud data corresponding to each node.

[0142] Optionally, the robot can determine the three-dimensional position of each target node in the candidate global point cloud map through a triangulation method, and construct a global point cloud map of the target area based on the target node pose, the point cloud data corresponding to each node, the three-dimensional positions of multiple target nodes, and loop constraints. Optionally, the robot can determine the reprojection constraint relationship of the target nodes in the candidate global point cloud map through a reprojection method, and construct a global point cloud map of the target area based on the target node pose, the point cloud data corresponding to each node, the constraint relationship, and the reprojection constraint relationship.

[0143] In the above-mentioned map merging method, the method inputs the node poses and loop constraints in the candidate point cloud map into a preset optimizer, optimizes the node poses through the optimizer, obtains the target node poses, and constructs a global point cloud map of the target area based on the target node poses and the point cloud data corresponding to each node. The loop constraints in this method include the node loop constraints between candidate matching node pairs, the loop constraints in each local point cloud map, and the node loop constraints of adjacent nodes in each local point cloud map. Since the optimizer incorporates multiple strong constraints and optimizes the candidate point cloud map based on the optimized poses corresponding to the optimized nodes, it has a good optimization effect, making the global point cloud map of the target area more accurate.

[0144] Figure 11 A flowchart of a map merging method provided in an embodiment of the present application. The embodiment of the present application relates to an optional implementation method for obtaining multiple local point cloud maps in a target area. Figure 2 Based on the embodiment shown, Figure 11 As shown, the above method may include the following steps:

[0145] S1101, constructing a pose graph corresponding to each initial local point cloud map based on initial local point cloud maps of multiple areas in the target area.

[0146] Specifically, the initial local point cloud map is generated by the robot's SLAM system. Based on the initial local point cloud map, the robot's driving path can be obtained, and the pose graph corresponding to the initial local point cloud map is constructed based on the driving path. The pose graphs corresponding to multiple initial local point cloud maps are constructed using the above method.

[0147] S1102 , transforming the point cloud data of each node into the global coordinate system of each initial local point cloud map through the node pose of each node in the pose graph, to obtain each local point cloud map.

[0148] Specifically, the point cloud data of each node in the pose graph is Euclidean transformed using the node pose of each node. The point cloud data of each node is then transformed to the global coordinate system of each initial local point cloud map, resulting in multiple local point cloud maps. Simultaneously, each local point cloud map and the node information of each pose graph node are stored to disk. For example, the first initial local point cloud map is transformed to the global coordinate system w0, and the second initial local point cloud map is transformed to the global coordinate system w1.

[0149] In the above-mentioned map merging method, the method constructs a pose graph corresponding to each initial local point cloud map based on the initial local point cloud maps of multiple areas in the target area. Using the node pose of each node in the pose graph, the point cloud data of each node is transformed to the global coordinate system of each initial local point cloud map to obtain each local point cloud map. This method can construct a pose graph based on each local point cloud map and use the pose of each node to transform the point cloud data of each node to the global coordinate system, facilitating the splicing of each local point cloud map and improving the efficiency of local point cloud map merging.

[0150] In one embodiment, in order to facilitate understanding by those skilled in the art, the map merging method is described in detail below. Figure 12 As shown, the method may include:

[0151] S1201, constructing a pose graph corresponding to each initial local point cloud map based on initial local point cloud maps of multiple areas in the target area;

[0152] S1202, transforming the point cloud data of each node into the global coordinate system of each initial local point cloud map according to the node pose of each node in the pose graph, to obtain each local point cloud map;

[0153] S1203, determining all adjacent local point cloud maps in each local point cloud map based on the overlapping areas of each local point cloud map;

[0154] S1204: For each adjacent local point cloud map, match the point cloud descriptor of each reference node in the second local point cloud map with the point cloud descriptor of each loop candidate node in the first local point cloud map;

[0155] S1205: Determine the reference node and the loop candidate node that are successfully matched by the point cloud descriptor as a matching node pair, and obtain multiple matching node pairs in each adjacent local point cloud map;

[0156] S1206, obtaining a reference frame corresponding to each reference node and a loop candidate frame corresponding to each loop candidate node;

[0157] S1207: performing a splicing operation on the loop candidate frames and the key frames according to the poses of a preset number of key frames before and after each loop candidate frame to obtain a loop candidate local point cloud map corresponding to each loop candidate node;

[0158] S1208, transforming each reference frame according to the pose of each loop closure candidate frame to obtain a reference local point cloud map corresponding to the reference node matching each loop closure candidate node;

[0159] S1209, performing point cloud registration on the matching loop candidate local point cloud map and the reference local point cloud map;

[0160] S1210, after the point cloud registration is completed, the relative pose between the loop candidate node and the reference node in each matching node pair is determined as the node loop constraint relationship between each matching node pair;

[0161] S1211, obtaining the root mean square error values between all matching node pairs in each adjacent local point cloud map;

[0162] S1212, obtaining the target matching node pair with the minimum root mean square error value;

[0163] S1213, determining the node loop constraint relationship between the reference node and the loop candidate node in the target matching node pair as the relative pose transformation between the starting nodes in each adjacent local point cloud map;

[0164] S1214, obtaining the difference between the root mean square error value of each matching node pair and the root mean square error value of the target matching node pair;

[0165] S1215, eliminating matching node pairs whose difference is greater than a preset difference to obtain candidate matching node pairs;

[0166] S1216, based on the relative pose transformation between the starting nodes in each adjacent local point cloud map, convert all node information in each local point cloud map into the same coordinate system to obtain an initial global point cloud map of the target area;

[0167] S1217: Input the node poses and loop constraint relationships in the candidate point cloud map into a preset optimizer, and optimize the node poses through the optimizer to obtain the target node poses;

[0168] S1218: Construct a global point cloud map of the target area based on the target node pose and the point cloud data corresponding to each node.

[0169] It should be noted that for the descriptions in S1201-S1218 above, reference can be made to the relevant descriptions in the above embodiments, and the effects are similar, so this embodiment will not be repeated here.

[0170] For example, take two pre-built point cloud maps as an example, and record the two point cloud maps as map0 and map1 respectively, and there is a certain overlap area in map0 and map1. The process of merging map0 and map1 includes the following steps: (1) Use Euclidean transformation to transform the poses of the nodes in map0 and map1 to the global coordinate system of their respective point cloud maps, and transform map0 and map1 to the w0 and w1 global coordinate systems respectively; (2) Match the point cloud descriptors of all nodes in map0 and map1. If the match is successful, the node with successful matching in map0 is determined as the current node, and the node with successful matching in map1 is determined as the loop candidate node; (3) Obtain the point cloud of the 25 key frames before and after the loop candidate frame corresponding to the loop candidate node in map1, and splice the 50 key frames and the loop candidate frame into a local point cloud map, recorded as pointcloud1; According to the poses of the current frame and loop candidate frame corresponding to the current node, transform the current frame to the w0 coordinate system, recorded as pointcloud2, and compare pointcloud1 with p ointcloud2 performs point cloud registration. If the registration is successful, the relative pose between the current frame and the loop candidate frame is determined as the loop constraint; (4) The loop constraint with the smallest root mean square error is determined as the relative pose transformation between the start points of map0 and map1, and the loop constraints that do not meet the preset conditions are eliminated in the loop constraint; (5) According to the relative pose transformation between the start points of map0 and map1, the node poses of all nodes in map1 are converted to the w0 coordinate system to obtain an initial global point cloud map; (6) The node poses and all constraints of all nodes in the initial global point cloud map are input into the optimizer, and the node poses of all nodes are optimized by the optimizer so that the node poses of all nodes meet all constraints to obtain the optimized node poses. The point cloud data of the nodes are spliced according to the node poses of all nodes to obtain the global point cloud data after the merger of map0 and map1.

[0171] There are overlapping areas between the local point cloud maps in the map merging method provided in this embodiment, and each local point cloud map includes the node information of the robot in the process of constructing the local point cloud map. The relative posture between the starting nodes in each adjacent local point cloud map can be directly obtained through multiple local point cloud maps. According to the relative posture of the starting nodes, the local point cloud maps can be converted to the same coordinate system to obtain a global point cloud map. This process only involves the calculation and processing of point cloud map data, while the existing technology involves the calculation and processing of point cloud maps and GPS positioning data. Compared with the existing technology, the data volume and data dimension of this method are smaller, the calculation process is relatively simple, and it can reduce the complexity of merging multiple local point cloud maps into a global point cloud map.

[0172] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0173] Based on the same inventive concept, embodiments of the present application also provide a map merging device for implementing the aforementioned map merging method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more map merging device embodiments provided below can be found in the above-described limitations of the map merging method and will not be further elaborated here.

[0174] In one embodiment, Figure 13 As shown, a map merging device is provided, comprising: a first acquisition module 11, a determination module 12 and a conversion module 13, wherein:

[0175] The first acquisition module 11 is used to acquire multiple local point cloud maps in the target area, where there are overlapping areas between the local point cloud maps, and each local point cloud map includes node information of the robot during the construction of the local point cloud map;

[0176] A determination module 12 is configured to determine a relative pose transformation between starting nodes in adjacent local point cloud maps based on the overlapped areas of the local point cloud maps and the node information in the local point cloud maps;

[0177] The conversion module 13 is used to perform coordinate alignment conversion on all node information in each local point cloud map based on the relative pose transformation between the starting nodes in each adjacent local point cloud map to obtain a global point cloud map of the target area.

[0178] The map merging device provided in this embodiment can execute the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail here.

[0179] In one embodiment, the determination module includes: a first determination unit, a first acquisition unit, and a second acquisition unit, wherein:

[0180] A first determining unit is configured to determine all adjacent local point cloud maps in each local point cloud map based on an overlapping area of each local point cloud map;

[0181] A first acquiring unit is configured to acquire a plurality of matching node pairs in adjacent local point cloud maps based on node information in each local point cloud map;

[0182] The second acquisition unit is used to obtain the node loop constraint relationship between each matching node pair in each adjacent local point cloud map;

[0183] The second determining unit is used to determine the relative pose transformation between the starting nodes in each adjacent local point cloud map according to the node loop constraint relationship between each matching node pair in each adjacent local point cloud map.

[0184] The map merging device provided in this embodiment can execute the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail here.

[0185] Optionally, the above-mentioned first acquisition unit is also used to, when each adjacent local point cloud map includes a first local point cloud map and a second local point cloud map, and the node information includes a point cloud descriptor of the node, for each adjacent local point cloud map, match the point cloud descriptors of each reference node in the second local point cloud map with the point cloud descriptors of each loop candidate node in the first local point cloud map; determine the reference nodes and loop candidate nodes whose point cloud descriptors are successfully matched as matching node pairs, and obtain multiple matching node pairs in each adjacent local point cloud map.

[0186] The map merging device provided in this embodiment can execute the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail here.

[0187] Optionally, the second acquisition unit is also used to obtain, for any matching node pair, a loop candidate local point cloud map corresponding to each loop candidate node in the matching node pair, and a reference local point cloud map corresponding to the reference node matching each loop candidate node; perform point cloud registration on the matching loop candidate local point cloud map and the reference local point cloud map; after the point cloud registration is completed, determine the relative pose between the loop candidate node and the reference node in each matching node pair as the node loop constraint relationship between each matching node pair.

[0188] The map merging device provided in this embodiment can execute the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail here.

[0189] Optionally, the second acquisition unit is further used to obtain reference frames corresponding to each reference node and loop candidate frames corresponding to each loop candidate node; according to the postures of a preset number of key frames adjacent to and before each loop candidate frame, the loop candidate frames and the key frames are spliced to obtain a loop candidate local point cloud map corresponding to each loop candidate node; according to the postures of each loop candidate frame, each reference frame is converted to obtain a reference local point cloud map corresponding to a reference node matching each loop candidate node.

[0190] The map merging device provided in this embodiment can execute the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail here.

[0191] Optionally, the above-mentioned second determination unit is also used to obtain the root mean square error value between all matching node pairs in each adjacent local point cloud map; obtain the target matching node pair with the smallest root mean square error value; determine the node loop constraint relationship between the reference node and the loop candidate node in the target matching node pair as the relative pose transformation between the starting nodes in each adjacent local point cloud map; wherein, the reference node in the target matching node pair is the starting node in the second local point cloud map, and the loop candidate node in the target matching node pair is the starting node in the first local point cloud map.

[0192] The map merging device provided in this embodiment can execute the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail here.

[0193] In one embodiment, the map merging device further includes: a second acquisition module and a removal module, wherein:

[0194] A second acquisition module is used to obtain the difference between the root mean square error value of each matching node pair and the root mean square error value of the target matching node pair;

[0195] The elimination module is used to eliminate matching node pairs whose difference is greater than a preset difference to obtain candidate matching node pairs.

[0196] The map merging device provided in this embodiment can execute the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail here.

[0197] In one embodiment, the conversion module further includes an optimization unit and a construction unit, wherein:

[0198] The optimization unit is used to input the node poses and loop constraints in the candidate point cloud map into a preset optimizer, and optimize the node poses through the optimizer to obtain the target node poses; the loop constraints include the node loop constraints between candidate matching node pairs, the loop constraints in each local point cloud map, and the node loop constraints between adjacent nodes in each local point cloud map;

[0199] The construction unit is used to construct a global point cloud map of the target area based on the target node pose and the point cloud data corresponding to each node.

[0200] The map merging device provided in this embodiment can execute the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail here.

[0201] Optionally, the construction unit is also used to construct a pose graph corresponding to each initial local point cloud map based on the initial local point cloud maps of multiple areas in the target area; through the node pose of each node in the pose graph, the point cloud data of each node is transformed into the global coordinate system of each initial local point cloud map to obtain each local point cloud map.

[0202] The map merging device provided in this embodiment can execute the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail here.

[0203] Each module in the map merging device described above can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of the robot's processor in hardware form, or stored in the robot's memory in software form, allowing the processor to call and execute the corresponding operations of each module.

[0204] In one embodiment, a robot is provided, comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor implements all the contents of the above method embodiments when executing the computer program.

[0205] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, all the contents of the above method embodiments are implemented.

[0206] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, all the contents of the above method embodiments are implemented.

[0207] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0208] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0209] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0210] The above embodiments merely illustrate several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A map merging method, applied to a robot, characterized in that: The method comprises: Acquire multiple local point cloud maps in the target area, where there are overlapping areas between the local point cloud maps, and each of the local point cloud maps includes node information used to construct the local point cloud map; Determining a relative position transformation relationship between starting nodes in adjacent local point cloud maps based on the overlapping areas of the local point cloud maps and the node information in the local point cloud maps; Based on the relative pose transformation relationship between the starting nodes in each of the adjacent local point cloud maps, coordinate alignment transformation is performed on all node information in each of the local point cloud maps to obtain a global point cloud map of the target area; The method of determining the relative pose transformation relationship between the starting nodes in each adjacent local point cloud map based on the overlapping area of each local point cloud map and the node information in each local point cloud map includes: determining all adjacent local point cloud maps in each local point cloud map based on the overlapping area of each local point cloud map; obtaining matching node pairs in each adjacent local point cloud map based on the node information in each local point cloud map; obtaining node loop constraint relationships between each matching node pair in each adjacent local point cloud map; and determining the relative pose transformation relationship between the starting nodes in each adjacent local point cloud map based on the node loop constraint relationships between each matching node pair in each adjacent local point cloud map. The method further includes: obtaining root mean square error values between all matching node pairs in each of the adjacent local point cloud maps; obtaining a target matching node pair with the smallest root mean square error value; obtaining the difference between the root mean square error value of each matching node pair and the root mean square error value of the target matching node pair; and eliminating matching node pairs with a difference greater than a preset difference to obtain candidate matching node pairs; The method comprises: performing coordinate alignment conversion on all node information in each of the local point cloud maps based on the relative pose transformation between the starting nodes in each of the adjacent local point cloud maps to obtain a global point cloud map of the target area, including: converting all node information in each of the local point cloud maps into the same coordinate system according to the relative pose transformation between the starting nodes in each of the adjacent local point cloud maps to obtain an initial global point cloud map of the target area; optimizing the poses of the nodes in the initial global point cloud map by a preset optimizer to obtain a global point cloud map of the target area; The method of optimizing the poses of the nodes in the initial global point cloud map by a preset optimizer to obtain a global point cloud map of the target area includes: inputting the poses and loop constraints of the nodes in the initial global point cloud map into the preset optimizer, optimizing the poses of the nodes in the initial global point cloud map by the preset optimizer to obtain the poses of the target nodes; the loop constraints include the node loop constraints between the candidate matching node pairs, the loop constraints in each of the local point cloud maps, and the node loop constraints between adjacent nodes in each of the local point cloud maps; and constructing a global point cloud map of the target area based on the target node poses and the point cloud data corresponding to each node.

2. The method according to claim 1, characterized in that If each adjacent local point cloud map includes a first local point cloud map and a second local point cloud map; and the node information in each of the local point cloud maps includes a point cloud descriptor of the node; Then, obtaining matching node pairs in each of the adjacent local point cloud maps according to the node information in each of the local point cloud maps includes: For each of the adjacent local point cloud maps, matching the point cloud descriptors of each reference node in the second local point cloud map with the point cloud descriptors of each loop candidate node in the first local point cloud map; The reference node and the loop candidate node that are successfully matched by the point cloud descriptor are determined as matching node pairs, and the matching node pairs in each of the adjacent local point cloud maps are obtained.

3. The method according to claim 2, characterized in that The obtaining of the node loop constraint relationship between each matching node pair in each adjacent local point cloud map includes: For any matching node pair, obtain the loop candidate local point cloud map corresponding to each loop candidate node in the matching node pair, and the reference local point cloud map corresponding to the reference node matching each loop candidate node; Perform point cloud registration on the matching loop candidate local point cloud map and the reference local point cloud map; After the point cloud registration is completed, the relative pose between the loop candidate node and the reference node in each matching node pair is determined as the node loop constraint relationship between each matching node pair.

4. The method according to claim 3, characterized in that The obtaining of a loop candidate local point cloud map corresponding to each loop candidate node in the matching node pair and a reference local point cloud map corresponding to a reference node matching each loop candidate node includes: Acquire a reference frame corresponding to each reference node and a loop candidate frame corresponding to each loop candidate node; According to the poses of a preset number of key frames adjacent to and preceding each loop candidate frame, the loop candidate frame is spliced with the preset number of key frames to obtain a loop candidate local point cloud map corresponding to each loop candidate node; According to the pose of each of the loop candidate frames, each of the reference frames is converted to obtain a reference local point cloud map corresponding to a reference node that matches each of the loop candidate nodes.

5. The method according to any one of claims 2 to 4, characterized in that: The determining, based on the node loop constraint relationship between each matching node pair in each adjacent local point cloud map, the relative pose transformation between the starting nodes in each adjacent local point cloud map comprises: The node loop constraint relationship between the reference node and the loop candidate node in the target matching node pair is determined as the relative pose transformation between the starting nodes in each of the adjacent local point cloud maps; wherein, the reference node in the target matching node pair is the starting node in the second local point cloud map, and the loop candidate node in the target matching node pair is the starting node in the first local point cloud map.

6. The method according to any one of claims 1 to 4, characterized in that The obtaining of multiple local point cloud maps in the target area includes: Constructing a pose graph corresponding to each of the initial local point cloud maps according to the initial local point cloud maps of the plurality of areas in the target area; Through the node pose of each node in the pose graph, the point cloud data of each node is transformed into the global coordinate system of each initial local point cloud map to obtain each local point cloud map.

7. A map merging device, characterized in that: The device comprises: A first acquisition module is configured to acquire a plurality of local point cloud maps in a target area, wherein the local point cloud maps have overlapping areas, and each of the local point cloud maps includes node information of the robot during the construction of the local point cloud map; a determination module, configured to determine a relative pose transformation between starting nodes in adjacent local point cloud maps based on an overlapping area of each local point cloud map and node information in each local point cloud map; A conversion module, configured to perform coordinate alignment conversion on all node information in each of the local point cloud maps based on the relative pose transformation between the starting nodes in each of the adjacent local point cloud maps, to obtain a global point cloud map of the target area; The determination module includes a first determination unit, a second determination unit, a first acquisition unit, and a second acquisition unit, wherein the first determination unit is used to determine all adjacent local point cloud maps in each of the local point cloud maps based on the overlapping area of each of the local point cloud maps; the first acquisition unit is used to acquire matching node pairs in the adjacent local point cloud maps based on the node information in each of the local point cloud maps; the second acquisition unit is used to acquire the node loop constraint relationship between each matching node pair in each of the adjacent local point cloud maps; the second determination unit is used to determine the relative pose transformation relationship between the starting nodes in each of the adjacent local point cloud maps based on the node loop constraint relationship between each matching node pair in each of the adjacent local point cloud maps; The second determining unit is further configured to obtain a root mean square error value between all matching node pairs in each of the adjacent local point cloud maps; and obtain a target matching node pair with the smallest root mean square error value. The map merging device further includes a second obtaining module and a eliminating module; the second obtaining module is configured to obtain a difference between the root mean square error value of each matching node pair and the root mean square error value of the target matching node pair; the eliminating module is configured to eliminate matching node pairs with a difference greater than a preset difference, thereby obtaining candidate matching node pairs; The conversion module is specifically configured to convert all node information in each of the local point cloud maps into the same coordinate system based on the relative pose transformation between the starting nodes in each of the adjacent local point cloud maps, thereby obtaining an initial global point cloud map of the target area; and optimize the poses of the nodes in the initial global point cloud map using a preset optimizer to obtain a global point cloud map of the target area; The conversion module includes an optimization unit and a construction unit. The optimization unit is used to input the pose and loop constraint relationship of the nodes in the initial global point cloud map into the preset optimizer, and optimize the pose of the nodes in the initial global point cloud map through the preset optimizer to obtain the target node pose; the loop constraint includes the node loop constraint relationship between the candidate matching node pairs, the loop constraint relationship in each of the local point cloud maps, and the node loop constraint relationship between adjacent nodes in each of the local point cloud maps; the construction unit is used to construct a global point cloud map of the target area based on the target node pose and the point cloud data corresponding to each node.

8. The device according to claim 7, characterized in that If each adjacent local point cloud map includes a first local point cloud map and a second local point cloud map; and the node information in each of the local point cloud maps includes the point cloud descriptor of the node; then the first acquisition unit is specifically used to match the point cloud descriptors of each reference node in the second local point cloud map with the point cloud descriptors of each loop candidate node in the first local point cloud map for each of the adjacent local point cloud maps; determine the reference node and the loop candidate node whose point cloud descriptors are successfully matched as a matching node pair, and obtain the matching node pair in each of the adjacent local point cloud maps.

9. The device according to claim 8, characterized in that The second acquisition unit is specifically configured to acquire, for any matching node pair, a loop candidate local point cloud map corresponding to each loop candidate node in the matching node pair, and a reference local point cloud map corresponding to a reference node matching each loop candidate node; and perform point cloud registration on the matching loop candidate local point cloud map and the reference local point cloud map; After the point cloud registration is completed, the relative pose between the loop candidate node and the reference node in each matching node pair is determined as the node loop constraint relationship between each matching node pair.

10. A robot comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

Images