Map alignment method, device, equipment and storage medium

Abstract

The present invention provides a map alignment method, apparatus, device, and storage medium, comprising: obtaining multiple submaps generated for different areas in a target scene; determining a first set of pose transformation relationships corresponding to the multiple submaps based on map features corresponding to the multiple submaps, and determining credible pose transformation relationships therefrom to form a second set of pose transformation relationships; establishing a directed graph corresponding to the multiple submaps based on the second set of pose transformation relationships; starting with the vertices corresponding to the reference map, determining a target path containing the largest number of vertices from the loops and / or links corresponding to the directed graph; and aligning the submaps corresponding to the multiple vertices of the target path from the reference map based on the second set of pose transformation relationships to obtain a global map of the target scene. Because the pose transformation relationships between the submaps corresponding to the target path are all credible and cover a large number of submaps, an accurate and complete global map of the target scene can be obtained.

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and in particular to a map alignment method, device, equipment and storage medium. Background Art

[0002] When constructing a map of a target scene, an intelligent robot can first scan different areas of the target scene using its sensory devices (such as cameras and laser sensors) to obtain scene information for each area and construct corresponding submaps. The submaps are then aligned and merged to form a complete map of the target scene. Related techniques often align all submaps in a random order, which can easily lead to mismatches and misalignments between submaps, resulting in inconsistencies between the final merged map and the actual target scene. Summary of the Invention

[0003] Embodiments of the present invention provide a map alignment method, apparatus, device, and storage medium for improving the accuracy of generating a global map of a target scene.

[0004] In a first aspect, an embodiment of the present invention provides a map alignment method, the method comprising:

[0005] Obtain multiple submaps generated for different areas in the target scene;

[0006] Determining, based on the map features corresponding to the multiple submaps, pose transformation relationships between each of the multiple submaps to form a first pose transformation relationship set;

[0007] Determining a second pose transformation relationship set, where the second pose transformation relationship set includes credible pose transformation relationships in the first pose transformation relationship set;

[0008] Establish a directed graph corresponding to the multiple sub-maps; wherein the vertices in the directed graph are connected to the multiple sub-maps Figure 1 One-to-one, an edge between any two vertices in the directed graph matches a posture transformation relationship in the second posture transformation relationship set;

[0009] Taking the vertex corresponding to the reference map as a starting point, determining a target path containing the largest number of vertices from the loops and / or links corresponding to the directed graph, wherein the reference map is the starting submap when aligning the multiple submaps;

[0010] Based on the second pose transformation relationship set, submaps corresponding to multiple vertices of the target path are aligned starting from the reference map to obtain a global map of the target scene.

[0011] In a second aspect, an embodiment of the present invention provides a map alignment device, the device comprising:

[0012] An acquisition module, used to acquire multiple sub-maps generated for different areas in the target scene;

[0013] The processing module is configured to determine the pose transformation relationships between the plurality of submaps according to the map features corresponding to the plurality of submaps, so as to form a first pose transformation relationship set; determine a second pose transformation relationship set, wherein the second pose transformation relationship set includes the credible pose transformation relationships in the first pose transformation relationship set; establish a directed graph corresponding to the plurality of submaps; wherein the vertices in the directed graph are connected to the plurality of submaps; Figure 1 One-to-one correspondence, the edge between any two vertices in the directed graph is matched with the pose transformation relationship in the second pose transformation relationship set; taking the vertex corresponding to the reference map as the starting point, determining the target path containing the largest number of vertices from the loop and / or link corresponding to the directed graph, and the reference map is the starting submap when the multiple submaps are aligned; based on the second pose transformation relationship set, starting from the reference map, aligning the submaps corresponding to the multiple vertices of the target path to obtain a global map of the target scene.

[0014] In a third aspect, an embodiment of the present invention provides an electronic device comprising: an inertial measurement unit, a camera, a laser sensor, a memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the map alignment method as described in the first aspect.

[0015] In a fourth aspect, an embodiment of the present invention provides a non-transitory machine-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor can at least implement the map alignment method described in the first aspect.

[0016] In an embodiment of the present invention, for multiple submaps corresponding to different areas in a target scene, first, a credible pose transformation relationship is screened out from the pose transformation relationships between the multiple submaps; then, based on the credible pose transformation relationship, the starting submap (i.e., the reference map) and the directed graph corresponding to the multiple submaps are determined when aligning the multiple submaps; thereafter, with the vertex corresponding to the reference map as the starting point, the target path containing the largest number of vertices is determined from the loop and / or link corresponding to the directed graph; based on the second pose transformation relationship set, the submaps corresponding to the multiple vertices of the target path are aligned starting from the reference map to obtain a global map of the target scene. Since the pose transformation relationships between the submaps corresponding to each vertex in the target path are all credible pose transformation relationships and the target path covers more submaps, the submaps are aligned according to the connection order of the vertices in the target path, so that an accurate and complete global map of the target scene can be obtained, thereby avoiding the misalignment of submaps caused by untrustworthy pose transformation relationships. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 A flowchart of a map alignment method provided by an embodiment of the present invention;

[0019] Figure 2 A schematic diagram of an overlapping area provided by an embodiment of the present invention;

[0020] Figure 3 A schematic diagram of a sub-map provided by an embodiment of the present invention;

[0021] Figure 4 A flowchart of a method for obtaining a posture transformation relationship provided by an embodiment of the present invention;

[0022] Figure 5 A schematic diagram of a matching key frame provided by an embodiment of the present invention;

[0023] Figure 6 A flow chart of a method for verifying posture transformation relationships provided by an embodiment of the present invention;

[0024] Figure 7 A schematic diagram of a directed graph provided by an embodiment of the present invention;

[0025] Figure 8 A schematic structural diagram of a map alignment device provided by an embodiment of the present invention;

[0026] Figure 9 This is a schematic structural diagram of an electronic device provided in this embodiment. DETAILED DESCRIPTION

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0028] In addition, the step sequence in the following method embodiments is only an example and not a strict limitation.

[0029] Figure 1 A flowchart of a map alignment method provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method includes the following steps:

[0030] 101. Obtain multiple submaps generated for different areas in the target scene.

[0031] 102. Determine the pose transformation relationship between each of the plurality of submaps based on the map features corresponding to the plurality of submaps to form a first pose transformation relationship set.

[0032] 103. Determine a second pose transformation relationship set, where the second pose transformation relationship set includes credible pose transformation relationships in the first pose transformation relationship set.

[0033] 104. Create a directed graph corresponding to multiple sub-maps; where the vertices in the directed graph are connected to multiple sub-maps. Figure 1 One-to-one correspondence, the edge between any two vertices in the directed graph matches the pose transformation relationship in the second pose transformation relationship set.

[0034] 105. Taking the vertex corresponding to the base map as the starting point, determine the target path containing the largest number of vertices from the loops and / or links corresponding to the directed graph. The base map is the starting submap when aligning multiple submaps.

[0035] 106. Based on the second pose transformation relationship set, align the submaps corresponding to multiple vertices of the target path from the reference map to obtain a global map of the target scene.

[0036] The map alignment method provided by the embodiment of the present invention can be applied to a robot, or can be applied to a robot client installed on a smart phone, a laptop computer or a server. The robot client can interact with the robot through network communication to exchange information and instructions.

[0037] When mapping a target scene, it's typically necessary to first perceive the scene using sensing devices (such as inertial measurement units, laser sensors, and cameras). Mapping is then performed based on the scene information (such as laser point cloud data and visual images) acquired by the sensing devices. In practice, when mapping larger target scenes (such as outdoor scenes), it's difficult to obtain all the scene information for the entire scene at once, and errors in the mapping process often require re-building the map.

[0038] In this embodiment, when generating a global map of the target scene, first, the target scene is divided into different areas, where there are overlapping areas between different areas, which can also be called common view areas; then, corresponding scene information is collected for different areas through sensing devices, and sub-maps of each area are constructed based on the corresponding scene information of each area; then, multiple sub-maps corresponding to different areas in the target scene are aligned to obtain a global map of the target scene.

[0039] In step 101, the multiple sub-maps of different areas in the target scene obtained may be visual maps or laser maps, etc. This embodiment does not limit the map type of the sub-maps.

[0040] When generating a global map of the target scene, aligning multiple submaps essentially means aligning the parts of the submaps that represent the same overlapping area, that is, making the parts of the submaps that represent the same overlapping area overlap.

[0041] For ease of understanding, for example, Figure 2 A schematic diagram of an overlapping area provided by an embodiment of the present invention, such as Figure 2 As shown in FIG, it is assumed that there is an overlapping area ab (gray area) between area a and area b in the target scene. Based on this assumption, when collecting scene information in area a, the scene information of the overlapping area ab will be collected, and when collecting scene information in area b, the scene information of the overlapping area ab will also be collected. Thus, as shown in FIG. Figure 3 As shown, the generated submap Ma of area a contains a partial map Ma' corresponding to the overlapping area ab, and the generated submap Mb of area b contains a partial map Mb' corresponding to the overlapping area ab. Figure 3A schematic diagram of a submap provided by an embodiment of the present invention. Aligning the submap Ma with the submap Mb means that part of the map Ma' overlaps part of the map Mb'.

[0042] In this embodiment, to achieve alignment of multiple submaps, after obtaining multiple submaps, it is first determined whether there are partial maps representing the same overlapping area between the multiple submaps. If so, based on the partial maps in the two submaps corresponding to the same overlapping area, the posture transformation relationship between the two maps is determined, and the two submaps are aligned based on the posture transformation relationship.

[0043] As an optional implementation, in step 102 , the position transformation relationship between the two submaps having the same overlapping area can be determined by performing feature matching on map features corresponding to the two submaps.

[0044] Among them, map features can be understood as features corresponding to scene information used to construct sub-maps, which are a type of map data. In actual applications, different types of maps correspond to different types of scene information, and thus the types of map features they correspond to are also different. For example, if the sub-map of area X is a visual map, the map features corresponding to the sub-map include: two-dimensional visual feature points extracted from the visual image corresponding to area X, and / or three-dimensional visual feature points reconstructed from the two-dimensional visual feature points through methods such as triangulation; if the sub-map of area X is a laser map, the map features corresponding to the sub-map include: two-dimensional laser point cloud, and / or three-dimensional laser point cloud.

[0045] The following combination Figure 4 , the specific process of determining the posture transformation relationship between the multiple sub-maps according to the map features corresponding to the multiple sub-maps in step 102 is described.

[0046] Figure 4 A flowchart of a method for obtaining a posture transformation relationship provided by an embodiment of the present invention is shown in FIG. Figure 4 As shown, the method includes the following steps:

[0047] 401. Determine similar feature points between each of the plurality of sub-maps based on map features corresponding to the plurality of sub-maps.

[0048] 402. Determine key frames that match each other between multiple submaps based on similar feature points. The key frames contain scene information of the corresponding areas of the submaps and are used to generate the submaps.

[0049] 403. Determine the posture transformation relationship between each of the multiple sub-maps based on the posture information corresponding to the matched key frames.

[0050] Among them, based on the map features corresponding to multiple sub-maps, similar feature points in the map features corresponding to any two sub-maps can be determined through a preset feature matching algorithm. For example, if the map feature corresponding to the sub-map is a three-dimensional laser point cloud, the similar laser point cloud data in the three-dimensional laser point cloud corresponding to the two sub-maps can be determined as the similar feature points between the two sub-maps through an iterative closest point (ICP) algorithm; if the map feature corresponding to the sub-map is a two-dimensional visual feature point, the distance between the descriptors corresponding to the two-dimensional visual feature points of different sub-maps can be calculated, such as the Euclidean distance or the Hamming distance. If the distance between the descriptors of the two two-dimensional visual feature points is less than a set threshold, the two two-dimensional visual feature points are determined to be similar feature points.

[0051] It's understood that similar feature points between two submaps exist in pairs, and the two feature points in a pair correspond to map features of different submaps. For any submap, the visual image or laser point cloud data that contains scene information for the submap's corresponding area and is used to generate the submap is called a keyframe for that submap. Any keyframe contains at least one map feature, for example, a visual keyframe contains multiple two-dimensional visual feature points.

[0052] After determining the similar feature points between the two submaps, we can further determine the keyframes corresponding to the similar feature points. These keyframes are called the matched keyframes between the two submaps. Similar to similar feature points, matched keyframes also exist in pairs, and multiple pairs of similar feature points between the two submaps may correspond to the same pair of matched keyframes.

[0053] Combine Figure 5 For example, Figure 5 A schematic diagram of a matching key frame provided by an embodiment of the present invention, such as Figure 5 As shown, assuming that keyframe a1 of submap Ma contains map features Fa1, Fa2, and Fa3, and keyframe b2 of submap Mb contains map features Fb1, Fb2, and Fb3, a preset matching algorithm is used to determine that the similar feature points between submap Ma and submap Mb are: Fa1 and Fb2, and Fa2 and Fb3. Based on the similar feature points Fa1 and Fb2, the matching keyframes between submap Ma and submap Mb are determined to be keyframe a1 and keyframe b2; based on the similar feature points Fa2 and Fb3, the matching keyframes between submap Ma and submap Mb are also determined to be keyframe a1 and keyframe b2. It can be understood that the greater the number of similar feature points corresponding to a pair of matching keyframes, the greater the probability that the scene information contained in the matching keyframes corresponds to the same area.

[0054] After determining the matching keyframes between the two submaps, the pose transformation relationship between the two submaps is determined based on the pose information corresponding to the matching keyframes. The pose information corresponding to any keyframe is predetermined when the submap is generated.

[0055] During implementation, there may be more than one pair of matching keyframes in the two submaps. Based on the pose transformation relationship determined for each pair of matching keyframes, the pose transformation relationship between the two submaps can be determined. For example, suppose that submap Ma and submap Mb have matching keyframes a1 and b2, as well as matching keyframes a3 and b4. Keyframes a1 and a3 correspond to submap Ma, and keyframes b2 and b4 correspond to submap Mb. Assuming that pose transformation relationship T12 is determined based on the pose transformation information corresponding to keyframes a1 and b2, and pose transformation relationship T34 is determined based on the pose transformation information corresponding to keyframes a3 and a4, then pose transformation relationship Tab between submap Ma and submap Mb is determined based on T12 and T34.

[0056] based on Figure 4 The posture transformation acquisition method shown can obtain the posture transformation relationship between multiple submaps, and the posture transformation relationship between multiple submaps constitutes a first posture transformation relationship set.

[0057] However, in practical applications, the pose transformation relationships in the first pose transformation relationship set are not always reliable. For example, there is actually no overlap between area c corresponding to submap Mc and area d corresponding to submap Md. However, due to some noise in the map features corresponding to submaps Mc and Md, incorrect similar feature points are mistakenly matched during feature matching, and the pose transformation relationship Tcd between submaps Mc and Md is further determined. If submaps Mc and Md are aligned based on the pose transformation relationship Tcd, the resulting global map of the target scene will inevitably be inaccurate. Therefore, the pose transformation relationships in the first pose transformation relationship set must be verified.

[0058] In this embodiment, the pose transformation relationship between any two submaps includes: a first pose transformation relationship and a second pose transformation relationship. The first pose transformation relationship is the pose transformation relationship from the first submap to the second submap, and the second pose transformation relationship is the pose transformation relationship from the second submap to the first submap. The first submap and the second submap are any two submaps from a plurality of submaps.

[0059] In practice, if there is overlap between the areas corresponding to the two submaps, then there will be a certain number of correctly matched similar feature points in the map features corresponding to the two submaps, and the number of matching keyframes corresponding to the similar feature points will also be relatively large. Furthermore, if the first pose transformation relationship and the second pose transformation relationship between the two submaps are correct, then the first matrix representing the first pose transformation relationship and the second matrix representing the second pose transformation relationship are mutually inverse.

[0060] As an optional implementation, you can Figure 6 The method shown verifies whether each pose transformation relationship in the first pose transformation relationship set is credible. Figure 6 A flow chart of a method for verifying posture transformation relationship provided by an embodiment of the present invention is as follows: Figure 6 As shown, the following steps are included:

[0061] 601. Input similar feature points between two sub-maps corresponding to a target pose transformation relationship into a preset random sampling consistency model to determine the inlier rate of similar feature points corresponding to the target pose transformation relationship, where the target pose transformation relationship is any pose transformation relationship in the first pose transformation relationship set.

[0062] 602. If the product of the first pose transformation relationship and the second pose transformation relationship corresponding to the target pose transformation relationship is a unit matrix, and the inlier rate of similar feature points corresponding to the target pose transformation relationship is greater than or equal to the first threshold, and the number of key frames matching between the two sub-maps corresponding to the target pose transformation relationship is greater than the second threshold, then the target pose transformation relationship is credible.

[0063] In this embodiment, the product of the first and second pose transformation relationships corresponding to the target pose transformation relationship, the inlier rate of similar feature points corresponding to the target pose transformation relationship, and the number of matching keyframes between the two submaps corresponding to the target pose transformation relationship are collectively used as parameters to determine whether the target pose transformation relationship is credible. In practical applications, one or more of these parameters can be selected as parameters to determine whether the target pose transformation relationship is credible.

[0064] In step 601, the Random Sample Consensus (RANSAC) model is used to detect outliers, i.e., incorrectly matched similar feature points, among the similar feature points between the two submaps. Assuming that the number of similar feature points between the two submaps corresponding to the target pose transformation relationship is A, and the number of outliers detected by the preset RANSAC model is A1, then the inlier rate r of the similar feature points corresponding to the target pose transformation relationship is r = (A-A1) / A, where the inlier rate is the ratio of the number of correctly matched similar feature points to the total number of similar feature points. The larger the inlier rate of the similar feature points corresponding to the target pose transformation relationship, the greater the number of correctly matched similar feature points among the similar feature points between the two submaps corresponding to the target pose transformation relationship, and the more credible the target pose transformation relationship is.

[0065] In actual applications, due to factors such as calculation errors and perception device measurement errors, the product of the first pose transformation relationship T1 and the second pose transformation relationship T2 between the sub-maps corresponding to two overlapping areas may not be the unit matrix, but in fact T1 and T2 are credible.

[0066] In order to avoid screening out credible pose transformation relationships whose product of the first pose transformation relationship and the second pose transformation relationship is not the identity matrix, optionally, first, the product of the first pose transformation relationship and the second pose transformation relationship corresponding to the target pose transformation relationship is determined; then, the product is decomposed into a rotation matrix for representing rotation parameters and a translation matrix for representing translation parameters; finally, if the modulus of the difference between the rotation matrix and the identity matrix is less than a third threshold and the modulus of the translation matrix is less than a fourth threshold, then the product of the first pose transformation relationship and the second pose transformation relationship corresponding to the target pose transformation relationship is determined to be the identity matrix. By setting the third and fourth thresholds, it is possible to avoid screening out some pose transformation relationships whose product of the first pose transformation relationship and the second pose transformation relationship is approximately the identity matrix.

[0067] based on Figure 6 The method for verifying the posture transformation relationship shown can filter out credible posture transformation relationships from the first posture transformation relationship set, and form the second posture transformation relationship set in step 103 with the credible posture transformation relationships.

[0068] Then, step 104 is executed to establish a directed graph corresponding to multiple sub-maps. Figure 1 One-to-one correspondence, the edge between any two vertices in the directed graph matches the pose transformation relationship in the second pose transformation relationship set.

[0069] When establishing a directed graph corresponding to multiple submaps, each submap corresponds to a vertex of the directed graph. If there is a posture transformation relationship between two submaps in the second posture transformation relationship set, two edges indicating opposite directions are added between the vertices corresponding to the two submaps. Figure 7 A schematic diagram of a directed graph provided for an embodiment of the present invention assumes that the target scene is divided into four areas: area a, area b, area c, and area d, and the corresponding submaps are: submap Ma, submap Mb, submap Mc, and submap Md, respectively. The second pose transformation relationship set includes: the pose transformation relationship Tab between submap Ma and submap Mb, the pose transformation relationship Tbc between submap Mb and submap Mc, the pose transformation relationship Tac between submap Ma and submap Mc, and the pose transformation relationship Tad between submap Ma and submap Md. Based on this assumption, submap Ma, submap Mb, submap Mc, and submap Md correspond to vertices A, B, C, and D of the directed graph, respectively, and based on Tab, Tbc, Tac, and Tad, two edges are added between vertices A and B, B and C, A and C, and A and D, respectively, to form a directed graph of four submaps, as shown in FIG. Figure 7 shown.

[0070] After determining the directed graphs of the multiple submaps, further, an alignment order of the multiple submaps is determined based on the directed graphs.

[0071] In step 105, starting from the vertex corresponding to the reference map, a target path containing the largest number of vertices is determined from the loops and / or links corresponding to the directed graph. The specific implementation process includes the following steps:

[0072] First, a submap is selected from multiple submaps to serve as the starting submap for alignment, also known as the base map. Optionally, the sum of the pose transformation relationships corresponding to each submap can be determined based on the second pose transformation relationship set. The submap corresponding to the largest sum of the pose transformation relationships corresponding to the multiple submaps is determined as the base map. Alternatively, based on the areas of the multiple submaps, the submap with the largest area can be determined as the base map.

[0073] Next, starting with the vertex corresponding to the reference map, determine the corresponding loops and / or links in the directed graph. A loop's start and end points are the same vertex, and within a loop, all vertices except the start and end points are unique. Within a link, the end point of the link is connected only by an edge to the previous vertex, and all vertices within the same link are unique.

[0074] Finally, a target path containing the largest number of vertices is determined from the corresponding cycles and / or links of the directed graph.

[0075] For ease of understanding, for example, Figure 7 For example, let's assume that the pose transformation relationships for submap Ma include Tab, Tac, and Tad, so the total number of pose transformation relationships for submap Ma is 3. The pose transformation relationships for submap Mb include Tab and Tbc, so the total number of pose transformation relationships for submap Mb is 2. The pose transformation relationships for submap Mc include Tbc and Tac, so the total number of pose transformation relationships for submap Mc is 2. The pose transformation relationships for submap Md include Tad, so the total number of pose transformation relationships for submap Md is 1. Based on this, submap Ma is determined to be the baseline map.

[0076] Afterwards, take the vertex A corresponding to the submap Ma as the starting point and determine Figure 7 The directed graph shown corresponds to a cycle: A→B→C→A, and a link: A→D. Because the cycle (3 vertices) is greater than the link (2 vertices), the cycle A→B→C→A is determined to be the target path. In practice, a directed graph may have more than one cycle or link.

[0077] It's important to note that the process of determining the target path can be understood as a secondary screening of the pose transformation relationships between multiple submaps. The purpose of this secondary screening is to confirm the correct submap alignment order. It's understandable that the more vertices a target path contains, the more submaps it corresponds to, and thus, a more complete representation of the target scene after alignment.

[0078] After the target path is determined, based on the second pose transformation relationship set, the submaps corresponding to multiple vertices of the target path are aligned starting from the reference map to obtain a global map of the target scene.

[0079] Specifically, the alignment order of the submaps corresponding to the multiple vertices is first determined according to the connection order of the multiple vertices of the target path; then, the pose transformation relationship between adjacent submaps in the submaps corresponding to the multiple vertices is determined according to the second pose transformation relationship set; according to the alignment order and the pose transformation relationship between adjacent submaps, the submaps corresponding to the multiple vertices are aligned in sequence starting from the baseline map to obtain the global map of the target scene.

[0080] For example, still based on Figure 7Correspondingly, after determining the target path as a loop A→B→C→A, the submaps used for alignment to generate the global map of the target scene are determined to be submaps Ma, Mb, and Mc. The corresponding alignment order, from first to last, is: submap Ma, submap Mb, and submap Mc. The corresponding pose transformation relationships between adjacent submaps are: Tab, Tbc, and Tac. Next, starting with submap Ma as the reference map, submap Mb is aligned with submap Ma based on Tab to obtain map Mab. Submap Mc is then aligned with map Mab based on Tbc and Tac. The resulting alignment is the global map of the target scene.

[0081] In an embodiment of the present invention, for multiple submaps corresponding to different areas in a target scene, first, a credible pose transformation relationship is screened out from the pose transformation relationships between the multiple submaps; then, based on the credible pose transformation relationship, the starting submap (i.e., the reference map) and the directed graph corresponding to the multiple submaps are determined when aligning the multiple submaps; thereafter, with the vertex corresponding to the reference map as the starting point, the target path containing the largest number of vertices is determined from the loop and / or link corresponding to the directed graph; based on the second pose transformation relationship set, the submaps corresponding to the multiple vertices of the target path are aligned starting from the reference map to obtain a global map of the target scene. Since the pose transformation relationships between the submaps corresponding to each vertex in the target path are all credible pose transformation relationships and the target path covers more submaps, the submaps are aligned according to the connection order of the vertices in the target path, so that an accurate and complete global map of the target scene can be obtained, thereby avoiding the misalignment of submaps caused by untrustworthy pose transformation relationships.

[0082] The map alignment device of one or more embodiments of the present invention will be described in detail below. Those skilled in the art will appreciate that these devices can be constructed using commercially available hardware components and configured according to the steps taught in this solution.

[0083] Figure 8 A structural diagram of a map alignment device provided by an embodiment of the present invention is shown in FIG. Figure 8 As shown, the device includes: an acquisition module 11 and a processing module 12.

[0084] The acquisition module 11 is configured to acquire multiple sub-maps generated for different areas in the target scene.

[0085] The processing module 12 is configured to determine the pose transformation relationships between the plurality of submaps according to the map features corresponding to the plurality of submaps, so as to form a first pose transformation relationship set; determine a second pose transformation relationship set, wherein the second pose transformation relationship set includes the credible pose transformation relationships in the first pose transformation relationship set; establish a directed graph corresponding to the plurality of submaps; wherein the vertices in the directed graph are connected to the plurality of submaps; Figure 1 One-to-one correspondence, the edge between any two vertices in the directed graph is matched with the pose transformation relationship in the second pose transformation relationship set; taking the vertex corresponding to the reference map as the starting point, determining the target path containing the largest number of vertices from the loop and / or link corresponding to the directed graph, and the reference map is the starting submap when the multiple submaps are aligned; based on the second pose transformation relationship set, starting from the reference map, aligning the submaps corresponding to the multiple vertices of the target path to obtain a global map of the target scene.

[0086] Optionally, the processing module 12 is specifically configured to determine similar feature points between each of the multiple submaps based on the map features corresponding to the multiple submaps; determine key frames that match each of the multiple submaps based on the similar feature points, wherein the key frames contain scene information of the corresponding areas of the submaps and are used to generate the submaps; and determine the posture transformation relationship between each of the multiple submaps based on the posture information corresponding to the matched key frames.

[0087] Optionally, the posture transformation relationship between any two submaps includes a first posture transformation relationship and a second posture transformation relationship; wherein, the first posture transformation relationship is the posture transformation relationship from the first submap to the second submap, and the second posture transformation relationship is the posture transformation relationship from the second submap to the first submap. The first posture transformation relationship and the second posture transformation relationship are used to determine whether the posture transformation relationship between the corresponding two submaps is credible, and the first submap and the second submap are any two submaps among the multiple submaps. The processing module 12 is further specifically used to input similar feature points between two sub-maps corresponding to the target pose transformation relationship into a preset random sampling consistency model to determine the inlier rate of similar feature points corresponding to the target pose transformation relationship, where the target pose transformation relationship is any pose transformation relationship in the first pose transformation relationship set; if the product of the first pose transformation relationship and the second pose transformation relationship corresponding to the target pose transformation relationship is a unit matrix, and the inlier rate of similar feature points corresponding to the target pose transformation relationship is greater than or equal to a first threshold and the number of key frames matching between the two sub-maps corresponding to the target pose transformation relationship is greater than a second threshold, then the target pose transformation relationship is credible.

[0088] Optionally, the processing module 12 is further specifically used to determine the product of the first posture transformation relationship and the second posture transformation relationship corresponding to the target posture transformation relationship; decompose the product into a rotation matrix for representing rotation parameters, and a translation matrix for representing translation parameters; if the modulus of the difference between the rotation matrix and the unit matrix is less than a third threshold and the modulus of the translation matrix is less than a fourth threshold, then determine that the product is the unit matrix.

[0089] Optionally, the processing module 12 is further specifically used to determine the total number of posture transformation relationships corresponding to each sub-map based on the second posture transformation relationship set; and determine the sub-map corresponding to the largest total number of posture transformation relationships among the sums of the multiple posture transformation relationships corresponding to the multiple sub-maps as the reference map.

[0090] Optionally, the processing module 12 is further specifically configured to determine an alignment order of the submaps corresponding to the multiple vertices of the target path according to a connection order of the multiple vertices; determine a pose transformation relationship between adjacent submaps in the submaps corresponding to the multiple vertices according to the second pose transformation relationship set; and align the submaps corresponding to the multiple vertices in sequence starting from the reference map according to the alignment order and the pose transformation relationship between the adjacent submaps to obtain a global map of the target scene.

[0091] Figure 8 The device shown can execute the steps in the aforementioned embodiments. For detailed execution process and technical effects, please refer to the description in the aforementioned embodiments and will not be repeated here.

[0092] In one possible design, the above Figure 8 The structure of the map alignment device shown can be implemented as an electronic device. Figure 9 As shown, the electronic device may include: a memory 21, a processor 22, and a communication interface 23. The memory 21 stores executable code, and when the executable code is executed by the processor 22, the processor 22 can at least implement the map alignment method provided in the above embodiments.

[0093] In addition, an embodiment of the present invention provides a non-transitory machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the map alignment method provided in the aforementioned embodiment.

[0094] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Those skilled in the art can understand and implement the present invention without inventive effort.

[0095] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by a combination of hardware and software. Based on this understanding, the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a computer product. The present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A map alignment method, characterized in that: The method comprises: Obtain multiple submaps generated for different areas in the target scene; Determining, based on the map features corresponding to the multiple submaps, pose transformation relationships between each of the multiple submaps to form a first pose transformation relationship set; Determining a second pose transformation relationship set, where the second pose transformation relationship set includes credible pose transformation relationships in the first pose transformation relationship set; Establishing a directed graph corresponding to the multiple submaps; wherein the vertices in the directed graph correspond one-to-one to the multiple submaps, and the edges between any two vertices in the directed graph match the pose transformation relationships in the second pose transformation relationship set; Taking the vertex corresponding to the reference map as a starting point, determining a target path containing the largest number of vertices from the loops and / or links corresponding to the directed graph, wherein the reference map is the starting submap when aligning the multiple submaps; Based on the second pose transformation relationship set, submaps corresponding to multiple vertices of the target path are aligned starting from the reference map to obtain a global map of the target scene.

2. The method according to claim 1, characterized in that The determining, based on the map features corresponding to the plurality of submaps, the pose transformation relationship between each of the plurality of submaps comprises: Determining similar feature points between any two of the submaps based on the map features corresponding to the submaps; Determining key frames that match each other between the plurality of submaps based on the similar feature points, wherein the key frames include scene information of the corresponding areas of the submaps, and are used to generate the submaps; The pose transformation relationship between each of the plurality of submaps is determined according to the pose information corresponding to the matched key frames.

3. The method according to claim 2, characterized in that The posture transformation relationship between any two submaps includes a first posture transformation relationship and a second posture transformation relationship; wherein the first posture transformation relationship is the posture transformation relationship from the first submap to the second submap, and the second posture transformation relationship is the posture transformation relationship from the second submap to the first submap. The first posture transformation relationship and the second posture transformation relationship are used to determine whether the posture transformation relationship between the corresponding two submaps is credible, and the first submap and the second submap are any two submaps among the multiple submaps.

4. The method according to claim 3, characterized in that Determine whether each pose transformation relationship in the first pose transformation relationship set is credible by: Inputting similar feature points between two submaps corresponding to a target pose transformation relationship into a preset random sampling consistency model, and determining an inlier rate of similar feature points corresponding to the target pose transformation relationship, wherein the target pose transformation relationship is any pose transformation relationship in the first pose transformation relationship set; If the product of the first pose transformation relationship and the second pose transformation relationship corresponding to the target pose transformation relationship is a unit matrix, and the inlier rate of similar feature points corresponding to the target pose transformation relationship is greater than or equal to a first threshold, and the number of key frames matching between the two submaps corresponding to the target pose transformation relationship is greater than a second threshold, then the target pose transformation relationship is credible.

5. The method according to claim 4, characterized in that The method further comprises: Determine the product of the first pose transformation relationship and the second pose transformation relationship corresponding to the target pose transformation relationship; Decomposing the product into a rotation matrix for representing the rotation parameters and a translation matrix for representing the translation parameters; If the modulus of the difference between the rotation matrix and the identity matrix is less than a third threshold and the modulus of the translation matrix is less than a fourth threshold, it is determined that the product is the identity matrix.

6. The method according to claim 1, characterized in that The method further comprises: Determining the total number of pose transformation relationships corresponding to each submap according to the second pose transformation relationship set; The submap corresponding to the largest sum of the number of posture transformation relationships among the sums of the number of posture transformation relationships corresponding to the multiple submaps is determined as the reference map.

7. The method according to claim 1, characterized in that The step of aligning submaps corresponding to a plurality of vertices of the target path from the reference map based on the second pose transformation relationship set to obtain a global map of the target scene includes: Determining, according to a connection order of the plurality of vertices of the target path, an alignment order of the submaps corresponding to the plurality of vertices; Determining, according to the second pose transformation relationship set, a pose transformation relationship between adjacent submaps in the submaps corresponding to the multiple vertices; According to the alignment order and the pose transformation relationship between the adjacent submaps, the submaps corresponding to the multiple vertices are aligned in sequence starting from the reference map to obtain a global map of the target scene.

8. A map alignment device, characterized in that: The device comprises: An acquisition module, used to acquire multiple sub-maps generated for different areas in the target scene; A processing module is used to determine the pose transformation relationship between the multiple submaps according to the map features corresponding to the multiple submaps respectively, so as to form a first pose transformation relationship set; determine a second pose transformation relationship set, the second pose transformation relationship set includes the credible pose transformation relationships in the first pose transformation relationship set; establish a directed graph corresponding to the multiple submaps; wherein the vertices in the directed graph correspond one-to-one to the multiple submaps, and the edges between any two vertices in the directed graph match the pose transformation relationships in the second pose transformation relationship set; with the vertex corresponding to the base map as the starting point, determine the target path containing the largest number of vertices from the loops and / or links corresponding to the directed graph, the base map being included in the multiple submaps; based on the second pose transformation relationship set, align the submaps corresponding to the multiple vertices of the target path starting from the base map to obtain a global map of the target scene.

9. An electronic device, characterized in that: include: A memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the map alignment method according to any one of claims 1 to 7.

10. A non-transitory machine-readable storage medium, characterized in that The non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor is caused to perform the map alignment method according to any one of claims 1 to 7.

Images