Global Relocation Method, Device, Electronic Device and Storage Medium

Abstract

The present application relates to a global relocalization method, apparatus, electronic device, and storage medium. Among them, the global relocalization method includes: determining a target semantic map and a semantic map to be matched of a target device, where the target semantic map includes a number of standard semantic targets, and the semantic map to be matched includes a number of semantic targets to be matched; determining a minimum spanning tree based on an undirected complete graph based on the semantic map to be matched, and the nodes of the minimum spanning tree are the semantic targets to be matched; constructing a search tree based on the target semantic map, and the nodes of the search tree are the standard semantic targets; determining a growth tree that matches the minimum spanning tree from the constructed search tree; and determining the pose of the target device according to the matching relationship between the standard semantic targets corresponding to the growth tree and the semantic targets to be matched corresponding to the minimum spanning tree.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular, to a global relocalization method, apparatus, electronic device, and storage medium. Background Art

[0002] With the continuous development of positioning technology and automatic control technology, many intelligent devices have realized autonomous movement functions, ranging from driverless vehicles to cleaning robots, etc. Generally, when an intelligent device operates in a target area, it can determine its position in the map of the target area with the help of sensor information, and can achieve autonomous navigation by combining relevant information in the map.

[0003] The map of the target area Figure 1 Generally, it can be constructed as a visual feature map, a laser point cloud map, a probability grid map, a semantic map, etc. Among them, the semantic map is an enhanced representation of the environment, which contains both geometric information and high-level qualitative features and belongs to high-level features. Semantic objects are least affected by environmental factors such as light and season and have higher robustness. In addition, semantic features contain a large amount of information and are sparsely distributed, greatly streamlining the information expression of the map and reducing the required capacity of the map. Therefore, the semantic map is relatively more suitable for large-scale positioning and mapping tasks, such as high-speed positioning and navigation, autonomous parking and other fields.

[0004] The sensor information of the intelligent device may have errors, and the error accumulation will lead to inaccurate positioning information, thereby affecting the navigation accuracy. Therefore, it is necessary to re-determine the pose. In addition, when the intelligent device is powered on or kidnapped and appears in a new environment, it may also be difficult to determine the pose. Therefore, how to improve the global positioning efficiency of intelligent devices has become an important issue in improving navigation accuracy at present. Summary of the Invention

[0005] To solve the above technical problems, the present application provides a global relocalization method, apparatus, electronic device, and storage medium, so as to improve the global positioning efficiency of intelligent devices and thereby improve the navigation accuracy.

[0006] In a first aspect, the present application provides a global relocalization method, including:

[0007] Determine a target semantic map and a to-be-matched semantic map of a target device, where the target semantic map includes a plurality of standard semantic targets, and the to-be-matched semantic map includes a plurality of to-be-matched semantic targets;

[0008] Based on the to-be-matched semantic map, determine a minimum spanning tree based on an undirected complete graph, and the nodes of the minimum spanning tree are the to-be-matched semantic targets;

[0009] Based on the target semantic map, construct a search tree, and the nodes of the search tree are the standard semantic targets;

[0010] Determine a growth tree that matches the minimum spanning tree from the constructed search tree;

[0011] Determine the pose of the target device according to the matching relationship between the standard semantic target corresponding to the growth tree and the semantic target to be matched corresponding to the minimum spanning tree.

[0012] Optionally, determining the minimum spanning tree based on a complete graph based on the semantic map to be matched, where the nodes of the minimum spanning tree are the semantic targets to be matched, includes:

[0013] For each semantic target to be matched, construct an undirected complete graph with the semantic target to be matched as a node;

[0014] For any two semantic targets to be matched, determine the weight of the edge between the nodes constructed by the two semantic targets to be matched according to the translation information of the two semantic targets to be matched;

[0015] Determine the minimum spanning tree of the undirected complete graph based on the weights of the edges in the undirected complete graph.

[0016] Optionally, the determining a growth tree that matches the minimum spanning tree from the constructed search tree includes:

[0017] Construct a survival rule tree based on the survival rule corresponding to each branch in the minimum spanning tree;

[0018] Based on the survival rule tree, determine a growth tree from the constructed search tree, where the growth tree satisfies the survival rule represented by the survival rule tree.

[0019] Optionally, the survival rule corresponding to each branch includes: the relative pose of the node pair corresponding to each branch, the estimated error variance;

[0020] The constructing a survival rule tree based on the survival rule corresponding to each branch in the minimum spanning tree includes:

[0021] For each node pair corresponding to a branch in the minimum spanning tree, calculate the relative pose corresponding to the node pair;

[0022] For each node pair corresponding to a branch in the minimum spanning tree, calculate the estimated error variance corresponding to the node pair;

[0023] Generate the survival rule corresponding to each branch by combining the position of each branch in the minimum spanning tree with the relative pose and the estimated error variance of the node pair corresponding to the branch;

[0024] Determine the survival rule tree according to the survival rules corresponding to each branch in the minimum spanning tree.

[0025] Optionally, the survival rule corresponding to each branch further includes: the attribute characteristics of each node;

[0026] Constructing the survival rule tree based on the survival rules corresponding to each branch in the minimum spanning tree includes:

[0027] For each node pair corresponding to a branch in the minimum spanning tree, calculate the relative pose corresponding to the node pair;

[0028] For each node pair corresponding to a branch in the minimum spanning tree, calculate the estimated error variance corresponding to the node pair;

[0029] Determine the attribute characteristics of each node corresponding to each branch in the minimum spanning tree;

[0030] Generate the survival rule corresponding to each branch by combining the position of each branch in the minimum spanning tree with the relative pose, estimated error variance, and attribute characteristics of each node of the node pair corresponding to the branch;

[0031] Determine the survival rule tree according to the survival rules corresponding to each branch in the minimum spanning tree.

[0032] Optionally, calculating the estimated error variance corresponding to each node pair corresponding to a branch in the minimum spanning tree includes:

[0033] Determine the observation error and distance error corresponding to the target device;

[0034] For each node pair corresponding to a branch in the minimum spanning tree, determine the distance between the target objects to be matched corresponding to the node pair;

[0035] Determine the estimated error variance corresponding to the node pair according to the observation error, the distance error, and the distance corresponding to the node pair.

[0036] Optionally, determining the growth tree from the constructed search tree based on the survival rule tree, where the growth tree satisfies the survival rules represented by the survival rule tree includes:

[0037] Select a node from the minimum spanning tree as the root node;

[0038] Select at least one standard semantic object with the same attribute characteristics as the root node from the target semantic map as the seed of the growth tree;

[0039] Starting from the root node, traverse each node of the minimum spanning tree upward;

[0040] For each node of the minimum spanning tree, perform the following steps:

[0041] Determine the branch to be grown corresponding to the node and the branch nodes that form the branch to be grown with the node;

[0042] According to the survival rule tree, search in the search tree for a standard semantic target that matches the branch node as the growth node of the growth tree;

[0043] After traversing the minimum spanning tree, determine the growth tree according to the seed and the corresponding growth node.

[0044] Optionally, selecting a node from the minimum spanning tree as the root node includes:

[0045] For each node in the minimum spanning tree, determine the attribute characteristics of the node;

[0046] For each attribute characteristic of the node, determine the first quantity corresponding to the node in the minimum spanning tree with the same attribute characteristic value;

[0047] For each attribute characteristic of the node, determine the second quantity corresponding to the standard semantic target in the target semantic map with the same attribute characteristic value;

[0048] According to the first quantity and the second quantity corresponding to each attribute characteristic of the node, determine the particularity score of the node;

[0049] Select the node with the highest particularity score as the root node.

[0050] Optionally, the step of searching in the search tree for a standard semantic target that matches the branch node as the growth node of the growth tree according to the survival rule tree includes:

[0051] Calculate the circumscribed cube of the minimum spanning tree;

[0052] Calculate the distances from the root node to each vertex of the cube;

[0053] Take the maximum distance among the distances from the root node to each vertex of the cube as the search radius;

[0054] For each seed, based on the search radius, determine the corresponding sub-search tree from the search tree;

[0055] For each node of the minimum spanning tree, according to the survival rule corresponding to the node, search for the standard semantic target matching the node from each of the sub-search trees as the growth node of the growth tree.

[0056] Optionally, the step of searching for the standard semantic target matching the branch node from the search tree according to the survival rule tree as the growth node of the growth tree includes:

[0057] For each sub-search tree corresponding to each seed, perform the following steps:

[0058] Determine the survival rule of the branch to be grown from the survival rule tree;

[0059] Determine the current node in the sub-search tree;

[0060] In the sub-search tree, search for the nodes within the search radius of the current node of the search tree as the first candidate node set;

[0061] According to the survival rule of the branch to be grown, determine the final candidate nodes from the first candidate node set;

[0062] Take the final candidate nodes as the growth nodes of the growth tree;

[0063] If no final candidate nodes are determined, delete the growth tree corresponding to the seed.

[0064] Optionally, the step of determining the survival rule of the branch to be grown from the survival rule tree includes:

[0065] Determine the relative pose and estimated error variance of the node pair corresponding to the branch to be grown from the survival rule tree;

[0066] The step of determining the final candidate nodes from the first candidate node set according to the survival rule of the branch to be grown includes:

[0067] For each node in the first candidate node set, determine the relative pose between the node and the current node of the sub-search tree;

[0068] According to the relative pose between the node and the current node of the sub-search tree, the relative pose of the node pair corresponding to the branch to be grown, and the estimated error variance, determine whether the node matches the branch node;

[0069] If the node matches the branch node, determine the node as the final candidate node.

[0070] Optionally, the step of determining the survival rule of the branch to be grown from the survival rule tree includes:

[0071] Determine the relative pose, estimated error variance, and attribute characteristics of the branch nodes corresponding to the branch to be grown from the survival rule tree;

[0072] Determining the final candidate nodes from the first candidate node set according to the survival rule of the branch to be grown includes:

[0073] For each node in the first candidate node set, determine whether the attribute characteristics of the node are the same as those of the branch nodes;

[0074] If the attribute characteristics of the node are different from those of the branch nodes, delete the node from the first candidate node set to obtain a second candidate node set;

[0075] For each node in the second candidate node set, determine the relative pose of the node with respect to the current node of the sub-search tree;

[0076] Determine whether the node matches the branch node according to the relative pose of the node with respect to the current node of the sub-search tree, the relative pose of the node pair corresponding to the branch to be grown, and the estimated error variance;

[0077] If the node matches the branch node, determine the node as the final candidate node.

[0078] Optionally, determining whether the node matches the branch node according to the relative pose of the node with respect to the current node of the sub-search tree, the relative pose of the node pair corresponding to the branch to be grown, and the estimated error variance includes:

[0079] Determine the mismatch error according to the relative pose of the node with respect to the current node of the sub-search tree;

[0080] Determine the actual error variance according to the mismatch error;

[0081] Determine whether the node matches the branch node according to the actual error variance, the estimated error variance, and in combination with the chi-square test.

[0082] In a second aspect, the present application provides a global relocalization device, including:

[0083] A semantic target determination module for determining a target semantic map and a semantic map to be matched of a target device, where the target semantic map includes a number of standard semantic targets, and the semantic map to be matched includes a number of semantic targets to be matched;

[0084] The minimum spanning tree determination module is used to determine a minimum spanning tree based on an undirected complete graph according to the to-be-matched semantic map, and the nodes of the minimum spanning tree are the to-be-matched semantic targets;

[0085] The search tree determination module is used to construct a search tree based on the target semantic map, and the nodes of the search tree are the standard semantic targets;

[0086] The growth tree determination module is used to determine a growth tree that matches the minimum spanning tree from the constructed search tree;

[0087] The pose determination module is used to determine the pose of the target device according to the matching relationship between the standard semantic target corresponding to the growth tree and the to-be-matched semantic target corresponding to the minimum spanning tree.

[0088] Optionally, the minimum spanning tree determination module is specifically configured to:

[0089] For each to-be-matched semantic target, construct an undirected complete graph with the to-be-matched semantic target as a node;

[0090] For any two to-be-matched semantic targets, determine the weight of the edge between the nodes constructed by the two to-be-matched semantic targets according to the translation information of the two to-be-matched semantic targets;

[0091] Based on the weights of the edges in the undirected complete graph, determine the minimum spanning tree of the undirected complete graph.

[0092] Optionally, the growth tree determination module is specifically configured to:

[0093] Construct a survival rule tree based on the survival rule corresponding to each branch in the minimum spanning tree;

[0094] Based on the survival rule tree, determine a growth tree from the constructed search tree, and the growth tree satisfies the survival rule represented by the survival rule tree.

[0095] Optionally, the survival rule corresponding to each branch includes: the relative pose of the node pair corresponding to each branch, and the estimated error variance;

[0096] When constructing the survival rule tree based on the survival rule corresponding to each branch in the minimum spanning tree, the growth tree determination module is specifically configured to:

[0097] For each node pair corresponding to a branch in the minimum spanning tree, calculate the relative pose corresponding to the node pair;

[0098] For each node pair corresponding to a branch in the minimum spanning tree, calculate the estimated error variance corresponding to the node pair;

[0099] Generate the survival rule corresponding to each branch in the minimum spanning tree based on the relative pose and estimated error variance of the node pair corresponding to the branch.

[0100] Determine the survival rule tree according to the survival rules corresponding to each branch in the minimum spanning tree.

[0101] Optionally, the survival rule corresponding to each branch further includes: the attribute characteristics of each node;

[0102] When constructing the survival rule tree based on the survival rules corresponding to each branch in the minimum spanning tree, the growth tree determination module is specifically configured to:

[0103] For each node pair corresponding to a branch in the minimum spanning tree, calculate the relative pose corresponding to the node pair;

[0104] For each node pair corresponding to a branch in the minimum spanning tree, calculate the estimated error variance corresponding to the node pair;

[0105] Determine the attribute characteristics of each node corresponding to each branch in the minimum spanning tree;

[0106] Generate the survival rule corresponding to each branch based on the position of each branch in the minimum spanning tree, the relative pose of the node pair corresponding to the branch, the estimated error variance, and the attribute characteristics of each node;

[0107] Determine the survival rule tree according to the survival rules corresponding to each branch in the minimum spanning tree.

[0108] Optionally, when calculating the estimated error variance corresponding to each node pair corresponding to a branch in the minimum spanning tree, the growth tree determination module is specifically configured to:

[0109] Determine the observation error and distance error corresponding to the target device;

[0110] For each node pair corresponding to a branch in the minimum spanning tree, determine the distance between the target objects to be matched corresponding to the node pair;

[0111] Determine the estimated error variance corresponding to the node pair according to the observation error, the distance error, and the distance corresponding to the node pair.

[0112] Optionally, when determining the growth tree from the constructed search tree based on the survival rule tree and the growth tree satisfies the survival rules represented by the survival rule tree, the growth tree determination module is specifically configured to:

[0113] Select a node from the minimum spanning tree as the root node;

[0114] Select at least one standard semantic target with the same attribute characteristics as the root node from the target semantic map as the seed of the growth tree;

[0115] Starting from the root node, traverse each node of the minimum spanning tree upward;

[0116] For each node of the minimum spanning tree, perform the following steps:

[0117] Determine the to-be-grown branch corresponding to the node and the branch node that forms the to-be-grown branch with the node;

[0118] According to the survival rule tree, search in the search tree for a standard semantic target that matches the branch node as the growth node of the growth tree;

[0119] After traversing the minimum spanning tree, determine the growth tree according to the seed and the corresponding growth node.

[0120] Optionally, when the growth tree determination module selects a node as the root node from the minimum spanning tree, it is specifically used for:

[0121] For each node in the minimum spanning tree, determine the attribute characteristics of the node;

[0122] For each attribute characteristic of the node, determine the first quantity corresponding to the node in the minimum spanning tree with the same attribute characteristic value;

[0123] For each attribute characteristic of the node, determine the second quantity corresponding to the standard semantic target in the target semantic map with the same attribute characteristic value as the node;

[0124] According to the first quantity and the second quantity corresponding to each attribute characteristic of the node, determine the particularity score of the node;

[0125] Select the node with the highest particularity score as the root node.

[0126] Optionally, when the growth tree determination module searches in the search tree for a standard semantic target that matches the branch node as the growth node of the growth tree according to the survival rule tree, it is specifically used for:

[0127] Calculate the circumscribed cube of the minimum spanning tree;

[0128] Calculate the distances from the root node to each vertex of the cube;

[0129] Take the maximum distance among the distances from the root node to each vertex of the cube as the search radius;

[0130] For each seed, based on the search radius, determine a corresponding sub-search tree from the search tree;

[0131] For each node of the minimum spanning tree, according to the survival rule corresponding to the node, search for a standard semantic target that matches the node from each sub-search tree as the growth node of the growth tree.

[0132] Optionally, when the growth tree determination module searches for a standard semantic target that matches the branch node from the search tree according to the survival rule tree as the growth node of the growth tree, it is specifically configured to:

[0133] For each sub-search tree corresponding to a seed, perform the following steps:

[0134] Determine the survival rule of the branch to be grown from the survival rule tree;

[0135] Determine the current node in the sub-search tree;

[0136] In the sub-search tree, search for nodes within the search radius of the current node of the search tree as the first candidate node set;

[0137] Determine the final candidate nodes from the first candidate node set according to the survival rule of the branch to be grown;

[0138] Use the final candidate nodes as the growth nodes of the growth tree;

[0139] If no final candidate nodes are determined, delete the growth tree corresponding to the seed.

[0140] Optionally, when the growth tree determination module determines the survival rule of the branch to be grown from the survival rule tree, it is specifically configured to:

[0141] Determine the relative pose and estimated error variance of the node pair corresponding to the branch to be grown from the survival rule tree;

[0142] When the growth tree determination module determines the final candidate nodes from the first candidate node set according to the survival rule of the branch to be grown, it is specifically configured to:

[0143] For each node in the first candidate node set, determine the relative pose between the node and the current node of the sub-search tree;

[0144] Determine whether the node matches the branch node according to the relative pose between the node and the current node of the sub-search tree, the relative pose of the node pair corresponding to the branch to be grown, and the estimated error variance;

[0145] If the node matches the branch node, determine that the node is a final candidate node.

[0146] Optionally, when determining the survival rule of the branch to be grown from the survival rule tree, the growth tree determination module is specifically configured to:

[0147] Determine the relative pose, estimated error variance, and attribute characteristics of the branch node of the node pair corresponding to the branch to be grown from the survival rule tree;

[0148] When determining the final candidate node from the first candidate node set according to the survival rule of the branch to be grown, the growth tree determination module is specifically configured to:

[0149] For each node in the first candidate node set, determine whether the attribute characteristics of the node are the same as the attribute characteristics of the branch node;

[0150] If the attribute characteristics of the node are different from the attribute characteristics of the branch node, delete the node from the first candidate node set to obtain a second candidate node set;

[0151] For each node in the second candidate node set, determine the relative pose of the node with respect to the current node of the sub-search tree;

[0152] Determine whether the node matches the branch node according to the relative pose of the node with respect to the current node of the sub-search tree, the relative pose of the node pair corresponding to the branch to be grown, and the estimated error variance;

[0153] If the node matches the branch node, determine that the node is a final candidate node.

[0154] Optionally, when determining whether the node matches the branch node according to the relative pose of the node with respect to the current node of the sub-search tree, the relative pose of the node pair corresponding to the branch to be grown, and the estimated error variance, the growth tree determination module is specifically configured to:

[0155] Determine the mismatch error according to the relative pose of the node with respect to the current node of the sub-search tree;

[0156] Determine the actual error variance according to the mismatch error;

[0157] Determine whether the node matches the branch node according to the actual error variance, the estimated error variance, and the chi-square test.

[0158] In a third aspect, the present application provides an electronic device, including: a memory and a processor;

[0159] The memory is used to store program instructions;

[0160] The processor is used to call and execute the program instructions in the memory and execute the method described in any item of the first aspect.

[0161] In a fourth aspect, the present application provides a computer-readable storage medium in which a computer program is stored; when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0162] In a fifth aspect, the present application provides a computer program product, including: a computer program; when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0163] The present application provides a global relocalization method, apparatus, electronic device and storage medium. The global relocalization method includes: determining a target semantic map and a to-be-matched semantic map of a target device, where the target semantic map includes a plurality of standard semantic targets, and the to-be-matched semantic map includes a plurality of to-be-matched semantic targets; determining a minimum spanning tree based on an undirected complete graph based on the to-be-matched semantic map, where nodes of the minimum spanning tree are the to-be-matched semantic targets; constructing a search tree based on the target semantic map, where nodes of the search tree are the standard semantic targets; determining a growing tree matching the minimum spanning tree from the constructed search tree; and determining the pose of the target device according to a matching relationship between the standard semantic targets corresponding to the growing tree and the to-be-matched semantic targets corresponding to the minimum spanning tree. The problem of target matching is converted into the problem of the growth of the growing tree. The growth of the growing tree can select the best nodes based on all the standard semantic targets in the target semantic map and grow into a growing tree that best matches the minimum spanning tree. Target matching can be performed based on all the semantic targets in the target semantic map, truly realizing global relocalization and improving the global positioning efficiency of intelligent devices. Description of the Drawings

[0164] Figure 1 is a schematic diagram of an application scenario provided by the present application.

[0165] Figure 2 is a flowchart of a global relocalization method provided by an embodiment of the present application.

[0166] Figure 3 is a flowchart of determining a growing node provided by an embodiment of the present application.

[0167] Figure 4 is a structural diagram of a growing tree provided by an embodiment of the present application.

[0168] Figure 5Schematic diagram of a global relocalization device provided by an embodiment of the present application.

[0169] Figure 6 Schematic diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0170] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are some but not all of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0171] In addition, the term "and / or" in this document is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this document generally represents an "or" relationship between the associated objects before and after, unless otherwise specified.

[0172] The embodiments of the present application will be further described in detail below with reference to the accompanying drawings of the specification.

[0173] The basic symbol conventions in the present application are as follows:

[0174] Key symbols:

[0175] The target semantic map (Map) is represented by ;

[0176] The set of to-be-matched semantic targets (Landmarks set) in the to-be-matched semantic map is represented by ;

[0177] The attitude quantity in the pose representation is an orthogonal matrix (Orthogonal Matrix), represented by ; At the same time, it can also represent the Lie algebra represented by this rotation; the displacement / translation vector (Translation Vector) is represented by ;

[0178] Subscripts:

[0179] Unless otherwise specified, subscripts in the present application are generally used to represent the index or code of an individual. For example, all target element individuals are represented by subscripts. In the form of represents in element, Indicate in element Indicate generated by generated

[0180] Superscript:

[0181] Unless otherwise specified, superscripts are generally used in this invention to distinguish attribution or category. For example, a symbol shaped like represents the category below and quantity related to

[0182] The above conventions apply to most symbol representations in the embodiments of this application. They do not actually point to a certain meaning for the time being. The formal explanations are only for easy understanding. Different symbols are explained in the embodiments of this application.

[0183] Semantic objects in the semantic map are least affected by environmental factors such as lighting and seasons, so they have higher robustness. Moreover, semantic features contain a high amount of information and are sparsely distributed, which can greatly streamline the information expression of the map and reduce the required capacity of the map. Therefore, semantic maps stand out among many map types and are more suitable for large-scale positioning and mapping tasks, such as high-speed positioning and navigation, autonomous parking and other fields.

[0184] Existing global matching and relocalization techniques based on semantic objects often have the following problems.

[0185] First, existing relocalization techniques based on semantic maps require a certain initial position state as a condition. For example, the relocalization of outdoor high-precision maps depends on real-time kinematic (RTK) / Global Positioning System (GPS) to give the initial state, and then performs local search and matching for relocalization; the relocalization of semantic segmentation / target-level maps in indoor parking lots depends on ultra-wideband (UWB) positioning technology / initial reference parking points.

[0186] Second, semantic objects used in, such as triangulation, have less information. They cannot cover information such as categories and directions.

[0187] Third, the number of features used is small. Often only a small number of frames of observed objects can be adopted.

[0188] Fourth, the coverage range of semantic features used for matching is small. If semantic features covering a large range are included, they often cannot pass the verification in the geometric constraint link.

[0189] ​​Fifth, there is a need for descriptors, which are often used for clustering and similarity judgment, and are strongly related to the environment or description method. Moreover, their description methods require additional memory and computing resources.

[0190] Sixth, target matching can only handle short-term object association and tracking to a greater extent, and it is difficult to give an associated target solution set at the global level.

[0191] To solve the problems in the prior art, this application proposes a global relocalization method, aiming to structure the semantic features in the target map, match them with the semantic features in the map to be matched, and complete relocalization. The present invention does not require a given initial state and a quantified feature descriptor, can support a large number of structured semantic objects, and is a non-fully rigid match.

[0192] The global relocalization method provided by this application can be applied to aspects such as data association, power-on positioning of intelligent devices, loop detection, and relocalization after positioning anomalies. Correspondingly, the target semantic map can be a complete semantic map that has been established, and the map to be matched can be a semantic map constructed at the moment when relocalization is required. By matching the map to be matched with the target semantic map, the current pose can be relocalized.

[0193] Figure 1 This is a schematic diagram of an application scenario provided by this application. Taking the above-mentioned power-on positioning scenario as an example, the process of an intelligent device applying the method of this application for relocalization is described. As Figure 1 shown, the household intelligent sweeping robot 101 runs out of power and shuts down during a certain cleaning process. After charging and powering on, it is necessary to relocalize the current pose to determine the current location and then re-plan the cleaning route. Immediately after powering on, an image of the current moment (the map to be matched) is obtained, combined with the pre-constructed map of the indoor area (the target semantic map), and the global positioning method of this application is used to match the semantic targets in the two maps to determine the current pose of the robot. In this scenario, objects such as furniture in the room can be used as semantic targets in the map.

[0194] The specific implementation method of relocalizing the current pose by matching the map to be matched with the target semantic map can refer to the following embodiments.

[0195] Figure 2 This is a flowchart of a global relocalization method provided by an embodiment of this application. The method of this embodiment can be applied to intelligent devices (target devices) that need to perform relocalization, such as unmanned vehicles, drones, and household intelligent sweeping robots in the above scenarios. The method of this embodiment includes:

[0196] S201. Determine the target semantic map and the semantic map to be matched of the target device. The target semantic map includes several standard semantic targets, and the semantic map to be matched includes several semantic targets to be matched.

[0197] When determining the pose of the target device by feature matching, a semantic map of the current location of the target device (referred to as the semantic map to be matched in this application) and a relatively standard semantic map (which can accurately reflect the real environment) in the current environment (referred to as the target semantic map in this application) are required.

[0198] The target device can obtain the semantic map information of its current location through sensors (depth cameras, lidar, etc.) equipped on itself. Identify or objects in the surrounding environment are used as semantic targets in the semantic map to be matched (referred to as semantic targets to be matched in this application).

[0199] In different application scenarios, the selection of the target semantic map is not the same. For example, in Figure 1 the described power-on positioning scenario, there is generally an accurate map of the environment where the target device is located that has been constructed, and the constructed accurate map can be directly obtained as the target semantic map. Loop detection is generally applied to pose adjustment during continuous positioning to eliminate drift caused by errors; repositioning after an anomaly is generally applied to pose confirmation after positioning is lost. Therefore, in these two scenarios, the semantic map information obtained during the positioning process before re-determining the pose is relatively accurate and can be directly used as the target semantic map. In this application, the semantic targets in the target semantic map are called standard semantic targets.

[0200] After determining the target semantic map and the semantic map to be matched of the target device, the features (semantic targets) in the two maps can be matched to determine the pose corresponding to the semantic map to be matched.

[0201] The following steps are a method for semantic target matching provided by this application.

[0202] S202. Based on the semantic map to be matched, determine the minimum spanning tree based on the undirected complete graph. The nodes of the minimum spanning tree are the semantic targets to be matched.

[0203] By constructing the minimum spanning tree corresponding to the semantic map to be matched, the feature information of each semantic target to be matched in the semantic map to be matched can be transformed into the structural information of the minimum spanning tree.

[0204] S203. Construct a search tree based on the target semantic map. The nodes of the search tree are the standard semantic targets.

[0205] Use each standard semantic target in the target semantic map as a node to construct a search tree.

[0206] S204. Determine the growth tree that matches the minimum spanning tree from the constructed search tree.

[0207] In this application, "matching" means that the feature information of each semantic target is basically the same (it can be basically determined that it is different manifestations of the same target in two semantic maps). Then, the constructed growth tree matches the minimum spanning tree, that is, the feature information of the standard semantic target corresponding to each node of the growth tree is basically the same as the feature information of the semantic target to be matched corresponding to each node of the minimum spanning tree.

[0208] Specifically, it is possible to start traversing from the root node of the minimum spanning tree, and one by one determine the standard semantic target that matches it from the search tree as the nodes of the growth tree, and construct the growth tree.

[0209] Furthermore, the process of matching semantic targets in the two maps can be transformed into the process of the growth of the growth tree.

[0210] In some embodiments, the search tree can be a K-d tree. The structure of the K-d tree and related construction algorithms can optimize the structure of the search tree. When determining the growth tree from it, the number of operations is small, the operation pressure is small, and thus the search efficiency of each node is improved.

[0211] S205. Determine the pose of the target device according to the matching relationship between the standard semantic target corresponding to the growth tree and the semantic target to be matched corresponding to the minimum spanning tree.

[0212] According to the corresponding relationship between the nodes in the constructed growth tree and the nodes in the minimum spanning tree, the matching relationship between the standard semantic target and the semantic target to be matched can be determined, that is, which standard semantic target in the standard semantic map matches which semantic target to be matched in the semantic map to be matched can be determined.

[0213] In this way, the process of matching semantic targets between the target semantic map and the semantic map to be matched is completed.

[0214] After determining the matching relationship between the standard semantic target and the semantic target to be matched, the pose change of each semantic target to be matched relative to the standard semantic target it matches can be determined. Combining the pose of the standard semantic target in the target semantic map, the pose of the semantic target to be matched in the semantic map to be matched can be determined, that is, the current pose of the target device.

[0215] The global relocalization method provided by this embodiment includes: determining a target semantic map and a semantic map to be matched of a target device, where the target semantic map includes several standard semantic targets, and the semantic map to be matched includes several semantic targets to be matched; based on the semantic map to be matched, determining a minimum spanning tree based on an undirected complete graph, where the nodes of the minimum spanning tree are the semantic targets to be matched; constructing a search tree based on the target semantic map, where the nodes of the search tree are the standard semantic targets; determining a growing tree that matches the minimum spanning tree from the constructed search tree; and determining the pose of the target device according to the matching relationship between the standard semantic targets corresponding to the growing tree and the semantic targets to be matched corresponding to the minimum spanning tree. The problem of target matching is converted into the problem of the growth of the growing tree. The growth of the growing tree can select the best nodes based on all the standard semantic targets in the target semantic map and grow into a growing tree that best matches the minimum spanning tree. Target matching can be performed based on all the semantic targets in the target semantic map, truly realizing global relocalization and improving the global localization efficiency of intelligent devices.

[0216] In some specific implementation manners, the above method for determining a minimum spanning tree based on a complete graph based on the semantic map to be matched may include: for each semantic target to be matched, constructing an undirected complete graph with this semantic target to be matched as a node; for any two semantic targets to be matched, determining the weight of the edge between the nodes constructed by the two semantic targets to be matched according to the translation information of the two semantic targets to be matched; and determining the minimum spanning tree of the undirected complete graph based on the weights of the edges in the undirected complete graph.

[0217] That is, each semantic target to be matched in the semantic map to be matched is used as a node of the undirected complete graph, and the translation information of the two semantic targets to be matched is used as the weight of the edge corresponding to the nodes corresponding to the two semantic targets to be matched.

[0218] Among them, the translation information between two semantic targets to be matched can be obtained from the semantic map to be matched. For example, according to the relative positions of the two semantic targets to be matched in the semantic map to be matched, the translation vector corresponding to the two semantic targets to be matched can be determined as the translation information.

[0219] In some implementation manners, the method for determining the minimum spanning tree of the undirected complete graph based on the weights of the edges in the undirected complete graph may include: listing all the spanning trees corresponding to the undirected complete graph; calculating the sum of the weights corresponding to all the edges in each spanning tree, and the spanning tree with the minimum sum of weights is determined as the minimum spanning tree. Here, only one method for generating the minimum spanning tree is listed. In some other implementation manners, the Prim Algorithm, Kruskal algorithm, or other variants of the spanning tree algorithm can also be used to determine the minimum spanning tree. Details are not elaborated here one by one.

[0220] In a specific example, an undirected complete graph is constructed based on the pure spatial structure information of the set to be matched (the set of semantic targets to be matched). The weights of the edges in the complete graph can be calculated using only translational information , and the corresponding weight function can refer to the following formula (1):

[0221] (1);

[0222] where represents the weight of the edge between the nodes constructed by a pair of semantic targets and ; , respectively represent the translational vectors of the semantic targets and .

[0223] Finally, a minimum spanning tree is generated , and this tree has the minimum weight among any complete trees in the set to be matched , and its mathematical representation can refer to formula (2):

[0224] (2);

[0225] By constructing the minimum spanning tree, the measurement error introduced by the observation space scale range during the conversion from the semantic map to the graph can be minimized as much as possible. Moreover, matching based on the minimum spanning tree can avoid the disorder caused by random matching, avoid the rigid requirements introduced by graph matching, conform to the spatio-temporal characteristics of added observations, conform to the accumulativeness of errors, and can well satisfy the error propagation relationship.

[0226] In some embodiments, the above method of determining the growing tree that matches the minimum spanning tree from the constructed search tree may include: constructing a survival rule tree based on the survival rule corresponding to each branch in the minimum spanning tree; determining the growing tree from the constructed search tree based on the survival rule tree, where the growing tree satisfies the survival rule represented by the survival rule tree.

[0227] The characteristic information of each semantic target to be matched in the semantic map to be matched represented by the minimum spanning tree can be used as a constraint condition (survival rule) to construct the survival rule tree. Based on the survival rule represented by the survival rule tree, the construction of the growing tree can be guided, and finally a growing tree that completely conforms to the survival rule tree can be constructed. If there is no ambiguity between the target semantic map and the semantic map to be matched, a growing tree that completely conforms to the survival rule tree will ultimately be constructed from the search tree.

[0228] In some embodiments, the survival rule corresponding to each branch described above includes: the relative pose and the estimated error variance of the node pair corresponding to each branch. Correspondingly, constructing a survival rule tree based on the survival rule corresponding to each branch in the minimum spanning tree includes: calculating the relative pose corresponding to the node pair for each branch in the minimum spanning tree; calculating the estimated error variance corresponding to the node pair for each branch in the minimum spanning tree; generating the survival rule corresponding to each branch by combining the position of each branch in the minimum spanning tree with the relative pose and the estimated error variance of the node pair corresponding to the branch; and determining the survival rule tree according to the survival rule corresponding to each branch in the minimum spanning tree.

[0229] Mapping the survival rule corresponding to each branch in the minimum spanning tree to the branch can generate a survival rule tree corresponding to the minimum spanning tree. The survival rule corresponding to each branch actually includes the feature information of the two nodes corresponding to the branch. In some embodiments, the feature information of the two nodes can be represented by the relative pose between the two nodes and the estimated error variance between the two nodes. Among them, the relative pose between the two nodes can be calculated and determined by the relative positions of the two nodes; the estimated error variance between the two nodes refers to the variance relationship of the estimated positioning errors of the semantic targets corresponding to the two nodes, and can be estimated according to the error of the positioning system of the target device.

[0230] In some scenarios, there are also the attribute features of each semantic target in both the to-be-matched semantic map and the target semantic map. In this scenario, the survival rule corresponding to each branch can further include: the attribute feature of each node. Correspondingly, constructing a survival rule tree based on the survival rule corresponding to each branch in the minimum spanning tree can further include: determining the attribute feature of each node corresponding to each branch in the minimum spanning tree; generating the survival rule corresponding to each branch by combining the position of each branch in the minimum spanning tree with the relative pose, the estimated error variance, and the attribute feature of each node corresponding to the branch.

[0231] Among them, the attribute features of the semantic target can include but are not limited to the size, shape, category, color, etc. of the target. The corresponding specific attribute can be called an attribute feature value, such as the color is yellow and the shape is a cube, etc.

[0232] It can be understood that the more feature information is applied, the more detailed the constraint conditions of the survival rule are. When determining the growth tree from the constructed search tree based on the survival rule tree, the processing efficiency will be higher and the accuracy will also be higher.

[0233] Specifically, for each node pair corresponding to a branch in the minimum spanning tree, calculating the estimated error variance corresponding to the node pair may include: determining the observation error and distance error corresponding to the target device; for each node pair corresponding to a branch in the minimum spanning tree, determining the distance between the target objects to be matched corresponding to the node pair; and determining the estimated error variance corresponding to the node pair according to the observation error, distance error, and the distance corresponding to the node pair.

[0234] Due to certain errors in the sensors and processing algorithms equipped in the target device itself, when the target device generates a semantic map using the information collected by the sensors, there will also be errors in the position information of each semantic object in the map. In this application, this kind of error is called the observation error. In addition, as the running distance of the target device increases, error accumulation will also cause errors in the positioning results. In this application, this kind of error is called the distance error. By analyzing the historical positioning data of the target device, the observation error and distance error can be evaluated. The evaluated observation error and distance error can well estimate the positioning error of each semantic object in the semantic map, which is the estimated error. This embodiment uses the variance between the estimated errors of two semantic objects (estimated error variance) as one of the growth rules for the branches corresponding to the two semantic objects.

[0235] In one implementation, the formula (3) can be used to calculate the estimated error variance. The estimated error in this implementation can be used to calculate the translation error and rotation error. This formula describes the calculation method of the estimated error variance between two matching objects and represents the variance level between the errors of the two objects.

[0236] (3);

[0237] Wherein, is the estimated error variance between two matching objects ; is the distance between two matching objects , with the unit of m; is the uncertainty (distance error) generated by the system itself as the running distance increases; is the inherent observation error (observation error), which can be obtained by evaluating the sensor accuracy and calculation error of the system itself.

[0238] The distance error can be obtained according to the error propagation characteristics of the target device itself. In one implementation, the composed of rotation and translation. The observation error can be obtained according to the sensor accuracy and calculation error of the target device itself. In one implementation, the observation error is .

[0239] The solution of this embodiment can calculate and compare the errors in translation and rotation separately. In other implementation manners, a combined calculation and comparison method can also be used.

[0240] Corresponding to the above-described scenarios with attribute characteristics described later, the construction process of the survival rule tree can refer to the following content.

[0241] Traverse each pair of nodes in the minimum spanning tree , calculate and . The calculation method can refer to formula (4):

[0242] (4);

[0243] Traverse each pair of nodes in the minimum spanning tree , and use the above formula (3) to calculate the theoretical level of the estimated error variance between each node except the root node and the previous node .

[0244] Record the attribute characteristics corresponding to each node .

[0245] Correspond the growth rules corresponding to each pair of nodes , , , with the branches of the minimum spanning tree to form a survival rule for the branches . Then, combined with the index number , form the growth rules for each branch, thereby constructing the rule tree . In some embodiments, a slack variable can also be set for each survival rule to appropriately expand the range defined by the survival rule and prevent correct matching results from being determined as not conforming to the survival rule due to excessive errors.

[0246] Starting from the root of the tree and constructing survival rules for each branch one by one according to the minimum spanning tree , and finally form the survival rule tree. Therefore, each branch formed by adding a pair of nodes must satisfy that the branches already existing in the rule tree must contain one node of the newly constructed branch. The complete survival rule tree can be aligned and constructed according to formula (5):

[0247] (5);

[0248] Among them, represents the survival rule tree; represents that the tree contains branches composed of the node number set ; the symbol ' ' means updating the left end with its right end.

[0249] Thus, the construction of the survival rule tree can be completed.

[0250] In some embodiments, based on the above-mentioned survival rule tree, the method for determining a growth tree from the constructed search tree, where the growth tree satisfies the survival rule represented by the survival rule tree, includes: selecting a node from the minimum spanning tree as the root node; selecting at least one standard semantic target with the same attribute characteristics as the root node from the target semantic map as the seed of the growth tree; starting from the root node, traversing each node of the minimum spanning tree upward; for each node of the minimum spanning tree, perform the following steps: determining the to-be-grown branch corresponding to the node and the branch nodes that form the to-be-grown branch with the node; according to the survival rule tree, searching for the standard semantic target matching the branch node from the search tree as the growth node of the growth tree. After traversing the minimum spanning tree, determine the growth tree according to the seeds and the corresponding growth nodes.

[0251] In some implementation manners, a node can be randomly selected from the minimum spanning tree as the root node. After determining the root node, according to the attribute characteristics of the root node, a node matching it can be determined from the search tree as the seed of the growth tree. The seeds may be one or more. A growth tree can be generated based on each seed.

[0252] Select seeds from the target semantic map. The specific method can be to traverse the target semantic map all elements in it, and find the semantic targets whose category to which the root node belongs is exactly the same as the seeds seeds required for the forest, represented by a script indicating the set of seeds, which can be represented by formula (6):

[0253] (6);

[0254] Among them, represents the corresponding seed.

[0255] The implementation process of this embodiment can be referred to Figure 3Starting from the root node of the tree, traverse each node of the minimum spanning tree. When traversing each node of the minimum spanning tree, first determine the to-be-grown branch corresponding to the node according to the growth direction, and call the other node on the to-be-grown branch the branch node. The branch node corresponds to the next growth node to be grown on the growth tree. Search for the standard semantic target matching the branch node from the search tree corresponding to the target semantic map as the next growth node of the growth tree. If there is no branch consistent with the growth direction on this node, continue to traverse the next node. If there are multiple to-be-grown branches corresponding to this node, search for the growth nodes corresponding to each branch node one by one. After traversing the minimum spanning tree, each growth node grown based on the seed on the growth tree is determined, and the growth of the growth tree is completed.

[0256] For the sake of easy understanding, refer to Figure 4 for further illustration. Figure 4 In [Figure], a is the minimum spanning tree, and b is the growth tree. There are three nodes a1, a2, and a3 in a, and a1 is determined as the root node. b1 in b is one of the seeds found according to a1. The pre-determined traversal direction is shown by the dotted arrow in a. The growth process of the growth tree b is as follows: At the beginning, based on the two branch nodes a2 and a3 corresponding to the a1 node, search for the matching b2 and b3 from the search tree as two new growth nodes in b. Traverse downward to a2. There is no corresponding to-be-grown branch for a2, move to the next node a3. a3 is the last node of the minimum spanning tree, and the traversal ends. The traversal process is completed. The growth of b is completed.

[0257] In some other implementation manners, in order to improve the generation efficiency of the search tree, a special node in the minimum spanning tree can also be selected as the root node. In one implementation manner, the above-mentioned selecting a node from the minimum spanning tree as the root node of the tree includes: for each node in the minimum spanning tree, determine the attribute characteristics of the node; for each attribute characteristic of the node, determine the first quantity corresponding to the node in the minimum spanning tree with the same attribute characteristic value as the node; for each attribute characteristic of the node, determine the second quantity corresponding to the standard semantic target in the target semantic map with the same attribute characteristic value as the node; according to the first quantity and the second quantity corresponding to each attribute characteristic of the node, determine the particularity score of the node; select the node with the highest particularity score as the root node. That is, according to the distribution of the attribute characteristic values of each semantic target in the target semantic map and the to-be-matched semantic map, determine a root node from the nodes of the minimum spanning tree.

[0258] Among them, the particularity score can be a value negatively correlated with the first quantity and the second quantity. For example, the particularity score is negatively correlated with the sum of all the first quantities and all the second quantities. In this way, when the repetition of the respective attribute feature values of a certain node in the minimum target semantic map and the semantic map to be matched is relatively low, the higher its particularity score, the more special this node is. Since the number of semantic targets with the same attribute feature values is small, the difficulty of finding its matching semantic target is lower, and the matching efficiency is higher, and the growth process speed is faster.

[0259] In a specific implementation manner, the first quantity and the second quantity can be reflected by a normalized histogram. Specifically, for any attribute feature of any semantic target to be matched, determine the quantity information corresponding to this attribute feature in the first normalized histogram and the quantity information corresponding to this attribute feature in the second normalized histogram; for any semantic target to be matched, based on the quantity information corresponding to each attribute feature of this semantic target to be matched in the first normalized histogram and the quantity information corresponding to each attribute feature in the second normalized histogram, determine the particularity score of this semantic target to be matched.

[0260] Among them, for any semantic target to be matched, determining the particularity score of the semantic target to be matched according to the attribute features, the first normalized histogram, and the second normalized histogram of the semantic target to be matched includes: for any attribute feature of any semantic target to be matched, determine the quantity information corresponding to the attribute feature in the first normalized histogram and the quantity information corresponding to the attribute feature in the second normalized histogram; for any semantic target to be matched, based on the quantity information corresponding to each attribute feature of the semantic target to be matched in the first normalized histogram and the quantity information corresponding to each attribute feature in the second normalized histogram, determine the particularity score of the semantic target to be matched.

[0261] In some implementation manners, the process of selecting the root node is as follows.

[0262] Calculate the normalized histogram (histogram) of the category - count in the target semantic map, denoted by , denote the abstract attribute the attribution in the corresponding histogram value. Without changing the target map, this step (the calculation process of the histogram related to the target semantic map) only needs to be calculated once, and the calculation method can refer to formulas (7), (8), and (9):

[0263] (7);

[0264] (8);

[0265] (9);

[0266] Among them, The operator is the attribution of the object in the category ; '==' is for the same category matching judgment, 1 if the types are the same, 0 if the types are different; Indicates the attribution in the value of the attribution in; Indicates in the histogram, the value of the highest attribution, that is, the maximum value of the vector ; Indicates the object traversed by the subscript in ; Indicates the attribute feature, without loss of generality is an abstract attribute in, is the different differentiations corresponding to a certain abstract attribute in (for example, defining color as , and the color attributions are divided into red, green, and blue corresponding to respectively).

[0267] Calculate the normalized histogram (histogram) of the category - count in the map to be matched. and are calculated in the same way as formulas (8) (9), The calculation of only needs to change the source of the object, as shown in formula (10):

[0268] (10).

[0269] Define the speciality score, and represent it with the script letter . Calculate the scores of all objects in the set to be matched, as shown in formula (11):

[0270] (11).

[0271] Among them, is the speciality scores of all categories to which the target belongs; is the natural constant; is the attribute of m in the normalized histogram of; is the attribute of m Values in the normalized histogram.

[0272] Select the semantic object with the highest particularity score , and use to represent as the root of the minimum spanning tree, then satisfies formula (12):

[0273] (12);

[0274] where represents the particularity score vector of all objects in

[0275] In this embodiment, the calculation of the particularity score comprehensively considers the object to be matched and all semantic objects in the target map. Therefore, the selected root is the semantic object in the semantic map to be matched that is most likely to match the semantic target in the target map. Of course, in other implementation manners, weighted sums or other statistical ideas can also be considered in the calculation of this score.

[0276] In some embodiments, the above method of finding the standard semantic target matching the branch node from the search tree as the growth node of the growth tree based on the survival law tree may include: calculating the circumscribed cube of the minimum spanning tree; calculating the distances from the root node to the vertices of the cube; taking the maximum distance among the distances from the root node to the vertices of the cube as the search radius; for each seed, determining the corresponding sub-search tree from the search tree based on the search radius; for each node of the minimum spanning tree, finding the standard semantic target matching the node from each sub-search tree according to the survival law corresponding to the node as the growth node of the growth tree.

[0277] Constructing the sub-search tree of the search tree within a certain range corresponding to each seed can, to a certain extent, narrow the search range and improve the search efficiency. Among them, this range (search radius) can be determined according to the range of the minimum spanning tree. As long as it is ensured that the range of the sub-search tree is not smaller than the range of the minimum spanning tree, it can be ensured that the nodes in the sub-search tree are sufficient for the growth of the growth tree.

[0278] One way to determine the range of the sub-search tree is to determine the range of the minimum spanning tree according to the circumscribed cube of the minimum spanning tree and use it as the search radius.

[0279] Calculate the circumscribed cube of the minimum spanning tree (Oriented bounding box, OBB) , and using the regular bounding box to replace the irregular minimum spanning tree itself for calculation will be more efficient and simple.

[0280] Calculate the distances from the root of the tree to the vertices of the cube (8 vertices), and take the maximum distance among them as the search radius. , which can refer to formula (13):

[0281] (13);

[0282] Among them, represents the point represented by; represents each vertex of the circumscribed cube of the minimum spanning tree; is the root of the tree.

[0283] Construct a K-dimensional tree (KDtree) for the target semantic map, and combine as the search radius to construct a sub KDtree (search tree) that meets the size of the search radius range for each seed, denoted by .

[0284] Among them, the lowercase letter represents the number of sub KDtrees; and , where has been described above, and here it is emphasized that it is the target semantic map.

[0285] It can be seen from here that there may be multiple seeds with the same attribute characteristics as the root node of the tree. The number of seeds determines the number of growth trees that can be generated, and numerous growth trees form a forest. Therefore, the growth of the growth trees in this application can also be regarded as the growth of a forest.

[0286] In the above embodiment, according to the survival rule tree, a standard semantic target matching the branch node is searched from the search tree as the growth node of the growth tree. Specifically, it can include: for each sub-search tree corresponding to a seed, perform the following steps: determine the survival rule of the branch to be grown from the survival rule tree; determine the current node in the sub-search tree; search for the nodes within the search radius of the current node in the sub-search tree as the first candidate node set; determine the final candidate node from the first candidate node set according to the survival rule of the branch to be grown; use the final candidate node as the growth node of the growth tree; if no final candidate node is determined, delete the growth tree corresponding to the seed.

[0287] Corresponding to the above Figure 4The scenario will be described. For example, currently, when looking for the growth node corresponding to a2, take the determination process of the growth node corresponding to the seed b1 as an example. First, determine the survival rule of the branch a1a2 from the survival rule tree. Since the current traversal reaches the a1 node in a, and the branch a1a2 needs to grow, it can be determined that the current node in the sub-search tree is the latest growth node b1 in the growth tree b. Search for the nodes within the search radius of b1 in the sub-search tree as the first candidate node set. Determine the nodes that conform to the survival rule of the branch a1a2 from the set as the final candidate nodes, that is, the latest growth node b2.

[0288] Of course, since the seed b1 may not be the matching target of a1, it is very likely that no node conforming to the survival rule of the branch a1a2 can be found in the first candidate node set. This indicates that b is not the growth tree matching a, and this tree can be deleted. During the subsequent continuous growth process, no corresponding growth process will be performed on b, reducing the computational amount.

[0289] Among them, the specific method for determining the survival rule of the branch to be grown from the survival rule tree may include: determining the relative pose and the estimated error variance of the node pair corresponding to the branch to be grown from the survival rule tree. Correspondingly, determining the final candidate nodes from the first candidate node set according to the survival rule of the branch to be grown includes: for each node in the first candidate node set, determining the relative pose between the node and the current node of the sub-search tree; according to the relative pose between the node and the current node of the sub-search tree, the relative pose of the node pair corresponding to the branch to be grown, and the estimated error variance, determining whether the node matches the branch node; if the node matches the branch node, determining the node as the final candidate node.

[0290] The branch formed by the candidate node and the current node in the generation tree is called the candidate branch here. The solution of this embodiment is to determine whether the branch to be grown matches each candidate branch in each growth tree by comparing the pose relationship represented by the branch to be grown in the minimum spanning tree with the pose relationships of each candidate branch in each growth tree, and further determine whether the branch node matches each candidate node.

[0291] Specifically, it can be determined whether they match through the following method. Determine the mismatch error according to the relative pose between the node and the current node of the sub-search tree; determine the actual error variance according to the mismatch error; determine whether the node matches the branch node according to the actual error variance, the estimated error variance, and in combination with the chi-square test.

[0292] That is, a chi-square test is performed on the actual error variance between two targets in the target semantic map and the estimated error variance of the target device to verify whether the gap is within the preset range. If the gap is within the preset range, it can be determined that the two targets in the target semantic map match the two targets in the semantic map to be matched. The principle on which this solution is based is that if two targets in the target semantic map match two targets in the semantic map to be matched, then the error variance between these two targets in the two semantic maps must follow the law of the estimated error variance.

[0293] In a specific implementation, assuming that the targets are all represented in a certain fixed coordinate system in their respective sets, the actual error variance between the targets is calculated as shown in formula (14):

[0294] (14);

[0295] Where, represents a certain quantity in the relative pose, which can be a displacement quantity , or it can be an attitude quantity ; represents the map in the semantic target between a quantitative representation of a relative pose relationship; represents the set in the semantic target between the inverse element of a relative pose relationship, which can also be represented by , and there is no ambiguity between the two. is for in the object a guessed association; the binary operation ' ' represents the addition operation defined in the algebraic system where the variable is located; represents the set in the semantic target and in the semantic target mismatch error, which is and a way of measuring the difference between the two; is the actual error variance calculated from the mismatch error.

[0296] The chi-square test is a one-tailed test. Its significance level refers to the risk level that must be borne when rejecting the H0 hypothesis when the H0 hypothesis is true, also known as the probability level. The significance level is generally selected as 0.05 or 0.01, indicating that there is a 95% or 99% probability that H1 is in the rejection region. Therefore, in this case, being in the rejection region excludes H0, which will generally cause the chi-square to tend to the H0 hypothesis.

[0297] In contrast, during the implementation of the present invention, it does not favor H0 but favors H1. When the null hypothesis H0 is true, accepting the null hypothesis H0 bears a very low risk level. Therefore, in the chi-square test of this solution, the mean error is used as the hypothesized value, 'the two are related' is used as H0, and a left-tailed test is selected, that is, the significance level .

[0298] In this embodiment, the semantic target is a planar semantic target. When its pose is represented separately, the translation amount has 2 degrees of freedom, and the pose amount has 1 degree of freedom. By referring to the chi-square distribution table, the rejection regions under different degrees of freedom can be obtained. At a significance level of 0.9, and the corresponding critical values of the test are 0.211 and 0.016 respectively, as shown below:

[0299] (15).

[0300] Use the slack variable coefficients , to set the linear buffer of the hypothesis test to counteract errors from other unknown sources. In this embodiment, is used as the slack variable coefficient.

[0301] The magnitude of the (chi-square) between can be approximated by the calculation method of formula (16), which represents the difference between the actual error variance and the estimated error variance (rather than estimating the same distribution):

[0302] ) (16).

[0303] This embodiment is for the determination method in the scenario without attribute feature information. The determination method in the scenario with attribute feature information can refer to the next embodiment.

[0304] In the scenario with attribute feature information, the relative pose, estimated error variance, and attribute features of the node pair corresponding to the branch to be grown can be determined from the survival law tree. Correspondingly, determining the final candidate node from the first candidate node set according to the survival law of the branch to be grown includes: for each node in the first candidate node set, determining whether the attribute features of the node are the same as those of the branch node; if the attribute features of the node are different from those of the branch node, deleting the node from the first candidate node set to obtain a second candidate node set; for each node in the second candidate node set, determining the relative pose of the node and the current node of the sub-search tree; determining whether the node matches the branch node according to the relative pose of the node and the current node of the sub-search tree, the relative pose of the node pair corresponding to the branch to be grown, and the estimated error variance; if the node matches the branch node, determining the node as the final candidate node.

[0305] In the scenario of this embodiment, due to the addition of attribute features, the first candidate node set can be preliminarily screened, which can greatly simplify the processing process and improve the processing efficiency.

[0306] In a specific embodiment, the process of forest growth is as follows.

[0307] Traverse the survival law tree and each sub-search tree formed with a seed , and start growing from the seed. The goal of each seed is to gradually branch out and grow into a tree that satisfies the survival law, using to represent the growing tree grown from the seed.

[0308] The growth steps are as follows:

[0309] Start: Starting from the first node (root node) in the survival law tree, use to represent the current node in each loop; find the node (branch node) corresponding to form a branch with it, use to represent, that is , and obtain all its rules { , , , }.

[0310] First-level candidate node set: Use to represent the current node in the growing tree, and find all nodes in that satisfy the condition with the current node within a radius of , and use to represent, . Among them, represents the one that is The corresponding set of candidate nodes.

[0311] The second-level candidate node set: For all nodes in the set , exclude them according to the category , and update the set with the nodes that can satisfy formula (17), then the set evolves into the second-level candidate node set.

[0312] (17).

[0313] Final candidate nodes: For all the remaining nodes in the set , calculate using formula (15) and retain the result. According to the critical value obtained from the above table lookup, those that satisfy formula (18) become the final candidate nodes.

[0314] (18).

[0315] Up to this point, for a real physical space and a complete map, there is usually only one final candidate point. If an extreme situation occurs, that is, there are multiple points passing the chi-square test (theoretically, this does not exist unless the map is ambiguous), then sort according to the value and select the smallest as the final candidate node.

[0316] Kill all the trees that do not pass the survival law test , and update the forest, as shown in formula (19):

[0317] (19).

[0318] Repeat the growth step, and the last n surviving trees are all the intersections of the map to be matched and the target map. Finally, expand each surviving tree, and each tree can obtain a matching solution set represented as having elements 。

[0319] If there is , then there is no solution. If there is , then there are multiple solution sets, but there must be a correct solution among them. They can be solved separately.

[0320] For the obtained matching solution set , the nodes according to the survival law are Restore the corresponding relationship to obtain the one-to-one correspondence of semantic targets in the map to be matched and the target map. 。

[0321] Subsequently, an optimization problem can be constructed to solve the pose relationship between the maps determined by the tree. , which can be solved using formula (20):

[0322] (20).

[0323] Other algorithms such as ICP can also be used for the solution method.

[0324] For a map with large-scale deformation caused by the cumulative error of temporary map inference, only the n latest corresponding matches in the tree can be taken as the semantic observation target closest to the truth, discard the nodes that are too far in time or distance, and still construct and solve the problem according to formula (20), then the pose almost independent of the recursive error can be obtained.

[0325] Theoretically, the requirement for the semantic observation target is at least 1, and the number of observations required under the corresponding accuracy is related to the error level of the system. In the case of 6dof, if the pose information needs to be discarded to obtain the effect of accelerated solution, at least 3 non-collinear targets are required. In the case of discarding the pose information of the semantic target, the problem becomes formula (21):

[0326] (21).

[0327] Figure 5 The following is a schematic structural diagram of a global relocalization device provided by an embodiment of the present application. As Figure 5 shown, the global relocalization device 500 in this embodiment includes: a semantic target determination module 501, a minimum spanning tree determination module 502, a search tree determination module 503, a growth tree determination module 504, and a pose determination module 505.

[0328] The semantic target determination module 501 is used to determine the target semantic map and the map to be matched of the target device. The target semantic map includes several standard semantic targets, and the map to be matched includes several semantic targets to be matched;

[0329] The minimum spanning tree determination module 502 is used to determine a minimum spanning tree based on the undirected complete graph based on the map to be matched, and the nodes of the minimum spanning tree are the semantic targets to be matched;

[0330] The search tree determination module 503 is used to construct a search tree based on the target semantic map, and the nodes of the search tree are the standard semantic targets;

[0331] The growth tree determination module 504 is configured to determine a growth tree that matches the minimum spanning tree from the constructed search tree;

[0332] The pose determination module 505 is configured to determine the pose of the target device according to the matching relationship between the standard semantic target corresponding to the growth tree and the to-be-matched semantic target corresponding to the minimum spanning tree.

[0333] Optionally, the minimum spanning tree determination module 502 is specifically configured to:

[0334] For each to-be-matched semantic target, construct an undirected complete graph with the to-be-matched semantic target as a node;

[0335] For any two to-be-matched semantic targets, determine the weight of the edge between the nodes constructed by the two to-be-matched semantic targets according to the translation information of the two to-be-matched semantic targets;

[0336] Based on the weights of the edges in the undirected complete graph, determine the minimum spanning tree of the undirected complete graph.

[0337] Optionally, the growth tree determination module 504 is specifically configured to:

[0338] Based on the survival rule corresponding to each branch in the minimum spanning tree, construct a survival rule tree;

[0339] Based on the survival rule tree, determine a growth tree from the constructed search tree, and the growth tree satisfies the survival rule represented by the survival rule tree.

[0340] Optionally, the survival rule corresponding to each branch includes: the relative pose of the node pair corresponding to each branch, the estimated error variance;

[0341] When constructing the survival rule tree based on the survival rule corresponding to each branch in the minimum spanning tree, the growth tree determination module 504 is specifically configured to:

[0342] For each node pair corresponding to a branch in the minimum spanning tree, calculate the relative pose corresponding to the node pair;

[0343] For each node pair corresponding to a branch in the minimum spanning tree, calculate the estimated error variance corresponding to the node pair;

[0344] Generate the survival rule corresponding to each branch by combining the position of each branch in the minimum spanning tree with the relative pose and the estimated error variance of the node pair corresponding to the branch;

[0345] Determine the survival rule tree according to the survival rule corresponding to each branch in the minimum spanning tree.

[0346] Optionally, the survival rule corresponding to each branch further includes: the attribute characteristics of each node;

[0347] When constructing the survival rule tree based on the survival rules corresponding to each branch in the minimum spanning tree, the growth tree determination module 504 is specifically configured to:

[0348] For each pair of nodes corresponding to a branch in the minimum spanning tree, calculate the relative pose corresponding to the pair of nodes;

[0349] For each pair of nodes corresponding to a branch in the minimum spanning tree, calculate the estimated error variance corresponding to the pair of nodes;

[0350] Determine the attribute characteristics of each node corresponding to each branch in the minimum spanning tree;

[0351] Generate the survival rule corresponding to the branch by combining the position of each branch in the minimum spanning tree with the relative pose, estimated error variance, and attribute characteristics of each node corresponding to the pair of nodes of the branch;

[0352] Determine the survival rule tree according to the survival rules corresponding to each branch in the minimum spanning tree.

[0353] Optionally, when calculating the estimated error variance corresponding to each pair of nodes corresponding to a branch in the minimum spanning tree, the growth tree determination module 504 is specifically configured to:

[0354] Determine the observation error and distance error corresponding to the target device;

[0355] For each pair of nodes corresponding to a branch in the minimum spanning tree, determine the distance between the target objects to be matched corresponding to the pair of nodes;

[0356] Determine the estimated error variance corresponding to the pair of nodes according to the observation error, distance error, and the distance corresponding to the pair of nodes.

[0357] Optionally, when determining the growth tree from the constructed search tree based on the survival rule tree and the growth tree satisfies the survival rules represented by the survival rule tree, the growth tree determination module 504 is specifically configured to:

[0358] Select a node from the minimum spanning tree as the root node;

[0359] Select at least one standard semantic object with the same attribute characteristics as the root node from the target semantic map as the seed of the growth tree;

[0360] Starting from the root node, traverse each node of the minimum spanning tree upward;

[0361] For each node of the minimum spanning tree, perform the following steps:

[0362] Determine the branch to be grown corresponding to the node and the branch node that forms the branch to be grown with the node;

[0363] According to the survival rule tree, search for the standard semantic target that matches the branch node from the search tree as the growth node of the growth tree;

[0364] After traversing the minimum spanning tree, determine the growth tree according to the seed and the corresponding growth node.

[0365] Optionally, when the growth tree determination module 504 selects a node from the minimum spanning tree as the root node, it is specifically used for:

[0366] For each node in the minimum spanning tree, determine the attribute characteristics of the node;

[0367] For each attribute characteristic of the node, determine the first quantity corresponding to the node in the minimum spanning tree with the same attribute characteristic value;

[0368] For each attribute characteristic of the node, determine the second quantity corresponding to the standard semantic target with the same attribute characteristic value in the target semantic map;

[0369] According to the first quantity and the second quantity corresponding to each attribute characteristic of the node, determine the particularity score of the node;

[0370] Select the node with the highest particularity score as the root node.

[0371] Optionally, when the growth tree determination module 504 searches for the standard semantic target that matches the branch node from the search tree as the growth node of the growth tree according to the survival rule tree, it is specifically used for:

[0372] Calculate the circumscribed cube of the minimum spanning tree;

[0373] Calculate the distances from the root node to each vertex of the cube;

[0374] Take the maximum distance among the distances from the root node to each vertex of the cube as the search radius;

[0375] For each seed, based on the search radius, determine the corresponding sub-search tree from the search tree;

[0376] For each node of the minimum spanning tree, according to the survival rule corresponding to the node, search for the standard semantic target that matches the node from each sub-search tree as the growth node of the growth tree.

[0377] Optionally, when the growth tree determination module 504 searches for the standard semantic target that matches the branch node from the search tree as the growth node of the growth tree according to the survival rule tree, it is specifically used for:

[0378] For each sub-search tree corresponding to each seed, perform the following steps:

[0379] Determine the survival rule of the branch to be grown from the survival rule tree;

[0380] Determine the current node in the sub-search tree;

[0381] In the sub-search tree, search for the nodes within the search radius of the current node of the search tree as the first candidate node set;

[0382] Determine the final candidate nodes from the first candidate node set according to the survival rule of the branch to be grown;

[0383] Take the final candidate nodes as the growth nodes of the growth tree;

[0384] If no final candidate nodes are determined, delete the growth tree corresponding to the seed.

[0385] Optionally, when determining the survival rule of the branch to be grown from the survival rule tree, the growth tree determination module 504 is specifically used for:

[0386] Determine the relative pose and estimated error variance of the node pair corresponding to the branch to be grown from the survival rule tree;

[0387] When determining the final candidate nodes from the first candidate node set according to the survival rule of the branch to be grown, the growth tree determination module 504 is specifically used for:

[0388] For each node in the first candidate node set, determine the relative pose between the node and the current node of the sub-search tree;

[0389] Determine whether the node matches the branch node according to the relative pose between the node and the current node of the sub-search tree, the relative pose of the node pair corresponding to the branch to be grown, and the estimated error variance;

[0390] If the node matches the branch node, determine the node as the final candidate node.

[0391] Optionally, when determining the survival rule of the branch to be grown from the survival rule tree, the growth tree determination module 504 is specifically used for:

[0392] Determine the relative pose, estimated error variance, and attribute characteristics of the branch node of the node pair corresponding to the branch to be grown from the survival rule tree;

[0393] When determining the final candidate nodes from the first candidate node set according to the survival rule of the branch to be grown, the growth tree determination module 504 is specifically used for:

[0394] For each node in the first candidate node set, determine whether the attribute characteristics of the node are the same as those of the branch node;

[0395] If the attribute features of the node are different from those of the branch node, the node is deleted from the first candidate node set to obtain a second candidate node set;

[0396] For each node in the second candidate node set, determine the relative pose of the node with respect to the current node of the sub-search tree;

[0397] Based on the relative pose of the node with respect to the current node of the sub-search tree, the relative pose of the node pair corresponding to the branch to be grown, and the estimated error variance, determine whether the node matches the branch node;

[0398] If the node matches the branch node, determine the node as the final candidate node.

[0399] Optionally, when the growth tree determination module 504 determines whether the node matches the branch node based on the relative pose of the node with respect to the current node of the sub-search tree, the relative pose of the node pair corresponding to the branch to be grown, and the estimated error variance, it is specifically configured to:

[0400] Determine the mismatch error based on the relative pose of the node with respect to the current node of the sub-search tree;

[0401] Determine the actual error variance based on the mismatch error;

[0402] Based on the actual error variance, the estimated error variance, and in combination with the chi-square test, determine whether the node matches the branch node.

[0403] The device of this embodiment can be used to execute the method of any of the above embodiments, and its implementation principle and technical effects are similar, which will not be elaborated here.

[0404] Figure 6 The following is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 6 shown, the electronic device 600 of this embodiment may include: a memory 601 and a processor 602.

[0405] The memory 601 stores a computer program that can be loaded and executed by the processor 602 to execute the method in the above embodiments.

[0406] Among them, the processor 602 and the memory 601 are connected, such as through a bus.

[0407] Optionally, the electronic device 600 may further include a transceiver. It should be noted that in practical applications, the transceiver is not limited to one, and the structure of the electronic device 600 does not constitute a limitation to the embodiments of the present application.

[0408] The processor 602 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logical blocks, modules, and circuits described in connection with the disclosure of this application. The processor 602 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0409] The bus may include a path for transmitting information between the above components. The bus may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity, only a thick line is shown in the figure, but it does not mean that there is only one bus or one type of bus.

[0410] The memory 601 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or it may also be an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0411] The memory 601 is used to store the application program code for executing the solution of this application and is controlled by the processor 602 for execution. The processor 602 is used to execute the application program code stored in the memory 601 to implement the content shown in the foregoing method embodiments.

[0412] Among them, the electronic device includes, but is not limited to: mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. It can also be a server, etc. Figure 6 The electronic device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present application.

[0413] The electronic device of this embodiment can be used to execute the method of any of the above embodiments. The implementation principles and technical effects are similar and will not be elaborated here.

[0414] The present application also provides a computer-readable storage medium storing a computer program that can be loaded and executed by a processor to perform the method in the above embodiments.

[0415] Those of ordinary skill in the art can understand that all or part of the steps for implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps including the above method embodiments; and the foregoing storage medium includes: various media such as ROM, RAM, magnetic disks, or optical discs that can store program codes.

Claims

1. A global relocalization method, characterized in that, it includes: Determine the target semantic map and the semantic map to be matched of the target device. The target semantic map includes several standard semantic targets, and the semantic map to be matched includes several semantic targets to be matched; Based on the semantic map to be matched, determine the minimum spanning tree based on the undirected complete graph, and the nodes of the minimum spanning tree are the semantic targets to be matched; Construct a search tree based on the target semantic map, and the nodes of the search tree are the standard semantic targets; Determine the growth tree matching the minimum spanning tree from the constructed search tree; According to the matching relationship between the standard semantic targets corresponding to the growth tree and the semantic targets to be matched corresponding to the minimum spanning tree, determine the pose of the target device; The determining the growth tree matching the minimum spanning tree from the constructed search tree includes: Construct a survival rule tree based on the survival rules corresponding to each branch in the minimum spanning tree; Based on the survival rule tree, determine the growth tree from the constructed search tree, and the growth tree satisfies the survival rules represented by the survival rule tree; The survival rules corresponding to each branch include: the relative pose and the estimated error variance of the node pair corresponding to each branch; The constructing a survival rule tree based on the survival rules corresponding to each branch in the minimum spanning tree includes: For each node pair corresponding to a branch in the minimum spanning tree, calculate the relative pose corresponding to the node pair; For each node pair corresponding to a branch in the minimum spanning tree, calculate the estimated error variance corresponding to the node pair; Generate the survival rule corresponding to each branch by combining the position of each branch in the minimum spanning tree with the relative pose and the estimated error variance of the node pair corresponding to the branch; Determine the survival rule tree according to the survival rules corresponding to each branch in the minimum spanning tree.

2. The method according to claim 1, characterized in that, the determining the minimum spanning tree based on the complete graph based on the semantic map to be matched, and the nodes of the minimum spanning tree are the semantic targets to be matched, includes: For each semantic target to be matched, construct an undirected complete graph with the semantic target to be matched as a node; For any two semantic targets to be matched, determine the weight of the edge between the nodes constructed by the two semantic targets to be matched according to the translation information of the two semantic targets to be matched; Based on the weights of the edges in the undirected complete graph, determine the minimum spanning tree of the undirected complete graph.

3. The method according to claim 1 or 2, characterized in that, the survival rules corresponding to each branch further include: the attribute characteristics of each node; the constructing a survival rule tree based on the survival rules corresponding to each branch in the minimum spanning tree includes: For each node pair corresponding to a branch in the minimum spanning tree, calculate the relative pose corresponding to the node pair; For each node pair corresponding to a branch in the minimum spanning tree, calculate the estimated error variance corresponding to the node pair; Determine the attribute characteristics of each node corresponding to each branch in the minimum spanning tree; Generate the survival rule corresponding to each branch in the minimum spanning tree based on the relative pose, estimated error variance, and attribute features of each node of the node pair corresponding to the branch. Determine the survival rule tree according to the survival rule corresponding to each branch in the minimum spanning tree.

4. The method according to claim 3, wherein, calculating the estimated error variance corresponding to the node pair corresponding to each branch in the minimum spanning tree includes: determining the observation error and distance error corresponding to the target device; for each node pair corresponding to a branch in the minimum spanning tree, determining the distance between the target objects to be matched corresponding to the node pair; determining the estimated error variance corresponding to the node pair according to the observation error, the distance error, and the distance corresponding to the node pair.

5. The method according to claim 1 or 2, wherein, determining the growth tree from the constructed search tree based on the survival rule tree, where the growth tree satisfies the survival rule represented by the survival rule tree, includes: selecting a node from the minimum spanning tree as the root node; selecting at least one standard semantic target with the same attribute features as the root node from the target semantic map as the seed of the growth tree; starting from the root node, traversing each node of the minimum spanning tree upward; for each node of the minimum spanning tree, perform the following steps: determining the branch to be grown corresponding to the node and the branch node that forms the branch to be grown with the node; searching in the search tree for a standard semantic target that matches the branch node according to the survival rule tree as the growth node of the growth tree; after traversing the minimum spanning tree, determining the growth tree according to the seed and the corresponding growth node.

6. The method according to claim 5, wherein, selecting a node from the minimum spanning tree as the root node includes: for each node in the minimum spanning tree, determining the attribute features of the node; for each attribute feature of the node, determining the first quantity corresponding to the node with the same attribute feature value in the minimum spanning tree; for each attribute feature of the node, determining the second quantity corresponding to the standard semantic target with the same attribute feature value as the node in the target semantic map; determining the particularity score of the node according to the first quantity and the second quantity corresponding to each attribute feature of the node; selecting the node with the highest particularity score as the root node.

7. The method according to claim 5, wherein, searching in the search tree for a standard semantic target that matches the branch node according to the survival rule tree as the growth node of the growth tree includes: calculating the circumscribed cube of the minimum spanning tree; calculating the distances from the root node to each vertex of the cube; taking the maximum distance among the distances from the root node to each vertex of the cube as the search radius; for each seed, determining the corresponding sub-search tree from the search tree based on the search radius. For each node of the minimum spanning tree, according to the survival rule corresponding to the node, search for a standard semantic target that matches the node from each of the sub-search trees as the growth node of the growth tree.

8. The method according to claim 7, wherein, the searching for a standard semantic target that matches the branch node from the search tree according to the survival rule tree as the growth node of the growth tree includes: For each sub-search tree corresponding to a seed, perform the following steps: Determine the survival rule of the branch to be grown from the survival rule tree; Determine the current node in the sub-search tree; In the sub-search tree, search for the nodes within the search radius of the current node of the search tree as the first candidate node set; Determine the final candidate nodes from the first candidate node set according to the survival rule of the branch to be grown; Take the final candidate nodes as the growth nodes of the growth tree; If no final candidate nodes are determined, delete the growth tree corresponding to the seed.

9. The method according to claim 8, wherein, the determining the survival rule of the branch to be grown from the survival rule tree includes: Determine the relative pose and the estimated error variance of the node pair corresponding to the branch to be grown from the survival rule tree; the determining the final candidate nodes from the first candidate node set according to the survival rule of the branch to be grown includes: For each node in the first candidate node set, determine the relative pose between the node and the current node of the sub-search tree; Determine whether the node matches the branch node according to the relative pose between the node and the current node of the sub-search tree, the relative pose of the node pair corresponding to the branch to be grown, and the estimated error variance; If the node matches the branch node, determine the node as the final candidate node.

10. The method according to claim 8, wherein, the determining the survival rule of the branch to be grown from the survival rule tree includes: Determine the relative pose, the estimated error variance, and the attribute characteristics of the branch node of the node pair corresponding to the branch to be grown from the survival rule tree; the determining the final candidate nodes from the first candidate node set according to the survival rule of the branch to be grown includes: For each node in the first candidate node set, determine whether the attribute characteristics of the node and the attribute characteristics of the branch node are the same; If the attribute characteristics of the node and the attribute characteristics of the branch node are different, delete the node from the first candidate node set to obtain a second candidate node set; For each node in the second candidate node set, determine the relative pose between the node and the current node of the sub-search tree; Determine whether the node matches the branch node according to the relative pose between the node and the current node of the sub-search tree, the relative pose of the node pair corresponding to the branch to be grown, and the estimated error variance; If the node matches the branch node, determine the node as the final candidate node.

11. The method according to claim 9 or 10, characterized in that, determining whether the node matches the branch node according to the relative pose between the node and the current node of the sub-search tree, the relative pose of the node pair corresponding to the branch to be grown, and the estimated error variance includes: determining a mismatch error according to the relative pose between the node and the current node of the sub-search tree; determining an actual error variance according to the mismatch error; determining whether the node matches the branch node according to the actual error variance and the estimated error variance in combination with a chi-square test.

12. A global relocalization device, characterized in that, comprising: a semantic target determination module, configured to determine a target semantic map and a semantic map to be matched of a target device, wherein the target semantic map includes a plurality of standard semantic targets, and the semantic map to be matched includes a plurality of semantic targets to be matched; a minimum spanning tree determination module, configured to determine a minimum spanning tree based on an undirected complete graph based on the semantic map to be matched, and the nodes of the minimum spanning tree are the semantic targets to be matched; a search tree determination module, configured to construct a search tree based on the target semantic map, and the nodes of the search tree are the standard semantic targets; a growth tree determination module, configured to determine a growth tree that matches the minimum spanning tree from the constructed search tree; a pose determination module, configured to determine the pose of the target device according to the matching relationship between the standard semantic target corresponding to the growth tree and the semantic target to be matched corresponding to the minimum spanning tree; the growth tree determination module is specifically configured to construct a survival rule tree based on the survival rule corresponding to each branch in the minimum spanning tree; and determine a growth tree from the constructed search tree based on the survival rule tree, where the growth tree satisfies the survival rule represented by the survival rule tree; the survival rule corresponding to each branch includes: the relative pose of the node pair corresponding to each branch, and the estimated error variance; when constructing the survival rule tree based on the survival rule corresponding to each branch in the minimum spanning tree, the growth tree determination module is specifically configured to calculate the relative pose corresponding to the node pair for each branch corresponding to the node pair in the minimum spanning tree; calculate the estimated error variance corresponding to the node pair for each branch corresponding to the node pair in the minimum spanning tree; generate the survival rule corresponding to the branch by combining the position of each branch in the minimum spanning tree with the relative pose and the estimated error variance of the node pair corresponding to the branch; and determine the survival rule tree according to the survival rule corresponding to each branch in the minimum spanning tree.

13. An electronic device, characterized in that, comprising: a memory and a processor; the memory is configured to store program instructions; the processor is configured to call and execute the program instructions in the memory to execute the method according to any one of claims 1-11.

14. A computer-readable storage medium, characterized in that, A computer-readable storage medium stores a computer program; when the computer program is executed by a processor, the method according to any one of claims 1-11 is implemented.

15. A computer program product, characterized in that it includes: a computer program; when the computer program is executed by a processor, the method according to any one of claims 1-11 is implemented.

Images