Global Relocation Method, Device, Equipment and Storage Medium

Abstract

The present application provides a global relocalization method, apparatus, device and storage medium. By acquiring the observation features of the geographical space where the vehicle is located; comparing the observation features and the semantic features of each reference position in the semantic database to determine a target reference position that meets the preset positioning requirements from each reference position; determining the initial pose of the vehicle relative to the feature map according to the first semantic element, the observation features and the target reference position of the feature map, the initial pose including: an initial position and an initial attitude; determining the current pose of the vehicle within the global range according to the initial pose, the observation features and the feature map. Without relying on an external positioning source, the positioning registration calculation amount is reduced through three positioning steps with different granularities, and the technical problems of easy failure and large calculation amount existing in the existing relocalization technology are solved.

Description

Technical Field

[0001] The present application relates to the technical field of positioning, and in particular, to a global relocalization method, apparatus, device, and storage medium. Background Art

[0002] Positioning technology is one of the basic and core technologies for intelligent machine applications such as autonomous driving, providing position and attitude information, i.e., pose information, for intelligent machines or vehicles. Relocalization is the process of determining the pose information of a robot or vehicle globally without prior pose information. Currently, there are mainly two relocalization technologies, namely geometric relocalization and feature relocalization.

[0003] Geometric relocalization measures the distance or angle to a reference device at a known position, and then determines the position of the robot or vehicle through geometric calculations. However, due to the instability of the reference device signal, its relocalization / positioning may fail. For example, positioning based on GPS (Global Positioning System) may fail in scenarios such as inside buildings, tunnels, and under overpasses.

[0004] Feature relocalization determines the pose of a robot or vehicle by matching observed features with a pre-established feature map. Its disadvantages are: first, when factors such as the scene, environment, and occlusion cause the quality of the pre-established feature map or the observed features obtained in real time to decline, feature relocalization is prone to failure; second, searching for features that match the current observed features among all features globally is a computationally intensive process.

[0005] Therefore, there are technical problems in the existing relocalization technologies of being prone to failure and having a large amount of computation. Summary of the Invention

[0006] The present application provides a global relocalization method, apparatus, device, and storage medium to solve the technical problems of being prone to failure and having a large amount of computation in the existing relocalization technologies.

[0007] In a first aspect, the present application provides a global relocalization method, including:

[0008] Obtaining observed features of the geographical space where the vehicle is located;

[0009] Comparing the observed features with the semantic features of each reference position in the semantic database to determine a target reference position that meets the preset positioning requirements from each reference position;

[0010] Determining an initial pose of the vehicle relative to the feature map according to the first semantic element of the feature map, the observed features, and the target reference position, where the initial pose includes: an initial position and an initial attitude;

[0011] Determine the current pose of the vehicle within the global scope based on the initial pose, observed features, and feature map.

[0012] Optionally, obtain the observed features of the geographical space where the vehicle is located, including:

[0013] In the case of no historical pose information, obtain the observed features of the geographical space where the vehicle is located at multiple observation points, and splice the multiple observed features into a multi-frame observed feature.

[0014] In a possible design, the semantic database includes multiple reference feature sets, and the reference feature sets are used to characterize: the first relative position relationship between multiple semantic features observed at the same reference position and the reference position;

[0015] Compare the observed features with the semantic features at each reference position in the semantic database to determine the target reference position that meets the preset positioning requirements, including:

[0016] Use the semantic description model to semantically describe the observed features to determine the corresponding semantic descriptor, and the semantic descriptor is used to characterize the second relative position relationship between each second semantic element in the observed features and the current observation point, and the current observation point is the latest observation point;

[0017] Compare the semantic descriptor with each reference feature set to determine the comparison result;

[0018] If the comparison result meets the preset positioning requirements, determine the reference position corresponding to the reference feature set as the target reference position.

[0019] In a possible design, the first relative position relationship includes: the first distance between the first position corresponding to the semantic feature and the reference position, and the second relative position relationship includes: the second distance between the second position corresponding to the second semantic element and the current observation point;

[0020] Correspondingly, compare the semantic descriptor with each reference feature set to determine the comparison result, including:

[0021] Determine the registration distance between the semantic descriptor and each reference feature set according to the first distance and the second distance;

[0022] Take the registration distance as the comparison result;

[0023] Correspondingly, if the comparison result meets the preset positioning requirements, determine the reference position corresponding to the reference feature set as the target reference position, including:

[0024] Filter out the minimum value in the comparison results, and determine the reference position corresponding to the minimum value as the target reference position.

[0025] In a possible design, determining the registration distance between the semantic descriptor and each reference feature set according to the first distance and the second distance includes:

[0026] Calculating the difference between each first distance in the semantic descriptor and each second distance in the reference feature set;

[0027] Judging whether the types of each second semantic element and each semantic feature are the same;

[0028] If they are the same, multiplying the difference by the first correction coefficient to determine the first corrected difference;

[0029] If they are not the same, multiplying the difference by the second correction coefficient to determine the second corrected difference, where the first correction coefficient is less than the second correction coefficient;

[0030] Summing all the first corrected differences and all the second corrected differences to determine the registration distance.

[0031] In a possible design, determining the initial pose of the vehicle relative to the feature map according to the first semantic element of the feature map, the observed feature, and the target reference position includes:

[0032] Performing principal component analysis on the observed feature to determine the first transformation matrix;

[0033] Performing principal component analysis on the first semantic element to determine the second transformation matrix, where the first semantic element corresponds to the observed feature;

[0034] Determining the relative pose of the vehicle relative to the feature map according to the first transformation matrix and the inverse matrix corresponding to the second transformation matrix;

[0035] Determining the initial pose according to the relative pose.

[0036] In a possible design, the observed feature includes multiple second semantic elements, and the feature map includes multiple first semantic elements, where the first semantic element corresponds to the second semantic element;

[0037] Correspondingly, determining the current pose of the vehicle in the global range according to the initial pose, the observed feature, and the feature map includes:

[0038] Calculating the distance between each second semantic element and the corresponding first semantic element according to the initial pose;

[0039] Weighted-summing each distance with the observation confidence corresponding to the distance as the weight to determine the optimization reference index;

[0040] Adjusting the value of the initial pose according to the optimization reference index, and when the optimization reference index reaches the minimum value, taking the current value of the initial pose as the current pose.

[0041] In a possible design, the observed features include: multi-frame observed features formed by splicing a plurality of single-frame observed features. Obtaining the observed features of the geographical space where the vehicle is located includes:

[0042] Obtain a plurality of single-frame observed features;

[0043] Calculate the first relative pose of each single-frame observed feature with respect to the latest single-frame observed feature;

[0044] According to each first relative pose, convert each single-frame observed feature into a to-be-combined observed feature, where the to-be-combined observed feature is a single-frame observed feature described in the latest pose, and the latest pose is the pose corresponding to the latest single-frame observed feature;

[0045] Superimpose and combine all the to-be-combined observed features to determine the multi-frame observed features.

[0046] In a second aspect, the present application provides a global relocalization device, including:

[0047] An acquisition module, configured to acquire the observed features of the geographical space where the vehicle is located;

[0048] A dictionary localization layer module, configured to compare the observed features with the semantic features of each reference position in the semantic database, so as to determine a target reference position that meets the preset localization requirements from each of the reference positions;

[0049] A dimension localization layer module, configured to determine an initial pose of the vehicle relative to the feature map according to the first semantic element of the feature map, the observed features, and the target reference position, where the initial pose includes: an initial position and an initial attitude;

[0050] A high-precision localization layer module, configured to determine the current pose of the vehicle within the global range according to the initial pose, the observed features, and the feature map.

[0051] In a possible design, the semantic database includes a plurality of reference feature sets, and the reference feature sets are used to characterize: the first relative position relationship between a plurality of semantic features observed at the same reference position and the reference position;

[0052] The dictionary localization layer module is configured to:

[0053] Use a semantic description model to semantically describe the observed features to determine a corresponding semantic descriptor, where the semantic descriptor is used to characterize the second relative position relationship between each second semantic element in the observed features and the current observation point, and the current observation point is the latest observation point;

[0054] Compare the semantic descriptor with each reference feature set to determine a comparison result;

[0055] If the comparison result meets the preset positioning requirement, the reference position corresponding to the reference feature set is determined as the target reference position.

[0056] In a possible design, the first relative position relationship includes: the first distance between the first position corresponding to the semantic feature and the reference position, and the second relative position relationship includes: the second distance between the second position corresponding to the second semantic element and the current observation point;

[0057] Correspondingly, the dictionary positioning layer module is used for:

[0058] According to the first distance and the second distance, determine the registration distance between the semantic descriptor and each reference feature set;

[0059] Take the registration distance as the comparison result;

[0060] Correspondingly, if the comparison result meets the preset positioning requirement, determining the reference position corresponding to the reference feature set as the target reference position includes:

[0061] Select the minimum value in the comparison result, and determine the reference position corresponding to the minimum value as the target reference position.

[0062] In a possible design, the dictionary positioning layer module is used for:

[0063] Calculate the difference between each first distance in the semantic descriptor and each second distance in the reference feature set;

[0064] Judge whether the types of each second semantic element and each semantic feature are the same;

[0065] If they are the same, multiply the difference by the first correction coefficient to determine the first corrected difference;

[0066] If they are not the same, multiply the difference by the second correction coefficient to determine the second corrected difference, where the first correction coefficient is less than the second correction coefficient;

[0067] Sum all the first corrected differences and all the second corrected differences to determine the registration distance.

[0068] In a possible design, the dimension positioning layer module is used for:

[0069] Perform principal component analysis on the observed features to determine the first transformation matrix;

[0070] Perform principal component analysis on the first semantic element to determine the second transformation matrix, where the first semantic element corresponds to the observed features;

[0071] According to the first transformation matrix and the inverse matrix corresponding to the second transformation matrix, determine the relative pose of the vehicle with respect to the feature map;

[0072] Determine the initial pose according to the relative pose.

[0073] In a possible design, the observed features include multiple second semantic elements, and the feature map includes multiple first semantic elements, and the first semantic elements correspond to the second semantic elements;

[0074] Correspondingly, the high-precision positioning layer module is used for:

[0075] Calculate the distance between each second semantic element and the corresponding first semantic element according to the initial pose;

[0076] Perform a weighted sum of each distance with the observation confidence corresponding to the distance as the weight to determine the optimization reference index;

[0077] Adjust the value of the initial pose according to the optimization reference index. When the optimization reference index reaches the minimum value, use the current value of the initial pose as the current pose.

[0078] In a possible design, the observed features include: multi-frame observed features formed by splicing multiple single-frame observed features, and the acquisition module is used for:

[0079] Acquire multiple single-frame observed features;

[0080] Calculate the first relative pose of each single-frame observed feature and the latest single-frame observed feature;

[0081] Convert each single-frame observed feature into a to-be-combined observed feature according to each first relative pose. The to-be-combined observed feature is a single-frame observed feature described in the latest pose, and the latest pose is the pose corresponding to the latest single-frame observed feature;

[0082] Superimpose and combine all the to-be-combined observed features to determine the multi-frame observed features.

[0083] In a third aspect, the present application provides an electronic device, including:

[0084] A memory for storing program instructions;

[0085] A processor for calling and executing the program instructions in the memory and executing any possible global relocalization method provided in the first aspect.

[0086] In a fourth aspect, the present application provides a vehicle, including the electronic device provided in the third aspect.

[0087] In a fifth aspect, the present application provides a storage medium, in which a computer program is stored, and the computer program is used to execute any possible global relocalization method provided in the first aspect.

[0088] In a sixth aspect, the present application further provides a computer program product, including a computer program which, when executed by a processor, implements any possible global relocalization system method provided in the first aspect.

[0089] The present application provides a global relocalization method, apparatus, device and storage medium. By acquiring the observed features of the geographical space where the vehicle is located; comparing the observed features with the semantic features of each reference position in the semantic database to determine a target reference position that meets the preset positioning requirements from each reference position; determining the initial pose of the vehicle relative to the feature map according to the first semantic element of the feature map, the observed features and the target reference position, the initial pose including: an initial position and an initial attitude; and determining the current pose of the vehicle within the global range according to the initial pose, the observed features and the feature map. The technical problems of easy failure and large computational amount existing in the existing relocalization technology are solved. Without relying on an external positioning source, the computational amount of positioning registration is reduced through three positioning steps with different granularities, achieving the technical effects of improving the positioning accuracy and efficiency and effectively avoiding positioning failure. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0091] Figure 1 It is a data flow schematic diagram of a global relocalization provided for an embodiment of the present application;

[0092] Figure 2 It is a schematic diagram of a high-precision feature map provided by the present application;

[0093] Figure 3 It is a schematic diagram of the construction of a semantic descriptor provided by the present application;

[0094] Figure 4 It is a schematic flowchart of a global relocalization method provided for an embodiment of the present application;

[0095] Figure 5 It is a schematic flowchart of another global relocalization method provided for an implementation of the present application;

[0096] Figure 6 It is a schematic structural diagram of a global relocalization apparatus provided for an embodiment of the present application;

[0097] Figure 7 It is a schematic structural diagram of an electronic device provided by the present application.

[0098] Through the above-mentioned drawings, specific embodiments of the present application have been shown, and more detailed descriptions will be provided hereinafter. These drawings and textual descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. Detailed Embodiments

[0099] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are only some, rather than all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts, including but not limited to combinations of multiple embodiments, fall within the scope of protection of the present application.

[0100] The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims, and accompanying drawings of the present application are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented, for example, in an order different from those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0101] First, the definitions of the professional terms involved in the present application will be explained and introduced as follows:

[0102] Positioning technology: It is one of the basic and core technologies of intelligent machines (or vehicles) such as autonomous driving, providing position and attitude information, i.e., pose information, for intelligent machines (or vehicles). According to different positioning principles, positioning technology can be divided into geometric positioning, dead reckoning, and feature positioning.

[0103] Geometric positioning: It measures the distance or angle to a reference device with a known position, and then determines its own position through geometric calculations. It includes technologies such as GNSS (Global Navigation Satellite System), UWB (Ultra Wide Band, a wireless carrier communication technology), Bluetooth, and 5G, providing absolute positioning information. In intelligent vehicle applications, GNSS technology is the most widely used. GNSS positioning is based on satellite positioning technology, which is divided into single-point positioning, differential GPS (Global Positioning System) positioning, and RTK (Real-Time Kinematic) positioning. Among them, single-point positioning provides a positioning accuracy of 3 - 10 meters, differential GPS provides a positioning accuracy of 0.5 - 2 meters, and RTK GPS provides centimeter-level positioning accuracy. Its limitation is that it depends on positioning facilities and is affected by signal occlusion, reflection, etc., and fails in scenarios such as tunnels and elevated roads.

[0104] Dead Reckoning: It starts from the position at the previous moment and calculates the position at the next moment based on the motion data of sensors such as IMU (Inertial Measurement Unit) and wheel speed sensors, providing relative positioning information. Its limitation is that as the calculated distance increases, the positioning error will continuously accumulate and increase.

[0105] Feature positioning: It first obtains several features of the surrounding environment, such as base station ID (Identity Document), Wifi fingerprint, image, LiDAR (Laser Detection and Ranging) point cloud, etc. Then it matches the observed features with a pre-established feature map to determine the position in the feature map, providing absolute positioning information. The direct factors affecting feature positioning are the quantity, quality, and distinctiveness of the features. Its limitation is that when the feature observation is affected by factors such as the scene and environment, the positioning accuracy and stability will decrease.

[0106] Relocalization: It is the process of determining the pose information of a robot or vehicle within a global range without prior pose information. Relocalization is generally considered as a type or part of positioning. Generally, relocalization is divided into two situations. One is the initialization positioning when a robot or vehicle switches from a dormant state to a working state, that is, determining the initial pose at the start of the working state. The other is that due to temporary functional failures of sensors installed on the robot or vehicle, or the lack of real-time observed environmental features for some unpredictable reasons, resulting in the invalidation of the positioning pose, that is, unable to obtain an effective historical pose at the previous moment, thus requiring relocalization.

[0107] Global scope: A relative concept referring to the location range where all robots or vehicles may exist or operate. For example, the global map indoors or outdoors.

[0108] The inventors of the present application found that there are currently two main relocalization techniques, namely geometric relocalization and feature relocalization. Geometric relocalization measures the distance or angle to a reference device at a known location and then determines the position of the robot or vehicle through geometric calculations. However, due to the instability of the reference device signal, its relocalization / positioning may fail. For example, positioning based on GPS (Global Positioning System) may fail in scenarios such as inside buildings, tunnels, and under overpasses.

[0109] Feature relocalization determines the pose of the robot or vehicle by matching the observed features with a pre - established feature map. Its disadvantages are: First, when factors such as the scene, environment, and occlusion cause the quality of the pre - established feature map or the observed features obtained in real - time to decline, feature relocalization is prone to failure; Second, searching for features that match the current observed features among all features in the global scope is a computationally intensive process.

[0110] Therefore, there are technical problems in the existing relocalization techniques, such as being prone to failure and having a large computational amount.

[0111] To solve the above - mentioned technical problems, the inventive concept of the present application is:

[0112] To propose a global relocalization method based on semantic and geometric feature information. This method realizes fast global relocalization through three relocalization steps with different granularities. The three relocalization steps are the dictionary localization layer, the dimension localization layer, and the high - precision localization layer, ultimately realizing fast high - precision relocalization within the global scope.

[0113] The following uses specific embodiments to elaborate in detail on the technical solution of the present application and how the technical solution of the present application solves the above - mentioned technical problems. These specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0114] Figure 1 This is a data flow diagram of global relocalization provided for the embodiments of the present application. As Figure 1 shown, the IMU inertial measurement unit 101 and the wheel speed / vehicle speed sensor 102 transmit the detected data to the DR dead reckoning system 103, and the DR dead reckoning system 103 calculates the relative pose estimation parameters of the vehicle.

[0115] Meanwhile, the environmental observation sensor 104 acquires the three-dimensional observation features 105 of the surrounding environment frame by frame, stitches multiple three-dimensional observation features through the relative pose calculation parameters to obtain the three-dimensional multi-frame observation features 106, and then inputs the three-dimensional multi-frame observation features 106 into the positioning module 108 for multi-level positioning through the dictionary positioning layer, dimension positioning layer, and high-precision positioning layer, from the three-dimensional coordinates of 3 degrees of freedom with low precision to the initial pose of 6 degrees of freedom, and finally accurately locates to the current pose 109 of 6 degrees of freedom with high precision.

[0116] The following will introduce in detail how to implement the global relocalization method provided by this application.

[0117] It should be noted that, for the convenience of elaborating the solutions provided by each embodiment of this application, it is first necessary to clarify the definition of coordinates. The following coordinate system definitions are involved in this application:

[0118] (1) Define the world coordinate system W, which has a fixed relationship with the actual geographical location. For example, the Earth-Centered, Earth-Fixed (ECEF) coordinate system can be adopted.

[0119] (2) Define the vehicle body coordinate system B, which can also be called the vehicle body coordinate system for a vehicle or vehicle. It is fixed at a certain fixed position of the vehicle, such as the center of the rear axle of the vehicle. The vehicle pose is the 6Dof (Degree of Freedom) pose T of the vehicle body coordinate system in the world coordinate system. WB .

[0120] (3) Define the sensor coordinate system S, also called the observation coordinate system. The measurement data acquired by the sensor is all based on the sensor coordinate system. Usually, the sensor is fixed on the vehicle and moves as a rigid body with the vehicle. Therefore, there is a fixed conversion relationship between the sensor coordinate system and the vehicle body coordinate system, that is, the pose T BS , which is also called the extrinsic parameter.

[0121] It should also be noted that for the high-precision feature map required for the following embodiments, its establishment process is as follows:

[0122] Collect road information through high-precision positioning equipment and sensors, and establish a high-precision map. The high-precision map stores the road semantic information in the form of vector information. It includes, but is not limited to, storing the information of road surface objects such as lamp posts, road signs, and road edges on the road surface in the form of vector information such as points, lines, and surfaces, as well as road surface marking information such as solid lines, dashed lines, arrows, and text.

[0123] Figure 2 This is a schematic diagram of a high-precision feature map provided by this application. As Figure 2As shown, the semantic elements in the feature map, or the first semantic elements, include: indicating arrow 201, lamp post 202, stop line 203, solid lane line 204, dashed lane line 205, and zebra crossing 206.

[0124] For the semantic dictionary required for the following embodiments, it contains multiple key-value pairs within a global scope, and the format of the key-value pair is: "semantic descriptor - position". Among them, the semantic descriptor is a specific observation position, such as the mathematical description of all semantic information within a preset range around an observation point with three-dimensional coordinates (x, y, z). The establishment process of the semantic dictionary is as follows:

[0125] Figure 3 It is a schematic diagram for constructing a semantic descriptor provided by this application. As Figure 3 shown, in a high-precision feature map, the feature acquisition vehicle 301 travels along the center line of the lane and samples at a preset sampling interval (such as 10m). The position where each sampling occurs is the sampling point. Then, at each sampling point, all semantic elements within a preset effective observation range (such as an observation radius of 50m) around are extracted, such as: lamp post 302, lane line 303, stop line 304, zebra crossing 305, etc. It should be noted that the specific value of the observation radius can be consistent with the effective observation distance of the sensor.

[0126] Semantic elements include fixed markers in the environment. Markers include, but are not limited to, objects such as lamp posts, road signs, and curbs on the road surface, as well as road markings composed of solid lines, dashed lines, arrows, text, etc. The semantic elements are arranged in ascending order of the distance from the current sampling point as: semantic element 1, semantic element 2, semantic element 3... semantic element n. Then, in a predetermined format, taking the basic unit (hereinafter referred to as a tuple), a semantic descriptor is formed.

[0127] For example: basic unit = (semantic element category, distance between the semantic element and the current sampling point). As Figure 3 shown, the distance D between lamp post A and the current sampling point A The basic unit formed can be denoted as (302, D A ), and the distance D between lane line B and the current sampling point A The basic unit formed can be denoted as (301, D B ).

[0128] Among them, for the identification and classification category judgment of semantic elements, mature methods in the industry such as detection, segmentation, and recognition can be used. Currently, deep learning methods are generally used. Those skilled in the art can select specific recognition models according to the actual situation, and this application does not make limitations.

[0129] The semantic descriptor is a set composed of multiple basic units, that is, multiple tuples: {(type 1, distance 1), (type 2, distance 2), (type 3, distance 3)... (type n, distance n)}.

[0130] Then, the position of the current sampling point and the semantic descriptor are used together to form a dictionary entry: "semantic descriptor - position". Each dictionary entry: "semantic descriptor - position" formed by all sampling points in the high-precision feature map constitutes the semantic dictionary.

[0131] Figure 4 It is a schematic flowchart of a global relocalization method provided by an embodiment of the present application. As Figure 4 shown, the specific steps of this global relocalization method include:

[0132] S401. Obtain the observation features of the geographical space where the vehicle is located.

[0133] In this step, the observation features are three-dimensional multi-frame observation features.

[0134] Specifically, in the case of no historical pose information, three-dimensional observation features of the geographical space where the vehicle is located are obtained at multiple observation points, and multiple three-dimensional observation features are spliced into a three-dimensional multi-frame observation feature. Each three-dimensional observation feature collected each time is a single-frame feature. The three-dimensional observation features include: geometric shapes or identifiers represented by points, lines, and planes, feature points containing abstract information, points, lines, and planes containing semantic information.

[0135] In this embodiment, the process of obtaining the three-dimensional multi-frame observation features can be divided into the following steps:

[0136] Obtain multiple single-frame observation features;

[0137] Calculate the first relative pose of each single-frame observation feature with respect to the latest single-frame observation feature;

[0138] According to each first relative pose, convert each single-frame observation feature into a to-be-combined observation feature. The to-be-combined observation feature is a single-frame observation feature described in the latest pose, and the latest pose is the pose corresponding to the latest single-frame observation feature;

[0139] Superpose and combine all the to-be-combined observation features to determine the multi-frame observation feature.

[0140] Specifically, in the first step, on a vehicle or a robot, the sensors equipped include at least one of a camera, LiDAR (Laser Detection and Ranging), or other sensors, or a combination of these sensors. Through the above-mentioned various sensors or sensor combinations, observation data is obtained, and then steps such as detection, segmentation, and recognition are performed on the observation data to obtain various road semantic information, or semantic elements.

[0141] Through the extrinsic parameters of the sensor, that is, the pose T BS , these road semantic information, namely semantic elements, are transformed into the vehicle coordinate system. Specifically, let the observation of sensor A in the sensor coordinate be P A , and the extrinsic parameter of the sensor be T BS , then the observation P in the vehicle coordinate system B = T BS * P A .

[0142] It should be noted that since these road semantic information contain three-dimensional information, also known as three-dimensional semantic features, that is, the three-dimensional observation features contain semantic information and three-dimensional information.

[0143] Among them, semantics and three-dimensional are two aspects of the description of information. On the one hand, the obtained information is semantic, for example, the observed object is a street lamp. On the other hand, "three-dimensional" means that the object information is described in three dimensions (i.e., x, y, z), for example, the three-dimensional coordinates of the street lamp in the vehicle body coordinate system are obtained.

[0144] In the implementation process of positioning, the three-dimensional information is conducive to the direct implementation of operations such as registration, while semantics provides another dimension of information, which is conducive to reducing the occurrence of situations similar to no registration results, or ensuring the availability of registration results, or it can also be said that. That is, the three-dimensional observation features described in this application include at least four dimensions of information, ensuring that the registration result is non-empty.

[0145] In the second step, while obtaining the three-dimensional observation features, the relative pose estimation parameters of the vehicle are also obtained, such as the above-mentioned pose T WB .

[0146] Specifically, on a vehicle or a robot, sensors such as an IMU (Inertial Measurement Unit), a wheel speedometer, or a vehicle speedometer are equipped. The relative pose estimation parameters of the vehicle are obtained through DR (Dead Reckoning). The relative pose estimation parameters refer to the relative pose provided by DR from point a to point b. Specifically, in the DR coordinate system (this coordinate system is defined by DR, and generally, the pose when DR obtains the first frame of observation can be taken as the origin), the pose of point a is Ta and the pose of point b is T b , then the relative pose T between a and b can be obtained ba = T a - inverse *T b .

[0147] Furthermore, a possible implementation of the relative pose calculation parameter T a-inverse is the inverse matrix of the pose T a .

[0148] It should be noted that the first step and the second step can be carried out synchronously without order requirements.

[0149] In the third step, through the above relative pose calculation parameters, the three-dimensional observation features obtained from multiple observation points are spliced together to obtain three-dimensional multi-frame observation features.

[0150] Among them, the number of three-dimensional observation features corresponding to a three-dimensional multi-frame observation feature, that is, the number of frames, needs to be comprehensively determined by weighing the computing power of the processor and the computing efficiency of the algorithm or the processor. The greater the information density of each frame and the more frames, the greater the overall computing power required, which finally leads to a decrease in computing efficiency. Therefore, it needs to be determined by combining the computing power of the computing platform, hardware parameters, algorithm parameters, etc.

[0151] It should also be noted that the sampling time interval between two adjacent three-dimensional observation features can be not fixed, because the three-dimensional observation features are frame-sampled at spatial intervals. For example, a sampling frame is taken every 5m. At different driving speeds of the vehicle or robot, the time taken for the same spatial interval may be different. And the spatial interval of frame sampling needs to be determined by combining the sensor characteristics. Taking Lidar as an example, the scanning of the multi-line laser on the environment is relatively dense, so the sampling spatial interval of multiple frames can be appropriately reduced. Those skilled in the art can select according to the actual situation, and this application does not make a limitation.

[0152] In a possible design, this step specifically includes:

[0153] (1) The multiple single-frame road observation features obtained are F1, F2…Fn, and the poses provided by the corresponding DR dead reckoning are T1, T2…Tn. Among them, Fn is the latest single-frame road observation feature.

[0154] (2) Calculate the relative pose between each single-frame road observation feature and the latest single-frame road observation feature. For the i-th frame, the relative pose T ni = T i-inverse *Tn.

[0155] (3) Transform each single-frame road observation feature to the latest single-frame road observation feature through relative poses. For the i-th frame, its observation after transformation to the latest frame is nFi = T ni *Fi, where the 'n' in nFi represents taking the latest single-frame road observation feature as a reference.

[0156] (4) Directly accumulate all the transformed single-frame road observation features to obtain the stitched multi-frame road observation feature. That is, nF1 + nF2 + …… + nFn

[0157] Compared with the single-frame three-dimensional observation feature, the three-dimensional multi-frame observation feature contains a larger observation range and richer road semantic feature information.

[0158] Next, enter the three positioning steps with different granularities in the embodiments of the present application:

[0159] S402. Compare the observation feature and the semantic features of each reference position in the semantic database to determine a target reference position that meets the preset positioning requirements from each reference position.

[0160] In this step, compare the three-dimensional multi-frame observation feature and the semantic features in the semantic database (also called semantic dictionary) to determine a target reference position that meets the preset positioning requirements from each reference position. The semantic feature is various geometric shapes and / or identification information observed at the reference position, such as Figure 2 various semantic elements in it. The semantic database (also called semantic dictionary) includes multiple reference feature sets, and the reference feature set is used to represent: the first relative position relationship between multiple semantic features observed at the same reference position and the reference position.

[0161] In this embodiment, use a semantic description model to semantically describe the multi-frame observation feature to determine the corresponding semantic descriptor. The semantic descriptor is used to represent the second relative position relationship between each semantic element in the multi-frame observation feature and the current observation point, and the current observation point is the latest observation point;

[0162] Compare the semantic descriptor with each reference feature set to determine the comparison result;

[0163] If the comparison result meets the preset positioning requirements, determine the reference position corresponding to the reference feature set as the target reference position.

[0164] It should be noted that this step is the first step of multi - layer positioning with different granularities. Its function is as follows: Based on the preset reference positions in the semantic database (also known as the semantic dictionary), quickly find the reference position closest to the current position of the vehicle or robot, and obtain the low - precision position information of the vehicle or robot. This is more efficient than directly searching for high - precision positions from the beginning and can effectively reduce the amount of calculation. Therefore, this step can also be called the positioning of the dictionary positioning layer, which is used to determine the low - precision three - dimensional position coordinates.

[0165] S403. Determine the initial pose of the vehicle relative to the feature map according to the first semantic element of the feature map, the observed features, and the target reference position.

[0166] In this step, on the feature map (also known as the high - precision map), determine the initial pose of the vehicle according to the three - dimensional multi - frame observed features and the target reference position. The initial pose includes: the initial position and the initial attitude.

[0167] Specifically, use a preset analysis algorithm to determine the first transformation matrix according to the three - dimensional multi - frame observed features; use a preset analysis algorithm to determine the second transformation matrix according to the map features in the feature map. The map features correspond to the three - dimensional multi - frame observed features, that is, the map features are the observed forms of each semantic element corresponding to the three - dimensional multi - frame observed features collected in the feature map.

[0168] Determine the relative pose of the vehicle relative to the feature map according to the first transformation matrix and the inverse matrix corresponding to the second transformation matrix, and decompose the relative pose to determine the initial pose.

[0169] It should be noted that the role of the preset analysis algorithm is to convert each feature into a matrix - type mathematical expression in a mathematical analysis manner for subsequent calculations. The initial pose is a six - degree - of - freedom data, including the three - dimensional coordinates of the initial position: x, y, z. It also includes the rotation parameters on the corresponding coordinate axes, that is, the attitude information in the other three dimensions: roll, pitch, yaw. The information in these six dimensions together constitutes the initial pose.

[0170] It should also be noted that this step is based on the low - precision three - dimensional position coordinates located in S402, and uses mathematical analysis to obtain the three - dimensional rotation attitude corresponding to the low - precision three - dimensional position coordinates. Therefore, this step is also called the positioning of the dimension positioning layer, which is used to complete the six - degree - of - freedom or six - dimensional initial pose.

[0171] Through S402 and S403, the problem of no historical pose in relocalization is solved. The low - precision six - degree - of - freedom initial pose is used as the latest historical pose, and then the normal positioning process can be entered.

[0172] S404. Determine the current pose of the vehicle within the global scope based on the initial pose, the observed features, and the feature map.

[0173] In this step, using a preset registration algorithm, determine the current pose of the vehicle within the global scope based on the initial pose, multiple frames of observed features, and the feature map. The multiple frames of observed features are the observed features, including multiple second semantic elements, and the feature map includes multiple first semantic elements, with the first semantic elements corresponding to the second semantic elements.

[0174] Specifically, after obtaining the initial pose with six degrees of freedom, a normal feature positioning method can be adopted to perform feature positioning in combination with the previously established feature map to correct the initial pose, and through the high-precision feature map, improve the positioning accuracy.

[0175] Therefore, this step can also be referred to as the positioning of the high-precision positioning layer, whose function is to convert the low-precision initial pose into a high-precision current pose for output. In this way, through the positioning of three levels, namely the reference positioning layer, the dimension positioning layer, and the high-precision positioning layer, global fast high-precision repositioning from 3D to 6D and from low precision to high precision is achieved.

[0176] This embodiment provides a global repositioning method. By obtaining the observed features of the geographical space where the vehicle is located; comparing the observed features and the semantic features of each reference position in the semantic database to determine the target reference position that meets the preset positioning requirements from each reference position; determining the initial pose of the vehicle relative to the feature map according to the first semantic element of the feature map, the observed features, and the target reference position, where the initial pose includes: the initial position and the initial attitude; determining the current pose of the vehicle within the global scope based on the initial pose, the observed features, and the feature map. This solves the technical problems of easy failure and large computational amount existing in the existing repositioning technology. It realizes not relying on an external positioning source, reduces the positioning registration calculation amount through three different granularity positioning steps, achieves the technical effects of improving the positioning accuracy and efficiency, and effectively avoiding positioning failure.

[0177] For ease of understanding, a more detailed introduction to a possible implementation manner in S402 - S404 will be given below with another embodiment.

[0178] Figure 5 It is a schematic flowchart of another global repositioning method provided for the implementation of this application. As Figure 5 shown, the specific steps of this global repositioning method include:

[0179] S501. Obtain the observed features of the geographical space where the vehicle is located.

[0180] In this step, in the absence of historical pose information, three-dimensional observation features of the geographical space where the vehicle is located are obtained at multiple observation points, and multiple three-dimensional observation features are stitched together into a three-dimensional multi-frame observation feature.

[0181] For the specific noun explanations and principle explanations of this step, reference can be made to S401, which will not be elaborated here.

[0182] The following provides a more detailed description of the specific positioning steps of the dictionary positioning layer, which includes steps S502 to S504.

[0183] S502. Use the semantic description model to semantically describe the observation features to determine the corresponding semantic descriptors.

[0184] In this embodiment, the semantic database (also known as the semantic dictionary) includes multiple reference feature sets, and the reference feature sets are used to characterize: the first relative position relationship between multiple semantic features observed at the same reference position and the reference position, and the first relative position relationship includes: the first distance between the first position corresponding to the semantic feature and the reference position.

[0185] In this step, use the semantic description model to semantically describe the multi-frame observation features to determine the corresponding semantic descriptors. The semantic descriptors are used to characterize the second relative position relationship between each second semantic element in the multi-frame observation features and the current observation point. The second relative position relationship includes: the second distance between the second position corresponding to the second semantic element and the current observation point, and the current observation point is the latest observation point where the vehicle or robot collects observation data.

[0186] Specifically, the multi-frame observation features are in the carrier coordinate system B. Take all the observed semantic elements within a certain surrounding range range-distance, such as all the semantic elements within 50m. This range is consistent with the effective observation range range-distance used when generating the semantic database (also known as the semantic dictionary).

[0187] Arrange each semantic element in ascending order of the distance from the vehicle or robot as: semantic element 1, semantic element 2, semantic element 3... semantic element m.

[0188] Then arrange them in the format of: "semantic element category, distance between the semantic element and the current sampling point" as the basic tuple to form a semantic descriptor. That is, the format of the semantic descriptor of the multi-frame observation feature is: "(type 1, distance 1); (type 2, distance 2); (type 3, distance 3)... (type m, distance m)".

[0189] S503. Determine the registration distance between the semantic descriptor and each reference feature set according to the first distance and the second distance, and use the registration distance as the comparison result.

[0190] In this step, using a preset distance algorithm, based on the first distance and the second distance, determine the registration distance between the semantic descriptor and each reference feature set, and use the registration distance as the comparison result between the semantic descriptor and each reference feature set.

[0191] A possible implementation of the preset distance algorithm includes:

[0192] Calculate the difference between each first distance in the semantic descriptor and each second distance in the reference feature set;

[0193] Determine whether the types of each second semantic element and each semantic feature are the same;

[0194] If they are the same, multiply the difference by a first correction coefficient to determine a first corrected difference;

[0195] If they are not the same, multiply the difference by a second correction coefficient to determine a second corrected difference, where the first correction coefficient is less than the second correction coefficient;

[0196] Sum all the first corrected differences and all the second corrected differences to determine the registration distance.

[0197] For example, sequentially find the registration distances between the semantic descriptors of three-dimensional multi-frame observation features and all reference feature sets in the semantic dictionary. Assume that the semantic descriptor of the three-dimensional multi-frame observation features is "(Tc1, Dc1), (Tc2, Dc2), (Tc3, Dc3) …… (Tcm, Dcm)", and a certain reference feature set a in the semantic dictionary is "(Ta1, Da1), (Ta2, Da2), (Ta3, Da3) …… (Tan, Dan)", where T represents the type of semantic element, D represents the distance, the subscript c represents the semantic descriptor c of the three-dimensional multi-frame observation features, and a represents the reference feature set a. Then the method for obtaining the registration distance between the semantic descriptor of the three-dimensional multi-frame observation features and the reference feature set a is as follows:

[0198] (i) For the semantic element i in the semantic descriptor of the three-dimensional multi-frame observation features, its corresponding tuple is (Tci, Dci). Traverse all semantic elements of the reference feature set a, that is, all semantic features, and find the semantic element with the distance component Daj closest to (i.e., the smallest distance component) the distance component Dci of the semantic element i, denoted as j, and its corresponding tuple is (Taj, Daj).

[0199] (ii) The registration distance between the semantic descriptor c of the multi-frame observation features and the reference feature set a can be expressed by formula (1):

[0200] Dist(c, a) = SUM{(Dci - Daj) * TYPE-CHECK(Tci, Taj)} (1)

[0201] where SUM{*} is the summation function, which sums the values of ((Dci - Daj) * TYPE-CHECK(Tci, Taj)) for all semantic elements i of the semantic descriptors of multi-frame observation features;

[0202] TYPE-CHECK(Tci, Taj) is the type-checking function, which takes the value 1 when the types of semantic elements Tci and Taj are the same, and 2 when they are different. It should be noted that for the values when the types are the same and different, other numerical values can also be used, which are not limited in this application, and those skilled in the art can select according to actual needs.

[0203] S504. If the comparison result meets the preset positioning requirement, then determine the reference position corresponding to the reference feature set as the target reference position.

[0204] In this step, the preset positioning requirement includes: screening out the minimum value in the comparison result, and determining the reference position corresponding to the minimum value as the target reference position. Specifically, select the reference position in the reference feature set corresponding to the minimum registration distance as the current position of the vehicle or robot, and this position is a 3-degree-of-freedom position, that is, the coordinates of the position are x, y, and z.

[0205] It should be noted that when generating the semantic database, sampling points are taken at a certain interval (such as 10m) along the center line of the lane. The sampling interval determines the positioning accuracy of the reference positioning layer. The larger the sampling interval, the fewer the entries included in the reference, which is beneficial to the retrieval speed but reduces the positioning accuracy. The smaller the sampling interval, the more entries the reference contains, the slower the retrieval speed, but the positioning accuracy is improved. In practice, those skilled in the art need to make a trade-off according to the computational amount and accuracy requirements. Optionally, the sampling interval can be 5m - 20m, such as 10m, 15m, etc.

[0206] The following provides a more detailed introduction to the specific positioning steps of the dimension positioning layer:

[0207] S505. Determine the initial pose of the vehicle relative to the feature map according to the first semantic element of the feature map, the observation feature, and the target reference position.

[0208] In this embodiment, on the feature map (also known as the high-precision map), according to the multi-frame observation features and the target reference position, determine the initial pose of the vehicle, and use the preset analysis algorithm to determine the first transformation matrix according to the multi-frame observation features;

[0209] Using a preset analysis algorithm, determine a second transformation matrix according to the map features in the feature map, where the map features correspond to multi-frame observation features;

[0210] According to the first transformation matrix and the inverse matrix corresponding to the second transformation matrix, determine the relative pose of the vehicle with respect to the feature map, and decompose the relative pose to determine the initial pose.

[0211] Optionally, the preset analysis algorithm includes a principal component analysis algorithm.

[0212] That is, in a possible design, this step includes:

[0213] Perform principal component analysis on the observation features to determine the first transformation matrix;

[0214] Perform principal component analysis on the first semantic element to determine the second transformation matrix, where the first semantic element corresponds to the observation features;

[0215] According to the first transformation matrix and the inverse matrix corresponding to the second transformation matrix, determine the relative pose of the vehicle with respect to the feature map;

[0216] Determine the initial pose according to the relative pose.

[0217] Specifically, for example:

[0218] In the previous step, only the coordinates of the target reference position of the vehicle or robot were determined, namely x, y, and z. In the dimension positioning layer, determine the parameter values of the other three dimensions, namely roll, pitch, and yaw.

[0219] (a) Perform a PCA (Principal Component Analysis) transformation on the multi-frame observation features. The multi-frame observation features are in the carrier coordinate system B. For the multi-frame observation features, take all semantic elements within a certain range range-distance around, and perform a PCA transformation to obtain a transformation matrix Tcn.

[0220] It should be noted that the specific steps of the PCA transformation are well-known to those skilled in the art or can be retrieved, and for the sake of brevity, they will not be elaborated here.

[0221] (b) Perform a PCA transformation on the map features corresponding to the multi-frame observation features in the feature map. In S503, the coordinates of the target reference position of the vehicle or robot were determined, namely x, y, and z. Take all map features within a certain range range-distance around this position and perform a PCA transformation to obtain a transformation matrix Tmn.

[0222] (c) Calculate the relative pose Tcm of the vehicle or robot with respect to the feature map, as shown in Equation (2):

[0223] Tcm = Tcn * Tmn-inverse (2)

[0224] where Tmn-inverse is the inverse matrix of the relative pose Tmn. Decompose the relative pose Tcm to obtain the initial pose T of the vehicle or robot with six degrees of freedom WB .

[0225] The following provides a more detailed introduction to the specific positioning steps of the high-precision positioning layer:

[0226] S506. Calculate the distance between each second semantic element and the corresponding first semantic element according to the initial pose.

[0227] In this embodiment, according to the initial pose, calculate the reprojection error, i.e., the distance, between each semantic element and the corresponding first semantic element. The multi-frame observation features, i.e., the observation features, include multiple second semantic elements, and the feature map includes multiple first semantic elements. The first semantic elements correspond to the second semantic elements in the observation features.

[0228] In this step, assume that all the second semantic elements in the multi-frame observation features include: B P1, B P2, B P5... B Pn, and the first semantic elements corresponding to each second semantic element in the feature map are W P1, W P2, W P5... W Pn. The reprojection error ε B between any semantic element W Pi and the first semantic element i Pi in the feature map can be expressed by Equation (3):

[0229] ε i = DIST(T WB * B Pi, W Pi) (3)

[0230] where DIST(*) represents the reprojection error between the observed element B Pi and the feature element W Pi in the feature map, and T WB is the initial pose.

[0231] S507. Weighted sum is performed on each distance with the observation confidence corresponding to the distance as the weight to determine the optimized reference index.

[0232] In this step, the observation confidence refers to the detection accuracy of the observed features. In actual engineering practice, the quality of different semantic information is different. For example, the lane lines on the road surface are white regular shapes, and their detection accuracy is relatively high. Some elements in the air, such as poles, signs, lights, etc., have relatively low detection accuracy. The detection accuracy is expressed by the precision-recall rate in practice. Therefore, in the relocalization method provided in this application, the precision-recall rate is used as the value of the confidence.

[0233] In this embodiment, when the optimized reference index is described in mathematical language, it can also be called a cost function.

[0234] Specifically, the cost function F(T WB ) can be expressed by formula (4):

[0235] F(T WB ) = SUM(ρi * ε i ) (4)

[0236] Among them, each reprojection error ε i corresponds to the confidence ρ i .

[0237] S508. According to the optimized reference index, adjust the value of the initial pose. When the optimized reference index reaches the minimum value, take the current value of the initial pose as the current pose.

[0238] In this step, using a preset optimization algorithm, the initial pose is optimized according to the cost function to determine the candidate pose.

[0239] The optimization solution process can be expressed by formula (5):

[0240] T WB-calculate = argmin(F(T WB )) (5)

[0241] Among them, argmin(*) represents finding the optimal T WB that minimizes the value of the cost function.

[0242] It should be noted that in the optimization solution process, it is necessary to repeatedly replace the old candidate pose with the new candidate pose T WB-calculate , and recalculate the standby function with the new candidate pose T WB-calculate , that is, replace T WB in formula (4) with T WB-calculate .

[0243] When the cost function reaches its minimum value, the current latest candidate pose is determined as the current pose. The high-precision registration and positioning of the low-precision initial pose is completed, and the high-precision current pose is obtained.

[0244] This embodiment provides a global relocalization method, which includes obtaining the observation features of the geographical space where the vehicle is located; comparing the observation features and the semantic features of each reference position in the semantic database to determine a target reference position that meets the preset positioning requirements from each reference position; determining the initial pose of the vehicle relative to the feature map according to the first semantic element of the feature map, the observation features, and the target reference position, where the initial pose includes: an initial position and an initial attitude; and determining the current pose of the vehicle within the global range according to the initial pose, the observation features, and the feature map. This solves the technical problems of easy failure and large computational amount existing in the existing relocalization technology. It realizes not relying on an external positioning source, reduces the positioning registration calculation amount through three positioning steps with different granularities, achieves the technical effects of improving the positioning accuracy and efficiency, and effectively avoiding positioning failure.

[0245] Figure 6 It is a schematic structural diagram of a global relocalization device provided by an embodiment of the present application. The global relocalization device 600 can be implemented by software, hardware, or a combination of both.

[0246] As Figure 6 shown, the global relocalization device 600 includes:

[0247] An acquisition module 601, configured to acquire the observation features of the geographical space where the vehicle is located;

[0248] A dictionary positioning layer module 602, configured to compare the observation features and the semantic features of each reference position in the semantic database to determine a target reference position that meets the preset positioning requirements from each of the reference positions;

[0249] A dimension positioning layer module 603, configured to determine the initial pose of the vehicle relative to the feature map according to the first semantic element of the feature map, the observation features, and the target reference position, where the initial pose includes: an initial position and an initial attitude;

[0250] A high-precision positioning layer module 604, configured to determine the current pose of the vehicle within the global range according to the initial pose, the observation features, and the feature map.

[0251] In a possible design, the semantic database includes multiple reference feature sets, and the reference feature set is used to represent: the first relative position relationship between multiple semantic features observed at the same reference position and the reference position;

[0252] The dictionary positioning layer module 602 is configured to:

[0253] Use a semantic description model to semantically describe the observed features to determine the corresponding semantic descriptors, where the semantic descriptors are used to characterize the second relative position relationship between each second semantic element in the observed features and the current observation point, and the current observation point is the latest observation point;

[0254] Compare the semantic descriptors with each reference feature set to determine the comparison result;

[0255] If the comparison result meets the preset positioning requirements, determine the reference position corresponding to the reference feature set as the target reference position.

[0256] In a possible design, the first relative position relationship includes: the first distance between the first position corresponding to the semantic feature and the reference position, and the second relative position relationship includes: the second distance between the second position corresponding to the second semantic element and the current observation point;

[0257] Correspondingly, the dictionary positioning layer module 602 is used for:

[0258] Determine the registration distance between the semantic descriptor and each reference feature set according to the first distance and the second distance;

[0259] Take the registration distance as the comparison result;

[0260] Correspondingly, if the comparison result meets the preset positioning requirements, determining the reference position corresponding to the reference feature set as the target reference position includes:

[0261] Filter out the minimum value in the comparison result, and determine the reference position corresponding to the minimum value as the target reference position.

[0262] In a possible design, the dictionary positioning layer module 602 is used for:

[0263] Calculate the difference between each first distance in the semantic descriptor and each second distance in the reference feature set;

[0264] Judge whether the types of each second semantic element and each semantic feature are the same;

[0265] If they are the same, multiply the difference by the first correction coefficient to determine the first corrected difference;

[0266] If they are not the same, multiply the difference by the second correction coefficient to determine the second corrected difference, where the first correction coefficient is less than the second correction coefficient;

[0267] Sum all the first corrected differences and all the second corrected differences to determine the registration distance.

[0268] In a possible design, the dimension positioning layer module 603 is used for:

[0269] Perform principal component analysis on the observed features to determine the first transformation matrix;

[0270] Perform principal component analysis on the first semantic elements to determine the second transformation matrix, where the first semantic elements correspond to the observed features;

[0271] Determine the relative pose of the vehicle with respect to the feature map according to the first transformation matrix and the inverse matrix corresponding to the second transformation matrix;

[0272] Determine the initial pose according to the relative pose.

[0273] In a possible design, the observed features include multiple second semantic elements, and the feature map includes multiple first semantic elements, where the first semantic elements correspond to the second semantic elements;

[0274] Correspondingly, the high-precision positioning layer module 604 is used for:

[0275] Calculate the distance between each second semantic element and the corresponding first semantic element according to the initial pose;

[0276] Perform weighted summation on each distance with the observation confidence corresponding to the distance as the weight to determine the optimization reference index;

[0277] Adjust the value of the initial pose according to the optimization reference index. When the optimization reference index reaches the minimum value, use the current value of the initial pose as the current pose.

[0278] In a possible design, the observed features include: multi-frame observed features formed by splicing multiple single-frame observed features. The acquisition module 601 is used for:

[0279] Acquire multiple single-frame observed features;

[0280] Calculate the first relative pose of each single-frame observed feature with respect to the latest single-frame observed feature;

[0281] Convert each single-frame observed feature into a to-be-combined observed feature according to each first relative pose. The to-be-combined observed feature is a single-frame observed feature described in the latest pose, and the latest pose is the pose corresponding to the latest single-frame observed feature;

[0282] Superimpose and combine all the to-be-combined observed features to determine the multi-frame observed features.

[0283] It should be noted that Figure 6 The device provided in the illustrated embodiment can execute the method provided in any of the above method embodiments. The specific implementation principles, technical features, explanations of professional terms, and technical effects are similar, and will not be elaborated here.

[0284] Figure 7 This is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 7 shown, the electronic device 700 may include: at least one processor 701 and a memory 702. Figure 7 The electronic device shown takes one processor as an example.

[0285] The memory 702 is used to store programs. Specifically, the program may include program code, and the program code includes computer operation instructions.

[0286] The memory 702 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.

[0287] The processor 701 is used to execute the computer execution instructions stored in the memory 702 to implement the methods described in the above method embodiments.

[0288] Among them, the processor 701 may be a central processing unit (abbreviated as CPU), or an application specific integrated circuit (abbreviated as ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0289] Optionally, the memory 702 may be either independent or integrated with the processor 701. When the memory 702 is a device independent of the processor 701, the electronic device 700 may further include:

[0290] A bus 703 for connecting the processor 701 and the memory 702. The bus may be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc., but it does not mean that there is only one bus or one type of bus.

[0291] Optionally, in a specific implementation, if the memory 702 and the processor 701 are integrated on a chip, the memory 702 and the processor 701 may communicate through an internal interface.

[0292] In a possible design, the processor 701 and the memory 702 are integrated into any vehicle information interaction terminal such as an in-vehicle central computing platform architecture, or a central super brain, or a central computer, or a central domain controller, or an integrated ECU, or a driving brain, or an SPB, or a car machine, or a DHU, or an IHU, or an IVI (In-Vehicle Infotainment).

[0293] Among them, SPB (super brain) is a central domain controller defined as the brain of the vehicle

[0294] IHU (Infotainment Head Unit) information entertainment host refers to an in-vehicle integrated information processing device that uses a dedicated in-vehicle central processor and is based on the vehicle body bus system and Internet services. It can implement a series of applications including 3D navigation, real-time traffic conditions, IPTV, assisted driving, fault detection, vehicle information, body control, mobile office, wireless communication, online-based entertainment functions, and TSP (Telematics Service Provider), etc., greatly improving the vehicle's electronic, networked, and intelligent levels.

[0295] DHU (Driver Head Unit) intelligent cockpit controller, DHU = IHU + DIM. DHU is the abbreviation of combining IHU and DIM. It takes the "D" of DIM and replaces the "I" of IHU to become "DHU".

[0296] DIM (Driver Information Module or Dash Integration Module) driver information module, also known as the "instrument", is a display screen used to display information related to vehicle driving and functions. It is generally placed behind the steering wheel, in the position where the driver can most easily see it.

[0297] The embodiment of the present application also provides a vehicle, including Figure 7 any one of the possible electronic devices shown in the embodiments above.

[0298] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium may include: various media that can store program codes such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs, etc. Specifically, the computer-readable storage medium stores program instructions, and the program instructions are used for the methods in the above method embodiments.

[0299] An embodiment of the present application also provides a computer program product, including a computer program, which implements the methods in the above method embodiments when executed by a processor.

[0300] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.

[0301] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A global relocalization method, characterized in that, it includes: Obtaining the observation features of the geographical space where the vehicle is located; The observation features include: three-dimensional multi-frame observation features formed by splicing a plurality of three-dimensional single-frame observation features; Determining a semantic descriptor corresponding to the observation features, the semantic descriptor being used to characterize the second relative position relationship between each second semantic element in the observation features and the current observation point, and the current observation point being the latest one; Comparing the semantic descriptor corresponding to the observation features with each reference feature set included in the semantic database to determine a comparison result; the reference feature set is used to characterize the first relative position relationship between a plurality of semantic features observed at the same reference position and the reference position; Selecting the minimum value in the comparison result, and determining the target reference position as the reference position corresponding to the minimum value; Determining the initial pose of the vehicle relative to the feature map according to the first semantic element of the feature map, the observation features, and the target reference position, the initial pose including: an initial position and an initial attitude; Determining the current pose of the vehicle within the global range according to the initial pose, the observation features, and the feature map; The obtaining the observation features of the geographical space where the vehicle is located includes: obtaining a plurality of the three-dimensional single-frame observation features; calculating the first relative pose between each three-dimensional single-frame observation feature and the latest three-dimensional single-frame observation feature; according to each of the first relative poses, converting each three-dimensional single-frame observation feature into a to-be-combined observation feature, the to-be-combined observation feature being the three-dimensional single-frame observation feature described in the latest pose, and the latest pose being the pose corresponding to the latest three-dimensional single-frame observation feature; superimposing and combining all the to-be-combined observation features to determine the three-dimensional multi-frame observation features.

2. The global relocalization method according to claim 1, characterized in that, the determining the semantic descriptor corresponding to the observation features includes: Using a semantic description model to perform semantic description on the observation features to determine the corresponding semantic descriptor.

3. The global relocalization method according to claim 1 or 2, characterized in that, the first relative position relationship includes: the first distance between the first position corresponding to the semantic feature and the reference position, and the second relative position relationship includes: the second distance between the second position corresponding to the second semantic element and the current observation point; Correspondingly, the comparing the semantic descriptor corresponding to the observation features with each reference feature set included in the semantic database to determine a comparison result includes: Determining the registration distance between the semantic descriptor and each reference feature set according to the first distance and the second distance; Taking the registration distance as the comparison result.

4. The global relocalization method according to claim 3, characterized in that, the determining the registration distance between the semantic descriptor and each reference feature set according to the first distance and the second distance includes: Calculate the difference between each of the first distances in the semantic descriptor and each of the second distances in the reference feature set; Determine whether the type of each of the second semantic elements is the same as that of each of the semantic features; If they are the same, multiply the difference by a first correction coefficient to determine a first corrected difference; If they are not the same, multiply the difference by a second correction coefficient to determine a second corrected difference, where the first correction coefficient is less than the second correction coefficient; Sum all the first corrected differences and all the second corrected differences to determine the registration distance.

5. The global relocalization method according to claim 1, wherein, the determining the initial pose of the vehicle relative to the feature map according to the first semantic element of the feature map, the observed feature, and the target reference position includes: Performing principal component analysis on the observed feature to determine a first transformation matrix; Performing the principal component analysis on the first semantic element to determine a second transformation matrix, where the first semantic element corresponds to the observed feature; Determine the relative pose of the vehicle relative to the feature map according to the first transformation matrix and the inverse matrix corresponding to the second transformation matrix; Determine the initial pose according to the relative pose.

6. The global relocalization method according to claim 1, wherein, the observed feature includes a plurality of second semantic elements, and the feature map includes a plurality of the first semantic elements, where the first semantic element corresponds to the second semantic element; Correspondingly, the determining the current pose of the vehicle in the global range according to the initial pose, the observed feature, and the feature map includes: Calculating the distance between each of the second semantic elements and the corresponding first semantic element according to the initial pose; Performing a weighted sum of each of the distances with the observation confidence corresponding to the distance as a weight to determine an optimized reference index; Adjust the value of the initial pose according to the optimized reference index, and when the optimized reference index obtains the minimum value, use the current value of the initial pose as the current pose.

7. A global relocalization device, wherein, comprising: An acquisition module for acquiring the observed features of the geographical space where the vehicle is located; The observed feature includes: a three-dimensional multi-frame observed feature formed by splicing a plurality of three-dimensional single-frame observed features; A dictionary localization layer module for determining the semantic descriptor corresponding to the observed feature, where the semantic descriptor is used to characterize the second relative position relationship between each second semantic element in the observed feature and the current observation point, and the current observation point is the latest observation point; comparing the semantic descriptor corresponding to the observed feature with each reference feature set included in the semantic database to determine a comparison result; the reference feature set is used to characterize the first relative position relationship between a plurality of semantic features observed at the same reference position and the reference position; screening out the minimum value in the comparison result, and determining the reference position corresponding to the minimum value as the target reference position; A dimension positioning layer module, configured to determine an initial pose of the vehicle relative to the feature map according to a first semantic element of the feature map, the observed feature, and the target reference position, where the initial pose includes: an initial position and an initial attitude; A high-precision positioning layer module, configured to determine a current pose of the vehicle within a global range according to the initial pose, the observed feature, and the feature map; The obtaining module is specifically configured to obtain a plurality of the three-dimensional single-frame observed features; calculate a first relative pose between each of the three-dimensional single-frame observed features and the latest three-dimensional single-frame observed feature; according to each of the first relative poses, convert each of the three-dimensional single-frame observed features into a to-be-combined observed feature, where the to-be-combined observed feature is the three-dimensional single-frame observed feature described in the latest pose, and the latest pose is the pose corresponding to the latest three-dimensional single-frame observed feature; superimpose and combine all the to-be-combined observed features to determine the three-dimensional multi-frame observed feature.

8. An electronic device, characterized in that, it includes: a processor and a memory; the memory is configured to store a computer program of the processor; the processor is configured to execute the global repositioning method according to any one of claims 1 to 6 by executing the computer program.

9. A vehicle, characterized in that, it includes the electronic device according to claim 8.

10. A computer-readable storage medium, on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the global repositioning method according to any one of claims 1 to 6 is implemented.

Images