Method for locating a vehicle on the basis of features
Patent Information
- Authority / Receiving Office
- EP · EP
- Patent Type
- Applications
- Current Assignee / Owner
- MERCEDES BENZ GROUP AG
- Filing Date
- 2024-06-21
- Publication Date
- 2026-07-01
Smart Images
Figure EP2024067411_27022025_PF_FP_ABST
Abstract
Description
[0001] Method for feature-based localization of a vehicle
[0002] The invention relates to a method for feature-based localization of a vehicle according to the features of the preamble of claim 1.
[0003] As described in DE 102019 001 450 A1, a method for providing data for locating a vehicle based on landmark information stored in a digital map is known from the prior art. Outside the vehicle, on a server external to the vehicle, locations in the digital map are determined where ambiguities regarding the landmark information exist. These locations are registered in an ambiguity layer of the digital map, and the digital map, together with the ambiguity layer, is made available to the vehicle for retrieval.
[0004] DE 102017 126 925 A1 describes an automated co-pilot control system for autonomous vehicles. A control system for a vehicle includes at least one controller. The controller is programmed to receive first sensor readings from a first group of sensors and to obtain a first vehicle pose based on the first sensor readings. The first vehicle pose includes a first location and a first orientation of the vehicle. The controller is also programmed to receive second sensor readings from a second group of sensors and to provide a second vehicle pose based on the second sensor readings. The second vehicle pose includes a second location and a second orientation of the vehicle. The controller is further programmed to generate a diagnostic signal in response to the first vehicle pose being outside a predetermined range of the second vehicle pose.The invention is based on the object of providing a method for feature-based localization of a vehicle that is improved compared to the prior art.
[0005] The object is achieved according to the invention by a method for feature-based localization of a vehicle having the features of claim 1.
[0006] Advantageous embodiments of the invention are the subject of the subclaims.
[0007] In a method for feature-based localization of a vehicle, also referred to as landmark-based localization, a comparison of sensor-detected infrastructure features, i.e. features of infrastructure elements from an environment of the vehicle, is carried out with reference features stored in a digital map, i.e. stored features of infrastructure elements.
[0008] According to the invention, feature-based localization is performed twice. Feature-based localization is performed in two parallel signal branches, wherein the infrastructure features are detected in the first signal branch with a first sensor arrangement of the vehicle in a first sensor space, and in the second signal branch with a second sensor arrangement of the vehicle in a second sensor space. The reference features are stored in the first signal branch in a first memory space of the digital map, and in the second signal branch in a second memory space of the digital map.
[0009] It is intended that common infrastructure features or common reference features, i.e. the features of the same infrastructure elements, are identified in the two sensor spaces and / or storage spaces and that the identified common infrastructure features or reference features are hidden from one of the two sensor spaces or storage spaces and are left out of consideration during localization, i.e. are not taken into account.
[0010] This means that in the two sensor spaces, common infrastructure features, i.e. the features of identical infrastructure elements from the vehicle's surroundings, are identified and hidden from one of the two sensor spaces and left unconsidered during localization, and / or in the two memory spaces, common reference features, i.e. the features of identical infrastructure elements stored in the digital map, are identified and hidden from one of the two memory spaces and left unconsidered during localization.
[0011] In one possible embodiment of the method, the localization results obtained in the two signal branches are fused or used for mutual plausibility checks.
[0012] The described solution enables a combination of two feature-based localization methods and greater localization reliability. In particular, it enables extremely reliable vehicle localization in the digital map, which is advantageously implemented as an HD map. This vehicle localization is particularly necessary for automated driving functions. Localization reliability requirements can be very high, especially at SAE levels above SAE Level 2. It is therefore advantageous to combine several independent localization methods so that they can monitor each other. One challenge is ensuring their independence.
[0013] For example, the current state of the art uses a combination of global navigation satellite system (GNSS)-based localization and feature-based localization to achieve a high level of localization reliability. However, GNSS-based positioning systems with integrity require specialized hardware and expensive correction data services.
[0014] In solutions that combine multiple feature-based localization methods, the localizations are based on different sensors, if possible even on different measurement methods, for example, camera measurement methods, radar measurement methods, and lidar measurement methods. In particular, separate sensor-specific localization layers in the digital map are used to ensure the required independence. However, this does not completely rule out common-cause errors. Some localization errors are related to the ambiguity or repeatability of the structure of the vehicle's environment. This can lead to two feature-based localizations making the same errors. The method described here specifically addresses this problem.
[0015] In the solution described here, it is first identified which of the
[0016] Features in both sensor rooms are related to the same infrastructure elements. By masking out the common features, particularly from one of the two sensor rooms, common-cause errors can now be avoided.
[0017] The described solution makes it possible to provide a safer decomposition with two feature-based localization algorithms, as common-cause errors related to the ambiguity or repeatability of the structure of the vehicle's environment can be avoided.
[0018] In one possible embodiment, the feature-based localization is carried out simultaneously in two passes, wherein in the first pass the identified common infrastructure features or reference features from the first sensor space or from the first memory space are masked out and left unconsidered during the localization, and in the second pass the identified common infrastructure features or reference features from the second sensor space or from the second memory space are masked out and left unconsidered during the localization.
[0019] This means that in the first pass, common infrastructure features, i.e. the features of identical infrastructure elements from the vehicle's surroundings, are identified in the two sensor spaces and masked out from the first sensor space and left out of the localization, and / or in the two memory spaces, common reference features, i.e. the features of identical infrastructure elements stored in the digital map, are identified and masked out from the first memory space and left out of the localization. At the same time, in the second pass, common infrastructure features, i.e. the features of identical infrastructure elements from the vehicle's surroundings, are identified in the two sensor spaces and masked out from the second sensor space and left out of the localization, and / or in the two memory spaces, common reference features, i.e.the characteristics of identical infrastructure elements stored in the digital map are identified and hidden from the second storage space and left out of the localization.
[0020] In one possible embodiment of the method, the localization results obtained in the two signal branches are merged or used for mutual plausibility checks in each pass. In one possible embodiment of the method, the localization results obtained in both signal branches in both passes and / or the results of the fusions or plausibility checks in the two passes are used for arbitration.
[0021] This approach with the two simultaneous runs is particularly advantageous when the feature densities are comparable or very situation-dependent.
[0022] Embodiments of the invention are explained in more detail below with reference to drawings.
[0023] Showing:
[0024] Fig. 1 schematically shows an embodiment of a method for feature-based localization of a vehicle,
[0025] Fig. 2 schematically shows a hiding of common features, and
[0026] Fig. 3 shows a schematic further development of the method.
[0027] Corresponding parts are provided with the same reference numerals in all figures.
[0028] In the following, a combination of two feature-based localization methods with a higher degree of reliability in the localization of a vehicle 1 is described using Figures 1 to 3.
[0029] Particularly for automated driving functions, extremely reliable vehicle localization in a digital map designed as an HD map is required. For highly automated driving in particular, vehicle 1 must have precise knowledge of its surroundings in order to plan maneuvers accordingly. The range of corresponding information from vehicle 1's sensors is very limited. A digital map designed as an HD map (i.e., a high-resolution map) can provide this environmental information, provided vehicle 1 can be correctly positioned in the digital map. With the precisely aligned position of vehicle 1 in the digital map, the digital map can provide a range-independent and occlusion-free extension of sensor-based lane detection.
[0030] There are different types of localization, in particular localization based on a global navigation satellite system (GNSS), localization based on semantic features, and localization based on low-level sensor features. GNSS-based localization requires only georeferencing, relies on satellite reception, can have local offsets, and does not allow for inherent lane-level positioning in the digital map. Localization based on semantic features uses, for example, line markings, barriers, and signs. The semantic features are often already present in the digital map. This localization is sensor-independent and has a low feature density. Localization based on low-level sensor features has a high feature density, is sensor-specific, and requires an additional dedicated layer in the digital map.
[0031] Localization reliability requirements can be very high, especially at SAE levels above SAE Level 2. It is therefore advantageous to combine multiple independent localization modules so they can monitor each other. This allows each to have lower ASIL and SOTIF requirements. Furthermore, this combination can be used to increase localization accuracy. This is achieved, for example, through data fusion.
[0032] For example, the current state of the art uses a combination of GNSS-based localization and feature-based localization to achieve a high level of localization reliability. However, GNSS-based positioning systems with integrity require specialized hardware and expensive correction data services.
[0033] In solutions that combine multiple feature-based localization methods, the localizations are based, in particular, on different sensors, if possible even on different measurement methods, for example, camera measurement methods, radar measurement methods, and lidar measurement methods. In particular, separate sensor-specific localization layers in the digital map are used to ensure the required independence. However, this does not completely rule out common-cause errors. Some localization errors are related to the ambiguity or repeatability of the structure of the vehicle's environment. This can lead to two feature-based localizations making the same errors.
[0034] The procedure described in more detail below solves this problem in particular.
[0035] In this method for feature-based localization of the vehicle 1 , also referred to as landmark-based localization, sensor-detected infrastructure features IM, ie features of infrastructure elements from the environment of the vehicle 1 , are compared with reference features RM stored in the digital map, ie stored features of infrastructure elements.
[0036] In the method described here, feature-based localization is carried out twice, i.e. in particular in two parallel signal branches Z1, Z2, in particular using two different localization methods. The infrastructure features IM are detected in the first signal branch Z1 with a first sensor arrangement S1 of the vehicle 1 in a first sensor space SR1 and in the second signal branch Z2 with a second sensor arrangement S2 of the vehicle 1 in a second sensor space SR2, wherein the reference features RM are stored in the first signal branch Z1 in a first memory space SP1 of the digital map and in the second signal branch Z2 in a second memory space SP2 of the digital map.
[0037] It is provided that in the two sensor rooms SR1, SR2 and / or storage rooms SP1, SP2 common infrastructure features IM or common reference features RM, ie the features of the same infrastructure elements, are identified and that the identified common infrastructure features IM or reference features RM from one of the two sensor rooms SR1, SR2 or
[0038] Storage spaces SP1, SP2 are hidden and are left out of the localization, ie are not taken into account.
[0039] This means that in the two sensor spaces SR1 , SR2 common infrastructure features IM, ie the features of the same infrastructure elements from the environment of the vehicle 1 , are identified and from one of the two
[0040] Sensor spaces SR1, SR2 are hidden and left unconsidered during localization, and / or in the two storage spaces SP1, SP2, common reference features RM, ie the features of identical infrastructure elements stored in the digital map, are identified and hidden from one of the two storage spaces SP1, SP2 and left unconsidered during localization.
[0041] In a possible embodiment of the method, the localization results E1, E2 obtained in the two signal branches Z1, Z2 are fused or used for mutual plausibility P.
[0042] Figure 1 shows an exemplary embodiment of the feature-based localization of the vehicle 1 using two different localization methods. The first localization method can, for example, be localization using semantic landmarks as infrastructure features IM, which are detected in the first signal branch Z1 with the first sensor arrangement S1 of the vehicle 1, for example with a camera, in the first sensor space SR1. The second localization method can, for example, be point clouds of infrastructure features IM, which are detected in the second signal branch Z2 with the second sensor arrangement S2 of the vehicle 1, for example with a lidar or radar, in the second sensor space SR2.
[0043] In the first signal branch Z1, for localization, the infrastructure features IM detected by the first sensor array S1 of vehicle 1 in the first sensor space SR1 are compared with the reference features RM stored in the first memory space SP1 of the digital map, thereby determining a localization result E1 in the first signal branch Z1. Likewise, in the second signal branch Z2, for localization, the infrastructure features IM detected by the second sensor array S2 of vehicle 1 in the second sensor space SR2 are compared with the reference features RM stored in the second memory space SP2 of the digital map, thereby determining a localization result E2 in the second signal branch Z2.
[0044] Advantageously, these localization results E1, E2 obtained in the two signal branches Z1, Z2 are used for decomposition D, mutual plausibility check P, and / or fusion F. Fusion F can refer to any method used to combine the two localization results E1, E2 and their accuracy estimates.
[0045] In the solution described here, it is additionally provided that, as already mentioned above, common infrastructure features IM or common reference features RM are identified in the two sensor rooms SR1, SR2 and / or storage rooms SP1, SP2 and that the identified common infrastructure features IM or reference features RM are hidden from one of the two sensor rooms SR1, SR2 or storage rooms SP1, SP2 and are not taken into account during localization.
[0046] In the solution described here, in particular, it is first identified which of the features in both sensor spaces SR1, SR2 are associated with the same infrastructure elements, i.e., both sensor technologies detect a localization feature on the same structure. In the simplest implementation, this indication can simply be provided by spatial proximity, i.e., by determining that the features of both sensor technologies are approximately on top of each other. The accuracy of this association can be increased by also processing feature descriptors and / or cluster shapes to determine whether these are structural classes that typically generate common features. This can be either an explicit classification, e.g., "this is a pole that is typically detected by both radar and camera," or a machine learning-based classifier that has learned the association implicitly.
[0047] By masking out the common features, specifically from one of the two sensor spaces SR1, SR2, common-cause errors can now be avoided. If one of the sensor spaces SR1, SR2 is consistently much denser in features than the other, the masking can simply be performed statically there.
[0048] In the example according to Figure 2, the infrastructure features IM are thus recorded in the first signal branch Z1 with the first sensor arrangement S1 of vehicle 1 in the first sensor space SR1, and in the second signal branch Z2 with the second sensor arrangement S2 of vehicle 1 in the second sensor space SR2. Subsequently, the common features are identified I, and then the identified common features are masked A. In the example shown, these common features are masked out as common reference features RM in the second signal branch Z2 from the second storage space SP2, so that in the second signal branch Z2 only correspondingly thinned reference features DRM are used for localization.
[0049] Localization is then advantageously carried out as described above for Figure 1, except that the thinned reference features DRM are used for this purpose in the second signal branch Z2. This means that in the first signal branch Z1, for localization purposes, the infrastructure features IM detected by the first sensor arrangement S1 of the vehicle 1 in the first sensor space SR1 are compared with the reference features RM stored in the first memory space SP1 of the digital map, and in this way a localization result E1 is determined in the first signal branch Z1. In the second signal branch Z2, for localization purposes, the infrastructure features IM detected by the second sensor arrangement S2 of the vehicle 1 in the second sensor space SR2 are compared with the thinned reference features DRM, and in this way a localization result E2 is determined in the second signal branch Z2.
[0050] Advantageously, these localization results E1, E2 obtained in the two signal branches Z1, Z2 are also used for decomposition D, mutual plausibility check P, and / or fusion F. Fusion F can also refer to any method used to combine the two localization results E1, E2 and their accuracy estimates.
[0051] The joint feature detection and masking / thinning can, as mentioned above, be applied to the sensor spaces SR1 , SR2 and / or to the memory spaces SP1 , SP2. Applying it to both can be advantageous because it reduces the complexity of feature matching and thus shortens the runtime, but it can also result in more information being lost. Most localization algorithms are resistant to missed detections, so masking A of the sensor detections, i.e., masking A from one of the two
[0052] Sensor spaces SR1, SR2, works better than hiding A from one of the two memory spaces SP1, SP2 of the digital map.
[0053] Particularly when the feature densities are comparable or very situation-dependent, an optional embodiment of the method can be advantageous in which two instances, i.e. two passes, of each localization are performed, one of which uses the full feature set and one of which does not use the common features. An arbitration AB then decides which combination of localizations works better at a specific point in time, provided that one of the sensor spaces SR1, SR2 and / or one of the memory spaces SP1, SP2 always has to do without common features. The decomposition D, plausibility check P and / or fusion F on the worse combination can be skipped if the arbitration AB can already determine this from the localization results, in particular if these contain a live performance estimate. This optional embodiment of the method is shown as an example in Figure 3.The localization is carried out here by masking out the identified common infrastructure features IM or reference features RM from the second sensor space SR2 or memory space SP2 and ignoring them during the localization, so that in the first signal branch Z1 a result E1 without masking out A of the common features is determined and in the second signal branch Z2 a result E2 with masking out A of the common features is determined. Here, too, the localization results E1, E2 obtained in the two signal branches Z1, Z2 are used for decomposition D, mutual plausibility check P and / or fusion F. Furthermore, the localization is carried out simultaneously using the reverse procedure, i.e. by masking out the identified common infrastructure features IM or reference features RM from the first sensor space SR1 orMemory space SP1 are hidden and left out of the localization, so that in the first signal branch Z1 a result E1 with hiding A of the common features is determined and in the second signal branch Z2 a result E2 without hiding A of the common features is determined. Here, too, the localization results E1, E2 obtained in the two signal branches Z1, Z2 are used for decomposition D, mutual plausibility check P and / or fusion F. All localization results E1, E2 and the results of decomposition D, mutual plausibility check P and / or fusion F are also used for arbitration AB.
Claims
Patent claims 1. A method for feature-based localization of a vehicle (1), wherein sensor-detected infrastructure features (IM) are compared with reference features (RM) stored in a digital map, characterized in that the feature-based localization is carried out in two parallel signal branches (Z1, Z2), wherein the infrastructure features (IM) are detected in the first signal branch (Z1) with a first sensor arrangement (S1) of the vehicle (1) in a first sensor space (SR1) and are detected in the second signal branch (Z2) with a second sensor arrangement (S2) of the vehicle (1) in a second sensor space (SR2), wherein the reference features (RM) in the first signal branch (Z1) are stored in a first memory space (SP1) of the digital map and in the second signal branch (Z2) are stored in a second memory space (SP2) of the digital map, wherein in the two sensor spaces (SR1, SR2) and / or memory spaces (SP1, SP2) common infrastructure features (IM) orcommon reference features (RM) are identified and the identified common infrastructure features (IM) or reference features (RM) from one of the two sensor spaces (SR1, SR2) or storage spaces (SP1, SP2) are hidden and left out of the localization.
2. Method according to claim 1, characterized in that the localization results (E1, E2) obtained in the two signal branches (Z1, Z2) are merged or used for mutual plausibility check (P).
3. Method according to one of the preceding claims, characterized in that the feature-based localization is carried out simultaneously in two rounds, where - in the first pass, the identified common infrastructure features (IM) or reference features (RM) from the first sensor space (SR1) or from the first storage space (SP1) are hidden and left out of the localization process, and - in the second pass, the identified common infrastructure features (IM) or reference features (RM) from the second sensor space (SR2) or from the second storage space (SP2) are hidden and are not taken into account in the localization.
4. Method according to claim 3, characterized in that in the respective pass the results (E1, E2) of the localization obtained in the two signal branches (Z1, Z2) are fused or used for mutual plausibility check (P).
5. Method according to claim 3 or 4, characterized in that the results of the localization (E1, E2) and / or results of the fusions (F) or plausibility checks (P) obtained in both passes in the two signal branches (Z1, Z2) in the two passes are used for an arbitration (AB).