Vehicle location determination method based on feature points

Abstract

The present invention relates to a method for determining the location of a vehicle (1) based on feature points, wherein infrastructure feature points (IM) detected by a sensor are compared with reference feature points (RM) stored in memory in a digital map. According to the present invention, location determination based on feature points is performed at two parallel signal branches (Z1, Z2), where the infrastructure feature points (IM) are detected by a first sensor device (S1) of the vehicle (1) in a first sensor space (SR1) at the first signal branch (Z1), and by a second sensor device (S2) of the vehicle (1) in a second sensor space (SR2) at the second signal branch (Z2), and the reference feature points (RM) are stored in a first memory space (SP) in a digital map at the first signal branch (Z1). 1) is stored in the first location, and at the second signal branch (Z2), it is also stored in the second memory space (SP2) in the digital map. Common infrastructure feature points (IM) or common reference feature points (RM) are identified in the two sensor spaces (SR1, SR2) and / or memory spaces (SP1, SP2). The identified common infrastructure feature points (IM) or reference feature points (RM) are removed from one of the two sensor spaces (SR1, SR2) or memory spaces (SP1, SP2) and ignored during location determination.

Description

Technical Field

[0001] The present invention relates to a method for identifying the position of a vehicle based on the feature points described in the preamble of claim 1.

Background Art

[0002] As described in German Patent Application Publication No. 102019001450, a method for providing data for identifying the position of a vehicle using landmark information stored in a digital map is known from the prior art. Outside the vehicle, on a server outside the vehicle, a location where ambiguity exists with respect to landmark information is identified on the digital map, and the location is registered in the ambiguity layer of the digital map, and the digital map is provided to the vehicle in a form that can be acquired together with the ambiguity layer.

[0003] German Patent Application Publication No. 102017126925 describes an automated co-pilot control system for an autonomous vehicle. The control system for the vehicle includes at least one control system. The control system is programmed to receive a first sensor measurement value from a first group of sensors and obtain a first vehicle attitude based on the first sensor measurement value. The first vehicle attitude includes a first position and a first orientation of the vehicle. The control system is also programmed to receive a second sensor measurement value from a second group of sensors and provide a second vehicle attitude based on the second sensor measurement value. The second vehicle attitude includes a second position and a second orientation of the vehicle. Further, the control system is programmed to generate a diagnostic signal in response to the first vehicle attitude being outside a predetermined range of the second vehicle attitude.

Summary of the Invention

Problems to be Solved by the Invention

[0004] An object of the present invention is to provide a method for identifying the position of a vehicle based on feature points, which is improved compared to the prior art. [Means for solving the problem]

[0005] This objective is achieved by the present invention through a vehicle positioning method based on feature points, having the configuration of claim 1.

[0006] Advantageous embodiments of the present invention are subject to the dependent claims.

[0007] In a feature point-based vehicle positioning method, also known as landmark-based positioning, infrastructure feature points detected by sensors—that is, feature points of infrastructure elements around the vehicle—are compared with reference feature points stored in a digital map—that is, feature points of stored infrastructure elements.

[0008] According to the present invention, location determination based on feature points is performed twice. In this case, location determination based on feature points is performed at two parallel signal branches, and infrastructure feature points are detected by a first sensor device of the vehicle in a first sensor space at the first signal branch and by a second sensor device of the vehicle in a second sensor space at the second signal branch, and reference feature points are stored in a first memory space of the digital map at the first signal branch and in a second memory space of the digital map at the second signal branch.

[0009] In this case, common infrastructure feature points or common reference feature points, i.e., feature points of the same infrastructure element, are identified in the two sensor spaces and / or memory spaces, and the identified common infrastructure feature points or reference feature points are removed from one of the two sensor spaces or memory spaces and ignored, i.e., not considered, during localization.

[0010] In other words, in both sensor spaces, common infrastructure feature points, i.e., feature points of the same infrastructure element around the vehicle, are identified, removed from one of the sensor spaces, and ignored during localization; and / or in both memory spaces, common reference feature points, i.e., feature points of the same infrastructure element stored in the digital map, are identified, removed from one of the memory spaces, and ignored during localization.

[0011] In one possible embodiment of this method, the location determination results obtained at the two signal branches are either merged or used for mutual validation.

[0012] The above solution improves security in location identification by combining location identification methods based on two feature points. This solution enables highly reliable vehicle location identification, preferably on a digital map formed as an HD map. Vehicle location identification is particularly required for automated driving functions. Security requirements for location identification can be very high, especially at SAE levels above SAE level 2. Therefore, it is preferable to combine multiple independent location identification methods so that they can monitor each other. One challenge here is ensuring their independence.

[0013] Conventional technologies employ combinations of, for example, global navigation satellite system (GNSS)-based positioning and feature point-based positioning to achieve a high degree of positioning accuracy. However, GNSS-based positioning systems with integrity require special hardware and expensive correction data services.

[0014] In a solution that combines localization methods based on multiple feature points, localization is based on different sensors, and if possible, on different measurement methods, such as camera-based, radar-based, or LiDAR-based methods, and in particular, on independent localization levels for each sensor within a digital map to ensure the necessary independence. However, this does not completely eliminate errors due to common causes. Some localization errors are related to the ambiguity and reproducibility of the structure around the vehicle. This can lead to localization based on two feature points producing the same error. The method described here specifically addresses this problem.

[0015] The solution described here first identifies which feature points in both sensor spaces are associated with the same infrastructure element. In particular, by removing common feature points from one of the two sensor spaces, it is possible to avoid errors caused by a common factor.

[0016] The above-described solution makes it possible to avoid errors that often occur due to common causes related to ambiguity or reproducibility of the structure surrounding the vehicle, thereby providing a more reliable decomposition using a positioning algorithm based on two feature points.

[0017] In one possible embodiment, feature point-based localization is performed simultaneously in two passes, where in the first pass, identified common infrastructure feature points or reference feature points are removed from the first sensor space or first memory space and ignored during localization, and in the second pass, identified common infrastructure feature points or reference feature points are removed from the second sensor space or second memory space and ignored during localization.

[0018] In other words, in the first pass, common infrastructure feature points, i.e., feature points of the same infrastructure elements around the vehicle, are identified in both sensor spaces, removed from the first sensor space, and ignored during localization, and / or common reference feature points, i.e., feature points of the same infrastructure elements stored in the digital map, are identified in both memory spaces, removed from the first memory space, and ignored during localization. Simultaneously, in the second pass, common infrastructure feature points, i.e., feature points of the same infrastructure elements around the vehicle, are identified in both sensor spaces, removed from the second sensor space, and ignored during localization, and / or common reference feature points, i.e., feature points of the same infrastructure elements stored in the digital map, are identified in both memory spaces, removed from the second memory space, and ignored during localization.

[0019] In one possible embodiment of this method, the location determination results obtained at both signal branches are either merged in each path or used for mutual validation.

[0020] In one possible embodiment of this method, the positioning results obtained in each bus at both signal branches, and / or the fusion or validation results in each path, are used in the determination unit.

[0021] This approach, using two simultaneous passes, is particularly advantageous when the density of feature points is equivalent or highly situational.

[0022] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. [Brief explanation of the drawing]

[0023] [Figure 1] This figure schematically illustrates one embodiment of a method for determining the location of a vehicle based on characteristic points. [Figure 2] This diagram schematically illustrates the removal of common feature points. [Figure 3] This diagram schematically illustrates the development and formation of location determination methods. [Modes for carrying out the invention]

[0024] In all of the figures, the parts corresponding to each other are designated by the same reference numerals.

[0025] Hereinafter, referring to FIGS. 1 to 3, a combination of position identification methods based on two feature points, which involves an improvement in the certainty of identifying the position of the vehicle 1, will be described.

[0026] In the automatic driving function, in particular, highly reliable identification of the position of the vehicle on the digital map formed as the HD map is required. This is because, especially in highly automated driving, the vehicle 1 needs to accurately recognize its surroundings in order to appropriately plan the driving operation. The corresponding information of the sensors of the vehicle 1 is very limited within its detection range. The HD map, that is, the digital map formed as a high-resolution map, can provide the surrounding information as long as the vehicle 1 can be accurately positioned within the digital map. Due to the accurately oriented position of the vehicle 1 within the digital map, the digital map can provide an extension of sensor-supported lane recognition that is independent of the detection range and free from occlusion.

[0027] There are various types of position identification. In particular, there are position identifications based on the global navigation satellite system (GNSS), position identifications based on semantic feature points, and position identifications based on low-level sensor feature points. The position identification based on GNSS only requires georeferencing, depends on satellite reception, may have local offsets, and does not enable unique positioning at the lane level within the digital map. The position identification based on semantic feature points uses, for example, lane markers, gates, and signs. Semantic feature points often already exist within the digital map. This position identification is sensor-independent. This position identification has a low feature point density. The position identification based on low-level sensor feature points has a high feature point density, is specific to the sensor, and requires an additional dedicated layer within the digital map.

[0028] Security requirements for localization can be very high, especially at SAE levels above SAE Level 2. Therefore, it is advantageous to combine multiple independent localization modules so that they can monitor each other. This allows each module to have lower ASIL and SOTIF requirements. Furthermore, this combination can be used to improve localization accuracy, which can be achieved, for example, through data fusion.

[0029] Conventional technologies employ combinations of GNSS-based and feature-point-based positioning to achieve a high degree of accuracy in localization. However, GNSS-based localization systems with integrity require special hardware and expensive correction data services.

[0030] In a solution that combines positioning methods based on multiple feature points, the positioning is based on different sensors, and if possible, on different measurement methods, such as camera-based, radar-based, or LiDAR-based measurement methods, and in particular, on independent positioning levels for each sensor within a digital map to ensure the necessary independence. However, this does not completely eliminate errors due to common causes. Some positioning errors are related to the ambiguity and reproducibility of the structure around vehicle 1. This can lead to positioning based on two feature points producing the same error.

[0031] The problem in question is resolved by the method described in more detail below.

[0032] In this method for locating a vehicle 1 based on feature points, also known as landmark-based location, infrastructure feature points IM, i.e., feature points of infrastructure elements around the vehicle 1, detected by sensors are compared with reference feature points RM, i.e., stored feature points of stored infrastructure elements, stored in a digital map.

[0033] In the method described here, location determination based on feature points is performed twice, specifically by two different location determination methods at two parallel signal branches Z1 and Z2. Infrastructure feature points IM are detected by the first sensor device S1 of vehicle 1 in the first sensor space SR1 at the first signal branch Z1, and by the second sensor device S2 of vehicle 1 in the second sensor space SR2 at the second signal branch Z2. Reference feature points RM are stored in the first memory space SP1 of the digital map at the first signal branch Z1, and in the second memory space SP2 of the digital map at the second signal branch Z2.

[0034] In this process, common infrastructure feature points IM or common reference feature points RM, i.e., feature points of the same infrastructure element, are identified in the two sensor spaces SR1, SR2 and / or memory spaces SP1, SP2. The identified common infrastructure feature points IM or reference feature points RM are then removed from one of the two sensor spaces SR1, SR2 or memory spaces SP1, SP2 and ignored, or not taken into consideration, during location determination.

[0035] In other words, in both sensor spaces SR1 and SR2, common infrastructure feature points IM, i.e., feature points of the same infrastructure elements around vehicle 1, are identified, removed from one of the two sensor spaces SR1 and SR2, and ignored during localization; and / or, in the two memory spaces SP1 and SP2, common reference feature points RM, i.e., feature points of the same infrastructure elements stored in the digital map, are identified, removed from one of the two memory spaces SP1 and SP2, and ignored during localization.

[0036] In one possible embodiment of this method, the location determination results E1 and E2 obtained at the two signal branches Z1 and Z2 are either merged or used for mutual validation checks P.

[0037] Figure 1 shows an exemplary embodiment of the positioning of vehicle 1 based on feature points using two different positioning methods. The first positioning method is, for example, positioning using semantic landmarks as infrastructure feature points IM, which can be detected at a first signal branch Z1 by a first sensor device S1 of vehicle 1, such as a camera, in a first sensor space SR1. The second positioning method is, for example, positioning using a point cloud of infrastructure feature points IM, which can be detected at a second signal branch Z2 by a second sensor device S2 of vehicle 1, such as a lidar or radar, in a second sensor space SR2.

[0038] At the first signal branch Z1, for location determination, infrastructure feature points IM detected by the first sensor device S1 of vehicle 1 in the first sensor space SR1 are compared with reference feature points RM stored in the first memory space SP1 of the digital map, thereby determining the location determination result E1 at the first signal branch Z1. Similarly, at the second signal branch Z2, for location determination, infrastructure feature points IM detected by the second sensor device S2 of vehicle 1 in the second sensor space SR2 are compared with reference feature points RM stored in the second memory space SP2 of the digital map, thereby determining the location determination result E2 at the second signal branch Z2.

[0039] Advantageously, the localization results E1 and E2 obtained at the two signal branches Z1 and Z2 are used for decomposition D, mutual validation P, and / or fusion F. Fusion F may be related to any method used to combine the two localization results E1 and E2 and their accuracy estimates.

[0040] In the solution described here, as mentioned above, a common infrastructure feature point IM or a common reference feature point RM is identified in both sensor spaces SR1, SR2 and / or memory spaces SP1, SP2. The identified common infrastructure feature point IM or reference feature point RM is removed from either sensor space SR1, SR2 or memory space SP1, SP2 and ignored during location determination.

[0041] In the solution described here, first, it is identified which feature points in both sensor spaces SR1 and SR2 are associated with the same infrastructure element; that is, both sensor technologies detect localization feature points in the same structure. In the simplest implementation, this indication can be made simply by spatial proximity, i.e., by identifying that the feature points of both sensor technologies nearly overlap. The accuracy of this association can be improved by also processing feature point descriptors and / or cluster shapes to identify whether they are structural classes that typically generate common features. This can be an explicit classification, such as "this is a pole that is typically detected by both radar and cameras," or a classifier based on machine learning that implicitly learns associations.

[0042] In particular, by removing common feature points from one of the two sensor spaces SR1 and SR2, it is possible to avoid errors caused by common factors. If one of the sensor spaces SR1 and SR2 is consistently much denser in terms of feature points than the other, the removal can be performed simply statically there.

[0043] Therefore, in the example shown in Figure 2, the infrastructure feature point IM is detected by the first sensor device S1 of vehicle 1 in the first sensor space SR1 at the first signal branch Z1, and by the second sensor device S2 of vehicle 1 in the second sensor space SR2 at the second signal branch Z2. Subsequently, the identification of common feature points I is performed, followed by the removal A of the identified common feature points. In the illustrated example, the common feature point is removed from the second memory space SP2 as a common reference feature point RM at the second signal branch Z2, so at the second signal branch Z2, only the correspondingly removed reference feature point DRM is used for location determination.

[0044] Then, except that the thinned reference feature points DRM are used for this purpose at the second signal branch Z2, location determination is performed advantageously as described in Figure 1. That is, at the first signal branch Z1, for location determination, infrastructure feature points IM detected by the first sensor device S1 of vehicle 1 in the first sensor space SR1 are compared with reference feature points RM stored in the first memory space SP1 of the digital map, thereby determining the location determination result E1 at the first signal branch Z1. At the second signal branch Z2, for location determination, infrastructure feature points IM detected by the second sensor device S2 of vehicle 1 in the second sensor space SR2 are compared with the thinned reference feature points DRM, thereby determining the location determination result E2 at the second signal branch Z2.

[0045] Advantageously, here again, the localization results E1, E2 obtained at the two signal branches Z1, Z2 are used for decomposition D, mutual validity check P, and / or fusion F. Here again, fusion F may be related to any method used to combine both localization results E1, E2 and their accuracy estimates.

[0046] As described above, common feature point recognition and removal / decimation can be applied to sensor spaces SR1, SR2 and / or memory spaces SP1, SP2. Applying it to both is advantageous in reducing the complexity of feature point matching and thus shortening execution time, but it may also result in the loss of more information. Since many localization algorithms are tolerant of missed recognitions, removal A from sensor recognition, i.e., removal A from one of the two sensor spaces SR1, SR2, performs better than removal A from one of the two memory spaces SP1, SP2 of the digital map.

[0047] In particular, when the feature point density is equivalent or highly situation-dependent, an optional embodiment of the method may be advantageous in which two instances, i.e., two passes, are performed for each location identification, one of which uses the complete set of feature points and the other does not use common feature points. Then, assuming that one of the sensor spaces SR1, SR2 and / or one of the memory spaces SP1, SP2 must always discard common feature points, the determination unit AB determines at a given time which location identification combination works better. If the determination unit AB can already read this based on the location identification result, especially if the location identification result includes live performance estimation, it is possible to skip the decomposition D to the worse combination, the validity check P and / or fusion F.

[0048] An optional embodiment of this method is illustrated in Figure 3. Here, localization is performed, and the identified common infrastructure feature point IM or reference feature point RM is removed from the second sensor space SR2 or memory space SP2 and ignored during localization. As a result, result E1 is identified at the first signal branch Z1 without the removal of common feature point A, while result E2 is identified at the second signal branch Z2 with the removal of common feature point A. Here again, the localization results E1 and E2 obtained at the two signal branches Z1 and Z2 are used again for decomposition D, mutual validation P and / or fusion F. Furthermore, since the localization is performed simultaneously with the reverse approach, by removing the identified common infrastructure feature point IM or reference feature point RM from the first sensor space SR1 or memory space SP1 and ignoring it during localization, the first signal branch Z1 identifies result E1 with the removal of common feature point A, and the second signal branch Z2 identifies result E2 without the removal of common feature point A. Here again, the localization results E1 and E2 obtained at the two signal branches Z1 and Z2 are used again for decomposition D, mutual validation P and / or fusion F. In addition, all localization results E1 and E2, as well as the results of decomposition D, mutual validation P and / or fusion F, are also used for determination unit AB. [Prior art documents] [Patent Documents]

[0049] [Patent Document 1] German Patent Application Publication No. 102019001450 [Patent Document 2] German Patent Application Publication No. 102017126925

Claims

1. A method for determining the location of a vehicle (1) based on feature points, wherein infrastructure feature points (IM) detected by a sensor are compared with reference feature points (RM) stored in memory on a digital map, The location determination based on the feature points is performed at two parallel signal branches (Z1, Z2), the infrastructure feature points (IM) are detected at the first signal branch (Z1) by the first sensor device (S1) of the vehicle (1) in the first sensor space (SR1), and at the second signal branch (Z2) by the second sensor device (S2) of the vehicle (1) in the second sensor space (SR2), and the reference feature points (RM) are stored at the first signal branch (Z1) in the first memory space (SP1) of the digital map, and at the second signal branch (Z2) in the second memory space (SP2) of the digital map. Within the two sensor spaces (SR1, SR2) and / or memory spaces (SP1, SP2), a common infrastructure feature point (IM) or a common reference feature point (RM) is identified, and the identified common infrastructure feature point (IM) or reference feature point (RM) is removed from one of the two sensor spaces (SR1, SR2) or memory spaces (SP1, SP2) and ignored during localization. A method for determining location, characterized by the features described above.

2. The positioning results (E1, E2) obtained at the two signal branches (Z1, Z2) are either merged or used for mutual validation (P). The method for determining location according to feature 1.

3. The position determination based on the aforementioned feature points is performed simultaneously in two paths. - In the first path, the identified common infrastructure feature point (IM) or reference feature point (RM) is removed from the first sensor space (SR1) or the first memory space (SP1) and ignored during localization. - In the second path, the identified common infrastructure feature point (IM) or reference feature point (RM) is removed from the second sensor space (SR2) or the second memory space (SP2) and ignored during localization. The location identification method according to feature 1 or 2.

4. In each of the aforementioned paths, the positioning results (E1, E2) obtained at each of the aforementioned signal branches (Z1, Z2) are either merged or used for mutual validation (P). The location determination method according to feature 3.

5. The position determination results (E1, E2) obtained at both signal branching points (Z1, Z2) in each of the aforementioned paths, and / or the fusion (F) or validation check (P) results in each of the aforementioned paths are used for the determination unit (AB). The method for determining location according to feature 3 or 4.

Images