DETERMINATION OF AN ABSOLUTE INITIAL POSITION OF A VEHICLE

Description

[0001] The present invention relates to a method for determining an absolute initial position of a vehicle. Furthermore, the present invention relates to a corresponding device with which the absolute initial position of the vehicle can be determined. Furthermore, the present invention relates to a corresponding vehicle.

[0002] For vehicle navigation, it is important that the vehicle's position is known. Knowing the vehicle's position is particularly essential for autonomous or semi-autonomous vehicles. The absolute position can be determined, at least roughly, using global positioning systems. However, such rough positioning is often insufficient for autonomous driving. Furthermore, global positioning systems are generally not available in parking garages. Therefore, it is necessary to rely on camera-based or other relative localization methods in such situations.

[0003] Many methods for camera-based vehicle localization using landmarks first require an initial position and orientation of the vehicle in order to perform tracking in the subsequent image sequence with an improved self-position estimation. One tool for this are landmarks whose detections can be clearly identified with objects on a map. One example of this is so-called AR (augmented reality) markers, but also other unmistakable objects such as parking bay numbers. The general goal is to identify a few such objects and calculate an initial pose or position by comparing them with the map.

[0004] There are various methods used in technology for global localization based on landmarks. One such well-known method is based on AR markers. By detecting the four corners of the AR marker in the image and comparing them with the four corners of the same marker in the 3D map, an initial pose can be determined using a classic PnP (Perspective-n-Point) method. This is done, for example, in the publicly available OpenCV Aruco library. These methods are relatively inaccurate, especially for distant markers, and require knowledge not only of the marker's position in the 3D map, but also its precise orientation and dimensions. They are not suitable for objects for which precise contours cannot be extracted.

[0005] The PnP method is based on an n-point pose problem, according to which the pose of a calibrated camera is to be estimated based on a set of n 3D points in the world and their corresponding 2D projections in the image. The camera position typically has 6 degrees of freedom, which consist of the three rotations and three translations of the camera with respect to the world. For the classic PnP method, several (at least 3) objects are extracted from a single camera image and compared with the map. From this, the pose can then be determined. The disadvantage is that three objects must be visible simultaneously, which places a severe restriction on the geometry of the landmarks. Furthermore, this method cannot be generalized to multi-camera systems (such as surround-view systems) because the lines of sight of the detections must converge at a single point.

[0006] Such a P3P method is described, for example, in: Nistér, David & Stewénius, Henrik, (2007), "A Minimal Solution to the generalised 3-Point Pose Problem", Journal of Mathematical Imaging and Vision; 27; pages 67 to 79.

[0007] A so-called generalized PnP method is described in: Camposeco et al. "Minimal Solvers for Generalized Pose and Scale Estimation from Two Rays and One Point", ECCV 2016. In this generalized PnP method, the lines of sight do not intersect.

[0008] Accordingly, Nishimura et al., "A linear Generalized Camera Calibration from Three Intersecting Reference Planes," ICCV 2015, presents a so-called "generalized camera" that also generates non-intersecting lines of sight. This concept was already introduced in Grossberg, Nayar, "The Raxel Imaging Model and Ray-Based Calibration," International Journal of Computer Vision 61(2) 2005.

[0009] Another approach to localization based on landmarks involves traveling a certain distance with the vehicle and performing a 3D reconstruction of the extracted objects in the vehicle coordinate system. The resulting 3D map can be compared with the external map using a 3D-3D matching process to determine the vehicle's position. The disadvantage of this approach is that a 3D reconstruction must be performed with sufficient accuracy, which in turn requires tracking the objects over an extended period of time. In particular, individual object detections cannot be used.

[0010] From the document US 10 21 55 71 B2, a method for localization and mapping is known which comprises capturing an image with a camera mounted on a vehicle. The vehicle is assigned a global system location. A landmark represented in the image is identified using a landmark identification module of the vehicle. The identified landmark has a geographical location and a known parameter. A feature extraction module extracts a set of landmark parameters from the image. A relative position between the vehicle and the geographical location of the landmark is determined based on a comparison between the extracted set of landmark parameters and the known parameter. The global system location is updated based on the relative position.

[0011] Furthermore, US Pat. No. 9,488,483 B2 discloses a system for creating a classification template for road markings and for vehicle localization based on such road markings. A database of templates for classifying road markings contains templates of training images originating from various navigation environments. A training image with a road marking can be rectified and enhanced. The corners of the road marking can be calculated using the rectified or enhanced image. Positions for the corners can be determined and saved as part of a template. Furthermore, the corners for a runtime image can be calculated and compared with the corners of templates from the template database. Based on the comparison, the location or position of a vehicle can be determined. In this way, vehicle localization is enabled in which drift or other GPS problems (e.g.Concealment) can be mitigated.

[0012] Document WO 2016 / 130719 A2 describes the use of a coarse map for navigating an autonomous vehicle. The positions of landmarks relative to a vehicle can be determined. Furthermore, a vehicle's current position can be determined based on intersections of direction vectors.

[0013] The object of the present invention is to be able to reliably determine an initial position of a vehicle.

[0014] According to the invention, this object is achieved by a method and a device according to the independent claims. Advantageous developments of the invention emerge from the subclaims.

[0015] According to the invention, a method is provided for determining an absolute initial position of a vehicle. The initial position is, for example, the position at the beginning of a trajectory or any movement of the vehicle and serves as the basis for calculating the vehicle's own position at other times.

[0016] First, the vehicle is moved along a trajectory starting from the absolute initial position. This means that the vehicle is driven, for example, a certain distance and this distance corresponds to the trajectory. The starting point of the distance or trajectory is the absolute initial position. However, the vehicle or the user does not yet know this absolute initial position, in particular its coordinates. To determine this absolute initial position, the trajectory is recorded using relative navigation starting from a relative initial position. For example, the relative initial position is determined by setting its coordinates to 0 or to another predetermined value. The trajectory resulting from the movement of the vehicle is recorded using so-called relative navigation using suitable sensors, preferably on the vehicle.The relative initial position is set as the starting point of the trajectory.

[0017] In a further step of the method according to the invention, the trajectory is recorded by odometry starting from a defined initial position, which is assigned to the absolute initial position. Since the absolute initial position is not yet known, the starting point of the trajectory is defined as the initial position by assigning this initial position, for example, the coordinates 0 / 0. The defined initial position can in principle contain any coordinates. The coordinates defined in this way form the defined initial position at which the odometry can start. The movement of the vehicle is recorded by the odometry. Using the odometry, the position and, if applicable, also the orientation of a mobile system can be estimated by the system itself. Finally, a trajectory starting at the defined initial position can be obtained from this estimate.

[0018] Then, (at least) three objects are detected from the vehicle located on the trajectory, with a detection beam being recorded for each detection. The objects are detected without contact. For example, images of the vehicle's surroundings are acquired and predetermined objects are identified in the images. The detection directions of the detected objects are recorded, for example in relation to the vehicle or the trajectory. This results in a detection beam for each detected object, which emanates from the current position of the vehicle (detection position) or the detection device on the trajectory and points in the direction of the detected object. More than three objects can also be detected and used for further analysis with their respective detection beams. The at least three objects are detected from at least two different points on the trajectory.This means that the detection beams for at least three objects do not all intersect. Furthermore, this means that the trajectory and, in particular, the detection beams are not all captured simultaneously. Rather, a virtual structure is created from the trajectory and the detection beams from the temporally offset images or captures.

[0019] Furthermore, the recorded trajectory, including the recorded detection beams (virtual structure), is compared with a digital map in which the three objects are represented with their respective absolute positions to determine the location of the defined initial position relative to the objects. Since the absolute positions of at least three objects are known from the digital map, as is the relative relationship between the objects and the defined initial position, the absolute initial position can be assigned to the defined initial position from this data. This allows the absolute initial position of the vehicle moving along the trajectory to be estimated.

[0020] In an advantageous embodiment of the method according to the invention, a corresponding orientation is obtained from the comparison in addition to the absolute initial position of the vehicle. For this purpose, the poses during the trajectory acquisition are naturally also recorded. Simultaneously with the absolute initial position, the orientation of the vehicle in the initial position, i.e., the initial pose, can also be determined. This allows the absolute pose of the vehicle at the beginning of the trajectory to be obtained.

[0021] In corresponding embodiments, the odometry is optical odometry, radar odometry, or wheel odometry. With optical odometry, the flow of objects in images is determined. From this flow of objects, conclusions can be drawn about the vehicle's own motion. If there are a sufficient number of objects in the flow, the position and orientation of the vehicle can be estimated. Similarly, odometry can be based on radar instead of light. Here, too, the flow of objects in radar images can be monitored, and the position and orientation of the vehicle can be deduced from this. The most common method, however, is wheel odometry, in which the position and orientation of the vehicle can be estimated based on data from the vehicle's propulsion system. For example, wheel revolutions and steering angles are taken into account for the estimation.

[0022] In one embodiment of the method for determining the absolute initial position of the vehicle, at least one of the objects is detected multiple times. If one of the at least three objects is detected multiple times, a corresponding number of detection beams are generated for the respective object. This increased number of detection beams increases the accuracy of determining the absolute initial position. If necessary, an intersection point of detection beams that all detected a common object can also be used for comparison with the digital map. In particular, barring detection errors, the respective object should be located at the intersection point. Appropriate optimization algorithms can be used for comparison.

[0023] As already indicated above, the detection of the three objects can be achieved by optical detection, radar detection, and / or ultrasonic detection. Optical detection can preferably be achieved by one or more cameras. The detection can be based on appropriate image analysis. The same applies to radar detection and ultrasonic detection. Here, too, corresponding radar images or ultrasonic images of the vehicle's surroundings can be obtained in order to identify predetermined objects or object types through appropriate image analysis.

[0024] In the method according to the invention, the matching involves a generalized PnP method and, in particular, a generalized P3P method. The PnP method (Perspective-n-Point) is a problem for estimating the pose of a calibrated camera given a set of n 3D points in the world and their corresponding 2D projections in the image. The camera pose has six degrees of freedom regarding the various rotations and translations. Especially with three objects, i.e., when n = 3, the well-known P3P method, which is available as open source software, can be used. In the "generalized PnP method," the detection beams or line of sight do not intersect, at least not all of them. The detection beams are therefore recorded, for example, by several cameras of a multi-camera system ("generalized camera").In the present case, the detection beams are usually recorded sequentially from different locations on the trajectory, which is why we can speak of a "virtual generalized camera".

[0025] In a special embodiment of the method, the detection beams and the trajectory are recorded in three-dimensional space. This allows corresponding 3D information regarding the absolute initial position or initial pose to be obtained. This allows corresponding spatial information regarding the initial position or initial pose to be determined.

[0026] Specifically, within the scope of the invention, a method can be provided for determining the self-position of a vehicle by determining an absolute initial position of the vehicle in the manner described above and obtaining the self-position based on the initial position by odometry and / or by camera-based localization. The determined absolute initial position is therefore used in the vehicle itself in order to be able to determine the self-position or self-pose, in particular absolutely. Such a determination of the self-position or self-pose is of outstanding importance in the autonomous or semi-autonomous operation of vehicles. This type of determination of the self-position or self-pose can be particularly advantageous in parking garages, since other systems cannot be used there.

[0027] The above-mentioned object is also achieved according to the invention by a device for determining an absolute initial position of a vehicle, comprising a detection device for detecting a trajectory by odometry starting from a defined initial position, wherein the detection device is movable along the trajectory starting from the absolute initial position, a detection device for detecting three objects from the detection device located on the trajectory, wherein a detection beam is detected during each detection, a comparison device for comparing the detected trajectory including the detected detection beams with a digital map in which the three objects are represented with their respective absolute positions, in order to determine a position of the defined initial position relative to the objects, wherein the comparison includes a generalized PnP method in which the detection beams do not intersect,and a determination device for determining the absolute initial position based on the location of the defined initial position relative to the objects. ,

[0028] Furthermore, according to the invention, a vehicle with such a device for determining the absolute initial position or self-position is also provided.

[0029] The invention also includes further developments of the device according to the invention or the vehicle according to the invention that have features already described in connection with the further developments of the method according to the invention. For this reason, the corresponding further developments of the device according to the invention or the vehicle according to the invention are not described again here.

[0030] The invention also includes combinations of the features of the described embodiments.

[0031] Exemplary embodiments of the invention are described below. Shown are: Fig. 1 shows a schematic block diagram of an embodiment of a method according to the invention; Fig. 2 shows a trajectory with detection beams; Fig. 3 shows objects in a digital map; Fig. 4 shows a comparison of the trajectory of Fig. 2 with the digital map of Fig. 3 ; and Fig. 5 a schematic representation of a vehicle with an exemplary device for determining an initial position.

[0032] The exemplary embodiments explained below are preferred exemplary embodiments of the invention. In the exemplary embodiments, the described components each represent individual, independently considered features of the invention, which also further develop the invention independently of one another and are thus also to be considered as components of the invention, either individually or in a combination other than that shown. Furthermore, the described exemplary embodiments can also be supplemented by further features of the invention already described.

[0033] In the figures, functionally identical elements are provided with the same reference numerals.

[0034] The position and, if applicable, the orientation of a vehicle should be estimated by extracting it, for example, from a camera image and comparing it with a map of the surroundings. Preferably, this estimated position (and optionally orientation) is used to initialize a downstream landmark-based localization.

[0035] In the exemplary embodiment described below, three technologies are combined to enable the initial localization of a vehicle equipped with a camera using an external 3D map. The first technology is the extraction of objects from the camera image that can be clearly identified with objects from the map (e.g., AR markers; augmented reality). For the extraction of the objects, the type of these objects must be specified. The second technology, which is to be combined with the first technology, is a method for "relative navigation" (i.e., odometry) of the vehicle over a section of road that is long enough to extract at least three different objects in the camera data (generally, image data). The relative navigation can be carried out using vehicle odometry (i.e., radodometry), a camera-based method, or optical odometry (e.g.,SLAM (Simultaneous Localization and Mapping), radar odometry, or a combination thereof. A third technology combines the two technologies mentioned above with a generalized method for solving a three-point pose problem (also known as "Perspective-3-Point" or, more generally, "Perspective-n-Point"), which enables the positioning of a generalized camera with at least three sightings of known objects or landmarks. Such a PnP method is described, for example, in the above-mentioned publication by Nistér et al.

[0036] In this combination of technologies, it is possible to consider the entire recorded trajectory with its temporally different sightings as the product of a single "virtual generalized camera" and to use the "generalized PnP method" to perform localization.

[0037] In Fig. 1 The individual process steps are shown schematically as a block diagram to determine the absolute initial position of a vehicle or its own position. In a first step S1, the vehicle is moved from the absolute initial position (whose coordinates are not known at the beginning) along a trajectory 1 (see Fig. 2 ). In a second method step S2, which usually runs concurrently with the first method step S1, the trajectory 1 is recorded by odometry starting from a defined initial position 2 or initial pose. This defined initial pose or initial position 2 corresponds to the initial pose or position, for example, in the vehicle's system as the starting point for movement along the trajectory. The vehicle thus moves and records images of its surroundings, e.g., with the camera (accumulation phase).

[0038] Subsequently, in step S3, at least three objects are detected from the vehicle located or moving along the trajectory. Each time an object is detected (usually from different locations or points along the trajectory), a detection beam is captured. This beam has a starting point (the current position of the vehicle), a direction, and, if applicable, an end point. Objects of a previously known type are thus extracted from the images. The accumulation phase lasts until a sufficient number of "sightings" of a sufficient number of different objects are available.

[0039] In the example of Fig. 2 A first object is sighted at points P1, P2, and P3 of trajectory 1. A respective viewing or detection beam 3 is determined from each point P1 to P3. Such a detection beam 3 is defined by its starting point P1, P2, P3 and, if applicable, including the respective orientations relative to the initial position 2 or the initial pose. Furthermore, the respective detection beams 3 are defined at least by their directions and, if applicable, also by endpoints.

[0040] During the course of the trajectory, the vehicle reaches a point P4, where it detects a second object with a detection beam 3. As it continues its journey, the vehicle reaches points P5 and P6 on the trajectory 2. From there, the vehicle detects a third object with respective detection beams 3. At all points along the trajectory where an object is sighted or detected, preferably not only the position but also the pose of the vehicle is recorded.

[0041] Fig. 3 symbolically represents a digital map 4. Three objects O1, O2, and O3 are marked on it. These objects O1 to O3 correspond to predefined object types that can be detected by the vehicle. In the present example, object O1 is detected or sighted by the vehicle at points P1 to P3. The second object O2 is detected only once at point P4 of trajectory 2. Object O3 is detected twice at points P5 and P6 on trajectory 2.

[0042] In a further step S4, the acquired trajectory 2, including the detection beams 3, is compared with the digital map 4. The digital map 4 contains the respective absolute positions of the objects O1 to O3. Of course, the digital map can also contain other objects and their absolute positions. The comparison is carried out according to the sketch of Fig. 4 This means that the trajectory 2 with its detection beams 3 according to Fig. 2 is positioned and aligned such that the respective detection beams 3 correlate with the corresponding objects O1 to O3 (e.g., landmarks). In the present example, the detection beams 3 pass through the objects O1 to O3. An optimization method based on the least square error, for example, can be used for the alignment. Other so-called map matching methods can also be used for this purpose.

[0043] Using relative navigation or the above-mentioned odometry methods, the pose of the vehicle can be located relative to the beginning of the trajectory, i.e., the defined initial position 2. Each sighting corresponds to a visual ray or detection ray, preferably in three-dimensional space, with the detection ray 3 passing through the optical center of, for example, the camera at the time of the sighting.

[0044] If an object is sighted multiple times, an average beam can be determined from the multiple sightings or detection beams 3. In the example of Fig. 4 For example, with respect to object O1, a beam could be averaged from three detection beams 3. A beam with respect to object O3 could be averaged from two detection beams 3, since object 3 was detected twice. Using relative navigation or odometry, each detection beam 3 can thus be related to the trajectory, if necessary in a 3D system. This creates a system of (3D) detection beams 3 (without a common starting point) and associated 3D points from the external or digital map 4. Using the generalized PnP method, the relative vehicle trajectory can be registered to the external map, thus determining the desired absolute initial pose or initial position of the vehicle (step S5).

[0045] Finally, in a further step S6, the vehicle's own position can be determined based on the absolute initial pose of the vehicle determined as above. The own position can be obtained from the initial position using odometry and / or camera-based localization.

[0046] Fig. 5 schematically shows a vehicle 7 capable of determining an absolute initial position or initial pose and, if applicable, also a current position of its own using the method described above. For this purpose, the vehicle has a device with the devices 8 to 13 explained below. A detection device 8 of the device or of the vehicle 7 serves to detect the trajectory 1 by odometry starting from the defined initial position 2. The detection device or the device or the vehicle 7 are moved along the trajectory 1 starting from the absolute initial position.

[0047] In addition, the device or vehicle 7 has a detection device 9 for detecting. The detection device 9, ie the device or vehicle 7, is located on the trajectory. With each detection, a detection beam 3 is captured. Fig. 5 the detection device 9 is symbolized as a camera on the exterior mirror 14.

[0048] In addition, the device or the vehicle 7 has a comparison device 10 for comparing the detected trajectory 2 including the detected detection beams 3 with a digital map 4 in which the three objects O1, O2, O3 are represented with their respective absolute positions, in order to determine a position of the defined initial position 2 relative to the objects O1, O2, O3.

[0049] Furthermore, the device or vehicle 7 has a determination device 12 for determining the absolute initial position based on the location of the defined initial position relative to the objects. Finally, the device or vehicle 7 also has a processing device 13 for determining a (current) own position or own pose of the vehicle based on the determined absolute initial position or initial pose and, if applicable, further odometry data or other detector data.

[0050] As the exemplary embodiments presented above demonstrate, a combination of relative navigation (odometry), object detection (landmark extraction), and triangulation or trilateration (e.g., generalized PnP method) for estimating a vehicle pose is possible within the scope of the invention. In particular, the objects detected at different times during a journey are interpreted as objects of a single virtual generalized camera in the initial pose or at the initial position.

[0051] Specifically, a corresponding vehicle can be provided with a system of surround-view cameras and vehicle odometry. Furthermore, relative navigation using a SLAM method and scale estimation via vehicle odometry are possible. AR markers can be used as unique landmarks.

[0052] Objects or landmarks can be clearly identifiable landmarks such as parking bay numbers, but also less clearly identifiable landmarks such as traffic signs. The latter may require a downstream process to eliminate misclassifications. Particularly advantageous is the use of individual sightings of objects or landmarks and sightings of the landmarks at different times. However, error-prone 3D reconstructions are not necessary. List of reference symbols

[0053] 1Trajectory 2Initial position 3Detection beam 4Digital map 6Generalized camera 7Vehicle 8Capture device 9Detection device 10Comparison device 12Determination device 13Processing device 14Exterior mirror O1Object O2Object O3Object P1Point P2Point P3Point P4Point P5Point P6Point S1Step S2Step S3Step S4Step S5Step S6Step

Claims

1. A method for determining an absolute initial position of a vehicle (7), wherein the method comprises the following steps: - moving (S1) the vehicle (7) along a trajectory starting from the absolute initial position, - capturing (S2) the trajectory (1) by odometry starting from a defined initial position (2), which is associated with the absolute initial position (2), - detecting (S3) three objects (O1, 02, 03) from the vehicle (7), which is located on the trajectory (1), wherein a detection beam (3) is captured upon each detection, - matching (S4) the captured trajectory (1) including the captured detection beams (3) with a digital map (4), in which the three objects (O1, 02, 03) are represented with their respective absolute positions, for determining a pose of the defined initial position (2) in relation to the objects (O1, 02, O3), wherein matching includes a generalized PnP method, in which the detection beams do not intersect each other, and - determining (S5) the absolute initial position based on the pose of the defined initial position (2) in relation to the objects.

2. The method according to claim 1, wherein an associated orientation to the absolute initial position of the vehicle (7) is also obtained from matching.

3. The method according to claim 1 or 2, wherein the odometry is optical odometry, radar odometry or wheel odometry.

4. The method according to any one of the preceding claims, wherein at least one of the objects is detected multiple times.

5. The method according to any one of the preceding claims, wherein the detection of the three objects (O1, 02, O3) is effected by optical detection, radar detection and / or ultrasonic detection.

6. The method according to any one of the preceding claims, wherein matching includes a generalized P3P method.

7. A method for determining (S6) an own position of a vehicle (7) by determining an absolute initial position (2) of the vehicle (2) according to any one of the preceding claims, and obtaining the own position starting from the initial position (2) by odometry and / or by camera-based localization.

8. An apparatus for determining an absolute initial position of a vehicle (7), comprising: - a capturing device (8) for capturing a trajectory (1) by odometry starting from a defined initial position (2), wherein the capturing device (8) is movable along the trajectory (1) starting from the absolute initial position, - a detection device (9) for detecting three objects (O1, 02, O3) from the detection device, which is located on the trajectory (1), wherein a detection beam is captured upon each detection, - a matching device (10) for matching the captured trajectory (1) including the captured detection beams (3) with a digital map (4), in which the three objects (O1, 02, 03) are represented with their respective absolute positions, for determining a pose of the defined initial position (2) in relation to the objects (O1, 02, O3), wherein matching includes a generalized PnP method, in which the detection beams do not intersect each other, and - a determining device (12) for determining the absolute initial position based on the pose of the defined initial position (2) in relation to the objects (O1, 02, 03).

9. A vehicle (7) with an apparatus according to claim 8.